JP7360023B2 - Shot management system, shot management method and program - Google Patents

Description

The present invention relates to a shot tally system, shot tally method, and program.

Patent Document 1 describes a device that can reproduce videos of swings before and after the moment when a golf ball is hit.

Also, golf measuring devices are known that can be used for trial shots in a golf shop or for golf practice at a golf driving range. Such golf measurement devices can measure, for example, values such as head speed, ball speed, ball launch angle, spin rate, simulated flight distance, head trajectory, dynamic loft, face angle, lie angle, and hitting position during a shot. is measurable.

Japanese Patent Application Publication No. 5-15628

Some golf measuring devices are capable of managing shot data indicating a measured value during a shot in association with the type (number, model name, model number, etc.) of the golf club used for the shot. By aggregating such shot data by type of golf club used for the shot, it is possible to utilize the shot data, for example, in club development for shot analysis or in providing shot data to customers. It is expected that performance will improve.

However, conventionally, in order to aggregate shot data for each type of golf club, an operator had to manually input the type of golf club, which was time-consuming.

The present invention has been made in view of the above circumstances, and one of its objects is to provide a shot aggregation system, a shot aggregation method, and a program that can reduce the effort of aggregating shot data for each type of golf club. It is in.

In order to solve the above problems, a shot tally system according to the present invention includes an estimating means for estimating the type of the golf club based on an image representing a golf ball and a golf club that shoots the golf ball; association means for associating club type data indicating an estimated type of the golf club with shot data indicating at least one of the behavior of the golf club or the behavior of the golf ball when making a shot; and calculation means for calculating a representative value of the values indicated by each of the plurality of shot data associated with the type of club.

In one aspect of the present invention, the calculation means calculates at least one of an average value, standard deviation, maximum value, or minimum value of the values indicated by the plurality of shot data.

Further, in one aspect of the present invention, the estimating means estimates the type of the golf club based on the image taken between the timing of the address in the shot and the timing of the shot.

In this aspect, the estimation means may estimate the type of the golf club based on the image taken a predetermined time before the timing of the detected shot.

In this case, the estimating means determines the type of the golf club based on the image taken a predetermined time before the timing at which a change from the state where the golf ball is placed to the state where the golf ball is not placed is detected. may be estimated.

Alternatively, the estimating means may estimate the type of the golf club based on the image taken a predetermined time before the timing at which the hitting sound was detected.

Alternatively, the estimating means may estimate the type of the golf club based on the image taken a predetermined time before the timing at which passage of the golf club was detected.

Alternatively, the estimating means may estimate the type of the golf club based on the image taken after the timing at which the placement of the ball was detected.

Further, the estimating means may estimate the type of the golf club based on the image taken by a photographing means that starts photographing at the timing of an address in the shot and ends photographing at the timing of the shot. .

Further, in one aspect of the present invention, the estimating means estimates the type of golf club represented by the image based on an output when the image is input to a trained machine learning model.

Further, the shot tallying method according to the present invention includes a step of estimating the type of the golf club based on an image representing a golf ball and a golf club with which the golf ball is shot; a step of associating club type data indicating an estimated type of the golf club with shot data indicating at least one of a behavior of a club or a behavior of the golf ball; and a step of associating club type data indicating an estimated type of the golf club; The method includes the step of calculating a representative value of the values indicated by each of the plurality of shot data.

The program according to the present invention also includes a procedure for estimating the type of golf club based on an image representing a golf ball and a golf club that shoots the golf ball, and a behavior of the golf club when shooting the golf ball. or a step of associating club type data indicating an estimated type of the golf club with shot data indicating at least one of the behaviors of the golf ball, a plurality of the shots being associated with a common type of the golf club; A computer is caused to execute a procedure for calculating a representative value of the values indicated by each piece of data.

1 is a diagram showing an example of the configuration of a shot measurement system according to an embodiment of the present invention. 1 is a diagram showing an example of the configuration of a shot measuring device according to an embodiment of the present invention. 1 is a diagram showing an example of the configuration of a learning device according to an embodiment of the present invention. FIG. 1 is a functional block diagram showing an example of functions of a learning device according to an embodiment of the present invention. FIG. 3 is a diagram schematically showing an example of a basic image. FIG. 3 is a diagram schematically showing an example of a learning input image. FIG. 3 is a diagram schematically showing an example of a basic image. FIG. 3 is a diagram schematically showing an example of a basic image. FIG. 2 is a flow diagram showing an example of the flow of processing performed by the learning device according to an embodiment of the present invention. 1 is a functional block diagram showing an example of functions of a shot management device according to an embodiment of the present invention. FIG. FIG. 2 is a flow diagram showing an example of the flow of processing performed by the shot management device according to an embodiment of the present invention. 1 is a diagram illustrating an example of the configuration of a totalizing device according to an embodiment of the present invention.

Hereinafter, one embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing an example of the configuration of a shot measurement system 1 according to an embodiment of the present invention. As shown in FIG. 1, the shot measurement system 1 according to the present embodiment includes a shot management device 10 and a shot measurement device 22.

The shot management device 10 according to this embodiment is a computer such as a personal computer. As shown in FIG. 1, the shot management device 10 includes, for example, a processor 12, a storage section 14, a communication section 16, a display section 18, and an operation section 20.

The processor 12 is a program-controlled device such as a CPU that operates according to a program installed in the shot management device 10, for example.

The storage unit 14 is a storage element such as ROM or RAM, a hard disk drive, or the like. The storage unit 14 stores programs and the like executed by the processor 12.

The communication unit 16 is, for example, a communication interface such as a network board. In this embodiment, the shot management device 10 can communicate with the shot measurement device 22 via the communication unit 16.

The display unit 18 is a display device such as a liquid crystal display, and displays various images according to instructions from the processor 12.

The operation unit 20 is a user interface such as a keyboard or a mouse, and receives user operation input and outputs a signal indicating the content to the processor 12.

Note that the shot management device 10 may include an optical disk drive for reading optical disks such as DVD-ROMs and Blu-ray (registered trademark) disks, a USB (Universal Serial Bus) port, and the like.

FIG. 2 is a diagram showing an example of the configuration of the shot measuring device 22 according to the present embodiment. As shown in FIG. 2, the shot measuring device 22 according to this embodiment includes a trajectory measuring device 30, a head behavior measuring device 32, and a head photographing device 34.

The trajectory measurement device 30 is provided with a total of nine flash light sources 40 such as LEDs, with three flash light sources 40 arranged vertically on the left, right, and center sides. Further, the trajectory measurement device 30 is provided with a stereo camera 42 between the left flash light source 40 and the center flash light source 40 and between the right flash light source 40 and the center flash light source 40. . Furthermore, a ball presence/absence determination sensor 44 is provided at the right end of the trajectory measuring device 30. Further, the trajectory measuring device 30 is provided with a head speed sensor 46.

In this embodiment, for example, a golf ball 48 having a plurality of markers of different types affixed to different locations on its surface is placed in front of the trajectory measuring device 30 .

Then, the trajectory of the golf ball 48 immediately after the shot is simultaneously photographed from a plurality of directions by a plurality of stereo cameras 42. In this embodiment, for example, an image including a plurality of images of the golf ball 48 is photographed by multiple exposure while the flash light source 40 is intermittently turned on a plurality of times. Note that the stereo camera 42 may be a high-speed camera, and images including the image of the golf ball 48 may be continuously captured by the high-speed camera.

The photographed image is then transmitted to the shot management device 10. Then, in the shot management device 10, three-dimensional measurement of the behavior of the golf ball 48 is performed using stereo image measurement technology based on the photographed images.

In this embodiment, the shot management device 10 generates partial ball data including ball speed data, launch angle data, spin amount data, and flight distance data based on the results of three-dimensional measurement of the golf ball 48. be done.

Here, the ball speed data is, for example, data indicating the ball speed (eg, initial speed) of the golf ball 48. Further, the launch angle data is, for example, data indicating the vertical and horizontal launch angles of the golf ball 48. Further, the spin amount data is, for example, data indicating the amount of backspin and the amount of side spin of the golf ball 48. Further, the flight distance data is, for example, data indicating a simulated flight distance of the golf ball 48.

Further, in the present embodiment, head speed data indicating the head speed of the golf club used for the shot of the golf ball 48, which is measured by the head speed sensor 46, is transmitted from the trajectory measuring device 30 to the shot management device 10.

In this embodiment, the shot management device 10 generates ball data including the head speed data transmitted from the trajectory measuring device 30 and the above-mentioned partial ball data.

The ball presence/absence determination sensor 44 is a sensor that detects the presence or absence of a golf ball 48 placed on the tee 50. In this embodiment, for example, when the ball presence/absence determination sensor 44 detects that the golf ball 48 has changed from not being placed on the tee 50 to being placed on the tee 50, the stereo camera 42 prepares for photographing. will be started. Then, when the golf ball 48 changes from being placed on the tee 50 to not being placed on the tee 50 by the ball presence/absence determination sensor 44, the stereo camera 42 takes an image.

Hereinafter, the detection that the golf ball 48 has changed from not being placed on the tee 50 to being placed on the tee 50 will be referred to as placement detection. Further, detection of a change from a state in which the golf ball 48 is placed on the tee 50 to a state in which it is not placed on the tee 50 will be referred to as shot detection.

Further, in this embodiment, the ball presence/absence discrimination sensor 44 may transmit a predetermined placement detection signal to the shot management device 10 at the timing of the placement detection by the ball presence/absence discrimination sensor 44 . Further, in this embodiment, the ball presence/absence discrimination sensor 44 may transmit a predetermined shot detection signal to the shot management device 10 at the timing of shot detection by the ball presence/absence discrimination sensor 44 .

Further, in this embodiment, an image representing a simulation result of the trajectory of the golf ball 48 can be displayed on the display unit 18 based on the results of three-dimensional measurement of the golf ball 48.

The head behavior measuring device 32 is provided with two units arranged side by side. Each unit is provided with a total of six flash light sources 52 such as LEDs, with three flash light sources 52 arranged horizontally on each of the upper and lower sides.

Furthermore, a camera 54 is provided at the center of each unit to photograph the club head of the golf club used to shoot the golf ball 48.

Further, on the left side of the right unit, a passage sensor 58 is provided that detects passage of a golf club used for a shot of the golf ball 48 by irradiating infrared rays or the like toward the front reflecting plate 56.

In the head behavior measuring device 32, the camera 54 photographs the club head of the golf club used to shoot the golf ball 48 from a plurality of directions. In this embodiment, for example, while the flash light source 52 is intermittently turned on a plurality of times, an image including a plurality of images of the club head is photographed by multiple exposure. In this embodiment, the image may be captured by the camera 54 at the timing when the passing sensor 58 detects passing of the golf club. Note that the camera 54 may be a high-speed camera, and images including the image of the club head may be continuously captured by the high-speed camera.

Furthermore, in the present embodiment, the passage sensor 58 may transmit a predetermined passage detection signal to the shot management device 10 at the timing when the passage of the golf club is detected by the passage sensor 58 .

The photographed image is then transmitted to the shot management device 10. Then, in the shot management device 10, three-dimensional measurement of the attitude and trajectory of the club head is performed based on the captured image. In addition, in this embodiment, a marker may be provided on the upper surface of the golf club used to shoot the golf ball 48. Then, three-dimensional measurement of the attitude and trajectory of the club head may be performed based on the three-dimensional coordinate values of the marker on the top surface of the golf club, which are specified based on the photographed image.

In this embodiment, the shot management device 10 determines, based on the three-dimensional measurement results, face speed data, head trajectory data, dynamic loft data, face angle data, lie angle data, hitting point position data, normal angle data, Then, head data including face rotation data is generated.

Here, the face speed data is, for example, data indicating the face speed of the club head at the time of impact. Further, the head trajectory data is, for example, data indicating the approach angle of the club head in the vertical direction and the horizontal direction at the time of impact. Further, the dynamic loft data is, for example, data indicating the loft angle of the club head at the time of impact. Further, the face angle data is, for example, data indicating the face angle of the club head at the time of impact. Further, the lie angle data is, for example, data indicating the lie angle of the club head at the time of impact. Further, the hitting point position data is, for example, data indicating the hitting point position of the golf ball 48 that has been shot. Further, the normal angle data is, for example, data indicating the direction of the face surface of the club head at the time of impact. The face rotation is, for example, data indicating the magnitude of the face rotation of the club head during a shot.

Furthermore, in this embodiment, the movement of the club head immediately before impact can be displayed in animation on the display unit 18 based on the results of three-dimensional measurement of the club head.

In this embodiment, the head photographing device 34 is provided with a camera 60 that photographs the club head of a golf club used for shooting a golf ball 48 from the side, for example. In this embodiment, the camera 60 captures frame images at a predetermined frame rate. Since the head photographing device 34 photographs the range in which the club head is positioned at the time of address as the photographing range, it is possible to reliably photograph the club head at the time of address. When photographing a club head, the time of photographing is also recorded.

The camera 60 of the head photographing device 34 may start photographing at the timing of the address in the shot, and may end photographing at the timing of the shot. For example, the head photographing device 34 may start photographing an image with the camera 60 at the timing when the ball presence/absence determination sensor 44 detects the placement. Then, the head photographing device 34 may end photographing of the image by the camera 60 at the timing when the ball presence/absence determination sensor 44 detects a shot, for example.

The photographed image is transmitted to the shot management device 10 after being associated with information on the photographing time. In this embodiment, the information on the photographing time is handled as information indicating the photographing timing. Then, in the shot management device 10, the type of golf club shown in the image is estimated using the learned machine learning model.

FIG. 3 is a diagram showing an example of the configuration of a learning device 70 used for learning the above-described machine learning model.

The learning device 70 according to this embodiment is a computer such as a personal computer. As shown in FIG. 3, the learning device 70 includes, for example, a processor 72, a storage section 74, a communication section 76, a display section 78, and an operation section 80.

The processor 72 is a program-controlled device such as a CPU that operates according to a program installed in the learning device 70, for example.

The storage unit 74 is a storage element such as ROM or RAM, a hard disk drive, or the like. The storage unit 74 stores programs and the like executed by the processor 72.

The communication unit 76 is, for example, a communication interface such as a network board.

The display unit 78 is a display device such as a liquid crystal display, and displays various images according to instructions from the processor 72.

The operation unit 80 is a user interface such as a keyboard or a mouse, and receives user operation input and outputs a signal indicating the content to the processor 72.

Note that the learning device 70 may include an optical disk drive for reading optical disks such as a DVD-ROM or a Blu-ray (registered trademark) disk, a USB (Universal Serial Bus) port, and the like.

Note that the shot management device 10 may be used as the learning device 70.

FIG. 4 is a functional block diagram showing an example of functions implemented in the learning device 70 according to this embodiment. Note that the learning device 70 according to this embodiment does not need to implement all of the functions shown in FIG. 4, and functions other than those shown in FIG. 4 may be implemented.

As shown in FIG. 4, the learning device 70 functionally includes, for example, a machine learning model 90, a learning data storage section 92, a learning data acquisition section 94, a preprocessing section 96, and a learning section 98. The machine learning model 90 is mainly implemented using a processor 72 and a storage unit 74.

The learning data storage section 92 is mainly implemented with the storage section 74. The learning data acquisition unit 94, preprocessing unit 96, and learning unit 98 are mainly implemented using the processor 72.

The above functions may be implemented by having the processor 72 execute a program installed in the learning device 70, which is a computer, and including commands corresponding to the above functions. This program may be supplied to the learning device 70, for example, via a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet.

In this embodiment, the machine learning model 90 is a machine learning model in which machine learning such as Adaboost, random forest, neural network, support vector machine (SVM), and nearest neighbor classifier is implemented.

In this embodiment, the machine learning model 90 estimates, for example, the type of golf club used for the shot (for example, at least one of the club count, model name, model number, etc.).

In this embodiment, the learning data storage unit 92 stores, for example, a plurality of learning data used for learning the machine learning model 90. The learning data includes, for example, a basic image representing at least a golf club and a golf ball 48, and teacher data indicating the type of the golf club. Here, it is desirable that the basic images included in each of the plurality of learning data include various types of golf clubs.

In this embodiment, the learning data acquisition unit 94 acquires learning data used for learning the machine learning model 90, for example. Here, the learning data acquisition unit 94 may acquire learning data stored in the learning data storage unit 92.

In this embodiment, the preprocessing unit 96 generates, for example, a learning input image representing at least the golf club and the golf ball 48, which is different from the basic image, based on the basic image included in the learning data. The preprocessing unit 96 uses, for example, template matching technology to identify the area occupied by the image of the golf ball 48 within the basic image. Specifically, for example, a white circular area within the basic image is specified as the area occupied by the image of the golf ball 48. Then, the preprocessing unit 96 generates a learning input image based on, for example, at least one of the position, size, or color of the area occupied by the image of the golf ball 48 in the basic image.

Here, the preprocessing unit 96 may generate a learning input image in which the position of the area occupied by the image of the golf ball 48 in the learning input image is a predetermined position.

Further, the preprocessing unit 96 may generate a learning input image in which the area occupied by the image of the golf ball 48 has a predetermined size.

FIG. 5 is a diagram schematically showing an example of a basic image. The basic image shown in FIG. 5 includes images of a golf ball 48, a tee 50, and a golf club 100. In the basic image shown in FIG. 5, the downward direction is the positive direction of the X-axis, and the direction to the right is the positive direction of the Y-axis. It is assumed that the vertical length of the basic image shown in FIG. 5 is x1, and the horizontal length is y1. Further, it is assumed that the diameter of the image of the golf ball 48 in the basic image is D1. It is also assumed that the coordinate values of the center of the image of the golf ball 48 in the basic image are (x2, y2).

FIG. 6 is a diagram schematically showing an example of a learning input image to be generated in this embodiment. In the learning input image shown in FIG. 6, it is assumed that the downward direction is the positive direction of the X-axis, and the direction to the right is the positive direction of the Y-axis. Further, it is assumed that the vertical length of the learning input image shown in FIG. 6 is x0, and the horizontal length is y0. Further, it is assumed that the diameter of the image of the golf ball 48 in the basic image is D0. It is also assumed that the coordinate values of the center of the image of the golf ball 48 in the learning input image are (a, b).

In this case, the preprocessing unit 96 determines the expansion/contraction ratio of the basic image to be D0/D1 based on the diameter D1 of the image of the golf ball 48 shown in the basic image shown in FIG. Then, the preprocessing unit 96 calculates the coordinate values of the four vertices from the basic image shown in FIG. D0)×a,y2+(D1/D0)×(y0-b)),(x2+(D1/D0)×(x0-a),y2-(D1/D0)×b),(x2+(D1/D0) )×(x0-a), y2+(D1/D0)×(y0-b)).

For example, if the diameter D1 of the image of the golf ball 48 in the basic image shown in FIG. 5 is the same as the diameter D0, the coordinate values of the four vertices from the basic image shown in FIG. -b), (x2-a, y2+(y0-b)), (x2+(x0-a), y2-b), (x2+(x0-a), y2+(y0-b)) R1 is cut out.

The preprocessing unit 96 then generates a learning input image in which the vertical and horizontal lengths of the cut out region R1 are multiplied by D0/D1. Note that if the diameter D1 of the image of the golf ball 48 in the basic image shown in FIG. 5 is the same as the diameter D0, the image itself obtained by cutting out the region R1 from the basic image becomes the learning input image. In this way, a learning input image corresponding to the basic image shown in FIG. 5 is generated.

FIG. 7 is a diagram schematically showing another example of the basic image. The basic image shown in FIG. 7 includes images of a golf ball 48, a tee 50, and a golf club 100. In the basic image shown in FIG. 7, the downward direction is the X-axis positive direction, and the right direction is the Y-axis positive direction. Assume that the basic image shown in FIG. 7 has a vertical length of x3 and a horizontal length of y3. Further, it is assumed that the diameter of the image of the golf ball 48 in the basic image is D2. It is also assumed that the coordinate values of the center of the image of the golf ball 48 in the basic image are (x4, y4).

In this case, the preprocessing unit 96 determines the expansion/contraction ratio of the basic image to be D0/D2 based on the diameter D2 of the image of the golf ball 48 shown in the basic image shown in FIG. Then, the preprocessing unit 96 calculates the coordinate values of the four vertices from the basic image shown in FIG. D0)×a, y4+(D2/D0)×(y0-b)), (x4+(D2/D0)×(x0-a), y4-(D2/D0)×b), (x4+(D2/D0) )×(x0-a), y4+(D2/D0)×(y0-b)).

For example, if the diameter D2 of the image of the golf ball 48 in the basic image shown in FIG. 7 is 125% of the diameter D0, the coordinate values of the four vertices from the basic image shown in FIG. 7 are (x4-1. 25×a, y4-1.25×b), (x4-1.25×a, y4+1.25×(y0-b)), (x4+1.25×(x0-a), y4-1.25× b), a rectangular region R2 of (x4+1.25×(x0-a), y4+1.25×(y0-b)) is cut out.

Then, the preprocessing unit 96 generates a learning input image in which the vertical and horizontal lengths of the cut out region R2 are doubled by D0/D. Note that if the diameter D2 of the image of the golf ball 48 in the basic image shown in FIG. 7 is 125% of the diameter D0, the learning input is such that the vertical and horizontal lengths of the image cut out from the basic image are 80%. An image is generated. In this way, a learning input image corresponding to the basic image shown in FIG. 7 is generated.

As described above, both the learning input image based on the basic image shown in FIG. 5 and the learning input image based on the basic image shown in FIG. The position (for example, the position where the center coordinate values are (a, b)). Furthermore, in both the learning input image based on the basic image shown in FIG. 5 and the learning input image based on the basic image shown in FIG. , D0).

Further, the basic image may be an image that further represents the tee 50. Then, the preprocessing unit 96 generates a learning input image in which the entire image is rotated so that the orientation of the tee 50 is in a predetermined orientation.

Furthermore, the basic image may be an image that further represents artificial turf. The preprocessing unit 96 may then generate a learning input image in which the entire image is rotated so that the edge of the artificial grass is oriented in a predetermined direction.

FIG. 8 is a diagram schematically showing yet another example of the basic image. Here, the preprocessing unit 96 may rotate the basic image shown in FIG. 8 so that the tee 50 is oriented in the vertical direction. Alternatively, the preprocessing unit 96 may rotate the basic image shown in FIG. 8 so that the edge 102 of the artificial grass is oriented in the left-right direction. Then, by performing processing similar to the above-described processing described with reference to the basic images shown in FIGS. 5 and 7 on the rotated basic image, the rectangular region R3 is extracted from the basic image shown in FIG. may be cut out. A learning input image may then be generated based on the cut out region R3. For example, the learning input image may be generated by enlarging or reducing the cut out region R3 as described above.

Further, the preprocessing unit 96 may generate a learning input image in which the entire image is color-corrected so that the color of the area occupied by the image of the golf ball 48 becomes a predetermined color. Here, for example, the preprocessing unit 96 identifies a color with the smallest color difference from among a plurality of predetermined colors (for example, white, yellow, pink) based on the color of the area occupied by the image of the golf ball 48. Good too. The preprocessing unit 96 may then generate the learning input image by color-correcting the entire image so that the area occupied by the image of the golf ball 48 has the specified color.

Further, when the basic image includes an image of artificial turf, the preprocessing unit 96 performs color correction on the entire image so that the color of the area indicated by the image of artificial turf is a predetermined color (for example, green). A learning input image may also be generated.

Note that the basic images shown in FIGS. 5, 7, and 8 include a driver. However, as described above, it is desirable that the basic images included in each of the plurality of learning data include golf clubs 100 of various sizes, including not only drivers but also fairway woods, utilities, irons, wedges, and the like. Furthermore, it is desirable that the basic images included in each of the plurality of pieces of learning data include golf clubs 100 of various models and model numbers.

In this embodiment, the learning unit 98 executes learning of the machine learning model 90 using, for example, an output when a learning input image is input to the machine learning model 90. Here, for example, the output when a learning input image generated based on a basic image included in the learning data is input into the machine learning model 90 and the teacher data included in the learning data may be compared. Then, supervised learning may be performed in which the values of the parameters of the machine learning model 90 are updated based on the results of the comparison.

Here, an example of the flow of the learning process performed by the learning device 70 according to the present embodiment will be described with reference to the flow diagram illustrated in FIG. 9. Here, for example, it is assumed that a plurality of pieces of learning data are stored in the learning data storage section 92.

First, the learning data acquisition unit 94 acquires one piece of learning data that has not been subjected to the processes shown in S102 to S104 from among the plurality of pieces of learning data stored in the learning data storage unit 92 (S101).

Then, the preprocessing unit 96 generates a learning input image corresponding to the basic image as described above, based on the basic image included in the learning data acquired in the process shown in S101 (S102).

Then, the learning unit 98 inputs the learning input image generated in the process shown in S102 to the machine learning model 90 (S103).

Then, the learning unit 98 executes learning of the machine learning model 90 using the output of the machine learning model 90 according to the input of the learning input image in the process shown in S103 (S104). Here, for example, the values of the parameters of the machine learning model 90 may be updated based on a comparison between the output and the teacher data included in the learning data acquired in the process shown in S101.

The learning unit 98 then checks whether the processes shown in S102 to S104 have been executed for all of the plurality of learning data stored in the learning data storage unit 92 (S105).

If it is confirmed that the processes shown in S102 to S104 have not been executed for all of the plurality of learning data stored in the learning data storage unit 92 (S105: N), the process returns to S101.

If it is confirmed that the processes shown in S102 to S104 have been executed for all of the plurality of learning data stored in the learning data storage unit 92 (S105: Y), the process shown in this processing example is ended. .

In this embodiment, a trained machine learning model 90 in which a plurality of learning data are trained is generated as described above. The generated machine learning model 90 is then stored in the storage unit 14 of the shot management device 10.

In this embodiment, the learned machine learning model 90 stored in the shot management device 10 is used to estimate the type of golf club shown in the image captured by the camera 60 of the head imaging device 34. be exposed.

Hereinafter, the functions of the shot management device 10 according to the present embodiment and the functions executed by the shot management device 10 will be mainly focused on estimating the type of golf club shown in the image photographed by the camera 60 of the head photographing device 34. The processing will be further explained.

FIG. 10 is a functional block diagram showing an example of functions implemented in the shot management device 10 according to this embodiment. Note that the shot management device 10 according to this embodiment does not need to implement all of the functions shown in FIG. 10, and functions other than those shown in FIG. 10 may be implemented.

As shown in FIG. 10, the shot management device 10 functionally includes, for example, a machine learning model 90, a reception section 110, a shot data generation section 112, a target image identification section 114, a club type estimation section 116, and shot management data. It includes a generation section 118, a shot management data storage section 120, an aggregation section 122, and a provision section 124.

The machine learning model 90 is mainly implemented using the processor 12 and the storage unit 14. The receiving section 110 is mainly implemented with the communication section 16. The shot data generation section 112, the target image identification section 114, the club type estimation section 116, the shot management data generation section 118, and the aggregation section 122 are mainly implemented using the processor 12. The shot management data storage section 120 is mainly implemented using the storage section 14 . The providing unit 124 is mainly implemented with the processor 12 and the display unit 18.

The above functions may be implemented by having the processor 12 execute a program that is installed in the shot management device 10, which is a computer, and includes commands corresponding to the above functions. This program may be supplied to the shot management device 10 via a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet. .

In this embodiment, the machine learning model 90 is, for example, a trained machine learning model generated as described with reference to FIGS. 4 to 9.

In this embodiment, the receiving unit 110 receives, for example, head speed data, an image taken by the stereo camera 42, an image taken by the camera 54, and an image taken by the camera 60 from the shot measuring device 22.

In this embodiment, every time one shot is performed, the receiving unit 110 receives head speed data associated with the shot from the trajectory measuring device 30. The receiving unit 110 also receives one or more images taken by the stereo camera 42 and associated with the shot from the trajectory measuring device 30. The receiving unit 110 also receives from the head behavior measuring device 32 one or more images taken by the camera 54 and associated with the shot. Further, the receiving unit 110 receives from the head photographing device 34 one or more images photographed by the camera 60 and associated with the shot.

Further, the receiving unit 110 may receive the above-described placement detection signal transmitted from the ball presence/absence determination sensor 44. Further, the receiving unit 110 may receive the above-described shot detection signal transmitted from the ball presence/absence determination sensor 44. Further, the receiving unit 110 may receive the above-mentioned passage detection signal transmitted from the passage sensor 58.

In this embodiment, for example, the shot data generation unit 112 generates shot data that indicates at least one of the behavior of the golf club that shoots the golf ball 48 or the behavior of the golf ball 48 when the golf ball 48 is shot. . Here, for example, shot data including the above-mentioned ball data and the above-mentioned club data is generated.

In this embodiment, the target image specifying unit 114 uses, for example, one or more images received by the receiving unit 110 and received by the camera 60 to estimate a golf club, and which is associated with one shot. Identify the target image that will be displayed. The target image according to this embodiment shows a golf ball 48 and a golf club that shoots the golf ball 48.

For example, the target image specifying unit 114 may specify, as the target image, an image photographed between the timing of the address in the shot and the timing of the shot. In this case, for example, the timing of receiving the placement detection signal may be treated as the timing of the address. Further, for example, the reception timing of the shot detection signal may be treated as the shot timing.

Here, the target image specifying unit 114 may specify, as the target image, for example, an image photographed by the camera 60 a predetermined time before the timing of the detected shot.

For example, the target image identifying unit 114 identifies, as the target image, an image taken by the camera 60 a predetermined time before the timing at which a change from the state where the golf ball 48 is placed to the state where the golf ball 48 is not placed is detected. You may. For example, an image taken by the camera 60 a predetermined time before the shot detection signal is received may be specified as the target image.

Further, the target image specifying unit 114 may specify, as the target image, an image photographed by the camera 60 a predetermined time before the timing at which passage of the golf club is detected. For example, an image taken by the camera 60 a predetermined time before the passing detection signal is received may be specified as the target image.

Further, the target image specifying unit 114 may specify that a shot has been taken when the receiving unit 110 receives a shot detection signal within a predetermined time after receiving the passing detection signal. When it is determined that a shot has been taken in this way, an image taken by the camera 60 a predetermined time before the timing at which the shot detection signal was received may be specified as the target image. Alternatively, when the shot timing is specified in this way, an image taken by the camera 60 a predetermined time before the timing at which the passing detection signal is received may be specified as the target image.

Furthermore, as described above, the camera 60 may start capturing images at the timing when the ball presence/absence determination sensor 44 detects the placement. Then, the target image specifying unit 114 may specify, as the target image, an image photographed by the camera 60 after the timing at which the arrangement is detected.

In this embodiment, the club type estimating unit 116 estimates the type of golf club used for the shot represented by the target image, for example, based on the target image. Here, the club type estimating unit 116 may estimate the type of golf club used for the shot represented by the target image based on the output when the target image is input to the trained machine learning model 90. . In this embodiment, the club type estimation unit 116 generates, for example, club type data indicating the estimated type of golf club.

In this embodiment, the shot management data generation unit 118 adds, for example, club type data indicating the type of golf club estimated by the club type estimation unit 116 for the shot to the above-mentioned shot data associated with one shot. associate. Here, for example, shot management data is generated that includes the above-mentioned shot data associated with one shot and club type data indicating the type of golf club estimated for the shot. Note that the shot management data may include shot timing data indicating shot timing such as the reception time of the shot detection signal, for example.

In this embodiment, the shot management data storage unit 120 stores, for example, shot management data generated by the shot management data generation unit 118. In this embodiment, each time a shot is performed, shot management data corresponding to the shot is generated. The generated shot management data is then stored in the shot management data storage section 120.

In this embodiment, the totaling unit 122 totals, for example, a plurality of shot management data stored in the shot management data storage unit 120. Here, the aggregation unit 122 may calculate, for example, a representative value of the values indicated by each of the plurality of shot data associated with a common type of golf club. For example, the aggregation unit 122 may specify a plurality of shot management data including club type data indicating the type specified by the user.

Then, the aggregation unit 122 may calculate a representative value of the values indicated by the shot data included in each of the plurality of shot management data.

For example, ball speed data, launch angle data, spin rate data, flight distance data, head speed data, face speed data, head trajectory data, dynamic loft data, face angle data, lie angle data, RBI position data, normal angle. For at least one of the data or face rotation data, a representative value (for example, at least one of the average value, standard deviation, maximum value, or minimum value) of the data may be calculated. good.

In this embodiment, the providing unit 124 provides the user with the total results calculated by the totaling unit 122. Here, in this embodiment, the providing unit 124 may display the total results calculated by the totaling unit 122 on the display unit 18. Further, the total results calculated by the total unit 122 may be printed out.

Here, for example, in response to the user's designation of the type of golf club, the aggregation unit 122 may calculate a representative value of the values indicated by each of the plurality of shot data associated with the type. good. For example, the totaling unit 122 may specify a plurality of shot management data including club type data indicating the specified type. Then, the aggregation unit 122 may calculate a representative value of the values indicated by the plurality of shot management data. Then, the providing unit 124 may display the calculated representative value on the display unit 18 or print it out. Here, the providing unit 124 may display on the display unit 18 or print out a list of values indicated by each of the plurality of shot management data.

Further, the providing unit 124 may calculate, for each type of golf club, a representative value of the values indicated by a plurality of shot management data corresponding to the type. Then, a list of representative values for each type of golf club may be displayed on the display unit 18 or may be printed out.

Here, an example of the flow of processing performed by the shot management device 10 according to the present embodiment will be described with reference to a flowchart illustrated in FIG. 11. In the processing example shown in FIG. 11, shot management data associated with one shot is generated and stored in response to reception of a shot detection signal.

First, the shot data generation unit 112 waits for reception of a shot detection signal by the reception unit 110 (S201).

Then, when receiving the shot detection signal by the receiving unit 110 is detected, the shot data generating unit 112 generates shot data (S202). Here, for example, the receiving unit 110 receives images received from the trajectory measuring device 30 and head behavior measuring device 32, and head speed received from the trajectory measuring device 30 within a predetermined time before or after the reception timing of the shot detection signal. Data is identified. Then, shot data is generated based on the specified image and head speed data.

Then, the target image identifying unit 114 identifies the target image as described above (S203).

Then, the club type estimating unit 116 estimates the type of golf club appearing in the target image based on the target image specified in the process shown in S203, and club type data indicating the estimated type of golf club. is generated (S204). Here, as described above, the type of golf club may be estimated based on the output when the target image is input to the machine learning model 90.

Then, the shot management data generation unit 118 generates shot management data including the shot data generated in the process shown in S202 and the club type data generated in the process shown in S204 (S205).

Then, the shot management data generation unit 118 stores the shot management data generated in the process shown in S205 in the shot management data storage unit 120 (S206), and the process shown in the present process example ends.

In this embodiment, learning of the machine learning model 90 is performed using a learning input image generated by performing preprocessing on a basic image. Therefore, the machine learning model 90 that can accurately estimate the type of golf club can be trained using fewer images.

Further, in the present embodiment, the aggregation unit 122 aggregates shot data for each type of golf club. Therefore, according to the present embodiment, it is possible to reduce the effort required to compile shot data for each type of golf club.

In this embodiment, the club type estimating unit 116 determines the golf club and the golf ball based on at least one of the position, size, or color of the area occupied by the image of the golf ball 48 in the target image. An input image different from the target image may be generated that represents at least 48. For example, the input image may be generated based on the target image using an algorithm similar to the algorithm used when generating the learning input image based on the basic image.

For example, the club type estimation unit 116 may generate an input image in which the position of the area occupied by the image of the golf ball 48 in the input image is a predetermined position.

Further, for example, the club type estimation unit 116 may generate an input image in which the area occupied by the image of the golf ball 48 has a predetermined size.

Further, for example, the club type estimating unit 116 may generate an input image in which the entire image is color-corrected so that the color of the area occupied by the image of the golf ball 48 is a predetermined color.

Further, for example, the target image is an image that further represents artificial turf, and the club type estimation unit 116 generates an input image in which the entire image is color-corrected so that the color of the area indicated by the artificial turf image is a predetermined color. You may.

Further, for example, the club type estimating unit 116 may generate an input image in which the entire image is rotated so that the edge of the artificial grass is oriented in a predetermined direction.

Further, for example, the target image is an image that further represents the tee 50, and the club type estimating unit 116 may generate an input image in which the entire image is rotated so that the orientation of the tee 50 is a predetermined orientation.

Then, the club type estimating unit 116 may estimate the type of golf club based on the output when the input image is input to the machine learning model 90.

Note that in this case, the machine learning model 90 does not have to be a machine learning model that has undergone learning using an output corresponding to the input of the learning input image generated by the preprocessing unit 96. For example, the learning input image may be preset to include a learning input image that is photographed so that the golf club and the golf ball 48 are oriented in a predetermined direction, at a predetermined angle of view, and have a predetermined color tone, and training data indicating the type of the golf club. It may be a machine learning model that has been trained using a plurality of created training data.

As described above, based on the output when an input image similar to the learning input image, which is generated based on the target image and relying on the images of the golf ball 48, the tee 50, and the artificial turf, is input to the machine learning model 90. The type of golf club may also be estimated. In this case, the machine learning model 90 can accurately determine the type of golf club regardless of whether learning has been performed using the output according to the input of the learning input image generated by the preprocessing unit 96. The machine learning model 90 that can be estimated can be trained using a small number of images.

Note that the present invention is not limited to the above-described embodiments.

For example, the shot measuring device 22 may include a hitting sound detection sensor that detects hitting sounds. Then, the hitting sound detection sensor may transmit a predetermined shot detection signal to the shot management device 10 at the timing when the hitting sound is detected by the hitting sound detection sensor. Then, the receiving unit 110 may receive the shot detection signal.

In this case, the target image specifying unit 114 may specify, as the target image, an image photographed by the camera 60 of the head photographing device 34 a predetermined time before the timing at which the tapping sound was detected. For example, an image photographed by the camera 60 a predetermined time before the timing at which the receiving unit 110 receives the shot detection signal transmitted by the hitting sound detection sensor may be specified as the target image.

Furthermore, the division of roles between the shot management device 10 and the shot measurement device 22 is not limited to the above-described one. For example, some or all of the functions shown in FIG. 10 implemented in the shot management device 10 may be implemented in the shot measurement device 22. Further, for example, head speed data may be generated by the shot management device 10.

Further, the functions of the head photographing device 34 may be implemented in the head behavior measuring device 32.

Further, for example, as shown in FIG. 12, a plurality of shot measurement systems 1 may be able to communicate with a totaling device 130, which is a computer such as a personal computer, similar to the shot management device 10. Then, each shot measurement system 1 may transmit the shot management data generated by the shot management data generation unit 118 to the aggregation device 130. In this case, the shot management data to be transmitted may be associated with identification information of the shot measurement system 1 that transmits the shot management data.

The aggregation device 130 may also have the functions of the aggregation section 122 and the provision section 124 described above. For example, the aggregation device 130 may aggregate a plurality of shot management data received from a plurality of shot measurement systems 1, as described above. The aggregation device 130 may then provide the user of the aggregation device 130 with the aggregation results calculated by the aggregation device 130.

By doing the above when each of a plurality of golf shops is provided with the shot measurement system 1, shot data generated at a plurality of golf shops can be analyzed in a cross-sectional manner. For example, shot data can be analyzed for each golf shot and for each type of golf club used for the shot.

Note that the location where the shot measurement system 1 according to the present embodiment is installed does not particularly matter; for example, the shot measurement system 1 may be installed at a golf driving range.

In addition to the functions of the aggregation unit 122 and the provision unit 124 described above, for example, the machine learning model 90 shown in FIG. A part or all of the functions of the shot management data generation section 118 and the shot management data storage section 120 may be implemented in the aggregation device 130.

Further, the club type estimating unit 116 may estimate the type of golf club used for the shot represented by the target image without using the trained machine learning model 90.

For example, the club type estimation unit 116 separates and extracts pixels having pixel values in a specific range corresponding to the club head in the target image from pixels having pixel values corresponding to a green background such as artificial turf. It's okay.

Then, first characteristic information regarding the size of the club head or the shape of the club head may be specified. Here, the total number of pixels of the extracted image of the club head may be taken as the size of the club head. Furthermore, from the extracted club head image area, information on the contour shapes of the crown and side images obtained by removing the image of the cylindrical socket into which the club head shaft is inserted is identified. Good too. In the case of wood-based golf clubs, the contour shape varies greatly depending on the type, so the type of golf club can be determined from the contour shape of the extracted crown image. The contour shape can be characterized, for example, by the inclination angle or curvature of a curved line representing the contour. Further, the aspect ratios of the images of the crown portion and the side portions may be specified.

Then, the type of golf club may be estimated based on the identified first feature information. For example, the club type estimating unit 116 may previously hold a reference table that associates the first feature information with the type of golf club. Then, the type of golf club may be estimated from the identified first feature information by referring to this reference table.

Further, the club type estimation unit 116 holds in advance reference images of a plurality of club heads having second characteristic information of at least one of a pattern, a symbol, or a color that can be associated with the model name or model number of the golf club. You may do so. Then, the model name or model number of the golf club is estimated by calling the retained reference images for the image of the club head in the target image and performing matching with the second feature information in each reference image. It's okay.

The second characteristic information includes the outline shape of the club head, the score line pattern provided on the face surface, the form of punch marks provided on the face surface, the decorative pattern formed on the face surface, or the shape of the score line provided on the face surface, or the It may include at least one of the logo marks.

Furthermore, in the present embodiment, the timing and range of photography by the stereo camera 42 of the trajectory measuring device 30 may be changed depending on the estimation result of the type of golf club.

Moreover, the above-mentioned specific numerical values are just examples, and the present invention is not limited to these numerical values.

1 shot measurement system, 10 shot management device, 12 processor, 14 storage unit, 16 communication unit, 18 display unit, 20 operation unit, 22 shot measurement device, 30 trajectory measurement device, 32 head behavior measurement device, 34 head photography device, 40 flash light source, 42 stereo camera, 44 ball presence/absence determination sensor, 46 head speed sensor, 48 golf ball, 50 tee, 52 flash light source, 54 camera, 56 reflector, 58 passing sensor, 60 camera, 70 learning device, 72 processor , 74 storage unit, 76 communication unit, 78 display unit, 80 operation unit, 90 machine learning model, 92 learning data storage unit, 94 learning data acquisition unit, 96 preprocessing unit, 98 learning unit, 100 golf club, 102 edge, Reference Signs List 110 receiving unit, 112 shot data generating unit, 114 target image specifying unit, 116 club type estimating unit, 118 shot management data generating unit, 120 shot management data storage unit, 122 aggregating unit, 124 providing unit, 130 aggregating device.

Claims (11)

A trained machine learning model that has learned an image containing an image of a golf ball and an image of a golf club is provided with at least a portion of an image taken between the timing of the address in the shot and the timing of the shot. estimating means for estimating the type of the golf club based on an output when inputting an image representing a golf ball shot in the shot and a golf club that shoots the golf ball in the shot;
an association for associating club type data indicating an estimated type of the golf club with shot data indicating at least one of the behavior of the golf club when hitting the golf ball or the behavior of the golf ball in the shot; means and
A shot management system comprising : further comprising calculation means for calculating a representative value of the values indicated by each of the plurality of shot data associated with the common type of golf club;
The shot management system according to claim 1, characterized in that : The calculation means calculates at least one of an average value, standard deviation, maximum value, or minimum value of the values indicated by the plurality of shot data.
The shot management system according to claim 2 , characterized in that: The estimating means estimates the type of the golf club based on the image taken a predetermined time before the timing of the detected shot.
The shot management system according to any one of claims 1 to 3 . The estimating means estimates the type of the golf club based on the image taken a predetermined time before a timing at which a change from a state where the golf ball is placed to a state where the golf ball is not placed is detected. ,
The shot management system according to any one of claims 1 to 3 . The estimating means estimates the type of the golf club based on the image taken a predetermined time before the timing at which the hitting sound was detected.
The shot management system according to any one of claims 1 to 3 . The estimating means estimates the type of the golf club based on the image taken a predetermined time before the timing at which passage of the golf club was detected.
The shot management system according to any one of claims 1 to 3 . The estimating means estimates the type of the golf club based on the image taken after the timing when the placement of the ball was detected.
The shot management system according to any one of claims 1 to 3 . The estimating means estimates the type of the golf club based on the image taken by the photographing means which starts photographing at the timing of the address in the shot and ends the photographing at the timing of the shot.
The shot management system according to any one of claims 1 to 8 . The estimation means applies at least one of the images taken between the timing of the address in the shot and the timing of the shot to the trained machine learning model that has learned the images including the image of the golf ball and the image of the golf club. a step of estimating the type of the golf club based on an output when inputting an image representing a golf ball shot in the shot and a golf club that shoots the golf ball in the shot;
The association means includes a club type indicating the estimated type of the golf club in shot data indicating at least one of the behavior of the golf club when hitting the golf ball or the behavior of the golf ball in the shot. a step of associating the data ;
A shot management method comprising : A trained machine learning model that has learned an image containing an image of a golf ball and an image of a golf club is provided with at least a portion of an image taken between the timing of the address in the shot and the timing of the shot. a step of estimating the type of the golf club based on an output when inputting an image representing a golf ball shot in the shot and a golf club that shoots the golf ball in the shot;
a step of associating club type data indicating an estimated type of the golf club with shot data indicating at least one of the behavior of the golf club when hitting the golf ball or the behavior of the golf ball in the shot; ,
A program that causes a computer to execute.

Images