JP2021029738A - Shot management system, shot management method and program - Google Patents

Abstract

To provide a shot management system, a shot management method and a program capable of reducing labor and time when managing shot data by associating the shot data with a user who made a shot.SOLUTION: A shot data generation unit 112 generates a plurality of pieces of shot data which indicates at least one of motion of a golf club and motion of a golf ball when a user makes a shot of a golf ball. A user determination unit 122 acquires a plurality of images of a user who made a shot associated with the shot data out of a plurality of users who is associated with one of the plurality of pieces of shot data. The user determination unit 122 determines the user with whom the shot data associated with the image is associated, out of the plurality of users, on the basis of respective images. A shot management data generation unit 124 associates the determined user with the shot data.SELECTED DRAWING: Figure 10

Description

The present invention relates to a shot management system, a shot management method and a program.

Patent Document 1 describes a device capable of reproducing an image of a swing before and after the moment when a golf ball is hit.

Further, a golf measuring device that can be used in a test hitting scene in a golf shop or a golf practice scene in a driving range is known. In such a golf measuring device, for example, values such as head speed, ball speed, ball launch angle, spin amount, simulation flight distance, head trajectory, dynamic loft, face angle, lie angle, and hitting point position at the time of a shot. Is measurable.

Japanese Unexamined Patent Publication No. 5-15628

It is conceivable to manage the shot data indicating the measured value at the time of the shot measured by the golf measuring device in association with the user who performed the shot. By doing so, for example, in a scene where a plurality of users take shots in order using one golf measuring device, for each user who has taken a shot, only the shot data of the shot by the user is extracted. It is convenient because it can be provided. Further, for example, in a lesson using a golf measuring device, it is convenient because the shot data of the student's shot can be extracted and provided by excluding the coach's shot.

However, if the operator needs to manually input user identification information such as the name of the user who took the shot each time a shot is taken or the user who takes the shot changes, the operator needs to enter the user's identification information manually. It takes time and effort.

The present invention has been made in view of the above circumstances, and one of the purposes thereof is a shot management system, a shot management method, and a program that can reduce the trouble of managing shot data in association with the user who performed the shot. Is to provide.

In order to solve the above problems, the shot management system according to the present invention generates shot data that generates a plurality of shot data indicating at least one of the behavior of a golf club or the behavior of the golf ball when the golf ball is shot. Based on the means, the acquisition means for acquiring a plurality of images of the user who has performed the shot corresponding to the shot data among the plurality of users associated with the plurality of shot data, and the respective images. Includes a determination means for determining which of the plurality of users the shot data associated with is associated with, and an association means for associating the shot data with the user to be determined.

In one aspect of the present invention, a calculation means for calculating a representative value of a value indicated by each of the plurality of shot data associated with the common user is further included.

In this aspect, the calculation means may calculate at least one of the mean value, standard deviation, maximum value, or minimum value of the values indicated by the plurality of shot data.

Further, in one aspect of the present invention, the acquisition means acquires an image of the user taken between the timing of the address in the shot and the timing of the shot.

In this aspect, the acquisition means may acquire an image of the user taken before a predetermined time of the timing of the shot to be detected.

In this case, the acquisition means may acquire an image of the user taken before a predetermined time at the timing when the change from the state in which the ball is arranged to the state in which the ball is not arranged is detected.

Alternatively, the acquisition means may acquire an image of the user taken before a predetermined time at the timing when the tapping sound is detected.

Alternatively, the acquisition means may acquire an image of the user taken before a predetermined time when the passage of the golf club is detected.

Alternatively, the acquisition means may acquire an image of the user taken after the timing when it is detected that the ball is arranged.

Further, the acquisition means may acquire an image of the user to be photographed by the photographing means that starts shooting at the timing of the address in the shot and ends shooting at the timing of the shot.

Further, in one aspect of the present invention, the determination means may determine which of the plurality of users the shot data is associated with by using a trained machine learning model.

Further, the shot management method according to the present invention includes a step of generating a plurality of shot data indicating at least one of the behavior of a golf club or the behavior of the golf ball when a golf ball is shot, and the plurality of shot data. A step of acquiring a plurality of images of a user who has performed a shot corresponding to the shot data among a plurality of users associated with each other, and a plurality of shot data associated with the image based on the respective images. It includes a step of determining which of the users is associated with, and a step of associating the determined user with the shot data.

Further, the program according to the present invention is associated with a procedure for generating a plurality of shot data indicating at least one of the behavior of a golf club or the behavior of the golf ball when shooting a golf ball, and the plurality of shot data, respectively. , A procedure for acquiring a plurality of images of a user who has taken a shot corresponding to the shot data among a plurality of users, and based on each of the images, the shot data associated with the image is among the plurality of users. The computer is made to execute a procedure for determining which one is associated with, and a procedure for associating the determined user with the shot data.

It is a figure which shows an example of the structure of the shot measurement system which concerns on one Embodiment of this invention. It is a figure which shows an example of the structure of the shot measuring apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the structure of the learning apparatus which concerns on one Embodiment of this invention. It is a functional block diagram which shows an example of the function of the learning apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the basic image schematically. It is a figure which shows an example of the learning input image schematically. It is a figure which shows an example of the basic image schematically. It is a figure which shows an example of the basic image schematically. It is a flow figure which shows an example of the flow of the process performed by the learning apparatus which concerns on one Embodiment of this invention. It is a functional block diagram which shows an example of the function of the shot management apparatus which concerns on one Embodiment of this invention. It is a flow chart which shows an example of the flow of the process performed by the shot management apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the structure of the terminal which concerns on one Embodiment of this invention.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing an example of a configuration of a shot measurement system 1 according to an embodiment of the present invention. As shown in FIG. 1, the shot measurement system 1 according to the present embodiment includes a shot management device 10 and a shot measurement device 22.

The shot management device 10 according to the present embodiment is a computer such as a personal computer. As shown in FIG. 1, the shot management device 10 includes, for example, a processor 12, a storage unit 14, a communication unit 16, a display unit 18, and an operation unit 20.

The processor 12 is, for example, a program control device such as a CPU that operates according to a program installed in the shot management device 10.

The storage unit 14 is a storage element such as a ROM or RAM, a hard disk drive, or the like. The storage unit 14 stores a program or the like executed by the processor 12.

The communication unit 16 is a communication interface such as a network board. In the present embodiment, the shot management device 10 can communicate with the shot measuring device 22 via the communication unit 16.

The display unit 18 is a display device such as a liquid crystal display, and displays various images according to the instructions of the processor 12.

The operation unit 20 is a user interface such as a keyboard or a mouse, receives an operation input of the user, and outputs a signal indicating the contents to the processor 12.

The shot management device 10 may include an optical disk drive for reading an optical disk such as a DVD-ROM or a Blu-ray (registered trademark) disc, a USB (Universal Serial Bus) port, or the like.

FIG. 2 is a diagram showing an example of the configuration of the shot measuring device 22 according to the present embodiment. As shown in FIG. 2, the shot measuring device 22 according to the present embodiment includes a ballistic measuring device 30, a head behavior measuring device 32, a head photographing device 34, and a user photographing device 36.

The ballistic measuring instrument 30 is provided with three flash light sources 40 such as LEDs arranged vertically on each of the left and right sides and the center, for a total of nine. Further, the ballistic measuring instrument 30 is provided with a stereo camera 42 between the flash light source 40 on the left side and the flash light source 40 in the center, and between the flash light source 40 on the right side and the flash light source 40 in the center. .. Further, a ball presence / absence determination sensor 44 is provided at the right end of the ballistic measuring instrument 30. Further, the ballistic measuring instrument 30 is provided with a head speed sensor 46.

In the present embodiment, for example, a golf ball 48 in which a plurality of markers of different types are attached to different locations on the surface is arranged in front of the ballistic measuring instrument 30.

Then, the trajectory of the golf ball 48 immediately after the shot is simultaneously photographed by the plurality of stereo cameras 42 from a plurality of directions. In the present embodiment, for example, while the flash light source 40 is intermittently turned on a plurality of times, an image including a plurality of images of the golf ball 48 is photographed by multiple exposure. The stereo camera 42 is a high-speed camera, and the high-speed camera may continuously capture images including an image of a golf ball 48.

Then, the captured image is transmitted to the shot management device 10. Then, in the shot management device 10, the behavior of the golf ball 48 is three-dimensionally measured based on the captured image by the stereo image measurement technique.

Then, in the present embodiment, the shot management device 10 generates partial ball data including ball speed data, launch angle data, spin amount data, and flight distance data based on the result of three-dimensional measurement of the golf ball 48. Will be done.

Here, the ball speed data is data indicating, for example, the ball speed (for example, the initial speed) of the golf ball 48. Further, the launch angle data is, for example, data indicating the vertical and horizontal launch angles of the golf ball 48. Further, the spin amount data is, for example, data showing the backspin amount and the side spin amount of the golf ball 48. Further, the flight distance data is, for example, data indicating a simulated flight distance of the golf ball 48.

Further, in the present embodiment, the head speed data indicating the head speed of the golf club used for the shot of the golf ball 48, which is measured by the head speed sensor 46, is transmitted from the ballistic measuring device 30 to the shot management device 10.

Then, in the present embodiment, the shot management device 10 generates ball data including the head speed data transmitted from the ballistic measuring device 30 and the above-mentioned partial ball data.

The ball presence / absence determination sensor 44 is a sensor that detects the presence / absence of the golf ball 48 arranged on the tee 50. In the present embodiment, for example, when the ball presence / absence determination sensor 44 detects that the golf ball 48 has changed from the state in which it is not placed on the tee 50 to the state in which it is placed, the stereo camera 42 prepares for shooting. It will be started. Then, when the golf ball 48 is changed from the state where the golf ball 48 is arranged on the tee 50 to the state where the golf ball 48 is not arranged on the tee 50 by the ball presence / absence determination sensor 44, the stereo camera 42 takes a picture.

Hereinafter, the detection that the golf ball 48 has changed from the state in which it is not placed on the tee 50 to the state in which it is placed is referred to as placement detection. Further, the detection that the golf ball 48 has changed from the state in which it is placed on the tee 50 to the state in which it is not placed is referred to as shot detection.

Further, in the present embodiment, the ball presence / absence determination sensor 44 may transmit a predetermined arrangement detection signal to the shot management device 10 at the timing of the arrangement detection by the ball presence / absence determination sensor 44. Further, in the present embodiment, the ball presence / absence determination sensor 44 may transmit a predetermined shot detection signal to the shot management device 10 at the timing of shot detection by the ball presence / absence determination sensor 44.

Further, in the present embodiment, an image showing the simulation result of the trajectory of the golf ball 48 can be displayed on the display unit 18 based on the result of the three-dimensional measurement of the golf ball 48.

The head behavior measuring instrument 32 is provided with two units arranged side by side on the left and right sides. Each unit is provided with three flash light sources 52 such as LEDs arranged side by side on each of the upper side and the lower side, for a total of six.

Further, in the center of each unit, a camera 54 for photographing the club head of the golf club used for the shot of the golf ball 48 is provided.

Further, on the left side of the unit on the right side, a passage sensor 58 is provided to detect the passage of the golf club used for the shot of the golf ball 48 by irradiating the front reflector 56 with infrared rays or the like.

In the head behavior measuring device 32, the club head of the golf club used for the shot of the golf ball 48 is photographed from a plurality of directions by the camera 54. In the present embodiment, for example, while the flash light source 52 is intermittently turned on a plurality of times, an image including a plurality of images of the club head is captured by multiple exposure. In the present embodiment, the image may be taken by the camera 54 at the timing when the passage sensor 58 detects the passage of the golf club. The camera 54 is a high-speed camera, and the high-speed camera may continuously capture images including an image of a club head.

Further, in the present embodiment, the passage sensor 58 may transmit a predetermined passage detection signal to the shot management device 10 at the timing when the passage sensor 58 detects the passage of the golf club.

Then, the captured image is transmitted to the shot management device 10. Then, the shot management device 10 performs three-dimensional measurement of the posture and trajectory of the club head based on the captured image. In the present embodiment, a marker may be provided on the upper surface of the golf club used for the shot of the golf ball 48. Then, the posture and trajectory of the club head may be three-dimensionally measured based on the three-dimensional coordinate values of the markers on the upper surface of the golf club, which are specified based on the captured image.

Then, in the present embodiment, in the shot management device 10, based on the result of the three-dimensional measurement, face speed data, head trajectory data, dynamic loft data, face angle data, lie angle data, hitting point position data, normal angle data, And head data including face rotation data is generated.

Here, the face speed data is, for example, data indicating the face speed of the club head at the time of impact. Further, the head trajectory data is, for example, data indicating an approach angle in the vertical direction and the horizontal direction of the club head at the time of impact. Further, the dynamic loft data is, for example, data indicating the loft angle of the club head at the time of impact. Further, the face angle data is, for example, data indicating the face angle of the club head at the time of impact. The lie angle data is, for example, data indicating the lie angle of the club head at the time of impact. Further, the hitting point position data is, for example, data indicating the hitting point position of the shot golf ball 48. The normal angle data is, for example, data indicating the orientation of the face surface of the club head at the time of impact. The face rotation is, for example, data indicating the magnitude of the face rotation of the club head at the time of a shot.

Further, in the present embodiment, the movement of the club head immediately before the impact can be animated and displayed on the display unit 18 based on the result of the three-dimensional measurement of the club head.

In the present embodiment, the head photographing device 34 is provided with, for example, a camera 60 that photographs the club head of the golf club used for the shot of the golf ball 48 from the side. In this embodiment, the camera 60 captures a frame image at a predetermined frame rate. Since the head photographing device 34 photographs the range in which the club head is located at the address as the photographing range, the club head at the time of the address can be reliably photographed. When shooting the club head, the shooting time is also recorded.

The camera 60 of the head photographing device 34 may start shooting at the timing of the address in the shot and end the shooting at the timing of the shot. For example, the head photographing device 34 may start photographing an image by the camera 60 at the timing when the arrangement is detected by the ball presence / absence determination sensor 44. Then, the head photographing device 34 may end the photographing of the image by the camera 60 at the timing when the shot is detected by the ball presence / absence determination sensor 44, for example.

The captured image is transmitted to the shot management device 10 after being associated with the shooting time information. In the present embodiment, the information on the shooting time is treated as information indicating the shooting timing. Then, the shot management device 10 estimates the type of golf club shown in the image by using the trained machine learning model.

FIG. 3 is a diagram showing an example of the configuration of the learning device 70 used for learning the above-mentioned machine learning model.

The learning device 70 according to the present embodiment is a computer such as a personal computer. As shown in FIG. 3, the learning device 70 includes, for example, a processor 72, a storage unit 74, a communication unit 76, a display unit 78, and an operation unit 80.

The processor 72 is, for example, a program control device such as a CPU that operates according to a program installed in the learning device 70.

The storage unit 74 is a storage element such as a ROM or RAM, a hard disk drive, or the like. The storage unit 74 stores a program or the like executed by the processor 72.

The communication unit 76 is a communication interface such as a network board.

The display unit 78 is a display device such as a liquid crystal display, and displays various images according to the instructions of the processor 72.

The operation unit 80 is a user interface such as a keyboard or a mouse, receives an operation input of the user, and outputs a signal indicating the contents to the processor 72.

The learning device 70 may include an optical disk drive for reading an optical disk such as a DVD-ROM or a Blu-ray (registered trademark) disc, a USB (Universal Serial Bus) port, or the like.

The shot management device 10 may be diverted to the learning device 70.

FIG. 4 is a functional block diagram showing an example of the functions implemented in the learning device 70 according to the present embodiment. It should be noted that the learning device 70 according to the present embodiment does not have to be equipped with all the functions shown in FIG. 4, and may be equipped with functions other than the functions shown in FIG.

As shown in FIG. 4, the learning device 70 functionally includes, for example, a machine learning model 90, a learning data storage unit 92, a learning data acquisition unit 94, a preprocessing unit 96, and a learning unit 98. The machine learning model 90 mainly implements the processor 72 and the storage unit 74.

The learning data storage unit 92 mainly implements the storage unit 74. The learning data acquisition unit 94, the preprocessing unit 96, and the learning unit 98 mainly implement the processor 72.

The above functions may be implemented by executing the program including the instructions corresponding to the above functions installed in the learning device 70, which is a computer, on the processor 72. This program may be supplied to the learning device 70 via a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet or the like.

In this embodiment, the machine learning model 90 is a machine learning model in which machine learning such as AdaBoost, Random Forest, Neural Network, Support Vector Machine (SVM), and Nearest Discriminator is implemented.

In this embodiment, the machine learning model 90 estimates, for example, the type of golf club used for the shot (for example, at least one of a count, a model name, a model number, and the like).

In the present embodiment, the learning data storage unit 92 stores a plurality of learning data used for learning the machine learning model 90, for example. The learning data includes, for example, a basic image showing at least a golf club and a golf ball 48, and teacher data showing the type of the golf club. Here, it is desirable that various types of golf clubs are shown in the basic image included in each of the plurality of learning data.

In the present embodiment, the learning data acquisition unit 94 acquires learning data used for learning the machine learning model 90, for example. Here, the learning data acquisition unit 94 may acquire the learning data stored in the learning data storage unit 92.

In the present embodiment, the preprocessing unit 96 generates, for example, a learning input image different from the basic image, which represents at least the golf club and the golf ball 48, based on the basic image included in the learning data. The preprocessing unit 96 specifies, for example, a region occupied by the image of the golf ball 48 in the basic image by using a template matching technique. Specifically, for example, the white circular region in the basic image is specified as the region occupied by the image of the golf ball 48. Then, the preprocessing unit 96 generates a learning input image based on, for example, at least one of the position, size, or color of the region occupied by the image of the golf ball 48 in the basic image.

Here, the preprocessing unit 96 may generate a learning input image in which the position of the region occupied by the image of the golf ball 48 in the learning input image is a predetermined position.

Further, the preprocessing unit 96 may generate a learning input image in which the size of the region occupied by the image of the golf ball 48 is a predetermined size.

FIG. 5 is a diagram schematically showing an example of a basic image. The basic image shown in FIG. 5 shows images of a golf ball 48, a tee 50, and a golf club 100. In the basic image shown in FIG. 5, it is assumed that the downward direction is the X-axis positive direction and the right direction is the Y-axis positive direction. Then, it is assumed that the vertical length of the basic image shown in FIG. 5 is x1 and the horizontal length is y1. Further, it is assumed that the diameter of the image of the golf ball 48 shown in the basic image is D1. Further, it is assumed that the coordinate value of the center of the image of the golf ball 48 shown in the basic image is (x2, y2).

FIG. 6 is a diagram schematically showing an example of a learning input image to be generated in the present embodiment. In the learning input image shown in FIG. 6, it is assumed that the downward direction is the X-axis positive direction and the right direction is the Y-axis positive direction. Further, it is assumed that the vertical length of the learning input image shown in FIG. 6 is x0 and the horizontal length is y0. Further, it is assumed that the diameter of the image of the golf ball 48 shown in the basic image is D0. Further, it is assumed that the coordinate values of the center of the image of the golf ball 48 shown in the learning input image are (a, b).

In this case, the preprocessing unit 96 determines the scaling ratio of the basic image to be D0 / D1 based on the diameter D1 of the image of the golf ball 48 shown in the basic image shown in FIG. Then, in the preprocessing unit 96, the coordinate values of the four vertices from the basic image shown in FIG. 5 are (x2- (D1 / D0) × a, y2- (D1 / D0) × b) and (x2- (D1 / D1 /), respectively. D0) xa, y2 + (D1 / D0) x (y0-b)), (x2 + (D1 / D0) x (x0-a), y2- (D1 / D0) xb), (x2 + (D1 / D0) ) X (x0-a), y2 + (D1 / D0) x (y0-b)).

Here, for example, when the diameter D1 of the image of the golf ball 48 shown in the basic image shown in FIG. 5 is the same as the diameter D0, the coordinate values of the four vertices from the basic image shown in FIG. 5 are (x2-a, y2, respectively). -B), (x2-a, y2 + (y0-b)), (x2 + (x0-a), y2-b), (x2 + (x0-a), y2 + (y0-b)) R1 is cut out.

Then, the preprocessing unit 96 generates a learning input image in which the vertical and horizontal lengths of the cut out region R1 are multiplied by D0 / D1. When the diameter D1 of the image of the golf ball 48 shown in the basic image shown in FIG. 5 is the same as the diameter D0, the image itself obtained by cutting out the region R1 from the basic image becomes the learning input image. In this way, the learning input image corresponding to the basic image shown in FIG. 5 is generated.

FIG. 7 is a diagram schematically showing another example of the basic image. The basic image shown in FIG. 7 shows images of a golf ball 48, a tee 50, and a golf club 100. In the basic image shown in FIG. 7, it is assumed that the downward direction is the X-axis positive direction and the right direction is the Y-axis positive direction. Then, it is assumed that the vertical length of the basic image shown in FIG. 7 is x3 and the horizontal length is y3. Further, it is assumed that the diameter of the image of the golf ball 48 shown in the basic image is D2. Further, it is assumed that the coordinate value of the center of the image of the golf ball 48 shown in the basic image is (x4, y4).

In this case, the preprocessing unit 96 determines the scaling ratio of the basic image to be D0 / D2 based on the diameter D2 of the image of the golf ball 48 shown in the basic image shown in FIG. Then, in the preprocessing unit 96, the coordinate values of the four vertices from the basic image shown in FIG. 7 are (x4- (D2 / D0) × a, y4- (D2 / D0) × b), (x4- (D2 /), respectively. D0) xa, y4 + (D2 / D0) x (y0-b)), (x4 + (D2 / D0) x (x0-a), y4- (D2 / D0) xb), (x4 + (D2 / D0) ) X (x0-a), y4 + (D2 / D0) x (y0-b)), and a rectangular region R2 is cut out.

Here, for example, when the diameter D2 of the image of the golf ball 48 shown in the basic image shown in FIG. 7 is 125% of the diameter D0, the coordinate values of the four vertices from the basic image shown in FIG. 7 are (x4-1. 25 × a, y4-1.25 × b), (x4-1.25 × a, y4 + 1.25 × (y0-b)), (x4 + 1.25 × (x0-a), y4-1.25 × b), (x4 + 1.25 × (x0-a), y4 + 1.25 × (y0-b)), a rectangular region R2 is cut out.

Then, the preprocessing unit 96 generates a learning input image in which the vertical and horizontal lengths of the cut out region R2 are doubled by D0 / D2. When the diameter D2 of the image of the golf ball 48 shown in the basic image shown in FIG. 7 is 125% of the diameter D0, the learning input in which the vertical and horizontal lengths of the image obtained by cutting out the region R2 from the basic image is 80%. An image is generated. In this way, the learning input image corresponding to the basic image shown in FIG. 7 is generated.

As described above, in both the learning input image based on the basic image shown in FIG. 5 and the learning input image based on the basic image shown in FIG. 7, the position of the region occupied by the image of the golf ball 48 in the learning input image is predetermined. It becomes a position (for example, a position where the coordinate value of the center is (a, b)). Further, in both the learning input image based on the basic image shown in FIG. 5 and the learning input image based on the basic image shown in FIG. 7, the diameter of the region occupied by the image of the golf ball 48 in the learning input image is a predetermined length (for example). , D0).

Further, the basic image may be an image further representing the tee 50. Then, the preprocessing unit 96 generates a learning input image in which the entire image is rotated so that the orientation of the tee 50 is a predetermined orientation.

Further, the basic image may be an image further representing the artificial turf. Then, the preprocessing unit 96 may generate a learning input image in which the entire image is rotated so that the orientation of the edges of the artificial turf is a predetermined orientation.

FIG. 8 is a diagram schematically showing still another example of the basic image. Here, the pretreatment unit 96 may rotate the basic image shown in FIG. 8 so that the tee 50 is oriented in the vertical direction. Alternatively, the pretreatment unit 96 may rotate the basic image shown in FIG. 8 so that the edge 102 of the artificial turf is oriented in the left-right direction. Then, by executing the same processing as the above-described processing described with reference to the basic images shown in FIGS. 5 and 7 on the rotated basic image, the rectangular region R3 from the basic image shown in FIG. 8 is executed. May be cut out. Then, a learning input image based on the cut out region R3 may be generated. For example, the learning input image may be generated by enlarging or reducing the cut out region R3 as described above.

Further, the preprocessing unit 96 may generate a learning input image in which the entire image is color-corrected so that the color of the region occupied by the image of the golf ball 48 becomes a predetermined color. Here, for example, the preprocessing unit 96 identifies a color having the smallest color difference from a plurality of predetermined colors (for example, white, yellow, and pink) based on the color of the area occupied by the image of the golf ball 48. May be good. Then, the preprocessing unit 96 may generate a learning input image by color-correcting the entire image so that the region occupied by the image of the golf ball 48 has a specified color.

Further, when the basic image includes an image of artificial turf, the preprocessing unit 96 color-corrects the entire image so that the color of the region indicated by the image of artificial turf becomes a predetermined color (for example, green). A learning input image may be generated.

The driver is shown in the basic images shown in FIGS. 5, 7, and 8. However, as described above, it is desirable that the basic image included in each of the plurality of training data shows not only the driver but also golf clubs 100 having various counts such as fairway woods, utilities, irons, and wedges. Further, it is desirable that the golf club 100 of various models and model numbers is shown in the basic image included in each of the plurality of learning data.

In the present embodiment, the learning unit 98 executes learning of the machine learning model 90 by using, for example, the output when the learning input image is input to the machine learning model 90. Here, for example, the output when the learning input image generated based on the basic image included in the learning data is input to the machine learning model 90 and the teacher data included in the learning data may be compared. Then, supervised learning in which the value of the parameter of the machine learning model 90 is updated based on the result of the comparison may be executed.

Here, an example of the flow of the learning process performed by the learning device 70 according to the present embodiment will be described with reference to the flow diagram illustrated in FIG. Here, for example, it is assumed that a plurality of learning data are stored in the learning data storage unit 92.

First, the learning data acquisition unit 94 acquires one of the plurality of learning data stored in the learning data storage unit 92 in which the processes shown in S102 to S104 have not been executed (S101).

Then, the preprocessing unit 96 generates a learning input image corresponding to the basic image as described above based on the basic image included in the learning data acquired in the process shown in S101 (S102).

Then, the learning unit 98 inputs the learning input image generated by the process shown in S102 into the machine learning model 90 (S103).

Then, the learning unit 98 executes learning of the machine learning model 90 using the output of the machine learning model 90 in response to the input of the learning input image in the process shown in S103 (S104). Here, for example, the value of the parameter of the machine learning model 90 may be updated based on the comparison between the output and the teacher data included in the learning data acquired in the process shown in S101.

Then, the learning unit 98 confirms whether or not the processes shown in S102 to S104 have been executed for all of the plurality of learning data stored in the learning data storage unit 92 (S105).

When it is confirmed that the processes shown in S102 to S104 have not been executed for all of the plurality of learning data stored in the learning data storage unit 92 (S105: N), the process returns to the process shown in S101.

When it is confirmed that the processes shown in S102 to S104 have been executed for all of the plurality of learning data stored in the learning data storage unit 92 (S105: Y), the processes shown in this processing example are terminated. ..

In the present embodiment, the trained machine learning model 90 in which the plurality of training data are learned as described above is generated. Then, the generated machine learning model 90 is stored in the storage unit 14 of the shot management device 10.

Then, in the present embodiment, the type of golf club shown in the image taken by the camera 60 of the head photographing apparatus 34 is estimated by using the learned machine learning model 90 stored in the shot management apparatus 10. It is said.

In the present embodiment, the user photographing device 36 is provided with, for example, a camera 62 that photographs a user who shoots a golf ball 48. In this embodiment, the camera 62 captures a frame image at a predetermined frame rate. Since the user photographing device 36 photographs the range in which the user who shoots the golf ball 48 is located as the photographing range, the user who shoots can be reliably photographed. When the user who shoots the golf ball 48 shoots, the shooting time is also recorded.

The camera 62 of the user photographing device 36 may start shooting at the timing of the address in the shot and end the shooting at the timing of the shot. For example, the user photographing device 36 may start photographing an image by the camera 62 at the timing when the arrangement is detected by the ball presence / absence determination sensor 44. Then, the user photographing device 36 may end the photographing of the image by the camera 62 at the timing when the shot is detected by the ball presence / absence determination sensor 44, for example.

The captured image is transmitted to the shot management device 10 after being associated with the shooting time information. In the present embodiment, the information on the shooting time is treated as information indicating the shooting timing. Then, the shot management device 10 determines the user in the image.

Hereinafter, the estimation of the type of golf club captured in the image captured by the camera 60 of the head photographing device 34 and the determination of the user captured in the image captured by the camera 62 of the user photographing device 36 will be mainly focused on. The function of the shot management device 10 and the process executed by the shot management device 10 according to the present embodiment will be further described.

FIG. 10 is a functional block diagram showing an example of the functions implemented in the shot management device 10 according to the present embodiment. It should be noted that the shot management device 10 according to the present embodiment does not need to be equipped with all the functions shown in FIG. 10, and may be equipped with functions other than the functions shown in FIG.

As shown in FIG. 10, functionally, the shot management device 10 includes a machine learning model 90, a reception unit 110, a shot data generation unit 112, a first target image identification unit 114, a club type estimation unit 116, and a first. 2. The target image identification unit 118, the user image data storage unit 120, the user determination unit 122, the shot management data generation unit 124, the shot management data storage unit 126, the aggregation unit 128, and the provision unit 130 are included.

The machine learning model 90 mainly implements the processor 12 and the storage unit 14. The receiving unit 110 mainly mounts the communication unit 16. The shot data generation unit 112, the first target image identification unit 114, the club type estimation unit 116, the second target image identification unit 118, the user determination unit 122, the shot management data generation unit 124, and the aggregation unit 128 mainly implement the processor 12. Will be done. The user image data storage unit 120 and the shot management data storage unit 126 are mainly implemented with the storage unit 14. The providing unit 130 mainly mounts the processor 12 and the display unit 18.

The above functions may be implemented by executing the program including the instructions corresponding to the above functions installed in the shot management device 10 which is a computer on the processor 12. This program may be supplied to the shot management device 10 via a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet or the like. ..

The machine learning model 90 is, for example, a trained machine learning model generated as described with reference to FIGS. 4 to 9 in the present embodiment.

In the present embodiment, the receiving unit 110 receives, for example, head speed data, an image taken by the stereo camera 42, an image taken by the camera 54, an image taken by the camera 60, and an image taken by the camera 62. Received from the shot measuring device 22.

In the present embodiment, each time a shot is taken, the receiving unit 110 receives the head speed data associated with the shot from the ballistic measuring instrument 30. Further, the receiving unit 110 receives one or a plurality of images taken by the stereo camera 42 associated with the shot from the ballistic measuring instrument 30. Further, the receiving unit 110 receives one or a plurality of images taken by the camera 54 associated with the shot from the head behavior measuring device 32. Further, the receiving unit 110 receives one or a plurality of images taken by the camera 60 associated with the shot from the head photographing device 34. Further, the receiving unit 110 receives one or a plurality of images taken by the camera 62 associated with the shot from the head photographing device 34.

Further, the receiving unit 110 may receive the above-mentioned arrangement detection signal transmitted from the ball presence / absence determination sensor 44. Further, the receiving unit 110 may receive the above-mentioned shot detection signal transmitted from the ball presence / absence determination sensor 44. Further, the receiving unit 110 may receive the above-mentioned passage detection signal transmitted from the passage sensor 58.

In the present embodiment, the shot data generation unit 112 generates a plurality of shot data indicating at least one of the behavior of the golf club that shoots the golf ball 48 and the behavior of the golf ball 48 when the golf ball 48 is shot, for example. To do. Here, for example, a plurality of shot data including the above-mentioned ball data and the above-mentioned club data are generated.

In the present embodiment, the first target image specifying unit 114 estimates the golf club from, for example, one or a plurality of images taken by the camera 60 associated with one shot received by the receiving unit 110. The first target image used for is specified. The first target image according to the present embodiment shows a golf ball 48 and a golf club that shoots the golf ball 48.

The first target image identification unit 114 may specify, for example, an image taken between the timing of the address in the shot and the timing of the shot as the first target image. In this case, for example, the reception timing of the arrangement detection signal may be treated as the address timing. Further, for example, the reception timing of the shot detection signal may be treated as the shot timing.

Here, the first target image identification unit 114 may specify, for example, an image taken by the camera 60 before a predetermined time of the timing of the detected shot as the first target image.

For example, the first target image identification unit 114 first obtains an image taken by the camera 60 before a predetermined time when a change from the state in which the golf ball 48 is arranged to the state in which the golf ball 48 is not arranged is detected. It may be specified as a target image. For example, an image taken by the camera 60 before a predetermined time when the shot detection signal is received may be specified as the first target image.

Further, the first target image specifying unit 114 may specify an image taken by the camera 60 as the first target image before a predetermined time when the passage of the golf club is detected. For example, an image taken by the camera 60 before a predetermined time when the passage detection signal is received may be specified as the first target image.

Further, the first target image specifying unit 114 may specify that the shot was taken when the receiving unit 110 received the shot detection signal within a predetermined time after receiving the passage detection signal. Then, when it is specified that the shot has been performed in this way, the image taken by the camera 60 before the predetermined time of the timing at which the shot detection signal is received may be specified as the first target image. .. Alternatively, when the shot timing is specified in this way, the image taken by the camera 60 before the predetermined time of the timing at which the passage detection signal is received may be specified as the first target image.

Further, as described above, the camera 60 may start taking an image at the timing when the arrangement is detected by the ball presence / absence determination sensor 44. Then, the first target image identification unit 114 may specify the image taken by the camera 60 after the timing when the arrangement is detected as the first target image.

In the present embodiment, the club type estimation unit 116 estimates the type of golf club used for the shot represented by the first target image, for example, based on the first target image. Here, the club type estimation unit 116 estimates the type of golf club used for the shot represented by the first target image based on the output when the first target image is input to the trained machine learning model 90. You may. Then, in the present embodiment, the club type estimation unit 116 generates, for example, club type data indicating the estimated golf club type.

In the present embodiment, the second target image specifying unit 118 selects the golf ball 48 from among one or a plurality of images taken by the camera 62, which are associated with one shot received by the receiving unit 110, for example. The second target image used for determining the shot user is specified.

The second target image specifying unit 118 may specify, for example, an image taken between the timing of the address in the shot and the timing of the shot as the second target image. In this case, for example, the reception timing of the arrangement detection signal may be treated as the address timing. Further, for example, the reception timing of the shot detection signal may be treated as the shot timing.

Here, the second target image identification unit 118 may specify, for example, an image taken by the camera 60 before a predetermined time of the timing of the detected shot as the second target image.

For example, the second target image identification unit 118 captures an image taken by the camera 62 a predetermined time before the timing at which the change from the state in which the golf ball 48 is arranged to the state in which the golf ball 48 is not arranged is detected. It may be specified as a target image. For example, an image taken by the camera 62 before a predetermined time when the shot detection signal is received may be specified as the second target image.

Further, the second target image specifying unit 118 may specify an image taken by the camera 62 as a second target image before a predetermined time when the passage of the golf club is detected. For example, an image taken by the camera 62 before a predetermined time when the passage detection signal is received may be specified as the second target image.

Further, the second target image specifying unit 118 may specify that the shot was taken when the receiving unit 110 received the shot detection signal within a predetermined time after receiving the passage detection signal. Then, when it is specified that the shot has been performed in this way, the image taken by the camera 62 before the predetermined time of the timing at which the shot detection signal is received may be specified as the second target image. .. Alternatively, when the shot timing is specified in this way, the image taken by the camera 62 before the predetermined time of the timing at which the passage detection signal is received may be specified as the second target image.

Further, as described above, the camera 62 may start taking an image at the timing when the arrangement is detected by the ball presence / absence determination sensor 44. Then, the second target image specifying unit 118 may specify the image taken by the camera 62 after the timing when the arrangement is detected as the second target image.

In the present embodiment, the user image data storage unit 120 stores user image data including, for example, a user ID which is user identification information (ID) and at least one image of the user. Here, for example, the user image data including the face image of the user and the user ID of the user, which are taken when the user is registered as a member, may be stored in the user image data storage unit 120.

In the present embodiment, the user determination unit 122 acquires, for example, a plurality of images of users who have taken shots corresponding to the shot data among the plurality of users, which are associated with the plurality of shot data. Here, for example, the user determination unit 122 acquires a plurality of second target images. Then, in the present embodiment, the user determination unit 122 determines which of the plurality of users the shot data associated with the second target image is associated with, for example, based on each second target image.

Here, the user determination unit 122 may extract a portion representing the user's face from the second target image by using, for example, an image recognition technique. Hereinafter, the portion extracted in this way will be referred to as an extracted partial image. Then, the user determination unit 122 may determine the user represented by the second target image based on the extracted partial image.

Here, for example, the user may use an image recognition technique to determine from among the images of the user stored in the user image data storage unit 120, those representing the same user as the user represented by the extracted partial image. .. Then, in the user image data, the user ID associated with the determined user's image may be specified. Then, the user determination unit 122 may add the extracted partial image to the user image data. After that, the extracted partial image will be treated as an image of the user included in the user image data.

Here, it may be determined that among the images of the user stored in the user image data storage unit 120, there is no image representing the same user as the user represented by the extracted partial image. In this case, the user determination unit 122 may generate new user image data including a new user ID that does not overlap with any user image data and an extracted partial image. Then, the user determination unit 122 may store the generated user image data in the user image data storage unit 120. After that, the extracted partial image will be treated as a user image included in the new user image data.

In the present embodiment, the shot management data generation unit 124 includes, for example, club type data indicating the type of golf club estimated by the club type estimation unit 116 for the shot in the above-mentioned shot data associated with one shot. Then, the user who is determined by the user determination unit 122 for the shot is associated with the shot. Here, for example, a shot including the above-mentioned shot data associated with one shot, club type data indicating the type of golf club estimated for the shot, and the user ID of the user determined for the shot. Management data is generated.

For example, when it is determined that the image of the user stored in the user image data storage unit 120 represents the same user as the user represented by the extracted partial image, the shot management data including the user ID of the user is stored. Will be generated. Further, for example, when it is determined that none of the images of the user stored in the user image data storage unit 120 represents the same user as the user represented by the extracted partial image, a new image is generated. Shot management data including the user ID included in the user image data is generated.

The shot management data may include shot timing data indicating the shot timing, such as the reception time of the shot detection signal.

In the present embodiment, the shot management data storage unit 126 stores, for example, the shot management data generated by the shot management data generation unit 124. In the present embodiment, each time a shot is taken, shot management data corresponding to the shot is generated. Then, the generated shot management data is stored in the shot management data storage unit 126.

In the present embodiment, the aggregation unit 128 aggregates a plurality of shot management data stored in the shot management data storage unit 126, for example.

Here, the aggregation unit 128 may calculate, for example, a representative value of a value indicated by each of a plurality of shot data associated with a common golf club type. For example, the aggregation unit 128 may specify a plurality of shot management data including club type data indicating the type specified by the user.

Further, the aggregation unit 128 may calculate, for example, a representative value of the value indicated by each of the plurality of shot data associated with the common user ID. For example, the aggregation unit 128 may specify a plurality of shot management data including a user ID specified by the user.

The shot management device 10 may include a photographing device such as a camera. Then, the totaling unit 128 collates the user's image taken by the photographing device with the user's image included in the user image data stored in the user image data storage unit 120 by the image recognition technology. , The user ID of the user may be determined. Then, the aggregation unit 128 may specify a plurality of shot management data including the determined user ID.

Further, the aggregation unit 128 may calculate, for example, a common user ID and a representative value of a value indicated by each of a plurality of shot data associated with a common golf club type. For example, the aggregation unit 128 may specify a plurality of shot management data including a user ID specified by the user and club type data indicating the type specified by the user. Further, a plurality of shot management data including a user ID determined based on the user's image taken by the above-mentioned photographing device and club type data indicating the type specified by the user may be specified.

Then, the aggregation unit 128 may calculate a representative value of the value indicated by the shot data included in each of the plurality of shot management data.

For example, ball speed data, launch angle data, spin amount data, flight distance data, head speed data, face speed data, head trajectory data, dynamic loft data, face angle data, lie angle data, hitting point position data, normal angle. For at least one of the data or face rotation data, even if a representative value of the value of the data (for example, at least one of the average value, the standard deviation, the maximum value, or the minimum value) is calculated. Good.

In the present embodiment, the providing unit 130 provides the user with the aggregation result calculated by the aggregation unit 128. Here, in the present embodiment, the providing unit 130 may display the aggregation result calculated by the aggregation unit 128 on the display unit 18. Further, the aggregation result calculated by the aggregation unit 128 may be printed out.

Here, for example, even if the totaling unit 128 calculates the representative value of the value indicated by each of the plurality of shot data associated with the type according to the user's designation of the golf club type. Good. For example, the aggregation unit 128 may specify a plurality of shot management data including club type data indicating the specified type. Then, the aggregation unit 128 may calculate a representative value of the values indicated by the plurality of shot management data.

Further, for example, the aggregation unit 128 may calculate the representative value of the value indicated by each of the plurality of shot data associated with the user ID according to the designation of the user ID by the user. For example, the aggregation unit 128 may specify a plurality of shot management data including the designated user ID. Alternatively, the aggregation unit 128 may specify a plurality of shot management data including a user ID determined based on the user's image taken by the above-mentioned photographing device. Then, the aggregation unit 128 may calculate a representative value of the values indicated by the plurality of shot management data.

Further, for example, the totaling unit 128 has the type of golf club and a plurality of shot data associated with the user ID according to the designation of the golf club type and the user ID by the user. The representative value of the value indicated by each of the above may be calculated. For example, the aggregation unit 128 may specify a plurality of shot management data including golf type data indicating a designated type and a designated user ID. Alternatively, the aggregation unit 128 identifies a plurality of shot management data including golf type data indicating the specified type and a user ID determined based on the image of the user photographed by the above-mentioned photographing device. May be good. Then, the aggregation unit 128 may calculate a representative value of the values indicated by the plurality of shot management data.

Then, the providing unit 130 may display the calculated representative value on the display unit 18 or print out the calculated representative value. Here, the providing unit 130 may display a list of values indicated by each of the plurality of shot management data on the display unit 18 or print out the list.

Further, the providing unit 130 may calculate a representative value of a value indicated by a plurality of shot management data corresponding to the type of the golf club for each type of golf club. Then, a list of representative values for each type of golf club may be displayed on the display unit 18 or printed out.

Further, the providing unit 130 may calculate a representative value of a value indicated by a plurality of shot management data corresponding to the user for each user who shoots the golf ball 48. Then, a list of representative values for each user may be displayed on the display unit 18 or printed out.

Further, the providing unit 130 may calculate a representative value of a value indicated by a plurality of shot management data corresponding to the user for each combination of the type of golf club and the user who shoots the golf ball 48. Then, a list of representative values for each combination of the type of golf club and the user who shoots the golf ball 48 may be displayed on the display unit 18 or printed out.

Here, an example of the flow of processing performed by the shot management device 10 according to the present embodiment will be described with reference to the flow chart illustrated in FIG. In the processing example shown in FIG. 11, shot management data associated with one shot is generated and stored in response to the reception of the shot detection signal.

First, the shot data generation unit 112 waits for the reception unit 110 to receive the shot detection signal (S201).

Then, when the reception of the shot detection signal by the receiving unit 110 is detected, the shot data generation unit 112 generates shot data (S202). Here, for example, the image received by the receiving unit 110 from the ballistic measuring device 30 and the head behavior measuring device 32 within a predetermined time before or after the reception timing of the shot detection signal, and the head speed received from the ballistic measuring device 30. The data is identified. Then, shot data is generated based on the specified image and head speed data.

Then, the first target image identification unit 114 identifies the first target image as described above (S203).

Then, the club type estimation unit 116 estimates the type of golf club appearing in the first target image based on the first target image specified by the process shown in S203, and determines the estimated type of golf club. The indicated club type data is generated (S204). Here, as described above, the type of golf club may be estimated based on the output when the first target image is input to the machine learning model 90.

Then, the second target image identification unit 118 identifies the second target image as described above (S205).

Then, the user determination unit 122 determines the user ID of the user appearing in the second target image based on the second target image specified in the process shown in S205 (S206).

Then, the user determination unit 122 executes a process for the user image data stored in the user image data storage unit 120 (S207).

In the process shown in S206, for example, in the user image data, the user ID associated with the image of the user representing the same user as the user represented by the extracted partial image is determined. Then, in this case, the extracted partial image is added to the user image data including the user ID by the process shown in S207.

Further, in the process shown in S206, for example, when it is determined that the image of the user representing the same user as the user represented by the extracted partial image does not exist, a new user ID is issued. Then, in this case, the process shown in S207 generates new user image data including the new user ID and the extracted partial image. Then, the generated user image data is stored in the user image data storage unit 120.

Then, the shot management data generation unit 124 includes the shot data generated by the process shown in S202, the club type data generated by the process shown in S204, and the user ID determined by the process shown in S206. Generate management data (S208).

In the process shown in S208, when it is determined that the image of the user stored in the user image data storage unit 120 represents the same user as the user represented by the extracted partial image, the user ID of the user is included. Shot management data is generated. Further, when it is determined that none of the user images stored in the user image data storage unit 120 represents the same user as the user represented by the extracted partial image, a newly generated user image is generated. Shot management data including the user ID included in the data is generated.

Then, the shot management data generation unit 124 stores the shot management data generated in the process shown in S208 in the shot management data storage unit 126 (S209), and the process shown in this processing example is completed.

In the present embodiment, the user determination unit 122 may determine which of the plurality of users the shot data is associated with by using the trained machine learning model. For example, images of a plurality of users may be classified by performing unsupervised learning such as clustering. Then, using the trained machine learning model in which the unsupervised learning is executed, the user determination unit 122 determines the user ID of the user appearing in the second target image based on the second target image. You may.

Further, for example, the user image data storage unit 120 may store the user image data including the user ID and the feature vector indicating the features of the user's image. The feature vector may be extracted, for example, by inputting a user's image into a trained self-encoder. Then, in this case, the feature vector corresponding to the second target image may be extracted by inputting the second target image into the trained self-encoder. Then, from the feature vectors included in the user image data, the one having the shortest distance from the feature vector corresponding to the second target image may be specified. Then, the user determination unit 122 may determine the user ID associated with the specified feature vector as the user ID of the user appearing in the second target image.

In the present embodiment, the learning of the machine learning model 90 is executed by the learning input image generated by executing the preprocessing on the basic image. Therefore, the machine learning model 90, which can accurately estimate the type of golf club, can be learned by using a small number of images.

Further, in the present embodiment, the totaling unit 128 aggregates shot data for each type of golf club. Therefore, according to the present embodiment, it is possible to reduce the time and effort required to aggregate shot data for each type of golf club.

Further, in the present embodiment, the shot management data generation unit 124 associates the user who shoots the golf ball 48 with the shot data. Therefore, it is possible to reduce the time and effort required to manage the shot data in association with the user who performed the shot.

Further, in the present embodiment, the aggregation unit 128 aggregates the shot data for each user who shoots the golf ball 48. Therefore, according to the present embodiment, it is possible to reduce the time and effort required to aggregate the shot data for each user who has performed the shot.

In the present embodiment, the club type estimation unit 116 sets the golf club and the golf club based on at least one of the position, size, or color of the area occupied by the image of the golf ball 48 in the first target image. An input image different from the first target image representing at least the golf ball 48 may be generated. For example, the input image may be generated based on the first target image by an algorithm similar to the algorithm used when generating the learning input image based on the basic image.

For example, the club type estimation unit 116 may generate an input image in which the position of the region occupied by the image of the golf ball 48 in the input image is a predetermined position.

Further, for example, the club type estimation unit 116 may generate an input image in which the size of the region occupied by the image of the golf ball 48 is a predetermined size.

Further, for example, the club type estimation unit 116 may generate an input image in which the entire image is color-corrected so that the color of the region occupied by the image of the golf ball 48 becomes a predetermined color.

Further, for example, the first target image is an image further representing the artificial turf, and the club type estimation unit 116 is an input image in which the entire image is color-corrected so that the color of the region indicated by the artificial turf image becomes a predetermined color. May be generated.

Further, for example, the club type estimation unit 116 may generate an input image in which the entire image is rotated so that the orientation of the edges of the artificial turf is a predetermined orientation.

Further, for example, even if the first target image is an image further representing the tee 50 and the club type estimation unit 116 generates an input image in which the entire image is rotated so that the orientation of the tee 50 becomes a predetermined orientation. Good.

Then, the club type estimation unit 116 may estimate the type of the golf club based on the output when the input image is input to the machine learning model 90.

In this case, the machine learning model 90 does not have to be a machine learning model in which learning using the output corresponding to the input of the learning input image generated by the preprocessing unit 96 is executed. For example, a learning input image taken so that the golf club and the golf ball 48 are oriented in a predetermined direction, have a predetermined angle of view, and have a predetermined color, and teacher data indicating the type of the golf club are included in advance. It may be a machine learning model in which training is executed using a plurality of created training data.

As described above, the output when an input image similar to the training input image generated based on the first target image is input to the machine learning model 90, relying on the images of the golf ball 48, the tee 50, and the artificial turf. The type of golf club may be estimated based on. In this case, regardless of whether or not the machine learning model 90 has been trained using the output corresponding to the input of the learning input image generated by the preprocessing unit 96, the type of golf club is accurately selected. Learning of the machine learning model 90 that can be estimated can be performed using a small number of images.

The present invention is not limited to the above-described embodiment.

For example, the shot measuring device 22 may include a tapping sound detection sensor that detects the tapping sound. Then, at the timing when the tapping sound is detected by the tapping sound detection sensor, the tapping sound detection sensor may transmit a predetermined shot detection signal to the shot management device 10. Then, the receiving unit 110 may receive the shot detection signal.

In this case, the first target image specifying unit 114 may specify the image taken by the camera 60 of the head photographing device 34 as the first target image at a predetermined time before the timing at which the tapping sound is detected. .. For example, an image taken by the camera 60 before a predetermined time when the receiving unit 110 receives the shot detection signal transmitted by the tapping sound detection sensor may be specified as the first target image.

Further, in this case, even if the second target image specifying unit 118 specifies the image taken by the camera 62 of the user photographing device 36 as the second target image before the predetermined time when the tapping sound is detected. Good. For example, an image taken by the camera 62 before a predetermined time when the receiving unit 110 receives the shot detection signal transmitted by the tapping sound detection sensor may be specified as the second target image.

Further, the division of roles between the shot management device 10 and the shot measuring device 22 is not limited to the above. For example, a part or all of the functions shown in FIG. 10 mounted on the shot management device 10 may be mounted on the shot measuring device 22. Further, for example, the shot management device 10 may generate head speed data.

Further, the function of the head photographing device 34 may be mounted on the head behavior measuring device 32.

Further, for example, as shown in FIG. 12, a plurality of shot measurement systems 1 may be able to communicate with a terminal 132 which is a computer such as a personal computer similar to the shot management device 10. Then, each shot measurement system 1 may transmit the shot management data generated by the shot management data generation unit 124 to the terminal 132. In this case, the shot management data to be transmitted may be associated with the identification information of the shot measurement system 1 that transmits the shot management data.

Then, the terminal 132 may have the functions of the above-mentioned aggregation unit 128 and provision unit 130. For example, as described above, the terminal 132 may aggregate a plurality of shot management data received from the plurality of shot measurement systems 1. Then, the terminal 132 may provide the aggregation result calculated by the terminal 132 to the user of the terminal 132.

When the shot measurement system 1 is provided for each turn at bat in the driving range, the shot data generated by the plurality of turn at bats can be analyzed in a cross-sectional manner by performing the above. For example, it is possible to analyze shot data for each shot, each type of golf club used for the shot, and each user who made the shot.

Further, by providing the terminal 132 at the reception desk of the driving range, for example, it is possible to provide the user who has practiced the shot at the driving range with the shot management data for the shot by the user and the aggregated result thereof. it can.

For example, a coach may make a model shot at bat during a lesson at a driving range. Even in such a situation, according to the present embodiment, the shot management data for shots by the coach is excluded from the practicing user (student), and the shot management data for the shots by the user and the aggregated result thereof are provided to the user (student). Can be provided.

Further, even if the user takes a shot at a plurality of at-bats, the shot management data and the aggregated result of the shots by the plurality of at-bats can be collectively provided to the user.

The place where the shot measurement system 1 according to the present embodiment is provided is not particularly limited, and for example, the shot measurement system 1 may be provided in a golf shop.

Further, the club type estimation unit 116 may estimate the type of golf club used for the shot represented by the first target image without using the trained machine learning model 90.

Further, the functions are not limited to the above-mentioned aggregation unit 128 and provision unit 130, and for example, the machine learning model 90, the reception unit 110, the shot data generation unit 112, the first target image identification unit 114, the club type estimation unit 116, and the second. A part or all of the functions of the target image identification unit 118, the user image data storage unit 120, the user determination unit 122, the shot management data generation unit 124, and the shot management data storage unit 126 may be implemented in the terminal 132.

For example, the club type estimation unit 116 separates pixels having a pixel value in a specific range corresponding to the club head in the first target image from pixels having a pixel value corresponding to a green background such as artificial turf. It may be extracted.

Then, the first feature information regarding the size of the club head or the shape of the club head may be specified. Here, the total number of pixels of the extracted club head image may be the size of the club head. Further, from the extracted area of the club head image, information on the contour shape of the image of the crown portion and the side portion obtained by removing the image of the tubular socket portion into which the shaft of the club head is inserted is specified. May be good. In the case of a wood-based golf club, the contour shape differs greatly depending on the type, so that the type of golf club can be determined from the contour shape of the extracted image of the crown portion. The contour shape can be characterized by, for example, the tilt angle of the curve representing the contour or the curvature of the curve. Further, the aspect ratio of the images of the crown portion and the side portion may be specified.

Then, the type of golf club may be estimated based on the specified first feature information. For example, the club type estimation unit 116 may hold in advance a reference table in which the first feature information and the type of the golf club are associated with each other. Then, the type of golf club may be estimated by referring to this reference table from the specified first feature information.

Further, the club type estimation unit 116 holds in advance reference images of a plurality of club heads having at least one second feature information of a pattern, a symbol, or a color, which can be associated with a model name or model number of a golf club. You may be doing it. Then, with respect to the image of the club head in the first target image, the held reference image is called and matched with the second feature information in each reference image, so that the model name or model number of the golf club can be changed. It may be estimated.

The second feature information is provided on the contour shape of the club head, the pattern of the score line provided on the face surface, the form of the punch mark provided on the face surface, the decorative pattern formed on the face surface, or the club head. It may contain at least one of the logo marks.

Further, in the present embodiment, the shooting timing and shooting range of the ballistic measuring instrument 30 by the stereo camera 42 may be changed according to the estimation result of the type of golf club.

Moreover, the above-mentioned specific numerical values are examples, and are not limited to these numerical values.

1 shot measurement system, 10 shot management device, 12 processor, 14 storage unit, 16 communication unit, 18 display unit, 20 operation unit, 22 shot measurement device, 30 ballistic measurement device, 32 head behavior measurement device, 34 head imaging device, 36 User shooting device, 40 flash light source, 42 stereo camera, 44 ball presence / absence detection sensor, 46 head speed sensor, 48 golf ball, 50 tee, 52 flash light source, 54 camera, 56 reflector, 58 pass sensor, 60 camera, 62 Camera, 70 learning device, 72 processor, 74 storage unit, 76 communication unit, 78 display unit, 80 operation unit, 90 machine learning model, 92 learning data storage unit, 94 learning data acquisition unit, 96 preprocessing unit, 98 learning unit , 100 golf club, 102 edge, 110 receiver, 112 shot data generation unit, 114 first target image identification unit, 116 club type estimation unit, 118 second target image identification unit, 120 user image data storage unit, 122 user determination Unit, 124 shot management data generation unit, 126 shot management data storage unit, 128 aggregation unit, 130 provision unit, 132 terminals.

Claims (13)

A shot data generation means for generating a plurality of shot data indicating at least one of the behavior of a golf club or the behavior of the golf ball when a golf ball is shot.
An acquisition means for acquiring a plurality of images of a user who has performed a shot corresponding to the shot data among a plurality of users, which are associated with each of the plurality of shot data.
Based on each of the images, a determination means for determining which of the plurality of users the shot data associated with the image is associated with, and
An association means for associating the determined user with the shot data,
A shot management system characterized by including. A calculation means for calculating a representative value of a value indicated by each of the plurality of shot data associated with the common user is further included.
The shot management system according to claim 1. The calculation means calculates at least one of the mean value, standard deviation, maximum value, or minimum value of the values indicated by the plurality of shot data.
The shot management system according to claim 2, wherein the shot management system is characterized in that. The acquisition means acquires an image of the user taken between the timing of the address in the shot and the timing of the shot.
The shot management system according to any one of claims 1 to 3, wherein the shot management system is characterized in that. The acquisition means acquires an image of the user taken before a predetermined time of the timing of the shot to be detected.
The shot management system according to claim 4, wherein the shot management system is characterized in that. The acquisition means acquires an image of the user taken before a predetermined time when a change from the state in which the ball is arranged to the state in which the ball is not arranged is detected.
The shot management system according to claim 5. The acquisition means acquires an image of the user taken before a predetermined time when the tapping sound is detected.
The shot management system according to claim 5. The acquisition means acquires an image of the user taken before a predetermined time when the passage of the golf club is detected.
The shot management system according to claim 5. The acquisition means acquires an image of the user taken after the timing when it is detected that the ball is arranged.
The shot management system according to claim 5. The acquisition means acquires an image of the user to be captured by the photographing means that starts shooting at the timing of the address in the shot and ends shooting at the timing of the shot.
The shot management system according to any one of claims 4 to 9, wherein the shot management system is characterized in that. The determination means uses a trained machine learning model to determine which of the plurality of users the shot data is associated with.
The shot management system according to any one of claims 1 to 10. A step of generating a plurality of shot data indicating at least one of the behavior of a golf club or the behavior of the golf ball when a golf ball is shot, and
A step of acquiring a plurality of images of a user who has performed a shot corresponding to the shot data among a plurality of users, which are associated with each of the plurality of shot data.
Based on each of the images, a step of determining which of the plurality of users the shot data associated with the image is associated with, and
A step of associating the determined user with the shot data,
A shot management method characterized by including. A procedure for generating a plurality of shot data indicating at least one of the behavior of a golf club or the behavior of the golf ball when a golf ball is shot.
A procedure for acquiring a plurality of images of a user who has taken a shot corresponding to the shot data among a plurality of users, which are associated with the plurality of shot data.
A procedure for determining which of the plurality of users the shot data associated with the image is associated with, based on each of the images.
A procedure for associating the determined user with the shot data,
A program characterized by having a computer execute.

Images