JP2021029484A - Determination verification system, determination verification method and computer program - Google Patents

Abstract

To shorten the time required for verification of the determination and enhance the reliability of the verification result.SOLUTION: A determination verification system 40 includes an information processing device 41, a photographing device 42, and a sensor 43. The photographing device 42 photographs a verification area that can be a verification target for verifying determination of a competition. The sensor 43 is arranged in the verification area and detects vibration, sound, or pressure. The information processing device 41 has a function for generating information to be used for verification of the determination of the competition using an image photographed by the photographing device 42 and a sensor signal output from the sensor 43.SELECTED DRAWING: Figure 4

Description

The present invention relates to a technique for verifying a judgment by a referee in a sports competition such as baseball.

From the 2018 season, the Nippon Professional Baseball Organization (NPB) has introduced a system (so-called request system) that allows you to request verification of a judgment when there is an objection to the judgment.

In the verification of the judgment, the video judgment is made by utilizing the video image (moving image) taken and recorded. However, even if the video image is used, for example, the touching base (the act of the runner touching the base and occupying the base) and the tag out (the defensive act of the fielder holding the ball touching the runner or the base) are almost simultaneous. Therefore, it may be difficult to determine which of the touch base and the tag out was faster. Also, in the video image, for example, the touch base and the tag out may not be clearly reflected in the shadow of the fielder, and even in this case, it is difficult to determine which of the touch base and the tag out was faster. ..

In addition, Patent Document 1 discloses a strike zone providing system, and Patent Document 2 discloses an automatic strike determination device.

Japanese Patent No. 5348004 Japanese Unexamined Patent Publication No. 9-2900037

As mentioned above, it may be difficult to determine which of the touch base and the tag out was faster even if the video is used, and in such a case, the audience may feel distrust of the verification result. .. In addition, it takes a lot of time (for example, 10 minutes or more) to determine the video, which causes a problem that the flow of the game is stopped and the spectators are whitened.

The present invention has been made to solve the above problems. That is. A main object of the present invention is to provide a technique capable of shortening the time required for verification of determination and improving the reliability of the verification result.

In order to achieve the above object, the determination verification system according to the present invention is, as one form thereof.
A shooting device that captures the verification area that can be the verification target for verifying the judgment of the competition,
A sensor that is placed in the verification area and detects vibration, sound, or pressure,
An information processing device that generates information to be used for verification of the determination by using an image captured by the photographing device and a sensor signal output from the sensor is provided.

The determination verification method according to the present invention is, as one form thereof.
Obtain an image taken by a shooting device that shoots a verification area that can be a verification target to verify the judgment of the competition.
Acquire the sensor signal output from the sensor that is placed in the verification area and detects vibration, sound, or pressure.
Using the captured image and the sensor signal, information used for verification of the determination is generated.

The computer program according to the present invention is, as one form thereof.
The process of acquiring the image taken by the photographing device that photographs the verification area that can be the verification target for verifying the judgment of the competition, and
The process of acquiring the sensor signal output from the sensor that is placed in the verification area and detects vibration, sound, or pressure.
Using the captured image and the sensor signal, a computer is made to execute a process of generating information used for verification of the determination.

According to the present invention, the time required for verification of determination can be shortened, and the reliability of the verification result can be improved.

It is a block diagram which shows the structure of the determination verification system of 1st Embodiment which concerns on this invention. It is a flowchart which shows an example of the control operation of the information processing apparatus which constitutes the determination verification system of 1st Embodiment. It is a figure explaining the determination verification system of the 2nd Embodiment which concerns on this invention. It is a block diagram which shows the other embodiment of the determination verification system which concerns on this invention.

Hereinafter, embodiments according to the present invention will be described with reference to the drawings.

<First Embodiment>
FIG. 1 is represented by a simplified block diagram of a configuration example of the determination verification system of the first embodiment according to the present invention. The judgment verification system 1 of the first embodiment is a system that verifies the judgment by the referee in a baseball game, particularly the judgment of the competition in the area around the four bases including the first base, the second base, the third base, and the home base. Is. The determination verification system 1 includes an information processing device 3, a photographing device 4, a sensor 5, a display device 6, and a speaker 7.

The photographing device 4 is a device that is installed at a position for photographing an area that can be a verification target for verifying the determination (hereinafter, also simply referred to as a verification area) and photographs the verification area. That is, in the first embodiment, the verification area is the area around the base. For this reason, the photographing device 4 is installed at a position for photographing the area around the base and photographs the area around the base. The shooting device 4 has a function of shooting a moving image. The frame rate of the moving image captured by the photographing device 4 is not limited, but is set to an appropriate frame rate in consideration of, for example, the accuracy of verification of determination and the time required for verification.

Further, the number of photographing devices 4 used for verification of determination is not limited. For example, a plurality of photographing devices 4 for photographing the areas around the four bases are installed, and the images captured by the plurality of photographing devices 4 are used for verification of determination. Further, for example, a plurality of photographing devices 4 for photographing one base peripheral area (verification area) from different directions are installed, and a plurality of photographing devices 4 are correspondingly installed in one base peripheral area. May be good.

The photographing device 4 does not have to be fixed. For example, if the photographing device 4 can photograph the area around the base (verification area) without interfering with the competition, the moving mechanism (rail type moving mechanism, small flying object (so-called drone (UCAV (Unmanned Combat Air Vehicle))), etc. ) May be mounted and movable.

In the first embodiment, the photographing device 4 has a communication function and is connected to the time supply source 22 wirelessly or by wire. The time supply source 22 has a function of providing time information (time information), for example, an NTP (Network Time Protocol) server or a GPS (Global Positioning System) satellite, and the photographing device 4 is transmitted from the time supply source 22. It has a function to acquire time information.

Further, the photographing device 4 can be connected to the information processing device 3 by the communication function. The photographing device 4 has a function of outputting to the information processing device 3 a photographed image in which the identification information for identifying the photographing device 4 and the time information acquired from the time supply source 22 are associated with each other.

The sensor 5 is a sensor that detects vibration, sound, or pressure, and is attached to a place where vibration, sound, or pressure caused by the movement of a competitor (runner or fielder) can be detected (that is, a verification area (base peripheral area)). Be done. Although only one sensor 5 is shown in FIG. 1, a plurality of sensors 5 are used in the first embodiment in order to improve the reliability of the verification of the determination. For example, a vibration sensor is embedded as a sensor 5 in, for example, the central portion of each of the four bases. Alternatively, a pressure sensor may be embedded in the base as the sensor 5 instead of the vibration sensor. Alternatively, both the vibration sensor and the pressure sensor (that is, a plurality of types of sensors) may be embedded in the base as the sensor 5. The vibration sensor and the pressure sensor output a sensor signal according to the vibration of the base caused by the competitor pressing (touching) the base and the pressure acting on the base.

Further, in the first embodiment, a microphone for collecting the sound in the region around the base is provided on the base as the sensor 5. The microphone can collect the sound caught by the glove. The microphone outputs a sensor signal according to the collected sound.

The various sensors 5 as described above are preferably small in size so as not to interfere with the competition and not to affect the movement of the athlete. Further, when the sensor 5 is installed on the base, it is assumed that a large force acts on the sensor 5. Therefore, the sensor 5 has strength according to the installation location, and protective measures are taken to protect the sensor 5. Further, when a plurality of sensors 5 are embedded in the same base, the plurality of sensors 5 may be integrated in a common housing to form one component. The installation location of the sensor 5 is not limited to the base. For example, the sensor 5 may be provided on a glove or a ball. In this case, the sensor 5 is preferably small in order to reduce the influence on the movement of the player and the movement of the ball, and the location of the sensor on the glove or ball is based on the results of experiments and simulations. Set.

The sensor 5 has a communication function, is connected to the same time supply source 22 as the photographing device 4, and has a function of acquiring time information from the time supply source 22. The sensor 5 can also be connected to the information processing device 3 by the communication function. The sensor 5 has a function of outputting a sensor signal associated with the identification information for identifying the sensor 5 and the time information acquired from the time supply source 22 to the information processing device 3.

The information processing device 3 is a computer device, and includes a control device (here, a processor) 10 and a storage device 20. The storage device 20 has a configuration for storing data and a computer program (hereinafter, also referred to as a program) 21. There are various types of storage devices 20, and any type of storage device may be adopted here. Further, although a plurality of storage devices may be mounted on the information processing device 3, in the description of this embodiment, they are collectively referred to as a storage device 20, and each of them is not distinguished. An external storage device 24 may be connected to the information processing device 3. When the external storage device 24 is connected to the information processing device 3, the information processing device 3 appropriately reads and writes data to and from the external storage device 24. The description of reading and writing data regarding the storage device 24 of the above is omitted.

The control device 10 can have a function corresponding to the program 21 by reading and executing the program 21 stored in the storage device 20. In the first embodiment, the control device 10 includes an acquisition unit 12, an analysis unit 13, a synchronization unit 14, and a provision unit 15 as functional units.

The acquisition unit 12 has a function of acquiring a photographed image from the photographing device 4 and a sensor signal from the sensor 5. In addition, the acquisition unit 12 further includes a function of writing the acquired captured image and sensor signal information to the storage device 20. Identification information is set for each of the areas around the base, which is the verification area. The storage device 20 stores information in which the identification information of each base peripheral area is associated with the identification information of the corresponding photographing device 4 and the sensor 5. Using the information, the acquisition unit 12 stores the captured image of the corresponding photographing device 4 and the sensor signal of the sensor 5 in the storage device 20 for each area around the base. At this time, the captured image and the sensor signal are associated with the identification information of the corresponding base peripheral area.

The analysis unit 13 has a function of executing an analysis process for extracting information to be extracted from the sensor signal. For example, when the sensor 5 is a vibration sensor or a pressure sensor, a vibration amplitude value or a pressure value as a threshold value is given in advance, and the analysis unit 13 determines the time when the vibration amplitude or pressure equal to or higher than the threshold value is detected. (Time) information is extracted. When the sensor 5 is a microphone, the sound information to be extracted (for example, the frequency and frequency of the sound when the glove catches the ball and the sound when the runner steps on the base or touches the base by hand). Information such as change patterns) is given in advance. The analysis unit 13 uses the sound information given in advance to extract the sound to be extracted and the time (time) information from which the sound is extracted from the sensor signal by the microphone.

At the timing when the analysis unit 13 starts the analysis process as described above, the user inputs an analysis start command to the information processing device 3 using the input device 8 (for example, a keyboard, a touch panel, or a mouse), and the information processing device 3 Recognizes the command to start analysis. More specifically, for example, a captured image in the area around the base and a rough time zone for which verification is required is read from the storage device 20 by a manual operation of the user, and the read captured image is displayed on the display device 6. It is assumed that it is displayed. In this state, the analysis starts after the time (time) for starting the analysis and the time (time) for ending the analysis are specified by the user's manual instruction while referring to the displayed captured image. Is input to the information processing device 3 by the input device 8. As a result, the analysis unit 13 starts the analysis process by reading the sensor signal of the time zone of the verification target (analysis target) in the area around the base where the verification is requested from the storage device 20.

The analysis unit 13 further includes a function of associating the information extracted from the sensor signal (information of the analysis result) with the identification information of the sensor signal of the extraction source and storing the information in the storage device 20.

The synchronization unit 14 has a function of synchronizing the captured image and the sensor signal by using the time information associated with the captured image and the sensor signal. Further, the synchronization unit 14 has a function of generating data for provision based on the captured image and the sensor signal after synchronization and the information of the analysis result by the analysis unit 13. Specific examples of the data to be provided will be described below.

The provided data is, for example, superimposing the extracted sound data (that is, the sound data when the glove catches the ball) on the captured image taken at the time when the sound data is detected by the sensor (microphone) 5. This is data for notifying the timing of catching a ball. Further, the data for provision is obtained by superimposing a screen display such as a mark or a color for notifying that the ball has been caught and an alarm sound on the captured image taken at the time when the sound data of the catching ball is detected by the sensor 5. It may be data for notifying the timing of catching a ball. Further, the provided data is, for example, a photograph taken at the time when the vibration or pressure (for example, the runner or the fielder touching the base) extracted from the sensor signal of the vibration sensor or the pressure sensor is detected by the sensor 5. The data may be, for example, for notifying the timing of touching by superimposing a screen display such as a mark or a color or an alarm sound on the image. As described above, the provision data is data for providing information that facilitates the verification of the judgment by the referee (in other words, the information used for the verification of the judgment).

The synchronization unit 14 further stores the provision data in the storage device 20 in a state in which the identification information of the corresponding base peripheral area is associated with the data.

Based on the data provided by the synchronization unit 14, the providing unit 15 uses the display device 6 and, if necessary, the speaker 7 to obtain information used for verification of the determination as a user (determination). It has a function to provide to the person who verifies it.

The determination verification system 1 of the first embodiment has the above-described configuration. Next, an example of the operation of the control device 10 of the information processing device 3 will be described with reference to the flowchart of FIG. The operation of the control device (processor) 10 shown in FIG. 2 is realized by executing the program 21.

For example, the acquisition unit 12 acquires a photographed image from the photographing device 4 and a sensor signal from the sensor 5 (step S101). The acquisition unit 12 stores the acquired information in the storage device 20 in association with the identification information of the corresponding base peripheral area. In other words, the acquisition unit 12 stores the acquired information in the storage device 20 in a state of being classified for each base peripheral area which is a verification area.

After that, when the information processing device 3 receives a command to start analysis based on the operation information of the input device 8 (step S102), the analysis unit 13 verifies the area around the base to be verified based on the user's input information. The captured image in the time zone of the target (analysis target) is read out from the storage device 20. Further, the analysis unit 13 reads out the sensor signal of the time zone of the verification target (analysis target) in the area around the base to be verified from the storage device 20. Then, the analysis unit 13 analyzes the sensor signal (step S103).

After that, the synchronization unit 14 synchronizes the captured image with the sensor signal by using the time information associated with the captured image and the sensor signal (step S104). Further, the synchronization unit 14 generates data for provision based on the captured image after synchronization, the sensor signal, and the analysis result by the analysis unit 13. Then, the providing unit 15 provides the information used for the verification of the determination based on the providing data by using the display device 6 or the like (step S105).

The determination verification system 1 of the first embodiment acquires information on vibration, pressure, and sound caused by the movements of a runner or a fielder (athlete) involved in the determination of a competition by using a sensor. Then, the determination verification system 1 can provide the person who verifies the determination by superimposing the information related to the movement of the runner or the fielder (athlete) based on such a sensor on the image captured by the photographing device 4. That is, the determination verification system 1 can provide information such as which of the touch base and the touch ball, which is difficult to obtain only from the captured image, is faster by using the sensor signal. As a result, the determination verification system 1 can shorten the time required for the determination verification and improve the reliability of the verification result.

<Second Embodiment>
The second embodiment according to the present invention will be described below with reference to the drawings. In the description of the second embodiment, the same reference numerals will be given to the components having the same names as the components constituting the determination verification system 1 of the first embodiment, and the duplicate description of the common parts will be omitted.

The determination verification system 1 of the second embodiment is provided with a determination unit 16 as shown in FIG. 3 in addition to the configuration of the first embodiment. Note that, in FIG. 3, the control device 10 in the determination verification system 1 is extracted and shown, and other configurations of the determination verification system 1 are omitted in FIG.

In the determination verification system 1 of the second embodiment, the providing unit 15 not only provides the information used for the verification of the determination to the person who verifies the determination by using the display device 6 or the like, but also the determination unit 16 The judgment result by is also provided to the person who verifies the judgment. That is, the determination unit 16 receives the data for provision by the synchronization unit 14 via the provision unit 15, and uses the determination information predetermined based on the rule information of the competition and the provision data to be verified. It has a function to judge the competition. The determination information is stored in the storage device 20. As a specific example of the determination information, for example, the base is stepped on based on the sensor signal of the sensor 5 that detects the pressure while the photographed image recognizes that the runner's shoes are located on the base. Is detected, and then, when it is detected that the ball has been caught based on the sensor signal of the sensor 5 that detects the sound, it is determined that the ball is safe. When such determination information is given, the analysis unit 13 uses AI (Artificial Intelligence) technology to indicate from the captured image that the runner's foot is located on the base and the position. It has a function to recognize the timing.

Further, as another specific example of the determination information, it is detected that the ball has been caught based on the sensor signal of the sensor 5 that detects the sound, and the fielder's body or glove is in contact with the base. When it is detected that the base is stepped on based on the sensor signal of the sensor 5 that detects the pressure while being recognized from the captured image, the information is such that it is determined to be out. When such determination information is given, the analysis unit 13 uses AI technology to indicate that the fielder's body or glove is in contact with the base from the captured image, and the timing of contact. It has a function to recognize.

The determination unit 16 generates data in which the information extracted from the sensor signal by the analysis unit 13 and the information recognized from the captured image are arranged in chronological order, and the data is collated with the determination information as described above for verification. It has a function to judge the target competition.

The determination unit 16 further has a function of providing the determination result to a person who verifies the determination by using, for example, the display device 6 or the speaker 7.

In addition to the configuration of the first embodiment, the judgment verification system 1 of the second embodiment judges the competition to be verified by using the captured image and the sensor signal, and provides the judgment result to the person who verifies the judgment. It has a function to do. As a result, the determination verification system 1 of the second embodiment can obtain the effect that the time required for the determination verification can be shortened.

<Other Embodiments>
The present invention is not limited to the first and second embodiments, and various embodiments can be adopted. For example, the determination verification system 1 of the first and second embodiments relates to baseball, but the determination verification system of the present invention can also be applied to a system for verifying determination of a competition other than baseball.

Further, in the first and second embodiments, at least one of a pressure sensor and a vibration sensor and a microphone are provided as the sensor 5, but for example, when it is desired to reduce the number of sensors 5, the microphone is used. It may be omitted.

Further, FIG. 4 shows a block diagram illustrating another embodiment of the determination verification system according to the present invention. The determination verification system 40 shown in FIG. 4 includes an information processing device 41, a photographing device 42, and a sensor 43. The photographing device 42 photographs a verification area that can be a verification target for verifying the determination of the competition. The sensor 43 is arranged in the verification area and detects vibration, sound, or pressure. The information processing device 41 has a function of generating information to be used for verification of a determination of a competition by using an image captured by the photographing device 42 and a sensor signal output from the sensor 43.

The determination verification system 40 can obtain the effect that the time required for verification of determination can be shortened and the reliability of the verification result can be improved.

1,40 Judgment verification system 3,41 Information processing device 4,42 Imaging device 5,43 Sensor

Claims (7)

A shooting device that captures the verification area that can be the verification target for verifying the judgment of the competition,
A sensor that is placed in the verification area and detects vibration, sound, or pressure,
A determination verification system including an information processing device that generates information used for verification of the determination by using an image captured by the imaging device and a sensor signal output from the sensor. The photographing device and the sensor are connected to a common time supply source that provides time information, and the photographing device includes a configuration that outputs the photographed image in association with the time information acquired from the time supply source. The sensor has a configuration for outputting the sensor signal in association with the time information acquired from the time supply source.
The determination verification system according to claim 1, wherein the information processing device has a function of synchronizing the captured image with the sensor signal by using the time information. The information processing device extracts vibration, sound, or pressure caused by the competition from the sensor signal, and applies the extracted vibration, sound, or pressure to the captured image captured at the time detected by the sensor. The determination verification system according to claim 2, further comprising a function of superimposing information for notifying that vibration, sound, or pressure has been generated. The sensor includes at least two types of sensors: a sensor that detects vibration, a sensor that detects sound, and a sensor that detects pressure.
The determination verification system according to any one of claims 1 to 3, wherein the information processing device uses sensor signals of a plurality of types of the sensors. The competition is baseball
The determination verification system according to any one of claims 1 to 4, wherein the sensor is embedded in a base. Obtain an image taken by a shooting device that shoots a verification area that can be a verification target to verify the judgment of the competition.
Acquire the sensor signal output from the sensor that is placed in the verification area and detects vibration, sound, or pressure.
A judgment verification method for generating information used for verification of the judgment using the captured image and the sensor signal. The process of acquiring the image taken by the photographing device that photographs the verification area that can be the verification target for verifying the judgment of the competition, and
The process of acquiring the sensor signal output from the sensor that is placed in the verification area and detects vibration, sound, or pressure.
A computer program that causes a computer to execute a process of generating information used for verification of the determination using the captured image and the sensor signal.

Images