JP6596804B1 - Position tracking system and position tracking method - Google Patents

Abstract

An object of the present invention is to accurately track the position of a measurement object while suppressing a processing load. A position tracking system irradiates a target area with a wave and detects detection information including at least a distance and a direction to the object based on a reflected wave from the object, and an image including the target area An acquisition unit that acquires and synchronizes detection information detected by the detection device, image information representing an image captured by the imaging device, and detection information acquired by the acquisition unit An identification processing unit that identifies an object as a measurement object based on a partial image in image information corresponding to the position of the object, and provides identification information for identifying the measurement object to the identified measurement object; The position trajectory of the measurement object in the target area generated based on the detection information and the image information is associated with the identification information given by the identification processing unit to generate a movement trajectory of the measurement object And a trajectory generation unit. [Selection] Figure 1

Description

  The present invention relates to a position tracking system and a position tracking method.

  In recent years, in sports, a position tracking system that measures the position of a measurement target such as a person or an object and tracks the position of the measurement target is known (see, for example, Patent Document 1). In such a position tracking system, the position of the measurement object is measured based on images taken from a plurality of cameras.

JP 2013-58132 A

  However, in the conventional position tracking system described above, the position of the measurement object is detected from the image. For example, when the measurement object is overlapped, the position of the measurement object is detected by more cameras. It was necessary to image. Therefore, in the conventional position tracking system, the processing load of position measurement by image processing is high, and it is difficult to track the position of the measurement object with high accuracy while suppressing the processing load.

  The present invention has been made to solve the above problems, and an object of the present invention is to provide a position tracking system and a position tracking method capable of tracking the position of a measurement object with high accuracy while suppressing processing load. There is.

  In order to solve the above-described problem, an aspect of the present invention provides a detection device that irradiates a target area with a wave and detects detection information including at least a distance and a direction to the object based on a reflected wave from the object. An acquisition unit that acquires and synchronizes the imaging device that captures an image including the target area, the detection information detected by the detection device, and image information that represents the image captured by the imaging device, and the acquisition The object is identified as a measurement object based on the partial image in the image information corresponding to the position of the object detected based on the detection information acquired by the unit, and the identified measurement object An identification processing unit for providing identification information for identifying the measurement object, position information of the measurement object in the target area generated based on the detection information, and the image information; In association with the identification information whose serial identification processor has given a position tracking system, characterized in that it comprises a trajectory generator for generating a movement trajectory of the measurement object.

  According to an aspect of the present invention, in the position tracking system, the detection information includes a contour of the object, and the identification processing unit is configured to detect the image based on the contour of the object detected by the detection device. The partial image in the information is extracted, and the object is identified as a measurement target based on the extracted partial image.

  One aspect of the present invention is the position tracking system, wherein the measurement target includes a target person, and the identification processing unit identifies the target person when the target person is identified as the measurement target. The target person is identified based on the image recognition of the face or clothes.

  According to an aspect of the present invention, in the position tracking system, the identification processing unit can identify the target person based on the image recognition of the target person's face or clothes, based on the partial image, A target person's skeleton or joint is estimated, and the target person is identified based on the estimated skeleton or joint and a predetermined characteristic of the target person's skeleton or joint.

  One aspect of the present invention is the position tracking system, wherein the trajectory generation unit includes first position information that is position information of the object in the target area based on the detection information, and the target area based on the image information. The position information of the measurement object is corrected by mutually complementing the second position information, which is the position information of the measurement object, and the corrected position information of the measurement object is provided by the identification processing unit The movement trajectory of the measurement object is generated in association with the identification information.

  According to one embodiment of the present invention, a detection device that irradiates a target area with waves and detects detection information including at least a distance and a direction to the object based on a reflected wave from the object, and an image including the target area A position tracking method of a position tracking system including an image pickup device that picks up an image, wherein the acquisition unit acquires the detection information detected by the detection device and image information representing an image picked up by the image pickup device. The acquisition step to synchronize, and the identification processing unit measures the object based on the partial image in the image information corresponding to the position of the object detected based on the detection information acquired by the acquisition step. An identification processing step for identifying the object to be measured, and adding identification information for identifying the object to be measured to the identified object to be measured; A trajectory for generating a movement trajectory of the measurement object by associating the position information of the measurement target in the target area generated based on the image information with the identification information given by the identification processing step. A position tracking method including a generation step.

  According to the present invention, it is possible to track the position of the measurement object with high accuracy while suppressing the processing load.

It is a block diagram which shows an example of the position tracking system by 1st Embodiment. It is a figure which shows the example of installation of the laser radar apparatus and imaging device in 1st Embodiment. It is a figure which shows the example of data of the target object information storage part in 1st Embodiment. It is a figure which shows the example of data of the movement locus | trajectory memory | storage part in 1st Embodiment. It is a figure which shows the example of data of the target object locus | trajectory memory | storage part in 1st Embodiment. It is a figure explaining an example of the production | generation process of the movement locus | trajectory in 1st Embodiment. It is a figure explaining an example of the calculation process of the three-dimensional coordinate in 1st Embodiment. It is a figure explaining an example of the identification process of the object person in 1st Embodiment. It is a flowchart which shows an example of operation | movement of the position tracking system by 1st Embodiment. It is a block diagram which shows an example of the play determination system by 2nd Embodiment. It is a flowchart which shows an example of operation | movement of the play determination system by 2nd Embodiment. It is a figure which shows an example of the display screen in the modification of the play determination system by 2nd Embodiment. It is a figure which shows another modification of the play determination system by 2nd Embodiment.

  Hereinafter, a position tracking system (position measurement system) and a play determination system according to embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram showing an example of a position tracking system 1 according to the first embodiment.
As illustrated in FIG. 1, the position tracking system 1 includes an information processing device 10, a laser radar device 20, and an imaging device 30. The position tracking system 1 is an example of a position measurement system.

  The laser radar device 20 (an example of a detection device) irradiates a target area with laser light (for example, laser light such as near infrared rays) and includes at least a distance and a direction to the object based on reflected light from the object. It is a lidar device that detects detection information. Here, the target area is, for example, a court or a game arena where sports are performed. The laser radar device 20 detects an object in the target area based on the reflected light from the object, and in addition to the distance and direction to the object, for example, various information such as the contour of the object and the shape of the surface of the object. To detect. The laser radar device 20 uses, for example, a distance to an object by a time of flight (TOF) system that detects the distance from the moving time of the laser light from the irradiation of the laser light to the reception of the reflected light reflected by the object. Is detected. The laser radar device 20 outputs the detected detection information to the information processing device 10.

The imaging device 30 is a camera device, for example, and captures an image including a target area. The imaging device 30 outputs image information representing the captured image to the information processing device 10.
For convenience of explanation, FIG. 1 shows one laser radar device 20 and one imaging device 30, but the position tracking system 1 includes a plurality of laser radar devices 20 and a plurality of imaging devices 30. May be provided.

FIG. 2 is a diagram illustrating an installation example of the laser radar device 20 and the imaging device 30 in the present embodiment.
As shown in FIG. 2, a plurality of laser radar devices 20 and a plurality of imaging devices 30 are arranged so that a three-dimensional position coordinate such as an object (for example, player PL1) on a court AR1 that is a target area can be detected. Yes. In the present embodiment, for example, the three-dimensional position coordinates are represented as an X axis and a Y axis orthogonal to the plane of the coat AR1, and a height direction as the Z axis.

The information processing apparatus 10 is a computer apparatus such as a PC (personal computer), for example. The information processing apparatus 10 generates position information of the target object in the target area based on the detection information detected by the laser radar apparatus 20 and the image information captured by the imaging apparatus 30, and based on the generated position information, A movement trajectory of the object is generated. Here, the target object is, for example, a target person such as a player or a player, and a target object indicating tools such as a ball, a racket, a net, or a goal.
Further, the information processing apparatus 10 includes a storage unit 11 and a control unit 12.

  The storage unit 11 stores information used for various processes of the information processing apparatus 10. The storage unit 11 includes a radar detection storage unit 111, an image information storage unit 112, an object information storage unit 113, a movement locus storage unit 114, and an object locus storage unit 115.

  The radar detection storage unit 111 stores detection information detected by the laser radar device 20 and time information in association with each other. In the radar detection storage unit 111, detection information acquired from the laser radar device 20 by a data acquisition unit 121 described later is stored in synchronization with image information captured by the imaging device 30.

  The image information storage unit 112 stores image information captured by the imaging device 30 and time information in association with each other. In the image information storage unit 112, image information acquired from the imaging device 30 by a data acquisition unit 121 described later is stored in synchronization with detection information detected by the laser radar device 20.

The object information storage unit 113 stores information for identifying an object that is a person or an object. Note that the target object includes a target person. Here, an example of data stored in the object information storage unit 113 will be described with reference to FIG.
FIG. 3 is a diagram illustrating a data example of the object information storage unit 113 in the present embodiment.
As illustrated in FIG. 3, the object information storage unit 113 stores “object name” and “feature amount information” in association with each other, for example.

Here, the “object name” is the name of the object and is identification information for identifying the object. “Feature amount information” indicates, for example, a feature amount for identifying an object from image information, and includes, for example, a shape, a color, a size, and an image pattern for determining the object. Further, the “feature information” may be a feature for face recognition or a clothing feature such as a uniform when the target is a target person such as a specific player.
In the example illustrated in FIG. 3, the “object name” is stored in the object information storage unit 113 in association with “player A” and the feature amount information of the “player A”. The “object name” is “ball” and the feature amount information of the “ball” is stored in association with each other.

  Returning to the description of FIG. 1, the movement trajectory storage unit 114 stores the movement trajectory information of the detected object. The movement trajectory storage unit 114 stores, for example, the detected object identification information, time information, and three-dimensional position coordinates in association with each other. Here, an example of data stored in the movement locus storage unit 114 will be described with reference to FIG.

FIG. 4 is a diagram illustrating an example of data in the movement trajectory storage unit 114 in the present embodiment.
As illustrated in FIG. 4, the movement trajectory storage unit 114 stores “identification ID”, “time”, and “coordinates” in association with each other. Here, “identification ID” is identification information for identifying an object detected by the laser radar device 20, for example, and “time” indicates time information. The time information may include date information. “Coordinates” are three-dimensional position coordinates, for example, two-dimensional coordinates in a target area as shown in FIG.
In the example shown in FIG. 4, it is shown that the movement locus with “ID 1” as “ID” is stored in association with “time” and “coordinates”.

  Returning to the description of FIG. 1 again, the object trajectory storage unit 115 stores the identification information of the object identified by the identification processing unit 123 described later in association with the movement trajectory of the object. Here, with reference to FIG. 5, the data example which the target locus | trajectory memory | storage part 115 in this embodiment memorize | stores is demonstrated.

FIG. 5 is a diagram illustrating a data example of the object locus storage unit 115 in the present embodiment.
In FIG. 5, the object trajectory storage unit 115 stores “object name”, “time”, and “coordinates” in association with each other. Here, the information in which “time” and “coordinates” are associated with each other corresponds to the above-described movement locus.
In the example illustrated in FIG. 5, the movement trajectory corresponding to the “object name” “player A” is stored in association with “time” and “coordinates”. In the example shown in FIG. 5, the “object name” is “player A” and indicates a movement locus corresponding to the object person, but the object locus storage unit 115 stores, for example, a ball or bat. A movement trajectory corresponding to a tool (target object) may be stored.

Returning to the description of FIG. 1 again, the control unit 12 is a processor including, for example, a CPU (Central Processing Unit), and controls the information processing apparatus 10 in an integrated manner. The control unit 12, for example, the position information (first position information) of the target object in the target area based on the detection information detected by the laser radar device 20 and the target area in the target area based on the image information captured by the imaging device 30. The position information (second position information) is complemented with each other to generate a movement trajectory of the object.
The control unit 12 includes a data acquisition unit 121, an image processing unit 122, an identification processing unit 123, and a locus generation unit 124.

  The data acquisition unit 121 (an example of an acquisition unit) acquires and synchronizes detection information detected by the laser radar device 20 and image information representing an image captured by the imaging device 30. The data acquisition unit 121 stores the detection information acquired from the laser radar device 20 and the time information in the radar detection storage unit 111 in association with each other. In addition, the data acquisition unit 121 stores the image information acquired from the imaging device 30 and the time information in the image information storage unit 112 in association with each other. The data acquisition unit 121 synchronizes detection information and image information by adding time information to the radar detection storage unit 111 and the image information storage unit 112.

  The image processing unit 122 executes various image processes based on the image information acquired by the data acquisition unit 121. For example, the image processing unit 122 generates position information (second position information) of the object in the target area based on the image information. The position information in the target area here is, for example, coordinate information of the center of gravity of the target object. Further, the image processing unit 122 extracts, for example, a partial image corresponding to the object detected by the laser radar device 20 from the image information, and based on the feature amount stored in the object information storage unit 113 for the partial image. Then, an image recognition process for identifying the object is executed. The image processing unit 122 executes, for example, face recognition processing based on feature amounts, pattern recognition processing based on feature amounts, and the like.

  The identification processing unit 123 identifies the object as a target based on the partial image in the image information corresponding to the position of the object detected based on the detection information acquired by the data acquisition unit 121, and identifies the identified target. On the other hand, identification information for identifying the object is given. For example, the identification processing unit 123 extracts a partial image in the image information based on the contour of the object detected by the laser radar device 20, and identifies the object as a target based on the extracted partial image. Specifically, the identification processing unit 123 causes the image processing unit 122 to extract a partial image based on the contour of the object and execute image processing such as face recognition on the partial image. The identification processing unit 123 identifies an object as a target based on the result of the image processing performed by the image processing unit 122, and gives identification information for identifying the target to the identified target.

  The trajectory generation unit 124 (an example of a position measurement unit) is based on detection information (for example, distance and direction to the object) detected by the laser radar device 20, and position information (first position information) of the target in the target area. Is generated. The trajectory generation unit 124 generates a movement trajectory of the object to which the object has moved based on the generated first position information and the above-described second position information. That is, the trajectory generation unit 124 complements the first position information and the second position information, corrects the position information of the object, and moves the object based on the corrected position information of the object. Generate a trajectory. Here, with reference to FIG. 6, a process for generating a movement locus by mutually complementing the first position information and the second position information by the locus generation unit 124 will be described.

FIG. 6 is a diagram for explaining an example of a movement locus generation process in the present embodiment.
FIG. 6A shows position coordinates (first position information) based on detection information detected by the laser radar device 20. FIG. 6B shows position coordinates (second position information) based on image information captured by the imaging device 30. Here, the position coordinates (first position information) shown in FIG. 6A are indicated by black circles, and the position coordinates (second position information) shown in FIG. 6B are indicated by white circles. Times t1 to t21 indicate time information when each position coordinate is detected. In this example, in order to simplify the description, the position coordinates will be described using two-dimensional coordinates on the XY axes.

  Although the laser radar device 20 can measure accurate position information without applying a processing load, since the sampling frequency is low, the position coordinates (first position information) shown in FIG. become. Further, since the image information can be continuously captured, the position coordinates (second position information) shown in FIG. 6B are continuously detected. However, the position coordinates (second position information) have a large processing load when detecting the position coordinates from the image information, and the accuracy is inferior to the position coordinates (first position information).

FIG. 6C shows a state where the position coordinates (first position information) shown in FIG. 6A and the position coordinates (second position information) shown in FIG. . FIG. 6D shows a locus in which the position coordinates (first position information) shown in FIG. 6A and the position coordinates (second position information) shown in FIG. ing. That is, the trajectory generation unit 124 generates the position coordinates of the target object by mutually complementing the position coordinates (first position information) based on the detection information and the position coordinates (second position information) based on the image information. Finally, a movement locus as shown by a movement locus L0 in FIG. 6D is generated. The trajectory generation unit 124 stores the generated movement trajectory in the movement trajectory storage unit 114 in association with object identification information, time information, and three-dimensional position coordinates, for example, as shown in FIG. The identification information here is an identification ID given to the detected object according to a predetermined rule.
Note that the laser radar device 20 and the imaging device 30 are arranged so that the three-dimensional coordinates based on the detection information in the target area coincide with the three-dimensional coordinates based on the image information.

  Further, the trajectory generation unit 124 corrects the position information on the plane of the target area based on the first position information based on the detection information, and the position in the height direction of the target area based on the second position information based on the image information. The information may be corrected to generate three-dimensional coordinates (three-dimensional position information) including position information on the plane and position information in the height direction. Here, with reference to FIG. 7, an example of the calculation process of the three-dimensional coordinate by this embodiment is demonstrated.

FIG. 7 is a diagram for explaining an example of the calculation process of the three-dimensional coordinates in the present embodiment.
In FIG. 7, an image G <b> 1 indicates image information captured by the imaging device 30. Here, it is assumed that the court AR2 is a volleyball court and the player PL2 is about to hit the ball B1. In such a case, it is difficult to determine whether or not the player PL2 is jumping from the image G1.

  Therefore, the trajectory generation unit 124 generates coordinates of the position PT1 on the XY plane in the court AR2 of the player PL2 based on the detection information detected by the laser radar device 20 in order to generate accurate three-dimensional coordinates of the player PL2. x1, y1) is calculated. Then, the trajectory generation unit 124 calculates the position coordinate (z1) in the height direction (Z-axis direction) of the player PL2 from the position of the player PL2 on the image information (image G1) captured by the imaging device 30. The three-dimensional coordinates (x1, y1, z1) of PL2 are generated.

  The trajectory generation unit 124 calculates the position coordinate (z1) of the player PL2, and from the positional relationship between the net NT, the poles (P1, P2), the line LN of the court AR2, and the player PL2 on the screen, The position coordinate (z1) of the player PL2 is calculated. Further, the laser radar device 20 and the imaging device 30 are based on the positional relationship among the net NT, the poles (P1, P2), and the line LN in the court AR2, and the three-dimensional coordinates based on the detection information and the image information 3 They are arranged so that their dimensional coordinates coincide. Further, the trajectory generation unit 124 may correct the position information on the plane of the target area based on the first position information and the second position information in the above-described processing.

  In addition, the trajectory generation unit 124 generates a movement trajectory of the target object by associating the position information of the target object in the generated target area with the identification information given by the identification processing unit 123. That is, the trajectory generation unit 124 reads the movement trajectory of the object stored in the movement trajectory storage unit 114, associates the movement trajectory of the object with the identification information of the object given by the identification processing unit 123, and Is stored in the object trajectory storage unit 115. Specifically, for example, as illustrated in FIG. 5, the trajectory generation unit 124 includes identification information of an object (“object name”), time information (“time”), and three-dimensional position coordinates (“coordinates”). Are associated with each other and stored in the movement trajectory storage unit 114.

In addition, the identification process part 123 identifies a target person based on the image recognition of a target person's face or clothes, when identifying a target person as a target object. That is, the identification processing unit 123 requests the image processing unit 122 to perform identification, and the image processing unit 122 identifies the target person based on the feature amount of the target person's face or clothing stored in the target object information storage unit 113. To do. The identification processing unit 123 identifies the target person based on the result of causing the image processing unit 122 to recognize the image.
Here, with reference to FIG. 8, the identification processing of the target person by the identification processing unit 123 will be described.

FIG. 8 is a diagram illustrating an example of target person identification processing in the present embodiment.
In FIG. 8, a process of recognizing the target person when the player PL3 (player B) and the player PL4 (player A) run across the court AR3 will be described.

  As shown in FIG. 8A, when the player PL3 (player B) and the player PL4 (player A) cross the court AR3, the trajectory generator 124 identifies the object detected by the laser radar device 20. While giving ID (for example, ID1-ID4), as shown in FIG.8 (b), the movement locus | trajectory of each object is produced | generated. In FIG. 8B, the horizontal axis indicates time, and the vertical axis indicates the identification ID of each object.

  Also, a circle (circle) and a cross (cross) in the graph indicate times when the identification processing unit 123 requests the image processing unit 122 to identify an object. In addition, a circle (circle) indicates that the object (player PL3 or player PL4) to which the identification ID is assigned can be identified by the image processing unit 122, and an x (cross) indicates that the identification ID is assigned. This indicates that object identification has failed. When the identification processing unit 123 fails to identify the object to which the identification ID is assigned, the identification processing unit 123 requests the image processing unit 122 to identify the object again after a certain interval, and when the object can be recognized, The request for identifying the object to which the identification ID is assigned is not performed. Thereby, the identification processing unit 123 reduces the processing load for identifying the object. The trajectory generation unit 124 stores the generated movement trajectory in the movement trajectory storage unit 114 as shown in FIG. 4 described above.

  In addition, the identification processing unit 123, for each movement trajectory, based on the partial image at the position where the object (player PL3 or player PL4) is detected and the feature amount stored in the object information storage unit 113, Player PL3 or player PL4) is identified, and an object name is assigned to the object (player PL3 or player PL4). For example, as shown in FIG. 8C, the identification processing unit 123 gives “player B” to the player PL3 and gives “player A” to the player PL4. As illustrated in FIG. 8D, the trajectory generation unit 124 associates the object name provided by the identification processing unit 123 with the movement trajectory stored in the movement trajectory storage unit 114.

  8C and 8D, the movement locus of “player A” is the movement locus L1, and the movement locus of “player B” is the movement locus L2. The trajectory generation unit 124 stores the movement trajectory to which the object names such as “player A” and “player B” are assigned in the object trajectory storage unit 115 as shown in FIG. 5, for example.

Next, the operation of the position tracking system 1 according to the present embodiment will be described with reference to the drawings.
FIG. 9 is a flowchart showing an example of the operation of the position tracking system 1 according to the present embodiment. In this figure, it is assumed that the laser radar device 20 steadily detects detection information, and the imaging device 30 steadily captures image information.

  In FIG. 9, the information processing apparatus 10 of the position tracking system 1 first acquires and synchronizes the detection information of the laser radar apparatus 20 and the image information (step S101). The data acquisition unit 121 of the information processing device 10 acquires detection information from the laser radar device 20 and acquires image information from the imaging device 30. The data acquisition unit 121 stores the detection information acquired from the laser radar device 20 and the time information in the radar detection storage unit 111 in association with each other. In addition, the data acquisition unit 121 stores the image information acquired from the imaging device 30 and the time information in the image information storage unit 112 in association with each other.

  Next, the information processing device 10 generates position information in the target area based on the detection information of the laser data device and the image information (step S102). The trajectory generation unit 124 of the information processing apparatus 10 is based on the detection information (for example, the distance and direction to the object) detected by the laser radar device 20, and the position information (first object) in the target area of the detected object (target object). Position information). Further, the image processing unit 122 generates position information (second position information) of an object (target object) in the target area based on the image information, for example. Here, the first position information and the second position information are, for example, three-dimensional coordinates of the center of gravity of the object. The trajectory generation unit 124 complements the first position information and the second position information and corrects and generates the three-dimensional coordinates of the object.

  Next, the information processing apparatus 10 cuts out a partial image corresponding to the position information, recognizes the partial image, and gives identification information (step S103). The identification processing unit 123 of the information processing apparatus 10 causes the image processing unit 122 to extract a partial image in the image information corresponding to the position of the object in the first position information described above. The identification processing unit 123 causes the image processing unit 122 to identify the target based on the extracted partial image and the feature amount stored in the target information storage unit 113, and for the identified target, Identification information for identifying an object is given.

  Next, the information processing apparatus 10 generates a movement locus of the object to which the identification information is given based on the position information (Step S104). The trajectory generation unit 124 generates a movement trajectory of the target object by associating the generated three-dimensional coordinates of the target object with the identification information provided by the identification processing unit 123 and the time information. For example, as illustrated in FIG. 5, the trajectory generation unit 124 associates the three-dimensional coordinates of the target object, the identification information of the target object, and the time information, and uses the movement trajectory storage unit 114 as a movement trajectory of the target object. Remember me. After the process of step S104, the trajectory generation unit 124 ends the process.

  As described above, the position tracking system 1 (position measurement system) according to the present embodiment includes the laser radar device 20 (detection device), the imaging device 30, the data acquisition unit 121 (acquisition unit), and the trajectory generation unit 124. (Position measuring unit). The laser radar device 20 irradiates the target area with laser light, and detects detection information including at least the distance and direction to the object based on the reflected light from the object. The imaging device 30 captures an image including the target area. The data acquisition unit 121 acquires and synchronizes detection information detected by the laser radar device 20 and image information representing an image captured by the imaging device 30. The trajectory generation unit 124 includes first position information that is position information of an object in the target area based on the detection information acquired by the data acquisition unit 121, and position information of the object in the target area based on the image information acquired by the data acquisition unit 121. The position information of the object is generated by complementing the second position information. The trajectory generation unit 124 generates a movement trajectory of the target object to which the target object has moved based on the generated position information of the target object.

  As a result, the position tracking system 1 according to the present embodiment can measure accurate position information without applying a processing load. However, since the sampling frequency is low, the laser radar device 20 that performs intermittent detection and the processing load are low. In addition to being large, the accuracy of the position information is inferior, but the position of the object can be measured with high accuracy while suppressing the processing load by using together with the imaging device 30 capable of continuously capturing images. The position tracking system 1 according to the present embodiment can realize, for example, accurate measurement of position coordinates in three dimensions, which is difficult with only the image information of the imaging device 30, by using the laser radar device 20 together. Therefore, the position tracking system 1 according to the present embodiment can track the position of the object with high accuracy while suppressing the processing load.

Further, in the present embodiment, the trajectory generation unit 124 complements the first position information and the second position information to correct the position information of the object, and based on the corrected position information of the object. The movement trajectory of the object is generated.
Thereby, the position tracking system 1 according to the present embodiment complements the first position information based on the detection information of the laser radar device 20 and the second position information based on the image information of the imaging device 30, for example. Thus, robust and highly accurate position measurement of an object can be performed without being affected by a complicated background or external environment. For example, as shown in FIG. 8, the position tracking system 1 according to the present embodiment can accurately generate a movement trajectory even when objects overlap.

  In addition, the position tracking system 1 according to the present embodiment does not need to increase the number of imaging devices 30 with a large processing load for image processing in order to detect position information with high accuracy by complementing each other. For example, real-time processing can be performed with a reduced processing load. Further, in the position tracking system 1 according to the present embodiment, for example, it is not necessary to attach an accessory such as a marker to an object, and the system configuration can be made a system (simplified).

In the present embodiment, the trajectory generation unit 124 corrects position information on the plane of the target area based on the first position information, and corrects position information in the height direction of the target area based on the second position information. Then, three-dimensional position information including position information on the plane and position information in the height direction is generated, and a trajectory of the object is generated based on the generated three-dimensional position information.
As a result, the position tracking system 1 according to the present embodiment can measure an accurate position in the height direction (Z-axis direction) that is difficult to measure from image information, as shown in FIG. The radar device 20 can measure an accurate position in the depth direction (XY axis direction). Therefore, the position tracking system 1 according to the present embodiment can generate the trajectory of the object with higher accuracy.

In the present embodiment, the laser radar device 20 and the imaging device 30 are arranged so that the first position information and the second position information in the target area match.
Thereby, the position tracking system 1 according to the present embodiment can measure the position of the object with higher accuracy.

The position tracking method (position measurement method) according to the present embodiment is a position tracking method of a position tracking system 1 (position measurement system) including a laser radar device 20 (detection device) and an imaging device 30, and is acquired. A step and a locus generation step (position measurement step). In the acquisition step, the data acquisition unit 121 acquires and synchronizes detection information detected by the laser radar device 20 and image information representing an image captured by the imaging device 30. In the trajectory generation step (position measurement step), the trajectory generation unit 124 (position measurement unit) acquires the first position information that is the position information of the object in the target area based on the detection information acquired in the acquisition step, and the acquisition step. The position information of the object is generated by complementing the second position information, which is the position information of the object in the target area based on the image information, and the object is moved based on the position information of the generated object. A movement trajectory of the object is generated.
Thereby, the position tracking method according to the present embodiment has the same effect as the position tracking system 1 described above, and can measure the position of the object with high accuracy while suppressing the processing load.

  Further, the position tracking system 1 according to the present embodiment includes a laser radar device 20, an imaging device 30, a data acquisition unit 121, an identification processing unit 123, and a locus generation unit 124. The data acquisition unit 121 acquires and synchronizes detection information detected by the laser radar device 20 and image information representing an image captured by the imaging device 30. The identification processing unit 123 uses the image processing unit 122 to detect the object based on the partial image in the image information corresponding to the position of the object detected based on the detection information acquired by the data acquisition unit 121. And identifying information for identifying the object is given to the identified object. The trajectory generation unit 124 generates a movement trajectory of the target object by associating the position information of the target object in the target area generated based on the detection information and the image information with the identification information given by the identification processing unit 123. To do.

  Thereby, the position tracking system 1 according to the present embodiment can identify the object with high accuracy by using the detection information detected by the laser radar device 20 and the image information representing the image captured by the imaging device 30. it can. Since the position tracking system 1 according to the present embodiment performs, for example, the identification process on the partial image corresponding to the detected position of the object, compared to the case where the identification process is performed on the entire image information, And the like, and the processing load can be greatly reduced. Therefore, the position tracking system 1 according to the present embodiment can track the position of the measurement object with high accuracy while suppressing the processing load.

In the present embodiment, the detection information of the laser radar device 20 includes the contour of the object. The identification processing unit 123 uses the image processing unit 122 to extract a partial image in the image information based on the contour of the object detected by the laser radar device 20, and based on the extracted partial image, the object is detected as a target. Identify as.
Thereby, the position tracking system 1 according to the present embodiment can further remove noise such as background by cutting out a partial image from the image information along the contour of the object, and recognizes the object by image processing. The accuracy can be further improved.

In the present embodiment, the target object includes a target person. When identifying the target person as an object, the identification processing unit 123 uses the image processing unit 122 to identify the target person based on the image recognition of the target person's face or clothes.
Thereby, the position tracking system 1 according to the present embodiment can appropriately identify a target person such as a player or a player.

In the present embodiment, the trajectory generation unit 124 includes first position information that is position information of an object in the target area based on detection information, and second position information that is position information of the target object in the target area based on image information. Are mutually corrected, the position information of the target is corrected, and the corrected position information of the target is associated with the identification information given by the identification processing unit 123 to generate a movement trajectory of the target.
Thereby, the position tracking system 1 according to the present embodiment can track the position of the measurement object with higher accuracy.

In addition, the position tracking method according to the present embodiment irradiates a target area with laser light, and the position tracking method of the position tracking system 1 including the laser radar device 20 and the imaging device 30 based on the reflected light from the object. The method includes an acquisition step, an identification processing step, and a trajectory generation step. In the acquisition step, the data acquisition unit 121 acquires and synchronizes detection information detected by the laser radar device 20 and image information representing an image captured by the imaging device 30. In the identification processing step, the identification processing unit 123 uses the image processing unit 122, based on the partial image in the image information corresponding to the position of the object detected based on the detection information acquired in the acquisition step. An object is identified as an object, and identification information for identifying the object is given to the identified object. In the trajectory generation step, the trajectory generation unit 124 associates the position information of the target object in the target area generated based on the detection information and the image information with the identification information given in the identification processing step, and The movement trajectory of is generated.
Thereby, the position tracking method according to the present embodiment has the same effect as the position tracking system 1, and can track the position of the measurement object with high accuracy while suppressing the processing load.

[Second Embodiment]
Next, a play determination system 1a according to the second embodiment will be described with reference to the drawings.
The present embodiment includes the position tracking system 1 described above, and a play determination system 1a that determines the play of the target person indicating the action and action of the target person based on the movement trajectory of the target person and the skeleton and joints of the target person. Will be described.

FIG. 10 is a block diagram showing an example of the play determination system 1a according to the present embodiment.
As illustrated in FIG. 10, the play determination system 1a (an example of a determination system) includes an information processing device 10a, a laser radar device 20, and an imaging device 30.
In this figure, the same components as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted. In the present embodiment, the laser radar device 20 and the imaging device 30 are an example of a detection device that can detect at least the shape and position information of an object.

The information processing apparatus 10a is a computer apparatus such as a PC, for example. Similar to the information processing apparatus 10 of the first embodiment described above, the information processing apparatus 10a is configured to detect a target in a target area based on detection information detected by the laser radar apparatus 20 and image information captured by the imaging apparatus 30. Object position information is generated, and a movement trajectory of the object is generated based on the generated position information. Further, the information processing apparatus 10a estimates the skeleton and joints of the target person from the image information, and determines the play of the target person from the skeleton and joints of the target person and the movement trajectory of the target object. Here, “play” indicates, for example, the movement and action of the target person in sports.
The information processing apparatus 10a includes a storage unit 11a and a control unit 12a.

  The storage unit 11a stores information used for various processes of the information processing apparatus 10a. The storage unit 11a includes a radar detection storage unit 111, an image information storage unit 112, an object information storage unit 113, a movement locus storage unit 114, an object locus storage unit 115, a skeletal joint information storage unit 116, The determination device information storage unit 117 and the determination result storage unit 118 are provided.

  The skeletal joint information storage unit 116 stores the skeletal joint information estimated by the skeleton estimation unit 125 described later. The skeletal joint information storage unit 116 stores, for example, target person identification information, time information, and skeletal joint information indicating the skeleton and joints in association with each other. Here, the skeletal joint information is, for example, three-dimensional position coordinates of the skeleton and joints.

The determiner information storage unit 117 stores information on determiners for determining play. The discriminator information storage unit 117 is machine-learned learning so as to determine the play of the target person indicating the action and action of the target person based on, for example, the movement trajectory of the target person in the past and the skeleton and joints of the target person. The result and the like are stored as information of the determiner. Here, the machine learning includes, for example, decision tree learning, neural network, support vector machine (SVM), various regression models, deep learning (deep learning), and the like. The information on the determiner includes various weighting information, rule information, various parameters, and the like.
In addition to the play type, the play determination includes success / failure of the play, performance, score determination, and the like.

  The determination result storage unit 118 stores a determination result determined by a play determination unit 126 described later. The determination result storage unit 118 stores, for example, target person identification information, time information, and a play determination result in association with each other.

The control unit 12a is, for example, a processor including a CPU and the like, and comprehensively controls the information processing apparatus 10a. The control unit 12a performs the same process as the control unit 12 of the first embodiment described above, and determines, for example, the play of the target person.
The control unit 12a includes a data acquisition unit 121, an image processing unit 122, an identification processing unit 123, a trajectory generation unit 124, a skeleton estimation unit 125, a play determination unit 126, and a play evaluation unit 127.

  The skeleton estimation unit 125 estimates the skeleton and joints of the target person based on detection information detected by the detection devices (for example, the laser radar device 20 and the imaging device 30). The skeleton estimation unit 125 estimates the skeleton and joints of the target person based on the contour of the target person detected by the laser radar device 20 and image information including the target person imaged by the imaging device 30. The skeleton estimation unit 125 causes the skeleton joint information storage unit 116 to store the skeleton joint information indicating the estimated skeleton and joint, the identification information of the target person, and the time information in association with each other. Note that the skeleton estimation unit 125 estimates the skeleton and joints of the target person using, for example, an existing skeleton tracking technique. Note that the skeleton estimation unit 125 may estimate the eye of the target person in addition to the skeleton and the joint. In this case, the skeletal joint information may include information indicating the eye line. Further, the skeleton estimation unit 125 may estimate one of the skeleton and joints of the target person.

  Since the image processing unit 122 and the identification processing unit 123 are the same as those in the first embodiment, descriptions thereof are omitted here. Note that the identification processing unit 123, when the target person cannot be identified based on the image recognition of the target person's face or clothes, based on the partial image, the target person's skeleton or joint (the target person's skeleton and joint, or One of the skeleton and joints of the target person may be estimated, and the target person may be identified based on the estimated skeleton or joint and the predetermined characteristics of the skeleton or joint of the target person. That is, the target object information storage unit 113 stores, for example, the feature amount of the target person's skeleton or joint in advance, and the identification processing unit 123 includes the skeleton or joint estimated by the skeleton estimation unit 125 and the target object. The target person may be identified based on the skeleton or joint feature quantity of the target person stored in the information storage unit 113.

Since the locus generation unit 124 is the same as that of the first embodiment, the description thereof is omitted here. That is, the trajectory generation unit 124 generates a movement trajectory of the target person based on detection information detected by the detection devices (for example, the laser radar device 20 and the imaging device 30).
In addition to the movement trajectory of the target person, the trajectory generation unit 124 according to the present embodiment includes a movement trajectory of an object around the target person, a movement trajectory of the skeleton and joints, and a part of the body such as the hand and foot of the target person. Or the like may be generated.

  The play determination unit 126 is based on the skeleton and joints of the target person estimated by the skeleton estimation unit 125, the movement trajectory of the target person generated by the trajectory generation unit 124, and the determiner that determines the play of the target person. Determine the play of a person. Here, the determination device is generated so as to determine the play of the target person indicating the action and behavior of the target person based on the movement trajectory of the target person in the past and the skeleton and joints of the target person. It is generated by learning. The play determination unit 126 determines the play of the target person from the skeleton and joints of the target person and the movement trajectory of the target person using the determination unit based on the determination unit information stored in the determination unit information storage unit 117.

  Here, the play of the target person is, for example, a player's receive, attack, serve, block or the like in volleyball or a screen play in basketball. In addition, for example, when determining play in volleyball, the play determination unit 126 is present in the movement trajectory of a tool such as a ball in addition to the skeleton and joints of the target person and the movement trajectory of the target person, that is, around the target person. The play of the target person may be determined based on the movement trajectory of the target object. In this case, the trajectory generation unit 124 generates a movement trajectory of an object (for example, a ball) existing around the target person, and the play determination unit 126 uses the determiner to move the target person's movement trajectory and skeleton. The play of the target person is determined from the joint and the movement trajectory (for example, a ball) of the target object.

In addition, for example, when determining the screen play in basketball, the play determination unit 126 may determine the play of the target person based on the movement trajectory of the persons around the target person. In this case, the trajectory generation unit 124 generates a movement trajectory of an object (for example, a peripheral person) existing around the target person, and the play determination unit 126 uses the determination unit to move the target person. The play of the target person is determined from the skeleton and joints and the movement trajectory of the target object (for example, a nearby person).
The play determination unit 126 stores information indicating the determined play as a determination result in the determination result storage unit 118 in association with the target person and time information.

  The play evaluation unit 127 evaluates the play based on the play determination result determined by the play determination unit 126. The play evaluation unit 127, for example, counts the number of executions of a play such as a jump during training from the play determination result, and evaluates the training effect, efficiency, overtraining, and the like. In addition, the play evaluation unit 127, for example, totals the plays executed in the game based on the play determination result, and evaluates the game content based on the play total result.

Next, the operation of the play determination system 1a according to the present embodiment will be described with reference to the drawings.
FIG. 11 is a flowchart showing an example of the operation of the play determination system 1a according to the present embodiment. In this figure, it is assumed that the laser radar device 20 steadily detects detection information, and the imaging device 30 steadily captures image information.
Since the processing from step S201 to step S203 shown in FIG. 11 is the same as the processing from step S101 to step S103 shown in FIG. 9, the description thereof is omitted here.

  In step S204, the information processing apparatus 10a estimates a skeleton and a joint based on the detection information and the image information. For example, the skeleton estimation unit 125 of the information processing device 10a calculates the skeleton and joints of the target person based on the contour of the target person detected by the laser radar device 20 and the image information including the target person captured by the imaging device 30. presume. The skeleton estimation unit 125 causes the skeleton joint information storage unit 116 to store the skeleton joint information indicating the estimated skeleton and joint, the identification information of the target person, and the time information in association with each other.

  Next, the information processing apparatus 10a generates a movement trajectory of the measurement object to which the identification information is added based on the position information (Step S205). The trajectory generation unit 124 of the information processing device 10a associates the generated three-dimensional coordinates of the target object, the identification information provided by the identification processing unit 123, and time information, as in step S104 illustrated in FIG. A movement trajectory of the object is generated.

  Next, the information processing apparatus 10a performs play determination based on the feature amount of the movement locus, the skeleton, and the joint (step S206). The play determination unit 126 of the information processing device 10a uses the determination device based on the determination device information stored in the determination device information storage unit 117, and uses the skeleton and joints of the target person and the feature amount of the movement trajectory of the target person. Determine the play of the target person. The play determination unit 126 stores the determination result of play in the determination result storage unit 118.

  Next, the information processing apparatus 10a determines whether or not the play determination is complete (step S207). If the play determination is finished (step S207: YES), the information processing apparatus 10a advances the process to step S208. If the play determination is not complete (step S207: NO), the information processing apparatus 10a returns the process to step S201 and repeats the processes from step S201 to step S207.

In step S208, the information processing apparatus 10a evaluates the play determination result. The play evaluation unit 127 of the information processing apparatus 10a evaluates the play based on the play determination result determined by the play determination unit 126. The play determination unit 126 ends the process after the process of step S208.
In the example described above, the skeleton estimation unit 125 estimates the skeleton and joints of the target person, and the play determination unit 126 determines play from the movement trajectory of the target person and the positions of the skeleton and joints of the target person. However, the skeleton estimation unit 125 estimates one of the skeleton and joints of the target person, and determines play based on the movement trajectory of the target person and the position of the skeleton and joints of the target person. You may make it do.

  As described above, the play determination system 1a (determination system) according to the present embodiment includes the detection device (for example, the laser radar device 20 and the imaging device 30), the skeleton estimation unit 125, the trajectory generation unit 124, and the play determination. Part 126. The detection device (for example, the laser radar device 20 and the imaging device 30) can detect at least the shape and position information of the object. The skeleton estimation unit 125 estimates the skeleton or joint of the target person based on the detection information detected by the detection device. The trajectory generation unit 124 generates a movement trajectory of the target person based on the detection information detected by the detection device. The play determination unit 126 determines the play of the target person based on the determiner, the skeleton or joint of the target person estimated by the skeleton estimation unit 125, and the movement trajectory of the target person generated by the trajectory generation unit 124. Here, the determiner is generated so as to determine the play of the target person indicating the action and action of the target person based on the past movement trajectory of the target person and the skeleton or joint of the target person.

  As a result, the play determination system 1a according to the present embodiment determines play based on the skeleton or joint of the target person and the movement trajectory of the target person. Judgment can be made. Further, in the play determination system 1a according to the present embodiment, for example, the player's play, position, time, success / failure of play, performance, score determination, etc. can be automatically recorded without any burden on the user. That is, the play determination system 1a according to the present embodiment can automatically record the score of a sports competition.

Further, in the present embodiment, the trajectory generation unit 124 generates a movement trajectory of an object existing around the target person based on detection information detected by the detection device (for example, the laser radar device 20 and the imaging device 30). To do. The play determination unit 126 determines the play of the target person from the movement trajectory of the target person, the skeleton or joint of the target person, and the movement trajectory of the target object using a determination device.
Thereby, the play determination system 1a according to the present embodiment can improve the accuracy of play determination by using the movement trajectory of an object which is a tool such as a ball or a racket.

For example, in volleyball, the play determination system 1a according to the present embodiment determines, for example, a receive, an attack, a serve, a block, and the like by making a play determination using a movement trajectory of an object such as a ball as a determiner. be able to.
In addition, for example, in the basket, the play determination system 1a according to the present embodiment accurately determines, for example, screen play by making a play determination using the movement trajectory of the person around the target person as a determination device. be able to.
Further, for example, in tennis, by using the skeleton of the target person and the movement trajectory of the target person, the ball, and the racket as a determiner, and making a play determination, a tennis shot type (eg, backhand, forehand, smash) , Slices, etc.), balls (for example, speed, spin type, rotation speed, etc.), success or failure of shots, and the like can be determined.

  In the present embodiment, the detection device includes the laser radar device 20 and the imaging device 30. The skeleton estimation unit 125 estimates the skeleton or joint of the target person based on the contour of the target person detected by the laser radar device 20 and the image information including the target person imaged by the imaging device 30. The trajectory generation unit 124 includes first position information that is position information of the target person in the target area based on detection by the laser radar device 20, and second position information that is position information of the target person in the target area based on image information. Complementing each other, the position information of the target person is corrected, and the movement trajectory of the target person is generated based on the corrected position information of the target person.

  Accordingly, the play determination system 1a according to the present embodiment performs processing by mutually complementing the first position information based on the detection information of the laser radar device 20 and the second position information based on the image information of the imaging device 30. The position of the object can be measured while suppressing the load. Therefore, the play determination system 1a according to the present embodiment can perform the target person's play determination with higher accuracy.

Further, the play determination system 1a according to the present embodiment identifies a target person based on the partial image in the image information corresponding to the first position information based on the detection information of the laser radar device 20, and for the identified target person And an identification processing unit 123 for providing identification information for identifying the target person. The trajectory generation unit 124 generates a movement trajectory of the target person in association with the identification information given by the identification processing unit 123.
Thereby, the play determination system 1a according to the present embodiment can identify an object with high accuracy by using detection information detected by the laser radar device 20 and image information representing an image captured by the imaging device 30. It is possible to track the position of the measurement object with high accuracy while suppressing the processing load. Therefore, the play determination system 1a according to the present embodiment can perform the target person's play determination with higher accuracy.

  In the play determination system 1a according to the present embodiment, the imaging apparatus 30 can detect position information with high accuracy by using position information that is difficult to acquire with high accuracy in combination with the laser radar device 20. . Therefore, the play determination system 1a according to the present embodiment can expand the types of play that can be determined, and can be applied to analysis of various competitions and games. In addition, the play determination system 1a according to the present embodiment improves the determination speed and the determination accuracy by using the imaging apparatus 30 together with the laser radar apparatus 20 in the determination of the play that requires a complicated and high processing load. be able to.

In the present embodiment, the identification processing unit 123 estimates the skeleton or joint of the target person based on the partial image when the target person cannot be identified based on the image recognition of the target person's face or clothes. The target person is identified based on the skeleton or joint and the characteristics of the target person's skeleton or joint.
Thereby, the play determination system 1a according to the present embodiment can identify the object with higher accuracy.

In the present embodiment, the locus generation unit 124 generates position information on the plane of the target area based on the detection information of the laser radar device 20, and the height direction of the target area based on the image information of the imaging device 30. Position information is generated, three-dimensional position information including position information on the plane and position information in the height direction is generated, and a trajectory of the target (target person) is generated based on the generated three-dimensional position information. To do.
Thereby, since the play determination system 1a according to the present embodiment can measure an accurate position, for example, a play determination with a jump can be performed with high accuracy, and the types of play that can be determined can be further expanded.

The play determination system 1a according to this embodiment includes a play evaluation unit 127 that evaluates play.
Thereby, the play determination system 1a according to the present embodiment can evaluate, for example, the mileage during practice, the number of jumps and the height, and the play operation of each player, and perform management such as overtraining in an integrated manner. Can do.

The determination method according to the present embodiment includes a skeleton estimation step, a trajectory generation step, and a play determination step. In the skeleton estimation step, the skeleton estimation unit 125 estimates the skeleton or joint of the target person based on detection information detected by at least a detection device that can detect the shape and position information of the object. In the trajectory generation step, the trajectory generation unit 124 generates a movement trajectory of the target person based on the detection information detected by the detection device. In the play determination step, the play determination unit 126 is generated so as to determine the play of the target person indicating the action and action of the target person based on the past movement trajectory of the target person and the skeleton or joint of the target person. The play of the target person is determined based on the determination unit, the skeleton or joint of the target person estimated by the skeleton estimation unit 125, and the movement trajectory of the target person generated by the trajectory generation unit 124.
Thereby, the determination method according to the present embodiment has the same effect as the above-described play determination system 1a, and can perform the play determination of the target person with high accuracy while suppressing the processing load.

The present invention is not limited to the above embodiments, and can be modified without departing from the spirit of the present invention.
For example, the number of the laser radar devices 20 and the imaging devices 30 included in the position tracking system 1 (play determination system 1a) is not limited to the above embodiments, and may include other numbers.

  In each of the above-described embodiments, the information processing apparatus 10 (10a) has been described as an example of a single apparatus. However, the present invention is not limited to this, and may be configured by a plurality of apparatuses. Further, the information processing apparatus 10 (10a) may be configured such that an external device includes part or all of the storage unit 11 (11a). In this case, the external device may be a storage device that can be connected to the information processing apparatus 10 (10a) via a network. Further, the information processing apparatus 10 (10a) may be configured such that an external device has a part of the functions of the control unit 12 (12a).

  In each of the above embodiments, a laser radar device is an example of a detection device that irradiates a target area with waves and detects detection information including at least the distance and direction to the object based on the reflected wave from the object. Although the example using 20 was demonstrated, it is not limited to this. For example, the detection device may irradiate light waves, radio waves, or sound waves, and detect detection information including at least the distance and direction to the object based on the reflected wave from the object. Here, the wave is a wave propagating in a space such as a light wave such as a laser beam, a radio wave, a sound wave such as an ultrasonic wave, and the like.

In the second embodiment described above, the play determination unit 126 describes an example in which play is determined from the movement trajectory of the target person, the positions of the skeleton and joints, and the movement trajectory of the target existing around the target person. However, the present invention is not limited to this. The play determination unit 126 is based on, for example, the movement trajectory of the skeleton and joints, the positional relationship with surrounding objects such as poles, nets, and court lines, the movement trajectory of a part of the body such as the hand and foot of the target person, and the like. The play may be determined. In addition, the play determination unit 126 may determine play from either the movement trajectory of the target person or the position of the skeleton or joint.
The play determined by the play determination unit 126 may be, for example, a soccer goal, a line-in or line-out such as tennis or volleyball, a name of a technique such as judo. In addition, the play determination unit 126 may determine success / failure of play (for example, success / failure of basketball shot), performance / score (for example, scoring of gymnastics), and the like as play determination.

  Moreover, in each said embodiment, although the target area demonstrated the example which is a court, a game field, etc. which are sports fields which perform a sport, for example, it is not limited to this. The target area to which the information processing apparatus 10 (10a) is applied may be, for example, an ice rink such as skates, a competition place on snow such as a ski jump or a half pipe, a swimming pool such as swimming or artistic swimming, or the like. It may be a place for other competitions.

Further, in each of the above embodiments, the calibration process in which the laser radar device 20 and the imaging device 30 are arranged so that the first position information and the second position information in the target area coincide with each other is as follows. You may go to
For example, in a competition using a court, the laser radar device 20 and the imaging device 30 may be calibrated simultaneously by using a colored sticker or the like created with a retroreflective material on the court line. In this case, the seal or the like may be used by being attached to a tool whose position is fixed during the competition, such as a net pole or a goal pole.

Further, the position measurement process of the information processing apparatus 10 (10a) is not limited to the above-described embodiments, and may be changed as follows depending on the competition item and purpose.
For example, the trajectory generation unit 124 of the information processing apparatus 10 (10a) performs position measurement in the normal state by supplementing mainly the first position information from the laser radar apparatus 20 with the second position information based on the image information. And when using it for a wheelchair basketball or a wheelchair rugby game, it is thought that it is difficult for the player to carry the ball and to detect the first position information by the laser radar device 20. In such a case, the locus generation unit 124 may supplement the second position information based on the image information mainly with the first position information from the laser radar device 20.

Further, as described above, the laser radar device 20 can accurately detect the position in real time, but the position information becomes fragmentary because the sampling frequency is low. In contrast, in position detection based on image information, continuous position information can be detected, but the processing load is high and time is required. Therefore, when emphasizing position detection in real time, the trajectory generation unit 124 mainly reduces the ratio, frequency, etc. of complementing the first position information by the laser radar device 20 with the second position information based on the image information. It may be. In addition, for example, when importance is placed on position detection accuracy, the trajectory generation unit 124 may increase the ratio, frequency, and the like that are supplemented with the second position information based on the image information.
As described above, the trajectory generation unit 124 may change a complementing method between the first position information and the second position information according to the competition item and the purpose.

  In each of the above embodiments, the information processing apparatus 10 (10a) may convert the generated movement trajectory into various lines of sight and orientations and display them on a display unit (not shown) or the like. For example, the trajectory generation unit 124 may convert the movement trajectory to the baseball catcher or soccer keeper's line of sight, and display the movement trajectory. The movement trajectory may be converted into the trajectory and displayed.

  Further, in each of the above-described embodiments, the information processing apparatus 10 (10a) can cope with a simple facility when it is desired to acquire the trajectory of a moving object in indoor competition such as bicycle track competition and speed skating. . In addition, when measuring speeds at medium and long distances on land, it is difficult to discriminate by video alone, such as when players overlap in the vicinity of the corner. With the conventional technology, calibration is difficult in situations where many cameras are required. is there. Even in such a case, the information processing apparatus 10 (10a) can acquire the movement locus with simple equipment.

In addition, by using the information processing apparatus 10 (10a) of each of the above embodiments, the following can be handled in the operation of a news camera or the like.
For example, the camera can be operated so as to always follow and photograph only the player or the ball. In addition, it is possible to process an image so that a ball or shuttle moving at a speed that cannot be followed by human eyes is conspicuous.

  In each of the above embodiments, the information processing apparatus 10 (10a) uses the position information and the movement trajectory to measure the movement trajectory and the flight distance in ski jumps, land, swimming, speed skating, etc. The recording measurement may be performed.

In the second embodiment, the play determination unit 126 may perform the following processing as the play determination from the position information and the movement locus, the movement of the skeleton / joint, and the locus of the surrounding area.
For example, the play determination unit 126 may notify a change in formation at the time of set play such as soccer, or may predict a success rate that is hit only by a form such as archery. In addition, the play determination unit 126 may estimate the condition and the degree of fatigue only with a form such as archery or weight lifting. Further, the play determination unit 126 may determine the play of a player who does not hold the ball, such as a basket screen play, a volleyball block, or the like. May be automatically measured.

  In the second embodiment, the play evaluation unit 127 may predict the success rate of the technique or the competition based on the play determination result and the past movement trajectory. The play evaluation unit 127 may predict the success rate of, for example, a figure skating jump or a gymnastic technique. Further, for example, in the weight lifting, the play evaluation unit 127 may predict the success rate from the trajectory of the barbell and the movement of the skeleton / joint. This makes it possible to manage the form appropriately.

  In the second embodiment described above, the play evaluation unit 127 may visualize the position of the object, the trajectory of the various objects identified, and the evaluation data. The play evaluation unit 127 may display a display screen (image G2) as shown in FIG.

FIG. 12 is a diagram illustrating an example of a display screen in a modification of the play determination system 1a according to the second embodiment.
The play evaluation unit 127 displays various information as shown below on the display unit (not shown) as a visualization application, as shown in the image G2.
The play evaluation unit 127 displays, as a visualization application, for example, one image or a time range in the image, thereby displaying the image of the part, position data, and inadequate evaluation data. In addition, in the image G2, video / position data and evaluation data can be displayed / not displayed.

  Further, the play evaluation unit 127 displays the movement of the position in the target area of the equipment such as the player, the ball, and the racket as the position data in the visualization application, and the position data includes the identified identification information (for example, the identification ID). ) Is displayed. In addition, the play evaluation unit 127 has a function of the visualization application, for example, video and position data are displayed in time synchronization, and tracking data (tracked movement trajectory) for each player is displayed as evaluation data. Processing such as displaying statistical data such as the number of times, height, maximum speed, and number of sprints is executed.

  In addition, as a function of the visualization application, for example, in the case of a speed curve, the play evaluation unit 127 displays video, position data, and point data in the graph in conjunction with each other, in addition to player-specific data. In addition to displaying statistical data and displaying statistics such as graphs, processing such as displaying a table based on a table is executed. Further, the play evaluation unit 127 may display the data of equipment such as a ball and a racket as the evaluation application function as the function of the visualization application, for example, the speed, the number of rotations, the spin type, the racket, etc. The speed, shot type, impact position, shot power data, etc. may be displayed.

Next, with reference to FIG. 13, another modified example of visualizing video and evaluation data in the play determination system 1a will be described.
Moreover, FIG. 13 is a figure which shows another modification of the play determination system 1a by 2nd Embodiment.

  In this figure, the play determination system 1a includes a screen 40 (an example of a display unit) installed on the upper part of a valley court AR4. The information processing device 10a displays the video imaged by the imaging device 30 on the screen 40 and displays the evaluation data associated with the play in an overlapping manner (image G3 in FIG. 13). For example, in the case of volleyball, the information processing apparatus 10a displays an attack hitting height, a ball speed, a ball angle, and the like as shown in an image G3. Further, it is assumed that the information processing apparatus 10a can specify the content and display position of evaluation data to be overlaid on the video. Further, when measurement is performed using a plurality of imaging devices 30, the information processing device 10 a can specify which imaging device 30 video is to be displayed on the screen 40. Moreover, although the example which uses the screen 40 is demonstrated as an example of a display part, if it is a display apparatus installed in the circumference | surroundings of the coat AR4, another display, a screen, a display apparatus, etc. may be sufficient.

In addition, since display of evaluation data requires measurement and determination processing, the video displayed on the screen 40 may be reproduced with a delay from real time. In this case, the information processing apparatus 10a may allow the user to set the delay time for displaying the evaluation data and the time for maintaining the display of the evaluation data. In this case, for example, if the delay time is set to 5 seconds in the attack practice, and a plurality of players attack one after another, the user can check the current evaluation data and form on the screen 40 after the attack. it can.
Further, for example, when performing the formation practice, the delay time is set to 20 seconds, and a series of play of receive, toss, and attack can be confirmed on the display such as the screen 40 after the play.

  In the second embodiment, the information processing apparatus 10a causes the evaluation data generated and accumulated by the play evaluation unit 127 to be output to the outside as text data, CSV file, or the like according to the user's specification. It may be. In this case, the evaluation data includes, for example, the transition of the hit point height of the attack in one game, the number of blocks and the success rate at the time of the attack, the number of jumps and the height in the practice of the day, and the like. In the play determination system 1a, by outputting these evaluation data to the outside, an analyst and a coach can efficiently perform game analysis and training evaluation from the output data.

Each of the components included in the position tracking system 1 and the play determination system 1a described above has a computer system inside. And the program for implement | achieving the function of each structure with which the position tracking system 1 and the play determination system 1a which were mentioned above are provided is recorded on a computer-readable recording medium, and the program recorded on this recording medium is read into a computer system. The processing in each configuration provided in the position tracking system 1 and the play determination system 1a described above may be performed by executing. Here, “loading and executing a program recorded on a recording medium into a computer system” includes installing the program in the computer system. The “computer system” here includes an OS and hardware such as peripheral devices.
Further, the “computer system” may include a plurality of computer devices connected via a network including a communication line such as the Internet, WAN, LAN, and dedicated line. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. As described above, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM.

  The recording medium also includes a recording medium provided inside or outside that is accessible from the distribution server in order to distribute the program. It should be noted that after the program is divided into a plurality of parts and downloaded at different timings, the structure incorporated in each structure included in the position tracking system 1 and the play determination system 1a and the distribution server that distributes each of the divided programs are different. May be. Furthermore, the “computer-readable recording medium” holds a program for a certain period of time, such as a volatile memory (RAM) inside a computer system that becomes a server or a client when the program is transmitted via a network. Including things. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  Moreover, you may implement | achieve part or all of the function mentioned above as integrated circuits, such as LSI (Large Scale Integration). Each function described above may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the advancement of semiconductor technology, an integrated circuit based on the technology may be used.

  DESCRIPTION OF SYMBOLS 1 ... Position tracking system, 1a ... Play determination system, 10, 10a ... Information processing apparatus, 11, 11a ... Memory | storage part, 12, 12a ... Control part, 20 ... Laser radar apparatus, 30 ... Imaging device, 40 ... Screen, 111 ... radar detection storage unit, 112 ... image information storage unit, 113 ... object information storage unit, 114 ... movement locus storage unit, 115 ... object locus storage unit, 116 ... skeletal joint information storage unit, 117 ... determiner information storage 118: Determination result storage unit, 121 ... Data acquisition unit, 122 ... Image processing unit, 123 ... Identification processing unit, 124 ... Trajectory generation unit, 125 ... Skeletal estimation unit, 126 ... Play determination unit, 127 ... Play evaluation unit

Claims (6)

A detection device that irradiates a target area with waves and detects detection information including at least a distance and a direction to the object based on a reflected wave from the object;
An imaging device that captures an image including the target area;
An acquisition unit that acquires and synchronizes the detection information detected by the detection device and image information representing an image captured by the imaging device;
The object is identified as a measurement object based on a partial image in the image information corresponding to the position of the object detected based on the detection information acquired by the acquisition unit, and the identified measurement object On the other hand, an identification processing unit for providing identification information for identifying the measurement object;
The movement trajectory of the measurement object is obtained by associating the position information of the measurement object in the target area generated based on the detection information and the image information with the identification information given by the identification processing unit. A position tracking system comprising: a locus generation unit that generates The detection information includes an outline of the object,
The identification processing unit extracts the partial image in the image information based on the contour of the object detected by the detection device, and identifies the object as a measurement target based on the extracted partial image. The position tracking system according to claim 1. The measurement object includes a target person,
The said identification process part identifies the said target person based on the image recognition of the said target person's face or clothes, when identifying the said target person as the said measurement target object. 2. The position tracking system according to 2. The identification processing unit estimates the skeleton or joint of the target person based on the partial image when the target person cannot be identified based on image recognition of the face or clothing of the target person, and the estimated skeleton The position tracking system according to claim 3, wherein the target person is identified based on the joint and a predetermined skeleton or joint characteristic of the target person. The trajectory generation unit includes first position information that is position information of the object in the target area based on the detection information, and second position information that is position information of the measurement target in the target area based on the image information. Are mutually corrected, the position information of the measurement object is corrected, the corrected position information of the measurement object is associated with the identification information provided by the identification processing unit, and the measurement object The position tracking system according to any one of claims 1 to 4, wherein a movement trajectory is generated. A detection device that irradiates a target area with waves and detects detection information including at least a distance and a direction to the object based on a reflected wave from the object, and an imaging device that captures an image including the target area A position tracking method for a position tracking system comprising:
An acquisition step in which an acquisition unit acquires and synchronizes the detection information detected by the detection device and image information representing an image captured by the imaging device;
An identification processing unit identifies the object as a measurement target based on a partial image in the image information corresponding to the position of the object detected based on the detection information acquired by the acquisition step, and performs identification. An identification processing step for providing identification information for identifying the measurement object to the measurement object;
A trajectory generation unit associates the position information of the measurement object in the target area generated based on the detection information and the image information, and the identification information given by the identification processing step, A position tracking method comprising: a locus generation step for generating a movement locus of a measurement object.

Images