JP2020030190A - Position tracking system and position tracking method - Google Patents

Abstract

To provide a position tracking system and a position tracking method, capable of highly accurately tracking a position of a measurement object while suppressing a processing load.SOLUTION: A position tracking system comprises: a detecting device which irradiates an object area with waves and detects detection information containing at least a distance to an object and a direction on the basis of a reflection wave from the object; an imaging device which captures an image containing the object area; an acquisition unit which acquires and synchronizes the detection information detected by the detecting device and image information representing the image captured by the imaging device; an identification processing unit which identifies the object as a measurement object on the basis of a partial image in the image information corresponding to a position of the object detected on the basis of the detection information acquired by the acquisition unit, and imparts identification information for identifying the measurement object to the identified measurement object; and a trajectory generation unit which associates position information on the measurement object in the object area generated on the basis of the detection information and the image information with identification information imparted by the identification processing unit, to generate a movement trajectory of the measurement object.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a position tracking system and a position tracking method.

  2. Description of the Related Art In recent years, in sports, a position tracking system that measures the position of a measurement target such as a person or an object and tracks the position of the measurement target is known (for example, see Patent Literature 1). In such a position tracking system, the position of the measurement target is measured based on images captured by a plurality of cameras.

JP 2013-58132 A

  However, in the conventional position tracking system described above, since the position of the measurement target is detected from the image, for example, when the measurement target overlaps, the image is detected by more cameras to detect the position of the measurement target. Need to be imaged. Therefore, in the conventional position tracking system, the processing load of position measurement by image processing is high, and it is difficult to track the position of the measurement target with high accuracy while suppressing the processing load.

  The present invention has been made to solve the above-described problem, and an object of the present invention is to provide a position tracking system and a position tracking method capable of tracking a position of a measurement target with high accuracy while suppressing a processing load. It is in.

  In order to solve the above problem, an embodiment of the present invention provides a detection device which irradiates a target area with a wave and detects detection information including at least a distance and a direction to the object based on a reflected wave from the object. An imaging device that captures an image including the target area; an acquisition unit that acquires and synchronizes the detection information detected by the detection device and image information representing an image captured by the imaging device; Corresponding to the position of the object detected based on the detection information obtained by the unit, based on the partial image in the image information, to identify the object as a measurement object, for the identified measurement object An identification processing unit that provides identification information for identifying the measurement target object, the detection information, and the position information of the measurement target object in the target area generated based on the image information, In association with the identification information whose serial identification processor has given a position tracking system, characterized in that it comprises a trajectory generator for generating a movement trajectory of the measurement object.

  One aspect of the present invention is the position tracking system, wherein the detection information includes an outline of the object, and the identification processing unit performs the image processing based on the outline of the object detected by the detection device. The method is characterized in that the partial image in information is extracted, and the object is identified as a measurement target based on the extracted partial image.

  One aspect of the present invention is the position tracking system, wherein the measurement target includes a target person, and the identification processing unit is configured to identify the target person as the measurement target when the target person is identified as the measurement target. The target person is identified based on image recognition of the face or clothes.

  One aspect of the present invention is the above-mentioned position tracking system, wherein the identification processing unit, when the target person cannot be identified based on image recognition of the face or clothes of the target person, based on the partial image, A skeleton or a joint of a target person is estimated, and the target person is identified based on the estimated skeleton or the joint and predetermined characteristics of the skeleton or joint of the target person.

  One aspect of the present invention is the position tracking system, wherein the trajectory generation unit is configured to: first position information that is position information of the object in the target area based on the detection information; and the target area based on the image information. The second position information, which is the position information of the measurement object, is complemented with each other to correct the position information of the measurement object, and the corrected position information of the measurement object and the identification processing unit are added. The moving trajectory of the measurement object is generated in association with the identification information.

  One embodiment of the present invention provides a detection device that irradiates a wave to a target area and detects detection information including at least a distance and a direction to the object based on a reflected wave from the object, and an image including the target area. An imaging device for capturing an image of the position tracking system of a position tracking system comprising an imaging device, wherein the acquisition unit acquires the detection information detected by the detection device and image information representing an image captured by the imaging device. An acquisition step of synchronizing, and the identification processing unit measures the object based on a partial image in the image information corresponding to the position of the object detected based on the detection information acquired in the acquisition step. An identification processing step of assigning identification information for identifying the measurement target object to the measurement target object identified as an object; A trajectory for generating a movement trajectory of the measurement target by associating position information of the measurement target in the target area generated based on the image information with the identification information given in the identification processing step And a generating step.

  ADVANTAGE OF THE INVENTION According to this invention, the position of a measuring object can be tracked with high precision, suppressing a processing load.

It is a block diagram showing an example of a position tracking system by a 1st embodiment. FIG. 2 is a diagram illustrating an example of installation of a laser radar device and an imaging device according to the first embodiment. FIG. 5 is a diagram illustrating an example of data in an object information storage unit according to the first embodiment. FIG. 5 is a diagram illustrating an example of data in a movement trajectory storage unit according to the first embodiment. FIG. 5 is a diagram illustrating an example of data in an object trajectory storage unit according to the first embodiment. FIG. 6 is a diagram illustrating an example of a movement trajectory generation process according to the first embodiment. FIG. 4 is a diagram illustrating an example of a process of calculating three-dimensional coordinates according to the first embodiment. FIG. 5 is a diagram illustrating an example of target person identification processing according to the first embodiment. 5 is a flowchart illustrating an example of an operation of the position tracking system according to the first embodiment. It is a block diagram showing an example of a play judging system by a 2nd embodiment. It is a flowchart which shows an example of operation | movement of the play determination system by 2nd Embodiment. It is a figure showing an example of the display screen in the modification of the play judging system by a 2nd embodiment. It is a figure showing another modification of the play judging system by a 2nd embodiment.

  Hereinafter, a position tracking system (position measurement system) and a play determination system according to an embodiment of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram illustrating an example of a position tracking system 1 according to the first embodiment.
As shown in FIG. 1, the position tracking system 1 includes an information processing device 10, a laser radar device 20, and an imaging device 30. The position tracking system 1 is an example of a position measurement system.

  The laser radar device 20 (an example of a detection device) irradiates a target area with laser light (for example, laser light such as near-infrared light) and includes at least a distance and a direction to the object based on reflected light from the object. This is a lidar device that detects detection information. Here, the target area is, for example, a court, a stadium, which is a stadium for sports. The laser radar device 20 detects an object in the target area based on the reflected light from the object, and, in addition to the distance and direction to the object, various information such as, for example, the contour of the object and the shape of the surface of the object. To detect. For example, the laser radar device 20 uses a TOF (Time of Flight) method that detects the distance to the object from the movement time of the laser light from the irradiation of the laser light to the reception of the reflected light reflected by the object, and the distance to the object. Is detected. The laser radar device 20 outputs the detected detection information to the information processing device 10.

The imaging device 30 is, for example, a camera device, and captures an image including the target area. The imaging device 30 outputs image information representing the captured image to the information processing device 10.
Although one laser radar device 20 and one imaging device 30 are shown in FIG. 1 for convenience of description, the position tracking system 1 includes a plurality of laser radar devices 20 and a plurality of imaging devices 30. May be provided.

FIG. 2 is a diagram illustrating an installation example of the laser radar device 20 and the imaging device 30 in the present embodiment.
As shown in FIG. 2, a plurality of laser radar devices 20 and a plurality of imaging devices 30 are arranged so as to detect three-dimensional position coordinates of an object (for example, a player PL1) on the court AR1, which is an object area. I have. In the present embodiment, the three-dimensional position coordinates are, for example, represented by the X-axis and the Y-axis orthogonal to the plane of the court AR1, and the height direction is represented by the Z-axis.

The information processing device 10 is, for example, a computer device such as a PC (personal computer). The information processing device 10 generates position information of the target in the target area based on the detection information detected by the laser radar device 20 and the image information captured by the imaging device 30, and based on the generated position information, Generate a movement trajectory of the object. Here, the target object is, for example, a target person such as a player or a player, and a target object indicating a tool such as a ball, a racket, a net, and a goal.
The information processing device 10 includes a storage unit 11 and a control unit 12.

  The storage unit 11 stores information used for various processes of the information processing device 10. The storage unit 11 includes a radar detection storage unit 111, an image information storage unit 112, an object information storage unit 113, a movement trajectory storage unit 114, and an object trajectory storage unit 115.

  The radar detection storage unit 111 stores detection information detected by the laser radar device 20 and time information in association with each other. In the radar detection storage unit 111, detection information acquired from the laser radar device 20 by a data acquisition unit 121 described later is stored in synchronization with image information captured by the imaging device 30.

  The image information storage unit 112 stores image information captured by the imaging device 30 and time information in association with each other. The image information storage unit 112 stores the image information acquired from the imaging device 30 by the data acquisition unit 121 described later in synchronization with the detection information detected by the laser radar device 20.

The object information storage unit 113 stores information for identifying an object that is a person or an object. Note that the target object here also includes the target person. Here, an example of data stored in the target information storage unit 113 will be described with reference to FIG.
FIG. 3 is a diagram illustrating an example of data in the object information storage unit 113 according to the present embodiment.
As illustrated in FIG. 3, the target information storage unit 113 stores, for example, “target name” and “feature amount information” in association with each other.

Here, the “object name” is a name of the object, and is identification information for identifying the object. The “feature amount information” indicates, for example, a feature amount for identifying a target from image information, and includes, for example, a shape, a color, a size, and an image pattern for determining the target. In addition, when the object is a target person such as a specific player, the “feature amount information” may be a feature amount for face recognition or a feature amount of clothing such as a uniform.
In the example illustrated in FIG. 3, the “target name” is stored in the target information storage unit 113 in association with “player A” and the feature amount information of the “player A”. The “ball name” is stored in association with “ball” and feature amount information of the “ball”.

  Returning to the description of FIG. 1, the movement trajectory storage unit 114 stores the movement trajectory information of the detected object. The movement trajectory storage unit 114 stores, for example, identification information of the detected object, time information, and three-dimensional position coordinates in association with each other. Here, an example of data stored in the movement trajectory storage unit 114 will be described with reference to FIG.

FIG. 4 is a diagram illustrating an example of data in the movement locus storage unit 114 according to the present embodiment.
As shown in FIG. 4, the movement trajectory storage unit 114 stores “identification ID”, “time”, and “coordinates” in association with each other. Here, “identification ID” is, for example, identification information for identifying an object detected by the laser radar device 20, and “time” indicates time information. Note that the time information may include date information. “Coordinates” are three-dimensional position coordinates, for example, two-dimensional coordinates in a target area as shown in FIG.
The example illustrated in FIG. 4 indicates that the movement trajectory whose “identification ID” is “ID1” is stored in association with “time” and “coordinate”.

  Returning again to the description of FIG. 1, the target trajectory storage unit 115 stores the identification information of the target identified by the identification processing unit 123 described later and the movement trajectory of the target in association with each other. Here, an example of data stored in the object locus storage unit 115 in the present embodiment will be described with reference to FIG.

FIG. 5 is a diagram illustrating an example of data in the object locus storage unit 115 according to the present embodiment.
In FIG. 5, the object locus storage unit 115 stores “object name”, “time”, and “coordinates” in association with each other. Here, the information in which “time” and “coordinate” are associated with each other corresponds to the above-described movement locus.
The example illustrated in FIG. 5 indicates that the movement trajectory in which the “object name” corresponds to “player A” is stored in association with “time” and “coordinates”. In the example illustrated in FIG. 5, the “target name” is “player A” and indicates a movement trajectory corresponding to the target person. For example, a moving trajectory corresponding to a tool (target object) such as a moving object may be stored.

Returning to the description of FIG. 1 again, the control unit 12 is a processor including, for example, a CPU (Central Processing Unit), and controls the information processing apparatus 10 in an integrated manner. For example, the control unit 12 controls the position information (first position information) of the target in the target area based on the detection information detected by the laser radar device 20 and the position information of the target in the target area based on the image information captured by the imaging device 30. The position information (second position information) and the position information (second position information) are complemented with each other to generate a movement trajectory of the target object.
The control unit 12 includes a data acquisition unit 121, an image processing unit 122, an identification processing unit 123, and a trajectory generation unit 124.

  The data acquisition unit 121 (an example of an acquisition unit) acquires and synchronizes detection information detected by the laser radar device 20 and image information representing an image captured by the imaging device 30. The data acquisition unit 121 stores the detection information acquired from the laser radar device 20 and the time information in the radar detection storage unit 111 in association with each other. Further, the data acquisition unit 121 stores the image information acquired from the imaging device 30 and the time information in the image information storage unit 112 in association with each other. The data acquisition unit 121 synchronizes the detection information with the image information by adding time information to the radar detection storage unit 111 and the image information storage unit 112.

  The image processing unit 122 performs various types of image processing based on the image information acquired by the data acquisition unit 121. The image processing unit 122 generates position information (second position information) of the target in the target area based on the image information, for example. The position information in the target area here is, for example, coordinate information of the center of gravity of the target object. Further, the image processing unit 122 extracts, for example, a partial image corresponding to the object detected by the laser radar device 20 from the image information, and based on the feature amount stored in the object information storage unit 113 for the partial image. Then, an image recognition process for identifying an object is executed. The image processing unit 122 performs, for example, a face recognition process using a feature amount, a pattern recognition process using a feature amount, and the like.

  The identification processing unit 123 identifies the object as an object based on the partial image in the image information corresponding to the position of the object detected based on the detection information acquired by the data acquisition unit 121, and assigns the identified object to the identified object. On the other hand, identification information for identifying the object is given. The identification processing unit 123 extracts, for example, a partial image in the image information based on the contour of the object detected by the laser radar device 20, and identifies the object as a target based on the extracted partial image. Specifically, the identification processing unit 123 causes the image processing unit 122 to extract a partial image based on the contour of the object and execute image processing such as face recognition on the partial image. The identification processing unit 123 identifies an object as a target object based on the result of the image processing performed by the image processing unit 122, and adds identification information for identifying the target object to the identified target object.

  The trajectory generation unit 124 (an example of a position measurement unit) is based on detection information (for example, distance and direction to the object) detected by the laser radar device 20, and the position information (first position information) of the target in the target area. Generate The trajectory generation unit 124 generates a movement trajectory of the object on which the object has moved based on the generated first position information and the above-described second position information. That is, the trajectory generation unit 124 corrects the position information of the target by complementing the first position information and the second position information with each other, and moves the target based on the corrected position information of the target. Generate a trajectory. Here, with reference to FIG. 6, a description will be given of processing performed by the trajectory generation unit 124 to generate the movement trajectory by mutually complementing the first position information and the second position information.

FIG. 6 is a diagram illustrating an example of a movement trajectory generation process according to the present embodiment.
FIG. 6A shows position coordinates (first position information) based on the detection information detected by the laser radar device 20. FIG. 6B shows position coordinates (second position information) based on image information captured by the imaging device 30. Here, the position coordinates (first position information) shown in FIG. 6A are indicated by black circles, and the position coordinates (second position information) shown in FIG. 6B are indicated by white circles. Times t1 to t21 indicate time information when each position coordinate is detected. In this example, to simplify the description, the position coordinates will be described using two-dimensional coordinates on the XY axes.

  The laser radar device 20 can measure accurate position information without applying a processing load. However, since the sampling frequency is low, the position coordinates (first position information) shown in FIG. become. Further, since image information can be continuously captured, the position coordinates (second position information) shown in FIG. 6B are continuously detected. However, the position coordinates (second position information) have a large processing load when detecting the position coordinates from the image information, and the accuracy is inferior to the position coordinates (first position information).

FIG. 6C shows a state in which the position coordinates (first position information) shown in FIG. 6A and the position coordinates (second position information) shown in FIG. 6B are superimposed. . FIG. 6D shows a trajectory in which the position coordinates (first position information) shown in FIG. 6A and the position coordinates (second position information) shown in FIG. 6B are mutually complemented. ing. That is, the trajectory generating unit 124 generates the position coordinates of the target object by mutually complementing the position coordinates (first position information) based on the detection information and the position coordinates (second position information) based on the image information. Finally, a movement trajectory as shown by a movement trajectory L0 in FIG. 6D is generated. The trajectory generation unit 124 causes the generated trajectory to be stored in the trajectory storage unit 114 in association with the identification information of the object, the time information, and the three-dimensional position coordinates, for example, as illustrated in FIG. Here, the identification information is an identification ID given to the detected object by a predetermined rule.
The laser radar device 20 and the imaging device 30 are arranged such that the three-dimensional coordinates based on the detection information in the target area and the three-dimensional coordinates based on the image information match.

  Further, the trajectory generation unit 124 corrects the position information on the plane of the target area based on the first position information based on the detection information, and positions the target area in the height direction based on the second position information based on the image information. The information may be corrected to generate three-dimensional coordinates (three-dimensional position information) including position information on the plane and position information in the height direction. Here, an example of a process of calculating three-dimensional coordinates according to the present embodiment will be described with reference to FIG.

FIG. 7 is a diagram illustrating an example of a process of calculating three-dimensional coordinates according to the present embodiment.
In FIG. 7, an image G1 indicates image information captured by the imaging device 30. Here, it is assumed that the court AR2 is a volleyball court, and the player PL2 is about to hit the ball B1. In such a case, it is difficult to determine whether or not the player PL2 is jumping from the image G1.

  Therefore, in order to generate accurate three-dimensional coordinates of the player PL2, the trajectory generating unit 124 uses the coordinates of the position PT1 on the XY plane on the court AR2 of the player PL2 based on the detection information detected by the laser radar device 20 ( x1, y1) is calculated. Then, the trajectory generation unit 124 calculates the position coordinates (z1) in the height direction (Z-axis direction) of the player PL2 from the position of the player PL2 on the image information (image G1) captured by the imaging device 30 and The three-dimensional coordinates (x1, y1, z1) of PL2 are generated.

  The trajectory generation unit 124 calculates the position coordinates (z1) of the player PL2, and calculates the position coordinates (z1) of the net NT, the poles (P1, P2), the line LN of the court AR2, and the like, and the positional relationship of the player PL2 on the screen. The position coordinates (z1) of the player PL2 are calculated. In addition, the laser radar device 20 and the imaging device 30 determine the three-dimensional coordinates based on the detection information and the three-dimensional coordinates based on the image information based on the positional relationship between the net NT, the poles (P1, P2), and the line LN in the court AR2. They are arranged so that their dimensional coordinates match. In addition, in the above-described processing, the trajectory generation unit 124 may correct the position information on the plane of the target area based on the first position information and the second position information.

  In addition, the trajectory generation unit 124 generates a movement trajectory of the target by associating the generated position information of the target in the target area with the identification information given by the identification processing unit 123. In other words, the trajectory generation unit 124 reads the trajectory of the object stored in the trajectory storage unit 114, associates the trajectory of the object with the identification information of the object given by the identification processing unit 123, and Is stored in the object locus storage unit 115 as the movement locus of the object. Specifically, for example, as shown in FIG. 5, the trajectory generation unit 124 determines the identification information of the object (“object name”), the time information (“time”), and the three-dimensional position coordinates (“coordinates”). ) Are stored in the movement trajectory storage unit 114 in association with each other.

When identifying the target person as the target, the identification processing unit 123 identifies the target person based on image recognition of the face or clothes of the target person. That is, the identification processing unit 123 requests the image processing unit 122 to perform identification, and the image processing unit 122 identifies the target person based on the feature amount of the face or clothes of the target person stored in the target information storage unit 113. I do. The identification processing unit 123 identifies a target person based on the result of the image processing unit 122 performing image recognition.
Here, the identification processing of the target person by the identification processing unit 123 will be described with reference to FIG.

FIG. 8 is a diagram illustrating an example of the target person identification processing according to the present embodiment.
In FIG. 8, a process of recognizing the target person when the player PL3 (player B) and the player PL4 (player A) run across the court AR3 will be described.

  As shown in FIG. 8A, when the player PL3 (player B) and the player PL4 (player A) cross the court AR3, the trajectory generator 124 identifies the object detected by the laser radar device 20 as an object. IDs (for example, ID1 to ID4) are given, and a moving trajectory of each object is generated as shown in FIG. In FIG. 8B, the horizontal axis represents time, and the vertical axis represents the identification ID of each object.

  In the graph, ○ (circle) and X (cross) indicate the time at which the identification processing unit 123 requests the image processing unit 122 to identify the object. A circle (circle) indicates that the object (player PL3 or player PL4) to which the identification ID has been assigned can be identified by the image processing unit 122, and a cross (cross) indicates that the identification ID has been assigned. Indicates that the identification of the object failed. When the identification processing unit 123 fails to identify the object to which the identification ID has been assigned, after a predetermined interval, requests the image processing unit 122 to identify the object again. It does not request identification of the object to which the identification ID has been assigned. Thereby, the identification processing unit 123 reduces the processing load for identifying the object. The trajectory generation unit 124 stores the generated trajectory in the trajectory storage unit 114 as shown in FIG.

  Further, the identification processing unit 123 determines the object (the player PL3 or the player PL4) based on the partial image of the position where the object (the player PL3 or the player PL4) is detected, and the feature ( The player (PL3 or PL4) is identified, and a target object name is given to the object (player PL3 or PL4). The identification processing unit 123 assigns “player B” to the player PL3 and “player A” to the player PL4, for example, as shown in 8 (c). The trajectory generation unit 124 associates the target object name given by the identification processing unit 123 with the trajectory stored in the trajectory storage unit 114, as shown in FIG.

  8C and 8D, the movement locus of “player A” is the movement locus L1, and the movement locus of “player B” is the movement locus L2. The trajectory generation unit 124 causes the target trajectory storage unit 115 to store the movement trajectory to which the target names such as “player A” and “player B” are given, for example, as illustrated in FIG.

Next, an operation of the position tracking system 1 according to the present embodiment will be described with reference to the drawings.
FIG. 9 is a flowchart illustrating an example of the operation of the position tracking system 1 according to the present embodiment. In this figure, it is assumed that the laser radar device 20 constantly detects the detection information, and the imaging device 30 constantly captures the image information.

  9, first, the information processing apparatus 10 of the position tracking system 1 acquires detection information of the laser radar apparatus 20 and image information and synchronizes them (step S101). The data acquisition unit 121 of the information processing device 10 acquires detection information from the laser radar device 20 and acquires image information from the imaging device 30. The data acquisition unit 121 stores the detection information acquired from the laser radar device 20 and the time information in the radar detection storage unit 111 in association with each other. Further, the data acquisition unit 121 stores the image information acquired from the imaging device 30 and the time information in the image information storage unit 112 in association with each other.

  Next, the information processing device 10 generates position information in the target area based on the detection information of the laser data device and the image information (Step S102). The trajectory generation unit 124 of the information processing device 10 detects position information (first information) of the detected object (target) in the target area based on the detection information (for example, the distance and direction to the object) detected by the laser radar device 20. Position information). Further, the image processing unit 122 generates position information (second position information) of the object (target) in the target area based on the image information, for example. Here, the first position information and the second position information are, for example, three-dimensional coordinates of the center of gravity of the target object. The trajectory generator 124 complements the first position information and the second position information, and corrects and generates the three-dimensional coordinates of the object.

  Next, the information processing apparatus 10 cuts out a partial image corresponding to the position information, performs image recognition on the partial image, and adds identification information (step S103). The identification processing unit 123 of the information processing device 10 causes the image processing unit 122 to extract a partial image in the image information corresponding to the position of the object in the first position information described above. The identification processing unit 123 causes the image processing unit 122 to identify an object based on the extracted partial image and the feature amount stored in the object information storage unit 113, and The identification information for identifying the object is given.

  Next, the information processing device 10 generates a movement trajectory of the target to which the identification information is added based on the position information (Step S104). The trajectory generation unit 124 generates a movement trajectory of the object by associating the generated three-dimensional coordinates of the object, the identification information given by the identification processing unit 123, and the time information. For example, as shown in FIG. 5, the trajectory generation unit 124 associates the three-dimensional coordinates of the target object, the identification information of the target object, and the time information, and sets the movement trajectory storage unit 114 as the movement trajectory of the target object. To memorize. After the processing in step S104, the trajectory generation unit 124 ends the processing.

  As described above, the position tracking system 1 (position measurement system) according to the present embodiment includes the laser radar device 20 (detection device), the imaging device 30, the data acquisition unit 121 (acquisition unit), and the trajectory generation unit 124. (Position measurement unit). The laser radar device 20 irradiates the target area with laser light, and detects detection information including at least the distance and direction to the object based on the reflected light from the object. The imaging device 30 captures an image including the target area. The data acquisition unit 121 acquires and synchronizes detection information detected by the laser radar device 20 and image information representing an image captured by the imaging device 30. The trajectory generation unit 124 includes first position information that is position information of the object in the target area based on the detection information acquired by the data acquisition unit 121 and position information of the object in the target area based on the image information acquired by the data acquisition unit 121 And the position information of the target object is generated. The trajectory generation unit 124 generates a movement trajectory of the target object based on the generated position information of the target object.

  As a result, the position tracking system 1 according to the present embodiment can measure accurate position information without applying a processing load. However, the laser radar device 20 that performs intermittent detection due to a low sampling frequency and a processing load By using together with the imaging device 30 that is large and has low accuracy of the position information but can continuously capture images, the position of the target object can be measured with high accuracy while suppressing the processing load. The position tracking system 1 according to the present embodiment can realize, for example, accurate measurement of three-dimensional position coordinates, which is difficult with only the image information of the imaging device 30, by using the laser radar device 20 together. Therefore, the position tracking system 1 according to the present embodiment can track the position of the target object with high accuracy while suppressing the processing load.

In the present embodiment, the trajectory generating unit 124 corrects the position information of the target by complementing the first position information and the second position information with each other, and based on the corrected position information of the target. , To generate a movement locus of the object.
Thereby, the position tracking system 1 according to the present embodiment mutually complements the first position information based on the detection information of the laser radar device 20 and the second position information based on the image information of the imaging device 30, for example, In addition, a robust and highly accurate position measurement of an object can be performed without being affected by a complicated background or an external environment. The position tracking system 1 according to the present embodiment can accurately generate a movement trajectory, for example, even when objects overlap as shown in FIG.

  In addition, the position tracking system 1 according to the present embodiment complements each other, so that it is not necessary to increase the number of imaging devices 30 having a large processing load for image processing in order to detect position information with high accuracy. For example, real-time processing can be performed while suppressing the processing load. Further, in the position tracking system 1 according to the present embodiment, for example, it is not necessary to attach an accessory such as a marker to the target object, and the system configuration can be simplified (system).

Further, in the present embodiment, the trajectory generation unit 124 corrects the position information on the plane of the target area based on the first position information, and corrects the position information in the height direction of the target area based on the second position information. Then, three-dimensional position information including the position information on the plane and the position information in the height direction is generated, and the trajectory of the object is generated based on the generated three-dimensional position information.
Thereby, the position tracking system 1 according to the present embodiment can measure an accurate position in the height direction (Z-axis direction), which is difficult to measure from image information, from the image information, as shown in FIG. An accurate position in the depth direction (XY axis direction) can be measured by the radar device 20. Therefore, the position tracking system 1 according to the present embodiment can generate the trajectory of the target with higher accuracy.

In the present embodiment, the laser radar device 20 and the imaging device 30 are arranged such that the first position information and the second position information in the target area match.
Thereby, the position tracking system 1 according to the present embodiment can measure the position of the target with higher accuracy.

The position tracking method (position measurement method) according to the present embodiment is a position tracking method of the position tracking system 1 (position measurement system) including the laser radar device 20 (detection device) and the imaging device 30. And a trajectory generation step (position measurement step). In the acquisition step, the data acquisition unit 121 acquires and synchronizes the detection information detected by the laser radar device 20 and the image information representing the image captured by the imaging device 30. In the trajectory generation step (position measurement step), the trajectory generation unit 124 (position measurement unit) acquires the first position information, which is the position information of the object in the target area based on the detection information acquired in the acquisition step, and the acquisition step. The position information of the target object is generated by complementing the second position information that is the position information of the object in the target area based on the generated image information, and the target object moves based on the generated position information of the target object. Generate a movement trajectory of the object.
Accordingly, the position tracking method according to the present embodiment has the same effect as the above-described position tracking system 1, and can measure the position of the target with high accuracy while suppressing the processing load.

  The position tracking system 1 according to the present embodiment includes a laser radar device 20, an imaging device 30, a data acquisition unit 121, an identification processing unit 123, and a trajectory generation unit 124. The data acquisition unit 121 acquires and synchronizes detection information detected by the laser radar device 20 and image information representing an image captured by the imaging device 30. The identification processing unit 123 uses the image processing unit 122 to identify the object based on the partial image in the image information corresponding to the position of the object detected based on the detection information acquired by the data acquisition unit 121. And assigns identification information for identifying the identified object to the identified object. The trajectory generation unit 124 generates a movement trajectory of the target object by associating the position information of the target object in the target area generated based on the detection information and the image information with the identification information given by the identification processing unit 123. I do.

  Thus, the position tracking system 1 according to the present embodiment can identify a target object with high accuracy by using the detection information detected by the laser radar device 20 and the image information representing the image captured by the imaging device 30. it can. The position tracking system 1 according to the present embodiment performs, for example, an identification process on a partial image corresponding to the position of a detected object. And the like, and the processing load can be greatly reduced. Therefore, the position tracking system 1 according to the present embodiment can track the position of the measurement target with high accuracy while suppressing the processing load.

In the present embodiment, the detection information of the laser radar device 20 includes the contour of the object. The identification processing unit 123 extracts a partial image in the image information based on the contour of the object detected by the laser radar device 20 using the image processing unit 122, and extracts the object based on the extracted partial image. Identify as
Thereby, the position tracking system 1 according to the present embodiment can further remove noise such as a background by cutting out a partial image from the image information along the contour of the object, and recognize the object by image processing. Accuracy can be further improved.

In the present embodiment, the target object includes the target person. When identifying the target person as the target, the identification processing unit 123 uses the image processing unit 122 to identify the target person based on image recognition of the face or clothes of the target person.
Thereby, the position tracking system 1 according to the present embodiment can appropriately identify a target person such as a player or a player, for example.

Further, in the present embodiment, the trajectory generation unit 124 includes first position information that is position information of the object in the target area based on the detection information and second position information that is position information of the object in the target area based on the image information. Are complemented with each other, the position information of the object is corrected, and the corrected position information of the object is associated with the identification information given by the identification processing unit 123 to generate a movement locus of the object.
Accordingly, the position tracking system 1 according to the present embodiment can track the position of the measurement target with higher accuracy.

Further, the position tracking method according to the present embodiment irradiates a target area with laser light, and based on reflected light from an object, a position tracking method of the position tracking system 1 including the laser radar device 20 and the imaging device 30. And includes an acquisition step, an identification processing step, and a trajectory generation step. In the acquisition step, the data acquisition unit 121 acquires and synchronizes the detection information detected by the laser radar device 20 and the image information representing the image captured by the imaging device 30. In the identification processing step, the identification processing unit 123 uses the image processing unit 122, based on the partial image in the image information corresponding to the position of the object detected based on the detection information acquired in the acquisition step, The object is identified as a target, and identification information for identifying the target is added to the identified target. In the trajectory generation step, the trajectory generation unit 124 associates the position information of the target in the target area generated based on the detection information and the image information with the identification information given in the identification processing step, and Is generated.
Thus, the position tracking method according to the present embodiment has the same effect as the position tracking system 1, and can track the position of the measurement target with high accuracy while suppressing the processing load.

[Second embodiment]
Next, a play determination system 1a according to a second embodiment will be described with reference to the drawings.
In the present embodiment, the play determination system 1a includes the above-described position tracking system 1 and determines the play of the target person indicating the motion and behavior of the target person based on the movement trajectory of the target person and the skeleton and joints of the target person. Will be described.

FIG. 10 is a block diagram illustrating an example of the play determination system 1a according to the present embodiment.
As shown in FIG. 10, the play determination system 1a (an example of a determination system) includes an information processing device 10a, a laser radar device 20, and an imaging device 30.
In this figure, the same components as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. In the present embodiment, the laser radar device 20 and the imaging device 30 are examples of a detection device that can detect at least shape and position information of an object.

The information processing device 10a is, for example, a computer device such as a PC. The information processing apparatus 10a, similar to the information processing apparatus 10 according to the above-described first embodiment, determines a target in the target area based on detection information detected by the laser radar device 20 and image information captured by the imaging device 30. The position information of the object is generated, and the movement locus of the object is generated based on the generated position information. Further, the information processing apparatus 10a estimates the skeleton and joints of the target person from the image information, and determines the play of the target person from the skeleton and joints of the target person and the movement locus of the target object. Here, the play indicates, for example, the action and behavior of the target person in sports.
Further, the information processing device 10a includes a storage unit 11a and a control unit 12a.

  The storage unit 11a stores information used for various processes of the information processing device 10a. The storage unit 11a includes a radar detection storage unit 111, an image information storage unit 112, an object information storage unit 113, a movement trajectory storage unit 114, an object trajectory storage unit 115, a skeletal joint information storage unit 116, It includes a determiner information storage unit 117 and a determination result storage unit 118.

  The skeletal joint information storage unit 116 stores skeletal joint information estimated by a skeleton estimating unit 125 described later. The skeletal joint information storage unit 116 stores, for example, identification information of the target person, time information, and skeletal joint information indicating a skeleton and a joint in association with each other. Here, the skeletal joint information is, for example, three-dimensional position coordinates of the skeleton and the joint.

The determiner information storage unit 117 stores information of a determiner for determining play. The determiner information storage unit 117 performs, for example, learning performed by machine learning so as to determine the play of the target person indicating the motion and behavior of the target person based on the past movement trajectory of the target person and the skeleton and joints of the target person. The result and the like are stored as information of the determiner. Here, the machine learning includes, for example, decision tree learning, a neural network, a support vector machine (SVM), various regression models, deep learning (deep learning), and the like. The information of the determiner includes various weighting information, rule information, various parameters, and the like.
In addition, the play determination includes, in addition to the type of play, success or failure of the play, performance, score determination, and the like.

  The determination result storage unit 118 stores the determination result determined by the play determination unit 126 described later. The determination result storage unit 118 stores, for example, the identification information of the target person, the time information, and the play determination result in association with each other.

The control unit 12a is, for example, a processor including a CPU and the like, and controls the information processing apparatus 10a in a comprehensive manner. The control unit 12a performs the same processing as the control unit 12 of the first embodiment described above, and determines, for example, the play of the target person.
The control unit 12a includes a data acquisition unit 121, an image processing unit 122, an identification processing unit 123, a trajectory generation unit 124, a skeleton estimation unit 125, a play determination unit 126, and a play evaluation unit 127.

  The skeleton estimating unit 125 estimates the skeleton and joints of the target person based on the detection information detected by the detection device (for example, the laser radar device 20 and the imaging device 30). The skeleton estimating unit 125 estimates the skeleton and joints of the target person based on the contour of the target person detected by the laser radar device 20 and image information including the target person captured by the imaging device 30. The skeleton estimation unit 125 causes the skeleton joint information storage unit 116 to store the skeleton and joint information indicating the estimated skeleton and joint, the identification information of the target person, and the time information in association with each other. The skeleton estimating unit 125 estimates the skeleton and joints of the target person using, for example, existing skeleton tracking technology. The skeleton estimating unit 125 may estimate the line of sight of the target person in addition to the skeleton and the joint. In this case, the skeletal joint information may include information indicating a line of sight. Further, the skeleton estimating unit 125 may estimate one of the skeleton and the joint of the target person.

  The image processing unit 122 and the identification processing unit 123 are the same as those in the first embodiment, and a description thereof will not be repeated. When the target person cannot be identified based on the image recognition of the face or clothes of the target person, the identification processing unit 123 determines the skeleton or joint of the target person (the skeleton and joints of the target person, or One of the skeleton and the joint of the target person) may be estimated, and the target person may be identified based on the estimated skeleton or joint and predetermined characteristics of the skeleton or the joint of the target person. That is, the object information storage unit 113 stores, for example, the feature amount of the skeleton or joint of the target person in advance, and the identification processing unit 123 stores the skeleton or joint estimated by the skeleton estimation unit 125 described above, The target person may be identified based on the feature amount of the target person's skeleton or joint stored in the information storage unit 113.

The trajectory generation unit 124 is the same as in the first embodiment, and a description thereof will not be repeated. That is, the trajectory generation unit 124 generates a movement trajectory of the target person based on the detection information detected by the detection device (for example, the laser radar device 20 and the imaging device 30).
Note that, in addition to the movement trajectory of the target person, the trajectory generation unit 124 according to the present embodiment includes a movement trajectory of an object around the target person, a trajectory of a skeleton and a joint, and a part of a body such as a hand or a foot of the target person. May be generated.

  The play determination unit 126 determines the target person's play based on the skeleton and joints of the target person estimated by the skeleton estimation unit 125, the movement trajectory of the target person generated by the trajectory generation unit 124, and a determinator that determines play of the target person. The play of the person is determined. Here, the determiner is generated so as to determine the play of the target person indicating the motion and action of the target person based on the past movement trajectory of the target person and the skeleton and joints of the target person. It is generated by learning. The play determination unit 126 determines play of the target person from the skeleton and joints of the target person and the movement trajectory of the target person, using a determiner based on the determiner information stored in the determiner information storage unit 117.

  Here, the play of the target person is, for example, a player's receive, attack, serve, block, or the like in volleyball, or screen play in basketball. In addition, for example, when determining play in volleyball, the play determination unit 126 determines, in addition to the skeleton and joints of the target person and the movement trajectory of the target person, a tool movement trajectory such as a ball, The play of the target person may be determined based on the movement trajectory of the target object. In this case, the trajectory generation unit 124 generates a movement trajectory of an object (for example, a ball) around the target person, and the play determination unit 126 uses a determination unit to determine the movement trajectory and skeleton of the target person. Then, the play of the target person is determined from the joint and the movement locus of the target object (for example, a ball).

Further, for example, when determining a screen play in a basketball, the play determination unit 126 may determine the play of the target person based on the movement trajectory of a person around the target person. In this case, the trajectory generation unit 124 generates a movement trajectory of an object (for example, a nearby person) existing around the target person, and the play determination unit 126 uses a determination unit to determine the movement trajectory of the target person. The play of the target person is determined from the skeleton, the joint, and the movement locus of the target object (for example, a nearby person).
The play determination unit 126 stores information indicating the determined play as the determination result in the determination result storage unit 118 in association with the target person and the time information.

  The play evaluation unit 127 evaluates play based on the play determination result determined by the play determination unit 126. The play evaluation unit 127, for example, totals the number of times of execution of a play such as a jump during training from the determination result of the play, and evaluates the effect, efficiency, overtraining, and the like of the training. Further, the play evaluation unit 127, for example, tallies the plays performed in a game or the like based on the determination result of the play, and evaluates the content of the game based on the tallied result of the play.

Next, the operation of the play determination system 1a according to the present embodiment will be described with reference to the drawings.
FIG. 11 is a flowchart illustrating an example of the operation of the play determination system 1a according to the present embodiment. In this figure, it is assumed that the laser radar device 20 constantly detects the detection information, and the imaging device 30 constantly captures the image information.
Since the processing from step S201 to step S203 shown in FIG. 11 is the same as the processing from step S101 to step S103 shown in FIG. 9, the description thereof is omitted here.

  In step S204, the information processing device 10a estimates a skeleton and a joint based on the detection information and the image information. The skeleton estimating unit 125 of the information processing device 10a, for example, determines the skeleton and joints of the target person based on the contour of the target person detected by the laser radar device 20 and image information including the target person captured by the imaging device 30. presume. The skeleton estimation unit 125 causes the skeleton joint information storage unit 116 to store the skeleton and joint information indicating the estimated skeleton and joint, the identification information of the target person, and the time information in association with each other.

  Next, the information processing device 10a generates a movement locus of the measurement target to which the identification information is added based on the position information (Step S205). The trajectory generation unit 124 of the information processing device 10a associates the generated three-dimensional coordinates of the target object, the identification information given by the identification processing unit 123, and the time information, as in step S104 illustrated in FIG. Generate a trajectory of the object.

  Next, the information processing device 10a makes a play determination based on the feature amount of the movement trajectory and the skeleton and the joint (step S206). The play determination unit 126 of the information processing apparatus 10a uses a determiner based on the determiner information stored in the determiner information storage unit 117 to determine the skeleton and joints of the target person and the feature amount of the movement trajectory of the target person. The play of the target person is determined. The play determination unit 126 causes the determination result storage unit 118 to store the determination result of the play.

  Next, the information processing device 10a determines whether or not the play determination has been completed (Step S207). If the play determination has been completed (step S207: YES), the information processing device 10a advances the process to step S208. If the play determination has not been completed (step S207: NO), the information processing device 10a returns the process to step S201, and repeats the processes from step S201 to step S207.

In step S208, the information processing device 10a evaluates the play determination result. The play evaluation unit 127 of the information processing device 10a evaluates the play based on the play determination result determined by the play determination unit 126. After the processing in step S208, the play determination unit 126 ends the processing.
In the example described above, the skeleton estimating unit 125 estimates the skeleton and joints of the target person, and the play determination unit 126 determines play based on the movement trajectory of the target person and the positions of the skeleton and joints of the target person. However, the skeleton estimating unit 125 estimates one of the skeleton and the joint of the target person, and determines the play from the movement trajectory of the target person and one of the positions of the skeleton and the joint of the target person. You may make it.

  As described above, the play determination system 1a (determination system) according to the present embodiment includes a detection device (for example, the laser radar device 20 and the imaging device 30), a skeleton estimation unit 125, a trajectory generation unit 124, and a play determination. A part 126. The detection device (for example, the laser radar device 20 and the imaging device 30) can detect at least the shape and position information of the object. The skeleton estimation unit 125 estimates the skeleton or joint of the target person based on the detection information detected by the detection device. The trajectory generation unit 124 generates a trajectory of the target person based on the detection information detected by the detection device. The play determination unit 126 determines the play of the target person based on the determiner, the skeleton or joint of the target person estimated by the skeleton estimation unit 125, and the movement trajectory of the target person generated by the trajectory generation unit 124. Here, the determiner is generated so as to determine the play of the target person indicating the motion and action of the target person based on the past movement trajectory of the target person and the skeleton or joint of the target person.

  Accordingly, the play determination system 1a according to the present embodiment determines play based on the skeleton or joint of the target person and the movement trajectory of the target person. A determination can be made. Further, the play determination system 1a according to the present embodiment can automatically record, for example, a player's play, position, time, success or failure of play, performance, score determination, and the like without any burden on the user. That is, the play determination system 1a according to the present embodiment can automatically record the score of the sports competition.

Further, in the present embodiment, the trajectory generation unit 124 generates a movement trajectory of an object existing around the target person based on detection information detected by the detection device (for example, the laser radar device 20 and the imaging device 30). I do. The play determination unit 126 determines play of the target person from the movement trajectory of the target person, the skeleton or joints of the target person, and the movement trajectory of the target object, using a determination unit.
Thus, the play determination system 1a according to the present embodiment can improve the accuracy of play determination by using the movement trajectory of an object such as a ball or a racket.

For example, in a volleyball, the play determination system 1a according to the present embodiment determines a play, an attack, a serve, a block, and the like, for example, by performing a play determination using a movement locus of an object such as a ball as a determiner. be able to.
In addition, for example, in the basket, the play determination system 1a according to the present embodiment accurately determines, for example, screen play by performing play determination using the movement locus of a person around the target person as a determiner. be able to.
In addition, for example, in tennis, the play determination is performed by using the skeleton of the target person and the movement trajectories of the target person, the ball, and the racket as a determiner, thereby determining the type of tennis shot (for example, backhand, forehand, smash). , Slice, etc.), a ball (eg, speed, spin type, rotation speed, etc.), success or failure of a shot, and the like.

  In the present embodiment, the detection device includes the laser radar device 20 and the imaging device 30. The skeleton estimating unit 125 estimates the skeleton or joint of the target person based on the contour of the target person detected by the laser radar device 20 and image information including the target person captured by the imaging device 30. The trajectory generation unit 124 converts first position information that is position information of the target person in the target area based on the detection by the laser radar device 20 and second position information that is position information of the target person in the target area based on the image information. Complementing each other, the position information of the target person is corrected, and the movement trajectory of the target person is generated based on the corrected target person position information.

  As a result, the play determination system 1a according to the present embodiment complements the first position information based on the detection information of the laser radar device 20 and the second position information based on the image information of the imaging device 30 with each other to perform processing. The position of the object can be measured while suppressing the load. Therefore, the play determination system 1a according to the present embodiment can perform the play determination of the target person with higher accuracy.

Further, the play determination system 1a according to the present embodiment identifies the target person based on the partial image in the image information corresponding to the first position information based on the detection information of the laser radar device 20. And an identification processing unit 123 for assigning identification information for identifying the target person. The trajectory generation unit 124 generates a movement trajectory of the target person in association with the identification information given by the identification processing unit 123.
Accordingly, the play determination system 1a according to the present embodiment can identify a target object with high accuracy by using the detection information detected by the laser radar device 20 and the image information representing the image captured by the imaging device 30. Thus, the position of the measurement target can be tracked with high accuracy while suppressing the processing load. Therefore, the play determination system 1a according to the present embodiment can perform the play determination of the target person with higher accuracy.

  In the play determination system 1a according to the present embodiment, the imaging device 30 can detect position information with high accuracy by using the position information that is difficult to obtain with high accuracy in combination with the laser radar device 20. . Therefore, the play determination system 1a according to the present embodiment can broaden the types of play that can be determined, and can be applied to analysis of various games and games. The play determination system 1a according to the present embodiment improves the determination speed and the determination accuracy by using the imaging device 30 in combination with the laser radar device 20 in a play determination that requires a complicated and high processing load. be able to.

Further, in the present embodiment, when the target person cannot be identified based on the image recognition of the face or clothes of the target person, the identification processing unit 123 estimates the skeleton or joint of the target person based on the partial image and estimates The target person is identified based on the determined skeleton or joint and predetermined characteristics of the target person's skeleton or joint.
Thereby, the play determination system 1a according to the present embodiment can identify the target with higher accuracy.

In the present embodiment, the trajectory generation unit 124 generates position information on the plane of the target area based on the detection information of the laser radar device 20, and based on the image information of the imaging device 30, the height direction of the target area. To generate three-dimensional position information including position information on a plane and position information in a height direction, and generate a trajectory of an object (target person) based on the generated three-dimensional position information. I do.
As a result, the play determination system 1a according to the present embodiment can accurately measure a position, so that a play determination involving a jump can be performed with high accuracy, for example, and the types of play that can be determined can be further expanded.

The play determination system 1a according to the present embodiment includes a play evaluation unit 127 that evaluates play.
Thereby, the play determination system 1a according to the present embodiment can evaluate, for example, the mileage, the number of jumps, the height, and the play motion of each player during the practice, and perform the unified management of overtraining and the like. Can be.

The determination method according to the present embodiment includes a skeleton estimation step, a trajectory generation step, and a play determination step. In the skeleton estimation step, the skeleton estimation unit 125 estimates the skeleton or joint of the target person based on at least detection information detected by a detection device capable of detecting shape and position information of the object. In the trajectory generation step, the trajectory generation unit 124 generates a trajectory of the target person based on the detection information detected by the detection device. In the play determination step, the play determination unit 126 is generated so as to determine the play of the target person indicating the motion and behavior of the target person based on the past movement trajectory of the target person and the skeleton or joint of the target person. The play of the target person is determined based on the determiner, the skeleton or joint of the target person estimated by the skeleton estimating unit 125, and the movement trajectory of the target person generated by the trajectory generation unit 124.
Accordingly, the determination method according to the present embodiment has the same effect as the above-described play determination system 1a, and can perform the play determination of the target person with high accuracy while suppressing the processing load.

The present invention is not limited to the above embodiments, and can be modified without departing from the spirit of the present invention.
For example, the numbers of the laser radar devices 20 and the imaging devices 30 included in the position tracking system 1 (play determination system 1a) are not limited to the above embodiments, and may include other numbers.

  In each of the above embodiments, the example in which the information processing device 10 (10a) is a single device has been described, but the information processing device 10 (10a) is not limited thereto, and may be configured with a plurality of devices. In the information processing device 10 (10a), a part or all of the storage unit 11 (11a) may be provided in an external device. In this case, the external device may be a storage device that can be connected to the information processing device 10 (10a) via a network. Further, the information processing device 10 (10a) may be configured such that an external device has a part of the function of the control unit 12 (12a).

  In each of the above embodiments, a laser radar device is used as an example of a detection device that irradiates a wave to a target area and detects detection information including at least a distance and a direction to the object based on a reflected wave from the object. Although the example using 20 has been described, the present invention is not limited to this. For example, the detection device may be a device that irradiates a light wave, a radio wave, or a sound wave, and detects detection information including at least a distance and a direction to the object based on a reflected wave from the object. Here, the wave is a wave propagating in space such as a light wave such as a laser beam, a radio wave, or a sound wave such as an ultrasonic wave.

In the above-described second embodiment, an example will be described in which the play determination unit 126 determines play from the movement trajectory of the target person, the positions of the skeleton and joints, and the movement trajectory of the target object around the target person. However, the present invention is not limited to this. The play determination unit 126 is based on, for example, a movement trajectory of a skeleton and a joint, a positional relationship with a nearby target such as a pole, a net, and a court line, and a movement trajectory of a part of a body such as a hand or a foot of the target person. Thus, play may be determined. The play determination unit 126 may determine the play from one of the movement trajectory of the target person and the positions of the skeleton and the joint.
The play determined by the play determination unit 126 may be, for example, a soccer goal, a line-in or line-out such as tennis or volleyball, a name of a technique such as judo, or the like. Further, the play determination unit 126 may determine, as play determination, success or failure of the play (for example, success / failure of a basketball shot), performance / score (for example, scoring for gymnastics), and the like.

  Further, in each of the above-described embodiments, an example has been described in which the target area is, for example, a court, a stadium, which is a stadium where sports are performed, but the present invention is not limited to this. The target area to which the information processing device 10 (10a) is applied may be, for example, an ice rink such as a skate, a competition place on the snow such as a ski jump or a half pipe, or a pool such as a swimming race or an artistic swimming. , Or other places where competitions are held.

In each of the above embodiments, the calibration processing in which the laser radar device 20 and the imaging device 30 are arranged so that the first position information and the second position information in the target area match each other is as follows. You may go to.
For example, in a competition using a court, the calibration process of the laser radar device 20 and the imaging device 30 may be performed simultaneously by using a colored seal or the like made of a retroreflective material on the court line. In this case, the seal or the like may be attached to a tool such as a net pole or a goal pole whose position is fixed during the competition and used.

Further, the processing of the position measurement of the information processing device 10 (10a) is not limited to each of the above embodiments, and may be changed as follows depending on an event or purpose.
For example, the trajectory generation unit 124 of the information processing device 10 (10a) measures the position by complementing the first position information mainly by the laser radar device 20 with the second position information based on the image information. Then, when used in a wheelchair basketball or a wheelchair rugby competition, the player often holds the ball, and it may be difficult for the laser radar device 20 to detect the first position information. In such a case, the trajectory generation unit 124 may mainly complement the second position information based on the image information with the first position information from the laser radar device 20.

Further, as described above, the laser radar device 20 can accurately detect a position in real time, but since the sampling frequency is low, the position information becomes fragmentary. On the other hand, in position detection based on image information, continuous position information can be detected, but the processing load is high and it takes time. Therefore, when emphasis is placed on real-time position detection, the trajectory generation unit 124 reduces the ratio, frequency, and the like of complementing the first position information mainly by the laser radar device 20 with the second position information based on image information. It may be. In addition, when importance is attached to the accuracy of position detection, for example, the trajectory generation unit 124 may increase the ratio, frequency, and the like, complemented by the second position information based on the image information.
In this way, the trajectory generation unit 124 may change the method of complementing between the first position information and the second position information according to the event or purpose.

  Further, in each of the above embodiments, the information processing device 10 (10a) may convert the generated movement trajectory into various lines of sight and directions and display it on a display unit (not shown). For example, the trajectory generation unit 124 may convert the movement trajectory into a line of sight of a baseball catcher or a soccer keeper, and may display the trajectory. The trajectory may be converted to a trajectory and displayed.

  Further, in each of the above embodiments, the information processing apparatus 10 (10a) can cope with a case where it is desired to acquire the trajectory of a moving object in an indoor competition such as a bicycle track competition or a speed skating with simple equipment. . In addition, in speed measurement on medium and long distances on land, it is difficult to determine only by video, such as when players are overlapping near the corner, and with the conventional technology, calibration is difficult in situations where many cameras are required. is there. Even in such a case, the information processing apparatus 10 (10a) can acquire the movement trajectory with simple equipment.

In addition, by using the information processing apparatus 10 (10a) of each of the above embodiments, the following can be handled in the operation of the news camera or the like.
For example, the camera can be operated so that the player or only the ball is always tracked and photographed. Also, the image processing can be performed so that a ball or shuttle moving at a speed that cannot be followed by human eyes is conspicuous.

  In each of the above embodiments, the information processing device 10 (10a) uses the position information and the movement trajectory to measure the movement trajectory and flight distance in ski jumping, land, swimming, speed skating, and the like. May be performed.

In the second embodiment, the play determination unit 126 may perform the following processing as the play determination from the position information and the movement trajectory, the movement of the skeleton / joint, and the trajectory of the surrounding area.
For example, the play determination unit 126 may notify a change in formation during a set play such as soccer, or may predict a success rate of hitting only with a form such as archery. Further, the play determination unit 126 may estimate the condition and the degree of fatigue only with a form such as archery or weight lifting. The play determination unit 126 may determine the play of a player who does not hold the ball, such as basketball play, volleyball block, and the like, and in athletics, pitch and stride based on movement of the skeleton and joints. May be automatically measured.

  In the second embodiment, the play evaluation unit 127 may predict a success rate of a technique or a game based on a play determination result and a past movement locus. The play evaluation unit 127 may predict, for example, a success rate of a figure skating jump or gymnastics technique. Further, for example, in the weight lifting, the play evaluation unit 127 may predict the success rate from the trajectory of the barbell and the movement of the skeleton / joint. Thereby, the form can be appropriately managed.

  In the second embodiment, the play evaluation unit 127 may visualize the position of the target, the trajectories of the identified various targets, and the evaluation data. The play evaluation section 127 may display a display screen (image G2) as shown in FIG.

FIG. 12 is a diagram illustrating an example of a display screen in a modification of the play determination system 1a according to the second embodiment.
The play evaluation unit 127 causes the display unit (not shown) to display various types of information as described below as a visualization application, as shown in an image G2.
As a visualization application, for example, by specifying one video or a time range in the video, the play evaluation unit 127 displays the video, the position data, and the accompanying evaluation data of the portion. In the image G2, display / non-display of the video, the position data, and the evaluation data can be selected.

  In the visualization application, the play evaluation unit 127 displays the movement of the position of the player, the ball, the racket, or the like in the target area as the position data, and the position data includes the identified identification information (for example, the identification ID). ) Is displayed. In addition, the play evaluation unit 127 includes, as functions of the visualization application, for example, displaying the video and the position data in synchronization with each other, displaying tracking data (tracked movement locus) for each player as evaluation data, and jumping the player. Processing such as displaying statistical data such as the number of times, the height, the maximum speed, and the number of sprints is executed.

  Further, the play evaluation unit 127 includes, as a function of the visualization application, for example, in the case of a speed curve, video and position data, and point data in a graph are displayed in association with each other. Processing such as displaying statistical data, displaying a table as a table in addition to displaying statistics such as graphs, and the like is executed. The play evaluation unit 127 may display equipment data such as a ball and a racket in the evaluation data as a function of the visualization application. For example, the play evaluation unit 127 may display the speed, the number of revolutions, the spin type, and the racket of the ball. Speed, shot type, impact position, shot power data, etc. may be displayed.

Next, another modified example of visualizing the video and the evaluation data in the play determination system 1a will be described with reference to FIG.
FIG. 13 is a diagram showing another modification of the play determination system 1a according to the second embodiment.

  In this figure, the play determination system 1a includes a screen 40 (an example of a display unit) installed above the valley court AR4. The information processing device 10a displays the video imaged by the imaging device 30 on the screen 40, and also displays the evaluation data accompanying the play in a superimposed manner (image G3 in FIG. 13). For example, in the case of a volleyball, the information processing device 10a displays an attack hitting height, a ball speed, a ball angle, and the like as shown in an image G3. In addition, the information processing device 10a allows the user to specify the content and display position of the evaluation data to be superimposed on the video. When measurement is performed using a plurality of imaging devices 30, the information processing device 10 a can specify which imaging device 30 displays an image on the screen 40 by a user. In addition, an example in which the screen 40 is used as an example of the display unit will be described. However, any other display, screen, display device, or the like may be used as long as the display device is installed around the court AR4.

In addition, since the display of the evaluation data requires measurement and determination processing, the video displayed on the screen 40 may be reproduced later than real time. In that case, the information processing apparatus 10a may allow the user to set a delay time for displaying the evaluation data and a time for maintaining the display of the evaluation data. In this case, for example, if the delay time is set to 5 seconds in the attack practice, and a plurality of players attack one after another, it is possible to check their own evaluation data and form on the screen 40 after the attack. it can.
Further, for example, when performing formation practice, the delay time is set to 20 seconds, and a series of plays of receive, toss, and attack can be confirmed on a display such as the screen 40 after play.

  In the second embodiment, the information processing apparatus 10a outputs the evaluation data generated and stored by the play evaluation unit 127 to the outside by text data or a CSV file according to a user's specification. It may be. In this case, the evaluation data includes, for example, the transition of the hit point height of the attack in one game, the number and the success rate of the blocks at the time of the attack, the number and the height of the jumps in the practice in one day, and the like. In the play determination system 1a, by outputting these evaluation data to the outside, an analyst or a coach can efficiently perform a game analysis or a training evaluation from the output data.

Each of the components included in the position tracking system 1 and the play determination system 1a has a computer system therein. Then, a program for realizing the functions of the components provided in the position tracking system 1 and the play determination system 1a is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system. Then, the processing in each configuration of the position tracking system 1 and the play determination system 1a described above may be performed by executing the processing. Here, "to make the computer system read and execute the program recorded on the recording medium" includes installing the program in the computer system. The “computer system” here includes an OS and hardware such as peripheral devices.
Further, the “computer system” may include a plurality of computer devices connected via a network including a communication line such as the Internet, a WAN, a LAN, and a dedicated line. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in a computer system. As described above, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM.

  The recording medium also includes an internal or external recording medium accessible from the distribution server for distributing the program. It should be noted that the program is divided into a plurality of parts, and after being downloaded at different timings, the configuration combined with each configuration provided in the position tracking system 1 and the play determination system 1a, and the distribution server that distributes each of the divided programs is different. You may. Further, the “computer-readable recording medium” holds a program for a certain period of time, such as a volatile memory (RAM) in a computer system serving as a server or a client when the program is transmitted via a network. Shall be included. Further, the program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  Further, some or all of the above functions may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each of the above-described functions may be individually formed into a processor, or a part or all of the functions may be integrated into a processor. The method of circuit integration is not limited to an LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, in the case where a technology for forming an integrated circuit that replaces the LSI appears due to the advance of the semiconductor technology, an integrated circuit based on the technology may be used.

  DESCRIPTION OF SYMBOLS 1 ... Position tracking system, 1a ... Play determination system, 10, 10a ... Information processing device, 11, 11a ... Storage part, 12, 12a ... Control part, 20 ... Laser radar device, 30 ... Imaging device, 40 ... Screen, 111 ... Radar detection storage unit, 112 ... Image information storage unit, 113 ... Object information storage unit, 114 ... Moving trajectory storage unit, 115 ... Object trajectory storage unit, 116 ... Skeletal joint information storage unit, 117 ... Determiner information storage Unit, 118: determination result storage unit, 121: data acquisition unit, 122: image processing unit, 123: identification processing unit, 124: trajectory generation unit, 125: skeleton estimation unit, 126: play determination unit, 127: play evaluation unit

Claims (6)

By irradiating the target area with a wave, based on the reflected wave from the object, a detection device that detects at least detection information including a distance and a direction to the object,
An imaging device that captures an image including the target area;
An acquisition unit that acquires and synchronizes the detection information detected by the detection device and image information representing an image captured by the imaging device,
Corresponding to the position of the object detected based on the detection information obtained by the obtaining unit, based on the partial image in the image information, the object is identified as a measurement target, the identified measurement target object On the other hand, an identification processing unit that gives identification information for identifying the measurement object,
The detection information and the position information of the measurement target in the target area generated based on the image information, and the identification information given by the identification processing unit are associated with each other, and the movement locus of the measurement target is And a trajectory generation unit that generates the position tracking system. The detection information includes a contour of the object,
The identification processing unit extracts the partial image in the image information based on a contour of the object detected by the detection device, and identifies the object as a measurement target based on the extracted partial image. The position tracking system according to claim 1, wherein: The measurement object includes a target person,
The said identification processing part, when identifying the said target person as the said measurement target object, identifies the said target person based on the image recognition of the face or the clothing of the said target person. The said person is characterized by the above-mentioned. 3. The position tracking system according to 2. The identification processing unit, when the target person cannot be identified based on the image recognition of the face or clothes of the target person, estimates the skeleton or joint of the target person based on the partial image, and estimates the estimated skeleton. 4. The position tracking system according to claim 3, wherein the target person is identified based on the joint and a predetermined characteristic of a skeleton or a joint of the target person. 5. The trajectory generation unit is configured to include first position information that is position information of the object in the target area based on the detection information, and second position information that is position information of the measurement target in the target area based on the image information. And the position information of the measurement object is corrected, and the corrected position information of the measurement object is associated with the identification information given by the identification processing unit, and the measurement object is corrected. The position tracking system according to any one of claims 1 to 4, wherein the movement trajectory is generated. By irradiating a wave to the target area, based on a reflected wave from the object, a detection device that detects detection information including at least the distance and direction to the object, and an imaging device that captures an image including the target area A location tracking method for a location tracking system comprising:
An acquisition unit configured to acquire and synchronize the detection information detected by the detection device with image information representing an image captured by the imaging device;
An identification processing unit that identifies the object as a measurement target based on a partial image in the image information corresponding to a position of the object detected based on the detection information acquired in the acquiring step, An identification processing step of assigning identification information for identifying the measurement object to the measurement object,
The trajectory generation unit associates the detection information and the position information of the measurement target in the target area generated based on the image information with the identification information given by the identification processing step, A trajectory generation step of generating a movement trajectory of the measurement target.

Images