JP2019190924A - Motion capturing system, method for capturing motion, marker used in method for capturing motion, and program used in method for capturing motion - Google Patents

Abstract

To provide a motion capturing system in which an imaging device can use not only such a special machine as an infrared camera but also a general machine as a digital camera and also can take an image of a marker even in a dark space.SOLUTION: A motion capture system 1 is a system for recording the motions of a person 3 as a target. Herein, the system uses a plurality of imaging means, which are cameras 5a and 5b, for a three-dimensional measurement. The person 3 is wearing a plurality of markers including markers 7a and 7b. The motion capturing system 1 has an image processing unit 9 for receiving images taken by the cameras 5a and 5b and performing image processing. The image processing unit 9 has a capturing part 11 for capturing motions of the markers 7a and 7b.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a motion capture system, a motion capture method, a marker used in the motion capture method, and a program used in the motion capture method, and more particularly to an optical motion capture system and the like.

  Motion capture technology is a technology that digitally records the movement of an object such as a person or an object. In addition to producing movies that take advantage of this, development is progressing in the sports and medical fields, for example. . Many techniques including improved techniques have been proposed. For example, there are Patent Documents 1, 2, and 3. The techniques of Patent Documents 1, 2, and 3 are included in what is called an optical type.

  In the optical system, in a bright space, light such as infrared rays or LEDs is applied to a marker attached to an object, and the reflected light is received by a camera (a plurality of three-dimensional images) as an imaging device, The segment corresponding to the marker portion is labeled by image processing. Then, after labeling, the coordinates in the marker space are calculated so that the entire locus of the marker, the time-series speed, the maximum speed, and the like can be displayed. Here, in particular, with respect to the markers, including those disclosed in Patent Documents 1 and 2, the so-called optical type has been basically assumed to reflect using infrared rays. Patent Document 3 discloses a device that emits an infrared LED, and a self-luminous LED marker has begun to be provided.

JP 2017-53739 A Japanese Patent Application Laid-Open No. 2014-211404 JP 2003-168115 A

  However, in the conventional optical system, it is basically assumed that reflection by a marker is performed, and the relationship between the position of the device that shines light, the marker that reflects, and the device that receives reflected light is, for example, Very important. In order to make it easy to recognize the marker position, an imaging device (camera) that is an apparatus that receives reflected light may use a dedicated machine such as an infrared camera that can shoot by limiting the frequency of light. There is a problem that it is necessary and there is a great restriction from that point. In order to overcome the problem that the marker is hidden, there is no method other than increasing the number of imaging devices, and a large number of dedicated infrared cameras that cannot be used for other purposes must be prepared. If this dedicated machine is not used, S / N is poor, noise is easily captured, and it is difficult to detect a reflected marker. As a result, imaging analysis cannot be performed accurately. The technique of Patent Document 3 also uses infrared rays, and in any case, it is necessary to use a dedicated machine such as an infrared camera. In addition, there is a restriction that the space needs a bright space. When the space is dark, it is difficult to capture light into the image sensor inside the camera, noise is likely to be included in the captured image, and the recognition accuracy of the light emitting marker is lowered, so that the accuracy of analysis is also lowered. Also, when measuring high-speed motion, the camera's image sensor generally needs to support a high shooting speed (the number of frames taken per second), but the faster the shooting speed, The amount to capture is reduced, making shooting difficult.

  In addition, in the case of the self-light-emitting infrared LED marker as in Patent Document 3, since the LED is used as a light-emitting device that does not emit infrared light, there are the following problems. First, in the case of visible light, an ordinary digital camera recognizes the color and shape, and when an LED is used, the LED becomes a point light source, making it difficult to recognize the shape of the marker. There's a problem. Secondly, regardless of visible light or infrared light, if you try to increase the brightness with the LED, there is a problem that the battery does not last long, and if you increase the battery, it will become heavier by the weight of the battery. .

  Therefore, the present invention has no problem of having a battery even if it emits light regardless of whether it is visible light or infrared light, and particularly for visible light, it is easier to recognize the shape than a self-light emitting LED marker. A motion capture system that can use a general-purpose device of a digital camera instead of a dedicated device such as an infrared camera and can capture a marker even in a dark imaging space, a motion capture method, a marker used in the motion capture method, and An object is to provide a program used in a motion capture method.

  According to a first aspect of the present invention, in the motion capture system capable of capturing the movement of an object to which a marker is attached, the marker is a chemiluminescent marker that emits light spontaneously and can capture light emitted by the marker. An image pickup means and a capture means for capturing the marker by performing image processing from an image picked up by the image pickup means. The light may be visible light or infrared light.

  In a second aspect of the present invention, in the first aspect, the marker is a flat light emitter marker.

  According to a third aspect of the present invention, in the first or second aspect, the marker has a color, a shape, or a combination thereof that can specify a predetermined position in the light emitting portion. is there.

  According to a fourth aspect of the present invention, in any one of the first to third aspects, the marker is a plurality of markers, and the plurality of markers emit light in different colors, shapes, or combinations thereof. The capturing means is characterized in that the marker can be labeled and captured by identifying a difference in the color, shape, or combination thereof.

  According to a fifth aspect of the present invention, in the motion capture method capable of capturing the movement of an object to which the marker is attached, the marker is a chemiluminescent marker that emits light spontaneously, and the imaging means is light emitted from the marker. The image capturing unit captures the marker by performing image processing from the image captured by the image capturing unit, and captures the movement of the object. The light may be visible light or infrared light.

  In a sixth aspect of the present invention, in the fifth aspect, the marker is a flat light emitting marker.

  According to a seventh aspect of the present invention, in the fifth or sixth aspect, a color, a shape, or a combination thereof that can specify a predetermined position in the light emitting portion is provided.

  An eighth aspect of the present invention is a marker used in the motion capture method according to any one of the fifth to seventh aspects.

  According to a ninth aspect of the present invention, the capturing means used in the motion capture method according to any one of the fifth to seventh aspects uses, as a computer, the color difference that the marker emits from the captured image. A program for executing an image processing operation to be labeled.

  According to the present invention, first, the marker is a marker that emits light by chemiluminescence, and does not reflect the marker when it is reflected. Therefore, the device portion that first applies the light is not necessary, and the system can be simplified. In particular, the problem of directivity and irregular reflection can be solved by using a planar light emitting marker as a marker. In particular, the brightness and intensity (for example, luminance) of self-luminous light can be easily set to a necessary level without problems of directivity and irregular reflection by using chemiluminescence technology. It is possible to use a general-purpose device such as a digital camera that can capture an image with a wavelength of visible light instead of such a dedicated device, and it is possible to capture a marker even in a dark imaging space. Here, in chemiluminescence, there are few restrictions on the shape of the marker that emits light (for example, a flat light emitter is possible), and this is of great significance in sports where it is desirable to avoid movement restrictions. Further, unlike LED self-emission, self-emission by chemiluminescence has the advantage that there is no problem with battery life, whether visible or infrared. Furthermore, the visible light has an advantage that the marker shape can be recognized correctly as compared with the self-emission of the LED.

  Further, according to the present invention, since a predetermined position in the light emitting portion of the marker can be specified by color, shape, or a combination thereof, it is possible to improve the accuracy (recognition accuracy) of capturing and capturing. In addition, since a plurality of markers can include those that emit light in different colors, shapes, or combinations thereof, and labeling can be performed by distinguishing differences in colors, shapes, or combinations. The time and burden of processing can be reduced.

It is the block diagram which showed the outline of the motion capture system which concerns on embodiment of this invention. It is a figure for demonstrating specifying the predetermined position in the light emission part of a marker with a color, a shape, or those combination.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited thereto.

  The motion capture system 1 is a system that records the movement of a person 3 as an object. Here, a plurality of imaging means called cameras 5a and 5b are used for three-dimensional measurement. The person 3 includes, for example, a person who is performing a group (team) competition such as badminton, soccer, or baseball in addition to an individual competition in the situation of performing sports. A plurality of markers such as markers 7a and 7b are attached to each person 3 of the object.

  Markers 7a and 7b are chemiluminescent self-luminous markers that can emit different colors by changing the fluorescent dye contained in the chemiluminescent solution, such as marker 7a "blue" and marker 7b "red". It is. In this chemiluminescence, for example, light having a visible light wavelength is generated using techniques such as Japanese Patent Application Laid-Open No. 2006-150956, Japanese Patent Application Laid-Open No. 2006-91823, and Japanese Patent Application Publication No. It is assumed that it can emit light.

  First, since the markers 7a and 7b are self-luminous, even if the cameras 5a and 5b are digital cameras, the color recognition function can be used to capture even a small amount of light, and the marker color can be recognized even in a dark place. Here, when the advantage of using chemiluminescence for self-emission is compared with the case of using a self-emission LED, for example, the following can be mentioned. First, the digital cameras 5a and 5b recognize the color and shape, and when the LED is used, the LED becomes a point light source, and it is difficult to recognize the shape of the markers 7a and 7b. Unlike the case of chemiluminescence, there is a problem that shape recognition becomes difficult. Second, there is a problem that if the LED is used to increase the brightness, the battery will be poor, and if the battery is increased, it will become heavier by the weight of the battery. In particular, the problem of battery life in the case of LED self-light emission is a common problem regardless of whether it is infrared rays or ultraviolet rays. These points do not occur in chemiluminescence, and can be said to be a great merit when considering actual operation. The brightness and intensity of light (for example, luminance) and the duration thereof can be adjusted by chemiluminescence self-emission. For example, in order to increase the recognition of the markers 7a and 7b, the amount of light can be increased even in a short time. It is also possible to do. The chemiluminescent self-luminous markers 7a and 7b capable of controlling such light are such that the brightness of the light is such that the markers 7a and 7b can be properly captured even by a general-purpose machine such as a digital camera. Since the light can be emitted by intensity (for example, luminance), the cameras 5a and 5b are configured to be a system that does not require the use of an infrared camera (dedicated device). In this way, the system can be simplified because a dedicated machine is not required. Further, the distance from the markers 7a and 7b to the cameras 5a and 5b can be set to a distance of several tens of meters depending on the setting, while the maximum distance is about 10 m in the conventional optical system using infrared reflection. As a result, it is possible to take an image from a high position in the sky and measure the movement of the entire team.

  Further, by using chemiluminescence as self-luminescence, the flexibility of the shape of the markers 7a and 7b is increased, and even if the person 3 is a sports player, a shape that does not hinder concentration on the competition is selected. It is supposed to be possible. Here, a marker that reflects infrared rays is basically spherical in terms of reflection efficiency, and a device using an infrared LED as a marker as in Patent Document 3 requires a device and is large.

  Here, the case where the markers 7a and 7b are planar light emitters will be described. As an example of the structure of the flat light emitter, for example, a structure in which a packaging structure having an oxidizing solution portion and a fluorescent solution portion is placed in an outer container that can be stored together with a porous sheet can be cited. With such a structure, the portion of the oxidizing solution and the portion of the fluorescent solution are broken by bending or the like, and both are chemically reacted with the porous sheet to emit light. And as a merit of a planar light-emitting body, the point which is easy to attach to a body is mentioned because it can be made from a soft material, for example. As another merit, it can be said that, for example, a flat light emitter uniformly shines in accordance with its shape, and the degree of freedom of shape is higher than that of a point light emitting LED. In addition, there is a method of attaching a ring (bracelet) -shaped chemiluminescent body to the arm, and there is a merit that the flexibility of responding flexibly to the competition situation and individual requests of the competitor is high.

  In the above description, the system using visible light is used. However, light having an infrared wavelength using chemiluminescence may be self-luminous. In addition, a system using only infrared light may be used instead of a system that always uses visible light. If infrared rays are used in this way, the movement of the athlete's marker can be captured, although not visible to the audience.

  The motion capture system 1 also includes an image processing unit 9. The image processing unit 9 receives images captured by the cameras 5a and 5b and performs image processing. The image processing unit 9 has a capturing unit 11, and the capturing unit 11 captures the movement of the markers 7a and 7b. The capturing process includes a labeling process, and the labeling process is performed by distinguishing the markers using the difference in color between the markers 7a and 7b. This process is performed automatically by a program. Unlike the binarization process using the conventional infrared rays, such a difference in color eliminates the need for post-processing (human processing), and has the great advantage that the labeling process can be automated. Has been obtained. A labeling process is performed, the movements of the markers 7 a and 7 b are captured, and the movement of the person 3 is recorded in the storage unit 13 of the image processing unit 9. In the markerless technique, complex algorithm processing is performed for labeling processing. However, labeling processing using a difference in color is very simple algorithm processing, and it is also added that the advantage is great.

  Here, an example in which a difference in color is used will be specifically described. For example, when capturing the movement of the joint of the elbow for one person, it is possible to automatically identify the attachment sites of the three types of color markers and capture the angle between the three points. For example, in a group sport such as soccer or volleyball, if a color is designated for each player and the positional relationship is automatically recognized, it can be used for game analysis.

  FIG. 2 is a diagram for explaining that a predetermined position in the light emitting portion of the marker is specified by color, shape, or a combination thereof.

  As shown in FIG. 2, the light emitting portion 15 that emits yellow light, for example, of the marker 7 includes, for example, one that can be captured in a square shape, and further divides the inside into four parts, so that the upper left and the right It has a characteristic shape 17 that is shielded so as not to transmit light emitted from below. By adopting this shape 17, even if movement occurs as shown in FIGS. 2 (A), 2 (B), and 2 (C), the center position 19 that is a predetermined position can be easily recognized with high accuracy. It can be captured. Note that if the inner square outline in FIG. 2 is thicker than the outer square outline, the video analysis may be difficult. To facilitate video analysis, the outer large square outline is changed to the inner square outline. It is preferable to make it thicker than the outline.

  Here, it has been described that the difference between the plurality of markers can be specified by the difference in color, but it will be described that the difference between the markers can also be specified by the difference in shape. Suppose that markers of the same color have occurred on the arms and torso. In this case, if the trunk side is hidden by the arm, it is difficult to distinguish the marker by the difference in color, but it can be determined by the difference in shape. That is, although the shape is characterized in FIG. 2, the marker can be distinguished even by the difference in color as described above, and the marker can be distinguished by combining them.

  Although the system has been described above, the present invention may be regarded as a motion capture method.

DESCRIPTION OF SYMBOLS 1 ... Motion capture system, 3 ... Person (object), 5a, 5b ... Camera, 7a, 7b ... Marker, 9 ... Image processing part, 11 ... Capture part, 13 ... Storage unit

Claims (9)

In a motion capture system that can capture the movement of an object with a marker attached,
The marker is a chemiluminescent marker that emits light,
Imaging means capable of imaging light emitted by the marker;
A motion capture system comprising: capture means for capturing the marker by performing image processing from an image captured by the image capturing means.   The motion capture system of claim 1, wherein the marker is a planar illuminator marker.   The motion capture system according to claim 1, wherein the marker has a color, a shape, or a combination thereof that can specify a predetermined position in the light emitting portion. The marker is a plurality of markers, and the plurality of markers include those that emit light in different colors, shapes, or combinations thereof,
The motion capture system according to any one of claims 1 to 3, wherein the capturing unit is capable of labeling and capturing the marker by identifying a difference in the color, shape, or a combination thereof. In a motion capture method that can capture the movement of an object with a marker attached,
The marker is a chemiluminescent marker that emits light,
The imaging means can capture the light emitted by the marker,
A motion capture method, wherein the capture unit captures the marker by performing image processing from an image captured by the imaging unit, and captures the movement of the object.   The motion capture method according to claim 5, wherein the marker is a planar light emitting marker.   The motion capture method according to claim 5, wherein the marker has a color, a shape, or a combination thereof that can specify a predetermined position in the light emitting portion.   A marker used in the motion capture method according to claim 5. The capture means used in the motion capture method according to any one of claims 5 to 7 causes a computer to execute an image processing operation for labeling the marker using a difference in color emitted by the marker from a captured image. ,program.

Images