JP2018022996A - Imaging system and imaging method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging system capable of accurately positioning an imaging device at a predetermined position, and an imaging method.SOLUTION: An imaging system 1 of the present invention comprises: a marker 10 which includes length information and positional information; and a smartphone 20 which has size information of an imaged person 30 and a display unit. By imaging the marker 10, teaching information to the imaged person 30 is displayed on a monitor 21.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging system and an imaging method.

In recent years, there has been a demand for an apparatus for analyzing the movement of a subject in various fields. For example, analyze the subject's exercise form such as a golf club, tennis racket, and baseball bat, such as a swing trajectory of an exercise device, and select an exercise device suitable for the subject and improve the exercise form from the analysis result. It can lead to improvement of competitiveness.
Patent Document 1 discloses a method in which a camera (digital camera or image-equipped mobile phone) is provided with a GPS function and a direction sensing function, and longitude / latitude information and direction information at the time of shooting are recorded. Therefore, since the camera can be installed at the same position and orientation as the previous time, it is possible to obtain images taken at the same position and orientation as the previous time, and by comparing the captured images with the previous time, the movement of the subject You can check the form.

JP 2004-48427 A

  However, in the method described in Patent Document 1 described above, since the height information of the camera and the distance information between the camera and the person to be photographed cannot be understood, the camera and the person to be photographed are accurately and quickly located at the positions where the camera and the person were photographed last time. There was a problem that it could not be placed. Therefore, the angle of the camera with respect to the subject and the size of the subject are different, and the two images cannot be accurately compared.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms or application examples.

  Application Example 1 An imaging system according to this application example includes an index including length information and position information, and an imaging device having size information of a subject and a display unit, and images the index. The display unit displays teaching information for the subject.

  According to this application example, it is possible to acquire length information and position information from an image obtained by photographing an index, and to detect the installation position and orientation of an imaging apparatus that has photographed the index using the index as a reference. For this reason, if there is size information of the photographed person, it is possible to calculate the installation position and orientation of the imaging apparatus in which the photographed person falls within the angle of view of the display unit. Therefore, in order to photograph an exercise form in which the whole body of the photographed person is captured, the installation position and orientation of the imaging device within which the photographed person is within the angle of view of the display section are displayed on the display section as teaching information, thereby The photographer can install the imaging apparatus in the optimum position and posture in a short time.

  Application Example 2 An imaging system according to this application example includes a first imaging device having an index including length information and position information, second relative position information of the index and the second imaging device, and a display unit. And displaying the difference between the first relative position information and the second relative position information of the index and the first imaging device on the display unit by photographing the index.

  According to this application example, the length information and the position information are acquired from the image obtained by photographing the index, and the first relative position information that is the installation position and orientation of the first imaging device that photographed the index using the index as a reference is detected. be able to. Therefore, if there is second relative position information between the index and the second imaging device, the difference between the first relative position information of the first imaging device and the first imaging device is displayed on the display unit, so that the person to be imaged can perform the first imaging in a short time. The apparatus can be installed at a position and posture that matches the second relative position information. Therefore, the first imaging device can capture the same position and posture as the image captured by the second imaging device, and the motion form of the subject can be accurately determined by comparing with the image captured by the second imaging device. Can be checked.

  Application Example 3 In the imaging system according to the application example described above, it is preferable that the size information includes a movable range of the subject.

  According to this application example, since the size information includes the movable range of the photographed person, even if the photographed person is an exercise form having an instrument or the like, the entire body of the photographed person including the instrument or the like is displayed on the display unit. Since it can be taken in the corner, it is possible to check the exercise form including equipment.

  Application Example 4 In the imaging system according to the application example described above, it is preferable that the size information includes posture information.

  According to this application example, since the size information includes the posture information, the imaging device can be installed in an optimum position and posture where the photographed person can check the posture of the photographed person in a short time.

  Application Example 5 In the imaging system according to the application example described above, it is preferable that the relative position information includes angle information of the imaging device.

  According to this application example, since the relative position information includes the angle information of the imaging device, the orientation of the first imaging device can be made the same as the orientation of the second imaging device.

  Application Example 6 In the imaging system according to the application example, it is preferable that the position information is a ball position.

  According to this application example, since the position information is the position of the ball, the distance between the ball and the imaging device can be detected with the position of the ball as a reference.

  Application Example 7 In the imaging system according to the application example described above, it is preferable that the index includes direction information.

  According to this application example, since the direction information is included in the index, the direction in which the imaging device is positioned with respect to the index can be detected.

  Application Example 8 In the imaging system according to the application example described above, it is preferable that the direction information is a flying direction of the ball and includes standing position information of the subject.

  According to this application example, since the direction information is the flying direction of the ball, it is possible to detect the direction in which the imaging apparatus is positioned with respect to the ball or the person to be photographed with reference to the flying direction of the ball. Further, by including the standing position information of the subject in the direction information, it is possible to determine whether the subject is right-handed or left-handed.

  Application Example 9 An imaging method according to this application example includes a step of inputting size information of a subject, a step of installing an index including length information and position information, a step of photographing the index, and a teaching And a step of displaying information.

  According to this application example, it is possible to acquire length information and position information from an image obtained by photographing an index, and to detect the installation position and orientation of an imaging apparatus that has photographed the index using the index as a reference. For this reason, if there is input size information of the photographed person, it is possible to calculate the installation position and orientation of the imaging apparatus in which the photographed person falls within the angle of view of the display unit. Therefore, in order to photograph an exercise form in which the whole body of the photographed person is captured, the installation position and orientation of the imaging device within which the photographed person is within the angle of view of the display section are displayed on the display section as teaching information, thereby The photographer can install the imaging apparatus in the optimum position and posture in a short time.

  Application Example 10 In the imaging method according to the application example, it is preferable that the teaching information includes a moving direction of the imaging device.

  According to this application example, since the moving direction of the imaging device is included in the teaching information, by moving the imaging device along the displayed moving direction of the imaging device, the coverage is within the angle of view of the display unit. The imaging device can be installed in a shorter time at a position and posture where the photographer can be accommodated.

  Application Example 11 An imaging method according to this application example includes a step of inputting an index including length information and position information and second relative position information of the second imaging device, a step of installing the index, A step of photographing the index, and a step of displaying a difference between the first relative position information of the index and the first imaging device and the second relative position information.

  According to this application example, the length information and the position information are acquired from the image obtained by photographing the index, and the first relative position information that is the installation position and orientation of the first imaging device that photographed the index using the index as a reference is detected. be able to. Therefore, the difference between the inputted relative index and the second relative position information of the second imaging device and the first relative position information of the first imaging device is displayed on the display unit, so that the subject can quickly take the first time. One imaging device can be installed at a position and posture that match the second relative position information. Therefore, the first imaging device can capture the same position and posture as the image captured by the second imaging device, and the motion form of the subject can be accurately determined by comparing with the image captured by the second imaging device. Can be checked.

  Application Example 12 In the imaging method according to the application example, it is preferable that the displaying step includes displaying the first relative position information.

  According to this application example, by displaying the first relative position information, it becomes easier to confirm the difference from the second relative position information, and the first imaging apparatus is more in the same position and orientation as the second imaging apparatus. It can be installed in a short time.

  Application Example 13 In the imaging method according to the application example described above, the displaying step includes displaying information that teaches the second relative position information and the first relative position information to coincide with each other. Is preferred.

  According to this application example, the first imaging device is displayed at the same position and orientation as the second imaging device by displaying on the display unit information that teaches that the second relative position information matches the first relative position information. In addition, it can be installed in a shorter time.

The top view explaining the composition of the imaging system concerning a 1st embodiment. 3 is a flowchart illustrating an imaging method according to the first embodiment. The top view which shows the parameter | index which concerns on this embodiment. Illustration of the world coordinate system. The display screen of the imaging device explaining the imaging method which concerns on 1st Embodiment. The display screen of the imaging device explaining the imaging method which concerns on 1st Embodiment. 9 is a flowchart illustrating an imaging method according to a second embodiment. The display screen of the imaging device explaining the imaging method which concerns on 2nd Embodiment. The display screen of the imaging device explaining the imaging method which concerns on 2nd Embodiment. 10 is a flowchart illustrating an imaging method according to a third embodiment. The display screen of the imaging device explaining the imaging method which concerns on 3rd Embodiment. The display screen of the imaging device explaining the imaging method which concerns on 3rd Embodiment. The display screen of the imaging device explaining the imaging method which concerns on 3rd Embodiment. The display screen of the imaging device explaining the imaging method which concerns on 3rd Embodiment.

  Hereinafter, as an embodiment of an imaging system and an imaging method, form shooting such as swing and stance in golf will be described as an example and described with reference to the drawings. Note that the drawings to be used are appropriately enlarged or reduced so that the part to be described can be recognized.

(First embodiment)
<Imaging system>
First, the configuration of the imaging system 1 according to the first embodiment of the present invention will be described with reference to FIG. Note that a smartphone (smartphone) that incorporates a camera will be described as an example of the imaging apparatus according to the present embodiment.
FIG. 1 is a plan view illustrating the configuration of the imaging system according to the first embodiment.

  As shown in FIG. 1, the imaging system 1 according to the present embodiment includes a marker 10 as an index including length information and position information, and a smartphone (smartphone or a data communication function) as an imaging device that captures the marker 10. Imaging terminal) 20. The smartphone 20 is installed behind the ball 36 in the flying direction X, that is, on the side opposite to the flying direction X. In addition, it may be installed on an extension line connecting the person to be photographed 30 and the ball 36, that is, in front of the person to be photographed 30 or obliquely in front of the person to be photographed 30.

  The imaging system 1 is provided in the smartphone 20 by taking length information and position information included in the marker 10 from an image obtained by photographing the marker 10 placed at a golf swing place or the like with a camera provided in the smartphone 20. A computer (CPU) performs a three-dimensional coordinate conversion process on the photographed image and calculates the PnP problem to detect the installation position and orientation of the smartphone 20. Thereafter, based on the size information and posture information of the subject 30 input in advance, an image in which the whole body of the subject 30 is optimally within the angle of view of the monitor 21 (see FIG. 5) as the display unit of the smartphone 20. The installation position and orientation of the smartphone 20 that can shoot the camera 20 are calculated, and the teaching information such as the moving direction of the smartphone 20 is displayed on the monitor 21 so that the optimal installation position and orientation are obtained. Is confirmed, and the smartphone 20 can be installed in an optimal position and posture.

<Imaging method>
Next, the imaging method according to the first embodiment will be described with reference to FIGS.
FIG. 2 is a flowchart illustrating the imaging method according to the first embodiment. In FIG. 2, description will be given by describing the index as a marker and the imaging device as a smartphone. FIG. 3 is a plan view showing an index according to the present embodiment. FIG. 4 is an explanatory diagram of the world coordinate system. 5 and 6 are display screens of the imaging apparatus for explaining the imaging method according to the first embodiment.

  In the imaging method according to the present embodiment, first, in the size information input step (step S1), when shooting the swing state, the golf club 34 is moved from the position of the ball 36 when the golf club 34 is gripped and swung. When photographing the movable range of the subject 30 such as the height dimension of the tip (head) to the highest position or the swing width dimension including the golf club, or the stance state of the subject 30 Posture information such as the height and stance width dimension (interval between both feet) of the person to be photographed 30 is input to the smartphone 20 as size information. Based on the size information including the movable range and posture information of the person to be photographed 30, teaching is performed so that the smartphone 20 can be installed at a position and posture where photographing can be performed so that the whole body of the person to be photographed 30 is within the angle of view of the smartphone 20. be able to.

  Next, in the marker placement step (step S2), the marker 10 is placed at the golf swing location. As shown in FIG. 3, the marker 10 as an index is a square whose shape is a square with a side length ML <b> 1, and a frame-shaped first marker 12 with a blank in the center, and an inner corner of the first marker 12. And a rectangular second marker 14 in contact with the two inner sides of the first marker 12. The length ML1 of one side of the marker 10 as length information is a reference for the length of each coordinate axis in the world coordinate system described later.

  In the marker 10, the position of the ball 36 as the position information (ball position P) is the position of the outer angle that faces the inner angle of the first marker 12 that is in contact with the rectangular second marker 14 and is located closest to the inner angle. Further, the direction passing the ball position P along one side of the marker 10 is the flying direction X of the ball 36 as direction information. The ball position P and the flying ball direction X of the marker 10 serve as a reference for an origin and a coordinate axis in the world coordinate system described later.

  In addition, on the side L1 ′ that passes through the ball position P and faces the side L1 of the marker 10 along the flying ball direction X, the person 30 to be photographed sets the front of the person 30 to be substantially parallel to the flying ball direction X. When standing and taking an address state, if the second marker 14 is located near the left foot side of the subject 30, the subject 30 is determined to be right-handed. When the direction passing through the ball position P in the direction orthogonal to the flying ball direction X is the flying ball direction X ′, the side L2 that passes through the ball position P and faces the one side L2 of the marker 10 in the flying ball direction X ′. When the subject 30 stands on the side with the front side of the subject 30 substantially parallel to the flying ball direction X 'and takes an address state, the second marker 14 is positioned near the right foot side of the subject 30. If there is, the subject 30 is determined to be left-handed.

  Accordingly, the flying ball direction X (X ′) as the direction information indicates whether the subject 30 is right-handed or left-handed depending on where the second marker 14 is located, that is, the flying ball direction X (X ′). On the other hand, it includes standing position information indicating whether the subject 30 is located on the left side or the right side.

  Note that the placement of the marker 10 is such that the ball position P of the marker 10 matches the position where the ball 36 is placed and the one side L1 of the marker 10 matches the flying ball direction X passing through the ball position P at the golf swing location. Hereinafter, a case where the subject 30 is right-handed will be described as an example, and a case where the subject 30 is shot from behind in the flying ball direction X will be described.

  Next, in the smartphone installation step (step S3), as shown in FIG. 1, the smartphone 20 mounted on the tripod 22 or the like for stable installation is rearward in the flying direction X passing the ball position P of the marker 10. In addition, the camera provided in the smartphone 20 is installed facing the marker 10.

  Thereafter, in the marker photographing step (step S4), the marker 10 is photographed by the camera provided in the smartphone 20.

Next, in the calibration process (step S5), an image obtained by photographing the marker 10 is analyzed, and the position and orientation of the smartphone 20 are estimated.
The procedure for estimating the position and orientation of the camera in the three-dimensional space from the two-dimensional image is as follows.

First, the marker 10 is detected from an image obtained by photographing the marker 10. This detection method can be performed using known methods such as template matching, feature amount extraction, and feature point extraction. These methods are generally used in AR (Augmented Reality).
Next, a world coordinate system is defined by the ball position P which is position information included in the marker 10 and the flying ball direction X which is direction information. For example, as shown in FIG. 4, the ball position P is the origin, the flying direction X of the ball 36 is the X axis, the direction orthogonal to the X axis is the Y axis, and the vertical direction orthogonal to the X axis and the Y axis is the Z axis. . Further, the length in one side, ML1, which is length information included in the marker 10, defines lengths in the X-axis direction, the Y-axis direction, and the Z-axis direction.

  Thereafter, a transformation matrix for transforming from the camera coordinate system (a coordinate system centered on the camera) to the world coordinate system is obtained. This transformation matrix is generally called an “external parameter or extrinsic parameter”. This series of calculations is called a PnP problem (Perspective n-Point Problem), and how to solve this problem is also a known technique. This calculation can be performed using a function called “sv :: solvePnP ()” implemented in OpenCV, which is an open source software library. The size, pattern, and color of the marker are determined in advance, and the coordinate value may be determined by the system recognizing the marker in the image.

In calculating external parameters, internal parameters (internal parameters or intrinsic parameters) are used. The internal parameter is a parameter for determining how information in the external three-dimensional space is projected onto the two-dimensional image sensor, and is information specific to the camera. Some models of the smartphone 20 have a function (API) for acquiring an internal parameter. In the model, the internal parameter can be acquired using this acquisition function. Therefore, the calculation described above is processed by a computer (CPU) provided in the smartphone 20.
With the above procedure, the position and orientation of the camera of the smartphone 20 in the three-dimensional space can be estimated. The camera posture means a rotation angle with the X axis of the camera as the rotation axis, a rotation angle with the Y axis as the rotation axis, and a rotation angle with the Z axis as the rotation axis. This is the reference position.

Thereafter, in the silhouette video display step (step S6), as shown in FIG. 5, the marker 10 photographed on the monitor 21 as the display unit of the smartphone 20 at the current position and posture of the smartphone 20, and the subject A silhouette image 40 assuming 30 is displayed.
This is because the current position and orientation of the smartphone 20 can be calculated in step S5, so that the marker 10 is predetermined based on the current position and orientation information of the smartphone 20 and the size information of the subject 30 in step S1. This is because it is possible to calculate the video when the subject 30 (silhouette video 40) is placed at the position of. Furthermore, by calculating the PnP problem, it is also possible to calculate the installation position and posture of the smartphone 20 that can capture an image in which the entire body of the subject 30 falls within the angle of view of the monitor 21 of the smartphone 20.
In FIG. 5, only the lower half of the silhouette image 40 is shown, which means that the camera position and posture of the smartphone 20 are closer to the installation position and posture where the whole body of the subject 30 can be photographed.

  Next, in step S7, it is determined whether the whole body of the silhouette image 40 is within the angle of view of the monitor 21, and as shown in FIG. 5, in the case of “No”, the smartphone installation position teaching step (step S8). Proceed to

  In the smartphone installation position teaching step (step S8), the current position and orientation information of the smartphone 20 calculated in the calibration step (step S5) and the image of the entire body of the subject 30 within the angle of view of the monitor 21 of the smartphone 20 From the installation position and posture information of the smartphone 20 that can be photographed, teaching information such as the moving direction and angle of the smartphone 20 is displayed on the monitor 21 so that the current position and posture of the smartphone 20 become the optimum installation position and posture. To do. In the case of FIG. 5, since the camera position is close, for example, characters such as “retract the camera by 1 m” are displayed on the monitor 21. Moreover, the method of teaching by voice from the smartphone 20 may be used.

  Next, in the smartphone installation position moving step (step S9), the subject 30 moves the installation position of the smartphone 20 according to the teaching information on the monitor 21 or the teaching information by voice.

  After moving the smartphone 20, the process returns to step S4 and the marker 10 is photographed again. In step S5, the position and posture of the smartphone 20 after movement are calculated, and in step S6, the silhouette image 40 at the position and posture of the smartphone 20 after movement is displayed. In step S7, it is determined whether the silhouette image 40 is accommodated in the monitor 21, and if not ("No"), the smartphone 20 is determined in step S9 according to the installation position and posture teaching information in step S8. Therefore, steps S4 to S9 are repeatedly performed until the silhouette image 40 fits in the monitor 21.

  After that, in step S6, as shown in FIG. 6, when the whole body of the silhouette video 40 is displayed in the monitor 21, it is determined “Yes” in step S7, and the process proceeds to the smartphone installation position fixing step (step S10). .

  In the smartphone installation position fixing step (step S <b> 10), the smartphone 20 is installed as an optimal position and posture in which the subject 30 falls within the angle of view of the monitor 21.

As described above, according to the imaging system 1 and the imaging method of the present embodiment, the following effects can be obtained.
Length information and position information is acquired from an image obtained by photographing the marker 10, and a three-dimensional coordinate conversion process is performed with the marker 10 as a reference to calculate a PnP problem. Can be detected. Therefore, if there is size information of the person 30 to be photographed, the installation position and posture of the smartphone 20 in which the person 30 is within the angle of view of the monitor 21 can be calculated. Therefore, in order to photograph an exercise form in which the whole body of the subject 30 is reflected, the installation position and posture of the smartphone 20 where the subject 30 falls within the angle of view of the monitor 21 are displayed on the monitor 21 as teaching information. The to-be-photographed person 30 can install the smartphone 20 in an optimal position and posture in a short time.

  In addition, since the size information includes the movable range of the subject 30, the whole body of the subject 30 including the golf club 34 is displayed on the monitor 21 even if the subject 30 is a swing form in which the golf club 34 is held. Since it is possible to shoot within the angle of view, it is possible to check an exercise form such as a swing including the golf club 34.

  Further, since the position information is the ball position P, the distance between the ball 36 and the smartphone 20 can be detected with the ball position P as a reference.

  Further, since the direction information included in the marker 10 is the flying direction X of the ball 36, it is possible to detect the direction in which the smartphone 20 is positioned with respect to the ball 36 and the subject 30 with reference to the flying direction X of the ball 36. it can. Further, since the position information of the subject 30 is included in the direction information, it can be determined whether the subject 30 is right-handed or left-handed.

(Second Embodiment)
Next, an imaging method according to the second embodiment will be described with reference to FIGS.
FIG. 7 is a flowchart illustrating an imaging method according to the second embodiment. In FIG. 7, as in FIG. 2, the index is described as a marker, and the imaging device is described as a smartphone. FIG. 8 and FIG. 9 are display screens of the imaging apparatus for explaining the imaging method according to the second embodiment.

  In the imaging method according to the second embodiment, in the imaging system 1 described above, posture information is input as size information, and the posture information and the person to be photographed 30 are displayed on the monitor 21, so that the smartphone 20 is installed at the optimal position. It is a method to do.

  First, in the posture information input step (step S1), as posture information, for example, the stance length that is the distance between the ball position P and the subject 30 in the stance state of the subject 30, the waist height of the subject 30 and the subject A dimension such as a stance width that is an interval between both feet of the photographer 30 is input to the smartphone 20.

  Thereafter, the marker 10 and the person to be photographed 30 are photographed simultaneously in the marker and photographed person photographing process (step S4) through the marker placement process (step S2) and the smartphone installation process (step S3).

  Next, as in the first embodiment, in the calibration process (step S5), an image of the marker 10 and the subject 30 is analyzed, and the position and orientation of the smartphone 20 are calculated. Further, the posture information of the subject 30 in the current position and posture information of the smartphone 20 is calculated from the current position and posture information of the smartphone 20 and the posture information of the subject 30 input in step S1. Furthermore, the installation position and posture of the smartphone 20 in which the posture and posture information of the subject 30 in the image match are calculated.

  In the posture information display step (step S6), as shown in FIG. 8 or FIG. 9, the posture information assumed based on the current position and posture information of the smartphone 20 in the image obtained by photographing the marker 10 and the person 30. Is displayed on the monitor 21. FIG. 8 shows an image captured from behind the flying ball direction X with the stance length as posture information indicated by a line 42 and the waist height indicated by a line 44. FIG. 9 is an image obtained by displaying the waist height, which is posture information, as a line 44 and the stance width as a line 46 in an image taken from the front of the subject 30.

  Thereafter, in step S7, it is determined whether the posture (for example, waist height, stance length, stance width, etc.) of the subject 30 matches the posture information. If it is determined that they match, the process proceeds to the smartphone installation position fixing step (step S10) as “Yes”, and the smartphone 20 is installed at the current position and posture. If it is determined that they do not match, “No” is determined and the process proceeds to the smartphone installation position teaching step (step S8).

  In the smartphone installation position teaching step (step S8), the current position and posture of the smartphone 20 calculated in the calibration step (step S5) and the posture and posture information of the subject 30 in the image match. The position and orientation information of the installation position and posture, and the difference between them are displayed on the monitor 21 with characters or the like, thereby teaching the smartphone 20 to have an optimum position and posture. Note that the teaching method is not limited to the display on the monitor 21, and a method of teaching by voice from the smartphone 20 may be used.

  Next, in the smartphone installation position moving step (step S9), the subject 30 moves the installation position of the smartphone 20 according to the teaching information on the monitor 21 or the teaching information by voice.

  After moving the smartphone 20, the process returns to step S4, the marker 10 and the subject 30 are photographed again, and steps S4 to S9 are performed until the posture of the subject 30 displayed on the monitor 21 matches the posture information. Repeatedly.

  It should be noted that it is very difficult for the subject 30 to photograph the marker 10 and the subject 30 alone in the marker and subject-photographing step (step S4). Therefore, as a method of photographing the marker 10 and the subject 30 together by one person, the timer function of the smartphone 20 is used to set the camera so that the camera shutter is released after several seconds, or the subject 30 utters “shooting” For example, the smartphone 20 may receive a voice such as “Start”, and the camera shutter may be released after several seconds.

As described above, according to the imaging method of the second embodiment, the following effects can be obtained.
Based on the image of the marker 10 and the person 30 to be photographed, with the marker 10 as a reference, the current position and posture of the smartphone 20 and the installation position and posture of the smartphone 20 in which the posture and posture information of the person 30 to be photographed coincide with each other. , Can be detected. For this reason, as the teaching information, the installation position and posture of the smartphone 20 in which the posture and posture information of the subject 30 coincide with each other are displayed on the monitor 21 of the smartphone 20, so The smartphone 20 can be installed in an optimum position and posture in which it can be checked in a short time.

(Third embodiment)
Next, an imaging method according to the third embodiment will be described with reference to FIGS.
FIG. 10 is a flowchart illustrating an imaging method according to the third embodiment. In FIG. 10, as in FIG. 2, the index is described as a marker, and the imaging device is described as a smartphone. FIG. 11 to FIG. 14 are display screens of the imaging apparatus for explaining the imaging method according to the third embodiment.

  In the imaging method according to the third embodiment, in the imaging system 1 described above, the installation position and orientation of the camera are detected from the image data of the smartphone 20 that was previously captured and the image data that is captured by a smartphone or video camera different from the smartphone 20. By displaying the difference between the current position and posture of the smartphone 20 on the monitor 21, the smartphone 20 can be installed in the position and posture of the previously photographed smartphone 20 or the camera of a different smartphone or video camera in a short time. It can be done.

In the present embodiment, the smartphone 20 is the first imaging device, the first relative position information is the position and posture of the smartphone 20, and the video camera taken for comparing the exercise forms is the second imaging device. The second relative position information will be described as the position and orientation of the video camera.
Each relative position information includes camera position and posture information. As described above, the posture information includes a rotation angle around the X axis, a rotation angle around the Y axis, and a rotation angle around the Z axis. Contains angle information.

  First, in the second relative position information input step (step S1), the image data of the video camera in which the marker 10 and the motion form are photographed are taken into a personal computer or the like, and the PnP problem is calculated from the length information and position information of the marker 10. Detecting the installation position and orientation of the video camera. The installation position and orientation of the video camera are input to the smartphone 20 as second relative position information.

  Thereafter, the process proceeds to the calibration step (step S5) through the marker placement step (step S2), the smartphone installation step (step S3), and the marker photographing step (step S4).

  In the calibration step (step S5), as in the first embodiment, an image obtained by photographing the marker 10 is analyzed, and the position and orientation of the smartphone 20 as first relative position information are calculated. Also, the difference between the position and orientation of the smartphone 20 as the calculated first relative position information and the installation position and orientation of the video camera as the second relative position information is calculated.

  In the relative position information display step (step S6), the difference between the calculated position and orientation of the smartphone 20 as the first relative position information and the installation position and orientation of the video camera as the second relative position information is monitored by characters 21 To display. Further, the first relative position information and the second relative position information may be displayed, or information that teaches the second relative position information and the first relative position information to match may be displayed.

  Thereafter, in step S7, it is determined whether the first relative position information and the second relative position information match. If it is determined that the first relative position information and the second relative position information match, the process proceeds to the smartphone installation position fixing step (step S10) as “Yes”, and the smartphone 20 is installed at the current position and posture. To do. If it is determined that they do not match, “No” is determined and the process proceeds to the smartphone installation position teaching step (step S8).

  In the smartphone installation position teaching step (step S8), as shown in FIG. 11, the second relative position information is displayed as a solid circle on the image obtained by photographing the marker 10, and the first relative position information is displayed. By displaying the display mark 52 as a dotted circle, the direction in which the smartphone 20 is moved is taught. The size of the diameter of the circle of the display mark 52 and the size of the diameter of the display mark 52 which is the current position and posture of the smartphone 20 represent perspective, and the position and posture of the video camera for which the diameter of the display mark 52 is a target. Is larger than the diameter of the display mark 50 indicating that the position of the smartphone 20 is too close to the marker 10 side. The display position of the display mark 52 represents a difference from the target installation position (display mark 50). When the display mark 52 is displayed on the left side of the display mark 50, the position of the smartphone 20 is the target installation position (display mark 50). In contrast to this, it is shifted to the left. Therefore, as shown in FIG. 12, the subject 30 moves the smartphone 20 so that the diameter and the position of the display mark 52 overlap the display mark 50, thereby moving the smartphone 20 to the position where the video camera is installed and Can be placed and fixed in posture.

  As a teaching method, as shown in FIG. 13, a display mark 54 by an arrow may be displayed so that the first relative position information matches the second relative position information. When the smartphone 20 is moved in the direction of the arrow of the display mark 54 and the “OK” display mark 56 is displayed on the monitor 21 as shown in FIG. 14, the position of the smartphone 20 is the target installation position (display mark 50 ), The smartphone 20 can be placed and fixed at the position and posture where the video camera was installed.

  Furthermore, as a teaching method, in addition to the above, a method of teaching a moving direction, a moving distance, an angle, etc., which are teaching information from the smartphone 20 by voice may be used.

As described above, according to the imaging method of the third embodiment, the following effects can be obtained.
Length information and position information is acquired from an image obtained by photographing the marker 10, and a three-dimensional coordinate conversion process is performed with the marker 10 as a reference to calculate a PnP problem. It is possible to detect the first relative position information. Therefore, the difference between the second relative position information of the marker 10 and the video camera and the first relative position information of the smartphone 20 is displayed on the monitor 21, so that the person to be photographed 30 can move the smartphone 20 to the second position in a short time. It can be installed at a position and posture that match the relative position information. Therefore, the smartphone 20 can be photographed at the same position and posture as the image photographed by the video camera. By comparing with the image photographed by the video camera, the exercise form such as the golf swing of the subject 30 can be accurately checked. can do.

  Further, since the relative position information includes angle information of the smartphone 20 and the video camera, the attitude of the smartphone 20 can be made the same as the attitude of the video camera. Therefore, it can be accurately compared with an image taken with a video camera.

  Also, displaying the first relative position information on the monitor 21 makes it easier to confirm the difference from the second relative position information, so that the smartphone 20 can be installed at the same position and posture as the video camera in a shorter time. it can.

  In addition, by displaying on the monitor 21 information that teaches that the second relative position information and the first relative position information match, the smartphone 20 can be installed in the same position and orientation as the video camera in a shorter time. Can do.

  The above-described imaging system and imaging method according to the present invention can be applied to sports in which exercise form is important, such as baseball batting form and batting stance, or tennis swing form and swing stance.

  DESCRIPTION OF SYMBOLS 1 ... Imaging system, 10 ... Marker as index, 12 ... 1st marker, 14 ... 2nd marker, 20 ... Smartphone as an imaging device, 21 ... Monitor as display part, 22 ... Tripod, 30 ... Subject, 34 ... Golf club, 36 ... Ball, 40 ... Silhouette video, 42, 44, 46 ... Line, 50, 52, 54, 56 ... Display mark.

Claims (13)

An indicator including length information and position information;
An imaging device having size information of the subject and a display unit,
An imaging system characterized by displaying the teaching information for the subject to be photographed on the display unit by photographing the index. An indicator including length information and position information;
A first imaging device having second relative position information of the index and the second imaging device and a display unit,
The imaging system, wherein the difference between the first relative position information and the second relative position information between the index and the first imaging device is displayed on the display unit by photographing the index.   The imaging system according to claim 1, wherein the size information includes a movable range of the subject.   The imaging system according to claim 1, wherein the size information includes posture information.   The imaging system according to claim 2, wherein the relative position information includes angle information of the imaging device.   The imaging system according to any one of claims 1 to 5, wherein the position information is a position of a ball.   The imaging system according to any one of claims 1 to 6, wherein the index includes direction information.   The imaging system according to claim 7, wherein the direction information is a flying direction of the ball and includes standing position information of the subject. Inputting the size information of the subject,
Installing an index including length information and position information;
Photographing the index;
Displaying the teaching information;
An imaging method comprising:   The imaging method according to claim 9, wherein the teaching information includes a moving direction of the imaging device. Inputting the second relative position information between the index including the length information and the position information and the second imaging device;
Installing the indicator;
Photographing the index;
Displaying a difference between the first relative position information of the index and the first imaging device and the second relative position information;
An imaging method comprising:   The imaging method according to claim 11, wherein the displaying includes displaying the first relative position information.   13. The imaging according to claim 11, wherein the displaying step includes displaying information that teaches that the second relative position information and the first relative position information match. Method.

Images