JP6787072B2 - Image processing equipment, analysis system, image processing method and program - Google Patents

Description

The present invention relates to an image processing apparatus, an analysis system, an image processing method and a program.

Conventionally, various methods for analyzing golf swings by images have been developed.
For example, Patent Document 1 proposes to evaluate swing motion by reproducing measurement data obtained by an acceleration sensor attached to the lumbar region together with a corresponding moving image.
However, the technique described in Patent Document 1 requires large-scale equipment such as a laboratory, and it is difficult for general users to easily utilize it.
On the other hand, in recent years, terminals such as high-speed digital cameras for consumers, smartphones, and motion sensors have become easily available, and general users bring these terminals to the practice area and use the functions of each terminal in combination. , It is also possible to easily analyze the golf swing.

Japanese Unexamined Patent Publication No. 2009-050721

However, when terminals such as high-speed digital cameras for consumers, smartphones, and motion sensors are used in combination, for example, although it is possible to acquire information from each terminal, there is a function of presenting analysis results from the acquired information. There was a problem that it was not enough. Therefore, it takes unnecessary time and effort for the user to grasp the operation content after performing an operation such as a swing, and efficient display is not possible.

The present invention has been made in view of such a situation, and an object of the present invention is to display the movement of the target person so that the user can grasp it more efficiently.

In order to achieve the above object, the image processing apparatus of one aspect of the present invention is
Image acquisition means for acquiring captured images taken by the subject,
A measurement result acquisition means for acquiring a measurement result obtained by measuring the movement of the target person with a sensor,
A first display control means for displaying the captured image acquired by the image acquisition means on the display means, and
A simple display image generation means for generating a simple display image relating to the movement of the target person, which is different from the captured image, based on the measurement result acquired by the measurement result acquisition means.
A second display control means for causing the display means to display the simple display image generated by the simple display image generation means prior to the display of the captured image by the first display control means .
The measurement result acquisition process by the measurement result acquisition means is characterized in that the processing time is shorter than the acquisition process of the captured image by the image acquisition means .

According to the present invention, it is possible to display the movement of the target person so that the user can grasp it more efficiently.

It is a system block diagram which shows the structure of the analysis system which concerns on one Embodiment of this invention. It is a schematic diagram which shows the example of use form of an analysis system. It is a block diagram which shows the hardware structure of a sensor unit. It is a block diagram which shows the hardware composition of a photographing apparatus. It is a block diagram which shows the hardware configuration of a processing apparatus. It is a functional block diagram which shows the functional configuration for executing information detection processing among the functional configurations of a sensor unit. It is a schematic diagram which shows the mounting state of a sensor unit. It is a schematic diagram which shows the example of the information detected by a sensor unit. It is a functional block diagram which shows the functional structure for executing a shooting process among the functional configurations of a shooting apparatus. It is a functional block diagram which shows the functional configuration for executing the analysis result display processing among the functional configurations of a processing apparatus. It is a schematic diagram which shows the display example in a real-time mode. It is a schematic diagram which shows the display example in a one-shot mode. It is a schematic diagram which shows the display example of the simple display image of the characteristic point in a golf swing. It is a flowchart explaining the flow of the information detection process executed by a sensor unit. It is a flowchart explaining the flow of the photographing process executed by the photographing apparatus. It is a flowchart explaining the flow of analysis result display processing executed by a processing apparatus. It is a schematic diagram which shows schematic the timing of the processing between each apparatus in a one-shot mode. It is a schematic diagram which shows the display form example of a BOX animation.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[System configuration]
FIG. 1 is a system configuration diagram showing a configuration of an analysis system S according to an embodiment of the present invention. Further, FIG. 2 is a schematic diagram showing an example of usage of the analysis system S.
As shown in FIGS. 1 and 2, the analysis system S includes a sensor unit 1, a photographing device 2, and a processing device 3. Further, the processing device 3, the sensor unit 1, and the photographing device 2 are configured to be communicable by Bluetooth low energy / Bluetooth LE (trademark) (hereinafter, referred to as “BLE”), and the photographing device 2, the sensor unit 1, and the processing are configured. The device 3 is configured to be communicable by Wi-Fi (Wireless Fieldy).
The sensor unit 1 is attached to the measurement target, senses the movement of the measurement target, and transmits the sensor information to the processing device 3. In the present embodiment, the sensor unit 1 is attached to the waist or the like of a person who performs a golf swing motion (hereinafter, referred to as “measurement target person P”), and the motion is sensed.

The photographing device 2 photographs the measurement target and transmits the moving image data showing the movement of the measurement target to the processing device 3. In the present embodiment, the photographing device 2 captures a moving image of the measurement target person P performing the golf swing operation.

The processing device 3 analyzes the sensor information acquired from the sensor unit 1 mounted on the measurement target, and measures the analysis result (in the present embodiment, the posture change and the movement state of the measurement target) from which the sensor information is acquired. It is displayed together with a video showing the movement of the target. In the present embodiment, the processing device 3 animates the analysis result of the measurement target with a box-shaped object (an index showing the three-dimensional movement of the measurement target) schematically showing the three-dimensional movement of the measurement target. indicate. Further, the processing device 3 displays an image that roughly represents the movement of the measurement target based on the sensor information until the analysis result of the measurement target is displayed.

[Hardware configuration]
FIG. 3 is a block diagram showing a hardware configuration of the sensor unit 1.
The sensor unit 1 is configured as a device including various sensors for detecting the movement of the measurement target.
As shown in FIG. 3, the sensor unit 1 includes a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, a bus 114, an input / output interface 115, and a sensor. It includes a unit 116, an input unit 117, an output unit 118, a storage unit 119, and a communication unit 120. The sensor unit 1 may be configured so that a removable medium made of a semiconductor memory or the like can be attached.

The CPU 111 executes various processes according to the program recorded in the ROM 112 or the program loaded from the storage unit 119 into the RAM 113.

Data and the like necessary for the CPU 111 to execute various processes are also appropriately stored in the RAM 113.

The CPU 111, ROM 112, and RAM 113 are connected to each other via the bus 114. An input / output interface 115 is also connected to the bus 114. A sensor unit 116, an input unit 117, an output unit 118, a storage unit 119, and a communication unit 120 are connected to the input / output interface 115.

The sensor unit 116 includes a 3-axis acceleration sensor that measures acceleration in the 3-axis direction, a 3-axis angular velocity sensor that measures the angular velocity in the 3-axis direction, and a 3-axis geomagnetic sensor that measures the geomagnetism in the 3-axis direction. .. The sensor unit 116 measures acceleration, angular velocity, and geomagnetism in the three-axis direction by a three-axis acceleration sensor, a three-axis angular velocity sensor, and a three-axis geomagnetic sensor every preset sampling cycle (for example, 0.001 second). The acceleration and angular velocity data measured by the sensor unit 116 are stored in the storage unit 119 or transmitted to the processing device 3 in association with the measurement time data.

The input unit 117 is composed of various buttons and the like, and inputs various information according to a user's instruction operation.
The output unit 118 is composed of a lamp, a speaker, a vibration motor, or the like, and outputs light, audio, or a vibration signal.
The storage unit 119 is composed of a semiconductor memory such as a DRAM (Dynamic Random Access Memory) and stores various data.
The communication unit 120 controls communication with other devices by direct wireless communication between terminals. In the present embodiment, the communication unit 120 communicates with the processing device 3 by BLE (registered trademark) and communicates with the photographing device 2 by Wi-Fi.

FIG. 4 is a block diagram showing a hardware configuration of the photographing device 2.
The photographing device 2 has an image capturing function, and is composed of, for example, a digital camera. In the present embodiment, the photographing device 2 is configured as a device in which the photographing function is unitized separately from the display.
As shown in FIG. 4, the photographing apparatus 2 includes a CPU 211, a ROM 212, a RAM 213, a bus 214, an input / output interface 215, an imaging unit 216, an input unit 217, an output unit 218, and a storage unit 219. A communication unit 220 and a drive 221 are provided.
Of these, the configurations other than the imaging unit 216, the communication unit 220, and the drive 221 are the same as the corresponding parts in FIG.

Although not shown, the image pickup unit 216 includes an optical lens unit and an image sensor.

The optical lens unit is composed of a lens that collects light, such as a focus lens or a zoom lens, in order to photograph a subject.
The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor. A zoom lens is a lens that freely changes the focal length within a certain range.
The optical lens unit is also provided with peripheral circuits for adjusting setting parameters such as focus, exposure, and white balance, if necessary.

The image sensor is composed of a photoelectric conversion element, an AFE (Analog Front End), and the like.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element performs photoelectric conversion (imaging) of the subject image, accumulates an image signal for a certain period of time, and sequentially supplies the accumulated image signal as an analog signal to AFE.
The AFE executes various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. A digital signal is generated by various signal processing and is output as an output signal of the imaging unit 216.
Such an output signal of the image pickup unit 216 is hereinafter referred to as "captured image data". The captured image data is appropriately supplied to the CPU 211 and the like.

The communication unit 220 controls communication with other devices by direct wireless communication between terminals. In the present embodiment, the communication unit 220 communicates with the processing device 3 by BLE (registered trademark) and communicates with the photographing device 2 and the processing device 3 by Wi-Fi.
A removable media 231 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted on the drive 221. The program read from the removable media 231 by the drive 221 is installed in the storage unit 219 as needed. Further, the removable media 231 can also store various data such as image data stored in the storage unit 219 in the same manner as the storage unit 219.

FIG. 5 is a block diagram showing a hardware configuration of the processing device 3.
The processing device 3 is an information processing device having an information display function, and is configured as, for example, a smartphone.

As shown in FIG. 5, the processing device 3 includes a CPU 311, a ROM 312, a RAM 313, a bus 314, an input / output interface 315, an image pickup unit 316, a sensor unit 317, an input unit 318, and an output unit 319. A storage unit 320, a communication unit 321 and a drive 322 are provided. A removable media 331 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted on the drive 322.
Of these, the configurations other than the input unit 318, the output unit 319, and the communication unit 321 are the same as the corresponding parts in FIGS. 3 and 4.

The input unit 318 is composed of various buttons, a touch panel, and the like, and inputs various information according to a user's instruction operation.
The output unit 319 is composed of a display, a speaker, or the like, and outputs an image or sound.
The communication unit 321 controls communication with other devices (not shown) via a network including the Internet. In addition, the communication unit 321 controls communication with other devices by direct wireless communication between terminals. In the present embodiment, the communication unit 321 communicates with the sensor unit 1 and the photographing device 2 by BLE (registered trademark), and communicates with the photographing device 2 by Wi-Fi.

[Functional configuration]
FIG. 6 is a functional block diagram showing a functional configuration for executing information detection processing among the functional configurations of the sensor unit 1.
The information detection process is a series of processes in which the sensor unit 1 mounted on the measurement target senses the movement of the measurement target when the analysis system S analyzes the movement of the measurement target.
When the information detection process is executed, as shown in FIG. 6, in the CPU 111, the first communication control unit 151, the second communication control unit 152, the calibration execution unit 153, the mode setting unit 154, and the synchronous control The unit 155, the detection processing unit 156, and the sensor information transmission control unit 157 function.
Further, a sensor information storage unit 171 is set in one area of the storage unit 119.

The first communication control unit 151 controls communication by the BLE of the sensor unit 1 and executes pairing processing with other devices and data transmission / reception processing.
The second communication control unit 152 controls communication by Wi-Fi of the sensor unit 1 and executes data transmission / reception processing with another device.

The calibration execution unit 153 acquires sensor information in a reference state in response to an instruction from the processing device 3, and executes calibration using the acquired result as a reference value. In the present embodiment, the calibration execution unit 153 is based on a state in which the sensor unit 1 is attached to the waist of the measurement target person P and takes a posture addressed to a position for performing a golf swing, and in this state. The acquired sensor information is used as a reference value.

FIG. 7A is a schematic diagram showing a mounted state of the sensor unit 1, and FIG. 7B is a schematic diagram showing an example of information detected by the sensor unit 1.
When calibration is performed, as shown in FIG. 7A, for example, the sensor unit 1 is attached to the waist of the measurement target person P by a belt or the like, and the measurement target person P is placed on the ball at a position for performing a golf swing. Rest for a predetermined time (for example, 2 seconds) in the opposite posture. At this time, as shown in FIG. 7B, the sensor unit 1 has sensor information (tilt angle in the front-rear direction of the body, based on the gravity direction detected by the 3-axis acceleration sensor and the orientation detected by the 3-axis geomagnetic sensor. The tilt angle in the left-right direction and the rotation angle around the vertical axis) are acquired, and each acquired sensor information is used as a reference value at the time of addressing. After that, when the measurement target person P swings, each sensor information detected in the sensor unit 1 is set for each sensor information with respect to the reference value at the time of addressing for a predetermined time (for example, 2 seconds) or more. When it is within the range of the threshold value, it is detected as the state of the address.
In addition to the information shown in FIG. 7B, since the sensor unit 1 is provided with a 3-axis acceleration sensor, it is possible to detect movement (translation, etc.) of the sensor unit 1 and is provided with a 3-axis geomagnetic sensor. Therefore, it is possible to detect the direction in which the sensor unit 1 is facing.

The mode setting unit 154 sets the operation mode to one of a plurality of prepared modes in response to an instruction from the processing device 3. In the present embodiment, the analysis system S has a real-time mode in which an animation generated from sensor information acquired by the sensor unit 1 is displayed together with a live view image of the measurement target person P, and a golf swing of the measurement target person P. There is a one-shot mode that synthesizes and displays the animation generated from the sensor information acquired by the sensor unit 1 in the swing.

After being set to the one-shot mode, the synchronization control unit 155 synchronizes the reference time in the sensor unit 1 and the photographing device 2 via communication by Wi-Fi. The sensor unit 1 and the photographing device 2 can synchronize the reference time by, for example, Simple Network Time Protocol (SNTP).

The detection processing unit 156 sequentially acquires various sensor information when the sensor unit 1 is set to the real-time mode and the one-shot mode, and corresponds the acquired sensor information with the acquired time. It is attached and stored in the sensor information storage unit 171. The sensor information stored in the sensor information storage unit 171 may be sequentially discarded when a predetermined time has elapsed from the acquired time. However, in the one-shot mode, the sensor information acquired from the golf swing address to the end of the swing is discarded at least after the transmission to the processing device 3 is completed.
Further, the detection processing unit 156 detects the timing of a predetermined characteristic point in the golf swing based on the sensor information acquired in the one-shot mode. For example, the detection processing unit 156 analyzes the waveform of the sensor information acquired in the one-shot mode, and detects the timing of each point of the address, top, downswing, impact, and follow in the golf swing.

Further, when the detection processing unit 156 detects the address timing and the follow timing in the one-shot mode, the detection processing unit 156 refers to a signal indicating that the address timing has been detected (hereinafter, referred to as an “address detection signal”. ) And a signal indicating that the follow timing has been detected (hereinafter, referred to as “follow detection signal”) are sequentially transmitted to the processing device 3 by the BLE.

Further, the detection processing unit 156 detects a range from the start to the end of the golf swing (hereinafter, appropriately referred to as "one shot"). In the present embodiment, the detection processing unit 156 detects the timing from the address to the follow of the golf swing of the measurement target person P as a one-shot range. Then, the detection processing unit 156 transmits a signal indicating the detected one-shot range (hereinafter, referred to as “one-shot range detection signal”) to the processing device 3 by BLE.

The sensor information transmission control unit 157 controls to transmit the sensor information acquired by the detection processing unit 156 to the processing device 3. In the present embodiment, the detection processing unit 156 can acquire sensor information at, for example, about 1000 samples / second. Then, the sensor information transmission control unit 157 converts the sensor information acquired by the detection processing unit 156 to a low sampling rate (for example, about 30 samples / second) in the real-time mode and transmits the sensor information to the processing device 3. In the shot mode, it is converted to a high sampling rate (for example, about 240 samples / sec) and transmitted to the processing device 3. In the one-shot mode, the sensor information in the one-shot range is transmitted to the processing device 3. In the present embodiment, the sensor information transmission control unit 157 provides sensor information of a predetermined characteristic point (address, top, downswing, impact, follow) in the golf swing of the measurement target person P within a one-shot range. It is transmitted to the processing device 3 prior to the sensor information. In this way, by transmitting the sensor information of a predetermined characteristic point to the processing device 3 in advance, the processing device 3 can display the simple display image at an earlier timing.

Further, when the sensor information transmission control unit 157 transmits the sensor information to the processing device 3 in the one-shot mode, the sensor information transmission control unit 157 determines the timing of each point of the address, top, downswing, impact, and follow detected by the detection processing unit 156. At the same time, it is transmitted to the processing device 3.
In the present embodiment, the sensor information acquired by the detection processing unit 156 is converted into a waveform in which the influence of noise is suppressed by filtering the output signals of various sensors, respectively, and the sensor information transmission control unit 157 The sensor information acquired from the waveform represented by the sensor information of the processing result is transmitted to the processing device 3.
As a result, it is possible to analyze the movement of the measurement target person P by referring to the information having higher reliability than the sensor information having a large variation due to the influence of noise.

Next, the functional configuration of the photographing apparatus 2 will be described.
FIG. 8 is a functional block diagram showing a functional configuration for executing a shooting process among the functional configurations of the photographing device 2.
The shooting process is a series of processes in which when the analysis system S analyzes the movement of the measurement target, a moving image representing the movement of the measurement target is shot in synchronization with the acquisition of the sensor information by the sensor unit 1.
When the shooting process is executed, as shown in FIG. 8, in the CPU 211, the first communication control unit 251 and the second communication control unit 252, the shooting control unit 253, the synchronization control unit 254, and the moving image transmission control unit 255 works.
In addition, a moving image data storage unit 271 is set in one area of the storage unit 219.

The first communication control unit 251 controls the communication by the BLE of the photographing device 2, and executes the pairing process with another device and the data transmission / reception process.
The second communication control unit 252 controls communication by the Wi-Fi of the photographing device 2 and executes data transmission / reception processing with another device. In the present embodiment, when communication by Wi-Fi is performed, the photographing device 2 becomes an access point (AP) and the other device becomes a station (ST).

When the processing device 3 is instructed to shift to the real-time mode, the shooting control unit 253 controls the imaging unit 216 to perform live at a shooting speed (for example, 30 fps) preset as the shooting speed of the live view image. Take a view image. Further, the shooting control unit 253 controls the imaging unit 216 when the processing device 3 instructs to shoot a moving image for recording, and sets a shooting speed preset as a shooting speed of the moving image for recording (for example, A moving image for recording is shot at 240 fps). The captured moving image for recording is stored in the moving image data storage unit 271.

After being set to the one-shot mode, the synchronization control unit 254 synchronizes the reference time in the sensor unit 1 and the photographing device 2 via communication by Wi-Fi.
The moving image transmission control unit 255 transmits a live view image or a moving image for recording taken under the control of the shooting control unit 253 to the processing device 3. Specifically, the moving image transmission control unit 255 transmits the captured live view image to the sequential processing device 3 when the processing device 3 gives an instruction to capture the live view image in the real-time mode. Further, in the one-shot mode, when the processing device 3 gives an instruction to shoot a moving image for recording, the moving image transmission control unit 255 has a range from the start to the end of the moving image for recording notified by the processing device 3. That is, the range of one shot) is cut out and transmitted to the processing device 3.

Next, the functional configuration of the processing device 3 will be described.
FIG. 9 is a functional block diagram showing a functional configuration for executing analysis result display processing among the functional configurations of the processing device 3.
The analysis result display processing is an animation that imitates the three-dimensional movement of the body of the measurement target person P by analyzing the sensor information detected by the sensor unit 1 at the time corresponding to the moving image captured by the photographing device 2. Is a series of processes for generating and displaying the image in combination with the moving image photographed by the photographing apparatus 2.
When the analysis result display process is executed, as shown in FIG. 9, the CPU 311 includes the first communication control unit 351 and the second communication control unit 352, the calibration management unit 353, the mode control unit 354, and one. The shot shooting management unit 355, the sensor information acquisition unit 356, the image acquisition unit 357, the analysis processing unit 358, the image generation unit 359, the simple display generation unit 360, and the writing processing unit 361 function.
Further, in one area of the storage unit 320, a moving image data storage unit 371, a sensor information storage unit 372, and a composite image storage unit 373 are set.

The first communication control unit 351 controls the communication by the BLE of the processing device 3, and executes the pairing process with another device and the data transmission / reception process. In the present embodiment, when communication by BLE is performed, the processing device 3 becomes the master and the other device becomes the slave.
The second communication control unit 352 controls communication by Wi-Fi and executes data transmission / reception processing with other devices.

The calibration management unit 353 instructs the sensor unit 1 to execute the calibration. In the present embodiment, the calibration in the sensor unit 1 is performed once after the start of the sensor unit 1 and before the transition to the real-time mode or the one-shot mode.

When the operation for shifting to the real-time mode is performed, the mode control unit 354 instructs the sensor unit 1 and the photographing device 2 to shift to the real-time mode by BLE. Further, the mode control unit 354 instructs the sensor unit 1 to shift to the one-shot mode when an operation for shifting to the one-shot mode is performed.

The one-shot shooting management unit 355 manages the shooting of a moving image in the range of one shot in the real-time mode. Specifically, when the one-shot shooting management unit 355 receives the address detection signal from the sensor unit 1, it instructs the shooting device 2 to shoot a moving image for recording. Further, when the one-shot shooting management unit 355 receives the follow detection signal from the sensor unit 1, the one-shot shooting management unit 355 instructs the shooting device 2 to end the shooting of the moving image for recording. Further, when the one-shot shooting management unit 355 receives the one-shot range detection signal from the sensor unit 1, the one-shot shooting management unit 355 notifies the shooting device 2 of the one-shot range to be cut out from the shot moving image. Further, the one-shot shooting management unit 355 instructs the shooting device 2 to transmit a moving image after the acquisition of the sensor information from the sensor unit 1 is completed in the one-shot mode.

In the real-time mode, the sensor information acquisition unit 356 sequentially acquires sensor information with a period (low sampling rate) corresponding to the shooting speed (for example, 30 fps) of the live view image from the sensor unit 1 by BLE. Further, in the one-shot mode, the sensor information acquisition unit 356 uses BLE to provide sensor information in a one-shot range with a period (high sampling rate) corresponding to a recording speed (for example, 240 fps) of a moving image for recording. Get from. Further, in the present embodiment, the sensor information acquisition unit 356 takes a predetermined characteristic point in the golf swing of the measurement target person P prior to acquiring the sensor information in the one-shot range in the one-shot mode. The sensor information (address, top, downswing, impact, follow) is acquired from the sensor unit 1 by BLE.

The image acquisition unit 357 sequentially acquires the live view image transmitted from the photographing device 2 by Wi-Fi in the real-time mode. Further, the image acquisition unit 357 captures a moving image for recording in the photographing device 2 in the one-shot mode, cuts out the range of the one-shot, and then collectively acquires the cut-out moving image by Wi-Fi. ..

The analysis processing unit 358 analyzes various sensor information acquired from the sensor unit 1 and an animation represented by a box-shaped object that imitates the three-dimensional movement of the body of the measurement target person P (hereinafter, appropriately "BOX animation"). ”) To generate data representing changes over time. In the present embodiment, in the real-time mode, the analysis processing unit 358 generates data representing a temporal change of the BOX animation from the sensor information of the cycle corresponding to the shooting speed of the live view image. Further, in the one-shot mode, the analysis processing unit 358 generates data representing a temporal change of the BOX animation from the sensor information of the cycle corresponding to the shooting speed of the moving image for recording.

The image generation unit 359 displays the BOX animation generated by the analysis processing unit 358 together with the live view image acquired in the real-time mode.
FIG. 10 is a schematic view showing a display example in the real-time mode.
As shown in FIG. 10, in the real-time mode, the image generation unit 359 displays, for example, the measurement target person P in the center and displays the BOX animation An at a preset predetermined position around the image of the measurement target person P. indicate. The BOX animation An may be displayed around the image of the measurement target person P and at a position close to the position of the body on which the sensor unit 1 is attached in the measurement target person P. Further, the image generation unit 359 displays the numerical value of the sensor information corresponding to the BOX animation An at a predetermined position (here, the lower left) in the image. Specifically, in the example shown in FIG. 10, the tilt angle in the front-back direction of the body is "forward tilt: 21 degrees", the rotation angle around the vertical axis is "rotation: 15 degrees", and the tilt angle in the left-right direction is "horizontal". : 7 degrees "is displayed.

Further, the image generation unit 359 synthesizes and displays the BOX animation generated by the analysis processing unit 358 with the moving image for recording acquired in the one-shot mode. Hereinafter, the image displayed by the image generation unit 359 in this way is appropriately referred to as an “analysis result moving image”.
FIG. 11 is a schematic view showing a display example in the one-shot mode.
As shown in FIG. 11, in the one-shot mode, the image generation unit 359 displays the analysis result moving image by, for example, superimposing the BOX animation An on the moving image of the measurement target person P by transmitting it. Further, in the present embodiment, the BOX animation An also displays straight lines indicating the axes of the sensor unit 1 (here, the vertical direction and the horizontal direction). The axis of the sensor unit 1 moves with the movement of the sensor unit 1. Further, in the present embodiment, in the display example in the one-shot mode, an axis fixed to the display screen (that is, an axis representing absolute coordinates) is also displayed.
The temporal change of the BOX animation is displayed in association with the timing of the video of the photographed measurement target person P.

By displaying the sensor information in the one-shot mode and the moving image of the measurement target person P in this way, the tendency of the body movement of the measurement target person P can be shown in an easy-to-understand manner by the BOX animation, and it is difficult to appear in the BOX animation. It is possible to clearly indicate the physical condition by sensor information. Further, by displaying the axis of the sensor unit 1 and the axis fixed to the display screen together, it is possible to display the translation of the measurement target person P, etc., and the movement of the measurement target person P's body can be further improved. It will be easy to grasp.

In the one-shot mode, the simple display generation unit 360 has a predetermined characteristic point (address, top, downswing, impact, etc.) in the golf swing of the measurement target person P acquired prior to the sensor information in the one-shot range. Refer to the data of (follow), and display the numerical values of each sensor information in order on the swing image corresponding to each characteristic point prepared in advance. Hereinafter, a series of images displayed by the simple display generation unit 360 will be appropriately referred to as "simple display images".

Here, in the one-shot mode, when the sensor unit 1 acquires the sensor information and the photographing device 2 captures a moving image, the processing device 3 first receives the sensor from the sensor unit 1 by the sensor information acquisition unit 356. Get information. Next, the image acquisition unit 357 acquires video data for recording from the photographing device 2. At this time, in order for the image acquisition unit 357 to acquire the data of the moving image for recording from the photographing device 2, it takes a certain amount of time because the amount of data is relatively large. Therefore, in the present embodiment, after the image acquisition unit 357 starts acquiring the data of the moving image for recording from the photographing device 2, the simple display generation unit 360 displays the simple display image. Then, in the simple display generation unit 360, after the image acquisition unit 357 completes the acquisition of the data of the moving image for recording from the photographing device 2, the image generation unit 359 synthesizes the BOX animation with the moving image for recording and displays it. During that time, the display of the simple display image is continued.

FIG. 12 is a schematic view showing a display example of a simple display image of characteristic points in a golf swing.
As shown in FIG. 12, the simple display generation unit 360 swings the measurement target person P with respect to a prepared image showing each characteristic point of the address, top, downswing, impact, and follow in the golf swing. In, the numerical values of the sensor information of the corresponding characteristic points are also displayed.
When the simple display generation unit 360 displays the simple display image, the simple display image of each characteristic point is sequentially displayed for a predetermined time, and after the simple display image of the follow is displayed, the simple display image of the top is displayed. It is possible to return to and display each simple display image cyclically.
By displaying the simple display image, information on the movement of the measurement target person P having a certain significance can be presented in advance.
Further, in FIG. 12, the tilt angle of the body of the measurement target person P at the time of address in the front-back direction and the left-right direction may be set to 0, and the rotation angle and various tilts of the body of the measurement target person from the time of address may be displayed. It may be configured to display the amount of change in the angle.

The writing processing unit 361 writes the analysis result moving image displayed by the image generation unit 359 as data in a predetermined storage format (MPEG or the like) to the storage unit 320 or the removable media 331. The writing processing unit 361 executes writing of the analysis result moving image when the operation of instructing the saving of the analysis result moving image is performed.

[motion]
Next, the operation of the analysis system S will be described.
FIG. 13 is a flowchart illustrating a flow of information detection processing executed by the sensor unit 1.
The information detection process is started when the sensor unit 1 is activated.

In step S1, the first communication control unit 151 is connected to the processing device 3 as a slave by BLE.
In step S2, the calibration execution unit 153 acquires the sensor information in the reference state in response to the instruction from the processing device 3, and executes the calibration using the acquisition result as the reference value.

In step S3, the calibration execution unit 153 notifies the processing device 3 by BLE that the calibration is completed.
In step S4, the mode setting unit 154 sets the real-time mode in response to an instruction from the processing device 3.
In step S5, the detection processing unit 156 sequentially acquires various sensor information, and starts a process of storing the acquired sensor information in the sensor information storage unit 171 in association with the acquired time.

In step S6, the sensor information transmission control unit 157 converts the sensor information acquired by the detection processing unit 156 into a low sampling rate (for example, about 30 samples / second) and sequentially transmits the sensor information to the processing device 3.
In step S7, the mode setting unit 154 sets the one-shot mode in response to an instruction from the processing device 3.

In step S8, the second communication control unit 152 is connected to the photographing device 2 as a station by Wi-Fi.
In step S9, the synchronization control unit 155 synchronizes the reference time in the sensor unit 1 and the photographing device 2 via communication by Wi-Fi.

In step S10, the detection processing unit 156 analyzes the waveform of the acquired sensor information, detects the timing of the address point in the golf swing, and transmits the address detection signal to the processing device 3 by BLE.

In step S11, the detection processing unit 156 analyzes the waveform of the acquired sensor information, detects the timing of the follow point in the golf swing, and transmits the follow detection signal to the processing device 3 by BLE.

In step S12, the detection processing unit 156 detects the timing from the address to the follow of the golf swing of the measurement target person P as a one-shot range, and transmits the one-shot range detection signal to the processing device 3.
In step S13, the detection processing unit 156 analyzes the waveform of the acquired sensor information, and sequentially detects the timing of each of the top, downswing, and impact points in the golf swing.

In step S14, the sensor information transmission control unit 157 transmits the sensor information of a predetermined characteristic point (address, top, downswing, impact, follow) in the golf swing of the measurement target person P to the processing device 3 by BLE. Send. In this way, by transmitting the sensor information of a predetermined characteristic point to the processing device 3 in advance, the processing device 3 can display the simple display image at an earlier timing. The processes of steps S13 and S14 may be executed before step S12.

In step S15, the sensor information transmission control unit 157 converts the sensor information in the one-shot range into a high sampling rate (for example, about 240 samples / second) and transmits it to the processing device 3 by BLE. When transmitting the sensor information in the range of one shot to the processing device 3, the sensor information of the characteristic point already transmitted in step S14 may be excluded from the transmission. In this case, it is possible to prevent the same sensor information from being transmitted in duplicate, so that the sensor information can be transmitted more efficiently.
After step S15, the information detection process ends.

Next, the shooting process executed by the shooting device 2 will be described.
FIG. 14 is a flowchart illustrating a flow of a shooting process executed by the shooting device 2.
The shooting process is started by performing an operation instructing the start of the shooting process via the input unit 217.

In step S21, the first communication control unit 251 connects to the processing device 3 as a slave by BLE.
In step S22, the second communication control unit 252 connects to the processing device 3 as an access point by Wi-Fi.

In step S23, the shooting control unit 253 controls the imaging unit 216 in response to the instruction to shift to the real-time mode from the processing device 3, and the shooting speed set in advance as the shooting speed of the live view image ( For example, the process of capturing a live view image at 30 fps) is started.
In step S24, the moving image transmission control unit 255 sequentially transmits the captured live view image to the processing device 3.

In step S25, the second communication control unit 252 connects to the sensor unit 1 as an access point by Wi-Fi.
In step S26, the synchronization control unit 254 synchronizes the reference time in the sensor unit 1 and the photographing device 2 via communication by Wi-Fi.

In step S27, the shooting control unit 253 controls the imaging unit 216 in response to an instruction from the processing device 3 to shoot a moving image for recording, and shoots a preset shooting speed of the moving image for recording. The process of shooting a moving image for recording is started at a speed (for example, 240 fps).
In step S28, the shooting control unit 253 controls the imaging unit 216 in response to the instruction from the processing device 3 to end the shooting of the moving image for recording, and ends the shooting of the moving image for recording.

In step S29, the moving image transmission control unit 256 cuts out the one-shot range of the moving image for recording in response to the notification of the one-shot range from the processing device 3.
In step S30, the moving image transmission control unit 256 starts the process of transmitting the moving image in which the range of one shot is cut out to the processing device 3 in response to the instruction to transmit the moving image from the processing device 3.
In step S31, the moving image transmission control unit 256 completes the transmission of the moving image started in step S30.
After step S31, the shooting process ends.

Next, the analysis result display process executed by the processing device 3 will be described.
FIG. 15 is a flowchart illustrating a flow of analysis result display processing executed by the processing device 3.
The analysis result display process is started by performing an operation instructing the start of the analysis result display process via the input unit 318.

In step S41, the first communication control unit 351 is connected to the sensor unit 1 and the photographing device 2 as a master by BLE.
In step S42, the second communication control unit 352 connects to the photographing device 2 as a station by Wi-Fi.
In step S43, the calibration management unit 353 instructs the sensor unit 1 to execute the calibration.

In step S44, the calibration management unit 353 receives a notification from the sensor unit 1 that the calibration execution is completed by BLE.
In step S45, the mode control unit 354 receives an input of an operation (instruction for remote shooting) for shifting to the real-time mode.
In step S46, the mode control unit 354 instructs the sensor unit 1 and the photographing device 2 to shift to the real-time mode by the BLE.

In step S47, the sensor information acquisition unit 356 acquires sensor information with a period (low sampling rate) corresponding to the shooting speed (for example, 30 fps) of the live view image from the sensor unit 1 by BLE. In addition, the image acquisition unit 357 sequentially acquires the live view image transmitted from the photographing device 2 by Wi-Fi.

In step S48, the analysis processing unit 358 sequentially generates data representing the temporal change of the BOX animation from the sensor information of the cycle corresponding to the shooting speed of the live view image acquired from the sensor unit 1, and the image generation unit. The 359 sequentially displays the acquired live view image together with the BOX animation generated by the analysis processing unit 358.

In step S49, the mode control unit 354 receives an input of an operation for shifting to the one-shot mode.
In step S50, the mode control unit 354 instructs the sensor unit 1 to shift to the one-shot mode by BLE. At this time, the mode control unit 354 also transmits the SSID (Service Set Identifier) of the photographing device 2 to the sensor unit 1.

In step S51, the one-shot shooting management unit 355 instructs the shooting device 2 to shoot a moving image for recording in response to receiving the address detection signal from the sensor unit 1.
In step S52, the one-shot shooting management unit 355 instructs the shooting device 2 to end shooting a moving image for recording in response to receiving the follow detection signal from the sensor unit 1.

In step S53, the one-shot shooting management unit 355 notifies the shooting device 2 of the range of the one-shot to be cut out from the shot moving image in response to receiving the one-shot range detection signal from the sensor unit 1.

In step S54, the sensor information acquisition unit 356 receives the sensor information of a predetermined characteristic point (address, top, downswing, impact, follow) in the golf swing of the measurement target person P from the sensor unit 1 by BLE. To do.

In step S55, the data of predetermined characteristic points (address, top, downswing, impact, follow) in the golf swing of the measurement target person P are referred to, and each characteristic point prepared in advance corresponds to the data. The numerical values of each sensor information are displayed in order on the swing image. As a result, a simple display image is displayed.

In step S56, the sensor information acquisition unit 356 acquires sensor information with a period (high sampling rate) corresponding to the shooting speed (for example, 240 fps) of the moving image for recording from the sensor unit 1 by BLE.

In step S57, the one-shot shooting management unit 355 receives a moving image from the shooting device 2.

In step S58, the one-shot shooting management unit 355 determines whether or not the reception of the moving image from the shooting device 2 is completed.
If the reception of the moving image from the photographing device 2 is not completed, NO is determined in step S58, and the process proceeds to step S57.
On the other hand, when the reception of the moving image from the photographing device 2 is completed, YES is determined in step S58, and the process proceeds to step S59.

In step S59, the analysis processing unit 358 generates data representing the temporal change of the BOX animation from the sensor information of the cycle corresponding to the shooting speed of the moving image for recording acquired from the sensor unit 1.

In step S60, the image generation unit 359 synthesizes and displays the BOX animation generated by the analysis processing unit 358 with the acquired moving image for recording. As a result, the analysis result moving image is displayed.

In step S61, the writing processing unit 361 determines whether or not an operation for instructing the saving of the analysis result moving image has been performed.
If the operation for instructing the saving of the analysis result moving image is not performed, NO is determined in step S61, and the analysis result display process ends.
On the other hand, when the operation of instructing the saving of the analysis result moving image is performed, YES is determined in step S61, and the process proceeds to step S62.

In step S62, the writing processing unit 361 writes the analysis result moving image displayed by the image generation unit 359 as data in a predetermined storage format (MPEG or the like) to the storage unit 320 or the removable media 331.
After step S62, the analysis result display process ends.

By such processing, in the analysis system S, the sensor information obtained by measuring the body movement of the measurement target person P and the moving image of the movement of the measurement target person P are acquired in synchronization with the time. be able to. Then, based on the sensor information, a BOX animation that imitates the three-dimensional movement of the body of the measurement target person P is generated, and the analysis result moving image is displayed by synthesizing the BOX animation with the moving image taken by the photographing device 2. Can be done.
In the analysis result video, the video of the measurement target person P himself, the BOX animation showing the three-dimensional movement of the measurement target person P in an easy-to-understand manner, and the sensor information obtained by measuring the body movement of the measurement target person P are displayed. The time is displayed in synchronization.
Therefore, information that is difficult to grasp only from the video of the movement of the measurement target person P can be intuitively and easily grasped by the BOX animation, and more detailed by the sensor information displayed in synchronization with the time. It is possible to clearly grasp the characteristics of movement. Further, since the temporal change of the BOX animation is associated with the sensor information, it is possible to grasp the accurate analysis result by referring to the BOX animation.

Further, in the analysis system S, in the one-shot mode, by displaying predetermined information regarding the movement of the measurement target person P prior to displaying the analysis result moving image, it is possible to suppress the occurrence of unnecessary waiting time. doing.
FIG. 16 is a schematic diagram schematically showing the timing of processing between the devices in the one-shot mode.
As shown in FIG. 16, in the one-shot mode, when the processing device 3 acquires the sensor information and the moving image in order to display the analysis result moving image, first, a predetermined characteristic characteristic of the golf swing of the measurement target person P is set. The sensor information of the point is acquired (step S54 in FIG. 15), and the sensor information in the one-shot range is acquired while displaying the information on the movement of the measurement target person P as a simple display image (step S55 in FIG. 15). (Step S56), and subsequently, the moving image data is received (step S57 in FIG. 15). Then, until the reception of the moving image data is completed (step S58 in FIG. 5: YES), the BOX animation is generated (step S59 in FIG. 5), and the analysis result moving image can be displayed (step S60 in FIG. 5). ), The simple display image is continuously displayed.
Therefore, the user can display the movement of the measurement target person P having a certain significance in advance rather than simply waiting for the processing device 3 to acquire the moving image data, so that the user can move the movement of the target person more efficiently. It becomes possible to grasp.
In addition, by presenting information on the movement of the measurement target person P in advance, a time zone in which the presented information is viewed by the user is additionally added to the minimum time required for generating the analysis result video. Can be created. Therefore, it is possible to secure a longer time to generate the analysis result moving image, and it is also possible to present a more effective analysis result moving image.

Further, in the analysis system S, in the one-shot mode, the address of the measurement target person P is automatically detected, and a moving image for recording can be recorded.
Therefore, the measurement target person P can sequentially shoot the moving image for recording by a simple operation by repeating the operation of addressing and swinging.

[Modification 1]
In the above-described embodiment, the display form of the BOX animation can be variously modified as long as it imitates the three-dimensional movement of the body of the measurement target person P.
FIG. 17 is a schematic diagram showing an example of a display form of the BOX animation An.
In the example shown in FIG. 17, a plurality of viewpoints such as front view, top view, and left side view can be selected as the viewpoint of the BOX animation An, and in the example shown in FIG. 17, the BOX animation of the selectable viewpoint is selected. An is displayed in a small size at the top of the screen.
Further, the display color of the BOX animation An can be selected to be a color in which the three-dimensional movement of the body of the measurement target person P is more easily understood.
In the example shown in FIG. 17, the display color of the BOX animation An of each viewpoint is set to a different color.
When the BOX animation is superimposed and displayed on the moving image of the measurement target person P, the background color may be detected and a predetermined color (for example, a complementary color) that is difficult to be assimilated with the background color may be selected. ..

[Modification 2]
In the above-described embodiment, as a display example in the real-time mode, a case where the measurement target person P is displayed in the center and the BOX animation An is displayed at a preset predetermined position around the image of the measurement target person P has been described. .. On the other hand, even in the real-time mode, the BOX animation An may be transparently superimposed on the moving image of the measurement target person P as in the display example of the one-shot mode shown in FIG.
On the contrary, also in the one-shot mode, as in the display example of the real-time mode shown in FIG. 10, the measurement target person P is displayed in the center and a predetermined predetermined position around the image of the measurement target person P. BOX animation An may be displayed in.

[Modification 3]
In the above-described embodiment, one sensor unit 1 is attached to the waist or the like of the measurement target person P to acquire sensor information, but the present invention is not limited to this. That is, a plurality of sensor units 1 may be attached to a plurality of body parts such as the shoulder and waist of the measurement target person P, and a BOX animation showing the movement of each of these body parts may be displayed.
In this case, the difference in movement of the measurement target person P for each body part (for example, the twist of the shoulder and the waist) can be displayed, and the measurement result can be presented more effectively.
In addition to the waist, the body part to which the sensor unit 1 is attached can be shoulders, neck, arms, thighs, head, and the like.

[Modification example 4]
In the above-described embodiment, in the display example in the one-shot mode, a continuous moving image of the movement of the measurement target person P is displayed. On the other hand, it is possible to display an intermittent moving image of the movement of the measurement target person P.
In this case, the processing load for generating the analysis result moving image is reduced, and the analysis result moving image can be presented earlier.

[Modification 5]
In the above-described embodiment, when the BOX animation is transparently superimposed on the moving image of the measurement target person P, the transparency can be set to various different values depending on the purpose. For example, when displaying a humanoid object as a guide for the movement to be performed by the measurement target person P, when the overlap between the measurement target person P's body and the humanoid object is large, the transparency of the BOX animation is increased ( That is, it is displayed lightly), and when the overlap between the body of the measurement target person P and the humanoid object is small, it is possible to reduce the transparency of the BOX animation (that is, display it darkly).
As a result, the BOX animation can be displayed in a more effective form when displaying the movement guide of the measurement target person P.

[Modification 6]
In the above-described embodiment, the mounting position of the sensor unit 1 can be automatically determined by photographing the measurement target person P wearing the sensor unit 1 with the photographing device 2.
In this case, in the display example in the real-time mode, when the BOX animation is displayed at a position close to the position of the body on which the sensor unit 1 is attached in the measurement target person P, the display position of the BOX animation is automatically determined. It becomes possible.

The analysis system S configured as described above includes a sensor unit 1, a photographing device 2, and a processing device 3. The sensor unit 1 is attached to the target person and includes a detection processing unit 156 that measures the movement of the target person by a sensor. The photographing device 2 includes a photographing control unit 253 that photographs an image of the target person. The processing device 3 includes an image acquisition unit 357, a sensor information acquisition unit 356, an analysis processing unit 358, an image generation unit 359, and a simple display generation unit 360.
The image acquisition unit 357 acquires an image taken by the subject.
The sensor information acquisition unit 356 acquires the measurement result obtained by measuring the movement of the target person by the sensor.
The image generation unit 359 causes the output unit 319 to display the image acquired by the image acquisition unit 357.
The simple display generation unit 360 causes the output unit 319 to display information on the movement of the target person based on the measurement result acquired by the sensor information acquisition unit 356 prior to the display by the image generation unit 359. ..
As a result, rather than simply waiting for the time until the analysis result is displayed by the image generation unit 359, information on the movement of the target person having a certain significance can be displayed in advance, so that the user can display the movement of the target person more. It becomes possible to grasp efficiently.
Therefore, according to the present invention, it is possible to display the movement of the target person so that the user can grasp it more efficiently.

The image generation unit 359 causes the output unit 319 to display both the image acquired by the image acquisition unit 357 and the measurement result.
Therefore, since the moving image showing the movement of the target person and the measurement result by the sensor when the movement in the moving image can be displayed together, the information can be displayed in a more effective form.

The measurement result acquisition process by the sensor information acquisition unit 356 has a shorter processing time than the image acquisition process by the image acquisition unit 357.
Therefore, it is possible to display useful information by using the measurement result that can be acquired earlier until the image that requires a long time for the acquisition process is acquired.

The sensor information acquisition unit 356 acquires at least one information of the angular velocity or the acceleration measured by the sensor attached to the subject as the measurement result.
As a result, more accurate measurement results can be obtained.

The image acquisition unit 357 acquires an image taken by a photographing device that photographs the target person.
As a result, a more appropriate image of the target person can be taken.

The image generation unit 359 displays the index superimposed on the image of the target person.
This makes it possible to easily grasp the movement of the target person and the change of the index.

The sensor information acquisition unit 356 acquires the measurement result prior to the image acquisition unit 357 acquiring the image.
The simple display generation unit 360 causes the output unit 319 to display information regarding the movement of the target person while the image acquisition unit 357 is acquiring the image.
As a result, the time during which the image is acquired by the image acquisition unit 357 can be effectively utilized.

The simple display generation unit 360 causes the output unit 319 to display the measurement result of the characteristic point and the image representing the characteristic point in the measurement result acquired by the sensor information acquisition unit 356.
This makes it possible to grasp the characteristics of the movement of the subject at an early stage.

In the measurement result acquired by the sensor information acquisition unit 356, the simple display generation unit 360 outputs the measurement result of the characteristic point and the image showing the characteristic point in order for a plurality of characteristic points in the time series. Displayed on 319.
This makes it possible to grasp the outline of the entire movement of the target person at an early stage.

The present invention is not limited to the above-described embodiment, and modifications, improvements, and the like within the range in which the object of the present invention can be achieved are included in the present invention.

In the above-described embodiment, when the processing device 3 displays the simple display image, the time required for the acquisition of the moving image from the photographing device 2 and the generation of the analysis result moving image is estimated, and the estimated time is calculated by the number of the simple display images. The simple display images may be displayed in order for each divided time. As a result, the analysis result can be displayed in a more understandable form while matching the display content of the simple display image with the display start timing of the analysis result moving image.

Further, in the above-described embodiment, the analysis system S is composed of three devices, a sensor unit 1, a photographing device 2, and a processing device 3, but the present invention is not limited to this. For example, the analysis system S may be configured as a device in which the sensor unit 1 and the processing device 3 are integrated by a smartphone or the like having the functions of the sensor unit 1 and the processing device 3.

Further, in the above-described embodiment, when displaying the temporal change of the BOX animation, the movement features such as the change in acceleration are color-coded or the index is displayed based on the sensor information acquired by the sensor unit 1. Depending on the form, it may be identified and displayed. For example, when the body is moving at an acceleration larger than the reference, it is possible to color the body of the measurement target person P in red.

In the above-described embodiment, when the BOX animation is displayed at a preset predetermined position around the image of the measurement target person P as in the display example in the real-time mode, the sensor unit 1 is attached to the measurement target person P. The position of the body may be identified and indicated by a leader line or a marker. Further, the BOX animation may be superimposed and displayed on the position of the body on which the sensor unit 1 is attached in the measurement target person P.

In the above-described embodiment, it is possible to determine whether or not the measurement target person P has performed a swing by setting detailed conditions based on the sensor information. For example, if the rotation angle around the vertical axis is equal to or greater than the first angle and rotates in the takeback direction, it is determined to be the top, and if it rotates from the top to the vertical axis at the first angular velocity or more, it is determined to be a downswing. When the rotation angle around the vertical axis is equal to or greater than the second angle and the rotation is in the follow direction, it is determined to be follow, and when all these conditions are satisfied, it can be determined that the swing has been performed. Further, the conditions may be set for the time when these determination conditions are satisfied. For example, it can be set as a condition that each condition is sequentially satisfied within 2 seconds.

In the above-described embodiment, when the timing of a predetermined characteristic point in the golf swing is detected based on the sensor information, the time-series data is analyzed in chronological order to determine the characteristic point. It is possible to discriminate other characteristic points by analyzing the time series retroactively from the timing that can be clearly discriminated as a specific point.
This makes it possible to more reliably detect the timing of a predetermined characteristic point in the golf swing.

In the above-described embodiment, when the simple display generation unit 360 displays the simple display image, the cancel button for canceling the processing related to the generation of the BOX animation data by the analysis processing unit 358 and the display of the analysis result moving image by the image generation unit 359 is pressed. It may be displayed.
As a result, by checking the simple display image, it is possible to prevent unnecessary processing from being performed when it is determined that the generation of the analysis result moving image is unnecessary.

In the above-described embodiment, when the sensor information of the one-shot range acquired in the one-shot mode is transmitted from the sensor unit 1 to the processing device 3, in step S14 of FIG. 13, a predetermined value in the golf swing of the measurement target person P is determined. In step S15 of FIG. 13, the entire sensor information in the one-shot range may be transmitted to the processing device 3 without transmitting the sensor information of the characteristic point in advance. In this case, the time required for transmitting the sensor information can be shortened as a whole.

In the above-described embodiment, the detection processing unit 156 detects the timing of a predetermined characteristic point in the golf swing based on the sensor information acquired in the one-shot mode. For example, the detection processing unit 156 may analyze the waveform of the sensor information acquired in the one-shot mode and detect the timing when the waveform starts to change from the state where the amount of change in the waveform is small as the address timing, and swing once. After detecting the range of the downswing and detecting the timing of the downswing, the time point that goes back a predetermined time from the timing of the downswing may be reset as the time point of the address to determine the range of one shot.

In the above-described embodiment, the case where the sensor unit 1 is attached to the measurement target person P who performs the golf swing and the analysis system S is used for the analysis of the golf swing has been described, but the present invention is not limited to this. That is, the analysis system S according to the present invention can be used for various sports such as baseball, tennis, and athletics, in which a player as a subject can be photographed at a fixed angle of view. For example, the analysis system S according to the present invention can be used for a batter swinging in a baseball at-bat, a pitcher throwing in a mound, a tennis server, a sprinter photographed by a parallel camera, and the like.

Further, in the above-described embodiment, the processing device 3 to which the present invention is applied has been described by taking a smartphone as an example, but the present invention is not particularly limited thereto.
For example, the present invention can be generally applied to electronic devices having an image processing function. Specifically, for example, the present invention can be applied to notebook personal computers, printers, television receivers, video cameras, portable navigation devices, mobile phones, portable game machines, and the like.

The series of processes described above can be executed by hardware or software.
In other words, the functional configurations of FIGS. 6, 8 and 9 are merely examples and are not particularly limited. That is, it suffices if the analysis system S is provided with a function capable of executing the above-mentioned series of processes as a whole, and what kind of functional block is used to realize this function is particularly an example of FIGS. 6, 8 and 9. Not limited to.
Further, one functional block may be configured by a single piece of hardware, a single piece of software, or a combination thereof.
The functional configuration in the present embodiment is realized by a processor that executes arithmetic processing, and the processor that can be used in the present embodiment is composed of various processing devices such as a single processor, a multiprocessor, and a multicore processor. In addition to the above, the present invention includes a combination of these various processing units and processing circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field-Programmable Gate Array).

When a series of processes are executed by software, the programs constituting the software are installed on a computer or the like from a network or a recording medium.
The computer may be a computer embedded in dedicated hardware. Further, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

The recording medium containing such a program is not only composed of the removable media 231 and 331 of FIGS. 4 and 5 distributed separately from the device main body in order to provide the program to the user, but is also preliminarily incorporated in the device main body. It is composed of a recording medium or the like provided to the user in this state. The removable media 231 and 331 are composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versaille Disk), a Blu-ray (registered trademark) Disc (Blu-ray Disc), or the like. The magneto-optical disk is composed of MD (Mini-Disk) or the like. The recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body is, for example, ROMs 112, 212, 312 of FIGS. 3 to 5 in which the program is recorded, and storage units 119, 219 of FIGS. , 320 and the like.

In the present specification, the steps for describing a program recorded on a recording medium are not necessarily processed in chronological order, but also in parallel or individually, even if they are not necessarily processed in chronological order. It also includes the processing to be executed.
Further, in the present specification, the term of the system means an overall device composed of a plurality of devices, a plurality of means, and the like.

Although some embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and further, various modifications such as omission and substitution can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in the present specification and the like, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

The inventions described in the claims at the time of filing the application of the present application are described below.
[Appendix 1]
Image acquisition means for acquiring images taken by the subject,
A measurement result acquisition means for acquiring a measurement result obtained by measuring the movement of the target person with a sensor,
A first display control means for causing the display means to display the image acquired by the image acquisition means, and
Prior to the display by the first display control means, the display means causes the display means to display information on the movement of the target person based on the measurement result acquired by the measurement result acquisition means. Control means and
An image processing device characterized by comprising.
[Appendix 2]
The image processing apparatus according to Appendix 1, wherein the first display control means causes the display means to display both the image acquired by the image acquisition means and the measurement result.
[Appendix 3]
The image processing apparatus according to Appendix 1 or 2, wherein the measurement result acquisition process by the measurement result acquisition means has a shorter processing time than the image acquisition process by the image acquisition means.
[Appendix 4]
The measurement result acquisition means obtains at least one of the information of the angular velocity or the acceleration measured by the sensor mounted on the subject as the measurement result, according to any one of the appendices 1 to 3. The image processing apparatus described.
[Appendix 5]
The image processing device according to any one of Supplementary note 1 to 4, wherein the image acquisition means acquires the image taken by the photographing device for photographing the target person.
[Appendix 6]
The measurement result acquisition means acquires the measurement result prior to the image acquisition means acquiring the image.
The second display control means is any one of Appendix 1 to 5, wherein the display means displays information on the movement of the target person while the image acquisition means is acquiring the image. The image processing apparatus according to 1.
[Appendix 7]
In the measurement result acquired by the measurement result acquisition means, the second display control means causes the display means to display the measurement result of the characteristic point and the image representing the characteristic point. The image processing apparatus according to any one of Supplementary Provisions 1 to 6, which is a feature.
[Appendix 8]
In the measurement result acquired by the measurement result acquisition means, the second display control means obtains the measurement result and the characteristic point of the characteristic point with respect to the plurality of the characteristic points in the time series. The image processing apparatus according to Appendix 7, wherein the images to be represented are sequentially displayed on the display means.
[Appendix 9]
A shooting device equipped with a shooting means for shooting an image of the target person,
A detection device that is attached to the target person and includes a measuring means for measuring the movement of the target person by a sensor.
An image acquisition means for acquiring an image captured by the subject from the imaging device, and
A measurement result acquisition means for acquiring a measurement result obtained by measuring the movement of the target person by a sensor from the detection device, and
A first display control means for causing the display means to display the image acquired by the image acquisition means, and
Prior to the display by the first display control means, the display means causes the display means to display information on the movement of the target person based on the measurement result acquired by the measurement result acquisition means. An image processing device including a control means,
An analysis system characterized by including.
[Appendix 10]
An image processing method executed by an image processing device.
An image acquisition step to acquire an image taken by the subject, and
The measurement result acquisition step of acquiring the measurement result obtained by measuring the movement of the target person by the sensor, and
A first display control step for causing the display means to display the image acquired by the image acquisition step, and
Prior to the display by the first display control step, the display means causes the display means to display information on the movement of the target person based on the measurement result acquired in the measurement result acquisition step. Control steps and
An image processing method characterized by including.
[Appendix 11]
To the computer that controls the image processing device
An image acquisition function that acquires images taken by the target person,
The measurement result acquisition function that acquires the measurement result of the movement of the target person measured by the sensor, and
The first display control function for displaying the image acquired by the image acquisition function on the display means, and
Prior to the display by the first display control function, the second display for displaying information on the movement of the target person on the display means based on the measurement result acquired by the measurement result acquisition function. Control function and
A program characterized by realizing.

S analysis system, 1 ... sensor unit, 2 ... photographing device, 3 ... processing device, 111, 211, 311 ... CPU, 112, 212, 312 ... ROM, 113, 213, 313 ... RAM, 114, 214, 314 ... Bus, 115, 215, 315 ... Input / output interface, 116, 317 ... Sensor unit, 117, 217, 318 ... Input unit, 118, 218 , 319 ... Output unit, 119, 219, 320 ... Storage unit, 120, 220, 321 ... Communication unit, 121, 322 ... Drive, 216, 316 ... Imaging unit, 231 and 331 ... Removable media, 151, 251, 351 ... 1st communication control unit, 152, 252, 352 ... 2nd communication control unit, 153 ... Calibration execution unit, 154 ... Mode setting unit , 155, 254 ... Synchronous control unit, 156 ... Detection processing unit, 171, 372 ... Sensor information storage unit, 253 ... Shooting control unit, 255 ... Movie transmission control unit, 271, 371 ... Video data storage unit, 353 ... Calibration management unit, 354 ... Mode control unit, 355 ... One-shot shooting management unit, 356 ... Sensor information acquisition unit, 357 ... Image acquisition Unit, 358 ... Analysis processing unit, 359 ... Image generation unit, 360 ... Simple display generation unit, 361 ... Export processing unit, 373 ... Composite image storage unit

Claims (11)

Image acquisition means for acquiring captured images taken by the subject,
A measurement result acquisition means for acquiring a measurement result obtained by measuring the movement of the target person with a sensor,
A first display control means for displaying the captured image acquired by the image acquisition means on the display means, and
A simple display image generation means for generating a simple display image relating to the movement of the target person, which is different from the captured image, based on the measurement result acquired by the measurement result acquisition means.
A second display control means for causing the display means to display the simple display image generated by the simple display image generation means prior to the display of the captured image by the first display control means .
An image processing apparatus characterized in that the measurement result acquisition process by the measurement result acquisition means has a shorter processing time than the acquisition process of the captured image by the image acquisition means . Image acquisition means for acquiring captured images taken by the subject,
A measurement result acquisition means for acquiring a measurement result obtained by measuring the movement of the target person with a sensor,
A first display control means for displaying the captured image acquired by the image acquisition means on the display means, and
A simple display image generation means for generating a simple display image relating to the movement of the target person, which is different from the captured image, based on the measurement result acquired by the measurement result acquisition means.
A second display control means for causing the display means to display the simple display image generated by the simple display image generation means prior to the display of the captured image by the first display control means.
The measurement result acquisition means acquires the measurement result prior to the image acquisition means acquiring the captured image.
The second display control means is an image processing device characterized in that the simple display image is displayed on the display means while the image acquisition means is acquiring the captured image. The image processing apparatus according to claim 1 or 2 , wherein the first display control means displays both the captured image acquired by the image acquisition means and the measurement result on the display means. Any one of claims 1 to 3, wherein the measurement result acquisition means acquires at least one information of an angular velocity or an acceleration measured by a sensor mounted on the subject as the measurement result. The image processing apparatus according to. The image processing device according to any one of claims 1 to 4, wherein the image acquisition means acquires the photographed image photographed by the photographing apparatus for photographing the target person. The simple display image generation means generates the simple display image representing a characteristic point in the measurement result acquired by the measurement result acquisition means.
As the simple display image , the second display control means obtains the measurement result of the characteristic point and the image corresponding to the characteristic point in the measurement result acquired by the measurement result acquisition means. The image processing apparatus according to any one of claims 1 to 5 , wherein the display means displays the image. Image acquisition means for acquiring captured images taken by the subject,
A measurement result acquisition means for acquiring a measurement result obtained by measuring the movement of the target person with a sensor,
A first display control means for displaying the captured image acquired by the image acquisition means on the display means, and
A simple display image generation means for generating a simple display image relating to the movement of the target person, which is different from the captured image, based on the measurement result acquired by the measurement result acquisition means.
A second display control means for causing the display means to display the simple display image generated by the simple display image generation means prior to the display of the captured image by the first display control means.
The simple display image generation means generates the simple display image representing a characteristic point in the measurement result acquired by the measurement result acquisition means.
As the simple display image, the second display control means obtains the measurement result of the characteristic point and the image corresponding to the characteristic point in the measurement result acquired by the measurement result acquisition means. Display on the display means
The simple display image generation means generates a plurality of the simple display images representing a plurality of the characteristic points in a time series in the measurement result acquired by the measurement result acquisition means.
The second display control means obtains the measurement results of the plurality of the characteristic points in the time series and the plurality of the characteristic points in the time series in the measurement result acquired by the measurement result acquisition means. the fact to be displayed on the display means the simple display image in order to represent image picture processor you said. When a predetermined operation by the target person is detected based on the measurement result acquired by the measurement result acquisition means, instruction information indicating an instruction regarding photography is transmitted to the photographing device for photographing the photographed image. Further equipped with instruction information transmission means,
Based on the measurement result, the instruction information transmitting means transmits the instruction information indicating that the imaging device starts photographing when the first operation by the target person is detected, and the target person transmits the instruction information. The invention according to any one of claims 1 to 7, wherein when the second operation different from the first operation is detected, the instruction information indicating that the photographing device is terminated is transmitted. Image processing equipment. A shooting device equipped with a shooting means for shooting an image shot by the subject,
A detection device that is attached to the target person and includes a measuring means for measuring the movement of the target person by a sensor.
An image acquisition means for acquiring a photographed image taken by the subject from the photographing device, and
A measurement result acquisition means for acquiring a measurement result obtained by measuring the movement of the target person by a sensor from the detection device, and
A first display control means for displaying the captured image acquired by the image acquisition means on the display means, and
A simple display image generation means for generating a simple display image relating to the movement of the target person, which is different from the captured image, based on the measurement result acquired by the measurement result acquisition means.
A second display control means for causing the display means to display the simple display image generated by the simple display image generation means prior to the display of the captured image by the first display control means .
An image processing apparatus characterized in that the measurement result acquisition process by the measurement result acquisition means has a shorter processing time than the acquisition process of the captured image by the image acquisition means .
An analysis system characterized by including. An image processing method executed by an image processing device.
An image acquisition step to acquire a captured image taken by the subject, and
The measurement result acquisition step of acquiring the measurement result obtained by measuring the movement of the target person by the sensor, and
A first display control step for displaying the captured image acquired by the image acquisition step on the display means, and
Based on the measurement result acquired by the measurement result acquisition step, a simple display image generation step for generating a simple display image relating to the movement of the target person, which is different from the captured image,
A second display control step of displaying the simple display image generated by the simple display image generation step on the display means prior to the display of the captured image by the first display control step is included.
An image processing method characterized in that the measurement result acquisition process by the measurement result acquisition step has a shorter processing time than the acquisition process of the captured image by the image acquisition step . To the computer that controls the image processing device
Image acquisition function for acquiring a photographed image of the subject is photographed,
Measurement result acquisition function for acquiring the measurement result obtained by measuring the motion of the subject by a sensor,
The first display control function of displaying the photographed image acquired by the image acquisition function on the display means,
The measurement result acquisition function based on the acquired measurement result by said different from the captured image, the simple display image generating function of generating a simplified display image related to the movement of the subject,
Prior to the display of the captured image by the first display control function, the simple display image generated by the simple display image generation function is realized as a second display control function to be displayed on the display means .
The measurement result acquisition process by the measurement result acquisition function is a program characterized in that the processing time is shorter than the acquisition process of the captured image by the image acquisition function .

Images