JP2018163417A - Image processing device, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a user to simply specifying a frame image to be displayed among moving images indicating a series of motions.SOLUTION: When displaying a moving image showing a golf swing action acquired by imaging a golf swing action, an image processing device displays a seek bar which is used by the user for specifying and operating a time axis position of the frame image to be displayed among the moving images, and in which respective sections corresponding to partial motions contained in the golf swing motion are identifiably displayed. Then, according to a change of position of knob T1 of the seek bar by the specifying operation on the seek bar by the user, a frame image to be displayed is selected from among the moving images.SELECTED DRAWING: Figure 8

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  2. Description of the Related Art Conventionally, a technique for detecting a user's action from a sensor attached to the user and setting chapter information corresponding to the action in a captured image based on the detected action is known (for example, Patent Document 1). reference).

JP2013-188426A

  However, in Patent Document 1, no consideration has been given to the display of chapters set in an image, leaving room for improvement in user convenience.

  The present invention has been made in view of such a problem, and is to enable a user to easily specify a frame image to be displayed from a moving image showing a series of operations.

In order to solve the above problem, an image processing apparatus according to claim 1 is provided.
Moving image acquisition means for acquiring a moving image showing a series of movements of the subject;
Among the moving images acquired by the moving image acquisition means, an operation image for a user to specify and operate a position on a time axis of a frame image to be displayed, and each of the partial operations included in the series of operations Display control means for causing the display means to display the operation image in which the corresponding section is displayed in an identifiable manner;
Image selecting means for selecting a frame image to be displayed from the moving images in accordance with a change in display mode of the operation image by a user's specifying operation;
It is characterized by providing.

An image processing apparatus according to a sixth aspect of the present invention provides:
Moving image acquisition means for acquiring a moving image indicating a rotation operation of the subject;
Rotation angle acquisition means for acquiring information on the rotation angle of the subject in each frame image of the moving image;
Display control means for causing the display means to display an operation image for the user to specify the rotation angle of the subject in the frame image to be displayed among the moving images acquired by the moving image acquisition means;
Image selecting means for selecting a frame image to be displayed from the moving images in response to a designation operation on the operation image by a user;
It is characterized by providing.

  According to the present invention, a user can easily specify a frame image to be displayed from a moving image showing a series of operations.

It is a figure which shows the example of whole structure of the display system of this embodiment. It is a figure which shows typically the mounting aspect to the user of the sensor of FIG. 1, and the direction of 3 axes | shafts. It is a block diagram which shows the functional structure of the sensor of FIG. It is a block diagram which shows the functional structure of the imaging device of FIG. It is a block diagram which shows the functional structure of the image processing apparatus of FIG. It is a figure which shows the measurement result of the angular velocity of the Z-axis direction in a user's golf swing operation | movement. It is a flowchart which shows the moving image display process A performed by CPU of FIG. It is a figure which shows the example of a seek bar. It is a flowchart which shows the moving image display process B performed by CPU of FIG. It is a figure which shows the example of a rotation angle controller.

  Embodiments according to the present invention will be described below in detail with reference to the accompanying drawings. The present invention is not limited to the illustrated example.

<First Embodiment>
[Configuration of Display System 100]
FIG. 1 is a diagram showing an example of the overall configuration of a display system 100 according to the first embodiment of the present invention. As shown in FIG. 1, the display system 100 includes a sensor 1, an imaging device 2, and an image processing device 3, and each device is configured to be communicably connected to each other.

[Configuration of Sensor 1]
As shown in FIG. 2, for example, the sensor 1 is attached to the waist of the user U, and measures the user U's action (in this embodiment, a golf swing action) to acquire sensor data. Send to.

  FIG. 3 is a block diagram illustrating a functional configuration of the sensor 1. As shown in FIG. 3, the sensor 1 includes a CPU 11, a RAM (Random Access Memory) 12, a storage unit 13, an operation unit 14, a sensor unit 15, a communication unit 16, a timing unit 17, and the like. 18 is connected.

  The CPU 11 is a processor that executes various programs stored in the storage unit 13 to perform predetermined calculations and control of each unit. For example, the CPU 11 performs time synchronization with the imaging device 2 according to the operation of the operation unit 14. Further, the CPU 11 starts measurement of the golf swing motion of the user U by the sensor unit 15 in response to the operation of the operation unit 14. When an address (see FIG. 6) in the golf swing motion is detected, the communication unit 16 performs the addressing. A detection signal is transmitted to the image processing apparatus 3. Further, when the finish (see FIG. 6) in the golf swing motion is detected, the communication unit 16 transmits a finish detection signal to the image processing device 3, and sensor data stored in the RAM 12 is processed by the communication unit 16. Transmit to device 3.

  The RAM 12 provides a working memory space to the CPU 11 and temporarily stores data. For example, the RAM 12 arranges the measurement results acquired by the sensor unit 15 in time series (corresponding to the time) and stores it as sensor data.

  The storage unit 13 is configured by a nonvolatile semiconductor memory, a hard disk, or the like. The storage unit 13 stores a system program executed by the CPU 11, various processing programs, data necessary for executing these programs, and the like.

  The operation unit 14 includes various function keys, accepts pressing input of each key by the user, and outputs the operation information to the CPU 11.

  The sensor unit 15 includes an X-axis, Y-axis, and Z-axis acceleration sensor, a 3-axis gyro sensor, a 3-axis geomagnetic sensor, and the like shown in FIG. Measure acceleration, angular velocity, geomagnetism, and the like, and output sensor data indicating the measurement results of each sensor to the CPU 11.

  The communication unit 16 has an interface for communicating with external devices such as the imaging device 2 and the image processing device 3. The communication unit 16 transmits / receives data to / from an external device in accordance with a wireless communication standard such as Bluetooth (registered trademark) or Wi-Fi.

  The timer unit 17 is composed of an RTC (Real Time Clock) or the like, and measures the current time and outputs it to the CPU 11.

[Configuration of Imaging Device 2]
The imaging device 2 acquires a moving image obtained by photographing the golf swing motion of the user U in synchronization with the measurement of the golf swing motion of the user U by the sensor 1 and transmits it to the image processing device 3.

  FIG. 4 is a block diagram illustrating a functional configuration of the imaging apparatus 2. As shown in FIG. 4, the imaging device 2 includes a CPU 21, a RAM 22, a storage unit 23, an operation unit 24, a display unit 25, an imaging unit 26, a communication unit 27, a timing unit 28, and the like. Connected by.

  The CPU 21 is a processor that executes various programs stored in the storage unit 23 to perform predetermined calculations and control of each unit. For example, the CPU 21 performs time synchronization with the sensor 1 according to the operation of the operation unit 24. Further, the CPU 21 starts moving image shooting when the image processing device 3 is instructed to shoot via the communication unit 27, and ends the moving image shooting when the end of shooting is instructed, and image data of the captured moving image. Is associated with the shooting time and transmitted to the image processing apparatus 3 by the communication unit 27 as a moving image file.

  The RAM 22 provides a working memory space to the CPU 21 and temporarily stores data. For example, the RAM 22 stores the frame image of the moving image acquired by the imaging unit 26 in association with the shooting time.

  The storage unit 23 is configured by a nonvolatile semiconductor memory, a hard disk, or the like. The storage unit 23 stores a system program executed by the CPU 21, various processing programs, data necessary for executing these programs, and the like.

  The operation unit 24 includes various function keys, accepts pressing input of each key by the user, and outputs the operation information to the CPU 21.

  The display unit 25 is configured by an LCD (Liquid Crystal Display) or the like, and performs various displays according to instructions of a display signal input from the CPU 21.

  The imaging unit 26 includes an optical lens unit, an imaging element such as an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-oxide Semiconductor), an A / D conversion circuit, and the like, and passes through the optical system. The optical image thus obtained is converted into a two-dimensional image signal to obtain image data.

  The communication unit 27 has an interface for communicating with external devices such as the sensor 1 and the image processing apparatus 3. The communication unit 27 transmits / receives data to / from an external device according to a wireless communication standard such as Bluetooth (registered trademark) or Wi-Fi. Alternatively, data may be transmitted / received to / from an external device in accordance with a wired communication standard such as USB (Universal Serial Bus).

  The timer unit 28 is composed of an RTC or the like, and measures the current time and outputs it to the CPU 21.

[Configuration of Image Processing Device 3]
The image processing device 3 displays an operation image for the user to specify a frame image to be displayed among the moving images transmitted from the imaging device 2, and the moving image of the moving image is displayed according to a user operation from the operation image. It is a device that displays a frame image designated from the inside. As the image processing device 3, for example, a smartphone, a tablet, a PC (Personal Computer), or the like is applicable.

  FIG. 5 is a block diagram illustrating a functional configuration of the image processing apparatus 3. As shown in FIG. 5, the image processing apparatus 3 includes a CPU 31, a RAM 32, a storage unit 33, an operation unit 34, a display unit 35, a first communication unit 36, a second communication unit 37, and the like. 38 is connected.

The CPU 31 is a processor that executes various programs stored in the program storage unit 331 of the storage unit 33 to perform predetermined calculations and control of each unit.
For example, when the golf swing display application stored in the program storage unit 331 is activated in the video acquisition mode, the CPU 31 waits for reception of an address detection signal from the sensor 1, and the second communication unit 37 causes the sensor 1 to When the address detection signal is received, a shooting instruction is transmitted to the imaging apparatus 2. Further, when the CPU 31 receives the finish detection signal from the sensor 1 by the second communication unit 37, the CPU 31 transmits a shooting end instruction to the imaging device 2. Further, when the CPU 31 receives the sensor data from the sensor 1 by the second communication unit 37, the CPU 31 stores the received sensor data in the sensor data storage unit 332 and receives the moving image image data from the imaging device 2. The image data is stored in the moving image storage unit 333.
Further, for example, when the golf swing display application is activated in the moving image display mode, the CPU 31 executes a moving image display process A described later, and displays a seek bar (see FIG. 8) on the display unit 35. Further, according to the operation of the seek bar by the user, a frame image is selected from the frame images constituting the moving image and displayed on the display unit 35. The CPU 11 functions as a moving image acquisition unit, a display control unit, and an image selection unit.

  Here, the seek bar is an operation image for the user to specify and operate the position on the time axis of the frame image displayed from the moving image. The seek bar in this embodiment is divided into sections corresponding to each of partial operations (back swing, down swing, follow-through) in which the time axis of the seek bar is included in the golf swing operation.

  The RAM 32 provides a working memory space to the CPU 31 and temporarily stores data.

  The storage unit 33 is configured by a nonvolatile semiconductor memory, a hard disk, or the like. The storage unit 33 includes a program storage unit 331, a sensor data storage unit 332, and a moving image storage unit 333.

  The program storage unit 331 stores a system program executed by the CPU 31, various processing application programs including a golf swing display application, data necessary for executing these programs, and the like.

  The sensor data storage unit 332 stores sensor data transmitted from the sensor 1. For example, the sensor data is stored with the same identification ID as the moving image corresponding to the sensor data (that is, the moving image obtained by photographing the golf swing motion when measuring the golf swing motion represented by the sensor data). .

  The moving image storage unit 333 stores moving image image data transmitted from the imaging device 2. For example, the same identification ID as the sensor data corresponding to the moving image (that is, the sensor data obtained by measuring the golf swing motion at the time of shooting the moving image) is assigned to the moving image and stored.

  Further, the storage unit 33 stores a seek bar image.

  The operation unit 34 includes various function keys, accepts pressing input of each key by the user, and outputs the operation information to the CPU 31. The operation unit 34 has a touch panel or the like in which transparent electrodes are arranged in a grid pattern so as to cover the surface of the display unit 35, detects various operations with fingers or a touch pen, and outputs the operation information to the CPU 31.

  The display unit 35 is configured by an LCD or the like, and performs various displays on the screen according to a display control signal from the CPU 31.

  The first communication unit 36 is wirelessly connected to a communication network such as the Internet including a mobile communication network via a base station or an access point, and communicates with an external device connected to the communication network.

  The second communication unit 37 transmits / receives data to / from an external device such as the sensor 1 or the imaging device 2 according to a wireless communication standard such as Bluetooth (registered trademark) or Wi-Fi. Alternatively, data may be transmitted / received to / from an external device in accordance with a wired communication standard such as USB (Universal Serial Bus).

[Golf Swing Operation]
FIG. 6 is a diagram illustrating an example of a series of golf swing operations and an angular velocity around the Z axis measured by the sensor unit 15 during a series of golf swing operations. As shown in FIG. 6, a series of operations of the golf swing includes address, top, half, impact, and finish points. Here, the address is just before starting takeback, the top is the point to switch from backswing to downswing, the half is the position where the shaft is horizontal during the downswing, the impact is the moment the golf club hits the ball, the finish is follow It means a switching point that returns to a normal posture after a through. When the forward swing direction is positive and the angular velocity around the Z axis of the user U is taken as a reference, the address is just before shifting from near zero to minus, the top turns from minus to plus, the half is near the maximum value, and the impact is maximum The point where a little has passed from the value, the finish is the point where it shifts from plus to near zero or minus.

[Operation of Display System 100]
Next, the operation of the display system 100 in the first embodiment will be described.
First, the operation | movement which acquires a series of moving images of a golf swing is demonstrated.
The user U activates the sensor 1 and the imaging device 2, and causes the sensor 1 and the imaging device 2 to perform time synchronization by a predetermined operation. In addition, the user U activates the golf swing display application of the image processing device 3 in the moving image acquisition mode. Next, the user U places the imaging device 2 at a position where the lens of the imaging device 2 captures the user U, attaches the sensor 1 to the waist, and operates the operation unit 14 to instruct start of measurement. Then, the user U performs a series of golf swing operations.

  When the sensor 1 is instructed to start measurement by operating the operation unit 14, the CPU 11 starts measuring the golf swing motion of the user U by the sensor unit 15 and waits for detection of the address start. Here, as shown in FIG. 6, when the user U starts a backswing from the address toward the top posture, the angular velocity around the Z axis output from the angular velocity sensor of the sensor unit 15 changes from 0 to minus. Therefore, for example, when the angular velocity around the Z-axis output from the sensor unit 15 changes by a predetermined threshold value (for example, −0.3 [rad / s]) in the minus direction, the CPU 11 starts addressing. Judge that it was detected. Then, the CPU 11 transmits an address detection signal to the image processing apparatus 3 through the communication unit 16.

  In the image processing device 3, when the second communication unit 37 receives an address detection signal from the sensor 1, the CPU 31 transmits a shooting start instruction to the imaging device 2 through the second communication unit 37.

  In the imaging device 2, when the imaging start instruction is received by the communication unit 27, the CPU 21 causes the imaging unit 26 to start capturing a moving image at a predetermined frame rate.

In the sensor 1, the CPU 11 waits for the finish to be detected by the sensor unit 15. Here, as shown in FIG. 6, the finish is a point where the angular velocity around the Z-axis output from the angular velocity sensor of the sensor unit 15 shifts from plus to near zero or minus.
Therefore, the CPU 11, for example, when the angular velocity around the Z axis output from the sensor unit 15 after detecting the start of the address becomes a predetermined threshold value (for example, 0.3 [rad / s]) or less from the plus direction, It is determined that a finish has been detected.

  When determining that the finish has been detected by the sensor unit 15, the CPU 11 transmits a finish detection signal to the image processing apparatus 3 through the communication unit 16. In addition, the sensor data acquired in the sensor unit 15 from the start of the address to the finish is transmitted to the image processing apparatus 3 by the communication unit 16.

  In the image processing device 3, when the second communication unit 37 receives a finish detection signal from the sensor 1, the CPU 31 transmits a shooting end instruction to the imaging device 2 through the second communication unit 37. Further, when sensor data from the sensor 1 is received by the second communication unit 37, an identification ID for identifying the data is issued, and the issued identification ID is given and stored in the sensor data storage unit 332.

  In the imaging device 2, when the communication unit 27 receives a shooting end instruction, the CPU 21 causes the imaging unit 26 to stop shooting, and the communication unit 27 uses the communication unit 27 to process the image data (frame image) of the moving image acquired by the shooting. 3 to send. The image data of the moving image is transmitted to the image processing device 3 as a moving image file, for example.

  In the image processing device 3, when image data of a moving image is received from the imaging device 2 by the second communication unit 37, the same identification ID as that given to the sensor data is assigned and stored in the moving image storage unit 333.

  In the image processing device 3, the moving image stored in the moving image storage unit 333 can be displayed by setting the operation mode of the golf swing display application to the moving image display mode.

  FIG. 7 is a flowchart showing a moving image display process A executed in cooperation with the CPU 31 and the golf swing display application in the moving image display mode. Hereinafter, the moving image display processing A will be described with reference to FIG.

  First, the CPU 31 displays a list of moving images stored in the moving image storage unit 333 on the display unit 35, and accepts a moving image selection operation by the user (step S1).

Next, the CPU 31 acquires image data of the selected moving image from the moving image storage unit 333 (step S2).
Next, the CPU 31 acquires sensor data (with the same identification ID) corresponding to the selected moving image from the sensor data storage unit 332 (step S3).

  Next, the CPU 31 analyzes the waveform of the acquired sensor data, and specifies the time of each point (address, top, impact, finish) of the golf swing motion in the moving image (step S4).

  Next, the CPU 31 calculates the time length of each section of the golf swing motion partial motion (back swing, down swing, follow-through) in the moving image from the time of each point (step S5). Here, the back swing section is from the address to the top, the down swing section is from the top to the impact, and the follow through section is from the impact to the finish.

Next, the CPU 31 reads the seek bar image from the storage unit 33 and assigns the calculated time length to each section of the golf swing motion at the seek bar (step S6).
Then, the CPU 31 displays the address frame image and the seek bar in the moving image on the display unit 35 (step S7).

  FIGS. 8A to 8C show examples of seek bars displayed on the display unit 35 in step S6. FIG. 8A is a diagram showing a bar-shaped seek bar. FIG. 8B is a diagram showing a triangular seek bar. FIG. 8C is a diagram showing a seek bar having a shape representing an operation image of a golf swing operation.

  As shown in FIGS. 8A to 8C, the seek bar is divided into a plurality of sections corresponding to each section of the golf swing movement (back swing section, down swing section, follow-through section). The position and name are displayed in an identifiable manner. The seek bar also displays the position of each point (address, top, impact, finish) in the golf swing operation in an identifiable manner. Each point overlaps a section break. The seek bar is provided with a knob T1. The default position of the knob T1 is an address position, and the position of the frame image to be displayed in the moving image on the time axis is moved by the user moving the knob T1 by a finger, a touch pen, or a physical key of the operation unit 34. Can be specified.

  As shown in FIGS. 8A to 8C, each section and each point of the golf swing motion are displayed on the seek bar so as to be identifiable, so that the user can perform golf swing motion such as top and impact on the seek bar. The point position and each section can be easily grasped, and the user can easily specify the position on the time axis of the frame image to be displayed from the series of moving images.

  Further, when the seek bar is a straight line, it may be difficult to stop the knob T1 at a point (section break) such as the top or impact, and it may be difficult to specify a desired frame image. However, as shown in FIG. By providing a corner for each point (section break), it is possible to prevent the knob T1 from moving to the next section past the point even though the knob T1 is moved to the point. The position can be easily specified.

  Moreover, in order to receive the operation by the user in correspondence with the length of each section of the seek bar shown in FIG. The user can easily specify a desired frame image even in an operation period in which the movement is fast and the number of frame images taken is small. With the above configuration, if the display is performed by simply matching the number of images included in each section of the partial motion and the length of each section of the seek bar, the section corresponding to the impact in the seek bar becomes extremely short, and the user It is possible to solve the problem that display becomes difficult to select.

  In addition, as shown in FIG. 8C, by making the seek bar a shape that expresses the image of the golf swing operation, the user can intuitively grasp the position of each point and each section on the seek bar, The user can easily specify the position on the time axis of the frame image to be displayed from the series of moving images.

  Next, the CPU 31 determines whether or not a movement operation of the knob T1 has been detected by the operation unit 34 (step S8). When it is determined that the movement operation of the knob T1 by the operation unit 34 is not detected (step S8; NO), the CPU 31 proceeds to step S11.

  When it is determined that the movement operation of the knob T1 is detected by the operation unit 34 (step S8; YES), the CPU 31 determines the distance from the start position of the movement destination section of the knob T1 to the position of the knob T1 on the seek bar, and the movement destination. A frame image corresponding to the position of the knob T1 is selected from the moving images based on the time length assigned to the section (step S9). That is, the CPU 31 selects a frame image to be displayed from among the moving images in accordance with a change in the display mode of the seek bar by a user operation (here, a change in the position of the knob T1). Then, the CPU 31 displays the moving image after the selected frame image on the display unit 35 (step S10), and proceeds to step S11.

In step S11, the CPU 31 determines whether or not a moving image display end instruction is detected by the operation unit 34 (step S11).
When it is determined by the operation unit 34 that an instruction to end moving image display has not been detected (step S11; NO), the CPU 31 returns to step S8.
When it is determined by the operation unit 34 that an instruction to end moving image display has been detected (step S11; YES), the CPU 31 ends the moving image display process A.

<Second Embodiment>
Next, a second embodiment of the present invention will be described.
In the second embodiment, the storage unit 33 stores an image of the rotation angle controller (see FIG. 10).
Since the system configuration, the configuration of each device, and the moving image acquisition operation in the other second embodiment are the same as those described in the first embodiment, the description is used, and hereinafter, the moving image in the second embodiment. Display processing (referred to as moving image display processing B) will be described.

  FIG. 9 is a flowchart showing a moving image display process B executed in cooperation with the CPU 31 and the golf swing display application in the moving image display mode. Hereinafter, the moving image display process B will be described with reference to FIG.

  First, the CPU 31 displays a list of moving images stored in the moving image storage unit 333 on the display unit 35, and accepts a moving image selection operation by the user (step S21).

Next, the CPU 31 acquires image data of the selected moving image from the moving image storage unit 333 (step S22).
Next, the CPU 31 acquires sensor data (with the same identification ID) corresponding to the selected moving image from the sensor data storage unit 332 (step S23).

  Next, the CPU 31 analyzes the waveform of the acquired sensor data, and specifies the time of each point (address, top, impact, finish) of the golf swing motion in the moving image (step S24).

  Next, the CPU 31 calculates the rotation angle around the Z axis of the waist of the user U at each time of the moving image by integrating the angular velocity around the Z axis of the sensor data from the top of the sensor data (step S25).

  Next, the CPU 31 reads the image of the rotation angle controller from the storage unit 33, and causes the display unit 35 to display the address frame image and the rotation angle controller in the moving image (step S26).

  FIG. 10 is a diagram illustrating an example of the rotation angle controller displayed on the display unit 35 in step S26. The rotation angle controller is an operation image for the user to specify and operate the rotation angle of the user's waist in the frame image to be displayed among the selected moving images. The rotation angle controller can be rotated in accordance with a user operation, for example, an elliptical controller 351 that gives an image of a waist, and axes indicating rotation angles of 0 °, 90 °, 180 ° (−180 °), and −90 °. 352. On the controller 351, a reference mark 351a is displayed. When the position where the reference mark 351a coincides with 0 ° is the position where the rotation angle of the controller 351 is 0 °, and the user rotates the controller 351, the waist of the moving image is rotated by the rotation angle. This frame image can be designated as a frame image to be displayed. Note that when the controller 351 is rotated clockwise, the rotation angle is displaced in the minus direction, and when the controller 351 is rotated counterclockwise, the rotation angle is displaced in the plus direction. FIG. 10A shows the controller 351 when the rotation angle is 0 °, and FIG. 10B shows the controller 351 when the rotation angle is −45 °.

  Thus, by displaying the controller 351 that rotates according to the user operation and inputs the rotated angle as the rotation angle as the operation image for designating the rotation angle of the user U in the frame image to be displayed, the user Can be intuitively designated as an image to display a frame image of a desired posture.

Next, the CPU 31 determines whether or not a rotation operation of the controller 351 is detected by the operation unit 34 (step S27).
When it is determined that the rotation operation of the controller 351 has not been detected (step S27; NO), the process proceeds to step S31.

  When determining that the rotation operation of the controller 351 has been detected (step S27; YES), the CPU 31 acquires the rotation angle and the rotation direction of the controller 351 (step S28).

Next, the CPU 31 selects a frame image from the moving images based on the acquired rotation angle and rotation direction (step S29).
Here, there is a moment when the hip rotation angle is the same during backswing and downswing, but during backswing it rotates in the negative direction (clockwise) and during downswing it is in the positive direction (counterclockwise). Rotate. Therefore, the frame image designated by the user can be specified from the moving images by the rotation angle and the rotation direction.

  Then, the CPU 31 displays the moving image after the selected frame image on the display unit 35 (step S30), and proceeds to step S31.

In step S31, the CPU 31 determines whether or not a moving image display end instruction is detected by the operation unit 34 (step S31).
When it is determined that the moving image display end instruction has not been detected by the operation unit 34 (step S31; NO), the CPU 31 returns to step S27 and repeatedly executes the processes of steps S27 to S31.
When it is determined by the operation unit 34 that an instruction to end moving image display has been detected (step S31; YES), the CPU 31 ends the moving image display process B.

  As described above, in the second embodiment, the rotation angle controller is displayed as an operation image for the user to specify and operate the frame image to be displayed. Therefore, the image that displays the frame image of the posture desired by the user intuitively. Can be specified as In addition, since the frame image to be displayed can be switched according to the rotation operation of the controller 351a by the user, the user can display the image of the attention posture at a desired speed by adjusting the speed of rotating the controller 351a. Can be viewed.

  The first and second embodiments of the present invention have been described above. However, the description in the above embodiment is a preferred example of the display system according to the present invention, and the present invention is not limited to this.

  For example, in the above-described embodiment, the case where the sensor 1 and the image processing apparatus 3 are separate has been described as an example. However, for example, a smartphone including a 3-axis acceleration sensor, a 3-axis gyro sensor, a 3-axis geomagnetic sensor, and the like The sensor 1 and the image processing device 3 may be integrally configured. Further, for example, the imaging device 2 and the image processing device 3 may be configured integrally, such as a smartphone equipped with a camera.

  In the above embodiment, the operation image and the moving image are displayed on the display unit 35 of the image processing apparatus 3. However, the operation image and the moving image may be displayed on a separate display device. Good.

  In the first embodiment, after a moving image is selected in the image processing device 3, the sensor data corresponding to the moving image is analyzed to determine the reproduction time of each point in the moving image and the time length of each section. As described above, the calculation is performed when the moving image is selected for the first time or in advance, and the reproduction time of each point and the time length of each section in the calculated moving image are associated with the moving image. It is good also to memorize. When a moving image is selected, the playback time of each point and the time length of each section stored in association with the moving image may be read and assigned to the seek bar.

  In the second embodiment, after a moving image is selected in the image processing device 3, the waist Z of the user U (subject) at each time of the moving image is based on sensor data corresponding to the moving image. Although the rotation angle around the axis is calculated, the calculation of the rotation angle is performed when the moving image is selected for the first time or in advance, and the calculated rotation angle is stored in association with the moving image. It is good also to leave. When a moving image is selected, the rotation angle at each time stored in association with the moving image may be read out and used for designating a frame image by the rotation angle controller.

  Further, in the second embodiment, it has been described that one rotatable controller 151 is displayed as the operation image and a frame image is selected from the moving image according to the rotation operation of the controller 151. The CPU 31 has a plurality of images representing a state in which the controller 151 is rotated at a plurality of rotation angles (for example, a plurality of images representing a state in which the controller 151 is rotated at a rotation angle corresponding to each point of address, top, impact, and finish). Image) may be displayed as an operation image, and when any image is designated and operated by the user, a frame image having a rotation angle corresponding to the designated image may be selected as a frame image to be displayed. As a result, the frame image of the point to be displayed can be specified by the user with a simpler operation.

  Moreover, in the said embodiment, although demonstrated as mounting the sensor 1 to the user's U waist, it is not limited to this. For example, the sensor 1 may be a wrist device such as a smart watch. Moreover, you may mount | wear with another site | part (a head, an arm, a wrist). Since it is easy, it is desirable to wear it on the body, but it is also possible to wear it on a golf club. Moreover, you may mount | wear with the sensor 1 in several places. For example, the sensor 1 may be attached to the waist and back. Since the torsional difference between the waist and shoulder is an important parameter in various competitions, it is effective.

  In the above embodiment, the sensor data is directly acquired from the sensor 1, but may be acquired via an external network.

  In the above embodiment, the sensor data is analyzed to calculate the time of each point of the moving image and the time length of each section. However, each frame image of the moving image is analyzed and calculated. It is good. Similarly, the waist rotation angle in each frame image of the moving image may be obtained by analyzing each frame image.

  In the above embodiment, the present invention has been described by taking an example in which a moving image obtained by shooting a golf swing motion is displayed. However, the present invention is not limited to this, and other motions such as a swing motion such as tennis or a gate ball are performed. The present invention can also be applied when displaying a captured moving image.

  In addition, the detailed configuration and detailed operation of each device constituting the display system can be appropriately changed without departing from the spirit of the invention.

Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
Moving image acquisition means for acquiring a moving image showing a series of movements of the subject;
Among the moving images acquired by the moving image acquisition means, an operation image for a user to specify and operate a position on a time axis of a frame image to be displayed, and each of the partial operations included in the series of operations Display control means for causing the display means to display the operation image in which the corresponding section is displayed in an identifiable manner;
Image selecting means for selecting a frame image to be displayed from the moving images in accordance with a change in display mode of the operation image by a user's specifying operation;
An image processing apparatus comprising:
<Claim 2>
The display control means further causes the display means to display the operation image on which specific points included in the series of operations are displayed in an identifiable manner,
The image processing apparatus according to claim 1.
<Claim 3>
The operation image has a shape having a corner for each section break,
The image processing apparatus according to claim 1, wherein:
<Claim 4>
The operation image is a shape having a corner for each specific point,
The image processing apparatus according to claim 2.
<Claim 5>
The operation image has a shape representing an operation image of the series of operations;
The image processing apparatus according to claim 1, wherein:
<Claim 6>
Moving image acquisition means for acquiring a moving image indicating a rotation operation of the subject;
Rotation angle acquisition means for acquiring information on the rotation angle of the subject in each frame image of the moving image;
Display control means for causing the display means to display an operation image for the user to specify the rotation angle of the subject in the frame image to be displayed among the moving images acquired by the moving image acquisition means;
Image selecting means for selecting a frame image to be displayed from the moving images in response to a designation operation on the operation image by a user;
An image processing apparatus comprising:
<Claim 7>
The operation image can be rotated according to a user operation, and an angle at which the user rotates the operation image can be designated as a rotation angle of the subject in the frame image to be displayed.
The image selecting means selects a frame image to be displayed from the moving images in response to a rotation operation of the operation image by a user;
The image processing apparatus according to claim 6.
<Claim 8>
The display control unit causes the display unit to display a plurality of images representing a state in which a predetermined image is rotated at a plurality of rotation angles as the operation image,
The image selecting means selects a frame image to be displayed from the moving images according to an image designated and operated by a user among the plurality of images in the operation image;
The image processing apparatus according to claim 6.
<Claim 9>
The display control means acquires the frame image selected by the image selection means from the moving images and causes the display means to display the frame image;
The image processing apparatus according to claim 1, wherein:
<Claim 10>
Acquiring a moving image showing a series of movements of the subject;
Among the acquired moving images, an operation image for a user to specify and operate a position on a time axis of a frame image to be displayed, and a section corresponding to each of the partial operations included in the series of operations is identified. A step of causing the display means to display the operation image displayed in a possible manner;
Selecting a frame image to be displayed from the moving images according to a change in the display mode of the operation image by a user's specifying operation;
Including an image processing method.
<Claim 11>
Obtaining a moving image showing the rotation of the subject;
Obtaining information on the rotation angle of the subject in each frame image of the moving image;
A step of causing a display unit to display an operation image for a user to designate and specify a rotation angle of the subject in a frame image to be displayed among the acquired moving images;
A step of selecting a frame image to be displayed from among the moving images in response to a designation operation on the operation image by a user;
Including an image processing method.
<Claim 12>
Computer
Moving image acquisition means for acquiring a moving image indicating a series of movements of the subject;
Among the moving images acquired by the moving image acquisition means, an operation image for a user to specify and operate a position on a time axis of a frame image to be displayed, and each of the partial operations included in the series of operations Display control means for displaying on the display means the operation image in which the corresponding section is displayed in an identifiable manner;
Image selecting means for selecting a frame image to be displayed from the moving images in accordance with a change in display mode of the operation image by a user's specifying operation;
Program to function as.
<Claim 13>
Computer
Moving image acquisition means for acquiring a moving image indicating the rotation of the subject;
Rotation angle acquisition means for acquiring information on the rotation angle of the subject in each frame image of the moving image;
Display control means for causing the display means to display an operation image for the user to specify the rotation angle of the subject in the frame image to be displayed among the moving images acquired by the moving image acquisition means;
Image selecting means for selecting a frame image to be displayed from the moving images in response to a designation operation on the operation image by a user;
Program to function as.

100 Display system 1 Sensor 11 CPU
12 RAM
13 Storage Unit 14 Operation Unit 15 Sensor Unit 16 Communication Unit 17 Timekeeping Unit 18 Bus 2 Imaging Device 21 CPU
22 RAM
23 Storage Unit 24 Operation Unit 25 Display Unit 26 Imaging Unit 27 Communication Unit 28 Timekeeping Unit 29 Bus 3 Image Processing Device 31 CPU
32 RAM
33 storage unit 331 program storage unit 332 sensor data storage unit 333 moving image storage unit 34 operation unit 35 display unit 36 first communication unit 37 second communication unit 38 bus

Claims (13)

Moving image acquisition means for acquiring a moving image showing a series of movements of the subject;
Among the moving images acquired by the moving image acquisition means, an operation image for a user to specify and operate a position on a time axis of a frame image to be displayed, and each of the partial operations included in the series of operations Display control means for causing the display means to display the operation image in which the corresponding section is displayed in an identifiable manner;
Image selecting means for selecting a frame image to be displayed from the moving images in accordance with a change in display mode of the operation image by a user's specifying operation;
An image processing apparatus comprising: The display control means further causes the display means to display the operation image on which specific points included in the series of operations are displayed in an identifiable manner,
The image processing apparatus according to claim 1. The operation image has a shape having a corner for each section break,
The image processing apparatus according to claim 1, wherein: The operation image is a shape having a corner for each specific point,
The image processing apparatus according to claim 2. The operation image has a shape representing an operation image of the series of operations;
The image processing apparatus according to claim 1, wherein: Moving image acquisition means for acquiring a moving image indicating a rotation operation of the subject;
Rotation angle acquisition means for acquiring information on the rotation angle of the subject in each frame image of the moving image;
Display control means for causing the display means to display an operation image for the user to specify the rotation angle of the subject in the frame image to be displayed among the moving images acquired by the moving image acquisition means;
Image selecting means for selecting a frame image to be displayed from the moving images in response to a designation operation on the operation image by a user;
An image processing apparatus comprising: The operation image can be rotated according to a user operation, and an angle at which the user rotates the operation image can be designated as a rotation angle of the subject in the frame image to be displayed.
The image selecting means selects a frame image to be displayed from the moving images in response to a rotation operation of the operation image by a user;
The image processing apparatus according to claim 6. The display control unit causes the display unit to display a plurality of images representing a state in which a predetermined image is rotated at a plurality of rotation angles as the operation image,
The image selecting means selects a frame image to be displayed from the moving images according to an image designated and operated by a user among the plurality of images in the operation image;
The image processing apparatus according to claim 6. The display control means acquires the frame image selected by the image selection means from the moving images and causes the display means to display the frame image;
The image processing apparatus according to claim 1, wherein: Acquiring a moving image showing a series of movements of the subject;
Among the acquired moving images, an operation image for a user to specify and operate a position on a time axis of a frame image to be displayed, and a section corresponding to each of the partial operations included in the series of operations is identified. A step of causing the display means to display the operation image displayed in a possible manner;
Selecting a frame image to be displayed from the moving images according to a change in the display mode of the operation image by a user's specifying operation;
Including an image processing method. Obtaining a moving image showing the rotation of the subject;
Obtaining information on the rotation angle of the subject in each frame image of the moving image;
A step of causing a display unit to display an operation image for a user to designate and specify a rotation angle of the subject in a frame image to be displayed among the acquired moving images;
A step of selecting a frame image to be displayed from among the moving images in response to a designation operation on the operation image by a user;
Including an image processing method. Computer
Moving image acquisition means for acquiring a moving image indicating a series of movements of the subject;
Among the moving images acquired by the moving image acquisition means, an operation image for a user to specify and operate a position on a time axis of a frame image to be displayed, and each of the partial operations included in the series of operations Display control means for causing the display means to display the operation image in which the corresponding section is displayed in an identifiable manner;
Image selecting means for selecting a frame image to be displayed from the moving images in accordance with a change in display mode of the operation image by a user's specifying operation;
Program to function as. Computer
Moving image acquisition means for acquiring a moving image indicating the rotation of the subject;
Rotation angle acquisition means for acquiring information on the rotation angle of the subject in each frame image of the moving image;
Display control means for causing the display means to display an operation image for the user to specify the rotation angle of the subject in the frame image to be displayed among the moving images acquired by the moving image acquisition means;
Image selecting means for selecting a frame image to be displayed from the moving images in response to a designation operation on the operation image by a user;
Program to function as.

Images