JP2014186167A - Display control device, display control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To display a change in attitude of an object without using a specific device such as a load sensor and the like.SOLUTION: An information display device 1 includes a point selection part 44, an indicator generation part 46 and an indicator display control part 47. The point selection part 44 selects plural points constituting a reference of an action in a start image of a video capturing an action of a golf swing of a player. The indicator generation part 46 respectively identifies plural points in an image of a series of swing, the points corresponding to the selected plural points, and generates an indicator diagram containing the plural points for the respectively identified plural points. The indicator display control part 47 causes a display part 17 to display the generated indicator diagram.

Description

  The present invention relates to a display control device, a display control method, and a program.

  Conventionally, as a technique for displaying a change in the posture of a subject based on a moving image of a golf player as a subject, a load sensor is provided on the board to display the state of the subject (subject) during a golf swing operation. The technique to do is known (for example, refer patent document 1).

JP-A-5-76634

  However, the technique described in Patent Document 1 requires a large-scale device such as a board provided with a load sensor, so that there is a problem that a place for displaying the state of the subject is limited.

  The present invention has been made in view of such a situation, and an object thereof is to grasp the state of a subject without using a special device such as a load sensor.

In order to achieve the above object, a display control device according to one embodiment of the present invention includes:
Selecting means for selecting a plurality of points serving as a reference for the series of actions in a predetermined image of the series of actions of the subject;
First identifying means for respectively identifying a plurality of points in the image of the series of operations corresponding to the plurality of points selected by the selecting means;
Index generating means for generating an index including the plurality of points for each of the plurality of points specified by the first specifying means;
Display control means for executing control to display each index generated by the index generation means on the display means;
Is provided.

  According to the present invention, the state of a subject can be grasped without using a special device such as a load sensor.

It is a block diagram which shows the hardware constitutions of the information display apparatus which concerns on one Embodiment of this invention. It is a functional block diagram which shows the functional structure for performing a 1st parameter | index display process among the functional structures of the information display apparatus of FIG. It is a figure which shows the position of the marking image in the operation | movement of a series of golf swings. It is a figure which shows a mode that the point in a start image was designated. It is a figure which shows the example of a display of the moving image on which the index figure was superimposed for every frame. It is a flowchart which shows the flow of the 1st parameter | index display process which the information display apparatus of FIG. 1 which has the function structure of FIG. 2 performs. It is a functional block diagram which shows the functional structure for performing a 2nd parameter | index display process among the functional structures of the information display apparatus of FIG. It is a figure which shows a mode that the point in a start image was designated. It is a figure which shows the example of a display of the moving image on which the index shape was superimposed for every flame | frame. It is a figure which shows a mode that the some indicator shape in each of several frames is superimposed on one screen of the display part 17, and is displayed simultaneously. It is a figure which shows a mode that the image of incidental information is displayed on the position screen of the display part 17 with the image of a parameter | index shape. It is a flowchart which shows the flow of the 2nd parameter | index display process which the information display apparatus of FIG. 1 which has the function structure of FIG. 7 performs. It is a figure explaining the example of the technique of selection of a some point. It is a figure explaining the example of the technique of selection of a some point.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
Hereinafter, in the description of the first embodiment, the present embodiment refers to the first embodiment.
[Constitution]
FIG. 1 is a block diagram showing a hardware configuration of the information display apparatus according to the first embodiment of the present invention.

  The information display device 1 is configured as a digital camera, for example, and includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, An input unit 16, a display unit 17, a storage unit 18, a communication unit 19, an imaging unit 20, and a drive 21 are provided.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 to the RAM 13.

  The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

  The CPU 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output interface 15 is also connected to the bus 14. An input unit 16, a display unit 17, a storage unit 18, a communication unit 19, an imaging unit 20, and a drive 21 are connected to the input / output interface 15.

The input unit 16 includes, for example, a capacitance type or resistive type position input sensor stacked on the display screen of the display unit 17, and detects the coordinates of the position where the touch operation is performed. Here, the touch operation refers to an operation of contact or proximity of an object (such as a user's finger or a touch pen) with respect to the input unit 16.
The display unit 17 includes a display and displays an image.
That is, in the present embodiment, the input unit 16 and the display unit 17 constitute a touch panel.
The storage unit 18 is composed of a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores various image data.
The communication unit 19 controls communication with other devices (not shown) via a network including the Internet.
The imaging unit 20 images a subject and supplies a digital signal (image signal) of an image including the image of the subject (hereinafter referred to as “captured image”) to the CPU 11. Here, the digital signal (image signal) of the captured image is appropriately referred to as “captured image data”.

  A removable medium 31 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 21. The program read from the removable medium 31 by the drive 21 is installed in the storage unit 18 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 18 in the same manner as the storage unit 18.

FIG. 2 is a functional block diagram showing a functional configuration for executing the first index display process among the functional configurations of the information display device 1.
The first index display process is, for example, a graphic image serving as an index indicating weight shift of a subject when displaying a moving image (including a continuous shot image) in which a person performing a golf swing motion is captured as a subject. This means a series of processes that are performed until each frame constituting a moving image is displayed in association with each frame.
Hereinafter, a figure that serves as an index indicating the weight shift of the subject is referred to as an “index figure”. In the present embodiment, a triangle formed on the crotch of the subject is employed as the index graphic.

  When the first index display process is executed, as shown in FIG. 2, in the CPU 11, a moving image acquisition unit 41, a moving image analysis unit 42, a start image display control unit 43, a point selection unit 44, The reproduction control unit 45, the index generation unit 46, the index display control unit 47, the comparison unit 48, the notification information generation unit 49, and the notification control unit 50 function.

  However, the functional configuration of FIG. 2 is an exemplification. For example, when the information display device 1 includes hardware for performing image processing such as GA (Graphics Accelerator), at least a part of each function of the above-described functional blocks. May be delegated from the CPU 11 to the hardware.

When the first index display process is executed, the moving image storage unit 61 and the comparison information storage unit 62 provided as one area of the storage unit 18 are used.
The moving image storage unit 61 stores data of one or more moving images (moving images composed of a plurality of frames) in advance. The moving image data stored in the moving image storage unit 61 may be captured image data captured by the imaging unit 20, or may be transmitted from another device (not shown) and received by the communication unit 19. The image data may also be used. In the present embodiment, the content of the moving image is a series of golf swings in a golf player.
The comparison information storage unit 62 stores in advance the index graphic data created based on the image of a series of swing movements by a golf expert such as a professional golfer in association with each image of the series of swing movements. Has been.

  The moving image acquisition unit 41 acquires moving image data stored in the moving image storage unit 61. Specifically, in the present embodiment, moving image data is stored in the moving image storage unit 61 in a state of being encoded in units of frames. Therefore, the moving image acquisition unit 41 decodes the moving image data stored in the moving image storage unit 61 in units of frames, and acquires the decoded moving image data.

  The moving image analysis unit 42 analyzes the data of the moving image acquired by the moving image acquisition unit 41, and thus, in the moving image, the operation from the setup to the top is shown in the section where the golf swing operation is reflected. A section (hereinafter referred to as “first section”) is distinguished from a section in which an operation from the top to the finish is reflected (hereinafter referred to as “second section”).

The method for identifying the first section and the second section is not particularly limited, but in the present embodiment, the technique shown in FIG. 3 is adopted.
FIG. 3 is a diagram illustrating a method for identifying the first section and the second section, and is a diagram illustrating the position of the marking image in a series of golf swing operations.
That is, in this embodiment, a golf player who appears as a subject in a moving image acquired by the moving image acquisition unit 41 performs a golf swing operation with a marking material wound around the back of the left hand at the time of imaging by the camera. It is assumed that FIG. 3 shows a state in which several representative frames among a plurality of frames constituting a moving image captured by the camera in this case are arranged according to the display order. The marking material only needs to be detectable by the camera, but in this embodiment, the color difference of the contrast difference is large so that the contrast difference is detected at maximum in the screen by the image pickup device of the camera. Marking material is used.
As shown in FIG. 3, each frame includes a marking image 91 indicating a marking material, and the moving image analysis unit 42 recognizes the spatial position of the marking image 91 in the frame for each frame. Shall be able to.
In this case, the moving image analysis unit 42 specifies the frame immediately before the marking image 91 is stationary for a predetermined time from the start of the moving image and starts moving upward as a setup frame. Further, the moving image analysis unit 42 specifies the frame at the time when the marking image 91 first reaches the maximum value as the top frame in the golf swing.
Here, the maximum value is, for example, when the left lower end of the frame is the origin, the horizontal axis is the X axis, and the vertical axis is the Y axis, the marking image 91 is in a series of consecutive frames. This is the Y coordinate value at the time of change in the moving direction when moving in the Y axis direction and then decreasing, that is, the maximum Y coordinate value during movement.
Further, the moving image analysis unit 42 identifies the frame at the time when the maximum value is reached next as the frame of the finish in the golf swing.
Then, as shown in FIG. 3, the moving image analysis unit 42 identifies the section from the setup frame to the top frame as the first section, and identifies the section from the top frame to the finish frame as the second section. To do.

  Returning to FIG. 2, the start image display control unit 43 is a still image (hereinafter referred to as a golf swing) starting from the setup frame among the frames constituting the moving image analyzed by the moving image analysis unit 42. , Referred to as “start image”).

FIG. 4 shows an example of a start image when the first index display process is executed.
In a state where the start image as shown in FIG. 4 is displayed on the display unit 17, the user inputs a plurality of points (hereinafter referred to as “point group”) necessary for forming the index graphic to the input unit 16. It is specified by touch operation on.
Thus, a plurality of points can be arbitrarily designated by the user. For example, it may be specified as three points of the player's shoulders and the joints of both hands. However, in the present embodiment, as described above, the triangle formed at the inseam of the subject is adopted as the index graphic, so three points for forming the triangle, specifically, in the example of FIG. Assume that a set of three points 92 in which the player's (subject) crotch and both ankles are displayed is designated as a point group.

Returning to FIG. 2, the point selection unit 44 receives an input signal from the input unit 16 (a signal indicating coordinates where the user's touch operation has been performed) and receives a plurality of points (in the example of FIG. 4) designated by the user. A set of three points 92) is recognized, and a plurality of recognized points are selected as a point group in the start image.
Note that the point selection unit 44 may generate a partial image (a symbol image such as a black circle) indicating the selected point group and display the partial image superimposed on the start image. That is, the point selection unit 44 may cause the display unit 17 to display a start image on which partial images showing a set 92 of three points are superimposed as shown in FIG. Thereby, the user can easily visually recognize the points for forming the index graphic.
Note that the method of selecting a point group by the point selection unit 44 is not particularly limited to the above-described method of selecting a set of a plurality of points designated by the user. For example, the point selection unit 44 creates index graphic candidates based on a plurality of points designated by the user, appropriately processes the candidates, completes the index graphic, and forms the index graphic Points (corrected points) may be selected as a point group. Further, for example, when a crotch detector (not shown) is mounted on the information display device 1, the point selection unit 44 is formed by a line segment connecting the player's crotch and both ankles in the start image. A triangle may be detected and each vertex of the triangle may be selected as a point group.

  Returning to FIG. 2, the playback control unit 45 includes a moving image in a period from a start image in which a point group is selected by the point selection unit 44 to a finish frame among the moving images acquired by the moving image acquisition unit 41. Control playback.

When the reproduction control unit 45 starts reproduction of the moving image, the index generation unit 46 generates index graphic image data for each frame.
First, the index generation unit 46 generates image data of an index graphic in the start image based on the point group selected by the point selection unit 44.
Thereafter, the index generation unit 46 generates image data of the index graphic by following a plurality of points corresponding to the selected point group for each frame constituting the moving image.
For example, in the present embodiment, the index generation unit 46 applies a known image matching technique to follow a plurality of points that move as the moving image is reproduced as points corresponding to the selected point group. Go. Specifically, for example, the index generation unit 46 sequentially sets a plurality of points corresponding to the previous frame as adjacent points in the frames adjacent in the reproduction order, and each point in the attention point group of the previous frame. An area composed of peripheral pixel groups including the center is specified. Thereafter, the index generation unit 46 searches the subsequent frame for a region that matches each identified region, and recognizes the center of each retrieved region as each point in the attention point group of the subsequent frame. Then, the index generation unit 46 generates data of an image within the range connecting the target point group as index graphic data in a subsequent frame.

The index display control unit 47 combines the index graphic data generated for each frame by the index generation unit 46 with the playback target moving image data, thereby displaying a moving image in which the index graphic is superimposed for each frame. Control to be displayed on the unit 17 is executed.
FIG. 5 shows a display example of a moving image in which index figures are superimposed for each frame.
As shown in FIG. 5, the index display control unit 47 divides the moving image on which the index image is superimposed on the display unit 17 for each of the first section and the second section identified by the moving image analysis unit 42. It may be displayed.
The index display control unit 47 only needs to display an image of the index graphic on the display unit 17, and does not necessarily display the index graphic and the moving image at the same time. For example, although not shown in the figure, the index display control unit 47 extracts some representative frames from the frames on which the index graphic is superimposed for each of the first section and the second section. It may be displayed on the display unit 17 on one screen, or may be sequentially displayed on the display unit 17 as a continuous image (an image in which still images are sequentially reproduced). For example, although not illustrated, the index display control unit 47 may sequentially display only the index graphic on the display unit 17 as a continuous image for each of the first section and the second section.

  Returning to FIG. 2, while the playback control unit 45 is playing back the moving image, the processing of the index generation unit 46 and the index display control unit 47 is repeatedly executed for each frame to be played back. As described above, the index graphic data is generated for each frame, and the index graphic is displayed.

The comparison unit 48 uses the image data of the index graphic created based on the image of a series of swing motions of a golfer such as a professional golfer stored in the comparison information storage unit 62 as a series of swing motions. Read out for each image.
Further, the comparison unit 48 compares each data of the index graphic for each frame generated by the index generation unit 46 and the index graphic for each image in a series of swing operations read from the comparison information storage unit 62.
At this time, the comparison unit 48 may divide the motion section of the golf swing into the first section and the second section and compare the data of the index figures of both.

The notification information generation unit 49 generates notification information according to the comparison result by the comparison unit 48. For example, the notification information generation unit 49 generates notification information for advising the player in accordance with the difference in shape between the index graphic generated by the index generation unit 46 and the index graphic read from the comparison information storage unit 62. To do. As an example of the notification information, for example, “it is better to put weight on the right in the whole swing” is considered.
At this time, the notification information generation unit 49 may create notification information for giving advice to the player separately for each of the first section and the second section. As an example of notification information in this case, for the takeback period, “It is better to put the weight a little more right in the takeback”, etc. For the second section, “the weight is a little left in the downswing” It ’s a good idea.

  The notification control unit 50 displays the notification information generated by the notification information generation unit 49 on the display unit 17 as an image, thereby executing control for notifying the user who views the display unit of the notification information.

[First index display processing]
Next, the first index display process executed by the information display device 1 will be described.
FIG. 6 is a flowchart showing an example of the flow of the first index display process executed by the information display device 1 of FIG. 1 having the functional configuration of FIG.

  In FIG. 6, when the input unit 16 of the information display device 1 is operated in order to start acquisition of moving image data, the following steps S <b> 1 to S <b> 11 are executed.

  In step S <b> 1, the moving image acquisition unit 41 acquires moving image data stored in the moving image storage unit 61.

In step S2, the moving image analysis unit 42 analyzes the moving image data acquired in step S1. Based on the analysis result, the moving image analysis unit 42 identifies frames corresponding to the setup, top, and finish of the golf swing. Then, the moving image analysis unit 42 identifies the operation interval from the setup to the top as the first interval and the operation interval from the top to the finish as the second interval based on the specific result.
In step S3, the start image display control unit 43 causes the display unit 17 to display the setup frame identified in step S2 as a start image.

  In step S4, the point selection unit 44 starts the start image displayed by the start image display control unit 43 based on the content of the touch operation on the input unit 16 by the user (the screen position of the display unit 17 where the touch operation has been performed). Select a point group.

  In step S5, the reproduction control unit 45 starts reproduction of the moving image acquired in step S1 from the start image from which the point group has been selected in step S4.

In step S6, the index generation unit 46 sets a frame to be played back as a target frame among a plurality of frames constituting the moving image that has been played back in step S5, and sets the index graphic image in the target frame. Generate data.
In step S7, the index display control unit 47 causes the display unit 17 to display the index graphic superimposed on the frame of interest.

In step S8, the reproduction control unit 45 determines whether or not the reproduction should be terminated.
Here, in the present embodiment, the reproduction control unit 45 determines whether or not a finish frame is displayed.
If the finish frame has not yet been displayed, it is determined as NO in step S8, the process returns to step S6, and the subsequent processes are repeated. That is, the next frame to be reproduced is set as the attention frame, index image data in the attention frame is generated (step S6), and the index image is displayed superimposed on the attention frame (step S7).
When the finish frame on which the index image is superimposed is displayed, it is determined as YES in Step S8, and the process proceeds to Step S9.

  In step S <b> 9, the comparison unit 48 compares the index graphic between the moving image acquired in step S <b> 1 and a series of swing motion images of a golfer such as a professional golfer.

In step S10, the notification information generation unit 49 generates notification information corresponding to the comparison result in step S9.
In step S11, the notification control unit 50 performs notification by causing the display unit 17 to display the notification information generated in step S10 as an image.
As a result, the first index display process ends.

As described above, in the first embodiment, a moving image showing a golf swing motion of a player (subject) is adopted as an image of a series of motions of the subject. The information display device 1 according to the first embodiment includes a point selection unit 44, an index generation unit 46, and an index display control unit 47.
The point selection unit 44 selects a start image as a predetermined image among the moving images of the golf swing motion of the player, and selects a plurality of points as reference points for the player's motion in the start image as a point group.
The index generation unit 46 generates index graphic data for each frame after the start image constituting the moving image based on the index graphic data including the selected point group.
The index display control unit 47 causes the display unit 17 to display the index graphic generated for each frame.
Therefore, the information display device 1 according to the first embodiment can display the change in the posture of the player as the change in the index graphic without using a special device such as a load sensor.

The index display control unit 47 performs control for superimposing each index graphic generated by the index generating unit 46 on a series of motion images (for example, moving images) corresponding to each index graphic and displaying them on the display unit 17. To do.
Therefore, the information display device 1 according to the first embodiment can display a change in the posture of the player as a change in the indicator graphic along with the swing motion of the player. Therefore, the player can grasp the change in posture in comparison with the actual swing motion.

The information display device 1 further includes a moving image analysis unit 42.
The moving image analysis unit 42 identifies a specific motion section (the first section and the second section in the above example) from the golf swing motion.
In this case, the index display control unit 47 can cause the display unit 17 to display an index graphic in a specific motion section.
Therefore, since the information display device 1 according to the first embodiment can display the change of the index graphic for each specific motion section, the user who views the display can play the player (even in the case of the user himself) in the specific motion section. Change in posture) can be easily grasped.

The information display device 1 further includes a notification control unit 50.
The notification control unit 50 performs control for notifying notification information corresponding to the index graphic generated by the index generation unit 46.
As notification information in this case, for example, advice information for a player based on a comparison between an index graphic for a swing motion of an expert such as a professional golfer and an index graphic for a swing motion of a player can be employed. Therefore, the user who is notified of such notification information can increase his / her insight into a series of actions of the subject and can take various actions such as correction of the series of actions.

The point selection unit selects a point group by a touch operation on the input unit 16 by the user.
Therefore, the user can easily select a desired point group.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.
Hereinafter, in the description of the second embodiment, the present embodiment refers to the second embodiment.

  The information display device 1 according to the second embodiment of the present invention has the same hardware configuration as that of the first embodiment. That is, FIG. 1 also shows a hardware configuration of the information display device 1 according to the second embodiment. Since the hardware configuration in FIG. 1 has already been described, the description thereof is omitted here.

FIG. 7 is a functional block diagram illustrating a functional configuration for executing the second index display process among the functional configurations of the information display device 1 illustrated in FIG. 1.
The second index display process is, for example, that when displaying a moving image in which a person performing a running motion is captured as a subject, an image having a shape that serves as an index indicating the movement of the subject (for example, weight shift) is configured as a moving image. This is a series of processing that is performed until the frames are displayed in association with each frame.
In the present embodiment, a shape serving as an index indicating the movement of the subject is referred to as an “index shape”. In the present embodiment, as the index shape, a line segment connecting two or more portions of the subject such as the subject's neck, waist, both knees, and both ankles as points is adopted.

Here, in FIG. 7, the same reference numerals as those in FIG. 2 are given to the functional blocks corresponding to those in FIG. As can be easily understood by comparing FIG. 2 and FIG. 7, when the second index display process is executed, the moving image analysis unit among the functional blocks that function when the first index display process is executed. 42 and the comparison unit 48 do not function, and other functional blocks function. Specifically, when the second index display process is executed, as shown in FIG. 7, in the CPU 11, the moving image acquisition unit 41, the start image display control unit 43, the point selection unit 44, and the playback The control unit 45, the index generation unit 46, the index display control unit 47, the notification information generation unit 49, and the notification control unit 50 function.
Since the main functions of these functional blocks have been described with reference to FIG. 2, only functions unique to the execution of the second index display process will be described below.

Further, when the second index display process is executed, the moving image storage unit 61 provided as one area of the storage unit 18 is used.
In the present embodiment, the content of the moving image data stored in the moving image storage unit 61 is the state of the runner's running operation.

  The point that the start image display control unit 43 performs control to display the start image on the display unit 17 is the same as in the first index display process. However, in the second index display process, the point that an arbitrary frame for starting reproduction can be selected as a start image among a plurality of frames constituting the moving image acquired by the moving image acquisition unit 41. This is different from the one-index display process.

FIG. 8 shows an example of the start image when the second index display process is executed.
The user designates a plurality of points necessary for forming the index shape by a touch operation on the input unit 16 in a state where the start image as shown in FIG. 8 is displayed on the display unit 17.
Thus, a plurality of points necessary for forming the index shape can be arbitrarily designated by the user. However, in the present embodiment, it is assumed that a set 93 of six points each displaying the neck, waist, both knees, and both ankles of the runner (subject) is designated.

Returning to FIG. 7, when reproduction of the moving image is started by the reproduction control unit 45, the point that the index generation unit 46 generates the image data of the index graphic in each frame is the same as the first index display process. is there. Therefore, the details of this point are the same as those in the first index display process and have been described, and thus the description thereof is omitted here.
The index generation unit 46 generates index graphic image data for each frame in the first index display process, whereas the index generation unit 46 generates index shape image data for each frame in the second index display process.

The index display control unit 47 combines the index shape data generated for each frame by the index generation unit 46 with the moving image data to be reproduced, thereby displaying a moving image in which the index shape is superimposed for each frame. Control to be displayed on the unit 17 is executed.
FIG. 9 shows a display example of a moving image in which the index shape is superimposed for each frame.
The index display control unit 47 only needs to display the index shape image on the display unit 17, and does not necessarily display the index shape and the moving image at the same time.
FIG. 10 shows a state in which the index shape in each of the plurality of frames is displayed simultaneously with being superimposed on one screen of the display unit 17.
Further, the index display control unit 47 may simultaneously display an image other than the moving image together with the index shape image.
FIG. 11 shows a state in which an image of supplementary information is displayed on the position screen of the display unit 17 together with an image of the index shape.
That is, as shown in FIG. 11, the index display control unit 47 sets the angle θ between the line 94 connecting the point 94 indicating the runner's neck and the point 95 indicating the runner's waist to the vertical direction as “inclination”. As “θ”, in each frame, an image indicating the inclination θ may be displayed on the display unit 17 together with an index shape (a line segment connecting the point 94 and the point 95 in the example in the figure). At this time, the index display control unit 47 stores information on the inclination θ for each frame in the RAM 13.

  Returning to FIG. 7, the processing of the index generation unit 46 and the index display control unit 47 is repeatedly executed for each frame to be played back while the playback control unit 45 is playing back the moving image. Thus, as described above, index shape data is generated and displayed for each frame.

The point that the notification information generation unit 49 generates notification information is the same as in the first index display process. However, the content of the generated notification information is different from the first index display process. That is, the notification information generation unit 49 generates, as notification information, a graph of the temporal transition of the inclination angle θ for each frame stored in the RAM 13 by the index display control unit 47, for example.
Note that the point that the notification control unit 50 performs control for notifying the user of notification information is the same as in the first index display process.

[Second index display processing]
Next, the second index display process executed by the information display device 1 will be described.
FIG. 12 is a flowchart showing an example of the flow of the second index display process executed by the information display device 1 of FIG. 1 having the functional configuration of FIG.

  In FIG. 12, when the input unit 16 of the information display device 1 is operated to start the acquisition of moving image data, the following processes of steps S21 to S29 are executed.

  In step S <b> 21, the moving image acquisition unit 41 acquires moving image data stored in the moving image storage unit 61.

  In step S22, the start image display control unit 43 selects any one of a plurality of frames constituting the moving image as the start image, and causes the display unit 17 to display the start image.

The processes in steps S23 to S27 are basically the same as the processes in steps S4 to S8 of the first index display process in FIG.
However, the difference is that the index generated and displayed for each frame is an index graphic in the first index display process, but is an index shape in the second index display process. Further, in step S26, the tilt angle θ is displayed together with the index shape, and the information for each frame of the tilt angle θ is stored in the RAM 13, which is different from the first index display process.

In step S28, the notification information generation unit 49 generates information indicating the tilt angle θ for each frame stored in the RAM 13 in step S26 as notification information.
In step S <b> 29, the notification control unit 50 performs notification by causing the display unit 17 to display the notification information.
Thereby, the second index display process is ended.

As described above, in the second embodiment, a moving image in which a runner (subject) runs is employed as an image of a series of movements of the subject.
The information display device 1 according to the second embodiment includes a point selection unit 44, an index generation unit 46, and an index display control unit 47.
The point selection unit 44 selects a start image as a predetermined image among the moving images of the runner's running motion, and selects a plurality of points serving as a reference for the runner's motion in the start image.
The index generation unit 46 generates index shape data for each of a plurality of frames constituting a moving image, based on index shape data including a plurality of selected points.
The index display control unit 47 causes the display unit 17 to display the index shape generated for each frame.
Therefore, the information display device 1 according to the second embodiment can display the change in the posture of the runner as the change in the index shape without using a special device such as a load sensor.

The index display control unit 47 performs control for superimposing each index graphic generated by the index generating unit 46 on a series of motion images (for example, moving images) corresponding to each index graphic and displaying them on the display unit 17. To do.
Therefore, the information display device 1 according to the second embodiment can display a change in the posture of the runner as a change in the indicator graphic along with the movement of the runner. Therefore, the runner can grasp the change in posture in comparison with the actual running motion.

The information display device 1 further includes a notification control unit 50.
The notification control unit 50 notifies the notification information based on the index shape generated by the index generation unit 46.
As the notification information in this case, for example, the angle formed between the back line of the runner as the index shape and the vertical direction is the inclination angle, and the inclination angle is sequentially arranged in time series, that is, the inclination about the runner. A graph of angular time variation can be employed. Therefore, the user who is notified of such notification information can grasp the angle change period and the maximum inclination angle for each inclination angle.

The point selection unit selects a plurality of points by a touch operation on the input unit 16 by the user.
Therefore, the user can easily select a plurality of desired points.

  The present invention is not limited to the first and second embodiments described above, but includes modifications and improvements as long as the object of the present invention can be achieved.

For example, in the above-described first embodiment, the case where the present invention is applied to the movement of the golf swing in the first embodiment and the movement of the runner in the above-described second embodiment is described as an example. The present invention is not limited to this, and can be applied to various operations. For example, the present invention can be applied to various sports operations such as a baseball bat swing, a tennis racket swing, and a dance choreography. Further, for example, the present invention can be applied to an operation of walking, an operation of moving the body by radio exercises, and the like.
In the second embodiment described above, the index shape in each of the plurality of frames is superimposed on one screen of the display unit 17 and displayed at the same time as shown in FIG. 10, but the present invention is not limited to this.
That is, only the index shape in each of the plurality of frames may be displayed as a moving image for each frame.

  In the first and second embodiments described above, a digital camera is employed as the information display device according to the present invention, but the present invention is not limited to this. The present invention can be applied to general electronic devices having an image processing function. Specifically, for example, the present invention can be applied to a notebook personal computer, a printer, a television receiver, a video camera, a portable navigation device, a mobile phone, a smartphone, a portable game machine, and the like.

In the first embodiment described above, as the plurality of points (points), the start image that is directly designated by the user's operation is selected first, and the subsequent frames are point points of the previous frame. The one that was followed was selected. However, the method for selecting a plurality of points is not particularly limited to the method employed in the first embodiment. For example, the method shown in FIGS. 13 and 14 can be employed as a method for selecting a plurality of points.
13 and 14 are diagrams for explaining another example of a method for selecting a plurality of points.
In the example of FIG. 13, a moving image in which a player (subject) performs a golf swing motion is employed. For this reason, in the examples of FIGS. 13 and 14, a triangular index graphic is adopted corresponding to the first embodiment. Therefore, hereinafter, for convenience of explanation, an example different from the first embodiment regarding a method of selecting three points forming a triangular index graphic will be described.
First, the user (which may be the subject himself / herself) operates the input unit 16 to display in advance information shown in FIG. 13 as basic information about the subject, that is, height, crotch length, and stance width. Input to the device 1 in advance.
In this case, since the information display device 1 is equally weighted on both feet of the subject, as shown in FIG. 13, an isosceles triangle whose base is a line segment connecting both feet is used as an indicator graphic at the time of setup. Three vertex coordinates of the isosceles triangle are calculated and calculated based on the basic information, and the three vertices located at these vertex coordinates are selected as three points at the time of setup.
Then, the information display device 1 gradually moves the triangle to the right side so that the weight of the right leg is gradually applied as shown in FIG. In order to do this, three points are calculated as follows. That is, the information display device 1 uses the vertex coordinates of the three points at the time of setup as start coordinates, and divides the triangle vertex coordinates little by little at appropriate time intervals until reaching the top. By moving to the right, three vertex coordinates at each time point are calculated and calculated, and the three vertices located at these vertex coordinates are selected as three points at each time point. In this case, the movement amount of the vertex coordinates at each time point is not particularly limited, and can be arbitrarily changed by a predetermined operation of the input unit 16 by the user.
Further, as a method for selecting three points in the start image, by storing a pattern for detecting the three points in the information of the discriminator known in the field of face image detection technology, the three points in the start image are stored. May be selected.
Further, the information display device 1 changes the change of the index graphic (weight shift) in the second section as a change opposite to the takeback period as shown in FIG. 14, that is, the top triangle is the start index. As the figure, assuming that the triangle gradually moves to the left side so that the weight is gradually applied to the left leg from the starting index figure, the three points are calculated as follows. That is, the information display device 1 uses the vertex coordinates of the three points at the top as the start coordinates, and divides the triangle vertex coordinates little by little at appropriate time intervals until reaching the impact. By moving to the left side, three vertex coordinates at each time point are calculated and calculated, and three vertices located at these vertex coordinates are selected as a point group at each time point. Also in this case, the movement amount of the vertex coordinates at each time point is not particularly limited, and can be arbitrarily changed by a predetermined operation of the input unit 16 by the user.
The change in the triangle formed by the three points selected by applying the method of FIGS. 13 and 14 (see FIG. 14) is an indicator graphic (hereinafter referred to as an ideal change in the weight shift of the subject). It is called “ideal index figure”. On the other hand, the change in the triangle formed by the three points selected by applying the method of the first embodiment is an index graphic (hereinafter referred to as “actual index graphic”) indicating the actual change in the weight shift of the subject. ).
Therefore, the information display device 1 may display the ideal index graphic and the actual index graphic independently of each other, but may display the ideal index graphic and the actual index graphic at the same time. The user can easily visually recognize the difference between the actual and ideal changes in weight by comparing the ideal index graphic and the actual index graphic displayed at the same time.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configurations of FIGS. 2 and 7 are merely examples, and are not particularly limited. That is, it is sufficient if the information display device 1 has a function capable of executing the above-described series of processing as a whole, and what functional blocks are used to realize this function is particularly the example shown in FIGS. It is not limited to.
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only constituted by the removable medium 31 of FIG. 1 distributed separately from the apparatus main body in order to provide the program to the user, but also in a state of being incorporated in the apparatus main body in advance. It is comprised with the recording medium etc. which are provided in this. The removable medium 31 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being incorporated in advance in the apparatus main body includes, for example, the ROM 12 in FIG. 1 in which the program is recorded, the hard disk included in the storage unit 18 in FIG.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalents thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
Selecting means for selecting a plurality of points serving as a reference for the series of actions in a predetermined image of the series of actions of the subject;
First identifying means for respectively identifying a plurality of points in the image of the series of operations corresponding to the plurality of points selected by the selecting means;
Index generating means for generating an index including the plurality of points for each of the plurality of points specified by the first specifying means;
Display control means for executing control to display each index generated by the index generation means on the display means;
A display control device.
[Appendix 2]
The display control means executes control for superimposing each index generated by the index generation means on the image of the series of operations corresponding to each index and causing the display means to display the display. Control device.
[Appendix 3]
A second specifying means for specifying a predetermined operation section of the series of operations;
The display control means performs control for causing the display means to display each index corresponding to a predetermined motion section specified by the second specifying means among the indices generated by the index generating means. The display control apparatus according to appendix 1 or 2.
[Appendix 4]
The display control apparatus according to any one of appendices 1 to 3, further comprising notification means for notifying notification information related to the series of operations based on the index generated by the index generation means.
[Appendix 5]
The display control apparatus according to any one of appendices 1 to 4, wherein the selection unit selects the plurality of points by an input operation by a user.
[Appendix 6]
A display control method executed by a display control device,
A selection step of selecting a plurality of points serving as a reference for the series of movements in a predetermined image of the series of movements of the subject;
A specifying step of specifying a plurality of points in the image of the series of operations corresponding to the plurality of points selected by the selection unit;
An index generating step for generating an index including the plurality of points for the plurality of points specified in the specifying step,
A display control step for executing control to display each index generated in the index generation step on a display means;
A display control method including:
[Appendix 7]
Computer
Selection means for selecting a plurality of points serving as a reference for the series of movements in a predetermined image of the series of movements of the subject;
A specifying unit that specifies a plurality of points in the image of the series of operations corresponding to the plurality of points selected by the selection unit;
Index generating means for generating an index including the plurality of points for each of the plurality of points specified by the specifying means;
Display control means for executing control to display each index generated by the index generation means on a display means;
A program characterized by functioning as

  DESCRIPTION OF SYMBOLS 1 ... Information display apparatus, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Bus, 15 ... Input / output interface, 16 ... Touch panel, 17 ... Display unit 18 ... Storage unit 19 ... Communication unit 20 ... Drive 31 ... Removable media 41 ... Moving image acquisition unit 42 ... Moving image analysis unit 43. ... Start image display control unit 44... Point selection unit 45. Reproduction control unit 46. Index generation unit 47. Index display control unit 48. .. Notification information generation unit, 50... Notification control unit, 61... Moving image storage unit, 62.

Claims (7)

Selecting means for selecting a plurality of points serving as a reference for the series of actions in a predetermined image of the series of actions of the subject;
First identifying means for respectively identifying a plurality of points in the image of the series of operations corresponding to the plurality of points selected by the selecting means;
Index generating means for generating an index including the plurality of points for each of the plurality of points specified by the first specifying means;
Display control means for executing control to display each index generated by the index generation means on the display means;
A display control device. 2. The control according to claim 1, wherein the display control unit performs control to superimpose each index generated by the index generation unit on an image of the series of operations corresponding to each index and display the image on the display unit. Display control device. A second specifying means for specifying a predetermined operation section of the series of operations;
The display control means performs control for causing the display means to display each index corresponding to a predetermined motion section specified by the second specifying means among the indices generated by the index generating means. The display control apparatus according to claim 1 or 2. The display control apparatus according to any one of claims 1 to 3, further comprising a notification unit configured to notify notification information related to the series of operations based on the index generated by the index generation unit.   The display control apparatus according to claim 1, wherein the selection unit selects the plurality of points by an input operation by a user. A display control method executed by a display control device,
A selection step of selecting a plurality of points serving as a reference for the series of movements in a predetermined image of the series of movements of the subject;
A specifying step of specifying a plurality of points in the image of the series of operations corresponding to the plurality of points selected by the selection unit;
An index generating step for generating an index including the plurality of points for the plurality of points specified in the specifying step,
A display control step for executing control to display each index generated in the index generation step on a display means;
A display control method including: Computer
Selection means for selecting a plurality of points serving as a reference for the series of movements in a predetermined image of the series of movements of the subject;
A specifying unit that specifies a plurality of points in the image of the series of operations corresponding to the plurality of points selected by the selection unit;
Index generating means for generating an index including the plurality of points for each of the plurality of points specified by the specifying means;
Display control means for executing control to display each index generated by the index generation means on a display means;
A program characterized by functioning as

Images