JP6641679B2 - Image editing method, apparatus, and program - Google Patents

Description

  The present invention relates to an image editing method, an image editing device, and an image editing program.

  2. Description of the Related Art Conventionally, there has been proposed a method for instructing a body movement on a video display, in which a video moving image such as a sport, a performance, or an action of a trainer or a student is made into a still image of one frame at a time. In this method, the user traces the basic line of the body characterizing the frame on the still image, switches the screen to the next frame while leaving the line on the screen, and further places the line on the image. Follow the basic line of the body that characterizes the top. Then, successively trace the basic lines of the body in the frame, fill in the image, and finally delete the image of the sport, acting or act, etc., and display only a plurality of basic line groups on one screen instead of moving images Then, make the students understand the change of the basic line of the body.

JP 2003-156994 A

  As described above, in the conventional teaching method using a moving image, only the basic line group written on the still image is displayed. However, the drawing performed on the moving image or the still image included in the moving image is performed by the moving image. Or, there is a case where it is desired to check with a still image included in a moving image.

  An object of the present invention is, as one aspect, to reflect drawing performed on a moving image on a still image or to reflect drawing performed on a still image on a moving image.

In one aspect, an input of a moving image is received, one or more frames to be displayed as a still image are extracted from the moving image, and an input of a figure or a character at any position on a display image displaying the moving image is performed. Accept. Then, the received graphic or character is superimposed and output on any of the positions on the display image displaying the moving image, and the position on the display image of the frame displayed as the still image is displayed. Is also superimposed and output at the position corresponding to.

  As one aspect, there is an effect that drawing performed on a moving image can be reflected on a still image, or drawing performed on a still image can be reflected on a moving image.

FIG. 3 is a functional block diagram of the moving image evaluation system according to the first and second embodiments. It is a figure showing an example of each table of a animation evaluation information database in a 1st embodiment. It is the schematic which shows an example of the screen for evaluators. It is the schematic which shows an example of a screen for users. FIG. 11 is a block diagram illustrating a schematic configuration of a computer that functions as an image editing device according to the first and second embodiments. It is a flowchart which shows an example of a process for users. It is a flow chart which shows an example of processing for evaluators in a 1st embodiment. It is a flowchart which shows an example of a browsing process. FIG. 9 is a sequence diagram illustrating a process at the time of an evaluation request. FIG. 9 is a sequence diagram illustrating a process at the time of evaluation. FIG. 9 is a sequence diagram showing a process at the time of browsing. FIG. 9 is an image diagram for describing drawing of an evaluation figure on a moving image. It is an image figure for explaining reflection of an evaluation figure to a still picture. It is an image figure for explaining reflection of an evaluation figure to a moving picture. It is an image figure for explaining reflection of an evaluation figure to a moving picture. It is a figure showing an example of each table of a moving picture evaluation information database in a 2nd embodiment. FIG. 9 is an image diagram for describing drawing of an evaluation figure on a still image. It is an image figure for explaining reflection of an evaluation figure to a moving picture. It is an image figure for explaining reflection of an evaluation figure to a still picture. It is a flow chart which shows an example of processing for evaluators in a 2nd embodiment.

  Hereinafter, an example of an embodiment according to the present invention will be described in detail with reference to the drawings.

<First embodiment>
As shown in FIG. 1, the moving image evaluation system 100 according to the first embodiment includes an image editing device 10, a plurality of user terminals 31, and a plurality of evaluator terminals 32. The image editing apparatus 10 is connected to each of the plurality of user terminals 31 and each of the plurality of evaluator terminals 32 via a network. In the moving image evaluation system 100, a service provided by the moving image evaluation system 100 is provided via a web browser operating on each of the user terminal 31 and the evaluator terminal 32. Specifically, the evaluator evaluates the moving image indicated by the moving image data transmitted from the user terminal 31 by the user at the evaluator terminal 32. The image editing apparatus 10 performs image editing on the still image indicated by the moving image data and each frame constituting the moving image data according to the evaluation of the moving image by the evaluator, and returns the evaluation result to the user terminal 31.

  The user terminal 31 and the evaluator terminal 32 are terminal devices such as a personal computer, a tablet terminal, a smartphone, and a mobile phone.

  As shown in FIG. 1, the image editing device 10 includes an evaluation request receiving unit 11, a still image extracting unit 12, an evaluation graphic receiving unit 13, and an output unit 14. The evaluation request receiving unit 11 is an example of a first receiving unit of the present invention, the still image extracting unit 12 is an example of an extracting unit of the present invention, and the evaluation graphic receiving unit 13 is an example of a second receiving unit. Further, the image editing apparatus 10 includes a storage unit 20, and the storage unit 20 includes a moving image evaluation information database (DB) 21, a moving image storage unit 22, and an evaluation figure storage unit 23. As shown in FIG. 2, the moving image evaluation information DB 21 includes a moving image evaluation information table 21A, a still image table 21B, and a moving image table 21C. Hereinafter, each functional unit and each table of the moving image evaluation information DB 21 will be described in detail.

  The evaluation request receiving unit 11 receives the evaluation request transmitted from the user terminal 31. The evaluation request includes moving image data indicating a moving image to be evaluated and a user ID for identifying a user who has transmitted the evaluation request. Further, the evaluation request can include text data such as a comment to the evaluator. In the case where a video that has been accepted in the past is set as the video to be evaluated, information specifying the video to be evaluated may be included in the evaluation request instead of the video data indicating the video to be evaluated. The evaluation request receiving unit 11 stores the moving image data included in the received evaluation request in the moving image storage unit 22.

  In this embodiment, a plurality of different evaluation requests can be received for the same moving image. Therefore, the evaluation request receiving unit 11 assigns a moving image evaluation ID for identifying the evaluation request for each evaluation request. Then, the evaluation request receiving unit 11 associates the moving image evaluation ID with the user ID and the moving image ID for identifying the moving image to be evaluated, and registers the moving image evaluation ID in, for example, the moving image evaluation information table 21A as illustrated in FIG. The moving image ID may be a file name of the moving image data, a path indicating a position on the moving image storage unit 22 where the moving image data is stored, or the like. Note that the moving image evaluation information table 21A illustrated in FIG. 2 also includes an item of “evaluator ID”, but when the evaluation request receiving unit 11 receives a moving image, the item of “evaluator ID” is blank. .

  The still image extracting unit 12 extracts a predetermined number of frames to be displayed as a still image from each frame constituting the moving image data according to a still image extraction rule. As the still image extraction rule, for example, a rule for extracting a frame in a predetermined pattern such as the tenth frame, the 50th frame, the 100th frame, and so on can be used. Instead of directly defining the frames to be extracted, a pattern in which the time from the start of the moving image is determined, such as the first, second, third, and so on, may be used.

  Alternatively, the still image extraction rule may be a rule that accepts the specification of a frame to be extracted from the evaluator and extracts the specified frame. The evaluator can specify a desired frame by, for example, playing the moving image frame by frame. Also in this case, not only the instruction of the frame to be extracted but also the designation by time may be accepted. When a frame to be extracted is specified by time, a frame corresponding to the specified time may be extracted. Further, a rule for extracting a frame having a predetermined characteristic may be used as the still image extraction rule. A frame indicating a predetermined feature can be extracted by, for example, pattern matching or a comparison between a feature amount extracted from an image indicated by the frame and a learning model learned in advance.

  The still image extraction unit 12 associates a still image extraction rule for extracting a frame to be displayed as a still image or a frame ID of an actually extracted frame with a moving image evaluation ID, as shown in FIG. It is registered in the still image table 21B. The frame ID is a serial number from the start frame. Note that the still image table 21B shown in FIG. 2 also includes an item of “evaluation graphic ID”, but when evaluation graphic data described later is not stored, the item of “evaluation graphic ID” is blank. .

  The evaluation graphic receiving unit 13 receives, on the evaluator terminal 32, evaluation graphic data indicating a graphic or a character drawn by the evaluator to evaluate a moving image (hereinafter, referred to as “evaluation graphic”), and receives the received evaluation graphic data. It is stored in the evaluation figure storage unit 23. Drawing of the evaluation graphic is performed on a display area where a moving image is reproduced on the evaluator terminal 32, as described later. The evaluation graphic data can be printed with a coordinate data group indicating which position in the display area has been drawn. Further, the evaluation graphic receiving unit 13 adds an evaluation graphic ID for identifying the evaluation graphic to the evaluation graphic data, and registers the evaluation graphic data in the “evaluation graphic ID” item of the corresponding moving image evaluation ID in the still image table 21B. .

  The output unit 14 displays, for example, an evaluator screen 70 as shown in FIG. 3 on a web browser operating on the evaluator terminal 32. The evaluator screen 70 includes a main display area 71 for playing a moving image or displaying a still image, and a still image indicated by a predetermined number (three in the example of FIG. 3) of frames extracted by the still image extracting unit 12. And a sub-display area 72 in which thumbnail images of images are displayed. In addition, on the evaluator screen 70, a drawing tool 73 is displayed so as to be selectable as an interface for realizing a drawing function.

  When the evaluator selects an evaluation request, the output unit 14 identifies the evaluator in the item “evaluator ID” of the selected evaluation request in the moving image evaluation information table 21A shown in FIG. Is temporarily registered. In addition, the output unit 14 refers to the moving image evaluation information table 21 </ b> A, specifies a moving image to be evaluated in the selected evaluation request, and acquires moving image data indicating the specified moving image from the moving image storage unit 22. The output unit 14 reproduces the moving image indicated by the obtained moving image data in the main display area 71. Further, when a still image is extracted by the still image extracting unit 12, the output unit 14 specifies a frame to be displayed as a still image with reference to the still image table 21B, and a thumbnail of the still image indicated by the specified frame. The image is displayed in the sub display area 72.

  Further, the output unit 14 receives a moving image reflection rule indicating in which section of the entire reproduction time of the moving image the reflecting graphic is to be reflected. The “section” of the moving image is an example of a partial image of the moving image of the present invention. The output unit 14 registers a moving image section in which the evaluation graphic is superimposed and displayed in accordance with the received moving image reflection rule and the received moving image reflection rule, for example, in the moving image table 21C as illustrated in FIG.

  In the example of FIG. 2, when the moving image reflection rule is “whole”, it indicates that the evaluation graphic is superimposed and displayed over the entire section of the moving image reproduction time. When the moving image reflection rule is “before and after the representative”, the evaluation figure is superimposed on a predetermined number of frames before and after the representative frame selected by the evaluator among a plurality of frames constituting the moving image. It indicates that it is to be displayed. For example, it is assumed that a frame with a frame ID of 60 is selected as a representative frame, and a section from 20 frames before to 30 frames after the representative frame is designated. In this case, the output unit 14 registers “40” as “display start frame ID” and “90” as “display end frame ID”. When the “display start frame ID” and the “display end frame ID” are “null”, it indicates that the number of frames before and after is not specified. In this case, a predetermined section based on the representative frame (for example, 15 frames before and after) can be set as a section in which the evaluation graphic is superimposed and displayed.

  When the moving image reflection rule is “after the representative”, it indicates that the evaluation graphic is superimposed and displayed in a section from the representative frame to the last frame of the moving image. Note that the representative frame can be selected from the frames extracted by the still image extracting unit 12. One representative frame may be selected for one evaluation request, or a plurality of representative frames may be selected.

  When the evaluation graphic data is received by the evaluation graphic receiving unit 13, the output unit 14 specifies an evaluation graphic corresponding to the moving image evaluation ID indicating the selected evaluation request with reference to the still image table 21B. Then, the output unit 14 acquires the evaluation graphic data indicating the specified evaluation graphic from the evaluation graphic storage unit 23. Further, the output unit 14 refers to the moving image table 21C to specify a moving image section in which the evaluation graphic is superimposed and displayed. The output unit 14 superimposes and displays the evaluation graphic indicated by the obtained evaluation graphic data in the specified section of the moving image being reproduced in the main display area 71. Further, the output unit 14 displays the evaluation graphic indicated by the obtained evaluation graphic data so as to be superimposed on the still image displayed in the sub display area 72. At this time, the evaluation graphic is reduced at a reduction ratio represented by the size of the thumbnail image with respect to the size of the main display area 71, and the evaluation graphic is superimposed on the still image corresponding to the position where the evaluation graphic is drawn on the moving image. I do.

  Further, the output unit 14 displays, for example, a user screen 76 as shown in FIG. 4 on a web browser operating on the user terminal 31. The user screen 76 is similar to the evaluator screen 70 except that the drawing tool 73 is not displayed. That is, the user screen 76 includes a main display area 77 and a sub display area 78.

  In addition, when the user requests viewing of the evaluation result, the output unit 14 plays back the target moving image in the main display area 77, and displays the corresponding evaluation graphic in the moving image section according to the corresponding moving image reflection rule. Display superimposed. In addition, the output unit 14 superimposes the corresponding evaluation graphic on the still image extracted from the target moving image according to the corresponding still image extraction rule, and displays the same as a thumbnail image in the sub-display area 78. The processing of playing a moving image, displaying a thumbnail image, and superimposing an evaluation figure is the same as the processing on the evaluator screen 70, and thus a detailed description is omitted. Further, when the user terminal 31 on which the evaluation result is viewed does not have a moving image reproduction function, the output unit 14 displays an evaluation graphic in the main display area 77 on a still image selected from any of the thumbnail images. Are superimposed and displayed. Note that the still image displayed in the main display area 77 is switched by changing the selection of the thumbnail image. Similarly, a still image may be displayed in the main display area 71 of the evaluator screen 70.

  The output unit 14 also displays a menu screen and the like on the user terminal 31 and the evaluator terminal 32, and transmits and receives various information. The output unit 14 displays various screens, moving images, and still images on the user terminal 31 or the evaluator terminal 32 by transmitting HTML (HyperText Markup Language) data indicating the screen to be displayed to the user terminal 31 or the evaluator terminal 32. It is done by doing.

  The image editing device 10 can be realized by, for example, a computer 40 illustrated in FIG. The computer 40 includes a CPU 41, a memory 42 as a temporary storage area, and a nonvolatile storage device 43. Further, the computer 40 includes an input / output interface (I / F) 44 to which an input / output device 48 is connected. Further, the computer 40 includes a read / write (R / W) unit 45 that controls reading and writing of data to and from the recording medium 49, and a network I / F 46 that is connected to a network such as the Internet. The CPU 41, the memory 42, the storage device 43, the input / output I / F 44, the R / W unit 45, and the network I / F 46 are connected to each other via a bus 47.

  The storage device 43 can be realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like. The storage device 43 as a storage medium stores an image editing program 50 for causing the computer 40 to function as the image editing device 10. Further, the storage device 43 has a moving image evaluation information storage area 61, a moving image data storage area 62, and an evaluation figure data storage area 63.

  The CPU 41 reads out the image editing program 50 from the storage device 43 and expands it in the memory 42, and sequentially executes the processes of the image editing program 50. The image editing program 50 includes an evaluation request receiving process 51, a still image extracting process 52, an evaluation graphic receiving process 53, and an output process 54.

  The CPU 41 operates as the evaluation request receiving unit 11 illustrated in FIG. 1 by executing the evaluation request receiving process 51. In addition, the CPU 41 operates as the still image extracting unit 12 illustrated in FIG. 1 by executing the still image extracting process 52. The CPU 41 operates as the evaluation graphic receiving unit 13 illustrated in FIG. 1 by executing the evaluation graphic receiving process 53. The CPU 41 operates as the output unit 14 illustrated in FIG. 1 by executing the output process 54. The CPU 41 reads information stored in the moving image evaluation information storage area 61 and expands the information in the memory 42 as, for example, each table of the moving image evaluation information DB 21 as shown in FIG. The moving image data storage area 62 functions as the moving image storage unit 22 shown in FIG. 1, and the evaluation graphic data storage area 63 functions as the evaluation graphic storage unit 23 shown in FIG. Thus, the computer 40 that has executed the image editing program 50 functions as the image editing apparatus 10.

  Note that the image editing apparatus 10 can be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit) or the like.

  Next, the operation of the moving image evaluation system 100 according to the first embodiment will be described. First, a user logs in to a site for receiving a service provided by the moving image evaluation system 100 via a web browser operating on the user terminal 31. In this case, the image editing apparatus 10 executes the user process shown in FIG. Then, the evaluator logs in to the site of the moving image evaluation system 100 via a web browser operating on the evaluator terminal 32. In this case, the image editing apparatus 10 executes the evaluator processing shown in FIG. Hereinafter, each process will be described with reference to the sequence diagrams of FIGS. 9 to 11. Note that, in the sequence diagrams of FIGS. 9 to 11, some descriptions of general processing such as login are omitted.

  In step S11 of the user process shown in FIG. 6, the output unit 14 acquires the user ID of the user input at the time of login. Next, in step S12, the output unit 14 displays on the user terminal 31 a menu screen for selecting either the evaluation request or the evaluation result browsing. Next, in step S13, the output unit 14 determines whether the user has selected an evaluation request from the menu screen. In the case of an affirmative determination, the process proceeds to step S14, and in the case of a negative determination, it is determined that browsing of the evaluation result has been selected from the menu screen, and the process proceeds to step S30. Be executed.

  In step S14, the output unit 14 displays an evaluation request screen for the user to make an evaluation request (81 in FIG. 9). The evaluation request screen is a screen on which a moving image to be evaluated, a comment to the evaluator, and the like can be input, and includes a transmission button selected when transmitting the evaluation request to the image editing apparatus 10.

  The user shoots a moving image to be evaluated, designates the moving image to be evaluated on the evaluation request screen, and inputs a comment or the like as necessary (82 in FIG. 9). Then, by selecting the transmission button, the evaluation request including the moving image data indicating the shot moving image, the user ID acquired in step S11, the comment, and the like is transmitted to the image editing apparatus 10 (83 in FIG. 9). . Note that the moving image data to be transmitted is not limited to the moving image data indicating the moving image captured at this time. The moving image data may be moving image data captured in advance and stored in the user terminal 31, or moving image data imported from the outside.

  Next, in step S15, the evaluation request receiving unit 11 receives the evaluation request transmitted from the user terminal 31. Next, in step S16, the evaluation request receiving unit 11 stores the moving image data included in the received evaluation request in the moving image storage unit 22. In addition, the evaluation request receiving unit 11 assigns a moving image evaluation ID to the received evaluation request and registers the moving image evaluation ID together with a moving image ID for identifying a moving image to be evaluated, for example, in a moving image evaluation information table 21A shown in FIG. Then, the user process ends.

  Next, in step S21 of the evaluator process shown in FIG. 7, the output unit 14 acquires the evaluator ID of the evaluator input at the time of login. Next, in step S22, it is determined whether there is an unprocessed evaluation request. In this determination, for example, an evaluation request with a blank “evaluator ID” in the moving image evaluation information table 21A can be determined as an unprocessed evaluation request. In the case of an affirmative determination, the process proceeds to step S23, and in the case of a negative determination, the evaluator process ends.

  In step S23, the output unit 14 receives the user ID from the evaluator terminal 32 (86 in FIG. 10) as the user's designation by the evaluator (85 in FIG. 10). Then, the output unit 14 refers to the video evaluation information table 21 </ b> A, specifies an unprocessed evaluation request for the specified user, and stores, in the evaluator terminal 32, a list of unprocessed evaluation requests for the specified user. Is displayed (86 in FIG. 10).

  Next, in step S24, the output unit 14 receives the moving image evaluation ID of the evaluation request selected from the evaluation request list (88 in FIG. 10) by the evaluator (89 in FIG. 10). Then, the output unit 14 displays an evaluator screen 70 as shown in FIG. 3 on the evaluator terminal 32 (90 in FIG. 10). Next, the output unit 14 temporarily registers the evaluator ID acquired in step S21 in the item of “evaluator ID” corresponding to the received moving image evaluation ID in the moving image evaluation information table 21A illustrated in FIG. In addition, the output unit 14 specifies a moving image corresponding to the received moving image evaluation ID with reference to the moving image evaluation information table 21A, and acquires moving image data indicating the specified moving image from the moving image storage unit 22 (91 in FIG. 10). ). Then, the output unit 14 reproduces the moving image indicated by the obtained moving image data in the main display area 71 (92 in FIG. 10).

  Next, in step S25, the output unit 14 displays a screen for selecting a still image extraction rule on the evaluator terminal 32. Then, the still image extraction unit 12 receives the still image extraction rule selected by the evaluator (93 in FIG. 10) (94 in FIG. 10). Next, the still image extracting unit 12 extracts a predetermined number of frames to be displayed as a still image from each frame constituting the moving image data according to the selected still image extraction rule. Then, the still image extracting unit 12 associates the selected still image extraction rule or the frame ID of the frame actually extracted according to the still image extraction rule with the moving image evaluation ID, and It is registered in the table 21B (95 in FIG. 10). Then, the output unit 14 refers to the still image table 21B, specifies a frame to be displayed as a still image, that is, specifies a frame extracted according to the still image extraction rule, and generates a thumbnail image of the still image indicated by the specified frame. It is displayed in the sub display area 72 (96 in FIG. 10).

  Next, in step S26, the evaluation graphic receiving unit 13 receives evaluation graphic data indicating an evaluation graphic drawn in the main display area 71 in a state where a moving image is being reproduced. For example, when drawing by the evaluator (97 in FIG. 10) is completed, the evaluator selects a completion button (not shown) provided on the evaluator screen 70 to indicate the drawn evaluation graphic. The evaluation graphic data is transmitted to the image editing apparatus 10 (98 in FIG. 10).

  Next, in step S27, the evaluation graphic receiving unit 13 stores the received evaluation graphic data in the evaluation graphic storage unit 23. Further, the evaluation graphic receiving unit 13 adds the evaluation graphic ID to the evaluation graphic data, and registers the evaluation graphic ID in the “evaluation graphic ID” item corresponding to the moving image evaluation ID received in step S24 in the still image table 21B ( 99 in FIG. 10).

  Next, in step S28, the output unit 14 specifies an evaluation graphic corresponding to the moving image evaluation ID received in step S24 with reference to the still image table 21B, and outputs evaluation graphic data indicating the specified evaluation graphic. Obtained from the evaluation figure storage unit 23. Then, the output unit 14 superimposes and displays the evaluation graphic indicated by the obtained evaluation graphic data on the still image displayed in the sub display area 72 (101 in FIG. 10). Thus, as shown in FIG. 12, the drawing of the evaluation graphic performed on the reproduced moving image, not the still image, can be reflected on the still image as shown in FIG.

  In step S29, the output unit 14 displays a screen for selecting a moving image reflection rule on the evaluator terminal 32. Then, the output unit 14 receives the moving image reflection rule selected by the evaluator (102 in FIG. 10) (103 in FIG. 10). The output unit 14 registers a moving image section in which the evaluation graphic is superimposed and displayed in accordance with the received moving image reflection rule and the moving image reflection rule, for example, in the moving image table 21C as illustrated in FIG. 2 (104 in FIG. 10). Then, the output unit 14 refers to the moving image table 21C, specifies a section of the moving image in which the evaluation pattern is superimposed and displays the evaluation pattern corresponding to the received moving image evaluation ID in the main display area 71. Are superimposed and displayed in the specified section (105 in FIG. 10). Specifically, the superimposed display of the evaluation graphic starts at the timing when the start frame of the specified section is displayed in the main display area 71 among the frames constituting the moving image to be reproduced. Then, the superimposed display of the evaluation graphic ends at the timing when the frame next to the end frame of the specified section is displayed in the main display area 71.

  Thus, as shown in FIG. 12, the drawing of the evaluation graphic performed on the moving image can be reflected as it is when the moving image is reproduced as shown in FIG. When a section of a moving image on which an evaluation graphic is superimposed is specified, as shown in FIG. 15, the evaluation graphic can be reflected only in a necessary section.

  Next, in step S30, the output unit 14 determines the evaluation result by permanently registering the evaluator ID that was temporarily registered in the moving image evaluation information table 21A in step S24. It should be noted that the provisional registration and the main registration can be distinguished from each other by providing the “evaluator ID” item with attribute information indicating whether the state is the provisional registration state or the main registration state. When the process of this step ends, the process for the evaluator ends.

  Next, the browsing process executed in step S30 of FIG. 6 will be described. In step S31 of the browsing process shown in FIG. 8, the output unit 14 acquires the user ID of the user input at the time of login. Next, in step S32, the output unit 14 refers to the moving image evaluation information table 21A, and corresponds to the acquired user ID, and the corresponding item of “evaluator ID” is the registered moving image evaluation ID of the moving image evaluation ID. The list is displayed on the user terminal 31 (106 in FIG. 11). Next, in step S33, the output unit 14 receives a moving image evaluation ID indicating the evaluation result selected from the list by the user (107 in FIG. 11) (108 in FIG. 11).

  Next, in step S34, the output unit 14 determines whether or not the user terminal 31 requesting viewing of the evaluation result is a video playback compatible terminal having a video playback function. This determination can be made, for example, by acquiring information that can determine whether or not the terminal is a video playback-compatible terminal, such as the model information of the user terminal 31, together with the user ID in step S31. If the user terminal 31 is a video playback compatible terminal, the process proceeds to step S35. If the user terminal 31 is a video playback non-compatible terminal, the process proceeds to step S36.

  In step S35, the output unit 14 displays, for example, a user screen 76 as shown in FIG. 4 on the web browser operating on the user terminal 31 (109 in FIG. 11). Then, the output unit 14 plays back the moving image corresponding to the moving image evaluation ID received in step S33 in the main display area 77, and superimposes and displays the evaluation graphic in the moving image section according to the moving image reflection rule. Further, the output unit 14 superimposes the evaluation figure on the still image extracted according to the still image extraction rule, and displays the evaluation figure as a thumbnail image in the sub-display area 78 (110 in FIG. 11).

  On the other hand, in step S36, the output unit 14 displays the user screen 76, and superimposes and displays the evaluation graphic on the still image selected from any of the thumbnail images in the main display area 77 (111 in FIG. 11). ). When the output unit 14 receives a change in selection of a thumbnail image, the output unit 14 switches the still image to be displayed in the main display area 77.

  As described above, according to the moving image evaluation system according to the first embodiment, the image editing apparatus extracts an evaluation graphic drawn in a display area where a moving image is being reproduced from a plurality of frames constituting the moving image. It is also superimposed and displayed on the still image. Thus, the drawing performed on the moving image can be reflected on the still image.

  In addition, by making it possible to designate a section of the moving image when the evaluation graphic is superimposed on the moving image, it is possible to prevent the evaluation graphic from being displayed more than necessary and becoming difficult to see.

  In the case where an evaluation graphic is drawn on a still image as in the related art, for example, the object is drawn because the speed of the movement of the body and the movement of the equipment accompanying the movement, such as a golf swing, is high. In some cases, the body or equipment may be blurred on a still image. In such a case, it may not be possible to properly draw the evaluation figure. In addition, since the purpose is to evaluate moving images, that is, to evaluate body movements and the like, there are cases where it is desired to draw an evaluation figure along with the movement, but this cannot be applied to a still image. On the other hand, the evaluation result may be browsed on a terminal that does not have a video playback function. In some cases, such as when it is desired to check an evaluation graphic for a certain point, it is better to reflect the evaluation graphic on a still image. According to the first embodiment, as described above, since the drawing performed on the moving image is reflected on the still image, these problems can be solved.

  Although the first embodiment does not consider which section is drawn with respect to the moving image, the evaluation figure may be reflected in the still image in consideration of this point. . For example, the frame ID of the frame displayed in the display area at the start of drawing and the frame ID of the frame displayed in the display area at the end of drawing are recorded. Then, a portion from the frame corresponding to the start of drawing to the frame corresponding to the end of drawing can be set as a partial section of the moving image. In this case, when reflecting the evaluation graphic in the still image, the evaluation graphic is superimposed on the still image extracted from the frame included in the above section, and the still image extracted from the frame outside the section is displayed. In other words, the evaluation figure can be prevented from being superimposed. As a result, it is possible to prevent the evaluation graphic from being superimposed on a still image irrelevant to the evaluation graphic. Further, when a still image is extracted by the still image extracting unit 12, a rule may be defined such that the still image is extracted from a section from a frame corresponding to the start of drawing to a frame corresponding to the end of drawing.

  Further, in the first embodiment, when the user makes an evaluation request, the evaluator is not designated, but an evaluation request in which the evaluator is designated may be made. In this case, the evaluation request includes information that can specify the nominated evaluator, and when the image editing device 10 receives the evaluation request, an item of “designated evaluator” is provided in the video evaluation information table 21A, The evaluator ID indicating the nominated evaluator may be registered together. When the evaluator logs in, a list of evaluation requests in which the “designated evaluator” item matches the evaluator ID of the logged-in evaluator may be presented to the evaluator.

<Second embodiment>
Next, a second embodiment will be described. In the first embodiment, the case where the evaluation graphic drawn on the moving image is reflected in the still image and the moving image has been described. In the second embodiment, the case where the evaluation graphic drawn on the still image is reflected on the moving image and the still image will be described. I do. The same parts as those of the moving image evaluation system 100 according to the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted.

  As shown in FIG. 1, the image editing device 210 according to the second embodiment includes an evaluation request receiving unit 11, a still image extracting unit 212, an evaluation graphic receiving unit 213, and an output unit 214. Further, the image editing device 210 includes a storage unit 20, and the storage unit 20 includes a moving image evaluation information DB 221, a moving image storage unit 22, and an evaluation figure storage unit 23. As shown in FIG. 16, the moving image evaluation information DB 221 includes a moving image evaluation information table 21A, a still image table 221B, and a moving image table 221C. Hereinafter, each functional unit and each table of the moving image evaluation information DB 221 will be described in detail mainly focusing on differences from the first embodiment.

  The still image extracting unit 212 extracts a predetermined number of frames to be displayed as a still image from each frame constituting the moving image data according to the still image extracting rule, similarly to the still image extracting unit 12 in the first embodiment. . The still image extracting unit 212 in the second embodiment registers each frame ID of the actually extracted frame in the still image table 221B as shown in FIG. 16 in association with the moving image evaluation ID. That is, the still image table 221B is a table of one row (one record) for each extracted frame.

  Drawing of the evaluation figure in the second embodiment is performed on the main display area 71 in a state where a still image is displayed. The evaluation graphic receiving unit 213 receives, together with the evaluation graphic data, a frame ID indicating a still image displayed in the main display area 71 when the evaluation graphic indicated by the evaluation graphic data is drawn. The evaluation graphic receiving unit 213 adds the evaluation graphic ID to the evaluation graphic data, and registers the evaluation graphic data in the “evaluation graphic ID” item of the corresponding moving image evaluation ID and the extracted frame ID in the still image table 21B.

  When the evaluation request is selected by the evaluator, the output unit 214 once reproduces the moving image corresponding to the selected evaluation request in the main display area 71, similarly to the output unit 14 of the first embodiment. Further, a thumbnail image of a still image extracted from the moving image corresponding to the selected evaluation request is displayed in the sub-display area 72. In this state, when any of the thumbnail images is selected by the evaluator, the output unit 214 displays a still image corresponding to the selected thumbnail image in the main display area 71, and renders the image displayable. As a result, as shown in FIG. 17, different evaluation figures can be drawn for each of the extracted still images.

  The output unit 214 also reflects the evaluation figure drawn on the still image in the moving image. Specifically, the output unit 214 receives, from the evaluator, a moving image reflection rule indicating in which section in the entire reproduction time of the moving image the evaluation graphic drawn for each still image is to be reflected. The output unit 214 registers a moving image section in which the evaluation graphic is superimposed and displayed according to the received moving image reflection rule, for example, in a moving image table 221C as illustrated in FIG.

  In the example of FIG. 16, the frame indicated by “extracted frame ID” is a frame extracted by the still image extracting unit 212. The “display start frame” and the “display end frame” indicate a section in which the evaluation graphic drawn on the still image of the frame indicated by the “extracted frame ID” is superimposed and displayed. The “display priority” determines which evaluation graphic is displayed with higher priority when sections displaying different evaluation graphics overlap each other. The example of FIG. 16 shows that the priorities are higher in the order of display priority = 1, 2, 3,.... For example, in the extracted frame IDs = 40 and 90 with the moving image evaluation ID = 4, the sections overlap at the frame ID = 40-50. In this case, in the portion of frame ID = 40 to 50, the evaluation graphic drawn for the still image of the frame with the extracted frame ID = 40 having the higher display priority is displayed. When the display priority is “null”, predetermined evaluation patterns such as displaying all evaluation patterns having overlapping sections in a superimposed manner or displaying the evaluation graphic displayed earlier with priority are set in advance. Follow the display method.

  When the evaluation graphic data is received by the evaluation graphic receiving unit 213, the output unit 214 specifies the evaluation graphic of each extracted frame corresponding to the moving image evaluation ID indicating the selected evaluation request with reference to the still image table 221B. I do. The output unit 214 acquires each of the evaluation graphic data indicating the specified evaluation graphic from the evaluation graphic storage unit 23. In addition, the output unit 214 refers to the moving image table 221C, and specifies a section of the moving image in which each evaluation pattern is superimposed and displayed for each evaluation pattern. The output unit 214 superimposes and displays the evaluation graphic indicated by the obtained evaluation graphic data in the specified section of the moving image being reproduced in the main display area 71.

  For example, in the example of the moving image evaluation ID = 4 registered in the moving image table 221C shown in FIG. 16, as shown in FIG. 18, in the section of the frame ID = 20 to 50, the still image of the frame with the frame ID = 40 Is superimposed on the evaluation graphic drawn. In the section of frame ID = 50 to 100, the evaluation figure drawn on the still image of the frame with frame ID = 90 is superimposed and displayed. Further, in the section of frame ID = 100 to 150, the evaluation graphic drawn on the still image of the frame with frame ID = 130 is superimposed and displayed.

  Further, the output unit 214 superimposes and displays each of the evaluation graphics indicated by the acquired evaluation graphic data on each of the corresponding still images displayed in the sub-display area 72, as shown in FIG. 19, for example. At this time, the evaluation graphic is reduced at a reduction ratio represented by the size of the thumbnail image with respect to the size of the main display area 71, and the evaluation graphic is superimposed on the still image corresponding to the position where the evaluation graphic is drawn on the moving image. indicate.

  The image editing device 210 can be realized by, for example, the computer 40 shown in FIG. The storage device 43 stores an image editing program 250 for causing the computer 40 to function as the image editing device 210. The CPU 41 reads out the image editing program 250 from the storage device 43 and expands it in the memory 42, and sequentially executes the processes of the image editing program 250. The image editing program 250 has an evaluation request reception process 51, a still image extraction process 252, an evaluation figure reception process 253, and an output process 254.

  The CPU 41 operates as the still image extracting unit 212 illustrated in FIG. 1 by executing the still image extracting process 252. The CPU 41 operates as the evaluation graphic receiving unit 213 illustrated in FIG. 1 by executing the evaluation graphic receiving process 253. The CPU 41 operates as the output unit 214 illustrated in FIG. 1 by executing the output process 254. Other processes and storage areas function in the same manner as in the first embodiment.

  Note that the image editing device 210 can also be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC or the like.

  Next, the operation of the moving image evaluation system 100 according to the second embodiment will be described. Since the process for the evaluator is different from that of the first embodiment, the process for the evaluator will be described with reference to FIG. The same processing as the evaluator terminal processing in the first embodiment is denoted by the same reference numeral, and detailed description is omitted.

  After steps S21 to S24 of the evaluator process shown in FIG. 20, in the next step S225, the still image extracting unit 212 extracts a predetermined number of frames to be displayed as a still image from each frame constituting the moving image data. , According to the still image extraction rules. Then, the still image extracting unit 212 registers each frame ID of the actually extracted frame in the still image table 221B as shown in FIG. 16 in association with the moving image evaluation ID. Further, the thumbnail image of the extracted still image is displayed in the sub display area 72.

  Next, in step S231, the output unit 214 displays a still image corresponding to the thumbnail image selected by the evaluator in the main display area 71, so that drawing is possible. Next, in step S226, the evaluation graphic receiving unit 213 receives the evaluation graphic data and the frame ID indicating the still image displayed in the main display area 71 when the evaluation graphic indicated by the evaluation graphic data is drawn. . Next, in step S227, the evaluation graphic receiving unit 213 stores the received evaluation graphic data in the evaluation graphic storage unit 23. Then, the evaluation graphic receiving unit 213 adds the evaluation graphic ID to the evaluation graphic data, and registers the evaluation graphic data in the “evaluation graphic ID” item of the corresponding moving image evaluation ID and the extracted frame ID in the still image table 221B.

  Next, in step S232, the output unit 214 determines whether or not the selection of the thumbnail image has been changed by the evaluator. If it has been changed, the process returns to step S231. If the selection has not been changed, the process proceeds to step S228. In step S228, the output unit 214 superimposes each of the evaluation figures indicated by the evaluation figure data corresponding to the moving image evaluation ID indicating the selected evaluation request on each of the corresponding still images displayed in the sub display area 72. To display.

  Next, in step S229, the output unit 214 receives a moving image reflection rule from the evaluator, and registers, in the moving image table 221C, for example, as illustrated in FIG. I do. Then, the output unit 214 specifies the evaluation graphic of each extracted frame corresponding to the moving image evaluation ID indicating the selected evaluation request with reference to the still image table 221B. The output unit 214 acquires each of the evaluation graphic data indicating the specified evaluation graphic from the evaluation graphic storage unit 23. In addition, the output unit 214 refers to the moving image table 221C, and specifies a section of the moving image in which each evaluation pattern is superimposed and displayed for each evaluation pattern. The output unit 214 superimposes and displays the evaluation graphic indicated by the obtained evaluation graphic data in the specified section of the moving image being reproduced in the main display area 71.

  As described above, according to the moving image evaluation system according to the second embodiment, the image editing apparatus superimposes the evaluation figure drawn on the display area displaying the still image on the moving image to be reproduced. indicate. Thus, the drawing performed on the still image can be reflected on the moving image.

  Note that some of the functions of the image editing apparatus according to the first and second embodiments may be realized by a web browser on a user terminal or an evaluator terminal.

  In the above description, the mode in which the image editing program 50 is stored (installed) in the storage device 43 in advance has been described. However, the image editing program 50 may be provided in a form recorded on a storage medium such as a CD-ROM or a DVD-ROM. It is.

  Regarding each of the above embodiments, the following supplementary notes are further disclosed.

(Appendix 1)
Accept video input,
Extracting one or more still images from the video,
Accepting input of a figure or character for any position on the display image displaying the moving image,
The received graphic or character is superimposed and output on any of the positions on the display image displaying the moving image, and is also superimposed on a position corresponding to the any position on the display image of the still image. Image editing method executed by computer.

(Appendix 2)
2. The image editing method according to claim 1, wherein the outputting includes superimposing the graphic or the character on all of the extracted one or more still images and outputting.

(Appendix 3)
The process of receiving an input of a figure or a character for the moving image includes receiving an input for some images of the moving image,
The output processing includes superimposing the figure or the character on a still image corresponding to a part of the moving image that has received the input of the figure or the character among the one or more extracted still images. 2. The image editing method according to claim 1, wherein the image is edited and output.

(Appendix 4)
Accept video input,
Extracting one or more still images from the video,
Receiving input of a graphic or character for a first still image of the extracted one or more still images,
The received graphic or character is superimposed on the first still image and output, and among a plurality of still images included in the moving image, one or more images including a still image corresponding to the first still image are included. An image editing method executed by a computer that outputs the moving image while being superimposed on the image.

(Appendix 5)
5. The image editing method according to claim 4, wherein the outputting includes superimposing the received figure or character on all images after the still image corresponding to the first still image.

(Appendix 6)
A first reception unit that receives input of a video,
An extracting unit that extracts one or more still images from the moving image;
A second accepting unit that accepts input of a graphic or character at any position on the display image displaying the moving image;
The received graphic or character is superimposed and output on any of the positions on the display image displaying the moving image, and is also superimposed on a position corresponding to the any position on the display image of the still image. And an output unit for outputting
An image editing device including:

(Appendix 7)
6. The image editing apparatus according to claim 5, wherein the output unit superimposes the graphic or the character on all of the one or more extracted still images and outputs the superimposed image.

(Appendix 8)
The second receiving unit receives an input for a part of images of the moving image,
The output unit superimposes the figure or the character on a still image corresponding to a part of the moving image that has received the input of the figure or the character among the one or more extracted still images. 7. The image editing apparatus according to claim 6, wherein the image editing apparatus outputs the image.

(Appendix 9)
A first reception unit that receives input of a video,
An extracting unit that extracts one or more still images from the moving image;
A second accepting unit that accepts input of a graphic or character for a first still image of the one or more extracted still images,
The received graphic or character is superimposed on the first still image and output, and among a plurality of still images included in the moving image, one or more images including a still image corresponding to the first still image are included. An output unit that outputs the moving image while being superimposed on the image,
An image editing device including:

(Appendix 10)
The image editing apparatus according to claim 9, wherein the output unit superimposes the received figure or character on all of the images after the still image corresponding to the first still image.

(Appendix 11)
Accept video input,
Extracting one or more still images from the video,
Accepting input of a figure or character for any position on the display image displaying the moving image,
The received graphic or character is superimposed and output on any of the positions on the display image displaying the moving image, and is also superimposed on a position corresponding to the any position on the display image of the still image. An image editing program that causes a computer to execute the output process.

(Appendix 12)
12. The computer-readable storage medium according to claim 11, wherein the outputting includes superimposing the graphic or the character on all of the extracted one or more still images and outputting the superimposed image.

(Appendix 13)
The process of receiving an input of a figure or a character for the moving image includes receiving an input for some images of the moving image,
The output processing includes superimposing the figure or the character on a still image corresponding to a part of the moving image that has received the input of the figure or the character among the one or more extracted still images. 14. The image editing program according to claim 11, wherein the image editing program outputs the image editing program.

(Appendix 14)
Accept video input,
Extracting one or more still images from the video,
Receiving input of a graphic or character for a first still image of the extracted one or more still images,
The received graphic or character is superimposed on the first still image and output, and among a plurality of still images included in the moving image, one or more images including a still image corresponding to the first still image are included. An image editing program for causing a computer to execute a process of outputting the moving image while being superimposed on an image.

(Appendix 15)
15. The image editing program according to claim 14, wherein the outputting includes superimposing the received figure or character on all images after the still image corresponding to the first still image.

100 Video evaluation system 10, 210 Image editing device 11 Evaluation request receiving unit 12, 212 Still image extracting unit 13, 213 Evaluation graphic receiving unit 14, 214 Output unit 20 Storage unit 21, 221 Video evaluation information database 21A Video evaluation information table 21B , 221B Still image table 21C, 221C Video table 22 Video storage unit 23 Evaluation graphic storage unit 31 User terminal 32 Evaluator terminal 40 Computer 41 CPU
42 memory 43 storage device 50, 250 image editing program 70 evaluator screen 71 main display area 72 sub display area 73 drawing tool 76 user screen 77 main display area 78 sub display area

Claims (12)

Accept video input,
Extracting one or more frames from the video for display as a still image;
Accepting input of a figure or character at any position in the display area where the moving image is being played ,
The received graphic or character is superimposed and output on any of the positions of the display area for displaying the moving image, and a position corresponding to the any of the positions of the display area of the frame displayed as the still image A computer-implemented image editing method that superimposes and outputs images.   2. The image editing method according to claim 1, wherein in the outputting, the graphic or character is superimposed on all of the extracted one or more frames and output. The process of receiving an input of a figure or a character for the moving image includes receiving an input for some images of the moving image,
The output processing includes superimposing the graphic or the character on a frame corresponding to a part of the image of the moving image that has received the input of the graphic or the character among the one or more extracted frames. The image editing method according to claim 1, wherein the image is output. Accept video input,
Extracting one or more frames from the video for display as a still image;
Receiving input of a graphic or character for each of the first frames displayed as the still image among the one or more extracted frames;
The graphics or characters received for each of the first frames are output by being superimposed on each of the first frames displayed as the still image, and among a plurality of frames included in the moving image, In the case where the moving image segments overlap each other in each of the moving image segments including each of the first frames , in the overlapping moving image segments, the graphic corresponding to the first frame having a higher priority given in advance is set. Alternatively, an image editing method executed by a computer that outputs the moving image with characters superimposed .   5. The image editing method according to claim 4, wherein in the outputting, the received graphic or character is superimposed on all frames after the first frame. A first reception unit that receives input of a video,
An extracting unit configured to extract one or more frames to be displayed as a still image from the moving image;
A second accepting unit that accepts input of a graphic or character at any position in the display area where the moving image is being played ;
The received graphic or character is superimposed and output on any of the positions of the display area for displaying the moving image, and a position corresponding to the any of the positions of the display area of the frame displayed as the still image An output unit for superimposing and outputting the
An image editing device including: A first reception unit that receives input of a video,
An extracting unit configured to extract one or more frames to be displayed as a still image from the moving image;
A second receiving unit that receives an input of a graphic or a character for each of the first frames displayed as the still image among the one or more extracted frames;
The graphics or characters received for each of the first frames are output by being superimposed on each of the first frames displayed as the still image, and among a plurality of frames included in the moving image, In the case where the moving image segments overlap each other in each of the moving image segments including each of the first frames , in the overlapping moving image segments, the graphic corresponding to the first frame having a higher priority given in advance is set. Or an output unit for outputting the moving image in a state where characters are superimposed ,
An image editing device including: Accept video input,
Extracting one or more frames from the video for display as a still image;
Accepting input of a figure or character at any position in the display area where the moving image is being played ,
The received graphic or character is superimposed and output on any of the positions of the display area for displaying the moving image, and a position corresponding to the any of the positions of the display area of the frame displayed as the still image An image editing program that causes a computer to execute the process of superimposing and outputting data. Accept video input,
Extracting one or more frames from the video for display as a still image;
Receiving input of a graphic or character for each of the first frames displayed as the still image among the one or more extracted frames;
The graphics or characters received for each of the first frames are output by being superimposed on each of the first frames displayed as the still image, and among a plurality of frames included in the moving image, In the case where the moving image segments overlap each other in each of the moving image segments including each of the first frames , in the overlapping moving image segments, the graphic corresponding to the first frame having a higher priority given in advance is set. Alternatively, an image editing program for causing a computer to execute a process of outputting the moving image with characters superimposed . Accept video input,
Extracting a plurality of still images from the video,
Accepting input of a figure or character at any position in the display area where the moving image is being played ,
In a state where the received figure or character is superimposed on a position corresponding to any one of the positions in the display area of the still image based on a predetermined moving image reflection rule for each of the plurality of still images, An image editing method executed by a computer in which two or more still images in a still image are output side by side. A first reception unit that receives input of a video,
An extraction unit configured to extract a plurality of still images from the moving image,
A second accepting unit that accepts input of a graphic or character at any position in the display area where the moving image is being played ;
In a state where the received figure or character is superimposed on a position corresponding to any one of the positions in the display area of the still image based on a predetermined moving image reflection rule for each of the plurality of still images, An output unit that outputs two or more still images of the still images side by side;
An image editing device including: Accept video input,
Extracting a plurality of still images from the video,
Accepting input of a figure or character at any position in the display area where the moving image is being played ,
In a state where the received figure or character is superimposed on a position corresponding to any one of the positions in the display area of the still image based on a predetermined moving image reflection rule for each of the plurality of still images, An image editing program for causing a computer to execute a process of outputting two or more still images in a still image side by side.

Images