JP6632360B2 - Information processing device - Google Patents

Description

  The present invention relates to an information processing device.

  A technique for generating both a high-resolution and high-frame-rate moving image (main moving image) and a low-resolution and low-frame-rate moving image (proxy moving image) having the same content as the main moving image has been proposed. By using the proxy moving image, the moving image can be reproduced and edited more comfortably than by using the main moving image. Also, a technique has been proposed in which the result of editing performed on a proxy moving image by a mobile terminal such as a smartphone or a tablet device is reflected on a main moving image recorded on a server or the like (Patent Document 1). Patent Literature 1 describes, as editing of a moving image, designation of a scene cut out from the moving image.

  However, in the conventional technology, as a process of reflecting the result of the proxy editing on the main body moving image, two main frames corresponding to the two proxy frames selected as the start point and the end point of the scene are set in consideration of the frame rate. Only a selection process is performed. The proxy editing is editing performed using the proxy moving image, the proxy frame is a frame of the proxy moving image, and the main frame is a frame of the main moving image. Since the frame rate of the proxy moving image is different from the frame rate of the main moving image (since the frame rate of the proxy moving image is lower than the frame rate of the main moving image), a desired frame does not always exist in the proxy moving image. Therefore, in the related art, in proxy editing, the user cannot confirm a desired frame, or a wrong frame different from the desired frame is selected.

JP 2010-246008 A

  An object of the present invention is to provide a technique that allows a desired frame to be selected with high accuracy.

A first aspect of the present invention provides:
First selecting means for selecting a selected frame that is one of a plurality of first frames constituting the first moving image;
First acquisition means for acquiring a frame rate of the first moving image;
A second acquisition unit that acquires a frame rate of a second moving image having a higher frame rate than the first moving image corresponding to the first moving image;
Among the plurality of second frames constituting the second moving image, the position of the selected frame in the plurality of first frames , the frame rate of the first moving image, and the second particular frame is identified based on the video frame rate and, determining means for determining a predetermined number of frames before or after the specific frame,
A third obtaining unit that obtains the target frame among a plurality of second frames configuring the second moving image;
An information processing apparatus characterized by having:

A second aspect of the present invention provides:
A selecting step of selecting a selected frame that is one of a plurality of first frames constituting the first moving image;
A first obtaining step of obtaining a frame rate of the first moving image;
A second obtaining step of obtaining a frame rate of a second moving image having a higher frame rate than the first moving image corresponding to the first moving image;
Among the plurality of second frames constituting the second moving image, the position of the selected frame in the plurality of first frames , the frame rate of the first moving image, and the second particular frame is identified based on the video frame rate and a determining step of determining a predetermined number of frames before or after the specific frame,
A third obtaining step of obtaining the target frame among a plurality of second frames constituting the second moving image;
An information processing method characterized by having:

  A third aspect of the present invention is a program that causes a computer to function as each unit of the information processing apparatus described above.

  According to the present invention, a desired frame can be selected with high accuracy.

FIG. 2 is a block diagram illustrating a configuration example of a playback device according to the present embodiment. FIG. 2 is a block diagram illustrating a configuration example of an imaging device according to the present embodiment. FIG. 3 is a diagram illustrating an example of a main body video and a proxy video according to the present embodiment. FIG. 3 is a diagram illustrating an example of a file configuration of a moving image according to the embodiment. FIG. 2 is a diagram illustrating an example of data communication according to the present embodiment. FIG. 6 is a diagram illustrating an example of a difference between a main body video and a proxy video according to the present embodiment. Flow chart illustrating an example of a processing flow of the system according to the present embodiment. FIG. 4 is a diagram illustrating an example of a GUI image according to the embodiment. 4 is a flowchart illustrating an example of a processing flow of the playback device according to the present embodiment. FIG. 4 is a diagram illustrating an example of a GUI image according to the embodiment. Flow chart illustrating an example of a processing flow of the system according to the present embodiment. FIG. 5 is a diagram illustrating an example of a notification process according to the embodiment.

  Hereinafter, embodiments of the present invention will be described. FIG. 1 is a block diagram illustrating a configuration example of a playback device 1000 that is an information processing device according to the present embodiment. The playback device 1000 includes a control unit 1001, a first storage unit 1002, a second storage unit 1003, a display unit 1004, an operation unit 1005, a communication unit 1006, an audio output unit 1007, and the like. Note that the information processing device according to the present embodiment is not limited to a playback device. For example, as an information processing apparatus according to the present embodiment, a PC (personal computer), a mobile terminal (smartphone, tablet device, portable media player, etc.), an image display device (a liquid crystal display device, an organic EL display device, a plasma display) Devices, MEMS shutter type display devices, etc.), servers, etc. may be used.

  The control unit 1001 controls various calculations and processes of each functional unit of the playback device 1000 according to a signal input to the control unit 1001, a program described below, and the like. As the control unit 1001, for example, a central processing unit (CPU) can be used. Note that one piece of hardware may be used as the control unit 1001, or a plurality of pieces of hardware may be used. The processing of the playback device 1000 may be controlled by a plurality of pieces of hardware sharing and executing the processing.

  The first storage unit 1002 is a storage unit that temporarily stores various data, and is used for processing (operation) of the control unit 1001. As the first storage unit 1002, for example, a volatile memory can be used. The second storage unit 1003 is a storage unit that can electrically store various data. The second storage unit 1003 stores, for example, various programs (including firmware) used in the playback device 1000, various setting information, and the like. As the second storage unit 1003, for example, a nonvolatile memory can be used.

  In the present embodiment, the data acquired from the external device by the playback device 1000 (communication unit 1006) is recorded in the second storage unit 1003, but is not limited to this. For example, data acquired from the external device by the playback device 1000 may be recorded in another storage unit (a storage unit that is detachable from the playback device 1000 or another storage unit built in the playback device 1000). Good. As the other storage unit, a semiconductor memory (memory card), a magnetic disk (hard disk), an optical disk (CD, DVD, Blu-ray disk), or the like can be used. In this case, the playback device 1000 only needs to have a function of accessing the other storage unit, reading and writing data from and to the other storage unit, deleting data stored in the other storage unit, and the like. .

  The display unit 1004 displays an image on a screen. The display unit 1004 displays an image based on image data, displays a graphic image (characters, icons, and the like) for interactive operation, and the like. As the display portion 1004, for example, a liquid crystal display panel, an organic EL display panel, a plasma display panel, a MEMS shutter display panel, or the like can be used. Note that an image display device separate from the playback device 1000 may be used as the display unit 1004. The playback device 1000 may have a function of controlling display on the display unit 1004.

  The operation unit 1005 receives a user operation on the playback device 1000. The operation unit 1005 outputs an operation signal corresponding to a user operation to the control unit 1001. Then, the control unit 1001 performs a process according to the operation signal. That is, the control unit 1001 performs processing according to a user operation on the playback device 1000. As the operation unit 1005, for example, an input device such as a physical button, a touch panel, and a keyboard can be used. Further, as the operation unit 1005, an input device separate from the playback device 1000 can be used. As an input device separate from the playback device 1000, for example, a keyboard, a mouse, a remote controller, and the like can be used. The playback device 1000 may have a function of receiving an electric signal corresponding to a user operation using the input device.

  The communication unit 1006 connects the playback device 1000 to an external device, and transmits and receives data (control commands, image data, and the like) between the playback device 1000 and the external device. As the communication protocol, for example, PTP (Picture Transfer Protocol), MTP (Media Transfer Protocol), or the like can be used. Note that the communication unit 1006 may connect the playback device 1000 to an external device using a cable such as a USB (Universal Serial Bus) cable. The transmission and reception of data between the playback device 1000 and the external device may be performed using a cable. In addition, the communication unit 1006 may wirelessly connect the playback device 1000 to an external device using a wireless LAN or the like. The playback device may be directly connected to an external device, or may be connected to the external device via a network such as the Internet, a server, or the like.

  The audio output unit 1007 outputs audio based on audio data. As the audio output unit 1007, for example, a speaker can be used. Note that an audio output device separate from the playback device 1000 may be used as the audio output unit 1007. The playback device 1000 only needs to have a function of controlling the output of audio by the audio output unit 1007.

  FIG. 2 is a block diagram illustrating a configuration example of an imaging device 2000 that is an external device of the playback device 1000. The imaging device 2000 includes a first storage unit 2001, a storage unit 2002, a control unit 2003, an operation unit 2004, an imaging unit 2005, a display unit 2006, an optical member 2007, an audio input unit 2008, a communication unit 2009, and a second storage unit. 2010, and the like. Note that the external device of the playback device 1000 is not limited to an imaging device. For example, a PC, a mobile terminal, an image display device, a playback device, a server, or the like may be used as an external device of the playback device 1000.

  The control unit 2003 controls various calculations and processes of each functional unit of the imaging device 2000 according to a signal input to the control unit 2003, a program described later, and the like. As the control unit 2003, for example, a CPU can be used. Note that one piece of hardware may be used as the control unit 2003, or a plurality of pieces of hardware may be used. The processing of the imaging device 2000 may be controlled by a plurality of pieces of hardware sharing and executing the processing.

  The first storage unit 2001 is a storage unit that temporarily stores various data, and is used for processing (operation) of the control unit 2003. As the first storage unit 2001, for example, a volatile memory (DRAM or the like) can be used. The second storage unit 2010 is a storage unit that can electrically store various data. The second storage unit 2010 stores, for example, various programs (including firmware) used in the imaging device 2000, various setting information, and the like. As the second storage unit 2010, for example, a nonvolatile memory (a flash memory or the like) can be used.

  The optical member 2007 includes a lens, a shutter, an aperture, and the like. The optical member 2007 focuses light from a subject on the imaging unit 2005. Specifically, the optical member 2007 controls the amount of light from the subject to an appropriate amount at an appropriate timing, and forms an image of the light after the amount of light is controlled on the imaging unit 2005.

  The imaging unit 2005 generates image data, which is digital data, by converting light (subject light) formed by the optical member 2007 into an electric signal (imaging; photographing). The imaging unit 2005 outputs the image data (photographed image data) generated by photographing. As the imaging unit 2005, for example, an imaging device such as a CMOS sensor or a CCD sensor can be used.

  The storage unit 2002 is a storage unit that can store various data. For example, the storage unit 2002 records captured image data. As the storage unit 2002, a semiconductor memory (memory card), a magnetic disk (hard disk), an optical disk (CD, DVD, Blu-ray disk), or the like can be used. Note that the storage unit 2002 may be a storage unit that is detachable from the imaging device 2000, or may be a storage unit built in the imaging device 2000. The imaging device 2000 only needs to have a function of accessing the storage unit 2002, reading and writing data from and to the storage unit 2002, deleting data stored in the storage unit 2002, and the like.

  The operation unit 2004 receives a user operation on the imaging device 2000. The operation unit 2004 outputs an operation signal according to a user operation to the control unit 2003. Then, the control unit 2003 performs a process according to the operation signal. That is, the control unit 2003 performs a process according to a user operation on the imaging device 2000. As the operation unit 2004, for example, an input device such as a physical button, a touch panel, and a keyboard can be used. In addition, as the operation unit 2004, an input device separate from the imaging device 2000 can be used. As an input device separate from the imaging device 2000, for example, a keyboard, a mouse, a remote controller, or the like can be used. The imaging device 2000 only needs to have a function of receiving an electric signal according to a user operation using the input device.

  The display unit 2006 displays an image on a screen. The display unit 2006 displays an image based on captured image data, displays a graphic image (characters, icons, and the like) for interactive operation, and the like. In addition, the display unit 2006 may display an image representing a shooting target at the time of shooting. That is, the display unit 2006 may be used as an electronic viewfinder. Note that a display device separate from the imaging device 2000 may be used as the display unit 2006. The imaging device 2000 only needs to have a function of controlling display on the display unit 2006.

  The voice input unit 2008 generates voice data (voice signal) based on the voice input to the voice input unit 2008. As the voice input unit 2008, for example, a microphone can be used. Note that a voice input device separate from the imaging device 2000 may be used as the voice input unit 2008. The imaging device 2000 only needs to have a function of acquiring audio data generated by the audio input unit 2008.

  The communication unit 2009 connects the imaging device 2000 to an external device, and transmits and receives data (control commands, image data, and the like) between the imaging device 2000 and the external device.

  FIG. 3 is a diagram illustrating an example of a captured image (captured image data) obtained by the imaging device 2000. In FIG. 3, reference numeral 3001 denotes the imaging device 2000 of FIG. The imaging device 3001 can perform moving image capturing for obtaining a moving image as a captured image. Then, the imaging device 3001 can generate the first moving image 3003 and the second moving image 3002 by one moving image shooting. Therefore, the first moving image 3003 is a moving image having the same content as the second moving image 3002. The second moving image 3002 corresponds to the first moving image 3003, and has a higher frame rate than the first moving image 3003. That is, the first moving image 3003 corresponds to the second moving image 3002, and has a lower frame rate than the second moving image 3002. Therefore, by using the first moving image 3003, the moving image can be reproduced and edited more comfortably than using the second moving image 3002.

  Although the image resolution of the first moving image 3003 and the image resolution of the second moving image 3002 are not particularly limited, in the example of FIG. 3, the image resolution of the first moving image 3003 is lower than the image resolution of the second moving image 3002. If the image resolution of the first moving image 3003 is lower than the image resolution of the second moving image 3002, the reproduction and editing of the moving image (first moving image 3003) can be performed more comfortably. Hereinafter, the first moving image is referred to as “proxy moving image”, and the second moving image is referred to as “body moving image”.

  Further, the imaging device 3001 can also perform still image shooting for obtaining a still image as a shot image. Specifically, the imaging device 3001 can perform various types of still image capturing to obtain various types of still images, various types of moving image capturing to obtain various types of moving images, and the like.

  Note that the generation of the main moving image 3002 and the generation of the proxy moving image 3003 may be performed in parallel, or the proxy moving image 3003 may be generated by performing image processing on the main moving image 3002 after generating the main moving image 3002. . The process of generating the proxy moving image 3003 from the main moving image 3002 may be performed by a device different from the imaging device 3001.

  In the present embodiment, an example will be described in which the first moving image and the second moving image are moving images obtained by shooting, but the first moving image and the second moving image are not particularly limited. For example, the first moving image and the second moving image may be moving image of broadcast content (broadcast program, CM (commercial message), etc.), CG (computer graphic) moving image, or animation. It may be a movie.

FIG. 4 is a diagram illustrating an example of a file configuration of a main body moving image. The main body moving image file (moving image file) 400 includes attribute information 410, related attribute information 420, frame attribute information 430,
The frame data 440 is included. The file structure of the proxy moving image is the same as the file structure of the main moving image. Specifically, the proxy moving image has a file configuration in which “proxy moving image” in the following description is replaced with “proxy moving image”.

  The attribute information 410 is attribute information of the main moving image (the main moving image corresponding to the moving image file 400). For example, the attribute information 410 includes information indicating the shooting date and time of the main body moving image (shooting date and time information), the model name of the imaging device (camera model name) used for shooting the main body moving image, the file size of the moving image file 400, Image resolution, start time code (time code of the first frame of the main video), end time code (time code of the last frame of the main video), duration of the main video, frame rate of the main video, main video (video file 400), and other additional information. As the ID, for example, a UMID (Unique Material Identifier) defined by SMPTE (Society of Motion Picture and Television Engineers) is used. UMID is often used as a moving image identifier.

  The related attribute information 420 is attribute information of a proxy moving image related to the main moving image (moving image file 400). For example, the related attribute information 420 includes the duration of the proxy moving image, the frame rate of the proxy moving image, the UMID of the proxy moving image, and other additional information. The attribute information 410 and the related attribute information 420 are described in the header of the moving image file 400.

  The frame attribute information 430 is attribute information of a frame of the main moving image. For example, the frame attribute information 430 includes a time code of the frame, shooting parameters (shutter speed, aperture value, ISO sensitivity, shooting mode, etc.) used for shooting the frame, and other additional information. The frame data 440 is image data of a frame of the main body moving image. The frame attribute information 430 is described in a header part of a data area related to the frame, and the frame data 440 is described in an image data part of the data area related to the frame. As shown in FIG. 4, the moving image file 400 includes frame attribute information 430 and frame data 440 for each frame of the main moving image.

  The data format of the frame data 440 of the main moving image and the data format of the frame data 440 of the proxy moving image are not particularly limited. For example, RAW format image data is used as frame data 440 of the main moving image, and H.264 is used as frame data 440 of the proxy moving image. Image data compressed in the H.264 format is used.

  FIG. 5 is a diagram illustrating an example of data transmission and reception between the imaging device and the playback device. In FIG. 5, reference numeral 5001 denotes the imaging device 2000 of FIG. 2, and reference numeral 5002 denotes the reproducing device 1000 of FIG. For example, data transmission and reception between the imaging device 5001 and the reproduction device 5002 are performed by wireless communication using Wi-Fi, Bluetooth, or the like. FIG. 5 shows a state where the proxy moving image 5003 is transmitted from the imaging device 5001 to the playback device 5002. Note that data transmission and reception between the imaging device 5001 and the playback device 5002 may be performed by wired communication. Alternatively, the imaging device 5001 may record data in an external storage such as an SD card, the external storage may be detached from the imaging device 5001 and attached to the playback device 5002, and the playback device may read data from the external storage.

FIG. 6 is a diagram illustrating an example of a difference between the main moving image and the proxy moving image. Hereinafter, each of the plurality of frames (second frame) constituting the main moving image is referred to as “main frame”, and each of the plurality of frames (first frame) forming the proxy moving image is referred to as “proxy frame”. In the example of FIG. 6, the frame rate of the main moving image 6001 is 120 fps, and the frame rate of the proxy moving image 6002 is 30 fps. And proxy video 6002
Is a moving image obtained by thinning out the main body frame from the main moving image 6001. In the example of FIG. 6, three main frames are thinned out between two main frames corresponding to two consecutive proxy frames. A thinning number M, which is the number of body frames thinned out between two body frames corresponding to two consecutive proxy frames, can be calculated using the following equation 1. In Equation 1, “FR_O” is the frame rate of the main moving image, and “FR_P” is the frame rate of the proxy moving image. In the example of FIG. 6, the thinning number M = 120 ÷ 30-1 = 3 is obtained.

M = FR_O ÷ FR_P-1 (Equation 1)

  Note that the proxy moving image need not be a moving image obtained by thinning out the main body frame from the main moving image. For example, the plurality of proxy frames constituting the proxy moving image may include an interpolation frame generated from two or more frames of the main moving image. In that case, at least one of the main frames used to generate the interpolated frame can be interpreted as a main frame corresponding to the interpolated frame. The main frame corresponding to the interpolation frame can be detected based on information on the interpolation method, information indicating the correspondence between the proxy frame and the main frame, and the like. Such information is included, for example, in a moving image file. Further, the frame rate of the main moving image and the frame rate of the proxy moving image are not particularly limited. The frame rate of the main moving image may be higher or lower than 120 fps, and the frame rate of the proxy moving image may be higher or lower than 30 fps.

  FIG. 7 is a flowchart illustrating an example of a processing flow of the system according to the present embodiment (a system including the playback device 1000 in FIG. 1 and the imaging device 2000 in FIG. 2). FIG. 7 shows a processing flow in the case where the first selection processing is performed by the playback device 1000. In the first selection process, any of a plurality of proxy frames is selected as a candidate for a desired frame (a frame to be cut out). At the start of the processing flow in FIG. 7, it is assumed that the second storage unit 1003 of the playback device 1000 stores the proxy moving image, and the storage unit 2002 of the imaging device 2000 stores the main body moving image.

  First, in step S7001, the control unit 1001 of the playback device 1000 selects one of a plurality of proxy frames included in a proxy moving image (first selection process). Hereinafter, the first frame selected by the first selection process is referred to as a “selected frame”. Details of the first selection process will be described later. Next, in step S7002, the control unit 1001 acquires frame rate information of the proxy moving image from the moving image file of the proxy moving image (the attribute information 410 in FIG. 4).

  In step S7003, the control unit 1001 requests the frame rate of the main body moving image to the imaging device 2000 via the communication unit 1006 of the playback device 1000. Next, in step S7004, the control unit 2003 of the imaging apparatus 2000 outputs frame rate information of the main body moving image in response to receiving the request in step S7003 from the playback apparatus 1000 via the communication unit 2009 of the imaging apparatus 2000. I do. More specifically, the control unit 2003 acquires frame rate information of the main moving image from the moving image file of the main moving image (the attribute information 410 in FIG. 4), and transmits the obtained information to the playback device 1000 via the communication unit 2009. Output. As a result, the control unit 1001 acquires the frame rate information of the main body moving image from the imaging device 2000 via the communication unit 1006.

For example, in S7003, the control unit 1001 obtains the UMID of the main body moving image from the moving image file of the proxy moving image (the related attribute information 420 in FIG. 4), and outputs the obtained UMID to the imaging device 2000. The control unit 2003 can uniquely determine the main body moving image from the UMID output from the playback device 1000. In step S7004, control unit 2003 outputs, to playback device 1000, information on the frame rate of the main moving image corresponding to the UMID output from playback device 1000.

  As shown in FIG. 4, the video file of the main video may include the frame rate information of the proxy video, or the video file of the proxy video may include the frame rate information of the main video. . When the moving image file of the proxy moving image includes the frame rate information of the main moving image, the control unit 1001 may acquire the frame rate information of the main moving image in the moving image file of the proxy moving image. Then, the processing of S7003 and S7004 may be omitted. The processing of S7003 and S7004 is necessary, for example, when the moving image file of the proxy moving image does not include the frame rate information of the main moving image.

  In the processing of S7005 and S7006, the control unit 1001 determines a specific section which is a partial section of the main moving image based on the frame rate of the main moving image, the frame rate of the proxy moving image, and the selected frame. For example, the specific section is a section to be referred to when selecting a desired frame (a frame to be cut out) from the section of the main moving image. The specific section is also a section that does not need to be referred to when selecting a desired frame among the sections of the main body moving image. In the present embodiment, a section including the main frame corresponding to the selected frame is determined as the specific section from among the plurality of main frames constituting the main moving image. This is because there is a high possibility that the desired frame exists around (before and after) the selected frame.

  Since the selected frame is a desired frame candidate, there is a high possibility that the desired frame is not included in the section before the proxy frame immediately before the selected frame. Therefore, it is considered that the main frame corresponding to the proxy frame immediately before the selected frame need not be referred to when selecting a desired frame. Therefore, it is preferable that the section not including the main frame corresponding to the proxy frame immediately before the selected frame is determined as the specific section.

  In addition, since the selected frame is a candidate for a desired frame, there is a high possibility that the desired frame is not included in a section subsequent to the proxy frame immediately after the selected frame. Therefore, it is considered that there is no need to refer to the main frame corresponding to the proxy frame immediately after the selected frame when selecting a desired frame. Therefore, it is preferable that a section not including the main frame corresponding to the proxy frame immediately after the selected frame is determined as the specific section.

  In the present embodiment, from the main body frame immediately after the main frame corresponding to the proxy frame immediately before the selected frame to the main body frame immediately before the main frame corresponding to the proxy frame immediately after the selected frame. An example of determining the section as a specific section will be described. However, the specific section is not limited to this. For example, a section up to the main moving picture frame corresponding to the selected frame, a section up to the main moving picture frame m1 before or after the main moving picture frame corresponding to the selected frame, and the like may be determined as the specific sections (m1 is a specific section). An integer greater than or equal to 1). A section from the main moving image frame corresponding to the selected frame, a section from the main moving image frame m2 before or after the main moving image frame corresponding to the selected frame, and the like may be determined as the specific section (m2 is 1 or more). Integer).

Further, a section from the main moving image frame corresponding to the proxy frame immediately before the selected frame may be determined as the specific section. A section from the main moving image frame m3 after the main moving image frame corresponding to the proxy frame immediately before the selected frame may be determined as the specific section (m3 is an integer of 2 or more). A section from the main moving image frame m4 before the main moving image frame corresponding to the proxy frame immediately before the selected frame may be determined as the specific section (m4 is an integer of 1 or more). The specific section may be determined based on the proxy frame M1 before the selected frame instead of the proxy frame immediately before the selected frame (M1 is an integer of 2 or more).

  A section up to the main moving image frame corresponding to the proxy frame immediately after the selected frame may be determined as the specific section. A section up to the main moving image frame m5 before the main moving image frame corresponding to the proxy frame immediately after the selected frame may be determined as the specific section (m5 is an integer of 2 or more). A section up to the main moving image frame m6 after the main moving image frame corresponding to the proxy frame immediately after the selected frame may be determined as the specific section (m6 is an integer of 1 or more). The specific section may be determined based on the proxy frame M2 after the selected frame, not the proxy frame immediately after the selected frame (M2 is an integer of 2 or more). The values of m1 to m6, M1, and M2 may be predetermined fixed values, may be values that can be changed by the user, or may be automatically determined according to the type of moving image and the like. Value may be used.

  In step S7005, the control unit 1001 thins out the number of target main frames that are the main frames to be acquired based on the information (the frame rate of the main moving image and the frame rate of the proxy moving image) obtained in S7002 and S7004. Get the number M. As described above, the thinning number M is the number of body frames thinned out between two body frames corresponding to two consecutive proxy frames, and can be calculated using Equation 1. In the present embodiment, the frame rate FR_O of the main moving image is 120 fps, and the frame rate FR_P of the proxy moving image is 30 fps. Therefore, the number M = 3 is obtained.

  In step S7006, the control unit 1001 determines a target main body frame based on the selected frame selected in step S7001, the information acquired in steps S7002 and S7004, and the number M determined in step S7005. All the main frames included in the specific section may be determined as the target main frame, but the main frame corresponding to the selected frame is unnecessary because the selected frame can be referred to when a desired frame is selected. There is. Therefore, in S7006, a main frame included in the specific section and not corresponding to the selected frame is determined as the target main frame. The processing of S7006 will be described in detail below.

First, the control unit 1001 detects a main body frame corresponding to the selected frame based on the selected frame, the frame rate FR_O of the main moving image, and the frame rate FR_P of the proxy moving image. Assuming that the first frame of the moving image is the 0th frame, the time position P_O of the main frame corresponding to the time position P_P of the selected frame can be calculated using the following equation 2. In the present embodiment, it is assumed that the P_P = 5th proxy frame of the proxy moving image is selected as the selected frame. Therefore, the P_O = 5 × (120 ÷ 30) = 20th main frame is detected as the main frame corresponding to the selected frame.

P_O = P_P × (FR_O ÷ FR_P) (Formula 2)

  Next, the control unit 1001 determines the target main body frame based on the time position P_O (the detection result of the main body frame corresponding to the selected frame) and the number M. Specifically, the M main frames immediately before the P_O-th main frame and the M main frames immediately after the P_O-th main frame are determined as target main frames. In the present embodiment, the time position P_O = 20 and the number M = 3. Therefore, three main frames immediately before the twentieth main frame and three main frames immediately after the twentieth main frame are determined as target main frames. That is, the 17, 18, 19, 21, 22, and 23rd frames of the main moving image are determined as the target main frames.

  In addition, since the section including the main body frame corresponding to the selected frame and the target main body frame is the specific section, the process of determining the target main body frame can also be referred to as “the process of determining the specific section”. Further, as described above, since the definition of the specific section is not particularly limited, the method of determining the specific section (target main body frame) is not particularly limited. For example, the specific section may be determined without acquiring the number M. Further, when information indicating the correspondence between the proxy frame and the main frame is prepared in advance, the main frame corresponding to the selected frame is detected based on the prepared information and the selected frame. Good.

  Next, in S7007, the control unit 1001 requests the imaging device 2000 via the communication unit 1006 for the target main body frame. Then, in S7008, in response to receiving the request in S7007 from playback device 1000 via communication unit 2009, control unit 2003 outputs the target main body frame to playback device 1000 via communication unit 2009. As a result, the control unit 1001 acquires the target main body frame from the imaging device 2000 via the communication unit 1006 (S7009).

  Note that the target main body frame may or may not be output from the imaging device 2000 to the playback device 1000 without performing image processing such as changing the image resolution. For example, the control unit 2003 may reduce the image resolution of the target main body frame. In the present embodiment, the image resolution of the main frame is higher than the image resolution of the proxy frame. In such a case, the control unit 2003 may reduce the image resolution of the target main body frame to an image resolution substantially equal to the image resolution of the proxy frame. Then, the control unit 2003 may output the target main body frame after the image resolution has been reduced to the playback device 1000. In the present embodiment, “substantially the same” includes “perfect match”. Further, only the thumbnail image of the target main body frame may be output from the imaging device 2000 to the playback device 1000. Accordingly, the playback device 1000 reduces the transmission amount and transmission time of the target body frame from the imaging device 2000 to the playback device 1000, and performs processing using the target body frame (display, deletion, and the like of the target body frame). Processing load at the time can be reduced.

  Further, in the present embodiment, the main body frame corresponding to the selected frame is not determined as the target main body frame. Therefore, the total number of target main frames can be reduced, and the transmission amount and transmission time of the target main frames from the imaging device 2000 to the reproducing device 1000 can be reduced. However, a high image resolution may be required for the selected frame. In such a case, it is preferable that the main body frame corresponding to the selected frame is further determined as the target main body frame.

  FIG. 8 is a diagram illustrating an example of a GUI (Graphical User Interface) image displayed on the display unit 1004 of the playback device 1000 when the first selection process in S7001 is performed. The control unit 1001 performs display control for displaying the GUI image shown in FIG. 8 on the display unit 1004 when the first selection process is performed. Reference numerals 8000 and 8100 indicate a method of the first selection processing. In the GUI image of FIG. 8, one or more proxy frames (one or more thumbnail images respectively corresponding to one or more proxy frames) are arranged. When a swipe operation of tracing the screen as indicated by arrow 8101 is performed, control unit 1001 switches the proxy frame (thumbnail image) displayed on display unit 1004 to another proxy frame by scrolling.

In the method 8000, the control unit 1001 selects a specified proxy frame as a selected frame in response to a user operation of specifying one of a plurality of proxy frames. Specifically, when a user operation for designating a thumbnail image of a proxy frame is performed as indicated by reference numeral 8001, the control unit 1001 causes the proxy frame (corresponding to the designated thumbnail image) Non-thumbnail image). Thereafter, when a touch operation is performed on determination button 8003, control unit 1001 determines the designated proxy frame as a selected frame.

  In the method 8100, the control unit 1001 selects a selected frame based on the proxy frame (thumbnail image) displayed on the display unit 1004. Specifically, as indicated by reference numeral 8102, the control unit 1001 transmits the proxy frame N frames after the proxy frame in which the thumbnail image is displayed at a predetermined position (center, right end, left end, etc.) of the display unit 1004. Is selected as the selection frame. N is an integer of 1 or more, for example, 50. The value of N may be a predetermined fixed value, may be a value that can be changed by the user, or may be a value that is automatically determined according to the type of moving image and the like. Good. Further, the value of N may be determined in consideration of at least the scroll speed at which the proxy frame (thumbnail image) displayed on the display unit 1004 is switched to another proxy frame by scrolling. For example, a larger value may be determined as the value of N as the scroll speed increases. The value of N may be determined in consideration of at least the transmission speed when transmitting the main body frame from the imaging device 2000 to the playback device 1000. For example, a smaller value may be determined as the value of N as the transmission speed is lower. The value of N may be determined based on one of the scroll speed and the transmission speed, or the value of N may be determined based on both the scroll speed and the transmission speed.

  The method of the first selection processing is not limited to the above method. For example, a proxy frame having a specific image feature may be automatically selected as a selected frame based on the image feature of each proxy frame. As long as a specific proxy frame can be selected, the proxy frame may be selected by any method.

  In the present embodiment, after acquiring the target main body frame, the control unit 1001 performs display control for displaying at least one of a plurality of proxy frames and a plurality of frames including the obtained target main body frame on the display unit 1004. . For example, the control unit 1001 adds the thumbnail image of the target main body frame to the GUI image of FIG. Thereby, the user of the playback device 1000 can view not only the proxy frame but also the target main body frame. The thumbnail image of the target main body frame may be generated inside the playback device 1000 or may be input to the playback device 1000 from outside.

  FIG. 9 is a flowchart illustrating an example of a processing flow of the playback device 1000. FIG. 9 shows a processing flow in the case where the second selection processing is performed by the playback device 1000. In the second selection process, one of a plurality of frames including a plurality of proxy frames and the acquired target main body frame is selected as a desired frame (a frame to be cut out) in accordance with a user operation. The processing flow of FIG. 9 is performed after the processing flow of FIG. 7 is performed.

  First, in S9001, the control unit 1001 performs display control to display at least one of a plurality of frames including the plurality of proxy frames and the target main body frame acquired in S7009 on the display unit 1004. Reference numeral 10000 in FIG. 10 indicates an example of a display form of the display unit 1004 after the processing of S9001 is performed. In S9001, display control for displaying a GUI image in which the selected frame 10001 and the six target main body frames 10002 are arranged as in the display mode 10000 is performed.

  Next, in step S9002, the control unit 1001 waits for a user operation to specify one of the displayed seven frames. When a user operation is performed, the control unit 1001 selects the instructed frame as a desired frame.

In S9003, control unit 1001 automatically deletes the target main body frame (non-selected frame) not selected in S9002. Specifically, in S7009 of FIG. 7, the control unit 1001 records the acquired target main body frame in the first storage unit 1002 or the second storage unit 1003. In step S9003, the control unit 1001 deletes the target main body frame not selected in step S9002 from the first storage unit 1002 or the second storage unit 1003. Note that the process of deleting the target main body frame may be a process of deleting the target main body frame from the screen instead of the process of deleting the target main body frame from the storage unit. Reference numeral 10100 in FIG. 10 indicates an example of a display mode of the display unit 1004 after the processing of S9003 is performed. When the main body frame 10101 is selected in S9002 as in the display mode 10100, the remaining five main body frames are deleted from the screen.

  As described above, according to the present embodiment, in the playback device, the specific section is determined, and the main body frame included in the specific section is obtained from the external device. Thus, when a desired frame is selected using the playback device, not only the proxy frame but also the main frame of the specific section can be used. As a result, a desired frame can be selected with high accuracy. For example, when the desired frame is the main body frame and there is no proxy frame corresponding to the desired frame, it is possible to prevent a proxy frame that is not the desired frame from being erroneously selected as the desired frame. .

  When the first selection process is performed, the user may be notified of information (notification information) regarding that the frame rate of the proxy moving image is lower than the frame rate of the main moving image. FIG. 11 is a flowchart illustrating an example of a processing flow of the system according to the present embodiment. FIG. 11 shows a processing flow when the notification processing is performed by the playback device 1000. In the notification process, the above-described notification information is notified to the user. For example, in the proxy moving image, information indicating that some main frames are thinned out, information indicating the frame rate of the proxy moving image and the frame rate of the main moving image, and the like are notified as notification information. Information indicating that a desired frame cannot be selected with high accuracy unless a main body frame is obtained, information indicating that processing for obtaining a main body frame is performed after the first selection processing, and the like may be notified as notification information. .

  First, in step S11001, the control unit 1001 activates (validates) a frame selection function that is a function of selecting a desired frame according to a user operation. Next, the processing of S11002 to S11005 is performed. Since the processing of S11002 to S11005 is the same as the processing of S7002 to S7005 in FIG. 7, the description of the processing of S11002 to S11005 is omitted. Subsequent to S11005, in S11006, the control unit 1001 performs the above-described notification processing. Specifically, as shown in FIG. 12, the control unit 1001 displays a thumbnail view image in which the thumbnail images of the respective proxy frames are arranged, and displays the thumbnail images of the thinned main frame as notification processing. Instead, display control for displaying a broken line 12000 is performed. The number and time position of the thinned main body frames can be determined from the thinned number N. The thumbnail view image is, for example, the GUI image in FIG. By performing such display control, when the user designates the selected frame, it is possible to easily grasp that some body frames have been thinned out from the body moving image.

  Note that the processing of S11002 to S11005 is the same as the processing of S7002 to S7005 in FIG. 7, and thus the processing of S7002 to S7005 may be omitted. For example, after the process of S11006, the processes of S7001, S7006 to S7009 of FIG. 7 may be performed, and then the process flow of FIG. 9 may be performed.

Note that the notification process is not limited to the above-described display control. For example, another graphic image (such as a predetermined icon) other than the broken line 12000 may be used as the graphic image indicating the thinned main body frame. As the notification processing, display control for displaying a text image in which notification information is described may be performed. The notification information may be notified by a method other than the image display. For example, the notification information may be notified by audio output, lighting of a lamp, or the like. The notification information may be notified to the user by any method as long as the user can grasp the notification information.

  As described above, the present invention has been described in detail based on the preferred embodiments. However, the present invention is not limited to this specific embodiment, and various forms that do not depart from the gist of the present invention are also included in the present invention. included. For example, the processing of S9003 of FIG. 9, the processing of S11006 of FIG. 11, and the like may be omitted. Further, even if processing P1 for selecting a desired frame from only a plurality of proxy frames and processing P2 for selecting a desired frame from a plurality of frames including a plurality of proxy frames and a target main body frame can be switched and executed. Good. For example, the playback device may be able to set one of an operation mode for executing the process P1 and an operation mode for executing the process P2. Then, the playback device may execute a process corresponding to the set operation mode. In addition, the playback device may ask the user whether to execute the process P1 or the process P2. Then, the playback device may execute a process specified by the user.

<Other embodiments>
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

  1000, 5002: playback device 1001: control unit 1006: communication unit

Claims (9)

First selecting means for selecting a selected frame that is one of a plurality of first frames constituting the first moving image;
First acquisition means for acquiring a frame rate of the first moving image;
A second acquisition unit that acquires a frame rate of a second moving image having a higher frame rate than the first moving image corresponding to the first moving image;
Among the plurality of second frames constituting the second moving image, the position of the selected frame in the plurality of first frames , the frame rate of the first moving image, and the second particular frame is identified based on the video frame rate and, determining means for determining a predetermined number of frames before or after the specific frame,
A third obtaining unit that obtains the target frame among a plurality of second frames configuring the second moving image;
An information processing apparatus comprising: 2. The information processing apparatus according to claim 1, further comprising control means for performing display control for displaying the target frame on a display unit. The image resolution of the second frame is higher than the image resolution of the first frame,
The information processing apparatus according to claim 2, wherein the control unit performs display control for displaying the target frame with reduced image resolution on the display unit. Said determining means includes a position of said selected frame in said plurality of first frame, second front Symbol plurality by the product of the second value the frame rate divided by the first moving frame rate of the moving the information processing apparatus according to any one of claims 1 to 3, characterized in that identifying the JP TAFE frame in the frame. When the first selecting means selects the selected frame, the first selecting means further includes a notifying means for notifying information that the frame rate of the first moving image is lower than the frame rate of the second moving image. The information processing apparatus according to claim 1. The information processing apparatus according to claim 1, wherein the third acquisition unit acquires the target frame from an external device. A second selection unit that selects one of a plurality of frames including the plurality of first frames and the target frame in accordance with a user operation;
Deleting means for deleting the target frame not selected by the second selecting means;
The information processing apparatus according to claim 1, further comprising: A selecting step of selecting a selected frame that is one of a plurality of first frames constituting the first moving image;
A first obtaining step of obtaining a frame rate of the first moving image;
A second obtaining step of obtaining a frame rate of a second moving image having a higher frame rate than the first moving image corresponding to the first moving image;
Among the plurality of second frames constituting the second moving image, the position of the selected frame in the plurality of first frames , the frame rate of the first moving image, and the second particular frame is identified based on the video frame rate and a determining step of determining a predetermined number of frames before or after the specific frame,
A third obtaining step of obtaining the target frame among a plurality of second frames constituting the second moving image;
An information processing method comprising:   A program for causing a computer to function as each unit of the information processing apparatus according to claim 1.

Images