JP7051406B2 - Mobile terminal for video editing, capture device, information processing system, information processing method, information generation method and information processing program - Google Patents

Description

The present invention relates to a mobile terminal for video editing, a capture device, an information processing system, an information processing method, an information generation method, and an information processing program.

When shooting a moving image, the user may concentrate on shooting for a relatively long time so as not to miss a decisive moment, or may set the setting to acquire high-quality and high-quality moving images. The moving image taken in this way has a quality (image quality or sound quality) suitable for recording. On the other hand, the moving image recorded by shooting is often a moving image whose highlights are difficult to understand because it is redundant. For this reason, such moving images are often unsuitable for viewing without editing. In addition, the data capacity of such moving images tends to be large.

Under these circumstances, there is a strong need for simple and impressive videos, especially in video sharing via the Internet in recent years. Therefore, there is a demand for technology related to transmission / reception and editing of moving image data. For example, Patent Document 1 discloses a technique relating to a video signal processing device capable of continuously outputting a still image to a monitor. Further, for example, Patent Document 2 is obtained as a result of receiving and decoding a plurality of coded video frames having a video time stamp VTS and a plurality of coded audio frames having an audio time stamp ATS. A technique relating to a content receiving device capable of accumulating a plurality of video frames VF1 and a plurality of audio frames AF1 and lip-syncing while maintaining audio continuity is disclosed.

Japanese Unexamined Patent Publication No. 2-93492 Japanese Unexamined Patent Publication No. 2005-102192

An object of the present invention is to provide a portable terminal, a capture device, an information processing system, an information processing method, an information generation method, and an information processing program for video editing that can realize simple video editing.

According to one aspect of the present invention, in a mobile terminal for moving image editing, a first image including a plurality of temporally continuous images is time-divided, and each of them has a first time information and is independent. A plurality of image pieces generated as an image that can be reproduced in the above and a first sound corresponding to the first image are time-divided, and each of them has a second time information and is generated as a sound that can be reproduced independently. A terminal-side communication control unit that acquires a plurality of voice pieces by communication, and a first time range from the time range of the first image is specified, and the time range of the first voice is selected. The terminal-side operation unit for designating the second time range, the image piece included in the first time range is selected from the received image pieces, and the received image pieces are plurality. It is provided with a terminal-side editorial unit that selects the voice piece included in the second time range from the voice pieces.

According to one aspect of the present invention, in the capture device for moving image editing, a first image including a plurality of images consecutive in time and a first sound corresponding to the first image are recorded. The recording unit and the first image are time-divided, and a plurality of image pieces, each of which is an image that can be reproduced independently by attaching the first time information, are generated, and the first sound is time-divided. A division unit that generates a plurality of audio pieces, each of which can be reproduced independently by attaching a second time information, and a device-side communication control unit that transmits the image piece and the audio piece to the outside. Be prepared.

According to one aspect of the present invention, the information processing system for video editing includes the portable terminal and the capture device.

According to one aspect of the present invention, in the information processing method for moving image editing, a first image including a plurality of temporally continuous images is time-divided, and each has a first time information. A plurality of image pieces generated as an image that can be reproduced independently and a first sound corresponding to the first image are time-divided, and each has a second time information and can be reproduced independently. Acquiring a plurality of generated audio pieces by communication, designating a first time range from the time range of the first image according to a user operation, and receiving the plurality of received images. Select the image piece included in the first time range from the pieces, specify the second time range from the time range of the first sound according to the user operation, and receive. It includes selecting the voice piece included in the second time range from the plurality of voice pieces.

According to one aspect of the present invention, in the information generation method for moving image editing, a first image including a plurality of temporally continuous images and a first sound corresponding to the first image are recorded. In the capture device, the first image is time-divided to generate a plurality of image pieces that can be reproduced independently, and the first time information is attached to each of the plurality of image pieces. The first sound piece is time-divided to generate a plurality of sound pieces that can be reproduced independently, the second time information is attached to each of the plurality of sound pieces, and the image piece and the sound piece are attached. Includes sending and to a mobile terminal.

According to one aspect of the present invention, in the information processing program for moving image editing, a first image including a plurality of temporally continuous images is time-divided, and each has a first time information. A plurality of image pieces generated as an image that can be reproduced independently and a first sound corresponding to the first image are time-divided, and each has a second time information and can be reproduced independently. Acquiring a plurality of generated audio pieces by communication, designating a first time range from the time range of the first image according to a user operation, and receiving the plurality of received images. Select the image piece included in the first time range from the pieces, specify the second time range from the time range of the first sound according to the user operation, and receive. The computer is made to select the sound piece included in the second time range from the plurality of sound pieces.

According to the present invention, it is possible to provide a portable terminal, a capture device, an information processing system, an information processing method, an information generation method, and an information processing program for video editing that can realize simple video editing.

FIG. 1A is a block diagram showing an outline of a configuration example of an information processing system according to a first embodiment. FIG. 1B is a block diagram showing an outline of a configuration example of the capture device according to the first embodiment. FIG. 1C is a block diagram showing an outline of a configuration example of a mobile terminal according to the first embodiment. FIG. 2A is a diagram for explaining an outline of the operation of the information processing system according to the first embodiment. FIG. 2B is a diagram for explaining an outline of the operation of the information processing system according to the first embodiment. FIG. 2C is a diagram for explaining an outline of the operation of the information processing system according to the first embodiment. FIG. 2D is a diagram for explaining an outline of the operation of the information processing system according to the first embodiment. FIG. 2E is a diagram for explaining an outline of the operation of the information processing system according to the first embodiment. FIG. 3 is a flowchart showing an example of the capture device control process according to the first embodiment. FIG. 4 is a flowchart showing an example of image piece and audio piece transmission processing according to the first embodiment. FIG. 5 is a flowchart showing an example of the reproduction process with temporary editing according to the first embodiment. FIG. 6A is a flowchart showing an example of the mobile terminal control process according to the first embodiment. FIG. 6B is a flowchart showing an example of the mobile terminal control process according to the first embodiment. FIG. 7A is a schematic diagram showing an example of display in the mobile terminal according to the first embodiment. FIG. 7B is a schematic diagram showing an example of display in the mobile terminal and the capture device according to the first embodiment. FIG. 7C is a schematic diagram showing an example of display in the mobile terminal according to the first embodiment. FIG. 7D is a schematic diagram showing an example of display in the mobile terminal according to the first embodiment. FIG. 7E is a schematic diagram showing an example of display in the mobile terminal according to the first embodiment. FIG. 8 is a flowchart showing an example of the separate editing process according to the first embodiment. FIG. 9A is a schematic diagram showing an example of display in the mobile terminal according to the first embodiment. FIG. 9B is a schematic diagram showing an example of display in the mobile terminal according to the first embodiment. FIG. 9C is a schematic diagram for explaining the separation editing process according to the first embodiment. FIG. 10A is a schematic diagram showing an example of display on a mobile terminal according to a modified example of the separation editing process. FIG. 10B is a schematic diagram for explaining a modification of the separation editing process. FIG. 11A is a flowchart showing another example of the reproduction process with temporary editing. FIG. 11B is a flowchart showing another example of the reproduction process with temporary editing. FIG. 12A is a schematic diagram showing an example of a display in a capture device according to a modified example of the reproduction process with temporary editing. FIG. 12B is a schematic diagram showing an example of a display in a capture device according to a modified example of the reproduction process with temporary editing. FIG. 12C is a schematic diagram showing an example of a display in a capture device according to a modified example of the reproduction process with temporary editing. FIG. 12D is a schematic diagram showing an example of a display in a capture device according to a modified example of the reproduction process with temporary editing. FIG. 12E is a schematic diagram showing an example of display in a mobile terminal and a capture device according to a modified example of the reproduction process with temporary editing. FIG. 12F is a schematic diagram showing an example of display on a mobile terminal according to a modified example of the reproduction process with temporary editing.

<Structure>
The first embodiment of the present invention will be described with reference to the drawings. FIG. 1A shows an outline of a configuration example of the information processing system 1 according to the present embodiment as a block diagram. The information processing system 1 has a function as an information acquisition device for acquiring information such as moving images and an information processing device for processing the acquired information such as moving images. In the present embodiment, as shown in FIG. 1A, an information processing system 1 including an information acquisition device, a capture device 100 having a function as an information processing device, and a portable terminal 200 having a function as an information processing device is taken as an example. I will explain.

FIG. 1B shows an outline of a configuration example of the capture device 100 according to the present embodiment as a block diagram. Further, FIG. 1C shows an outline of a configuration example of the mobile terminal 200 according to the present embodiment as a block diagram. The capture device 100 is a digital camera capable of shooting high-quality moving images such as 4K and 8K. Further, the mobile terminal 200 according to the present embodiment is an information terminal such as a smartphone or a tablet PC. The user operates the mobile terminal 200, and based on the high-quality video acquired by the capture device 100, the data capacity is reduced, it is easy to share with other users, and the user can easily create a video that can be easily watched. do.

The capture device 100 includes a device-side control unit 110, a device-side display unit 120, a device-side audio reproduction unit 130, a device-side operation unit 140, a device-side recording unit 150, a device-side communication unit 160, and a device-side. It includes an image pickup unit 170, a device-side sound collection unit 180, and a device-side clock unit 190. The capture device 100 may further include an image processing circuit that performs various image processing on the image data.

The device-side control unit 110 controls the operation of each unit of the capture device 100. The device-side control unit 110 has a function as the device-side information processing unit 111. The information processing unit 111 on the device side has functions as a compression unit 112 on the device side, an extension unit 113 on the device side, a division unit 114 on the device side, a communication control unit 115 on the device side, and a recording control unit 116 on the device side. The device-side compression unit 112 performs compression processing for still image data, audio data, and moving image data. Here, the moving image data includes a first image including a plurality of temporally continuous images, and a first sound corresponding to the first image. Further, the device-side compression unit 112 performs image data resizing processing. The device-side decompression unit 113 performs decompression processing of compressed image data, audio data, and moving image data. The device-side dividing unit 114 time-divides the first image and the first sound included in the moving image data. The device-side dividing unit 114 images the first image (video frame) included in the moving image data at specific timings. The device-side dividing unit 114 cuts out the first sound included in the moving image data at specific timings. Hereinafter, the image data and the audio data generated in this manner will be referred to as an image piece and an audio piece (audio clip), respectively. The device-side dividing unit 114 attaches the first time information to each image piece. Here, the first time information is the time position of each image piece in the first image. Similarly, the device-side dividing unit 114 attaches a second time information to each voice piece. Here, the second time information is the time position of each voice piece in the first voice. The device-side communication control unit 115 controls operations such as communication of the device-side communication unit 160. The device-side communication control unit 115 causes the mobile terminal 200 to transmit the image piece, the sound piece, and the like generated by the capture device 100. The device-side recording control unit 116 controls the recording of various information in the device-side recording unit 150. The device-side recording control unit 116 records the moving image data (first image and first sound) acquired by the capture device 100 in the image recording unit 151 and the sound recording unit 152 of the device-side recording unit 150, respectively. .. The device-side recording control unit 116 temporarily records or records the image piece, the sound piece, and the like generated by the capture device 100 in the image recording unit 151 and the sound recording unit 152 of the device-side recording unit 150, respectively.

Further, the device-side control unit 110 further includes functions as a device-side display control unit and a device-side editing unit. The device-side display control unit generates display information to be displayed on the device-side display unit 120 included in the capture device 100. The operation of the device-side display control unit during video editing is the same as that of the terminal-side display control unit 218, which will be described later. The device-side editorial unit performs processing related to video editing. The device-side editorial unit performs a process related to temporary editing for setting a range of image pieces and audio pieces to be transmitted to the mobile terminal 200.

The device-side display unit 120 displays the display information generated by the device-side display control unit. The display information includes image data. The device-side display unit 120 is, for example, a liquid crystal display or an organic EL display. The device-side display unit 120 has a function as an electronic viewfinder (EVF). The device-side audio reproduction unit 130 reproduces the audio information generated by the device-side control unit 110. The device-side audio reproduction unit 130 is, for example, a speaker. The device-side operation unit 140 includes an operation member for acquiring various operation results for the user's capture device 100. The operation unit 140 on the device side may be any element as long as it can acquire user operations, and may be, for example, a mechanical operation member such as a push button, a knob, a slider, a dial, or a lever, or a touch panel.

Various information such as information acquired by the capture device 100 and information related to the operation of the capture device 100 are recorded in the device-side recording unit 150. Further, the program and parameters used in each unit included in the capture device 100 are recorded in the device-side recording unit 150. This program includes an information processing program for editing video data. The device-side recording unit 150 records display data necessary for displaying various editing screens on the device-side display unit 120 when editing a moving image. Further, various data including the processing data during the operation of the capture device 100 are temporarily recorded in the device-side recording unit 150. The device-side recording unit 150 includes an image recording unit 151 and an audio recording unit 152. The image recording unit 151 includes still image data acquired by the capture device 100, a first image including time-consecutive images included in the moving image data, an image piece generated by the capture device 100, and these image data. Various image data such as a thumbnail image indicating the above are recorded. The voice recording unit 152 records various voice data such as voice data acquired by the capture device 100, a first voice corresponding to the first image, and a voice piece generated by the capture device 100. The voice recording unit 152 may record voice data such as sound effects used for generating an ornamental moving image. Here, the ornamental moving image is a moving image edited so as to be suitable for later viewing by a user or the like.

The device-side communication unit 160 includes a circuit for the capture device 100 to communicate with another device. Communication between the capture device 100 and the mobile terminal 200 is performed, for example, by wireless communication. The device-side communication unit 160 includes a first device-side communication unit 161 and a second device-side communication unit 162. The first device-side communication unit 161 includes a communication circuit corresponding to Bluetooth (registered trademark) communication or Bluetooth Low Energy (BLE) communication. The second device-side communication unit 162 includes a communication circuit corresponding to Wi-Fi (registered trademark) communication. For example, for communication of image data, voice data, etc., a first device-side communication unit 161 capable of executing Wi-Fi communication, which is a relatively large-capacity communication, is used. Further, for communication of control signals and the like, a second device-side communication unit 162 capable of executing BLE communication, which is communication with relatively low power consumption, is used.

Image data having a small data capacity such as a thumbnail image may be transmitted / received by BLE communication. The device-side communication unit 160 may further include a communication circuit corresponding to mobile phone communication. Further, communication between the capture device 100 and the outside of the capture device 100 such as the mobile terminal 200 may be performed by wired communication, or may be connected to each other by a telecommunication line such as the Internet to make a telecommunication line such as the Internet. Communication may be performed via. Even if the data is moved between the capture device 100 and the mobile terminal 200 or the like via a recording medium outside the information processing system 1 such as a USB memory, a CD-ROM, or a data server on the network. good.

The device-side image pickup unit 170 takes an image of the target and acquires image data related to the subject. The device-side image pickup unit 170 includes a lens and an image pickup element. The lens forms an image of a luminous flux from an object on a light receiving surface of an image pickup device. The lens may include a zoom lens, a focus lens, and the like. The image pickup device is, for example, a Complementary Metal-Oxide Semiconductor (CMOS) sensor or a Charge-Coupled Device (CCD) sensor, and converts a light flux incident on a light receiving surface into an electric signal (image signal). The image sensor also performs preprocessing such as amplifying an image signal and digitizing it to generate image data.

The device-side sound collecting unit 180 picks up the target voice and acquires the voice data related to the target. The device-side sound collecting unit 180 is, for example, a microphone, and converts a sound wave incident on the device-side sound collecting unit 180 into an electric signal (voice signal). In addition, the device-side sound collecting unit 180 also performs preprocessing such as amplifying the audio signal and digitizing it to generate audio data, and removing noise from the audio signal. It should be noted that these preprocessings may be realized by hardware or software.

The device-side clock unit 190 generates information related to the date and time and outputs the information to the device-side control unit 110.

As shown in FIG. 1C, the mobile terminal 200 includes a terminal side control unit 210, a terminal side display unit 220, a terminal side audio reproduction unit 230, a terminal side operation unit 240, a terminal side recording unit 250, and a terminal side. It is provided with a communication unit 260.

The terminal-side control unit 210 controls the operation of each unit of the mobile terminal 200. The terminal-side control unit 210 has a function as a terminal-side information processing unit 211. The terminal-side information processing unit 211 has functions as a terminal-side compression unit 212, a terminal-side communication control unit 215, a terminal-side recording control unit 216, a terminal-side editing unit 217, and a terminal-side display control unit 218. The terminal-side compression unit 212 performs compression processing for image data, audio data, and moving image data. Here, the moving image data includes a time-continuous image piece included in a specific time range (first time range) and a time-continuous sound included in a specific time range (second time range). Includes video data for ornamental videos generated based on the piece. Further, the terminal-side compression unit 212 performs image data resizing processing. The terminal-side communication control unit 215 controls operations such as communication of the terminal-side communication unit 260. The terminal-side communication control unit 215 receives the image piece, the sound piece, and the like generated by the capture device 100 from the capture device 100. The terminal-side recording control unit 216 controls recording of various information in the terminal-side recording unit 250. The terminal-side recording control unit 216 temporarily records the image piece and the audio piece acquired from the capture device 100 in the terminal-side recording unit 250 for each attached time information. The terminal-side recording control unit 216 causes the terminal-side recording unit 250 to record the moving image data of the ornamental moving image generated by the mobile terminal 200. The terminal-side editorial unit 217 controls various processes related to the generation of the ornamental moving image. The terminal-side editing unit 217 has a time range of an image piece (first time range) and a time range of an audio piece (second time range) used for generating an ornamental moving image based on the output of the terminal-side operation unit 240. ) And set. The terminal side editing unit 217 separates the time range between the image (image piece) and the sound (audio piece) and the separate editing that sets the time range between the image (image piece) and the sound (audio piece) separately. Performs join editing and setting all at once. In the separate editing, the terminal-side editing unit 217 sets the first time range and the second time range individually. The terminal-side editorial unit 217 generates an ornamental moving image based on the image piece included in the first time range and the audio piece included in the second time range. The terminal-side editorial unit 217 performs a process of adding a caption to an image piece and a process of adding an inserted image such as a title image, a sound effect, or the like to an ornamental moving image. The terminal side display control unit 218 generates display information to be displayed on the terminal side display unit 220. The display information includes a display screen for video editing. Here, the display screen for video editing includes a display screen for separate editing and a display screen for combined editing. The display screen for separate editing includes an image range selection bar for setting a first time range and an audio range selection bar for setting a second time range. The display screen for combined editing includes a common range selection bar which is a range selection bar for both image and sound.

The terminal side display unit 220 displays the display information generated by the terminal side display control unit 218. The display information includes image data. The terminal side display unit 220 is, for example, a liquid crystal display or an organic EL display.

The terminal-side audio reproduction unit 230 reproduces the audio information generated by the terminal-side control unit 210. The terminal-side audio reproduction unit 230 is, for example, a speaker.

The terminal-side operation unit 240 includes an operation member for acquiring various operation results for the user's mobile terminal 200. The terminal-side operation unit 240 may be an element that can acquire user operations, and may be, for example, a mechanical operation member such as a push button, a knob, a slider, a dial, or a lever, or may be a touch panel. The terminal-side operation unit 240 according to the present embodiment includes at least a touch panel provided on the terminal-side display unit 220.

Various information such as information acquired by the mobile terminal 200 and information related to the operation of the mobile terminal 200 are recorded in the terminal-side recording unit 250. Further, the terminal side recording unit 250 records programs and parameters used in each unit included in the mobile terminal 200. This program includes an information processing program for editing video data. The terminal-side recording unit 250 records display data necessary for displaying various editing screens on the terminal-side display unit 220 when editing a moving image. The terminal-side recording unit 250 includes a temporary recording unit 251 and an information recording unit 252. Various data including the processing data during the operation of the mobile terminal 200 are temporarily recorded in the temporary recording unit 251. Image pieces and audio pieces received from the capture device 100 are temporarily recorded in the temporary recording unit 251 for each attached time information. Various information such as an ornamental moving image generated by the mobile terminal 200 is recorded in the information recording unit 252. The information recording unit 252 may record an inserted image, a sound effect, or the like used for generating an ornamental moving image.

The terminal-side communication unit 260 includes a circuit for the mobile terminal 200 to communicate with other devices. The terminal-side communication unit 260 communicates with the capture device 100. The terminal-side communication unit 260 includes a first terminal-side communication unit 261 and a second terminal-side communication unit 262. The first terminal-side communication unit 261 includes a communication circuit corresponding to Bluetooth communication or BLE communication. The second terminal-side communication unit 262 includes a communication circuit corresponding to Wi-Fi communication. The terminal-side communication unit 260 may further include a communication circuit corresponding to mobile phone communication.

The device-side control unit 110 and the terminal-side control unit 210 include integrated circuits such as a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA). The device-side control unit 110 and the terminal-side control unit 210 may each be configured by one integrated circuit or the like, or may be configured by combining a plurality of integrated circuits or the like. The operation of these integrated circuits is performed according to a program recorded in, for example, the recording unit 150 on the device side or the recording unit 250 on the terminal side or the recording area in the integrated circuit, but is realized by hardware depending on the configuration of the integrated circuit. May be good.

The device-side recording unit 150 and the terminal-side recording unit 250 are non-volatile memories such as a flash memory, a hard disk drive (HDD), and a solid state drive (SSD). The device-side recording unit 150 and the terminal-side recording unit 250 may further have a volatile memory such as a Static Random Access Memory (SRAM) or a Dynamic Random Access Memory (DRAM). The device-side recording unit 150 and the terminal-side recording unit 250 may each be configured with one memory or the like, or may be configured by combining a plurality of memories or the like. Further, the device-side recording unit 150 and the terminal-side recording unit 250 may use an external data server or the like as a part of the memory thereof.

<Outline of operation>
Here, an outline of the operation of the information processing system 1 according to the present embodiment will be described with reference to FIGS. 2A to 2E.

FIG. 2A shows a scene in which a user U is shooting a moving image using a digital camera (capture device 100) in a wedding candle service scene. In such a case, the user U may concentrate on shooting a video for a relatively long time so as not to miss a decisive moment, or may be set to acquire a high-quality and high-quality video. do. The moving image taken in this way has a quality (image quality or sound quality) suitable for recording. On the other hand, the moving image recorded by shooting is often a moving image whose highlights are difficult to understand because it is redundant. For this reason, such moving images are often unsuitable for viewing without editing. In particular, in recent years, in video sharing via telecommunication lines such as the Internet, there is a strong need for simple and impressive videos.

As shown in FIG. 2B, the user U operates the smartphone (mobile terminal 200) at hand after shooting the moving image, for example, in the train on the way back, and the moving image recorded in the capture device 100 in the bag (original). Check the video). However, the original moving image is a moving image shot with high image quality or high sound quality over a relatively long period of time, and tends to have a large data capacity. It is difficult for such a moving image to be transmitted to the mobile terminal 200 in the hands of the user U by wireless communication. In addition, video files with a large data capacity are often unsuitable for video sharing via the Internet or the like.

Therefore, the capture device 100 according to the present embodiment stretches the original moving image and generates a first image (moving image frame) as one image (image piece) at each specific timing. In addition, time information (time stamp) is attached to each image piece. Hereinafter, the time information attached to the image piece will be referred to as the first time information. In FIG. 2C, a first image piece F1, a second image piece F2, a third image piece F3, and a fourth image piece F10 are shown as a series of image pieces F10 including a plurality of image pieces that are continuous in time. The case where the image piece F4 and the fifth image piece F5 are generated is shown as an example. Similarly, the capture device 100 expands the original moving image, separates the first audio at specific timings, and generates an audio piece (audio clip). In addition, time information (time stamp) is attached to each voice piece. Hereinafter, the time information attached to the voice piece will be described as the second time information. In FIG. 2C, as a series of audio pieces S10 including audio pieces corresponding to each image piece, a first audio piece S1, a second audio piece S2, a third audio piece S3, and a fourth sound piece S3 are shown. The case where the voice piece S4 and the fifth voice piece S5 are generated is shown as an example.

As described above, the capture device 100 according to the present embodiment generates an image piece as an image that can be reproduced independently by time-dividing the first image and attaching the first time information. The "image that can be reproduced independently" here is an image that can be reproduced without referring to another image, such as an image compressed between frames (time compression). Further, the capture device 100 divides the time of the first sound corresponding to the first image, attaches the second time information, and generates a sound piece as a sound that can be reproduced independently. The "sound that can be played independently" here is a sound that can be played without referring to other sounds, such as a time-compressed sound such as by taking a difference between adjacent samples. be. Here, the first time information indicates the time position of the image piece in the original moving image (first image). Similarly, the second time information indicates the time position of the audio piece in the original moving image (first audio).

The mobile terminal 200 according to the present embodiment receives the generated series of image pieces F10 and a series of audio pieces S10 from the capture device 100, and temporarily records each time information attached to each image piece or audio piece. do. The user U operates a smartphone (mobile terminal 200) at hand, and based on a series of image pieces F10 and a series of audio pieces S10 temporarily recorded for each time information, a moving image suitable for sharing (ornamental moving image). To create.

At this time, as shown in FIG. 2D, the user U selects a range (first time range) of the image piece used for creating the ornamental moving image as the material image F20 from the series of image pieces F10. Here, the material image F20 is a temporally continuous image piece within the first time range. A range of a series of image pieces F10 including an image piece representative of the scene (original moving image) may be selected as the first time range. For example, in the case of a candle service scene, the material image F20 includes a fourth image piece F4 indicating immediately before the candle is lit, and a fifth image piece F5 indicating immediately after the candle is lit. The material image time Tm indicating the time width of the first time range may be preset to a value such as 5 seconds, or may be appropriately adjusted and set by the user U.

Similarly, as shown in FIG. 2E, the user U selects a range (second time range) of the audio piece used for creating the ornamental moving image as the material audio S20 from the series of audio pieces S10. Here, the material audio S20 is a temporally continuous audio piece within the second time range. For example, if a range of a series of audio pieces S10 including audio pieces representing the scene (original moving image) such as when the BGM is excited or when everyone applauds is selected as the second time range. good. For example, in the case of a candle service scene, the material voice S20 includes a fourth voice piece S4 and a fifth voice piece S5 as voices before and after the candle is lit. The material audio time Ts indicating the time width of the second time range may be set according to the first time range (material image time Tm), or may be appropriately adjusted and set by the user U. ..

In this way, the user U can individually select the range to be used as the material image F20 and the material sound S20 for each of the image piece and the sound piece derived from the original moving image. At this time, the first time range and the second time range may be the same or different. Therefore, the information processing system 1 according to the present embodiment can generate a spectacular ornamental moving image in which the highlight of the image and the highlight of the sound recorded at different time positions in the original moving image are used.

Further, the user U creates an ornamental moving image by appropriately combining the generated material image F20 and the material sound S20. At this time, in addition to the material image F20 and the material sound S20, a title image or a sound effect may be further used, or an image piece included in the material image F20 may be subjected to image processing. The image processing for the image piece may include a process of adding a caption or the like to the image piece, a process of processing the image piece into a monotone or the like, and the like. Further, the material image F20 and the material sound S20 may be combined so that either one of them is repeatedly reproduced according to the material image time Tm and the material sound time Ts.

The ornamental moving image may be generated by the mobile terminal 200 such as a smartphone having abundant user interfaces (UI), but may be generated by the capture device 100. In this case, for example, the capture device 100 generates the material image F20 and the material sound S20 according to the user operation. The mobile terminal 200 generates an ornamental moving image based on the material image F20 and the material sound S20 received from the capture device 100 according to the user operation.

<Operation>
Here, an example of the capture device control process performed by the capture device 100 is shown in FIG. 3 as a flowchart, and the operation of the information processing system 1 according to the present embodiment will be described with reference to this. The following processing is started in a state where the capture device 100 is powered off and is waiting for communication for the first communication, which is low power consumption communication.

In step S101, the device-side control unit 110 determines whether or not to turn on the power of the capture device 100. In this determination, for example, when the power switch is turned on or the first communication start request is made, it is determined that the power is turned on. The process proceeds to step S102 if it is determined to turn on the power, ends the observation device control process if it is not determined, and waits for communication for the first communication. If it is not determined that the power is turned on, the process may proceed to step S112.

In step S102, the device-side control unit 110 turns on the power of the capture device 100 and determines whether or not the current operation mode of the capture device 100 is the shooting mode. The process proceeds to step S103 if it is determined to be in the shooting mode, and proceeds to step S109 if it is not determined.

In step S103, the device-side control unit 110 starts the live view display and the sound collection. In the live view display, the device-side control unit 110 causes the device-side image pickup unit 170 to start imaging, performs image processing for live view display on the image obtained by imaging, and uses the image after image processing as the device. It is displayed on the side display unit 120. The device-side control unit 110 causes the device-side sound collection unit 180 to start collecting sound. In step S104, the device-side control unit 110 determines whether or not the operation related to the photographing parameter has been performed based on the output of the device-side operation unit 140. The process proceeds to step S105 if it is determined that the shooting parameter operation has been performed, and proceeds to step S106 if it is not determined. In step S105, the device-side control unit 110 sets the values of various shooting parameters based on the output of the device-side operation unit 140 according to the input shooting parameters.

In step S106, the device-side control unit 110 determines whether or not the shooting or voice recording start operation has been performed based on the output of the device-side operation unit 140. The process proceeds to step S107 if it is determined that the start operation has been performed, and returns to step S101 if it is not determined. The start operation may be a start operation for shooting to acquire a still image, a start operation for recording (voice recording) to acquire audio data, or a shooting (shooting and shooting) to acquire a moving image. It may be the start operation of (recording). In step S107, the device-side control unit 110 starts shooting or voice recording (moving image shooting). In step S108, the device-side control unit 110 compresses the acquired image, voice, or image and voice (moving image) into a file, and records it in the device-side recording unit 150. The moving image recorded here is, for example, a moving image having an image quality or sound quality suitable for recording. The processing of steps S107 and S108 is continued until, for example, the end operation of shooting or voice recording (moving image shooting) is performed. After that, the process returns to step S101.

In step S109, the device-side control unit 110 determines whether or not to perform communication settings based on the output of the device-side operation unit 140. The process proceeds to step S110 if it is determined that the communication setting is to be performed, and proceeds to step S111 if it is not determined. In step S110, the device-side control unit 110 turns on the power of the capture device 100 and makes various settings (communication settings) related to communication. In the communication setting, information related to the communication partner of the capture device 100 such as the mobile terminal 200 is registered.

In step S111, the device-side control unit 110 determines whether or not the reproduction mode is selected based on the output of the device-side operation unit 140. The process proceeds to step S117 if it is determined that the reproduction mode is selected, and proceeds to step S112 if it is not determined.

In step S112, the device-side control unit 110 determines whether or not there is a communication request from the mobile terminal 200. The process proceeds to step S113 if it is determined that there is a communication request, and returns to step S101 if it is not determined. In step S113, the device-side control unit 110 causes the second device-side communication unit 162 to establish the second communication with the mobile terminal 200. The device-side control unit 110 reads out a representative image showing each of the still image, the sound, and the moving image recorded in the device-side recording unit 150 from the image recording unit 151, and transmits the representative image to the mobile terminal 200 that made the communication request. Here, the representative image includes an icon and a thumbnail image.

In step S114, the device-side control unit 110 determines whether or not a moving image (moving image file) has been selected based on the control signal received from the mobile terminal 200. The process proceeds to step S115 if it is determined that the moving image file is selected, and proceeds to step S116 if it is not determined. In step S115, the device-side control unit 110 performs image piece and audio piece transmission processing. The image piece and sound piece transmission process is a process of generating an image piece and a sound piece as described above. Details of the image piece and audio piece transmission processing will be described later. The process proceeds to step S116 after the image piece and sound piece transmission processing is completed. In step S116, the device-side control unit 110 determines whether or not to terminate the second communication established with the mobile terminal 200. The determination may be made based on the user operation, may be made based on whether or not the image piece and voice piece transmission processing is completed, and the time during which communication is not performed is set to a predetermined time. It may be done based on whether or not it has been reached. If it is determined that the process is terminated, the second communication is terminated and the process returns to step S112. If the process is not determined, the process returns to step S114.

In step S117, the device-side control unit 110 executes a reproduction process with temporary editing. The reproduction process with temporary editing is a process of reproducing a still image, sound, and moving image recorded in the recording unit 150 on the device side. Further, in the reproduction process with temporary editing, a temporary editing image (temporary editing frame) and a temporary editing sound can be generated based on the moving image file. Here, the temporary editing image and the temporary editing sound are temporally continuous image pieces and sound pieces included in an arbitrary time range (temporary editing time range). The details of the playback process with temporary editing will be described later. The process returns to step S111 after the reproduction process with temporary editing is completed.

Here, an example of the image piece and audio piece transmission processing performed in step S115 of the capture device control process is shown in FIG. 4 as a flowchart, and the operation of the information processing system 1 will be described with reference to this.

In step S201, the device-side control unit 110 selects the original moving image for generating the image piece and the sound piece according to the result of the user operation. The selection of the original moving image is based on the control signal received from the mobile terminal 200 in step S114 of the capture device control process. After that, the device-side control unit 110 starts expanding the selected moving image (moving image data).

In step S202, the device-side control unit 110 expands the moving image data and images (image fragmentation) the first image (moving image frame) at each specific timing. The specific timing for image fragmentation is, for example, 33 ms, but is not limited to this. It may be determined according to the original moving image file, it may be determined according to the timing of voice fragmentation, or it may be determined according to the communication capacity. In step S203, the device-side control unit 110 associates time information (first time information) with the image piece. The device-side control unit 110 causes the mobile terminal 200 to transmit the generated image piece.

In step S204, the device-side control unit 110 expands the moving image data and unifies the first sound (voice clip) at each specific timing. The specific timing for voice fragmentation is, for example, 40 ms, but is not limited to this. It may be determined according to the original moving image file, it may be determined according to the timing of image fragmentation, or it may be determined according to the communication capacity. In step S205, the device-side control unit 110 associates time information (second time information) with the voice piece. The device-side control unit 110 causes the mobile terminal 200 to transmit the generated voice piece.

In step S206, the device-side control unit 110 determines whether or not to end the image piece and sound piece transmission processing. In this determination, for example, it is determined that the image piece and audio piece transmission processing is completed when the generation and transmission of the image piece and the audio piece are completed for the selected moving image (original moving image). The process repeats the processes of steps S202 to S205 until it is determined to be completed, and when it is determined to be completed, the image piece and audio piece transmission processes are terminated, and the process proceeds to step S116 of the capture device control process.

Here, an example of the reproduction process with temporary editing performed in step S117 of the capture device control process is shown in FIG. 5 as a flowchart, and the operation of the information processing system 1 will be described with reference to this.

In step S301, the device-side control unit 110 causes the device-side display unit 120 to display a list of reproducible contents recorded in the device-side recording unit 150. Here, the playable content may be a moving image file, an image file, or an audio file.

In step S302, the device-side control unit 110 determines whether or not there has been an operation of selecting a representative image (thumbnail image). In this determination, for example, when the touch panel is arranged on the device side display unit 120, it is determined whether or not the representative image is touched. The process proceeds to step S304 if it is determined that the representative image has been selected, and proceeds to step S303 if it is not determined. In step S303, the device-side control unit 110 determines whether or not to end the reproduction process with temporary editing (reproduction mode) and return to the mode selection. This determination is based on, for example, the result of a user operation. If it is determined to return, the process ends the reproduction process with temporary editing and returns to step S111 of the capture device control process, and if not determined, returns to step S301. In step S304, the device-side control unit 110 reproduces the content selected in step S302.

In the determination of step S302, it may be determined that playable content other than the moving image file is selected. In this case, for example, after the reproduction of the content is started in step S304, it is continuously determined not to perform temporary editing in step S305, and the reproduction process related to the content is determined until it is determined that the reproduction is completed in step S312. Should be done. Hereinafter, the description will be continued by taking as an example the case where the moving image file is selected in step S302 and the reproduction of the moving image file is started in step S304.

In step S305, the device-side control unit 110 determines whether or not to perform temporary editing on the video being played. The temporary editing process according to the present embodiment is a process of generating a temporary editing image (temporary editing frame) and a temporary editing sound and transmitting them to the mobile terminal 200. Here, the tentatively edited image according to the present embodiment is an image piece included in a time range (temporary editing time range) selected by the user. Similarly, the tentatively edited voice according to the present embodiment is a voice piece included in the tentatively edited time range. In this determination, it is determined that temporary editing is performed when the user instructs temporary editing, for example, when the user touches the operation unit 140 (touch display) on the device side during video reproduction. The start position of the temporary editing may be set before the timing instructed by the user, such as by temporarily recording the moving image data expanded with the reproduction. Further, the range in which the temporary editing is performed may be from the timing when the user instructs the start to the timing when the user instructs the end to end, or from the timing when the user instructs the start to the elapse of a predetermined time. You may. The predetermined time may be set to a value such as 5 seconds in advance, or may be appropriately input and set by the user. The process proceeds to step S306 if it is determined to perform temporary editing, and proceeds to step S312 if it is not determined.

In step S306, the device-side control unit 110 generates an image piece from the original moving image (first image) stretched for reproduction in the same manner as in step S202. In step S307, the device-side control unit 110 attaches the first time information (time stamp) to the image piece in the same manner as in step S203. In step S308, the device-side control unit 110 generates an audio piece from the original moving image (first audio) in the same manner as in step S204. In step S309, the device-side control unit 110 attaches a second time information (time stamp) to the voice piece in the same manner as in step S205. In step S310, the device-side control unit 110 determines whether or not to end the temporary editing. This determination may be performed based on a user operation, or may be performed based on whether or not image fragmentation and audio fragmentation have been completed for a predetermined range set as a temporary editing time range. If it is determined that the process is completed, the process proceeds to step S311. If the process is not determined, the process returns to step S306.

In step S311, the device side control unit 110 generates a temporary editing image (temporary editing frame) based on the image piece generated by the temporary editing, and generates the temporary editing sound based on the audio piece generated by the temporary editing. .. The device-side control unit 110 causes the mobile terminal 200 to transmit the generated temporary editing image and the temporary editing sound in units of the image piece and the sound piece.

In step S312, the device-side control unit 110 determines whether or not to end the reproduction of the moving image (selected reproducible content). This determination is determined to end the reproduction when the reproduction time of the content ends, when there is an operation instructing the user to end the reproduction, or the like. If it is determined that the reproduction is finished, the process ends the reproduction process with temporary editing and returns to step S111 of the capture device control process, and if not determined, returns to step S305.

Here, FIGS. 6A and 6B show an example of the mobile terminal control process performed by the mobile terminal 200 as a flowchart, and the operation of the information processing system 1 will be described in more detail with reference to this.

In step S401, the terminal-side control unit 210 generates display information for displaying icons indicating various applications (various functions that can be executed) installed in the mobile terminal 200, and displays the generated display information on the terminal side. Displayed on the unit 220.

An example of the display at this time is shown in FIG. 7A. In the display example shown in FIG. 7A, as icons, a video editing icon 801 indicating a video editing application, a mail icon 802 indicating a mail application, and a telephone icon 803 indicating a telephone application are displayed. Further, in the display example shown in FIG. 7A, another icon 804 indicating an application other than the video editing application, the mail application, and the telephone application is further displayed. As described above, when another application is installed on the mobile terminal 200, the displayed icon can be appropriately changed depending on the installed application. The other icon 804 may be an icon that further displays an icon indicating another application.

In step S402, the terminal-side control unit 210 determines whether or not the video editing application has been selected. This determination is made based on the result of the user operation output by the terminal side operation unit 240. The user operation is, for example, an operation in which the user touches the position where the video editing icon 801 is displayed. In this determination, it is determined that the video editing application is selected when the video editing icon 801 is selected by the user. Further, as shown in FIG. 7A, an icon indicating the selected application may be highlighted. The process proceeds to step S404 if it is determined that the moving image acquisition mode is selected, and proceeds to step S403 if it is not determined.

In step S403, the terminal-side control unit 210 performs processing (other processing) related to the application indicated by the icon selected by the user after step S401. For example, when the mail icon 802 is selected, the terminal-side control unit 210 activates the mail application and performs processing related to the mail application. For example, when the telephone icon 803 is selected, the terminal-side control unit 210 activates the telephone application and performs processing related to the telephone application. The process returns to step S401 after the other processes are completed.

In step S404, the terminal-side control unit 210 activates the video editing application. The terminal-side control unit 210 generates display information for displaying icons indicating various functions in the video editing application, and causes the terminal-side display unit 220 to display the generated display information.

An example of the display at this time is shown in FIG. 7B. In the display example shown in FIG. 7B, a moving image acquisition icon 805a, a temporary edit data reception icon 805b, an image or audio data acquisition icon 805c, and a communication setting icon 805d are displayed as icons. Here, the moving image acquisition icon 805a is an icon indicating a function of acquiring moving image data from the capture device 100. The temporary editing data reception icon 805b is an icon indicating a function of acquiring temporary editing data (temporary editing image and temporary editing sound) from the capture device 100. The image or audio data acquisition icon 805c is an icon indicating a function of acquiring image or audio data from the capture device 100. The communication setting icon 805d is an icon indicating a function of registering information of the capture device 100 as a communication destination and making various settings related to communication with the capture device 100. Further, in the display example shown in FIG. 7B, the back button 806 is further displayed. The back button 806 is an icon indicating a function of returning to the previous display screen. For example, the back button 806 in the display example shown in FIG. 7B is an icon indicating a function of returning the display screen to the state of the display example shown in FIG. 7A.

In steps S401 and S404, the case where various icons are displayed in a list has been described as an example, but the present invention is not limited to this. The various icons may be displayed in sequence according to the swipe operation of the user or the like.

In step S405, the terminal-side control unit 210 determines whether or not the moving image acquisition icon 805a is selected. This determination is made based on the result of the user operation output by the terminal side operation unit 240. The user operation is, for example, an operation in which the user touches the position where the moving image acquisition icon 805a is displayed. Further, as shown in FIG. 7B, an icon indicating a function in the selected video editing application may be highlighted. The process proceeds to step S406 if it is determined that the moving image acquisition icon 805a is selected, and proceeds to step S418 if it is not determined.

In step S406, the terminal-side control unit 210 makes a communication request to the capture device 100 using the first communication. After that, the terminal-side control unit 210 establishes a second communication with the capture device 100. After the second communication is established, the terminal side control unit 210 acquires a thumbnail image showing each moving image data recorded in the capture device 100. The terminal side control unit 210 causes the terminal side display unit 220 to display the acquired thumbnail image.

An example of the display at this time is shown in FIG. 7C. In the display example shown in FIG. 7C, a first thumbnail image 807 showing the first moving image, a second thumbnail image 808 showing the second moving image, and a third thumbnail image 809 showing the third moving image are used. A fourth thumbnail image 810 showing the fourth moving image is displayed. Further, in the display example shown in FIG. 7C, the forward / return button 811, the selection button 812, and the back button 806 are further displayed. The forward / backward button 811 is a button selected by the user when switching the moving image data to be edited. For example, FIG. 7C shows a display example in which the fourth thumbnail image 810 is temporarily selected as a result of the operation of the forward / backward button 811 by the user. As shown in FIG. 7C, even if the tentatively selected thumbnail image is distinguished from other thumbnail images so that the user can know that the tentatively selected thumbnail image is tentatively selected, for example, by highlighting surrounded by a thick frame line. good. The selection button 812 is a button for selecting the moving image data tentatively selected by the operation of the forward / backward button 811 as the moving image data to be edited by the user. The back button 806 is an icon indicating a function of returning to the previous display screen. For example, in the display example shown in FIG. 7C, the icon indicates a function of returning the display screen to the state of the display example shown in FIG. 7B.

The destination of the communication request is, for example, a capture device 100 that is preset as a communication partner in the communication settings described later, but is not limited to this. For example, in this step, the capture device 100 may be searched for in the first communication, or the capture device 100 may be registered as a communication partner.

Although it has been explained that the thumbnail image is acquired from the capture device 100 by the second communication, the present invention is not limited to this. The thumbnail image may be acquired by the first communication.

In step S407, the terminal-side control unit 210 determines whether or not any of the moving image data has been selected. This determination is made based on the result of the user operation output by the terminal side operation unit 240. The user operation is, for example, an operation in which the user touches the selection button 812 while any thumbnail image is temporarily selected. The temporary selection of the thumbnail image by the user may be performed by touching the position where the thumbnail image that the user wants to edit is displayed, or the user operates the forward / backward button 811 to temporarily select the thumbnail images in sequence. It may be done while selecting. In this determination, when it is determined that the user has selected any thumbnail image, it is determined that any moving image data has been selected. The process proceeds to step S409 if it is determined that any of the moving image data has been selected, and proceeds to step S408 if it is not determined. In step S408, the terminal-side control unit 210 determines whether or not the return button 806 has been selected. The process returns to step S407 if it is not determined to be selected, and returns to step S404 if it is determined. Therefore, when it is determined that the return button 806 is selected in this step, the display screen returns from the display example shown in FIG. 7C to the display example shown in FIG. 7B.

In step S409, the terminal-side control unit 210 sequentially selects the image piece and the sound piece generated based on the moving image data determined in step S407 from the capture device 100 in units of the image piece and the sound piece. Receive. The image piece and the sound piece are received via the second communication. When the thumbnail image is received by the first communication in step S406, the second communication may be established prior to the processing of this step.

In step S410, the terminal-side control unit 210 temporarily records the received image piece and audio piece in the temporary recording unit 251 for each time information (time stamp) attached to each. FIG. 7D shows an example of the display after the image piece and the sound piece are received and provisionally recorded in this way. In the display example shown in FIG. 7D, the preview screen 813, the forward / backward button 811, the play button 814, the edit button 815, and the back button 806 are displayed. The user can play back and confirm the received moving image data (image piece and sound piece) on the preview screen 813 by operating the play button 814. The user can also operate the back button 806 to return to the display screen shown in FIG. 7C and select another moving image data. The user can also operate the forward / backward button 811 to reselect another moving image data. Further, when the user wants to edit the moving image data displayed on the preview screen 813, the user operates the edit button 815. The process proceeds to step S411 after the user operates the edit button 815.

In step S411, the terminal-side control unit 210 generates display information for combined editing and causes the terminal-side display unit 220 to display the display information. An example of the display at this time is shown in FIG. 7E. In the display example shown in FIG. 7E, the editing preview screen 817, the common time width 818, the common preview button 819, the common range selection bar 820, the separate editing button 825, the back button 826, and the enter button 827 are provided. It is displayed.

The editing preview screen 817 is a display screen for displaying a preview for reproducing and confirming the material moving image (material image) being edited. The common time width 818 displays the time width of the time range (common time range) indicated by the current common range selection bar, and in the display example shown in FIG. 7E, the time width of the common time range is 1 minute 30 seconds. It shows that. The common preview button 819 is a button for instructing the start of preview reproduction for reproducing and confirming the material moving image being edited.

The common range selection bar 820 is a slide bar for setting a time range (common time range) of an image piece and an audio piece used for generating an ornamental moving image in combined editing. The common range selection bar 820 includes a common time scale 821, a common start point 822, a common end point 823, and a common position indicator 824. In the common time scale 821, the left end indicates the start position of the original moving image, and the right end indicates the end position of the original moving image. The common start point 822 indicates the start position (start time) of the common time range. The common end point 823 indicates the end position (end time) of the common time range. The common position indicator 824 indicates the time position of the image piece currently displayed on the editing preview screen 817 with respect to the original moving image (received image piece and audio piece). The user moves the common start point 822 and the common end point 823 respectively on the common time scale 821, and selects the range between the common start point 822 and the common end point 823 as a range to be cut out as a material moving image. Further, the user can move the common position indicator 824 on the common time scale 821 to check the image piece (audio piece) at each position (time) on the common time scale 821. Further, when the user selects the common preview button 819, the video data is previewed (reproduced) from the position (time) of the common position indicator 824.

The separate edit button 825 is a button for instructing the start of separate edit. The details of separate editing will be described later. The back button 826 is a button for returning the display screen to the state of the display example shown in FIG. 7D. The decision button 827 is a button for determining the range indicated by the common range selection bar 820 as the range of the material moving image. When the enter button 827 is selected, the material image and the material sound are generated based on the image piece and the sound piece included in the range (common time range) indicated by the common range selection bar 820.

In step S412, the terminal-side control unit 210 determines whether or not separate editing is selected. This determination is made based on the result of the user operation output by the terminal side operation unit 240. The user operation is, for example, an operation in which the user touches the separate edit button 825. The process proceeds to step S413 if it is determined that the separate edit is selected, and proceeds to step S416 if it is not determined.

In step S413, the terminal-side control unit 210 performs a separate editing process. Here, an example of the separate editing process according to the present embodiment is shown in FIG. 8 as a flowchart.

In step S501, the terminal-side control unit 210 has a range corresponding to each of the image piece and the sound piece in order to allow the user to individually select each time range (first time range and second time range). Display each selection bar (separate display).

An example of the display at this time is shown in FIG. 9A. As shown in FIG. 9A, in the display in the separate editing process, the image range selection bar 820a and the audio range selection bar 820b are displayed instead of the common range selection bar 820. Here, the image range selection bar 820a is a range selection bar (image corresponding bar) for selecting a time range (first time range) of an image piece used as a material image. The voice range selection bar 820b is a range selection bar (image corresponding bar) for selecting a time range (second time range) of a voice piece used as a material voice. The common range selection bar 820 described above can also be expressed as an image-compatible bar that also serves as audio. That is, the common range selection bar 820 can also be expressed as a range selection bar for selecting a time range (common time range) in a state where the first time range and the second time range always match.

Similarly, the common time scale 821 is separately displayed on the image time scale 821a and the audio time scale 821b. The common start point 822 is separately displayed as an image start point 822a and an audio start point 822b. The common end point 823 is separately displayed at the image end point 823a and the audio end point 823b. The common position indicator 824 is separately displayed on the image position indicator 824a and the audio position indicator 824b. Further, the common time width 818 and the common preview button 819 are separately displayed as an image time width 818a and an audio time width 818b, and an image preview button 819a and an audio preview button 819b, respectively.

Further, next to the image time width 818a and the audio time width 818b, an image range selection button 828 and an audio range selection button are used to specify which range is to be selected, the image piece or the audio piece, respectively. 829 is provided. For example, when selecting a range (first time range) of an image piece, the user operates the image range selection bar 820a after selecting the image range selection button 828. When selecting the range of the image piece (first time range), the image range selection bar 820a may be directly operated without selecting the image range selection button 828. In the display example of FIG. 9A, the case where the image range selection button 828 is selected and highlighted is shown. The image range selection button 828 and the sound range selection button 829 may be icons for displaying which range selection of the image and the sound is being performed. In the display example of FIG. 9A, the back button 826 and the enter button 827 are further displayed. The return button 826 in the display example of FIG. 9A is a button for returning the display screen to the state of the display example shown in FIG. 7E. The decision button 827 is a button for determining the time range indicated by the current image range selection bar 820a and the audio range selection bar 820b as the first time range and the second time range, respectively.

In step S502, the terminal-side control unit 210 determines whether or not the operation related to the image range selection has been performed. In this determination, it can be determined that the operation related to the image range selection has been performed when the image range selection button 828 is operated or the image range selection bar 820a is operated. The process proceeds to step S503 if it is determined that the operation related to the image range selection has been performed, and proceeds to step S504 if it is not determined. In step S503, the terminal-side control unit 210 acquires the result of the user operation and displays the image start point 822a and the image end point 823a at the positions corresponding to the user operation. In addition, the changed image time width 818a is calculated and the display is updated. After that, the process proceeds to step S504.

Here, FIG. 9B shows an example of the display when the operation related to the image range is performed in step S503. The display example shown in FIG. 9B shows a display state when the positions of the image start point 822a and the image end point 823a of the image range selection bar 820a are changed from the display state shown in FIG. 9A. At this time, the updated image time width 818aa is displayed based on the positions of the changed image start point 822a and the image end point 823a. Hereinafter, the description will be continued by taking the case where the updated image time width 818aa is 30 seconds as shown in the display example of FIG. 9B as an example.

In step S504, the terminal-side control unit 210 determines whether or not the operation related to the voice range selection has been performed. In this determination, it can be determined that the operation related to the voice range has been performed when the voice range selection button 829 is operated or the voice range selection bar 820b is operated. The process proceeds to step S505 if it is determined that the operation related to the voice range selection has been performed, and proceeds to step S506 if it is not determined. In step S505, the terminal-side control unit 210 acquires the result of the user operation and displays the voice start point 822b and the voice end point 823b at the positions corresponding to the user operation. In addition, the changed voice time width 818b is calculated and the display is updated. After that, the process proceeds to step S506.

In step S506, the terminal-side control unit 210 determines whether or not there has been an operation such as confirmation / reproduction, feed, or return. In this determination, it is determined that the confirmation reproduction operation has been performed when it is determined that the image preview button 819a or the audio preview button 819b has been selected. In this determination, when it is determined that the image position indicator 824a or the audio position indicator 824b has been operated, it is determined that the feed or return operation has been performed. When the position of the image position indicator 824a or the audio position indicator 824b is moved toward the left end, it is determined that the return operation has been performed. Further, when the position of the image position indicator 824a or the audio position indicator 824b is moved toward the right end, it is determined that the feed operation has been performed. The process proceeds to step S507 if it is determined that there has been an operation such as confirmation / reproduction, feed, or return, and proceeds to step S508 if it is not determined. In step S507, the terminal-side control unit 210 performs processing according to user operations such as confirmation / reproduction, feed / return, and the like. After that, the process proceeds to step S508.

In step S508, the terminal-side control unit 210 determines whether or not to end the range designation in the separate editing. In this determination, when the decision button 827 is operated, it is determined that the range designation in the separate editing is finished. If it is determined that the process is completed, the process proceeds to step S509, and if it is not determined, the process returns to step S501.

In step S509, the terminal-side control unit 210 displays an edit screen such as a caption or a title. Further, the terminal-side control unit 210 performs image processing such as adding a caption to the image piece according to the operation result of the user, and generates an inserted image other than the image piece used for generating the ornamental moving image. .. In step S510, the terminal-side control unit 210 specifies a combination of the material image, the material sound, and other images whose range is specified for each.

Here, a diagram for explaining the ornamental moving image generated by the separation editing process is shown in FIG. 9C as a schematic diagram, and the processes of steps S509 and S510 will be described in more detail. As shown in FIG. 9C, the user creates an inserted image such as a title image F30 and an ending image F50. For example, the user inputs a title, a comment, or the like of an ornamental video such as Happy Wedding to create a title image F30. Further, the user specifies a combination of the title image F30, the material image F40, the ending image F50, the order of arrangement, and the like. At this time, the user specifies the positions of the material image and the inserted image while listening to the material sound whose range is specified separately from the material image. In FIG. 9C, the range from 3 minutes to 3 minutes and 30 seconds of the original video is used as the material image, and the range from 2 minutes and 30 seconds to 4 minutes of the original video is used as the material sound as an example. It is shown. For example, the user arranges a material image including an image (image piece) representing the scene at the position of the sound when the applause or cheers become loud and the venue is excited.

In step S511, the terminal-side control unit 210 determines whether or not to end the separate editing. In this determination, it is determined that the separation editing is terminated when it is determined that the decision button 827 is selected. When it is determined that the separate editing is finished, the terminal-side control unit 210 sets the display to the state of the display example of FIG. 7E. If it is not determined to end the process, the process returns to step S501, and if it is determined to end, the process proceeds to step S414 of the mobile terminal control process.

Here, with reference to FIGS. 6A and 6B again, the description of the mobile terminal control process will be continued. In step S414, the terminal-side control unit 210 determines whether or not to end the moving image editing. In this determination, it is determined that the moving image editing is finished when it is determined that the decision button 827 is selected. If it is determined that the process is completed, the process proceeds to step S415, and if it is not determined, the process returns to step S412.

In step S415, the terminal-side control unit 210 generates a moving image file of an ornamental moving image based on a material image, a material sound, an inserted image, and the like. Further, the terminal-side control unit 210 records the generated moving image file in the information recording unit 252 of the terminal-side recording unit 250. After that, the process returns to step S401.

In step S416, the terminal-side control unit 210 sets the range (common time range) of the image piece and the audio piece used for generating the ornamental moving image in the same manner as in the processes of steps S502 to S507 of the separation editing process. Executes the join editing process that is determined according to the operation. As described above, the process related to the combined editing is performed according to the operation of the common range selection bar 820 or the like by the user. Therefore, in the combined editing, the time position and time width of the image piece or sound piece designated as the material image or material sound, that is, the time range (common time range) is the same. In step S417, the terminal-side control unit 210 edits the caption, the title, and the like in the same manner as in step S508 of the separate editing process. After that, the process proceeds to step S414.

In step S418, the terminal-side control unit 210 determines whether or not to receive the temporary editing data (temporary editing image and temporary editing sound) from the capture device 100. The display at this time is the state of the display example shown in FIG. 7B. This determination may be performed based on whether or not the temporary edit data reception icon 805b is selected, or may be performed based on whether or not there is a communication request from the capture device 100. The process proceeds to step S419 if it is determined to be received, and proceeds to step S420 if it is not determined to be received. In step S419, the terminal-side control unit 210 receives the tentatively edited image and the tentatively edited sound from the capture device 100 in units of the image piece and the sound piece. The received temporary edit image and the image piece and the sound piece included in the temporary edit sound are temporarily recorded in the temporary recording unit 251 for each time information attached to each. The tentatively recorded tentatively edited image and tentatively edited sound can be edited in the same manner as in the processes of steps S411 to S414. After that, the process returns to step S404.

In step S420, the terminal-side control unit 210 determines whether or not to set the communication. The display at this time is the state of the display example shown in FIG. 7B. This determination may be performed based on whether or not the communication setting icon 805d is selected, or may be performed based on whether or not there is a communication request from the capture device 100. The process proceeds to step S421 if it is determined that the communication setting is to be performed, and proceeds to step S422 if it is not determined. In step S421, the terminal side control unit 210 sets the communication. In the communication setting, information related to the capture device 100 may be registered as a communication partner. After that, the process returns to step S404.

In step S422, the terminal-side control unit 210 determines whether or not to terminate the video editing application. The display at this time is the state of the display example shown in FIG. 7B. In this determination, it is determined that the video editing application is terminated when the back button 806 is selected. When it is determined that the video editing application is terminated, the terminal-side control unit 210 sets the display to the state of the display example shown in FIG. 7A. If it is not determined that the process is completed, the process returns to step S404, and if it is determined, the process returns to step S401.

<Advantage>
The capture device 100 according to the present embodiment visualizes the first image (video frame) of the original moving image at each specific timing to generate an image piece, and the first image corresponding to the first image at each specific timing. It is possible to separate the voice of and generate a voice piece (voice clip). The capture device 100 attaches time information (first time information and second time information) to each of these image pieces and sound pieces, and transmits the generated image pieces and sound pieces to the mobile terminal 200 in units of the generated image pieces and sound pieces. can. The image piece and the sound piece can be reproduced independently. Further, the capture device 100 according to the present embodiment can also specify a range (temporary editing time range) of an image piece and an audio piece to be transmitted to the mobile terminal 200.

In this way, the capture device 100 according to the present embodiment can reduce the data capacity related to communication as compared with the case where the high-quality original moving image is transmitted to the mobile terminal 200 as it is. Since the reduction of the data capacity to be communicated contributes to the reduction of power consumption and the saving of the recording capacity, it is particularly useful when the capture device 100 or the mobile terminal 200 is a mobile device.

Further, the mobile terminal 200 according to the present embodiment acquires an image piece and an audio piece from the capture device 100 and temporarily records each time information. Further, the mobile terminal 200 can generate an ornamental moving image by combined editing or separate editing based on the image piece and the audio piece temporarily recorded for each time information. Therefore, the user can easily acquire and check the moving image data on the mobile terminal 200 such as a smartphone, or edit and create an ornamental moving image suitable for sharing.

Further, in the separate editing, the terminal side display control unit 218 included in the mobile terminal 200 according to the present embodiment has an image range selection bar corresponding to the first time information and an audio range selection bar corresponding to the second time information. Generate display information to display and. Each time range indicated by the image range selection bar and the voice range selection bar changes according to the result of the user operation acquired by the terminal side operation unit 240. The terminal-side editorial unit 217 acquires the time range indicated by the image range selection bar as the time range (first time range) of the image piece used for generating the ornamental moving image, and sets the time range indicated by the audio range selection bar. , Acquired as the time range (second time range) of the voice piece. For this reason, the user can easily edit the video data on the mobile terminal 200 such as a smartphone, and enjoy the spectacular viewing including the highlight of the image and the highlight of the sound recorded at different time positions in the original video. You can generate a video.

From the above, the technique according to the present embodiment can realize a video editing system configured to obtain ornamental video data (editing video data with audio) that is shorter in time than the original video data. Can be expressed. Here, the original moving image data includes a first image (image data) including a plurality of temporally continuous images according to the above-described embodiment, and a first audio (audio data) corresponding to the first image. And include.

The moving image editing system includes a recording unit in which the image data and the audio data collected at the same time as the image data are simultaneously recorded as moving image data. This recording unit is, for example, the device-side recording unit 150 included in the capture device 100 according to the above-described embodiment, but may be the terminal-side recording unit 250 included in the mobile terminal 200. Further, the image data is acquired by, for example, the device-side image pickup unit 170 according to the above-described embodiment, but the mobile terminal 200 may be provided with the terminal-side image pickup unit and acquired by the terminal-side image pickup unit, or a moving image editing system. It may be obtained from the outside of. Similarly, the sound data collected at the same time as the image data is acquired by, for example, the device-side sound collecting unit 180 according to the above-described embodiment, but the mobile terminal 200 includes a terminal-side sound collecting unit and is provided with a terminal-side image pickup unit. It may be obtained by, or it may be obtained from outside the video editing system.

In the moving image editing system, an image piece is cut out from the recorded image data at the first timing. Further, the audio piece is cut out at the second timing from the audio data recorded at the same time as the image data. Here, the first timing and the second timing may be the same or different.

The video editing system has audio that is shorter than the original video data (image data and audio data) from the image fragment clipped at the first timing and the audio fragment clipped at the second timing. It is configured to obtain edited video data. The generation of the edited moving image data with audio is performed, for example, in the mobile terminal 200 according to the above-described embodiment, but may be performed in the capture device 100.

In addition, although it is described that the audio data is the audio data acquired at the same time as the image data, this is the image data (first) included in a single video file (original video data) by this video editing system. It shows that it is a system for editing image) and audio data (first audio). Therefore, the image data and the audio data need only be recorded as a single moving image file, and are not necessarily acquired at the same time.

<Modification example>
(Modified example of separate editing process)
In the separate editing process, the same material image may be repeatedly used and combined with the material sound. Similarly, the same material audio may be used repeatedly and combined with the material image. Here, an example of a display when the material image is repeatedly used to generate an ornamental moving image is shown in FIG. 10A, and a diagram for explaining the ornamental moving image in which the material image is repeatedly used is shown in FIG. 10B as a schematic diagram. show. With reference to FIGS. 10A and 10B, processing when the material image is repeatedly used to generate an ornamental moving image will be described in more detail. The display shown in FIG. 10A is a state after the image range selection bar 820a is operated and the image time width 818a is specified. For example, when the display position of the image time width 818a is touched and selected, the image repeat button 818ab is displayed at the display position. When the image repeat button 818ab is selected, as shown in FIG. 10B, the material image having the currently specified image time width 818a is repeatedly used to generate an ornamental moving image. At this time, an inserted image such as a title may be used, or a caption may be added to an image piece constituting the material image.

In the above-described embodiment and modification, the case where the inserted image is used in addition to the image piece and the audio piece to generate an ornamental moving image has been described as an example, but the present invention is not limited to this. The ornamental moving image may be generated based only on the image piece and the sound piece, or may be generated by using the sound effect or the inserted image and the sound effect in addition to the image piece and the sound piece.

In the above-described embodiment and modification, the case where the user inputs a title or the like to generate the inserted image has been described as an example, but the present invention is not limited to this. The inserted image and sound effect may be acquired from a server or the like on a telecommunication line such as the Internet, or may be acquired from a device-side recording unit 150 or a terminal-side recording unit 250. At this time, an image piece or an audio piece generated from another original moving image may be used as the inserted image or sound effect. When the mobile terminal 200 includes a camera, the image data captured and acquired by the mobile terminal 200 may be used as an inserted image. Similarly, when the mobile terminal 200 is provided with a microphone, the voice data acquired by the mobile terminal 200 may be used as a sound effect.

In the above-described embodiment and modification, the case where the inserted image is added before and after the image piece (material image) included in the first time range has been described as an example, but the present invention is not limited to this. The inserted image may be inserted between the image piece (material image) and the image piece (material image), or may be superimposed on the image piece. Similarly, the sound effect may be inserted between the sound piece (material sound) and the sound piece (material sound), or may be added so as to be superimposed on the sound piece.

A part of the functions of the information processing unit 111 on the device side may be provided outside the capture device 100. Further, the information processing unit 111 on the device side may generate an image piece and an audio piece based on the moving image data acquired from the outside of the capture device 100. Similarly, a part of the functions of the information processing unit 211 on the terminal side may be provided outside the mobile terminal 200 such as a server provided on the Internet. In this case, the image piece and the sound piece generated by the capture device 100 may be transmitted to the server.

(Modified example of playback processing with temporary editing)
In the above-described embodiment and modification, the reproduction process with temporary editing, which is executed by the capture device 100 and can select the time range (temporary editing time range) of the image piece and the audio piece transmitted to the mobile terminal 200, has been described. Further, the separate editing process executed by the mobile terminal 200 has been described.

Under such circumstances, the time range (temporary editing time range) of the image piece and the sound piece transmitted from the capture device 100 to the mobile terminal 200 may be different between the image piece and the sound piece. Therefore, in this modification, in the playback process with temporary editing executed by the capture device 100, a part of the separate editing process is performed, and the time range of the image piece selected as the temporary editing image (third time range). ) And the time range (fourth time range) of the audio piece selected as the temporary editing voice will be described with respect to the reproduction processing with temporary editing in which the time range (fourth time range) is individually selected.

Here, an example of the reproduction process with temporary editing executed by the capture device 100 according to the present modification is shown in FIGS. 11A and 11B as flowcharts, and the operation of the information processing system 1 will be described with reference to these. The reproduction process with temporary editing according to this modification is performed in step S117 of the capture device control process described with reference to FIG.

The processes of steps S601 to S609 are the same as the processes of steps S301 to S309 of the reproduction process with temporary editing according to the first embodiment, respectively, and the device side control unit 110 is an image from the selected original moving image. Generate pieces and audio pieces. The range of the original moving image in which the image piece and the audio piece are generated may be a part of the original moving image as in the provisional editing according to the first embodiment, or the image piece according to the first embodiment. And, as with the audio piece transmission process, the entire original moving image may be used. In step S610, the device-side control unit 110 determines whether or not to end the processing related to image fragmentation and audio fragmentation in the same manner as in step S310. If the process is not determined, the process returns to step S606, and the process related to image fragmentation and audio fragmentation is continued until it is determined that the process is completed in this step. If it is determined that the process ends, the process proceeds to step S611.

Here, an example of the display in step S601 is shown in FIG. 12A. In the display example shown in FIG. 12A, as in the display example shown in FIG. 7C, thumbnail images showing each moving image file recorded in the device-side recording unit 150 are displayed in a list. The user operates the operation unit 140 on the device side to select a moving image file to be played, edited, or the like.

Here, an example of the display in step S604 is shown in FIG. 12B. In the display example shown in FIG. 12B, a thumbnail image showing the moving image file selected in step S601 is displayed on the preview screen 813. Further, in the display example shown in FIG. 12B, the play button 814, the temporary edit button 815a, the back button 806, and the forward / back button 811 are further displayed. When the play button 814 is selected, the device-side control unit 110 starts playing the moving image file. When the temporary editing button 815a is selected, the device-side control unit 110 executes a process related to temporary editing of the moving image file. At this time, the process proceeds to step S606, and an image piece and an audio piece are generated. When the return button 806 is selected, the device-side control unit 110 returns the display to the state of the display example of FIG. 12A and waits for the user to select the thumbnail image. When the forward / backward button 811 is selected, the device-side control unit 110 switches the moving image file displayed on the preview screen 813, that is, the currently selected moving image file to another moving image file.

In step S611, the device-side control unit 110 temporarily records an image piece and an audio piece in the device-side recording unit 150 for each time information in the same manner as in step S410 of the mobile terminal control process executed by the mobile terminal 200. ..

The processes of steps S612 to S614 are the same as the processes of steps S411 to S413 of the mobile terminal control process, respectively. The device-side control unit 110 generates display information for combined editing, including a common range selection bar 820 for both image and audio, and displays it on the device-side display unit 120 included in the capture device 100.

Here, an example of the display in step S612 is shown in FIG. 12C. In the display example shown in FIG. 12C, the back button 806, the forward / back button 811, and the preview screen 813 are displayed. When the return button 806 is selected, the device-side control unit 110 returns the display to the state of the display example shown in FIG. 13B. Further, in the display example shown in FIG. 12C, a separate edit button 825 for instructing separate edit is displayed. When the separation / editing button 825 is selected, the device-side control unit 110 starts the processing related to the separation / editing according to this modification. At this time, the process proceeds to step S613.

In step S613, the device-side control unit 110 executes the separate editing process in the provisional editing in the same manner as the separate editing process described with reference to FIG. The temporary edit data generated by the separate edit process in the temporary edit executed by the capture device 100 is further edited (main edit) in the separate edit process executed by the portable terminal 200. Therefore, the separate editing process in the temporary editing executed by the capture device 100 may be simpler than the separate editing process executed by the mobile terminal 200. For example, in the separate editing process according to this modification, the process related to the input of captions, titles, and the like may not be performed. For example, in the separate editing process according to this modification, the process related to the combination designation is not performed. The mobile terminal 200 has a richer user interface than the capture device 100 in the processing related to input of captions, titles, etc. and the processing related to combination specification, and it is easy to acquire inserted images, exchange sounds, etc. via the Internet or the like. It may be done in this editing in.

Here, an example of the display in the separate editing process in the temporary editing in step S613 is shown in FIG. 12D. The display example shown in FIG. 12D is the same as the display example shown in FIG. 9B. However, in the display example shown in FIG. 12D, the decision transmission button 830 is displayed instead of the decision button 827 in the display example shown in FIG. 9B.

The processes of steps S615 and S616 are the same as the processes of steps S311 and S312 of the reproduction process with temporary editing according to the first embodiment, respectively. When the decision transmission button 830 is selected, the device-side control unit 110 ends the process related to the separate edit and the temporary edit process according to this modification, and generates the temporary edit data. Here, the temporary editing image (temporary editing frame) is an image piece included in the time range (third time range) indicated by the current image range selection bar 820a. Similarly, the provisionally edited voice is a voice piece included in the time range (fourth time range) indicated by the current voice range selection bar 820b. The device-side control unit 110 causes the mobile terminal 200 to transmit the generated temporary editing data (temporary editing image and temporary editing sound) in units of an image piece and a sound piece. After transmission, the device-side control unit 110 ends the reproduction process with temporary editing.

An example of the display in step S615 is shown in FIG. 12E. The display example shown in FIG. 12E shows the display when the temporary editing data is transmitted to the mobile terminal 200. In the display example shown in FIG. 12E, a preview screen 831 for sequentially displaying the temporarily edited images (image pieces) being transmitted, a transmission status display unit 832 indicating that the temporary editing data is being transmitted, and a table shown in FIG. 12D. A back button 833 for returning to the illustrated state is displayed. In the display example shown in FIG. 12E, the transmission status display unit 832 indicates that transmission is in progress. The preview screen 831 may display a thumbnail image showing the provisional editing data being transmitted. Further, the back button 833 may be a button instructing to return to step S111 of the capture device control process. Further, while the temporary editing data is being transmitted, the mobile terminal 200 displays a reception status display unit 834 indicating that the temporary editing data is being received. In the display example shown in FIG. 12E, the reception status display unit 834 indicates that a moving image is being received.

The processes of steps S617, S618 and S619 are the same as the processes of steps S411, S416 and S417 of the mobile terminal control process according to the first embodiment, respectively. The device-side control unit 110 generates temporary editing data based on the range (temporary editing time range) specified by operating the common range selection bar 820. However, the process related to the editing of captions, titles, etc. executed in step S619 may not be performed in the provisional editing as in the process of step S613.

The temporary editing data generated in this way and transmitted to the mobile terminal 200 is used as a material moving image for generating an ornamental moving image in the mobile terminal 200. Here, an example of the display on the mobile terminal 200 after receiving the temporary editing data is shown in FIG. 12F. As shown in the display example shown in FIG. 12F, the main editing button 835 for instructing the execution of the main editing is displayed. Here, this editing is a moving image editing using an image piece and an audio piece included in the provisional editing data. When the edit button 835 is selected, the terminal-side control unit 210 executes the separate edit process as described with reference to FIGS. 9A and 9B.

As described above, the capture device 100 according to the present modification can individually set the time range of the image piece and the sound piece to be transmitted to the mobile terminal 200 by separate editing. Therefore, in the information processing system 1 according to this modification, the amount of data to be communicated can be further reduced. Further, the user can roughly specify the range of the original moving image to be used for the ornamental moving image by using the capture device 100, for example, without forgetting the captured contents.

The present invention is not limited to the above-described embodiment and modification, and can be variously modified at the implementation stage without departing from the gist thereof. Further, the above-described embodiment and each modification may be carried out in combination as appropriate, and in that case, the combined effect can be obtained. Further, the above-described embodiments and modifications include various inventions, and various inventions can be extracted by a combination selected from a plurality of disclosed constituent requirements. For example, if the problem can be solved and the effect can be obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiments and modifications, the configuration in which the constituent elements are deleted is extracted as an invention. Can be done.

In the above-described embodiment and modification, the portion described as a section or a unit may be configured by combining a dedicated circuit or a plurality of general-purpose circuits, and if necessary, pre-programmed software. A microcomputer, a processor such as a CPU, or a sequencer that operates according to the above may be combined and configured. It may be partly or wholly designed with FPGA. Further, it is possible to design so that an external device takes over a part or all of the control, and in this case, a wired or wireless communication circuit intervenes. Communication may be performed by Bluetooth, Wi-Fi, a telephone line, or the like, or may be performed by USB or the like. A dedicated circuit, a general-purpose circuit, and a control unit may be integrally configured as an ASIC.

1 ... information processing system, 100 ... capture device, 110 ... device side control unit, 111 ... device side information processing unit, 112 ... device side compression unit, 113 ... device side expansion unit, 114 ... device side division unit, 115 ... device Side communication control unit, 116 ... device side recording control unit, 120 ... device side display unit, 130 ... device side voice reproduction unit, 140 ... device side operation unit, 150 ... device side recording unit, 151 ... image recording unit, 152 ... Audio recording unit, 160 ... device side communication unit, 161 ... first device side communication unit, 162 ... second device side communication unit, 170 ... device side image pickup unit, 180 ... device side sound pickup unit, 190 ... device side Clock unit, 200 ... Mobile terminal, 210 ... Terminal side control unit, 211 ... Terminal side information processing unit, 212 ... Terminal side compression unit, 215 ... Terminal side communication control unit, 216 ... Terminal side recording control unit, 217 ... Terminal side Editing unit, 218 ... Terminal side display control unit, 220 ... Terminal side display unit, 230 ... Terminal side audio reproduction unit, 240 ... Terminal side operation unit, 250 ... Terminal side recording unit, 251 ... Temporary recording unit, 252 ... Information recording Unit 260 ... Terminal side communication unit, 261 ... First terminal side communication unit, 262 ... Second terminal side communication unit.

Claims (21)

A plurality of image pieces including a plurality of images consecutive in time are time-divided, each of which has the first time information and is generated as an image that can be reproduced independently, and the first image. A terminal-side communication control unit that acquires a plurality of voice pieces generated as voices in which the first voice corresponding to an image is time-divided and each has a second time information and can be played independently. ,
A terminal-side operation unit for designating a first time range from the time range of the first image and designating a second time range from the time range of the first sound.
The image piece included in the first time range is selected from the plurality of received image pieces, and the sound included in the second time range is selected from the received sound pieces. A mobile terminal for video editing, equipped with a terminal-side editorial unit for selecting one piece. An image range selection bar for designating the first time range from the time range of the first image, and a second time range for designating the second time range from the time range of the first sound. The portable terminal according to claim 1 , further comprising a terminal-side display control unit that generates display information for displaying an audio range selection bar. Each of the first time information is the time position of each of the image pieces in the first image, and each of the second time information is the time of each of the voice pieces in the first voice. The mobile terminal according to claim 1 , which is a position. The mobile terminal according to claim 1 , further comprising a temporary recording unit for temporarily recording the image piece and the audio piece for each attached time information. The mobile terminal according to claim 1 , wherein the terminal-side editorial unit generates a moving image file based on the selected image piece and audio piece. The mobile terminal according to claim 1 , wherein the terminal-side editing unit controls separate editing capable of individually editing each time range of the first time range and the second time range. A recording unit in which a first image including a plurality of temporally continuous images and a first sound corresponding to the first image are recorded.
The first image is time-divided and the first time information is attached to generate a plurality of image pieces which are images that can be reproduced independently, and the first sound is time-divided and the second time. A split unit that generates multiple audio pieces that are audio that can be played independently by attaching information.
A capture device for video editing, comprising a device-side communication control unit for transmitting the image piece and the audio piece to the outside. An image pickup unit that captures an image of an object and outputs an image,
A sound collecting unit that collects the target sound and outputs the sound,
The seventh aspect of the present invention further comprises a recording control unit that records the first image and the first sound in the recording unit based on the image output by the imaging unit and the sound output by the sound collecting unit. The capture device described. A device-side operation unit for designating a third time range from the time range of the first image and designating a fourth time range from the time range of the first sound is further provided.
The capture device according to claim 7 , wherein the device-side communication control unit transmits the image piece included in the third time range and the voice piece included in the fourth time range to the outside. An image range selection bar for designating the third time range from the time range of the first image, and a fourth time range for designating the fourth time range from the time range of the first sound. The capture device according to claim 9 , further comprising a device-side display control unit that generates display information for displaying an audio range selection bar. The mobile terminal according to any one of claims 1 to 6 and the mobile terminal.
The capture device according to any one of claims 7 to 10 .
An information processing system equipped with. A plurality of image pieces including a plurality of images consecutive in time are time-divided, each of which has the first time information and is generated as an image that can be reproduced independently, and the first image. The first sound corresponding to the image is time-divided, and a plurality of sound pieces generated as voices, each of which has a second time information and can be played independently, are acquired by communication.
Specifying the first time range from the time range of the first image according to the user operation, and
To select the image piece included in the first time range from the plurality of received image pieces, and to select the image piece.
Specifying the second time range from the first voice time range according to the user operation, and
An information processing method for video editing, which comprises selecting the audio piece included in the second time range from a plurality of received audio pieces. An image range selection bar for designating the first time range from the time range of the first image, and a second time range for designating the second time range from the time range of the first sound. The information processing method according to claim 12 , further comprising displaying an audio range selection bar. The information processing method according to claim 12 , further comprising generating a moving image file based on the selected image piece and the audio piece. The information processing method according to claim 12 , wherein each time range of the first time range and the second time range is edited individually. In a capture device in which a first image including a plurality of temporally continuous images and a first sound corresponding to the first image are recorded.
To generate a plurality of image pieces that can be reproduced independently by time-dividing the first image,
Attaching the first time information to each of the plurality of image pieces,
To generate a plurality of audio pieces that can independently reproduce the first audio by dividing the time,
Attaching a second time information to each of the plurality of audio pieces,
An information generation method for video editing, which comprises transmitting the image piece and the sound piece to a mobile terminal. To generate the first image by imaging the object,
Collecting the target voice to generate the first voice,
The information generation method according to claim 16 , further comprising recording the first image and the first sound. A plurality of image pieces including a plurality of images consecutive in time are time-divided, each of which has the first time information and is generated as an image that can be reproduced independently, and the first image. The first sound corresponding to the image is time-divided, and a plurality of sound pieces generated as voices, each of which has a second time information and can be played independently, are acquired by communication.
Specifying the first time range from the time range of the first image according to the user operation, and
To select the image piece included in the first time range from the plurality of received image pieces, and to select the image piece.
Specifying the second time range from the first voice time range according to the user operation, and
An information processing program for video editing that causes a computer to select a voice piece included in the second time range from a plurality of received voice pieces. An image range selection bar for designating the first time range from the time range of the first image, and a second time range for designating the second time range from the time range of the first sound. The information processing program according to claim 18 , further causing a computer to display a voice range selection bar. The information processing program according to claim 18 , further causing a computer to generate a moving image file based on the selected image piece and the sound piece. The information processing program according to claim 18 , wherein each time range of the first time range and the second time range is edited individually.

Images