WO2011129059A1

WO2011129059A1 - Information processing device and information processing method

Info

Publication number: WO2011129059A1
Application number: PCT/JP2011/001725
Authority: WO
Inventors: 武紀米津
Original assignee: パナソニック株式会社
Priority date: 2010-04-16
Filing date: 2011-03-24
Publication date: 2011-10-20

Abstract

Even at scene changes, the disclosed information processing device can easily and appropriately control the amount of data produced in an encoding process. Said information processing device (100), which also allows easy seamless playback, is provided with a video clip storage unit (102), a clip end-image storage unit (201), a blending unit (202), and an encoding process unit (103). The video clip storage unit (102) stores a video clip comprising a plurality of taken images. The clip end-image storage unit (201) stores a clip end image, which is the last taken image from a single video clip obtained from the beginning to the end of imaging by an imaging device. The clip end image and an image taken at the beginning of the next imaging are inputted to the blending unit (202), which blends said images to generate image data. The encoding process unit (103) encodes the image data generated by the blending unit (202), and on the basis of control information, controls the size of the encoded image data.

Description

Information processing apparatus and information processing method

The present invention relates to an information processing apparatus and an information processing method, and more particularly to an information processing apparatus and an information processing method for realizing seamless reproduction without reproduction interruption.

For example, in shooting using a shooting device such as a camera or a camcorder, a desired scene, that is, a video is often shot (recorded) by repeating recording and pause (shooting start and shooting stop). In addition, when playing back images shot in such a manner, there is a demand for smoothly playing back joints between scenes (hereinafter referred to as seamless playback).

The video shot as described above is recorded on an information recording medium (recording medium) by image signal processing that performs data compression using, for example, the MPEG (Moving Picture Experts Group) technique, and when the recorded video is played back, seamless playback is performed. May not be possible. Hereinafter, when recording a video, a video composed of a plurality of shot images obtained from the start of shooting (recording) to the stop of shooting (recording) is referred to as a video clip.

FIG. 1 is a diagram showing a functional configuration of a conventional imaging system. FIG. 1 shows a conventional imaging system including an imaging device 901, an information processing device 900, a recording medium 905, and a display device 908.

The photographing device 901 is, for example, a camera or a camcorder, and photographs a video clip composed of a plurality of images. Specifically, the imaging device 901 captures a video clip by repeating recording start and stop according to an instruction from the photographer, and the information processing device 900 records it.

The recording medium 905 is, for example, a recording medium capable of recording such as SD (Secure Digital) memory, HDD (Hard Disk Drive), CD-R (Compact Disc Recordable), DVD-R (Digital Versatile Disk Recordable), and the like. The video clip photographed in 901 is recorded by the information processing apparatus 900.

Next, recording processing and reproduction processing of the conventional imaging system shown in FIG. 1 will be described.

First, the recording process will be described. An image signal (video clip) photographed by the photographing device 901 is subjected to signal processing, stored in a frame buffer 902, and encoded into a format that can be recorded on a recording medium by an encoder 903. The encoded image signal (video clip) is temporarily stored in the stream buffer 904, but is stored in the recording medium 905 at an appropriate timing according to the buffer amount managed by the information processing apparatus 900.

Next, the playback process will be described. A stream (video clip) necessary for decoding is temporarily stored in the stream buffer 904 from the recording medium 905 at an appropriate timing according to the buffer amount managed by the information processing apparatus 900. The temporarily accumulated stream (video clip) is decoded by the decoder 906 into a format that can be displayed by the display device 908, stored in the frame buffer 907, and then sent to the display device 908 such as a digital television.

In this way, the video imaged by the imaging device 901 is data-compressed by the encoder 903 that performs encoding and is decompressed by the decoder 906 that performs decoding. Specifically, when decoding the encoded data (video clip) read from the recording medium 905, the decoder 906 stores the encoded data (video clip) in the internal buffer (stream buffer 904). Decode the encoded data (video clip).

However, if the decoder 906 uses the internal buffer (stream buffer 904) for decoding, the internal buffer (stream buffer 904) may overflow.

On the other hand, in order to prevent the internal buffer (stream buffer 904) from overflowing, a method of controlling the playback of the next video clip after the playback of the previous video clip is conceivable. However, with this control, for example, when playback is performed across video clips, a discontinuity of continuity occurs, for example, the image immediately before the subsequent video clip is displayed in an extended manner.

Therefore, a technique for enabling seamless reproduction without causing interruption of continuity is disclosed (for example, Patent Documents 1 to 3).

As disclosed in Patent Documents 1 to 3, seamless playback can be performed without interruption of continuity even when playback is performed across a plurality of video clips when a specific condition is satisfied.

Specifically, a method of recording on a recording medium 905 after performing an encoding process for realizing seamless reproduction when recording an image using a photographing device 901 such as a camcorder is disclosed. That is, when video recording is performed at discontinuous timing, buffer status information or stream time stamp when controlling the amount of code managed by the encoder 903 at the recording stop time (shooting stop time) of the preceding video clip, etc. A method of using so-called seamless information for encoding a subsequent video clip is disclosed.

Thus, the encoding (seamless) is performed by the encoder 903 so that the internal buffer (stream buffer 904) used by the decoder 906 for decoding does not overflow when the video clip is reproduced, that is, the buffer model does not break down. Recording) can be performed. More specifically, the encoder 903 generates a stream that can be seamlessly reproduced and records the stream on the recording medium 905, and stores the seamless information in a memory such as a RAM provided separately in the information processing apparatus 900. Then, when the next video recording is performed (when the next video recording is started and the captured video clip is encoded), the seamless information is read to generate a video clip that can be seamlessly reproduced. In this way, encoding (seamless recording) processing by the encoder 903 can be performed so that the buffer model does not fail.

In the AVCHD (Advanced Video Codec High Definition) standard, there is a definition for generating a video clip that can be seamlessly played using the seamless information.

JP 2008-061032 A JP 2005-136633 A JP 2007-306258 A

However, in the seamless recording control to which the above-described seamless information is applied, the buffer status information when the code amount managed by the encoder 903 is controlled at the recording stop time (shooting stop time) of the preceding video clip is used. For this reason, when the subsequent video clip is shot, if the remaining amount of the buffer that can be used by the encoder 903 is small, the code amount that is originally intended to be allocated is not allocated, and there is a problem that the image quality is deteriorated.

For example, in a video clip created using a data compression technique typified by MPEG, a large number of rates are assigned to places where image quality degradation is easily perceived, and the rate distribution at places where image quality degradation is difficult to perceive is deleted. The recording rate of the video clip is dynamically changed. As a result, compression is performed so as not to deteriorate the image quality as a whole without assigning a rate more than necessary to a portion where it is difficult to perceive image quality deterioration. When the bit rate is changed in this way, the bit rate is controlled on the encoder 903 side so that the continuous video can be decoded without interruption on the decoder 906 side (so that seamless playback can be performed). That is, on the encoder 903 side, the data amount of the VBV buffer is controlled on the encoder 903 side so that a VBV (Video Buffering Verifier) buffer that temporarily holds data input to the decoder 906 does not overflow or underflow. .

For example, in the case of a video clip composed of normal moving images, the current picture is generated based on the generated code amount of the past picture by utilizing the fact that the correlation of the image contents between temporally adjacent pictures is high. There is a method of assigning a code amount. More specifically, in an image compression method that combines motion compensation represented by the MPEG standard and DCT (Discrete Cosine Transform), code amount control is performed so that the bit stream compressed by the encoder has a desired rate. Yes. Here, the code amount control is generally a method in which the quantization step is feedback-controlled based on the relationship between the previous quantization step and the code amount and the current average rate. That is, if the current average rate is higher than the target rate, the quantization step is slightly roughened to lower the total code amount, and if the current average rate is lower, the quantization step is slightly finer to increase the total code amount. . By controlling to achieve the target rate on average in this way, the total code amount will increase or decrease when viewed in fine time, but the average code amount will be set as the target value when viewed over a long time. Can do. This code amount control is proposed as a test model in MPEG2. That is, the code amount control here is performed by performing feedback control using the relationship between the remaining amount of the virtual buffer and the generated code amount at the time of previous encoding.

However, in the code amount control by the above feedback control, for example, the rate is instantaneously increased at a location such as a scene change where the correlation of the image contents between adjacent pictures is low. Therefore, depending on the application, the reproduced image may be broken, or a sufficient amount of code cannot be allocated, resulting in extreme image quality degradation.

As can be seen from the above example, if the remaining amount of buffer that can be used by the encoder 903 is small when a subsequent video clip is shot, the code amount that is originally intended to be allocated is not allocated, resulting in image quality degradation. There is a problem.

In addition, when reproducing a video clip that is not seamlessly connected (CC = 1 in the AVCHD standard), it is necessary to decode the subsequent video clip after completing the decoding of the previous video clip in order to prevent the decoding buffer from failing. . Therefore, in order to prevent the video of the last frame of the preceding video clip from being continued, the decoder of the playback device can pre-read and process the next video clip. However, this requires an image holding buffer before decoding and before display, and a buffer memory for the time required for the decoder to decode one frame and for the number of reference images necessary for decoding. It will be a significant cost increase.

On the other hand, as a means for solving this, the AVCHD standard includes a standard (CC = 5) for seamlessly connecting video clips. However, in the reproduction of the video clip as described above, it is necessary to ensure at the time of encoding that the buffer at the time of decoding does not fail. For this reason, it is necessary to implement a scene joining algorithm that is conscious of decoder processing in advance at the time of recording (encoding) so that a decoder of a playback device such as a TV can seamlessly reproduce the junction between scenes. In particular, when dealing with various scenes and use cases, a sufficient code amount cannot be assigned to the video information at the scene junction point, and the image quality deteriorates. Furthermore, it is necessary to prevent a buffer failure at the time of decoding or an encoding or decoding processing time performance failure, and the development difficulty becomes extremely high.

Furthermore, with the seamless playback described above, there is a problem that it is far from the images taken by professional photographers at events and broadcasts, simply by connecting the captured images before and after resuming shooting to the inorganic material. There are processing means using editing equipment and playback machines that reproduce by adding special effects, but the equipment itself is very expensive. Also, AVCHD and other H.264. Although the H.264 format video stream has high compatibility with AV equipment, the stream folder structure is complex, and at the same time, it requires high performance in terms of the required processor performance, and has high compatibility with PCs. It is not a thing. For this reason, even if PC editing software is used, an environment that satisfies the required performance is necessary in order to obtain a high video effect, and it cannot be said that it is practically used. That is, there is no versatility. In addition, since editing is performed after shooting is completed at a different location and time from shooting, there is a problem that viewing and enjoyment cannot be performed immediately.

The present invention has been made in view of the above-described circumstances, and an information processing apparatus and information processing capable of easily and appropriately controlling the amount of generated code in an encoding process even at a scene change location and easily seamlessly reproducing. It aims to provide a method.

In order to achieve the above object, an information processing apparatus according to an aspect of the present invention encodes and records a plurality of video clips shot by repeating start and stop of shooting a plurality of times in a shooting apparatus. A video holding unit that holds a video clip composed of a plurality of shot images shot by the shooting device, and one video clip obtained from the start to the stop of shooting of the shooting device, An end image holding unit that retains an end shot image as a clip end image, the clip end image, and a shot image shot at the start of the next shooting, and the input clip end image and input A blend unit that generates image data by blending the captured image captured at the start time of the next capturing, and the clip from the terminal image holding unit A synchronization control unit that synchronizes input of an end image and input of a captured image captured at the start of the next capturing from the video holding unit, and a blend control unit that controls the blending unit by control information And an encoding processing unit that encodes the image data generated by the blending unit and controls the code amount of the image data to be encoded based on the control information.

With this configuration, it is possible to easily and appropriately control the amount of generated code in the encoding process at a scene change location, and to realize an information processing apparatus that can be easily seamlessly reproduced. Thus, not only can the image be correlated with respect to a location such as a scene change, the generated code amount in the encoding process can be controlled easily and appropriately, and seamless reproduction can be easily performed.

The encoding processing unit includes an encoding unit that encodes the image data generated by the blend unit, and a code that controls a code amount of image data encoded by the encoding unit based on the control information. An amount control unit, and at the start of the next shooting, the code amount control unit controls the amount of code encoded by the encoding unit according to the control information, and the encoding unit includes the code The image data generated by the blending unit is encoded with the code amount controlled by the amount control unit, and the code amount control unit depends on the control information from the start to the stop of the photographing of the photographing apparatus. Without controlling the code amount encoded by the encoding unit for each encoding processing unit, the encoding unit is input to the blend unit with the code amount controlled by the code amount control unit After the start of the next shooting The image data generated by the blending unit may be encoded by blending only a plurality of photographed images. Here, the blend control unit may determine the control information based on a code amount preset in the information processing apparatus, and the encoding processing unit may be configured such that the end image holding unit is the clip end point. When holding the image, the code amount of the clip end image may be calculated, and the blend control unit may determine the control information based on the code amount calculated by the encoding processing unit.

Thereby, at the start of the next photographing of the photographing apparatus, the generated code amount in the encoding process can be controlled easily and appropriately by controlling the encoded code amount according to the control information.

Further, the blend control unit uses the information on the current code amount calculated by the encoding processing unit to cause the blend unit to capture the clip end image and a captured image captured at the start of the next capturing. The image data may be generated by changing the blending ratio.

With this configuration, the clip end image can be blended step by step with the captured image captured at the time when the next capturing is started.

Further, the blend control unit is configured to control the control information based on the code amount calculated by the encoding processing unit, the clip end image, and the image correlation of the captured image immediately before the clip end image. In accordance with the determined control information, the blending unit is caused to generate the image data by changing a ratio of blending the clip end image and the captured image captured at the start of the next capturing. It is good.

With this configuration, by controlling the rate of change in the ratio of blending the clip end image and the shot image shot at the next shooting start time, an unnecessary code amount from the viewpoint of image relevance is controlled. Occurrence can be suppressed, and buffer failure during seamless playback can be further suppressed.

The present invention can be realized not only as such an information processing apparatus, but also as an information processing method using steps characteristic of the information processing apparatus. It can also be realized as a program to be executed. Needless to say, such a program can be distributed via a recording medium such as a CD-R or a transmission medium such as the Internet.

According to the present invention, it is possible to easily and appropriately control the amount of generated code in an encoding process even at a scene change location, and to realize an information processing apparatus that can be easily seamlessly reproduced.

Specifically, unlike a still image, a moving image is a video recording method that accompanies a time axis rather than a moment, and therefore, images from recording to stop are recorded while repeating this operation. For example, when a recorded image (video) is to be viewed, there is a demand to seamlessly reproduce the image. According to the present invention, a method for easily realizing this at an extremely low cost is provided in the AVCHD standard. be able to.

More specifically, when data recording on the information recording medium (recording medium) is stopped, the data obtained up to the recording stop instruction is recorded on the information recording medium, and then one frame or 1 GOP (Group Of Pictures), etc. A very short data processing unit is held in the buffer, and is superposed on the data scheduled to be recorded at the start of subsequent recording. Thereby, it is possible to correlate the video information between the video clips, and it is possible to realize the seamless reproduction that prevents the buffer failure at the time of reproducing the subsequent clip across the video clips and prevents the deterioration of the image quality.

FIG. 1 is a diagram illustrating a functional configuration of a conventional imaging system. FIG. 2 is a diagram showing a main block configuration of the imaging system according to the present invention. FIG. 3 is a diagram showing a main functional configuration of the imaging system according to the present invention. FIG. 4 is a diagram for explaining an information processing method for captured images that enables seamless reproduction of the information processing apparatus according to the present invention. FIG. 5 is a diagram for explaining a method of recording a captured image that enables seamless reproduction of the information processing apparatus according to the present invention. FIG. 6 is a flowchart for explaining the information processing method of the imaging system according to the present invention. FIG. 7 is a diagram illustrating an example of an imaging apparatus equipped with the information processing apparatus according to the present invention. FIG. 8 is a diagram illustrating an example of an imaging apparatus equipped with the information processing apparatus according to the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment)
FIG. 2 is a diagram showing a main block configuration of the imaging system according to the present invention, and FIG. 3 is a diagram showing a main functional configuration of the imaging system according to the present invention.

The image capturing apparatus 101 is a moving image recording apparatus such as a camera or a camcorder, for example, and captures a video clip (captured image group) by capturing a plurality of images. Specifically, the photographing apparatus 101 records a video clip (captured image group) by repeatedly starting and stopping recording in accordance with a photographer's instruction.

The recording medium 105 is a recordable medium such as an SD memory, HDD, CD-R, or DVD-R, for example, and data (here, image data) is recorded by the information processing apparatus 100.

The information processing apparatus 100 shown in FIG. 2 encodes and records a plurality of video clips shot by repeating the start and stop of shooting a plurality of times in the shooting apparatus 101. Specifically, the information processing apparatus 100 encodes an image input from the photographing apparatus 101 by a technique such as MPEG2 or H264, and records the encoded image on the recording medium 105, for example. Note that a video clip is composed of a plurality of captured images.

The information processing apparatus 100 illustrated in FIG. 2 includes a video clip holding unit 102, an encoding processing unit 103, a stream buffer 104, a clip end image holding unit 201, a blending unit 202, a synchronization control unit 203, a blend And a control unit 204.

The video clip holding unit 102 holds a video clip composed of the number of shot images shot by the shooting device 101.

Specifically, the image signal photographed by the photographing apparatus 101 is subjected to signal processing and held as a photographed image signal. In addition, the video clip holding unit 102 holds image information for a processing unit sufficient to encode the end image information at a timing when a stop instruction is given to the photographing apparatus 101 by the photographer. Here, the image information for the processing unit sufficient to encode the end image information is, for example, a plurality of times corresponding to a minimum unit (1 GOP) or more composed of pictures having a reference relationship with the clip end image. A plurality of captured images such as 1 GOP after a stop instruction.

Note that the video clip holding unit 102 may hold one video clip obtained from the start to the stop of shooting of the shooting apparatus 101, and is shot by repeating the start and stop of shooting of the shooting apparatus 101 a plurality of times. A plurality of video clips may be held.

The clip end image holding unit 201 holds a terminal shot image as a clip end image among one video clip obtained from the start to the stop of shooting by the shooting apparatus 101.

Specifically, the clip end image holding unit 201 corresponds to a processing unit sufficient for the video clip holding unit 102 to encode the end image information at a timing when a stop instruction is given to the photographing apparatus 101 by the photographer. After the image information is held, the next one frame of image information is held as a clip end image signal (clip end image).

Note that the clip end image signal (clip end image) may be, for example, a plurality of captured images for a time equal to or longer than the minimum unit (1 GOP) composed of pictures having a reference relationship with the clip end image. Further, the clip end image signal (clip end image) is, for example, a final shot image of video clips (a plurality of shot images) obtained from the start to the stop of shooting by the shooting apparatus 101, and is equivalent to one frame. It may be a still image.

When resuming shooting (starting the next shooting), the synchronization control unit 203 controls the clip end image holding unit 201 and the video clip holding unit 102 to input the clip end image from the clip end image holding unit 201. Then, synchronization with the input of the photographed image taken at the start time of the next photography from the video clip holding unit 102 is performed.

Specifically, the synchronization control unit 203 receives the captured image signal with the actual time input from the image capturing apparatus 101 to the video clip holding unit 102 and the clip end point in response to a recapture start instruction (capture resumption instruction) by the photographer. The clip end image signal held in the image holding unit 201 is synchronized with the two inputs. The two inputs are sent to the blend unit 202.

The blend unit 202 includes a blend execution unit 2021 and a frame buffer 2022, and blends (superimposes) the input image to generate image data. Specifically, when resuming shooting (starting the next shooting), the blending unit 202 uses the clip end image input in synchronization by the synchronization control unit 203 and the shot image shot at the shooting restart point, Image data is generated by blending according to the control information. That is, the blending unit 202 receives the clip end image and the shot image shot at the start of the next shooting, and inputs the clip end image and the shot image shot at the start of the next shooting. Blend to generate image data.

The blend execution unit 2021 is controlled by the control information output from the blend control unit 204, and blends the two inputs according to the weighted superimposition ratio (value α indicating the blend ratio) indicated in the control information to generate image data. Is generated. The blend execution unit 2021 stores the generated image data in the frame buffer 2022. Specifically, as shown in FIG. 3, the blend execution unit 2021 converts the captured image held by the video clip holding unit 102 and the clip end image held by the clip end image holding unit 201 to α: (1- Mix at a ratio of α), that is, blend (superimpose).

Here, for example, the value α indicating the blend ratio takes a value from 0 to 1. When the blend ratio indicates a value of 0, the blend execution unit 2021 blends only the clip end image held in the clip end image holding unit 201 out of the two inputs, and blends the image data of only the clip end image. Generated as later video information. On the other hand, when the blend ratio indicates a value of 1, only the captured image held by the video clip holding unit 102 is blended, and image data of only the captured image is generated as the video information after blending. In this way, the value α indicating the blend ratio is determined by the value of α that can be taken when starting the next shooting (resuming the shooting), the clip end image, and the shot image shot at the next shooting start time. The ratio of blending is determined.

It should be noted that the value α indicating the blend ratio may be initialized to 0 by a re-shooting start instruction from the photographer, and the value α indicating the blend ratio is initialized to 0 by a shooting stop instruction from the photographer. It may be done.

Also, in the first shooting start instruction such as after the power of the shooting apparatus 101 is turned on, blending is not executed, that is, the value α indicating the blend ratio may be set to 1. Also, at the time of system boot when the power of the photographing apparatus 101 is turned on, instead of setting the value α indicating the blend ratio to 1, white data separately prepared in advance by the clip end image holding unit 201 using ROM data or the like And / or a fixed image such as black may be blended.

In this way, the clip end image can be blended in a stepwise manner with the shot image taken at the next start of shooting.

Note that the value α indicating the blend ratio may be changed based on, for example, control information including a code amount. In that case, by controlling the rate of change in the blend ratio, it is possible to suppress the generation of an unnecessary code amount from the relevance of the image, and to further suppress the buffer failure during seamless reproduction.

The frame buffer 2022 stores the image data generated by the blend execution unit 2021. The stored image data is read out by the encoding processing unit 103 during the encoding process.

The blend control unit 204 controls the blend unit 202 with control information. Specifically, the blend control unit 204 determines the control information based on the code amount calculated by the code amount control unit 1031 and the image correlation between the clip end image and the captured image immediately before the clip end image. decide. Then, according to the determined control information, the blend unit 202 changes the ratio of blending the clip end image and the photographed image taken at the next photographing start time to generate image data.

Note that the blend control unit 204 may determine control information based on a code amount preset in the information processing apparatus 100, for example. Further, the blend control unit 204, for example, when the clip end image holding unit 201 holds the clip end image, the control information is obtained based on the code amount calculated by the code amount control unit 1031 of the clip end image. It may be decided. Also, the blend control unit 204 uses the information on the current code amount calculated by the code amount control unit 1031, for example, to the blend unit 202, and the captured image captured at the time when the next capturing starts. The image data may be generated by changing the blend ratio.

The encoding processing unit 103 includes a code amount control unit 1031, a quantization unit 1032, and an encoding unit 1033. The encoding processing unit 103 encodes the image data generated by the blend unit 202 and encodes it based on the control information. Controls the code amount of image data.

The code amount control unit 1031 controls the code amount of the image data encoded by the encoding processing unit 103 based on the control information. Specifically, the code amount control unit 1031 controls the code amount encoded by the encoding unit 1033 according to the control information at the start of the next shooting, and from the start of shooting of the shooting apparatus 101 to the stop thereof. The amount of code encoded by the encoding unit 1033 is controlled for each encoding processing unit without depending on the control information.

The encoding unit 1033 encodes the image data generated by the blending unit 202. Specifically, the encoding unit 1033 encodes the image data generated by the blend unit 202 with the code amount controlled by the code amount control unit 1031 at the start of the next imaging, and the imaging device 101 performs imaging. From the start to the stop, the blend unit is obtained by blending only a plurality of photographed images taken after the start of the next photographing input to the blend unit 202 with the code amount controlled by the code amount control unit 1031. The image data generated in 202 is encoded.

Here, as described above, the code amount may be a small code amount set in advance, or may be a code amount obtained by the code amount control unit 1031 calculating the code amount of the clip end image in advance. Also good.

The stream buffer 104 temporarily stores a plurality of video clips encoded in a format that can be recorded on the recording medium 105 by the encoding processing unit 103. The plurality of encoded video clips temporarily stored are recorded on the recording medium 105.

The information processing apparatus 100 configured as described above includes a video clip holding unit 102 that holds a shot image signal input from the shooting apparatus 101, a clip end image holding unit 201 that holds a clip end image signal, and a clip end point. Blend unit 202 having as input the clip end image signal of image holding unit 201 and the captured image signal of video clip holding unit 102, synchronization control unit 203 that synchronizes the two inputs, and blend that controls blend unit 202 The encoding unit 103 includes a control unit 204 and an encoding processing unit 103. The encoding processing unit 103 includes a code amount control unit 1031 that controls the code amount based on control information of the blend control unit 204, and then starts shooting ( It is characterized by recording the image data (video) generated by blending the captured image and the clip end image at the time of shooting resumption) . With this configuration, seamless reproduction can be easily realized at a low cost and a high video effect can be obtained even at a point where the correlation of images such as a scene change is low.

Note that video (moving images) is a video recording method that is continuous with the time axis, not the moment, so the photographer can take multiple shot images (video clips) from the start of shooting (recording start) to the stop of shooting. Recording is performed while repeating the start and stop operations of shooting. Here, by assuming that the clip end image is video information of one frame, 1 GOP or 1 GOP or more after the shooting stop instruction of the photographer, a plurality of shot images (video clips) up to the shooting stop instruction of the photographer are stored on the hard disk. Alternatively, it may be stored in a storage medium such as a semiconductor memory. Thereby, it is possible to hold an image for blending for obtaining the effect of the present invention after sufficiently recording information as intended by the photographer.

FIG. 4 is a diagram for explaining a captured image information processing method that enables seamless playback of the information processing apparatus according to the present invention.

For convenience of explanation, the photographed image (image data) until being encoded and stored in the recording medium 105 according to the photographing stop instruction by the photographer is the (n−1) th (n is a natural number), the photographing start instruction (shooting restart instruction) The clip end image to be superimposed is set to the nth (n is a natural number) of the captured image (image data).

First, as shown in FIG. 4A, the video clip holding unit 102 has a processing unit sufficient to encode the end image information at the timing when the photographing apparatus is instructed to stop photographing by the photographer. Image information, that is, a video clip up to a photographed image (n-1) photographed by the photographing apparatus 101 and a photographed image (n) which is image information of the next one frame are held.

Further, as shown in FIG. 4B, the clip end image holding unit 201 is configured so that the video clip holding unit 102 takes the shot image (n−1) at the timing when the shooting is instructed to the shooting apparatus 101 by the photographer. After that, the image information of the next one frame, that is, the captured image (n) is held as the clip end image.

Here, (c) of FIG. 4 shows that the video clip holding unit 102 has a plurality of images captured by the imaging device 101 at the timing when the next imaging start instruction (imaging restart instruction) is given to the imaging device 101 by the photographer. A photographed image (video clip to be held from now on) is shown.

The blend unit 202 synchronizes and inputs the captured image (when the photographer next instructs the photographing apparatus 101 to perform photographing start (shooting restart instruction) when the photographing is started next (photographing restart instruction). The clip end image of n) and the shot image shot at the shooting start time (shooting restart time) are blended according to the value α indicating the blend ratio indicated in the control information, as shown in FIG. Then, the encoding processing unit 103 generates image data as shown in FIG.

In this way, the information processing apparatus 100 can correlate images with respect to parts such as scene changes.

For the sake of simplicity, the case where the clip end image (clip end image signal) is one frame of still image information immediately after the shooting stop instruction has been described. However, the present invention is not limited to this. An integer multiple of 1 GOP, which is a processing unit of a plurality of captured images for a time equal to or longer than a minimum unit (1 GOP) composed of a plurality of pictures in which a clip end image (clip end image signal) has a reference relationship with the clip end image Video information. Next, the case will be described.

FIG. 5 is a diagram for explaining an information processing method for captured images that enables seamless reproduction of the information processing apparatus according to the present invention. FIG. 5 shows a case where the clip end image (signal) is video information that is an integral multiple of 1 GOP. That is, a video in which a clip end image (clip end image signal) maintains a correlation between frames (captured images) in units of GOPs, which are a plurality of pictures as a minimum unit that establishes a reference relationship necessary for image compression processing. The case of information is shown. In the following, for convenience of explanation, a plurality of photographed images that have been encoded and stored in the recording medium 105 in accordance with a photographing stop instruction from a photographer are represented as n−1th (n is a natural number) of GOP (GOP data). Then, the clip end image, which is the moving image information of an integral multiple of 1 GOP, to be superimposed by the next shooting start instruction (shooting restart instruction) is set to the nth (n is a natural number) of the GOP (GOP data).

First, as shown in FIG. 5A, the video clip holding unit 102 has a processing unit sufficient to encode the end image information at the timing when the photographing apparatus is instructed to stop the photographing apparatus 101. Image information, that is, a plurality of photographed images (video clips) up to GOP (n−1) photographed by the photographing apparatus 101 and GOP (n) which is the next GOP are held.

In addition, as shown in FIG. 5B, the clip end image holding unit 201 causes the video clip holding unit 102 to set GOP (n−1) at the timing when the shooting is instructed to the shooting apparatus 101 by the photographer. After holding, the next GOP, that is, GOP (n) is held as the clip end image.

Here, (c) of FIG. 5 shows that the video clip holding unit 102 has a plurality of images captured by the imaging apparatus 101 at the timing when the next imaging start instruction (imaging restart instruction) is given to the imaging apparatus 101 by the photographer. A photographed image (video clip to be held from now on) is shown.

The blending unit 202 synchronizes and inputs the GOP (n) when the next photographing is started, that is, at the timing when the photographer gives the photographing device 101 a next photographing start instruction (shooting restart instruction). ) And the shot end image taken at the next shooting start time (shooting restart time), as shown in FIG. 5D, are blended according to the value α indicating the blend ratio indicated in the control information. . Then, the encoding processing unit 103 generates GOP data (image data) as shown in FIG.

In this way, the information processing apparatus 100 can have image correlation with respect to a part such as a scene change by being blended from the nth GOP and encoded later.

Next, an information processing method of the imaging system configured as described above will be described. FIG. 6 is a flowchart for explaining the information processing method of the imaging system according to the present invention.

First, at the start of shooting, as described above, for example, when the blending function is valid, such as at the start of the next shooting (when shooting is resumed) (YES in S101), the initial blend ratio is set. Based on the value (S102), the blending unit 202 starts blending, and the encoding processing unit 103 starts encoding (S103). Here, for example, the initial value of the set blend ratio is 0 (α = 0).

Next, it is determined whether or not the blending process is finished (S104).

If the blending process is not finished, that is, the blending process is in progress (NO in S104), the blend amount is changed until the blending process is finished at the maximum value of the blending ratio (for example, the value α = 1 indicating the blending ratio) ( For example, the blending ratio α is added) (S105). Here, changing the blend amount means changing the blend ratio by adding or subtracting the value α indicating the blend ratio in increments of 0.1, for example.

Then, since there is a possibility that the photographing stop instruction is given by the photographer, the photographing stop (recording stop) instruction is monitored, that is, whether or not the photographing is finished (S106). If a shooting stop instruction is issued by the photographer (YES in S106), processing sufficient for the video clip holding unit 102 to encode the end image information at the timing when the shooting instruction is given to the shooting apparatus 101 by the photographer. After holding the image information for the unit, the next one frame of image information is held as a clip end image.

On the other hand, when the blending process is completed, that is, the photographing is continued and the blending process is completed (YES in S104), the photographing stop instruction by the photographer is further monitored in the same manner as described above (S106). (YES in S106), after the video clip holding unit 102 holds image information for a processing unit sufficient to encode the above-described end image information at the timing when the photographing apparatus 101 instructs the photographing apparatus 101 to stop. Then, the image information of the next one frame is held as a clip end image (S107). Here, the case where the blending process is completed means a case where the value α = 1 indicating the blending ratio is reached.

The blending process can be performed by the above series of processing flows.

As described above, according to the present invention, it is possible to easily and appropriately control the amount of generated code in an encoding process even at a scene change location, and to realize an information processing apparatus and information processing method that can be easily seamlessly reproduced. Can do.

Specifically, in the camera work by the photographer, the correlation between image contents may be low between adjacent pictures, such as a scene change. There is a conventional method for assigning the generated code amount of the current picture based on the generated code amount of the past picture to such a moving image using the correlation between the pictures. However, the amount of generated codes to be allocated increases unnecessarily, and the virtual buffer for code amount smoothing may overflow, which is one factor that makes seamless reproduction extremely difficult. Further, the CC = 5 connection method of the AVCHD standard as means for solving this, that is, the code of the image after the scene change based on the coding amount of the image before the scene change and the buffer remaining amount at the time of decoding (decoding) Even if the amount of code is limited so that the buffer does not overflow during playback, the amount of code cannot be sufficiently allocated for the amount of code originally intended to be allocated, resulting in fatal image quality degradation. There was a risk of inviting.

On the other hand, in the present invention, at the recording resumption position (the next shooting start position) after the recording is stopped (after the shooting is stopped), between the clip end image and the video clip at the time of the recording restart (the next shooting start). A clip end image relative to the beginning of the video clip at the time of resuming recording (starting the next shooting), that is, the shot image after the shooting is stopped and shot at the time when the shooting is resumed. And superimpose (blend) video (image) with high correlation. Thereby, not only the unnecessary code amount allocation is unnecessary, but also a sufficient image quality can be reproduced with a small code amount.

Also, the encoding processing unit 103 includes a code amount control unit 1031 that controls the code amount based on the control information of the blend control unit 204. In the encoding processing unit 103, when encoding the image data that has been blended and generated immediately after the recording resumption instruction (shooting resumption instruction), the code amount control unit 1031 specifies, for example, a small code amount set in advance. Thus, the encoding unit 1033 performs encoding processing. Thereby, not only there is no buffer overflow at the time of reproduction, but also seamless reproduction can be realized without causing image quality degradation. When the clip end image holding unit 201 holds the clip end image, the code amount control unit 1031 may calculate the code amount of the clip end image in advance and use it as the code amount after the recording restart instruction. good.

Also, according to the information processing apparatus and information processing method of the present invention, it is possible to avoid the discontinuity of the video information by maintaining the correlation between the video clips. It becomes unnecessary to be aware of the processing on the code side. Accordingly, seamless reproduction can be realized easily at a low cost.

Furthermore, according to the information processing apparatus and the information processing method of the present invention, in seamless playback between video clips, editing using an expensive editing device is unnecessary, and a video effect with a high crossfade is obtained at the same time as shooting is completed. Can be provided. Further, the effect can be obtained without changing the basic operation flow performed when the photographer shoots a moving image. Furthermore, it can be realized by a combination of the functions that are the basis of the moving image photographing device, and has an effect that it can be obtained relatively easily, inexpensively, and extremely high effects.

In addition, in the image data of the video clip obtained by the information processing apparatus and the information processing method of the present invention, a management unit that manages the head position immediately after the image data of the blended part (joined part) as a playlist of the AVCHD standard And the management unit may select the head position by managing the time information of the head positions.

As described above, the information processing apparatus and the information processing method of the present invention have been described based on the embodiment, but the present invention is not limited to this embodiment. Unless it deviates from the meaning of this invention, the form which carried out the various deformation | transformation which those skilled in the art can think to this embodiment, and the structure constructed | assembled combining the component in different embodiment is also contained in the scope of the present invention. . In addition, an imaging apparatus equipped with the above-described information processing apparatus is also included in the scope of the present invention.

This imaging device is, for example, a camera (digital still camera) as shown in FIG. 7 or a camcorder as shown in FIG. 8, and captures a desired scene by repeating recording and pause (shooting start and shooting stop). Can be recorded.

INDUSTRIAL APPLICABILITY The present invention can be used for an information processing device and an information processing method, and is particularly used for home photographing using a photographing device such as a camera or a camcorder, and is desired by repeatedly recording and pausing (starting photographing and stopping photographing). It can be used in an information processing apparatus and an information processing method for photographing and recording a scene.

100, 900

Information processing device

101, 901 Image capturing device 102 Video clip holding unit 103

Encoding processing unit

104, 904

Stream buffer

105, 905 Recording medium 201 Clip end image holding unit 202 Blending unit 203 Synchronization control unit 204

Blend control unit

902, 907, 2022 Frame buffer 903 Encoder 906 Decoder 908 Display device 1031 Code amount control unit 1032 Quantization unit 1033 Encoding unit 2021 Blend execution unit

Claims

An information processing apparatus that encodes and records a plurality of video clips shot by repeating the start and stop of shooting a plurality of times in a shooting device,
A video holding unit for holding a video clip composed of a plurality of shot images shot by the shooting device;
Of one video clip obtained from the start to the stop of shooting of the shooting device, a terminal image holding unit that holds a terminal shot image as a clip terminal image;
The clip end image and a shot image taken at the start of the next shooting are input, and the input clip end image and the input shot image shot at the start of the next shooting are input. A blending unit that blends to generate image data;
A synchronization control unit that synchronizes the input of the clip end image from the end image holding unit and the input of the shot image shot at the start of the next shooting from the video holding unit;
A blend control unit for controlling the blend unit by control information;
An information processing apparatus comprising: an encoding processing unit that encodes image data generated by the blending unit and controls a code amount of image data to be encoded based on the control information.
The encoding processing unit includes:
An encoding unit for encoding the image data generated by the blend unit;
A code amount control unit that controls a code amount of image data encoded by the encoding unit based on the control information;
At the start of the next shooting,
The code amount control unit controls a code amount encoded by the encoding unit according to the control information, and the encoding unit is generated by the blend unit with a code amount controlled by the code amount control unit. Encoded image data,
From the start to the stop of shooting of the shooting device,
The code amount control unit controls the code amount encoded by the encoding unit for each encoding processing unit without relying on the control information, and the encoding unit is controlled by the code amount control unit. The image data generated by the blending unit is encoded by blending only a plurality of captured images captured after the start of the next imaging input to the blending unit with the code amount thus obtained. The information processing apparatus described.
The information processing apparatus according to claim 1, wherein the blend control unit determines the control information based on a code amount preset in the information processing apparatus.
The encoding processing unit calculates a code amount of the clip end image when the end image holding unit holds the clip end image,
The information processing apparatus according to claim 1, wherein the blend control unit determines the control information based on a code amount calculated by the encoding processing unit.
The blend control unit uses the information on the current code amount calculated by the encoding processing unit to blend the clip end image and the shot image shot at the start of the next shooting in the blend unit. The information processing apparatus according to any one of claims 1 to 4, wherein the image data is generated by changing a ratio to be generated.
The blend control unit determines the control information based on the code amount calculated by the encoding processing unit, the clip end image, and the image correlation of the photographed image immediately before the clip end image. And
The image data is generated by changing the ratio of blending the clip end image and the captured image captured at the start time of the next capturing according to the determined control information. The information processing apparatus according to 1 or 2.
The clip end image is a final photographed image of video clips composed of a plurality of photographed images obtained from the start to the stop of photographing by the photographing device, and is a still image for one frame. 7. The information processing apparatus according to any one of 1 to 6.
The clip end image is a minimum unit (1 GOP) composed of pictures having a reference relationship with the clip end image, out of video clips composed of a plurality of shot images obtained from the start to the stop of shooting by the shooting apparatus. The information processing apparatus according to any one of claims 1 to 7, wherein the information is a plurality of photographed images for the above time.
The information processing apparatus further includes:
9. A management unit that manages a head position that is a joint portion between the image data generated by the blending process at the start time of the photographing by the blending unit and the image data after the image data. The information processing apparatus according to item 1.
An imaging apparatus equipped with the information processing apparatus according to any one of claims 1 to 9.
An information processing method for encoding and recording a plurality of video clips shot by repeating the start and stop of shooting a plurality of times in a shooting device,
A video holding step for holding a video clip composed of a plurality of shot images shot by the shooting device;
Of one video clip obtained from the start to the stop of shooting by the shooting device, a terminal image holding step for holding a terminal shot image as a clip terminal image;
The clip end image and a shot image taken at the start of the next shooting are input, and the input clip end image and the shot image shot at the start of the next shooting are blended. A blending step for generating image data;
A synchronization control step for synchronizing the input of the clip end image and the input of the photographed image photographed at the start of the next photographing;
An encoding processing step that encodes the image data generated in the blending step and controls a code amount of the image data encoded based on the control information.