JP4158463B2 - Imaging device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image pickup apparatus that compresses and encodes a digital image captured from an image pickup element and a recording apparatus using the same.
[0002]
[Prior art]
Video camera devices that digitize and record images include zoom lenses, CCDs, image signal processing units, still image compression / decompression units, video compression / decompression units, microphones, speakers, audio compression / decompression units, control units (CPU) , LCD monitor, built-in flash memory, PC card type hard disk drive, and USB interface. The zoom lens constitutes an imaging optical system, and an optical image formed by the zoom lens is photoelectrically converted by a CCD as a solid-state imaging device.
[0003]
The imaging signal obtained by this photoelectric conversion is converted into a digital signal by the imaging signal processing unit, and after undergoing necessary image processing, it is sent to the image compression / decompression unit and recorded on a recording medium such as a flash memory or a hard disk drive. (See, for example, Patent Document 1).
[0004]
[Patent Document 1]
Japanese Patent Laid-Open No. 2001-352471
[Problems to be solved by the invention]
For example, in a recording apparatus that can record an input image by moving image encoding / still image encoding, the encoding format and resolution differ when, for example, it is desired to take a still image temporarily during moving image shooting. To temporarily encode an image, a method having two image encoding devices is conceivable. However, having two image encoding devices causes problems such as cost increase, circuit scale increase, and circuit complexity.
[0006]
In order to satisfy the above requirements with a single image encoding device, when still images are encoded, the moving image encoding is temporarily stopped, the still image is compressed and encoded, and the moving image encoding is performed again. A method of restarting can be considered.
[0007]
However, since the moving image encoding is temporarily stopped and restarted, the generated stream is not temporally continuous, and is shortened by a time shift of the still image encoding process. In addition, the generated streams are separated for stopping / resuming, which is inconvenient for file management, and power consumption and data amount increase caused by stopping / resuming cannot be ignored. Further, there is a problem that much time is required for the stop / restart process.
[0008]
The same problem as described above can be considered when an image input from a temporarily different camera unit is temporarily encoded.
[0009]
The present invention has been made in view of the above circumstances. For example, when a still image is desired to be shot during moving image shooting, for example, an image having a different encoding format or resolution, or input from a temporarily different camera unit is used. An object of the present invention is to provide an encoding method and an imaging apparatus capable of capturing a first image encoded stream without interruption when an image or the like is temporarily encoded.
[0010]
[Means for Solving the Problems]
In view of the above-described problems, the present invention includes, for example, the configuration described in the claims.
[0011]
The imaging apparatus of the present invention includes a camera unit that captures an image and generates an image signal, and an encoding unit that encodes the image signal output from the camera unit to generate an image encoded stream including a plurality of pictures. And a skip code inserting means for inserting a skip code that is inserted into the picture and that causes the picture to be skipped to another picture at the time of decoding. The encoding means encodes the first image output from the camera unit. During the generation of the image encoded stream, the encoding of the first image is temporarily stopped and the second image output from the camera unit is encoded. Inserts a skip code into the corresponding picture of the first image while the encoding of the first image is stopped. The skip code insertion means may be provided in a multiplexing circuit that multiplexes encoded image data and encoded audio data, or may be provided as a skip code insertion circuit independent of the multiplexing circuit.
[0012]
Further, in another embodiment of the imaging apparatus of the present invention, the image pickup apparatus further includes a control unit that controls switching between the first image and the second image encoded by the encoding unit, and the switching is performed by a user or a program. Follow the instructions.
[0013]
Moreover, in another embodiment of the imaging apparatus of the present invention, the encoding means further performs encoding of the first image and encoding of the second image in different encoding formats.
[0014]
Further, in another embodiment of the imaging apparatus of the present invention, it further has two camera units.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. First, the first embodiment will be described. FIG. 1 is a block diagram of the recording apparatus of the present invention. The analog signal captured by the lens 10 is photoelectrically converted by the CCD sensor 11. An imaging signal obtained by this photoelectric conversion is converted into a digital image signal by A / D conversion 12. The digital image is compressed and encoded by an image encoding circuit 13 capable of encoding a still image and a moving image, and a moving image is multiplexed with audio encoded data by a multiplexing circuit 17 to be recorded in a recording unit 20. Save to.
[0016]
The still image is stored in the recording unit 20 with the JPEG header added by the JPEG header adding unit 19. Also, the audio signal captured by the microphone 14 is converted into a digital signal by the A / D converter 15, the audio encoding circuit 16 performs audio compression encoding, and the multiplexing circuit 17 multiplexes it with the moving image encoded data. And stored in the recording unit 20.
[0017]
Here, the recording unit is assumed to be a recording device such as an HDD or a DVD-RAM drive that records encoded data on a recording medium such as HD, memory, DVD, etc. Can have. In addition, it is conceivable to transfer moving image / still image encoded data through another route / same route as a transfer route to the recording unit. This figure is an example when moving image / still image encoded data is recorded in the same recording unit by another path. Further, the recording apparatus in the present embodiment can be used as an imaging apparatus except for the recording unit.
[0018]
When performing still image encoding temporarily during moving image encoding as shown in FIG. 2, the control unit switches between moving image encoding and still image encoding. A skip code is inserted into the moving image encoded stream by a multiplexing circuit. Note that a skip code insertion circuit may be provided separately and the skip code insertion circuit may insert the skip code.
[0019]
Switching between still image and moving image encoding can be performed by user designation or time designation. When switching is designated, the control circuit switches between still images and moving images.
[0020]
Here, the skip code is a code that becomes the same picture as the picture to be referred to when decoding an image, and is inserted in units of pictures. The skip code insertion position is a position that can be uniquely determined when the image is decoded.
[0021]
An example will be described. The encoded data structure for encoding each picture is determined as follows on the encoding side and the decoding side. First, when encoding a picture, a picture start code is set, which is the start of the encoded data of this picture. This code is data that never occurs except for the Picture start code. After that, a skip code for determining whether or not to skip a picture is given to each picture. When decoding, it is possible to determine whether each picture is skipped by first detecting the picture start code and looking at the subsequent skip code. It can be realized by setting as above.
[0022]
Next, an implementation example in MPEG2, which is an international standard system for moving image coding, will be described. The skip code in MPEG2 is a picture skip in MBAI (Macroblock Address Increment), MBE (Macroblock Escape), or Low Delay mode. MBAI and MBE are codes that are the same as the Picture that is referenced when decoding an image. In the picture skip in the low delay mode, the display order (Temporal reference) of the images described in the stream is skipped, so that playback can be performed in a time equivalent to the actual recording time at the time of decoding.
[0023]
First, explanations are given when MBAI or MBE is used. When performing video encoding, I-Picture is encoded using only its own Picture information for each Picture unit, and its own Picture is predicted using the past Picture information as a reference Picture. The P-Picture to be encoded and the past and future Pictures as reference pictures are encoded into B-Pictures that are encoded by predicting their own pictures. In this method, skip codes cannot be inserted into I-Pictures in MPEG2, so skip codes are not inserted into I-Pictures, and skip codes are inserted into P-Pictures and B-Pictures.
[0024]
FIG. 3 shows an example in which a skip code is inserted in the P-Picture format. The upper part of FIG. 3 represents moving picture coding. The input in FIG. 3 represents images input from the camera unit in time order, and I, B, and P represent moving image encoding formats, respectively, and are encoded in that format when encoding. The subscripts 0 to 8 represent the temporal order. The encoding of FIG. 3 represents the thing which rearranged the input image in the encoding order. The skip code is inserted in picture units.
[0025]
PS in FIG. 3 means that a skip code is inserted in the P-Picture format. The lower part of FIG. 3 shows still picture coding. When the skip code is inserted into the moving picture stream, it is inserted in the P-Picture format before the P-Picture as shown in FIG. 3 (encoding in the figure: PS between B1 and P5). An example of decoding and displaying a stream generated by this method is shown in FIG. At this time, the skip code (PS) inserted in the P-Picture format is the same image as the reference image I2 at the time of decoding. By doing so, it is possible to maintain temporal continuity without the encoded stream being interrupted.
[0026]
Similarly, FIG. 5 shows a case where a skip code is inserted in the B-Picture format. The configuration of FIG. 5 is the same as that of FIG. In this case, the skip code is inserted in front of the P-Picture in the B-Picture format (BS in the figure). At this time, the B-Picture has a mode of referring to a future picture, a past picture, or both pictures. In this case, the B-Picture is set to a mode that refers to a past picture. FIG. 6 shows an example of decoding / display. From this, the validity of this method can be confirmed.
[0027]
Next, the insertion position of the skip code to be inserted into P-Picture and B-Picture will be described. FIG. 7 shows the MPEG2 stream structure. MPEG2 includes a sequence layer, a GOP layer, a picture layer, a slice layer, a macroblock layer, and a block layer for each stream. Since the skip code is located at the head of the macroblock information in FIG. 7, it is inserted at that position. In this case, skip codes cannot be added to the beginning and end macroblocks of the slice according to the MPEG2 standard. For such macroblocks, no motion information is predicted in the inter-frame coding mode. Add information with zero error. This makes it possible to display the same image as the reference image when decoding, as with the skip code. By the way, I, P and B-Picture formats are specified in the picture header.
[0028]
Next, a method for performing picture skip in the low delay mode will be described. In the low delay mode, the picture structure for encoding is only I-Picture and P-Picture. FIG. 12 shows a simple picture structure. "Input" I and P indicate I-Picture and P-Picture, respectively, and the numerical suffix indicates Temporal reference. As shown in FIG. 2, when switching between a moving image and a still image, moving image encoding is stopped and still image encoding is performed. At that time, a temporal reference is added to the moving image encoded stream in consideration of the stopped period. FIG. 12 shows an example in which still image encoding is performed between P5 and P7, and moving image encoding is newly restarted from P8. In this case, P5 to P7 that have stopped moving picture encoding are skipped, and when moving picture encoding is resumed, encoding is resumed with Temporal reference set to 8. In this way, playback can be performed in a time equivalent to the recording time by referring to the Temporal reference when decoding.
[0029]
A second embodiment of the invention will be described below with reference to the drawings. FIG. 8 is a block diagram of the recording apparatus of the present invention. The analog signal captured by the lens 10 is photoelectrically converted by the CCD 11. An imaging signal obtained by this photoelectric conversion is converted into a digital image signal by the A / D 12.
[0030]
The digital image is image-encoded by the image encoding circuit 13, and the audio encoded data is multiplexed by the multiplexing circuit 17 and stored in the recording unit 20. Also, the voice data input from the microphone 14 is digitized by the A / D 15, voice coded by the voice coding circuit 16, multiplexed with the moving image coded data by the multiplexing circuit 17, and stored in the recording unit 20. This recording unit is the same as in the first embodiment. Further, in this apparatus, a switch circuit 22 is provided in front of the moving image encoding circuit, and an input image can be switched by designation of the control circuit 18. The skip insertion circuit 21 can insert a skip code similar to that of the first embodiment into the moving image encoded data. It is also possible to insert a skip code by a multiplex circuit by providing the multiplex circuit with a skip code insertion function.
[0031]
As shown in FIG. 10, when a telephoto image (image input to the first lens) is encoded, a wide-angle image (image input to the second lens) is encoded by the control circuit. The switch circuit is switched to start wide-angle image encoding. The wide-angle image is multiplexed with audio and recorded in the recording unit. For telephoto images, a skip code is added by a skip insertion circuit, multiplexed with audio by a multiplexing circuit, and recorded in a recording unit. It is also possible to insert a skip code by a multiplex circuit by providing the multiplex circuit with a skip code insertion function.
[0032]
The skip code represents the same skip code as in the first embodiment. Further, when encoding of the telephoto image is selected by the control circuit, encoding of the wide-angle image is stopped and encoding of the telephoto image is restarted.
[0033]
Also, it is possible to control by the control circuit whether to insert a skip code into the wide-angle image encoded data or to insert a skip code into the telephoto image encoded data.
[0034]
FIG. 9 shows an apparatus that can insert a skip code into both the telephoto image (image input to the first lens) and wide-angle image (image input to the second lens) encoded data. . Accordingly, it is possible to insert a skip code into the wide-angle image at the time of telephoto image encoding, and to insert a skip code into the telephoto image encoded data at the time of wide-angle image encoding. The above input image switching and skip code insertion are controlled by user designation or time designation.
[0035]
A third embodiment of the invention will be described below with reference to the drawings. FIG. 11 is a block diagram of the recording apparatus of the present invention. The analog signal captured by the lens 10 is photoelectrically converted by the CCD 11. An imaging signal obtained by this photoelectric conversion is converted into a digital image signal by the A / D 12. The moving image encoding circuit 13 performs moving image encoding, and the multiplexing circuit 17 multiplexes it with the audio encoded data and stores it in the recording unit 20. The audio data input from the microphone 14 is digitized by the A / D 15, encoded by the audio encoding circuit 17, multiplexed with the moving image encoded data by the multiplexing circuit, and stored in the recording unit. Here, the recording unit is the same as the first example. The switch circuit 22 can switch the image input to the image encoding circuit to an image designated by the control circuit 18. Accordingly, it is possible to encode a telephoto / wide-angle image with a single encoding circuit, and to generate a stream that does not shift in time and does not drop.
[0036]
【The invention's effect】
The present invention can encode a plurality of images without interrupting the encoded stream.
[Brief description of the drawings]
FIG. 1 is a block diagram of a first embodiment.
FIG. 2 is a conceptual diagram of moving image / still image switching.
[Figure 3] P-Picture skip code insertion position.
FIG. 4 shows the display order after decoding.
[Fig. 5] B-Picture skip code insertion position.
FIG. 6 shows the display order after decoding.
FIG. 7: Skip code insertion position.
FIG. 8 is a block diagram of a second embodiment.
FIG. 9 is a block diagram of a second embodiment.
FIG. 10 is a conceptual diagram of telephoto / wide-angle image switching.
FIG. 11 is a block diagram of a third embodiment.
FIG. 12 is a conceptual diagram of the first embodiment.
[Explanation of symbols]
10 ... Lens 11 ... CCD
12 ... A / D
13 ... Image encoding circuit 14 ... Microphone 15 ... A / D
16 ... Audio encoding circuit 17 ... Multiplexing circuit 18 ... Control circuit 19 ... JPEG header addition circuit 20 ... Recording unit 21 ... Skip insertion circuit 22 ... Switch circuit

Claims (10)

A camera unit that captures an image and generates an image signal;
Encoding means for encoding an image signal output from the camera unit to generate an image encoded stream including a plurality of pictures;
Skip code insertion means for inserting a skip code, which is a code inserted into the picture and generating the picture as a copy of the whole other picture at the time of decoding,
The encoding means temporarily stops the encoding of the moving image and outputs it from the camera unit while encoding the moving image output from the camera unit and generating the image encoded stream. A still image
The skip code insertion means corresponds to the time when the moving picture is being encoded with respect to the picture of the moving picture corresponding to the time when the encoding means stops encoding the moving picture. An imaging apparatus, wherein the skip code is inserted so that a copy of the entire picture is generated.   The imaging apparatus according to claim 1, further comprising a control unit that controls switching between the moving image and the still image encoded by the encoding unit.   3. The imaging apparatus according to claim 1, wherein the encoding unit performs encoding of the moving image and encoding of the still image in different encoding formats.   The imaging apparatus according to claim 1, wherein the encoding unit converts each picture of a moving image to be encoded into an I-picture that is encoded using only information of the picture itself, and a past picture. The P-Picture is encoded using the prediction, and the B-Picture is encoded using the past and future pictures. The skip code inserting means converts the skip code into the P-Picture. An imaging apparatus characterized by being inserted in the form of   The imaging apparatus according to claim 1, wherein the encoding unit converts each picture of a moving image to be encoded into an I-picture that is encoded using only information of the picture itself, and a past picture. The P-Picture is encoded using the prediction, and the B-Picture is encoded using the past and future pictures. The skip code inserting means converts the skip code into the B-Picture. An imaging apparatus characterized by being inserted in the form of   6. The imaging apparatus according to claim 1, wherein said encoding means encodes an image in MPEG format, and said skip code inserting means has zero motion information for the first and last macroblocks of slice information. And an information with no prediction error, and the skip code is inserted into a macroblock constituting the picture to be skipped. In an imaging device that encodes images taken from at least two different camera units,
Switch means for switching input images ;
Encoding means for encoding an image signal output from the camera unit to generate an image encoded stream including a plurality of pictures;
Skip code insertion means for inserting a skip code that causes the picture to be generated as a copy of the entire other picture at the time of decoding ;
Control means for controlling switching of the switch means and controlling insertion of the skip code ;
With
The encoding means temporarily stops the encoding of the moving image and outputs it from the camera unit while encoding the moving image output from the camera unit and generating the image encoded stream. The skip code insertion unit executes encoding of the moving image with respect to a corresponding picture of the moving image while the encoding unit stops encoding of the moving image. An image pickup apparatus, wherein the skip code is inserted so that a copy of the entire picture corresponding to the generated picture is generated. The imaging apparatus according to any one of claims 1 to 7,
Comprising a change-over switch means for accepting a user instruction,
An imaging apparatus, wherein the change-over switch means stops encoding the moving image and encodes the still image according to a user instruction. The imaging apparatus according to any one of claims 1 to 7,
Provided with changeover switch means for accepting program instructions,
An image pickup apparatus, wherein the change-over switch means stops the encoding of the moving image and encodes the still image according to a program instruction.   10. The imaging apparatus according to claim 1, further comprising a recording unit that records and saves an encoded image on a recording medium.

Images