JP2009033586A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance encoding efficiency during photographing, when periodically repeating a plurality of different photographic conditions. <P>SOLUTION: An imaging device (12) outputs a frame image at a second frame rate which is equivalent to an n-fold first frame rate, while periodically repeating (n) kinds of different photographic conditions at the first frame rate. The frame distribution section (18) of a coding section (16) distributes frame images from a camera signal processing section (14) to data compression circuits (22-28), according to the photographic conditions so as to supply frame images on the same photographic conditions, to the same data compression circuits. Each of the data compression circuits (22-28) regards input frame images as moving images at the first frame rate and performs compression coding thereon. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, and more specifically to an imaging apparatus that records a moving image while periodically repeating a plurality of different imaging conditions.

A technique for shooting and recording a moving image while periodically repeating a plurality of different shooting conditions is known. Japanese Patent Application Laid-Open No. 2004-228561 describes a technique for reproducing a moving image compressed and recorded by the Motion JPEG method by such shooting according to shooting conditions.
JP 2006-33033 A

  When a moving image is recorded, the amount of data is enormous, and therefore it is usually recorded after compression encoding. In particular, inter-frame compression using correlation between temporally adjacent frames is widely performed in moving images.

  In an image shot while periodically repeating different shooting conditions, the inter-frame correlation is lowered, and the coding efficiency of compression coding is lowered. In particular, when the exposure condition is repeatedly changed as the photographing condition, the correlation between adjacent frames is remarkably lost, and the encoding efficiency is lowered.

  It is an object of the present invention to provide an imaging apparatus that improves the encoding efficiency when shooting while periodically repeating a plurality of different shooting conditions.

  An imaging device according to the present invention includes an imaging unit that captures an image at a high rate while periodically repeating a plurality of different imaging conditions, and images captured by the imaging unit are grouped under the same imaging condition. And encoding means for compressing and encoding as a normal-rate moving image.

  According to the present invention, since images periodically shot under a plurality of different shooting conditions are collectively compressed and encoded under the same shooting conditions, an encoding method with high encoding efficiency can be applied. Thereby, encoding efficiency can be improved.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of an embodiment of the present invention. The photographing lens 10 makes an optical image of a subject incident on the image sensor 12. The photographing lens 10 includes an optical diaphragm that controls the amount of incident light of the image sensor 12. The image sensor 12 converts an optical image formed on the imaging surface into an electrical signal. The camera signal processing unit 14 performs known processes such as gamma conversion and color balance adjustment on the image signal from the image sensor 12 and converts the image signal into image data of a predetermined format. A portion including the photographing lens 10, the image pickup element 12, and the camera signal processing unit 14 corresponds to the image pickup means in the claims.

  The encoding unit 16 compresses the image data from the camera signal processing unit 14. The encoding unit 16 distributes the image data from the camera signal processing unit 14 for each frame under the same shooting condition, and a compression process for compressing and encoding the frame distributed by the frame distribution unit 18 for each shooting condition. Part 20. The compression processing unit 20 includes four data compression circuits 22 to 28 in this embodiment. Each of the data compression circuits 22 to 28 compresses the input frame images together under the same shooting conditions, and generates stream data in a predetermined format composed of the compressed image data. Each of the data compression circuits 22 to 28 includes a frame memory for compression encoding.

  Stream data output from the data compression circuits 22 to 28 is recorded on the recording medium 30. The recording medium 30 is composed of, for example, an optical disk or a hard disk.

  The control unit 32 comprehensively controls the photographing lens 10, the image sensor 12, the camera signal processing unit 14, the encoding unit 16, and the recording medium 30.

  In this embodiment, the image sensor 12 is operated at a rate four times the normal rate (30 frames / second), that is, 120 frames / second (high-speed rate), and each frame is repeatedly imaged under four different shooting conditions. . The camera signal processing unit 14 and the frame distribution unit 18 also operate at 120 frames / second. On the other hand, each of the data compression circuits 22 to 28 operates at a normal rate, that is, 30 frames / second. The recording medium 30 has a writing capability for recording four streams of stream data output from the data compression circuits 22 to 28. Note that the four streams of data written to the recording medium 30 can be expanded (decoded) at the time of reproduction, and then any one of the streams can be selectively displayed on a display device (not shown). The display rate of the display device at this time is 30 frames / second, which corresponds to the frame rate of each stream data.

  With reference to FIG. 2 and FIG. 3, how the photographing frame is changed according to the present embodiment will be described. 2A, 2B, 2C, and 2D show examples of images corresponding to four different shooting conditions. The photographing condition is, for example, the exposure amount, and can be adjusted by the aperture of the photographing lens 10, the shutter speed of the image sensor 12, or the gain of the camera signal processing unit 14. FIG. 2A shows an image with the largest exposure amount, decreases as shown in FIG. 2B and FIG. 2C, and FIG. 2D shows an image with the smallest exposure amount.

  In this embodiment, the image sensor 12 is operated at a rate four times the normal rate, and four different shooting conditions are repeated in order for each frame. FIG. 3 shows how the frames shot in this way change. In FIG. 3, the horizontal axis indicates the time interval. Four frame images are taken between frames at the normal rate. The frame interval of each frame image is 1/120 second (sec).

  In FIG. 3, frames 1 a, 2 a, and 3 a are frames shot under the same shooting condition A. Frames 1b, 2b, and 3b are frames shot under the same shooting condition B. Frames 1c, 2c, and 3c are frames shot under the same shooting condition C. Frames 1d, 2d, and 3d are frames shot under the same shooting condition D. For example, the shooting condition A is the optimum exposure amount, and the shooting conditions B, C, and D are set to the exposure values corrected in order by a fixed exposure correction amount.

  The encoding operation of this embodiment for the frame image shown in FIG. 3 will be described. The control unit 32 controls the imaging device 12 to shoot at a rate four times the normal rate, and controls the aperture of the photographic lens 10 to change in four steps within one cycle of the normal rate. The camera signal processing unit 14 processes the quadruple rate frame image from the image sensor 12 and supplies it to the encoding unit 16.

  In the encoding unit 16, the frame distribution unit 18 distributes the frame image data from the camera signal processing unit 14 to the data compression circuits 22 to 28 corresponding to the shooting conditions A, B, C, and D, respectively. For example, the frame distribution unit 18 supplies the image data of the frames 1a, 2a, and 3a under the shooting condition A to the data compression circuit 22, and supplies the image data of the frames 1b, 2b, and 3b under the shooting condition B to the data compression circuit 24. To do. The frame distribution unit 18 supplies the image data of the frames 1c, 2c, and 3c under the shooting condition C to the data compression circuit 26, and supplies the image data of the frames 1d, 2d, and 3d under the shooting condition D to the data compression circuit 28. FIG. 4 shows a state of a frame processed by each data compression circuit 22-28.

  Each data compression circuit 22 to 28 sequentially compresses the input frame images to form stream data. For example, the image data of frames 1a, 2a, and 3a are sequentially input to the data compression circuit 22. The frames 1a, 2a, and 3a are spaced apart from each other when taken, but the shooting conditions, that is, the exposure conditions are the same. Furthermore, since the frame intervals of the frames 1a, 2a, and 3a are equal to the frame rate of the normal rate, it is possible to expect an inter-frame correlation equivalent to an image captured at the normal rate. As a result, the data compression circuit 22 can apply inter-frame compression processing to the frames 1a, 2a, and 3a, and can perform compression coding with high coding efficiency. Similar inter-frame correlation can be expected for the data compression circuits 24, 26, and 28. As a result, in this embodiment, each of the data compression circuits 22 to 28 can compress the input frame with high coding efficiency using inter-frame compression, and can realize high coding efficiency as a whole.

  Stream data generated by the compression encoding of each of the data compression circuits 22 to 28 is recorded on the recording medium 30. The control unit 32 creates management information describing the relationship between these stream data and records the management information on the recording medium 30. FIG. 5 is a schematic diagram of information recorded on the recording medium 30. Streams A to D are streams generated by compression by the data compression circuits 22 to 28, respectively. In the management information, at least the recorded streams A to D are image data photographed almost simultaneously by the same image sensor 12, and the photographing conditions of the streams A to D are recorded. With this management information, these streams A to D can be reproduced or edited later. For example, when each of the streams A to D is an image having the same composition photographed with different exposure conditions, the exposure conditions can be selected during reproduction. Alternatively, a plurality of these streams A to D can be combined and reproduced as a moving image with an expanded dynamic range.

  As described above, according to the present embodiment, the frame image having the same shooting condition is regarded as an independent moving image, is compressed and encoded, and stream data is formed and recorded. High encoding efficiency can be realized.

  In addition, although the case where it image | photographs with the frame rate 4 times the normal rate was demonstrated, it is not limited to 4 times. Basically, when n shooting conditions are periodically repeated, shooting is performed at a frame rate of n times and n streams are recorded.

  In this embodiment, the plurality of data compression circuits 22 to 28 are operated in parallel. However, one data compression circuit may be operated in a time division manner. By this time-sharing operation, it is possible to realize the same effect as if a plurality of data compression circuits are operating in parallel.

  Also, the frame image is an example, and it is obvious that the same effect can be obtained by forming a stream for each shooting condition in the same manner for the field image. Therefore, in this specification, it should be understood that a frame includes a field.

  Although the example in which the shooting condition is the exposure amount has been described, the shooting condition is not limited to the exposure amount. For example, the exposure amount determination method may be varied as the photographing condition. That is, a different range such as the entire screen, the central portion of the screen, or a part of the screen may be set as the image area when determining the exposure. Alternatively, a preset shooting mode may be sequentially selected. For example, a shooting mode set in accordance with a specific shooting screen such as a spotlight mode or a sports mode may be selected. Alternatively, the image adjustment parameters in the camera signal processing unit 14 may be different. That is, parameter settings such as contrast and dynamic range may be different.

  FIG. 6 shows a schematic block diagram of the second embodiment of the present invention. Constituent elements having the same functions as those in the embodiment shown in FIG.

  In the present embodiment, the functions of the data compression circuits 22a to 28a of the compression processing unit 20a of the encoding unit 16a are different from those of the data compression circuits 22 to 28. In the embodiment shown in FIG. 6, only the data compression circuit 24a has a motion vector search function (Motion Estimation), and the other data compression circuits 22a, 26a, and 28a have no motion vector search function. Use motion vector search results. In other words, only one data compression circuit 24a has a part of the function necessary for interframe compression (here, the motion vector search function), and the remaining data compression circuits 22a, 26a, and 28a are connected to the data compression circuit 24a. The calculation result of the function of has been used.

  The operation of the encoding unit 16a will be described. The frame distribution unit 18 distributes the image data of the shooting frame from the camera signal processing unit 14 to the data compression circuits 22a to 28a of the compression processing unit 20a for each frame under the same shooting conditions. The ME unit 40 of the data compression circuit 24a searches for a motion vector.

  The motion search process is adopted in typical inter-frame compression methods such as MPEG2 and MPEG4. Specifically, the image is divided into blocks, a block that approximates the block is searched from past or future images (reference frames), and the motion (motion vector) between the blocks, the reference frame, and the reference mode are determined. It is processing. In inter-frame compression, inter-block difference information is generated from a motion vector, a reference frame, and a reference mode obtained by motion search processing, and is compressed and encoded.

  The data compression circuit 24a compresses and encodes the input frame using the motion search result by the internal ME unit 40. The result of motion search by the ME unit 40 is supplied to the other data compression circuits 22a, 26a, and 28a. The other data compression circuits 22a, 26a, and 28a compress and encode the input frame using the search result of the ME unit 40 of the data compression circuit 24a, that is, the motion vector, the reference frame, and the reference mode. The stream data generated by the compression encoding of each of the data compression circuits 22a to 28a is recorded on the recording medium 30 as in the first embodiment.

  The frames processed by the data compression circuits 22a, 26a, and 28a have different shooting times from the frames processed by the data compression circuit 24a, and there is a difference in the movement of images processed by the data compression circuits 22a to 28a. Can occur. However, since the shooting is performed at a frame rate faster than the normal rate, this difference is slight, and the motion vector search result does not greatly change. Therefore, even if the compression process is performed by sharing the motion search result in the data compression circuit 24a, the difference due to the difference in motion vector is small, and the compression efficiency does not greatly decrease.

  The motion vector search process generally requires a large amount of processing, and has a large arithmetic process and circuit scale. In the present embodiment, since this is shared, the circuit scale or cost can be reduced as a whole. In particular, when a plurality of systems of data compression are operated in parallel by independent hardware, the circuit scale and power consumption for motion vector search processing can be greatly reduced.

  Although the plurality of data compression circuits 22a to 28a are operated in parallel, one data compression circuit may be operated in a time division manner. By this time-sharing operation, it is possible to realize the same effect as if a plurality of data compression circuits are operating in parallel. A motion vector may be searched for in the first data compression process in the repetition cycle of the shooting conditions, and the previous motion vector search result may be used in the subsequent data compression process. Although it is not useful for reducing the circuit scale and power consumption, it leads to shortening or omission of the compression encoding process and can reduce the calculation time.

  Although motion vector search has been described as an example of a processing function necessary for interframe compression, it is obvious that the present invention can similarly handle other functions necessary for interframe compression.

It is a schematic block diagram of Example 1 of the present invention. It is an example of an image from which exposure conditions differ. It is an example of a frame image at the time of repeating four different imaging conditions periodically. It is a schematic diagram which shows a mode that compression encoding is carried out for every flame | frame of the same imaging conditions. 3 is a configuration example of data recorded on a recording medium 30. FIG. It is a schematic block diagram of Example 2 of this invention.

Explanation of symbols

10: photographic lens 12: imaging device 14: camera signal processing unit 16, 16a: encoding unit 18: frame distribution unit 20, 20a: compression processing units 22 to 28, 22a to 28a: data compression circuit 30: recording medium 32: Control unit 40: motion vector search unit

Claims (11)

Imaging means for capturing images at a high rate while periodically repeating a plurality of different imaging conditions;
An image pickup apparatus comprising: an encoding unit that compresses and encodes images captured by the image capturing unit as moving images at a normal rate by grouping images having the same shooting conditions.   The imaging apparatus according to claim 1, further comprising a recording medium that records the image data encoded by the encoding unit as stream data for each shooting condition.   The imaging apparatus according to claim 2, further comprising means for recording management information for stream data for each shooting condition on the recording medium.   The imaging apparatus according to claim 3, wherein the management information includes a stream data relationship for each shooting condition and information indicating the shooting condition.   The imaging apparatus according to claim 1, wherein the encoding unit includes an inter-frame compression process. The encoding means includes a plurality of data compression means;
A distribution unit that distributes the image captured by the imaging unit to the plurality of data compression units so that an image under the same imaging condition is supplied to the same data compression unit according to the imaging condition. The imaging apparatus according to any one of claims 1 to 4.   The imaging apparatus according to claim 6, wherein each of the plurality of data compression units includes inter-frame compression processing. One data compression means of the plurality of data compression means has a processing function necessary for compression encoding of the image,
The imaging apparatus according to claim 6, wherein the remaining data compression unit uses a result of the processing function of the one data compression unit.   The imaging apparatus according to claim 8, wherein the processing function is a motion search process of an inter-frame compression process.   The imaging apparatus according to claim 1, wherein the photographing condition is any one of an exposure condition, a photographing mode, and an image adjustment parameter.   The imaging apparatus according to claim 10, wherein the exposure condition is at least one of an exposure amount and an exposure amount determination method.

Images