JPH07162853A - Image pickup device - Google Patents

Abstract

PURPOSE:To reduce picture quality degradation even to the image of low correla tion by performing coding based on camera control information outputted by a control circuit and coding the control information as well at the image pickup device provided with coding and decoding means. CONSTITUTION:The image of an incidental optical signal is formed at a sensor 2 by a lens 1. An A/D converting circuit 3 converts the image pickup signal to a digital signal. A signal processing circuit 4 processes the image pickup signal and outputs a digital video signal composed of a luminance signal and a color difference signal. A user is controlled by a control circuit 11. The user inputs the contents of an operation by using an operation switch 14. The control circuit 11 supplies the control information to a coding circuit 10 and at the circuit 10, coding is performed by using this control information. The digital video signal outputted from the circuit 4 is respectively outputted to an encoder 5 and a format transforming circuit 9. The circuit 9 transforms the output of the circuit 4 to any intermediate format required for coding.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup device, and in particular,
The present invention relates to an imaging device that outputs a coded video signal for compressing the amount of information of a captured image.

[0002]

2. Description of the Related Art With the progress of digitization of video signals, high-efficiency coding of moving images has come to be performed. An example of such an encoding algorithm is inter-frame predictive encoding that encodes a difference between frames in a video signal. This is an encoding method that reduces the amount of information by utilizing the correlation between adjacent frame images. Further, there is known a motion-compensated inter-frame predictive coding which adds motion compensation to this system to improve coding efficiency of an image including a fast motion.

Such an encoding system is based on the MPEG (Moving Picture Exp) which is an encoding standard for storage media.
ert Group), CCITT's H.264 standard, which is an encoding standard for communications such as videophones. It is adopted in 261 etc. The method of encoding / decoding in accordance with these standards is described in, for example, the Journal of Television Society; Vol. 45,
No. 7 (1991), page 793 to page 799, and page 807 to page 812.

[0004]

However, in these methods, since coding is performed by utilizing the correlation between frames, when the frame correlation is low and motion compensation is difficult, for example, scene switching or scene change is performed. There is a problem that the image quality is deteriorated when an operation such as fade or zoom is performed. Regarding the means for solving such a problem, for example, the above-mentioned conventional technique describes a method of switching from inter-frame predictive coding to intra-frame coding in which an input video signal is coded as it is, but this is, of course, It is hard to say optimal because the average code rate increases.
In addition, for example, Journal of Television Society; Vol. 35, N
o. 10 (1985), pages 949 to 954, a method of performing a fade prediction is described, but this increases the circuit scale significantly. In particular, when such an encoding function is mounted on an image pickup device which is required to be downsized, an increase in circuit scale becomes a problem.

An object of the present invention is to solve the above-mentioned problems and to obtain an image signal which outputs a highly efficient coded video signal with a small circuit configuration in which the image quality is less deteriorated even for an image having a low frame correlation. To provide a device.

[0006]

In order to achieve the above object, according to the present invention, an image pickup means for outputting a digital video signal, a control means for controlling the image pickup means, and a digital video signal output by the image pickup means are encoded. In an image pickup apparatus including an encoding unit, the control unit outputs control information of the image pickup unit, the encoding unit performs encoding based on the control information of the image pickup unit, and also encodes the control information together. To configure.

[0007]

The image pickup means subjects the image pickup signal to desired signal processing and outputs a digital video signal. The control means of the image pickup means sends control signals to the image pickup means such as start and end of image pickup (scene change), zoom, fade, pan, tilt, exposure state, white balance, etc.
Control the image pickup means. The coding means performs inter-frame coding on the video signal output by the image pickup means and outputs the video signal. At this time, the encoding means uses the control signal output from the control means to perform encoding according to the above-described photographing operation, and also encodes and outputs the control information as well.

By thus encoding using the control information of the image pickup means and also including the control information, it is possible to perform optimum encoding and decoding according to the image pickup state. As a result, it is possible to realize an image pickup apparatus that outputs a highly efficient coded video signal with a small circuit configuration that causes little image quality deterioration even for an image with low frame correlation.

[0009]

The present invention will be described below with reference to the illustrated embodiments. FIG. 1 is a block diagram showing the arrangement of an image pickup apparatus according to the first embodiment of the present invention. In the figure, reference numeral 1 is a lens, which receives an incident optical signal by a CCD (Charge Coupled D
The image is formed on the sensor 2 such as evice). The sensor 2 converts the formed optical signal into an electric signal and outputs it as an image pickup signal. An A / D conversion circuit 3 converts an analog image pickup signal into a digital signal. The signal processing circuit 4 performs known processing such as gamma correction and white balance correction on the digitized image pickup signal and outputs a digital video signal including a luminance signal and a color difference signal.

The signal processing circuit 4 is controlled by the control circuit 11. A user who operates this imaging device inputs the content of the operation by the operation switch 14.
Such camera operations include the start and end of imaging, zoom,
The control circuit 11 includes a fade, etc.
It controls the signal processing circuit 4, the lens drive circuit 12, and the sensor drive circuit 13. The lens drive circuit 12 sets the zoom magnification
The sensor drive circuit 13 controls the shutter speed and the like.
The control circuit 11 supplies the control information of the signal processing circuit 4 and the like to the encoding circuit 10, and the encoding circuit 10 performs encoding using this control information as described later.

The digital video signal output from the signal processing circuit 4 is output to the encoder 5 and the format conversion circuit 9, respectively. The encoder 5 converts the digital video signal into a signal according to the standard television system, and then outputs an analog signal from the output terminal 7 and a D / D signal from the output terminal 8.
A digital signal is output via the A conversion circuit 6. Therefore, although not shown here, the analog signal from the output terminal 7 may be input to a conventional analog recording device for analog recording, a viewfinder for monitoring the current imaging state, and the like. Become.

The format conversion circuit 9 converts the digital video signal output from the signal processing circuit 4 into an intermediate format required for encoding. This format depends on the encoding method performed by the encoding circuit 10. For example, H.264 for communication. When encoding according to the H.261 standard, CIF (Common Intermediate Format)
Or QCIF (Quarter CIF). When other encoding is performed, it may be converted into a format permitted by the encoding method.

The coding circuit 10 performs coding by a method such as interframe predictive coding. The coding may or may not be according to a particular standard. When encoding, the control information output from the control circuit 11 is used for encoding. The control information includes presence / absence of operations such as scene change, zoom, and fade that cause image quality deterioration (eg, zoom magnification, signal gain during fade, etc.).
In accordance with the contents of the camera operation, the encoding circuit 10 switches the encoding method so as to perform optimum encoding, and also encodes the control information used for the encoding. The signal encoded by the encoding circuit 10 is output from the output terminal 15.

Next, the coding in the coding circuit 10 will be described. Many encoding methods have been proposed and any method may be used.
The case of encoding based on PEG will be described as an example. FIG. 2 is a block diagram showing an example of the configuration of the encoding circuit 10 in FIG. The figure is based on a general configuration for realizing MPEG encoding, but in the motion detection circuit 210 and the prediction circuit 29 which determine the encoding method, the control information output from the control circuit 11 in FIG. It differs from the conventional encoding in that it is used and that the control information used for the encoding is encoded by the variable length encoding circuit 25.

The input signal of the frame memory 20 in FIG. 2 is the output signal of the format conversion circuit 9 in FIG. 1 and is stored in the frame memory 20. The subtractor 21 is
An inter-frame difference signal is generated by taking the difference between the output signal of the frame memory 21 and the signal output by the prediction circuit 29. The signal output from the prediction circuit 29 is MP
The control circuit 11 is newly added to the front-back and both-side prediction, which is a feature of EG.
The prediction is performed by adding the control information that is output by, and the accuracy is improved. The inter-frame difference signal is a DCT
After being intra-frame coded by the (Discrete Cosine Transform) circuit 23, it is quantized by the quantization circuit 24 in order to reduce the code amount. The quantized signal is output to the variable length coding circuit 25 and the dequantization circuit 26, respectively. In the variable length coding circuit 25, the quantized signal and the control information used for the prediction are multiplexed and further Huffman coded. The Huffman-coded signal is output from the output terminal 15 via the buffer memory 28.

On the other hand, the coded signal output from the quantizing circuit 24 is decoded by a decoding circuit composed of an inverse quantizing circuit 26, an IDCT (Inverse DCT) circuit 27 and an adder 22, and output to a predicting circuit 29. To be done. The prediction circuit 29 generates an output signal according to the above prediction and performs motion compensation using the motion vector output from the motion detection circuit 210. Here, the control information output from the control circuit 11 is input to the motion detection circuit 210, and the accuracy of the output motion vector is improved.

Next, the structure of a decoding circuit for decoding a signal coded by the above algorithm is shown in FIG. 3, and its operation will be briefly described.

A coded signal input from the input terminal to the decoding circuit 30 (a signal output from the output terminal 15 of FIG. 2)
Is input to the variable-length decoding circuit 32 via the buffer memory 31, Huffman-decoded, and separated into the quantized signal and the control information. The former quantized signal is decoded by the inverse quantization circuit 26 and the IDCT (Inverse DCT) circuit 27 into the inter-frame difference signal, and the adder 35 adds it to the prediction signal output from the predictor 36 to obtain a digital image. The signal is decoded, output from the output terminal 37, and input to the predictor 36. The latter control information is input to the predictor 36 together with the decoded digital video signal, and a prediction signal is generated for subsequent decoding of the digital video signal.

Next, the encoding circuit 10 of FIG. 2 performs encoding using the control information output from the control circuit 11 (FIG. 1), and the decoding circuit 30 of FIG. 3 decodes this. An example will be described with reference to FIG.

The present example is an operation example when a zoom-up operation is performed during image pickup. For simplification of explanation, the video signals of two consecutive frames are multiplied by a times (a ≧
1) The operation when zooming up will be described. In FIG. 4, 40 is the video signal of the previous frame, and 41 is the video signal of the current frame. When predicting the video signal of the current frame from the video signal 40 of the previous frame, the video signal 42 obtained by zooming up the video signal 40 to a times is used as the prediction signal by using the information of a times from the control information. It can be generated electronically by known techniques using memory. The frame difference signal 43 between the predicted signal 42 and the video signal 41 of the current frame is generated by the subtracter 21, quantized and Huffman coded as described above, and output.

In FIG. 4, if the above-mentioned signal is Huffman-decoded and dequantized, the inter-frame difference signal is 44, and the decoded video signal of the previous frame is 45.
A video signal 46 obtained by zooming up 5 times a times is used as a prediction signal, and is added to the inter-frame difference signal 44 by the adder 35 to be decoded into a digital video signal 47. Therefore, the information to be encoded by the encoding circuit 10 is only the inter-frame difference signal 43 and the zoom magnification a, and the encoding efficiency can be improved even at the time of zooming up (there is little movement between frames, and the predictive zoom is accurate). If done well, the inter-frame difference signal 43 approaches 0 without limit).

Next, the encoding circuit 10 of FIG. 2 performs encoding using the control information output from the control circuit 11 (FIG. 1), and the decoding circuit 30 of FIG. 3 decodes this. Another example will be described with reference to FIG.

This example is an operation example when a zoom-down operation is performed during image pickup. In this example as well, in order to simplify the description, the video signals of two consecutive frames are b
The operation at the time of double (b <1) zoom down will be described.
In FIG. 5, 50 is the video signal of the previous frame, and 51 is the video signal of the current frame. Video signal 5 of previous frame
When the video signal of the current frame is predicted from 0, the video signal 52 obtained by zooming down the video signal 50 to b times is used as the prediction signal by using the information of b times from the control information. The predictor signal 52 and the inter-frame difference signal 53 of the video signal 51 of the current frame are generated by the subtractor 21, and quantized and Huffman coded as described above, and output.

In FIG. 5, if the above-mentioned signal is Huffman-decoded and dequantized, the inter-frame difference signal is 54, and the decoded video signal of the previous frame is 55.
The video signal 56 obtained by zooming down 5 by b times is used as a prediction signal and is added to the inter-frame difference signal 54 by the adder 35 to be decoded into the digital video signal 57. Therefore, the information to be coded by the coding circuit 10 is only the inter-frame difference signal 53 and the zoom magnification b, and the coding efficiency can be improved even at the time of zooming down (however, the predicted signals 52 and 56 after zooming down are obtained). Indicates that the information in the portion indicated by the frame in the figure is missing, so that the inter-frame difference signals 53, 54 increase the information in the shaded portion).

Here, an example of encoding using zoom up / down control information has been described, but as other control information, fade, scene change, pan, tilt, exposure state, white balance, etc. may be used. Similarly, it is possible to improve the coding efficiency.

For example, during the fade operation, the control circuit 1
In No. 1, it is possible to grasp the changing values of the gain of the luminance signal, the color difference signal, and the setup amount in each field, so that an optimal encoding can be realized by generating the prediction signal based on this control information.

FIG. 6 is a block diagram showing the arrangement of an image pickup apparatus according to the second embodiment of the present invention. In the present embodiment, a recording / reproducing function is added to the first embodiment of FIG. 1, the same components as those in the previous embodiment are designated by the same reference numerals, and the description thereof will be omitted to avoid duplication ( Note that this is the same in each of the following examples).

In FIG. 6, reference numeral 61 denotes a recording circuit, which records the encoded signal on the recording medium 62. A reproducing circuit 63 reproduces a signal recorded on the recording medium 62 such as a magnetic tape. The switch 66 is a switch for switching between the reproduction signal and the encoded signal before recording. The output signal of the switch 66 is decoded by the decoding circuit 30 described above, and the format reverse conversion circuit 64 performs the reverse conversion of the conversion performed by the format conversion circuit 9. The switch 60 switches between an image pickup signal and a reproduction signal.

In the case of the present embodiment, since the start and end of recording mean a scene change, it is possible to improve the coding efficiency by using the control information. Operation example
Will be described with reference to FIG. When the recording end information is input to the prediction circuit 29 of the encoding circuit 10, the video signal 71 (FIG. 7) at the end and the recording end information are encoded and output, and the recording circuit 61 outputs the information. It is recorded on the recording medium 62. Thereafter, when the recording is started, the recording start information is input to the prediction circuit 29 in the same manner as described above, and the recording start information is encoded and output together with the video signal 72 (FIG. 7) at the time of the recording start. Then, it is recorded on the recording medium 62 by the recording circuit 61.

Thereafter, the signal recorded on the recording medium 62 such as a magnetic tape is reproduced by the reproducing circuit 63, and the switch 6
It is input to the decoding circuit 30 via 6. Decoding circuit 30
Then, by using the above-mentioned video signals 71 and 72 before and after the scene change, the former gradually coarsens the data and the latter gradually synthesizes the data in detail for a certain period of time. Control such as so-called cross-fade for switching to is performed. As a result, during the scene change period, it is sufficient to encode the video signals for the above two frames and the recording end and recording start information, and it is possible to improve the encoding efficiency.

The other parts of this embodiment have substantially the same structure as the first embodiment of FIG.
The same effect as the embodiment can be obtained.

FIG. 8 is a block diagram showing the arrangement of an image pickup apparatus according to the third embodiment of the present invention. The present embodiment is different from the second embodiment in FIG. 6 in that it has a camera shake correction function. In FIG. 8, reference numeral 81 is a motion detection circuit, and 80 is a zoom circuit. The motion detection circuit 81 detects a motion vector for each field (or one frame). For the motion vector, the screen is divided into blocks of about 16 × 16, and the motion vector is obtained for each block. Regarding the method of correcting camera shake from the obtained motion vector, for example,
Television Society Technical Report; Vol. 15, No. 7
(1991), pp. 43-48.
The camera shake correction is to detect the motion of the entire screen from the vector obtained for each block and move the output position of the image by the motion vector so as to stop the shake of the image and correct it. . However, if the output position of the screen is shifted, there is a problem that the end of the screen is cut off in the reproduction screen. Therefore, the zoom circuit 80 is used to prevent the edge of the screen from being chipped.

By the way, in this embodiment, this motion detection is used for encoding. In the first embodiment of FIG. 1, the encoding circuit 10 has the motion detection circuit 210, but in the present embodiment, the motion detection circuit 81 for camera shake correction is also used for the motion detection for encoding. In this way, since the motion detection circuit is shared, the motion detection in the encoding circuit 10 becomes unnecessary, and thus the circuit scale can be significantly reduced.

FIG. 9 is a block diagram showing the arrangement of an image pickup apparatus according to the fourth embodiment of the present invention. The image pickup apparatus of this embodiment also has a camera shake correction function, and in this embodiment, a motion detection circuit 90 using an angular velocity sensor or the like is provided to prevent camera shake. The motion vector thus obtained represents the uniform motion of the entire image, and the motion vector of each block in the frame cannot be obtained. However, camera operations such as panning are not affected by the noise of the video signal, so that the motion vector can be accurately obtained. For other effects,
This is similar to the first embodiment of FIG.

FIG. 10 is a block diagram showing the arrangement of an image pickup apparatus according to the fifth embodiment of the present invention. This embodiment is an application example to a videophone. In the configuration shown in FIG.
The encoded signal is output to the transmission line for communication through the transmission I / F circuit 101. The signal input to the transmission I / F circuit 101 via the communication path is decoded by the decoding circuit 30, converted into a video signal, and output to the monitor (display device) 102.

The encoding in this embodiment is performed according to the H.264 standard. 26
Although it is performed based on the communication standard such as 1 or the like, in the videophone as shown in FIG. 11, it can be assumed that the person 111 who interacts is imaged and the material 112 such as a document or a photograph is imaged. In order to deal with this, there is a configuration in which the image pickup unit 110 rotates or moves.
Since the start ((a) of FIG. 11) and the end ((b) of FIG. 11) of 10 mean a scene change, it is possible to improve the coding efficiency by using the control information. . This is the same as the second described in FIGS. 6 and 7.
In the embodiment, instead of the recording end and start information,
The information on the start and end of the rotation or movement of the image pickup unit 110 is used, and other effects are the same as those in the second embodiment.

[0037]

As described above, according to the present invention, in the image pickup apparatus having the encoding and decoding means, the start / end of image pickup, fade, zoom, scene change, pan, tilt, exposure which are output from the control circuit. Encoding is performed based on camera control information such as state and white balance, and the control information is also encoded, so that image quality degradation is small and small even for images with low frame correlation. It is possible to provide an imaging device that outputs a highly efficient encoded video signal with a large-scale circuit configuration.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image pickup apparatus according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a specific example of the encoding circuit in FIG.

FIG. 3 is a block diagram showing a specific example of a decoding circuit that decodes an encoded signal output from the encoding circuit in FIG.

FIG. 4 is an explanatory diagram showing an example of encoding and decoding operations during zoom-up according to the first embodiment of the present invention.

FIG. 5 is an explanatory diagram showing an example of encoding and decoding operations during zoom-down according to the first embodiment of the present invention.

FIG. 6 is a block diagram showing a configuration of an image pickup apparatus according to a second embodiment of the present invention.

FIG. 7 is an explanatory diagram showing an example of encoding and decoding operations at the time of a scene change according to the second embodiment of the present invention.

FIG. 8 is a block diagram showing a configuration of an image pickup apparatus according to a third embodiment of the present invention.

FIG. 9 is a block diagram showing a configuration of an image pickup apparatus according to a fourth embodiment of the present invention.

FIG. 10 is a block diagram showing a configuration of an image pickup apparatus according to a fifth embodiment of the present invention.

FIG. 11 is an explanatory diagram showing an example of a scene change operation of a videophone using the image pickup apparatus according to the fifth embodiment of the present invention.

[Explanation of symbols]

 1 lens 2 sensor 3 A / D conversion circuit 4 signal processing circuit 5 encoder 6 D / A conversion circuit 7 analog video output terminal 8 digital video output terminal 9 format conversion circuit 10 encoding circuit 11 control circuit 12 lens drive circuit 13 Sensor drive circuit 14 Operation switch 15 Coded video signal output terminal 61 Recording circuit 63 Playback circuit 80 Zoom circuit 81 Motion detection circuit 90 Motion detection circuit 101 Transmission I / F circuit 102 Monitor (display device)

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Noriyuki Iura Inventor Noriyuki Iura 292 Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa Inside the Video Media Research Laboratory, Hitachi, Ltd. (72) Taya Imade Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa 292 Address: Hitachi Media Co., Ltd. Video Media Research Laboratory (72) Inventor Tatsushi Nishimura Yoshida Town, Totsuka Ward, Yokohama City, Kanagawa 292 Address: Hitachi Co., Ltd. Video Media Research Center

Claims (8)

[Claims] 1. An image pickup apparatus comprising: an image pickup means for outputting a digital video signal; a control means for controlling the image pickup means; and an encoding means for encoding a digital video signal output by the image pickup means. The control means outputs the control information of the image pickup means, and the encoding means is configured to perform encoding based on the control information, and to perform encoding including the control information. Imaging device. 2. An image pickup means for outputting a digital video signal, an encoding means for encoding the digital video signal output by the image pickup means, and recording / reproducing the encoded video signal on a recording medium such as a magnetic tape. An image pickup apparatus comprising: a recording / reproducing means for performing the above operation; a decoding means for decoding a signal reproduced from the recording medium; and a control means for controlling the image pickup means and the recording / reproducing means. Outputs the control information of the imaging means and the recording / reproducing means, and the encoding means performs encoding based on the control information, and encodes the control information as well. A characteristic imaging device. 3. An image pickup means for outputting a digital video signal, a motion correction means for detecting and correcting a motion vector such as a camera shake generated by the image pickup means, and a code for encoding the digital video signal output by the image pickup means. Encoding means, recording / reproducing means for recording / reproducing the encoded video signal on a recording medium such as a magnetic tape, decoding means for decoding the signal reproduced from the recording medium, the imaging means and the above An image pickup apparatus comprising a motion correction means and a control means for controlling the recording / reproducing means, wherein the control means outputs control information of the image pickup means, the motion correcting means and the recording / reproducing means, and the code The imaging device is configured such that the encoding means performs encoding based on the control information, and includes the control information. 4. Imaging means for outputting a digital video signal, coding means for coding the digital video signal output by the imaging means, and means for outputting the output signal of the coding means to a digital communication terminal. An image pickup device comprising a decoding means for decoding a signal input from the digital communication terminal, and a control means for controlling input / output to / from the image pickup means and the digital communication terminal, wherein the control means is the image pickup device. An image pickup apparatus, characterized in that it is configured to output control information of means, the encoding means performs encoding based on the control information, and encodes also including the control information. 5. The control information according to claim 1, wherein the control information includes start / end of image pickup by the image pickup unit, fade, zoom, scene change, pan, tilt, exposure state, white balance and the like. An image pickup apparatus comprising at least one of operating state information and stop state information. 6. The image pickup device according to claim 2, wherein the control information includes recording start information and recording end information in the recording / reproducing means. 7. The image pickup device according to claim 3, wherein the control information includes motion vector information output from the motion correction means. 8. The imaging device according to claim 4, wherein the control information includes communication start information and communication end information.