JP2000209590A - Image encoder, image encoding method, storage medium and image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain encoding of an image at a compression rate different between a focused area and a nonfocused area. SOLUTION: A video capture device 46 captures a video signal outputted from a camera and writes the signal to a memory secured in a RAM 36. A computer 30 reads information denoting a position, its size of a focus detection object area of a captured video frame and a distance D up to an object from the camera via a bus interface 50. The computer 30 describes focus area information and a quantization table for a focus area and a nonfocused area to a header. A picture is divided into N square blocks of light pixels, and they are DCT-transformed. A focus area quantization table is applied to blocks in the focal area and the nonfocused area quantization table is applied to the nonfocused area for quantization. Huffman coding is applied to coefficients after quantization. The computer 30 stores the image that is compression coded to a secondary storage device 33 and/or distributed to the network via a network interface 48.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding apparatus and method, a storage medium, and an imaging apparatus. The present invention relates to a storage medium for storing software and an imaging device.

[0002]

2. Description of the Related Art Numerous methods have been developed for digitally recording images taken by image pickup means. Normally, image data has a huge amount of data, and is recorded by compression encoding. Typical image compression coding methods are the JPEG method and the MPEG method using transform coding. In these coding methods, after performing orthogonal transform such as discrete cosine transform (DCT) in order to remove the correlation between neighboring pixels inherent in an image, the obtained transform coefficients are quantized, and Huffman coding is performed. And other variable length coding.

[0003]

In the conventional example, coding is performed at the same compression ratio for a focused area near the subject and a background area (unfocused area) that is out of focus. Usually, most of the information desired by the user is considered to be in the in-focus area, so this method is not appropriate, and the in-focus area should have a larger amount of information than the out-of-focus area. .

In order to extract a focused area by image processing, it is necessary to convert an image into a frequency domain once and determine whether or not there is a high frequency component. Considering the calculation amount, this method cannot be applied to a moving image captured at a full frame rate (30 frames / sec).

An interactive camera which has a high affinity with a computer and can read and control various parameters (focus detection target area, zoom magnification value, etc.) in the image pickup apparatus by connecting to the computer. Are known.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image encoding apparatus and method for encoding an in-focus area and an out-of-focus area at different compression rates, a storage medium, and an imaging apparatus.

[0007]

According to the present invention, there is provided an image coding apparatus comprising: an input unit for inputting image information and information on a focus detection target area and a subject distance; and a focus area based on the subject distance information. An encoding unit that compresses and encodes the image information at a different compression ratio between the focus detection target region indicated by the information and the other region, and the encoding unit using the focus detection target region and the subject distance information as a header. Output means for outputting compression-encoded image information.

An image encoding method according to the present invention includes a capturing step of capturing image information and information of a focus detection target area and a subject distance, and a focusing step indicated by the focus area information in accordance with the subject distance information. An encoding step of compressing and encoding image information at a different compression ratio between the detection target area and the other areas, and the in-focus detection target area and the subject distance information as headers, which are compression-encoded in the encoding step. And outputting an image information.

A storage medium according to the present invention stores program software for executing the above-described image encoding method.

An image pickup apparatus according to the present invention comprises: an image pickup means for outputting photographed image information and information on a focus detection target area and a subject distance; and a focus indicated by the focus area information in accordance with the subject distance information. An encoding unit for compressing and encoding the captured image information at a different compression ratio between the detection target region and the other region; and a compression encoding by the encoding unit using the focus detection target region and the subject distance information as a header. And output means for outputting the obtained photographic image information.

[0011]

Embodiments of the present invention will be described below in detail with reference to the drawings.

First, the configuration of the interactive camera used in the present invention will be described. FIG. 1 shows a schematic block diagram of the configuration. The taking lens 12 of the camera 10 includes a zooming lens 12a and a focusing lens 12b for focusing. The imaging lens 12 couples an optical image of a subject to an imaging surface of an imaging device 14 such as a CCD. The imaging device 14 converts an optical image on the imaging surface into an electric signal and outputs the electric signal to the camera process circuit 16. Camera process circuit 1
6 is a well-known process (gain correction,
γ correction and color balance adjustment), and outputs a video signal of a predetermined format. The video signal output from the camera / process circuit 16 is supplied to a focus control circuit 18 and a moving object detection circuit 22 in the camera 10 and also to an external computer 30.

The focusing control circuit 18 controls the focusing lens 12 so as to focus on an object in the focus detection target area.
control b. FIG. 2 shows an example of the focus detection target area in the imaging screen. The position of the focus detection target area is determined by the overall control circuit 20 according to the user's designation.
In some cases, the overall control circuit 20 sets the focus control circuit 18 according to the detection result of the moving object detection circuit 22 using a known moving object detection method (such as background difference or color identification). When tracking a moving object, the latter is used. The focusing control circuit 18 adjusts the position of the focusing lens 12b (not shown) so that the high-frequency component contained in the video signal in the focus detection target region in the video signal from the camera process circuit 16 is maximized. (Stepping motor) to move and adjust in the optical axis direction. Thereby, the subject in the focus detection target area is automatically focused. The distance measurement circuit 23 measures the distance to the subject by a known method (calculated based on driving information of the stepping motor).

Information on the position and size of the focus detection target area and the distance to the subject is stored in a bus interface 24.
Is output to the computer 30 via the.

The magnification setting circuit 26 sets the magnification specified by the overall control circuit 20 in the zoom control circuit 28.
The zoom control circuit 28 drives the zooming lens 12a in the direction of the optical axis so that the set magnification is achieved.

The computer 30 is actually a camera 1
0 functions as an image recording apparatus for compressing and recording a video signal captured. FIG. 3 shows a schematic block diagram of the configuration. In this embodiment, the JPEG method is used as the compression encoding method. 32 is a CPU, 34 is a ROM, 36 is a RAM as a main memory, 38 is a secondary storage device such as a hard disk, 40 is a monitor for displaying images, 42 is a mouse,
Reference numeral 4 denotes a keyboard; 46, a video capture device for capturing a video signal output from the camera process circuit 16 of the camera 10; 48, a network interface for connecting the computer 30 to a network;
And a bus interface for exchanging various kinds of information (for example, position information of a focus detection target area, etc.) between them.

FIG. 4 is a flowchart showing an algorithm of image compression encoding in the present embodiment. The computer 30 includes the camera 10 (the camera process circuit 16 thereof).
The video signal output from is captured by the video capture device 46 and written into a memory (hereinafter, referred to as a frame memory) secured on the RAM 36 (S1). After this,
Alternatively, at the same time, the computer 30 reads, via the bus interface 50, information on the position and size of the focus detection target area of the video frame immediately before or currently captured and the distance D to the subject from the camera 10 (S
2). FIG. 5 shows an example of a parameter for specifying the focus detection target area. In this embodiment, the focus detection target area is specified by four parameters of the upper left coordinates (Xp, Yp) and its size (Xf, Yf) in the imaging screen.
The computer 30 sets an area specified by these parameters on the currently acquired video frame as an in-focus area.

Next, the computer 30 creates a header. First, JPEG comment segments (markers,
Second, information obtained from the camera 10 using the code 0xFFFE) is described in the header, and second, a quantization table for the in-focus area and a quantization table for the non-focus area are created and described in the header.

FIG. 6 shows an example of the structure and data arrangement of a comment segment. In the comment segment, following the code indicating the start of the segment and the segment length, information on the above-described in-focus area and information on the distance to the subject are described.

FIG. 7A shows an example of a quantization table for an out-of-focus area, and FIG. 7B shows an example of a quantization table for an in-focus area. However, here, only the quantization table for the luminance component is shown. The JPEG method uses a quantization table (hereinafter referred to as a recommended table) in which the quantization width of a high-frequency component is larger than that of a low-frequency component in consideration of human visual characteristics. FIG. 7A shows this recommendation table. In the present embodiment, the recommendation table is used for an out-of-focus area. On the other hand, in the focusing area, a quantization table as shown in FIG. 7B is used. In the quantization table shown in FIG. 7B, since the quantization width is set to the same value for all DCT coefficients, high frequency components can be stored as compared with the recommended table. Since all the values are smaller than the out-of-focus area, the compression ratio is lower than in the out-of-focus area, and the image quality is improved.

In practice, these quantization tables are scaled and used as follows according to the quality required by the user. That is, Q′a = Q · Ta [i, j] (1) Q′b = Q · Tb [i, j] · D (2) where D <1.0 Equations (1) and (2) In each of the defocusing tables Q′a, Q′a,
Q′b. Ta [i, j] and Tb [i, j] are quantization tables shown in FIGS. 7A and 7B, and Q is a scaling factor set by the user to obtain an image of desired quality. . In Expression (2), the quantization width of the in-focus area increases in proportion to the distance D to the subject, and the compression ratio increases. However, the distance D must be scaled so that the distance to the background is 1.

FIG. 8 shows the format of the header obtained from the above processing.

When the description of the header is completed, encoding is started next (S4 and subsequent steps). In the present embodiment, first, the image is divided into N eight-pixel square blocks (S4), a variable n indicating the encoding target block is initialized to 1 (S5), and DC
T (discrete cosine transform) is performed (S6). It is determined whether or not the block #n exists in the focus area (S
7). The determination method is based on which of the pixels in the block contains a large number of pixels. The quantization is performed using the quantization table for the in-focus area or the out-of-focus area according to the determination result (S8, S9), and the quantized coefficient is Huffman-coded (S10).

Step S6 and subsequent steps are repeated until n is incremented and all the blocks of one screen are encoded (S12).

The computer 30 stores the image thus compressed and encoded in the secondary storage device 38 and / or distributes the image on the network via the network interface 48.

FIG. 9 shows a flowchart of a process for decoding the image data thus encoded. By referring to the comment segment, information on the focused area is obtained (S21). Next, Huffman decoding is performed in units of 8-pixel square blocks (S23). If the target block belongs to the in-focus area, inverse quantization is performed using the in-focus area quantization table (S23).
25) If it belongs to the out-of-focus area, inverse quantization is performed using the out-of-focus area quantization table (S26). Next, inverse DCT is performed (S27). This is executed for all blocks of the target screen (S29).

In the above embodiment, the compression ratio in the in-focus area is reduced while the compression rate in the out-of-focus area is set to a normal level. In this case, since the code amount increases, it is not suitable for low-rate coding. When low-rate coding is required, for example, conversely, the overall code amount is smoothed by increasing the compression ratio of the focus area, and the information amount is reduced as compared with the conventional method (an encoding method that does not distinguish the focus area). May be reduced. That is, the value of the quantization table in the in-focus area is set to a value larger than that in the non-focus area. As in the case of the above embodiment, these quantization tables are scaled and used according to the quality required by the user. As the scaling method, the user can freely adopt a method depending on the distance to the subject or the like, but the quantization width of the focused area must be always larger than that of the non-focused area.

FIG. 10 shows the camera 10 and the computer 30.
FIG. 1 shows a schematic block diagram of a device in which is integrated. An overall control circuit 20a corresponding to the overall control circuit 20 controls the image recording device 30a corresponding to the computer 30. The image recording device 30a includes a video capture device 60 that captures a video signal output from the camera / process circuit 16,
An image compression encoding device 62 for compressing and encoding the video information captured by the video capture device 60, and a recording device 64 for recording the video information compressed by the image compression and encoding device 62 on a recording medium. The operation of the image recording device 30a is substantially the same as that of the computer 30.

Although the embodiment in which the JPEG system suitable for a still image is used as the image compression encoding system has been described, the present invention can be applied to a case where a moving image compression encoding system represented by the MPEG system is used. . As is well known, in the MPEG system, an arbitrarily selected frame A is first selected.
As a still image, and intra-frame encoding is performed using substantially the same algorithm as the JPEG system. Become

An intra frame can be set by a user at an arbitrary position in a moving image stream. Usually, when a severe change such as a scene change occurs between frames, a frame immediately after the scene change occurs is regarded as an intra frame. And When the present invention is applied, each of the above-described embodiments is applied to the coding of the intra frame, and different quantization tables are used for the in-focus area and the out-of-focus area.

According to the present invention, a storage medium storing software program codes for realizing the functions of the above-described embodiments is supplied to a system or an apparatus, and a computer (CPU or MPU) of the system or the apparatus stores the storage medium in the storage medium. This can also be achieved by reading and executing the stored program code. In this case, the program code itself read from the storage medium realizes the function of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

As a storage medium for supplying the program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM,
DR, magnetic tape, nonvolatile memory card and RO
M etc.

When the computer executes the readout program code, not only the functions of the above-described embodiment are realized, but also the operating system running on the computer based on the instruction of the program code. This also includes a case where some or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing.

Further, after the program code read from the storage medium is written into a memory provided on an expansion board or a function expansion unit inserted in the computer,
The present invention is also applicable to the case where the CPU or the like provided in the function expansion board or the function expansion unit performs part or all of the actual processing based on the instructions of the program code, and the processing realizes the functions of the above-described embodiments. Included in the range.

[0035]

As can be easily understood from the above description, according to the present invention, the in-focus area is encoded using the information on the in-focus detection target area and the distance to the subject obtained from inside the camera. Make the parameters different from other areas. By reducing the compression ratio of the in-focus area, the attention area can be recorded and transmitted more clearly. Conversely, by increasing the compression ratio of the in-focus area relative to the out-of-focus area, it is possible to perform compression encoding at a low rate while preserving a background portion having a small amount of information.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of a first embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of a focus detection target area.

FIG. 3 is a diagram showing a schematic configuration of a computer 30.

FIG. 4 is a flowchart of compression encoding according to the present embodiment.

FIG. 5 is a diagram showing an example of parameterization of a focus detection target area.

FIG. 6 is a diagram illustrating an example of a header description method of focus area information.

FIG. 7 is a diagram illustrating an example of a quantization table according to the first embodiment.

FIG. 8 is an example of the structure of a header portion in the first embodiment.

FIG. 9 is a flowchart of a process for decoding video information compressed in the present embodiment.

FIG. 10 is a schematic configuration block diagram of a second embodiment of the present invention.

[Explanation of symbols]

 10: Interactive camera 12: Photographing lens 12a: Zooming lens 12b: Focusing lens 14: Image sensor 16: Camera process circuit 18: Focus control circuit 20: Overall control circuit 20a: Overall control circuit 22: Moving object detection circuit 23: Distance Measurement circuit 24: Bus interface 26: Magnification setting circuit 28: Zoom control circuit 30: Computer 30a: Image recording device 32: CPU 34: ROM 36: RAM 38: Secondary storage device 40: Monitor 42: Mouse 44: Keyboard 46 : Video capture device 48: Network interface 50: Bus interface 60: Video capture device 62: Image compression encoding device 64: Recording device

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Takashi Kobayashi 3-30-2 Shimomaruko, Ota-ku, Tokyo Inside Canon Inc. (72) Inventor Yoshiki Ishii 3-30-2 Shimomaruko, Ota-ku, Tokyo 5C052 AA17 AB02 DD02 GA02 GA08 GA09 GB06 GC05 5C059 KK00 MA00 MA23 MC14 ME02 PP01 PP14 RC12 RC14 SS15 SS20 TA47 TB04 TC00 TD05 UA02 UA40 5C078 AA04 CA02 DA00 DA01 DA11 DB07 9A001 EK04 EE04

Claims (13)

[Claims] An input unit for inputting image information and information on a focus detection target area and a subject distance thereof; and a focus detection target area indicated by the focus area information and another area according to the subject distance information. Encoding means for compressing and encoding the image information at different compression rates, and output means for outputting the image information compressed and encoded by the encoding means using the focus detection target area and the subject distance information as a header. An image encoding device, comprising: 2. The image encoding apparatus according to claim 1, wherein the compression encoding unit includes a quantization unit, and uses different quantization tables for the focus detection target area and other areas. 3. The image encoding apparatus according to claim 2, wherein said compression encoding means uses a quantization table for storing a high-frequency component in the focus detection target area as compared with other areas. 4. The image encoding apparatus according to claim 1, wherein the header includes information on a position and a size of a focus detection target area on a screen and a subject distance. 5. A capturing step for capturing image information and information on a focus detection target area and a subject distance, and a focus detection target area indicated by the focus area information and other areas according to the subject distance information. An encoding step of compressing and encoding the image information at different compression ratios, and an output step of outputting the image information compressed and encoded in the encoding step using the focus detection target area and the subject distance information as a header. An image encoding method comprising: 6. The image encoding method according to claim 5, wherein the compression encoding step includes a quantization step, and different quantization tables are used for the focus detection target area and other areas. 7. The image encoding method according to claim 6, wherein the compression encoding step uses a quantization table for storing a higher frequency component in the focus detection target area than in other areas. 8. The image encoding method according to claim 5, wherein the header includes information on a position and a size of a focus detection target area on a screen and a subject distance. 9. A storage medium storing program software for executing the image coding method according to claim 5. 10. An imaging means for outputting photographed image information and information on a focus detection target area and a subject distance, and a focus detection target area indicated by the focus area information according to the subject distance information. Encoding means for compressing and encoding the photographed image information at a different compression ratio in the area of; and, using the focus detection target area and the subject distance information as a header, the photographed image information compressed and encoded by the encoding means. An image pickup apparatus comprising: an output unit that outputs an image. 11. The imaging apparatus according to claim 10, wherein the compression encoding unit includes a quantization unit, and uses different quantization tables for the focus detection target area and other areas. 12. The imaging apparatus according to claim 11, wherein the compression encoding unit uses a quantization table for storing a high-frequency component in the focus detection target area as compared with other areas. 13. The imaging apparatus according to claim 10, wherein the header includes information on a position and a size of a focus detection target area on a screen and a subject distance.