JP2001333430A - Image processing unit, method, and computer-readable storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing unit that can reproduce a moving picture and only part of consecutively shot still pictures and reproduce and display the image through magnification without the need for hardware such as a high-speed CPU. SOLUTION: In the case of expanding only a designated area by an instruction of a user, a system inverse quantization section 8 expands only an area designated by an area instruction section 11 at coding, and e.g. an inverse discrete wavelet transform section 9 substitutes a white or black pixel for an image signal in an area not decoded to allow only a star-shaped area part designated at coding to be decoded and to allow the other parts to be displayed as a white or black level, or substitutes an invalid value for the area not decoded as an image signal and segments and displays the part including the designated area.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing apparatus and method for encoding and decoding image data, and a computer-readable storage medium.

[0002]

2. Description of the Related Art Conventionally, video cameras and digital cameras having a moving image shooting function and a still image continuous shooting function have been proposed. In addition, video and digital cameras and personal computers have been used to reproduce moving images and continuous still images shot by these devices.

[0003]

However, moving images and continuously shot still images are usually compressed and stored. When these images are displayed in full frame and the entire image is reproduced and displayed, the compressed Must be decompressed, and hardware such as a high-speed CPU is required.

Also, the display device of a video camera or a digital camera is small, and there is a problem that, when trying to display the entire image, the main subject becomes small, making it difficult to determine.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems. By enabling reproduction of only a part of a moving image or a continuously shot still image, hardware such as a high-speed CPU can be implemented. It is not required, and it is another object of the present invention to enable reproduction and display of an enlarged image.

[0006]

An image processing apparatus according to the present invention is an image processing apparatus for encoding moving picture data or continuously shot still picture data, wherein each frame of the moving picture or the continuously shot picture is encoded. It is characterized in that it comprises an area designating means for designating a specific area for each frame of the still image, and an encoding means for encoding the specific area at a different compression ratio.

Another feature of the image processing apparatus of the present invention is that the moving image data or the continuously shot still image data is encoded by discrete wavelet transform.

An image processing apparatus according to the present invention is an image processing apparatus for decoding moving image data or continuously shot still image data in which a specific area is compressed at different compression ratios.
Decoding means for decoding only the specific area, and display for performing processing for continuously displaying only the decoded part on each frame of the moving image or each frame of the continuously shot still image It is characterized by having a processing means.

Another feature of the image processing apparatus of the present invention is that the moving picture data or the continuously shot still picture data is decoded by inverse discrete wavelet transform.

An image processing apparatus according to the present invention is an image processing apparatus for encoding and decoding moving image data or continuously shot still image data, wherein each frame of the moving image or the continuously shot still image is encoded. Area designating means for designating a specific area for each frame, encoding means for encoding the specific area at a different compression rate, decoding means for decoding only the specific area, Display processing means for continuously displaying only the converted portion on each frame of the moving image or each frame of the continuously shot still image.

According to another feature of the image processing apparatus of the present invention, the moving image data or the continuously shot still image data is encoded by a discrete wavelet transform and decoded by an inverse discrete wavelet transform. It is in a point.

According to another feature of the image processing apparatus of the present invention, the specific area encoded at the different compression ratio is an area including a focus position of the moving image or the continuously shot still image. It is in the point that is.

Another feature of the image processing apparatus according to the present invention is that the image processing apparatus further includes a visual line detecting means for detecting a visual line of a photographer,
The specific region encoded at the different compression ratio is a region including a line-of-sight position detected by the line-of-sight detection unit on the moving image or the continuously shot still image.

[0014] Another feature of the image processing apparatus of the present invention is that the encoding means decodes only the specific area and that the entirety of each frame of the moving image data or the continuous shooting is performed. That is, it is possible to selectively switch between decoding the entire frame of the still image.

Another feature of the image processing apparatus of the present invention is that the display means includes a portion including the specific area in each frame of the moving image data or each frame of the continuously shot still images. Only and the entirety of each frame of the moving image data or the entirety of each frame of the continuously shot still image can be selectively switched.

An image processing method according to the present invention is an image processing method for encoding moving image data or continuously shot still image data, wherein each frame of the moving image or each frame of the continuously shot still image is encoded. The method is characterized in that it has an area designating process for designating a specific region and a process of encoding the specific region at a different compression ratio.

An image processing method according to the present invention is an image processing method for decoding moving image data or continuously shot still image data in which a specific area is compressed at a different compression ratio,
In that the method includes a process of decoding only the specific region and a process of continuously displaying only the decoded portion on each frame of the moving image or each frame of the continuously shot still image. Has features.

An image processing method according to the present invention is an image processing method for encoding and decoding moving image data or continuously shot still image data, wherein each frame of the moving image or the continuously shot still image is encoded. A process of setting a specific region for each frame, a process of encoding the specific region at a different compression ratio, a process of decoding only the specific region, and a process of decoding only the decoded portion. Displaying continuously for each frame of the moving image or each frame of the continuously shot still image.

The computer-readable storage medium of the present invention is characterized in that a program for causing a computer to function as each of the above means is stored.

The computer-readable storage medium of the present invention is characterized in that a program for executing each of the above-described processes is stored.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of an image processing apparatus, a method, and a computer-readable storage medium according to the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing the configuration of the image processing apparatus 100. Reference numeral 10 denotes a photographing lens, 12 denotes a shutter having an aperture function, and 14 denotes an image sensor that converts an optical image into an electric signal. An A / D converter 16 converts an analog signal output of the image sensor 14 into a digital signal. Reference numeral 18 denotes a timing generation circuit that supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26, and is controlled by the memory control circuit 22 and the system control circuit 50. .

Reference numeral 20 denotes an image processing circuit which performs predetermined pixel interpolation processing and color conversion processing on data from the A / D converter 16 or data from the memory control circuit 22. In the image processing circuit 20, predetermined arithmetic processing is performed using the captured image data, and the system control circuit 50 controls the exposure control means 40 and the distance measurement control means 42 based on the obtained arithmetic results. TTL (through-the-lens) AF (autofocus) processing, AE
(Automatic exposure) processing and EF (flash pre-emission) processing are performed. Further, the image processing circuit 20 performs predetermined arithmetic processing using the captured image data, and also performs TTL AW (auto white balance) processing based on the obtained arithmetic result.

Reference numeral 22 denotes a memory control circuit, which includes an A / D converter 16, a timing generation circuit 18, an image processing circuit 20,
Image display memory 24, D / A converter 26, memory 30,
The compression / decompression circuit 32 is controlled. The data of the A / D converter 16 is sent to the image display memory 24 or the memory 30 via the image processing circuit 20 and the memory control circuit 22 or the data of the A / D converter 16 is sent directly to the image display memory 24 or the memory 30 via the memory control circuit 22. Written.

Reference numeral 24 denotes an image display memory; 26, a D / A converter; and 28, an image display unit comprising a TFT LCD or the like. The display image data written in the image display memory 24 is displayed by the image display unit 28 via the D / A converter 26.

If the captured image data is sequentially displayed using the image display section 28, an electronic finder function can be realized. Further, the image display unit 28 can arbitrarily turn off the display by 0N / OFF according to an instruction from the system control circuit 50. When the display is turned off, the power consumption of the image processing apparatus 100 can be significantly reduced. Can be. Further, the image display unit 28 is connected to the image processing apparatus 100 main body by a rotatable hinge unit, and can be freely set to any direction and angle to use the electronic finder function, the reproduction display function, and various display functions. It is possible. Further, it is possible to store the display portion of the image display unit 28 toward the image processing apparatus 100. In this case, the storage state is detected by an image display unit open / close detection unit (not shown).
The display operation of the image display unit 28 can be stopped.

Reference numeral 30 denotes a memory for storing photographed still images and moving images, which has a sufficient storage capacity to store a predetermined number of still images and moving images for a predetermined time. Accordingly, even in the case of continuous shooting or panoramic shooting in which a plurality of still images are continuously shot, it is possible to write a large amount of images to the memory 30 at high speed. Further, the memory 30 can also be used as a work area of the system control circuit 50.

Numeral 32 denotes DWT (discrete wavelet transform)
Is a compression / expansion circuit for compressing / expanding image data by reading the image stored in the memory 30 to perform a compression process or an expansion process, and storing the processed data in the memory 3
Write to 0. This compression / expansion circuit can perform compression / expansion processing of not only still images but also moving images and continuously shot still images. In the case of moving images or continuously shot still images, compression / expansion is performed by applying DWT to each frame, for example, in the same manner as for still images. Furthermore, as described later, only pixels included in a specific region (ROI: Region Of Interesting) of the image can be expanded. As a result, only the images included in the area can be displayed on the image display unit 28. Compression / expansion circuit 3 in this way
The operation 2 will be described later in detail.

Reference numeral 40 denotes an exposure control means for controlling the shutter 12 having an aperture function, which also has a flash dimming function in cooperation with the flash 48. 4
Reference numeral 2 denotes a distance measurement control unit that controls focusing of the photographing lens 10. The exposure control means 40 and the distance measurement control means 42 are controlled using the TTL method, and the system control circuit 50 controls the exposure control means 40 and the distance measurement based on the calculation result obtained by calculating the captured image data by the image processing circuit 20. The control unit 42 is controlled.

Reference numeral 44 denotes a zoom control unit for controlling zooming of the photographing lens 10, and 46 denotes a protection unit 10 as a barrier.
Reference numeral 48 denotes a barrier control unit for controlling the operation of No. 2, and a flash unit.

Reference numeral 50 denotes a system control circuit for controlling the entire image processing apparatus 100, and reference numeral 52 denotes a memory for storing constants, variables, programs, and the like for operating the system control circuit 50.

Reference numeral 54 denotes a display unit such as a liquid crystal display device or a speaker, which displays an operation state, a message, and the like using characters, images, sounds, and the like in accordance with execution of a program in the system control circuit 50. The display unit 54 displays the image processing device 100
A single or a plurality of units are installed at easy-to-recognize positions near the operation unit, and are constituted by, for example, a combination of LCDs, LEDs, sound emitting elements, and the like. The display unit 54 has a part of its functions installed in the optical finder 104.

Of the display contents of the display unit 54, those displayed on the LCD or the like include a single shot / continuous shooting display, a self-timer display, a compression ratio display, a recording pixel number display, a recording number display, and a remaining number of recordable images. Display, shutter speed display, aperture value display, exposure compensation display, flash display, red-eye reduction display, macro shooting display, buzzer setting display,
There are a remaining battery level display for a clock, a remaining battery level display, an error display, an information display using a plurality of digits, a detachable state display of the recording media 200 and 210, a communication I / F operation display, a date / time display, and the like.

Of the display contents of the display unit 54, those displayed in the optical viewfinder 104 include a focus display, a camera shake warning display, a flash charge display, a shutter speed display, an aperture value display, and an exposure correction display. is there.

Reference numeral 56 denotes an electrically erasable and recordable nonvolatile memory, for example, an EEPROM or the like. 5
Reference numeral 7 denotes a line-of-sight detection unit,
By detecting the line of sight when looking into the object 4, it is possible to determine which subject the operator is looking at.

60, 62, 64, 66, 68 and 70
Is an operation means for inputting various operation instructions of the system control circuit 50, and is constituted by one or a combination of a switch, a dial, a touch panel, pointing by gaze detection, a voice recognition device, and the like.

Reference numeral 60 denotes a mode dial switch, which can switch and set various function modes such as power-off, automatic photographing mode, photographing mode, panoramic photographing mode, reproducing mode, multi-screen reproducing / erasing mode, and PC connection mode.

Reference numeral 62 denotes a shutter switch SW1, which is set to 0N during the operation of a shutter button (not shown) to perform AF (auto focus) processing, AE (auto exposure) processing, and AWB.
An instruction to start operations such as (auto white balance) processing and EF (flash pre-emission) processing is given. Reference numeral 64 denotes a shutter switch SW2, which becomes 0N when the operation of a shutter button (not shown) is completed, and outputs a signal read from the image sensor 14 to A / A.
The memory 30 via the D converter 16 and the memory control circuit 22
Exposure processing of writing image data to the memory, development processing using the operation of the image processing circuit 20 and the memory control circuit 22, reading of image data from the memory 30, compression by the compression / decompression circuit 32, and It instructs to start the operation of a series of processing of recording data.

Reference numeral 66 denotes an image display 0N / OFF switch.
ON / OFF of the image display unit 28 can be set. With this function, when photographing using the optical viewfinder 104, the image display unit 2 composed of a TFT LCD or the like is used.
By cutting off the current supply to the power supply 8, power can be saved.

Reference numeral 68 denotes a single / continuous shooting switch, which is a single shooting mode in which a single frame is shot when the shutter switch SW2 is pressed and a standby state is set, and continuous shooting is performed while the shutter switch SW2 is pressed. A continuous shooting mode can be set.

Reference numeral 70 denotes an operation unit including various buttons, a touch panel, and the like. A menu button, a set button, a macro button, a multi-screen playback update button, a flash setting button, a single-shot / continuous-shot / self-timer switching button,
Menu move + (plus) button, menu move-(minus) button, playback image move + (plus) button, playback image-(minus) button, shooting image quality selection button, exposure compensation button, date / time setting button, panorama mode A selection / switch button for setting selection and switching of various functions when executing shooting and playback of a camera, etc., a button for starting audio recording, and determining and executing various functions when executing shooting and playback of a panorama mode and the like. OK / OFF button for setting the OK / OFF button for setting 0N / 0FF of the image display unit 28, a quick review ON / OFF switch for setting a quick review function for automatically playing back image data immediately after shooting, JPEG To select the compression ratio of the compression or by digitizing the image sensor signal as it is Compression mode switch, which is a switch for selecting a CCD RAW mode for recording on a recording medium, a reproduction mode switch for setting various function modes such as a reproduction mode, a multi-screen reproduction / deletion mode, and a PC connection mode, and a shooting mode state , A playback switch for instructing the start of a playback operation for reading a captured image from the memory 30 or the recording medium 200 and displaying the image on the image display unit 28, a drive button for changing the active drive, There are a reproduction display switching button for switching between image display and an information display button for displaying incidental information of a recorded image.

Numeral 80 denotes a power control means, a battery detection circuit, D
It comprises a C-DC converter, a switch circuit for switching a block to be energized, and the like, and detects the presence or absence of a battery, the type of battery, the remaining battery level, and performs DC based on the detection result and an instruction from the system control circuit 50. Control the DC converter to supply a required voltage to each unit including the recording medium for a required period.

Reference numerals 82 and 84 denote connectors, and reference numeral 86 denotes a primary battery such as an alkaline battery or a lithium battery, a NiCd battery or a NiMd battery.
It is a power supply unit including a secondary battery such as an H battery and a Li battery, an AC adapter, and the like.

Reference numeral 90 denotes an interface with a recording medium such as a memory card or a hard disk; 92, a connector for connecting to a recording medium such as a memory card or a hard disk;
Reference numeral 8 denotes a recording medium attachment / detachment detecting unit that detects whether the recording medium 200 is attached to the connector 92.

Although the present embodiment has been described as having one interface and connector for attaching a recording medium, these interfaces and connectors may have a single or plural number of systems. In addition, a configuration may be adopted in which interfaces and connectors of different standards are provided in combination. PCMC for interface and connector
What is necessary is just to comprise using what conformed to standards, such as IA card and CF (CompactFlash (trademark)) card. Further, when the interface 90 and the connector 92 are configured using a standard such as a PCMCIA card or a CF (compact flash) card, a LAN card, a modem card, a USB card,
By connecting various communication cards such as communication cards such as an IEEE 1394 card, a P1284 card, a SCSI card, and a PHS, image data and management information attached to the image data can be transferred to and from other peripheral devices such as a computer and a printer. You can compete with each other.

Reference numeral 102 denotes a protection unit, and the image processing apparatus 1
By covering the imaging unit including the imaging lens 10 of 00,
This is a barrier for preventing the imaging unit from being stained or damaged. Reference numeral 104 denotes an optical finder, which enables photographing using only the optical finder without using the electronic finder function of the image display unit 28. In the optical viewfinder 104, some functions of the display unit 54, such as a focus display, a camera shake warning display, a flash charge display,
A shutter speed display, an aperture value display, an exposure correction display, and the like are provided.

Reference numeral 110 denotes communication means, such as RS232C or US
It has various communication functions such as B, IEEE1394, P1284, SCSI, modem, LAN, and wireless communication. 11
Reference numeral 2 denotes a connector for connecting the image processing apparatus 100 to another device by the communication unit 110 or an antenna in the case of wireless communication.

Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium 200 includes a recording unit 202 including a semiconductor memory and a magnetic disk, an interface 204 with the image processing apparatus 100, and a connector 206 for connecting to the image processing apparatus 100.

Next, the operation of the compression / expansion circuit 32 will be described in detail. In the following, a compression / expansion method for one still image will be described. In the case of a moving image or a continuously shot still image, for example, as described above, for each frame, for example, the same as the still image Compression / expansion is possible by applying the above DWT.

First, the operation of the compression circuit will be described.
The compression circuit is configured as shown in FIG. In FIG. 2, 1 is an image input unit, 2 is a discrete wavelet transform unit,
Reference numeral 3 denotes a quantization unit, 4 denotes an entropy coding unit, 5 denotes a code output unit, and 11 denotes an area designation unit.

Pixel signals constituting an image to be encoded are input to the image input unit 1 in raster scan order, and the output is input to the discrete wavelet transform unit 2. The image input unit 1 corresponds to the memory 30 in FIG. In the following description, the image signal represents a monochrome multivalued image. However, if a plurality of color components such as a color image are encoded, the RGB color components, or the luminance and chromaticity components are used as the single color components. Just compress it.

The discrete wavelet transform unit 2 performs a two-dimensional discrete wavelet transform process on the input image signal, calculates a transform coefficient, and outputs the result. FIG.
2A shows the basic configuration of the discrete wavelet transform unit 2. The input image signals are stored in a memory 201, sequentially read out by a processing unit 202, subjected to a conversion process, and stored in the memory 201 again. Written.

In this embodiment, the configuration for processing in the processing section 202 is as shown in FIG. In the figure, an input image signal is separated into a signal of an even address and a signal of an odd address by a combination of a delay element and a downsampler, and is subjected to filter processing by two filters p and u. s and d are each 1
A low-pass coefficient and a bypass coefficient when one-level decomposition is performed on a dimensional image signal are calculated by the following equation.

D (n) = x (2 * n + 1) -floor ((x (2 * n) + x (2 * n + 2)) / 2) (1) s (n) = x (2 * n) + floor ((d (n-1) + d (n)) / 4) (2) where x (n) is an image signal to be converted.

With the above processing, one-dimensional discrete wavelet transform processing is performed on the image signal. In the two-dimensional discrete wavelet transform, one-dimensional transform is sequentially performed in the horizontal and vertical directions of an image, and details thereof are publicly known, and thus description thereof is omitted here.

FIG. 3C shows a configuration example of a two-level transform coefficient group obtained by a two-dimensional transform process. The image signal is composed of coefficient sequences HH1, HL1, LH of different frequency bands.
Decomposed into 1, ..., LL. In the following description, these coefficient sequences are called subbands. The coefficients of each subband are output to the quantization unit 3.

Returning to FIG. 2, the area designating section 11 decodes an area (ROI: Region Of Interesting) to be decoded with higher image quality compared to the surrounding area in the image to be encoded.
Is determined, and mask information indicating which coefficient belongs to the designated area when the target image is subjected to the discrete wavelet transform is generated.

A method for designating an area will be described. First, based on information from the distance measurement control means 42, it is checked which position of the image is in focus. This is because, in the case of a device having a plurality of distance measuring points, there is a possibility that the image is focused on a portion other than the center of the image. Thereafter, a predetermined shape such as a rectangle or a circle centered on the in-focus position is designated as the above-mentioned area.

Alternatively, an edge of an image may be calculated, and an area having an edge of a certain range of size may be traced from the in-focus position to obtain an area of the main subject, and the area may be designated. Alternatively, the line of sight of the photographer may be detected by the line of sight detecting means 57, and a fixed shape such as a rectangle or a circle centered on the position of the image where the line of sight of the photographer is located may be designated as the area. Further, the edge of the image may be traced from the line-of-sight position, the area of the main subject may be determined, and the area may be designated. With either of these techniques,
A region including a main subject in an image to be encoded with high image quality is designated.

FIG. 4A shows an example when generating mask information. As shown on the left side of the figure, when a star-shaped region in an image is designated by the region designation unit 11, the region designation unit 11 performs discrete wavelet transform on the image including the designated region, and the designated region becomes Calculate the portion occupied by each subband. Further, the area indicated by the mask information is necessary when restoring the image signal on the specified area boundary.
The range includes the surrounding conversion coefficients.

An example of the mask information calculated in this way is shown on the right side of FIG. In this example, mask information when a two-level discrete wavelet transform is performed on the image on the left side of the figure is calculated as shown in the figure. In the figure, the star-shaped portion is a designated area, and the bits of the mask information in this area are 1 and the bits of the other mask information are 0. Since the entire mask information is the same as the configuration of the transform coefficient by the two-dimensional discrete wavelet transform, it is possible to identify whether or not the coefficient at the corresponding position belongs to the designated area by inspecting the bits in the mask information. it can. The mask information generated in this way is used by the quantization unit 3
Is output to

Further, the area designating section 11 has a parameter for designating the image quality for the designated area. The parameter may be a numerical value representing a compression ratio to be assigned to the designated area or a numerical value representing image quality. The area specifying unit 11 calculates the bit shift amount B for the coefficient in the specified area from the parameter, and outputs the calculated bit shift amount B to the quantization unit 3 together with the mask information.

The quantizing section 3 quantizes the input coefficient by a predetermined quantization step and outputs an index for the quantized value. Here, the quantization is performed by the following equation.

Q = sign (c) floor (abs (c) / Δ) (3) sign (c) = 1; c ≧ 0 (4) sign (c) = − 1; c <0 (5) )

Here, c is a coefficient to be quantized. In the present embodiment, it is assumed that the value of Δ includes 1. In this case, no quantization is actually performed.

Next, the quantization unit 3 changes the quantization index according to the following equation based on the mask information and the bit shift amount B input from the area designation unit 11.

Q ′ = q * 2 ^ B; m = 1 (6) q ′ = q; m = 0 (7)

Here, m is the value of the mask at the position of the quantization index. By the above processing, only the quantization index belonging to the space area specified by the area specifying unit 11 is shifted up by B bits.

FIGS. 4B and 4C show the change of the quantization index due to the above shift-up. In FIG. 4B, when three quantization indices exist in each of the three subbands, and the value of the mask in the networked quantization index is 1 and the shift number B is 2, The quantization index is as shown in FIG. The quantization index changed in this way is output to the entropy encoding unit 4.

The entropy coding unit 4 decomposes the input quantization index into bit planes, performs binary arithmetic coding on a bit plane basis, and outputs a code stream. FIG. 5 is a diagram for explaining the operation of the entropy encoding unit 4. In this example, a non-zero quantization index is 3 in a region within a subband having a size of 4 × 4.
And has values of +13, -6, and +3, respectively. The entropy coding unit 4 scans this area to find the maximum value M, and calculates the number of bits S required to represent the maximum quantization index by the following equation.

S = ceil (log2 (abs (M))) (8)

Here, ceil (x) represents the smallest integer value among the integers greater than or equal to x. In FIG. 5, since the maximum coefficient value is 13, S is 4, and the 16 quantization indexes in the sequence are processed in units of four bit planes as shown in FIG. 5B. .

First, the entropy coding unit 4 sets each bit of the most significant bit plane (represented by the MSB in FIG. 5) to 2 bits.
Value arithmetic coding and output as bit stream. Next, the bit plane is lowered by one level, and similarly, each bit in the bit plane is encoded and output to the code output unit 5 until the target bit plane reaches the least significant bit plane (represented by LSB in FIG. 5). At this time, the code of each quantization index is entropy-encoded immediately after the first non-zero bit is detected in the bit plane scanning.

FIG. 6 is a schematic diagram showing the structure of a code string generated and output in this way. FIG. 6A shows the entire structure of the code string, where MH is a main header, TH is a tile header, and BS is a bit stream.

As shown in FIG. 6 (b), the main header MH has the image size (the number of pixels in the horizontal and vertical directions) of the image to be encoded, and the image when the image is divided into a plurality of rectangular areas as tiles. , The number of components representing the number of each color component, the size of each component, and component information representing the bit precision. In the present embodiment, since the image is not divided into tiles, the tile size and the image size take the same value, the target image is a monochrome multi-valued image, and the number of components is 1 when the image is an image.

As shown in FIG. 6C, the tile header TH includes a tile length including a bit stream length and a header length of the tile, an encoding parameter for the tile,
It consists of a mask indicating the designated area and the number of bit shifts for the coefficients belonging to the area. The coding parameters include the level of the discrete wavelet transform, the type of filter, and the like.

FIG. 6D shows the configuration of a bit stream according to the present embodiment. In the figure, bit streams are grouped for each sub-band, and are arranged in order of increasing resolution starting with the sub-band having the smaller resolution. Further, in each subband, codes are arranged in units of bit planes from the upper bit plane to the lower bit plane. With the above code arrangement, hierarchical decoding as described later with reference to FIG. 10 can be performed.

In the above-described embodiment, the compression rate of the entire image to be encoded can be controlled by changing the quantization step Δ.

As another method, it is also possible to limit (discard) the lower bits of the bit plane to be coded in the entropy coding unit 4 according to the required compression ratio. In this case, all the bit planes are not coded, and the bits from the upper bit plane to the bit planes corresponding to the desired compression ratio are coded and included in the final code string.

When the function of limiting the lower bit plane is used, only bits corresponding to the designated area shown in FIG. 4 are included in the code string in a large number. That is, only the designated area has a low compression rate and high image quality. It can be encoded as a simple image.

Next, the operation of the decoding circuit for decoding a bit stream by the above-described compression circuit will be described. The decoding circuit is configured as shown in FIG. In FIG. 7, 6 is a code input unit, 7 is an entropy decoding unit, 8
Is an inverse quantization unit, 9 is an inverse discrete wavelet transform unit, 10
Denotes an image output unit.

The code input section 6 inputs a code string constituting an image to be decompressed, analyzes a header included in the code string, extracts parameters necessary for the subsequent processing, and controls the flow of the processing when necessary, or The corresponding parameter is sent to the subsequent processing unit. Further, the bit stream included in the code string is output to the entropy decoding unit 7.

The entropy decoding unit 7 decodes the bit stream in bit plane units and outputs the result. FIG. 8 shows the decoding procedure at this time. FIG. 8A illustrates a flow of sequentially decoding one area of a sub-band to be decoded in units of bit planes and finally restoring a quantization index. Decrypted. The restored quantization index is output to the inverse quantization unit 8.

The inverse quantization unit 8 restores discrete wavelet transform coefficients from the input quantization index based on the following equation.

C ′ = Δ * q / 2 ^ U; q ≠ 0 (9) c ′ = 0; q = 0 (10) U = B; m = 1 (11) U = 0; m = 0 ... (12)

Here, q is a quantization index, Δ is a quantization step, and Δ is the same value used at the time of encoding. B is the number of bit shifts read from the tile header, and m is the value of the mask at the position of the quantization index. c ′ is a restored transform coefficient, which is a restored coefficient represented by S or d at the time of encoding. The transform coefficient c ′ is output to the inverse discrete wavelet transform unit 9.

FIG. 9A is a diagram showing the configuration of the inverse discrete wavelet transform unit 9.
01 is stored. The processing unit 902 performs a two-dimensional inverse discrete wavelet transform by performing a one-dimensional inverse discrete wavelet transform, sequentially reading transform coefficients from the memory 901 and performing processing.

The two-dimensional inverse discrete wavelet transform is executed in a procedure reverse to the forward transform, but the details are well known and will not be described. FIG. 9B shows the processing unit 90.
2 shows a configuration for processing in which the input transform coefficients are subjected to two filter processes of u and p, are upsampled and then superimposed to output an image signal x ′. These processes are performed by the following equations.

X ′ (2 * n) = S ′ (n) −floor ((d ′ (n−1) + d ′ (n)) / 4) (13) x ′ (2 * n + 1) = d ′ (n) + floor ((x '(2 * n) + x' (2 * n + 2)) / 4) (14)

Here, equations (1), (2) and (1)
Since the discrete wavelet transform in the forward and backward directions according to 3) and (14) satisfies the perfect reconstruction condition, the quantization step Δ is 1 in the present embodiment, and all bit planes are decoded in the bit plane decoding. Have been decoded, the restored image signal x 'is the signal x of the original image.
Matches.

Through the above processing, the image is restored and output to the image output unit 10. The image input unit 10 corresponds to the memory 30 in FIG.

A display mode of an image when an image is restored and displayed according to the above-described procedure will be described with reference to FIG.
FIG. 10A shows an example of a code string. The basic configuration is based on FIG. 6, but the entire image is set as a tile. Therefore, only one tile is included in the code string. A header and a bit stream are included.

As shown in the figure, the bit stream BS0 has a subband L corresponding to the lowest resolution.
The codes are arranged in order of increasing resolution from L, and the codes are arranged in each subband from the upper bit plane to the lower bit plane.

The decoding circuit sequentially reads the bit stream and displays an image when the code corresponding to each bit plane is decoded. FIG. 10B shows the correspondence between each subband and the size of the displayed image, and the change of the image accompanying decoding of the code string in the subband.
In the figure, the code sequence corresponding to LL is sequentially read, and the image quality is gradually improved as the decoding process of each bit plane progresses. At this time, the star-shaped part which has become the designated area at the time of encoding is restored with higher image quality than other parts.

This is because the quantization unit 3 shifts up the quantization index belonging to the specified area at the time of encoding, so that the quantization index is more effective than the other parts during bit plane decoding. This is because it is decoded at an early point. The reason why the designated area portion is decoded with high image quality is the same for other resolutions.

Further, when all the bit planes are decoded, the designated area and other parts have the same image quality. However, when the decoding is terminated in the middle, the designated area is higher than the other areas. An image restored to the image quality is obtained.

In the above-described embodiment, the amount of coded data to be received or processed is reduced by limiting (ignoring) the lower bit planes to be decoded in the entropy decoding unit 7, and consequently the compression ratio is controlled. Is possible. In this way, it is possible to obtain a decoded image of a desired image quality only from the encoded data of a necessary data amount. When the quantization step Δ at the time of encoding is 1 and all the bit planes are decoded at the time of decoding, lossless encoding / decoding in which the restored image matches the original image can be realized. .

If the function of limiting the lower bit plane is used, the code string to be decoded contains only bits corresponding to the designated area shown in FIG. 4 more than the other areas. As a result, an effect similar to that obtained by decoding data encoded as a high-quality image with a low compression ratio only in the specified area is obtained.

Also, by decoding only the area where the value of the mask is 1 in the inverse quantization section 8, only the area specified by the area specifying section 11 at the time of encoding can be expanded. For the region not to be decoded, the inverse discrete wavelet transform unit 9 determines whether a white pixel (all 255 bits for each of 8 bits of RGB) or a black pixel (8 pixels of each of RGB) is used.
In the case of a bit, all 0) are substituted for the image signal.

Switching between expansion of the entire image or expansion of only the ROI area is performed by the system control circuit 50 in response to a user's instruction using the operation unit 70 in FIG.

At this time, the display mode of the image is shown in FIG.
This will be described with reference to FIG. Basically, each bit plane is decoded and displayed in the same manner as in FIG. 10B. However, since only the star-shaped area specified at the time of encoding is decoded, Other parts are displayed as white or black.

In the area that is not decoded by the inverse discrete wavelet transform unit 9, an invalid value is used as an image signal (for example, in the case of 8 bits for RGB, all values are -1).
Can also be substituted. In this case, a region of a pixel having a valid value is determined from the image sent to the image output unit 10 according to an instruction from the system control circuit 50, and a rectangle circumscribing the region is cut out as an image. Then, for a pixel included in the rectangle and having an invalid value, a white pixel (all 255 bits for RGB each) or a black pixel (all 0 for RGB each 8 bits) Is substituted for the image signal.

The displayed image at this time is, for example, as shown in FIG.
As shown in 2, only a rectangular portion circumscribing the star-shaped area specified at the time of encoding is obtained. Areas not included in the star shape are displayed as white or black. If the aspect ratio of the rectangle circumscribing the area specified at the time of encoding is different from the aspect ratio of the display area of the image display unit 28, the white or black Fill pixels.

By decoding only a specific area in this way, the inverse discrete wavelet transform does not need to be performed for the other areas, so that the image can be expanded and displayed at high speed. In addition, by displaying only a specific area on the display device, an image can be greatly enlarged and displayed.

(Other Embodiments) In order to operate various devices for realizing the functions of the above-described embodiments, an apparatus connected to the various devices or a computer in a system is used in the above-described embodiments. Supply the software program code to realize the functions of
The present invention also includes those implemented by operating the various devices according to programs stored in a computer (CPU or MPU) of the system or apparatus.

In this case, the program code of the software implements the functions of the above-described embodiment, and the program code itself and means for supplying the program code to a computer are provided.
For example, a recording medium storing such a program code constitutes the present invention. As a recording medium for storing such a program code, for example, a floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, C
D-ROM, magnetic tape, nonvolatile memory card, R
OM or the like can be used.

When the computer executes the supplied program code, not only the functions of the above-described embodiment are realized, but also the OS (operating system) running on the computer or other It goes without saying that such program codes are also included in the embodiments of the present invention when the functions of the above-described embodiments are realized in cooperation with the application software or the like.

Further, after the supplied program code is stored in the memory provided in the function expansion board of the computer or the function expansion unit connected to the computer, the function expansion board or the function expansion unit is specified based on the instruction of the program code. It is needless to say that the present invention also includes a case where the CPU or the like provided for performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

The shapes and structures of the respective parts shown in the above-described embodiments are merely examples for embodying the present invention, and the technical scope of the present invention is thereby reduced. It should not be interpreted restrictively. That is, the present invention can be implemented in various forms without departing from the spirit or main features thereof.

[0110]

As described above, according to the present invention, only a part of a moving image or a continuously shot still image can be reproduced, so that a full-frame image can be reproduced in real time by a low-speed CPU or a processing device. Can be played.

Also, by reproducing only the area including the in-focus position or the line-of-sight position of the image, it is possible to reproduce only the subject important to the photographer.

Further, by reproducing only a part of the image on the display device, a part of the image is enlarged and displayed, so that the main subject can be clearly recognized even in a narrow display device.

Further, since not only a part of the image but also the entire image can be displayed, it is possible to selectively use the conventional reproducing method and the reproducing method of the present application.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of an image processing apparatus 100.

FIG. 2 is a diagram showing a configuration of a compression circuit in a compression / expansion circuit 32;

FIG. 3 is a diagram for explaining a discrete wavelet transform unit 2;

FIG. 4 is a diagram for explaining mask information and a quantization index.

FIG. 5 is a diagram for describing an operation of an entropy encoding unit 4.

FIG. 6 is a schematic diagram for explaining a configuration of a code string.

FIG. 7 is a diagram showing a configuration of a decoding circuit in the compression / decompression circuit 32.

FIG. 8 is a diagram for explaining an operation of an entropy decoding unit 7;

FIG. 9 is a diagram for explaining an inverse discrete wavelet transform unit 9;

FIG. 10 is a diagram illustrating a configuration of a code string and an example of a display mode of an image.

FIG. 11 is a diagram showing an example of a display mode of an image.

FIG. 12 is a diagram showing an example of a display mode of an image.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 image processing device 28 image display unit 32 compression / expansion circuit 1 image input unit 2 discrete wavelet transform unit 3 quantization unit 4 entropy encoding unit 5 code output unit 6 code input unit 7 entropy encoding unit 8 inverse quantization unit 9 Inverse discrete wavelet transform unit 10 Image output unit 11 Area designation unit

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C022 AA13 AB12 AB15 AB22 AB55 AB66 AB67 AC03 AC12 AC18 AC32 AC52 AC54 AC69 AC71 5C053 FA05 FA07 FA23 FA27 GA11 GB01 GB02 GB06 GB21 GB26 GB28 GB33 GB40 JA12 JA21 KA01 KA05 KA08 KA24 LA03 LA06 LA14 5C059 KK38 KK40 LA01 MA24 MA35 MB02 ME01 PP01 PP04 PP15 RC32 SS14 SS20 UA02 UA05 UA39 5C078 AA09 BA58 CA14 CA31 EA08 5J064 AA00 BA09 BA13 BA16 BC01 BC06 BC07 BC16 BD03 BD04

Claims (15)

[Claims] 1. An image processing apparatus for encoding moving image data or continuously shot still image data, wherein a specific area is specified for each frame of the moving image or each frame of the continuously shot still image. An image processing apparatus comprising: an area designating unit that designates a region; and an encoding unit that encodes the specific region at a different compression ratio. 2. The image processing apparatus according to claim 3, wherein the moving image data or the continuously shot still image data is encoded by discrete wavelet transform. 3. An image processing apparatus for decoding moving image data or continuously shot still image data in which a specific area is compressed at a different compression ratio, wherein the decoding is for decoding only the specific area And a display processing unit for performing a process of continuously displaying only the decoded portion on each frame of the moving image or each frame of the continuously shot still image. Image processing device. 4. The image processing apparatus according to claim 3, wherein the moving image data or the continuously shot still image data is decoded by an inverse discrete wavelet transform. 5. An image processing apparatus for encoding and decoding moving image data or continuously shot still image data, wherein each frame of the moving image or each frame of the continuously shot still image is Area specifying means for specifying a specific area; encoding means for encoding the specific area at a different compression rate; decoding means for decoding only the specific area; and only the decoded part An image processing apparatus comprising: a display processing unit configured to continuously display each frame of the moving image or each frame of the continuously shot still image. 6. The image according to claim 5, wherein the moving image data or the continuously shot still image data is encoded by a discrete wavelet transform and decoded by an inverse discrete wavelet transform. Processing equipment. 7. The specific area encoded at the different compression ratios is an area including a focus position of the moving image or the continuously shot still images.
An image processing apparatus according to claim 1. 8. A line-of-sight detecting means for detecting a line of sight of a photographer, wherein the specific area encoded at the different compression ratio is detected by the line-of-sight detecting means on the moving image or the continuously shot still image. The image processing apparatus according to claim 5, wherein the image processing apparatus is an area including a detected line-of-sight position. 9. The encoding unit according to claim 1, wherein the encoding unit decodes only the specific area, and decodes the entire frame of the moving image data or the entire frame of the continuously shot still image. The image processing apparatus according to any one of claims 3 to 8, wherein the image processing apparatus can be selectively switched. 10. The display processing means displays only a portion including the specific area in each frame of the moving image data or each frame of the continuously shot still image;
The display of the entire frame of the moving image data or the entire frame of the continuously shot still image is selectively switchable. An image processing apparatus according to claim 1. 11. An image processing method for encoding moving image data or continuously shot still image data, wherein a specific area is specified for each frame of the moving image or each frame of the continuously shot still image. An image processing method comprising: an area designating process for designating a specific region; and a process of encoding the specific region at a different compression ratio. 12. An image processing method for decoding moving image data or continuously shot still image data in which a specific area is compressed at a different compression ratio, wherein the processing includes decoding only the specific area. And a process of continuously displaying only the decoded portion on each frame of the moving image or each frame of the continuously shot still image. 13. An image processing method for encoding and decoding moving image data or continuously shot still image data, wherein each frame of the moving image or each frame of the continuously shot still image is A process of setting a specific region; a process of encoding the specific region at a different compression ratio; a process of decoding only the specific region; and only the decoded portion of each frame of the moving image or Continuously displaying each frame of the continuously shot still images. 14. A computer-readable storage medium storing a program for causing a computer to function as each of the means according to claim 1. Description: 15. A computer-readable storage medium storing a program for executing each of the processes according to claim 11. Description: