JP2002033904A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for processing of making transparent, without transmitting control parameters, separate from image data. SOLUTION: A detecting means (31) detects a control parameter embedded in first image data according to a prescribed format, and a compositing means (41) composites image from the first image data with an image from second image data, based on the detected control parameter. In this way, it is possible to detect the control parameter embedded in the image data, and to composite the image from the first image data with the image form the second image data based on the control parameter. Thus, it is possible to processing of making transparent, without transmitting control parameters, separate from the image data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing technique, and more particularly to a technique effective when applied to an MPEG4 video codec (encoding / decoding) LSI.

[0002]

2. Description of the Related Art As a method of superimposing two images to display one image, there are a complete transparency process and a translucent process.

[0003] The complete transparency process will be described.

When two images are superimposed and displayed, an image displayed on the side where the images overlap is referred to as an upper image, and an image not displayed is referred to as a lower image. Normally, the upper image is displayed in the area where the images overlap, but a complete transparency process is performed by making a part of the upper image transparent so that the lower image can be seen through.

[0005] The translucent processing will be described.

[0006] As the translucent treatment, an α blending method is known. This method has an effect of making the upper image semi-transparent and allowing the lower image to be transparent when two images are superimposed and displayed. For example, the pixel value to be displayed is obtained by adding the pixel values of the upper image and the lower image by the ratio α using the opacity α having a value in the range of 0 to 1.
For example, in a pixel where two images are superimposed,
Assuming that the pixel value of the upper image is a, the opacity α of the upper image is b, and the pixel value of the lower image is b, the pixel value c of this pixel value α after blending using blending is represented by the following equation.

[0007]

## EQU1 ## c = a × α + b × (1−α)

[0008] As an example of a document describing a method of performing image synthesis, there is "All of MPEG-4 (pages 60 to 60)" issued by the Industrial Research Council on September 30, 1998.

[0009]

In the complete transparency processing,
For example, as shown in FIG. 10, a shape to be clipped of the upper image is designated using a mask pattern (0 indicates a transparent area and 1 indicates a display area for one pixel), and the mask pattern is designated as By reference, a composite image with the lower image is generated. When the mask pattern is designated as a donut, the upper image is displayed in the above-described donut shape, and in other areas, the lower image is displayed.

In the translucent process, for example, as shown in FIG. 6, one α value indicating the opacity is set for one image, and the translucent process is performed at a uniform rate for the entire image. (First method). In this case, α
Since the values are uniform on the screen, the upper image and the lower image are blended at a fixed ratio in any pixel and displayed on the display means.

As shown in FIG. 7, it is conceivable to prepare α-value data for each pixel separately from the image data, and to use this to perform translucent processing at an independent rate for each pixel. (Second method). According to this method, by setting an α value for each pixel, (1)
Only the upper image is displayed (area of α = 1), (2) the upper image is made completely transparent and only the lower image is displayed (area of α = 0), and (3) the upper image is made semi-transparent and becomes the upper image. Three types of image display of an image obtained by combining the lower image (0 <α <1) can be performed independently for each pixel.

When performing image composition, control parameters are taken in separately from the upper image and the lower image to generate composite image data. The control parameter referred to here corresponds to a mask pattern in the above-described complete transparency processing, and corresponds to an α value in the semi-transparency processing.

In the present specification, "completely transparent processing" and "translucent processing" are collectively referred to as "transparency processing".

When performing the above-mentioned complete transparency processing, it is necessary to send 0/1 mask pattern data to the display unit in addition to the image data. The data amount of the mask pattern data increases in proportion to the image size.

In the case of performing the translucent processing, in the first method, since the degree of translucency of an image is uniform on a screen, one α value data is displayed for one image on a display unit. , And the data amount of the image hardly increases, but the transparency processing for each pixel cannot be performed. Further, in the second method, since the α value data is provided for each pixel, a flexible image composition can be performed by changing the value, but the data amount of the α value increases in proportion to the image size. Therefore, it is necessary to send a huge amount of α-value data separately from the image data.

It is an object of the present invention to provide a technique which enables a transparent process such as clipping of an image without transmitting a control parameter separately from image data.

Another object of the present invention is to provide a technique that enables α-blending processing for each pixel without transmitting control parameters separately from image data.

The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0019]

The following is a brief description of an outline of a typical invention among the inventions disclosed in the present application.

That is, an image based on the first image data;
When the image processing apparatus includes an image synthesizing unit that synthesizes an image based on the second image data, the image synthesizing unit detects a control parameter embedded in the first image data in a predetermined format. And synthesizing means for synthesizing an image based on the first image data and an image based on the second image data based on the detected control parameter.

According to the above means, the detecting means detects the control parameter embedded in the first image data in a predetermined format, and the synthesizing means detects the control parameter based on the detected control parameter. The image and the image based on the second image data are combined. Thus, the control parameter embedded in the image data is detected, and based on the control parameter, the image based on the first image data and the second
An image based on image data can be synthesized, which achieves transparency processing such as image clipping without transmitting control parameters separately from the image data.

At this time, the first image data has a range of pixel values not used due to band limitation, and data conversion for embedding the control parameters in the first image data using the range of pixel values. Means can be provided.

Also, an image based on the first image data and a second
When the image processing apparatus is configured to include an image synthesizing unit that synthesizes an image based on image data, the image synthesizing unit determines whether or not the luminance value of the input first image data belongs to a predetermined range. Brightness value detecting means for determining,
A luminance value conversion means for converting pixels having a luminance value not belonging to a predetermined range into non-conversion, and converting a pixel having a luminance value belonging to a predetermined range to luminance values of adjacent pixels, and a luminance value belonging to a predetermined range. Α indicates opacity for pixels without
For a pixel whose luminance value falls within a predetermined range, an α value generating means for obtaining an α value corresponding to the pixel value by referring to a predetermined table, an output signal of the luminance value converting means and the α value A synthesizing unit for performing a translucent process based on the output value of the generating unit.

According to the above means, the luminance value detecting means comprises:
It is determined whether or not the luminance value of the input first image data belongs to a predetermined range, and the luminance value conversion means performs no conversion for the pixel whose luminance value does not belong to the predetermined range, and sets the luminance value to The pixels belonging to the predetermined range are converted into luminance values of adjacent pixels, and the α value generating means sets the α value indicating opacity to 1 for pixels whose luminance values do not belong to the predetermined range, and sets the luminance value to 1. For pixels belonging to a predetermined range,
The α value corresponding thereto is obtained by referring to a predetermined table, and the image synthesizing unit performs a translucent process based on the output signal of the luminance value conversion unit and the output value of the α value generation unit. In this way, it is determined whether or not the luminance value of the input first image data belongs to a predetermined range. Pixels whose luminance value does not belong to the predetermined range are not converted, and the luminance value is Pixels belonging to the range are converted to luminance values of adjacent pixels. For pixels whose luminance values do not belong to the predetermined range, the α value indicating opacity is set to 1, and for pixels whose luminance values belong to the predetermined range, Is determined by referring to a predetermined table for the corresponding α value,
Translucent processing can be performed based on the output signal of the luminance value conversion means and the output value of the α value generation means, and this can be performed for each pixel without transmitting a control parameter separately from image data. Α blending process is achieved.

At this time, it is possible to provide a data conversion means for converting the luminance value of the pixel to be subjected to the translucency processing into a value obtained according to a predetermined table with reference to the α value indicating the opacity.

A means for decoding the MPEG4 object-based coded image data to obtain data to be converted by the data converter can be provided.

[0027]

FIG. 2 shows an MPEG4 video codec (encoding / decoding) LSI which is an example of an image processing apparatus according to the present invention.

An MPEG4 video codec (hereinafter, referred to as an MPEG4 video codec)
The LSI 100 (referred to as “MPEG4 codec”) is not particularly limited.
Video signal processing unit 106, memory interface 11
3 and a frame memory 114, which
17 are communicatively coupled to each other. Also, this MPEG4 codec LSI 100
Although not particularly limited, it is formed on one semiconductor substrate such as a single crystal silicon substrate by a known semiconductor integrated circuit manufacturing technique. When the MPEG4 codec LSI 100 is applied to, for example, a personal digital assistant, the MPEG4 codec
The MPEG4 codec LSI 1 is provided outside the 4 codec LSI 100 by executing a predetermined program.
CPU (central processing unit) 1 for controlling the operation of 00
10. Program ROM (Read Only Memory) for storing programs executed by CPU 110
111 and a program RAM (random access memory) 112 are connected.

The input / output interface unit 101 is not particularly limited, but includes a video interface 102, an audio interface 103, a serial and parallel port 104, and a video / graphics interface.
And a processor 105. The video interface 102 enables input and output of video signals to and from a video camera, a liquid crystal display, and the like (not shown). That is, a video signal obtained by shooting with a video camera (not shown) can be captured via the video interface 102, and the MPEG4 codec LSI1
The video signal processed in 00 can be output to an appropriate display means such as a liquid crystal display via the video interface 102. Also, this video
In the interface 102, as described later in detail, a process of synthesizing a plurality of images is performed based on the control parameters.
Video graphics processor 105 generates image data, such as a background. The image data generated here is transmitted to the video interface 102 and combined with another image. Audio interface 10
Numeral 3 enables input / output of a digital audio signal corresponding to the video signal. The serial port and the parallel port 104 enable input / output of various data in a serial format or a parallel format.

The video signal processing section 106 includes, but is not limited to, a controller 107, a DSP (digital signal processor) 108, a DCT / inverse DCT circuit 109, and a data conversion section 119. DSP10
8 are integrated, and perform motion compensation of an image by executing a SIMD type instruction. The controller 107
The operation of the DSP 108 is controlled. DCT / Inverse DCT
The circuit 109 performs, for example, discrete cosine transform in image compression processing when performing image data via the serial port and the parallel port 104, and inverse discrete cosine transform in image decompression processing. The data conversion unit 119 uses the MP
It includes means for decoding the EG4 object-based coded image, and data conversion means for embedding shape data in the image data at the time of decoding. Data conversion unit 1
The data conversion means in 19 refers to the shape data,
The brightness of the pixels in the region where the object to be displayed is present is kept as it is, and the brightness of the background portion to be made transparent is replaced with 0 which is a value outside the band limit to form control parameter embedded image data. In other words, since the value outside the band limit has no meaning as image data, the control parameter can be embedded by actively using it as a control parameter for image synthesis processing. The control parameter embedded image data is transmitted to the video interface 102 via the buses 116 and 117. As described above, since the control parameters for image synthesis are embedded in the image data and transmitted, it is not necessary to transmit the control parameters separately from the image data.

The frame memory 114 is not particularly limited, but may be a dynamic random access memory (D
RAM), and can store image data in frame units. The memory interface 113 enables input and output of image data between the frame memory 114 and the bus 117.

FIG. 3 shows the video interface 102.
Is shown.

As shown in FIG. 3, the video interface 102 includes, but is not limited to, a format conversion circuit 201, a synthesis processing circuit 202, and a synchronization circuit 20.
3 and a synchronization signal generation circuit 204. The format conversion circuit 201 converts the format of the image data read from the frame memory 114. The format-converted image data is input to the synthesis processing circuit 202. In the synthesis processing circuit 202, the format conversion circuit 201 and the MPEG4 codec LSI
An image input from a video camera disposed outside the video camera 100 and a video graphic processor 101 is synthesized. The synthesized image data is transmitted to the synchronization circuit 203. The synchronization circuit 203 synchronizes the output signal from the synthesis processing circuit 201 based on the synchronization signal output from the synchronization signal generation circuit 204. The output signal is transmitted to appropriate display means such as a liquid crystal display for displaying an image.

FIG. 1 shows a configuration example of the synthesis processing circuit 202.

For convenience of explanation, the image format is I
TU-R Recommendation BT. 601 (hereinafter referred to as BT.601). Pixel values not used due to band limitation of 601 (luminance: 0 to 15, 236 to 255, color difference: 0 to 1)
5, 241-255) are used as control parameters for the transparency.

As shown in FIG. 1, the synthesis processing circuit 202
Includes a brightness value detection unit 31 for detecting a brightness value from the image data transmitted from the data conversion unit 119, and an image synthesis unit 41 for performing an image synthesis process according to the brightness value detection result. Consists of The image synthesizing unit 202 includes:
Although not particularly limited, the multiplexer is a multiplexer, which controls the control parameters transmitted from the data converter 119 based on the luminance value detection result of the luminance value detector 31 to embed the embedded image data and the video / graphics data. Select the lower image data transmitted from the processor 105. That is, the data conversion unit 119
In BT. By using a pixel value (for example, 0) that is not used as a luminance indicator due to the band limitation of 601, the control parameter is embedded by replacing the luminance of the background portion to be made transparent with 0. In the luminance value detection unit 31, the control parameter can be detected by detecting a pixel having a luminance value of 0 from the input control parameter embedded image data, and based on this, the image synthesis unit 41 Image synthesis processing is performed. That is, the pixels of the upper image data are selected for the pixels for which the luminance value detection unit 31 has not detected the luminance value 0, and the pixels of the lower image data are selected for the pixels where the luminance value detection unit 31 has detected the luminance value 0. Of pixels are selected. As a result, the object in the upper image is cut out, and one composite image in which the lower image is superimposed on the background can be obtained. This image data is displayed on display means (not shown) via the synchronization circuit 203.

According to the above example, the following functions and effects can be obtained.

In the data converter 119, the BT. 6
By using a pixel value (for example, 0) that is not used as an indicator of luminance due to the band limitation of 01, the control parameter is embedded by replacing the luminance of the background portion to be transparentized with 0 with the luminance. The value detection unit 31 detects a pixel having a luminance value of 0 from the input control parameter embedded image data,
The control parameters can be detected, and based on the detected control parameters, the image composition processing is performed by the image composition unit 41. The pixels of the upper image data are selected for the pixels for which the luminance value detection unit 31 has not detected a luminance value of 0, and the pixels of the lower image data are selected for the pixels for which the luminance value detection unit 31 has detected the luminance value of 0. Is selected. As a result, the object in the upper image is cut out, and one composite image in which the lower image is superimposed on the background can be obtained.

FIG. 4 shows another example of the configuration of the synthesis processing circuit 202.

Although not particularly limited, the synthesis processing circuit 202 shown in FIG.
2, an α value generating unit 62 and an image synthesizing unit 42.

At this time, the data conversion section 119 generates control parameter embedded image data as follows.
With reference to the α value data, the luminance Y ′ after conversion is set to the value Y as it is with respect to the luminance Y of the pixel that is not translucent,
The luminance value of the pixel to be made translucent with the opacity α is replaced with Y ′ corresponding to α in the table shown in FIG. 8, and the control parameter embedded image data is output. However, in order to restore the image data at the time of synthesis, α other than complete transparency is set every other pixel in the horizontal direction.

The control parameter embedded image data is input to the synthesis processing circuit 202 as upper image data. In the brightness value detection unit 32, pixels whose brightness value Y ′ belongs to the range of 0 to 15 from the input upper image, and 1
Pixels belonging to the range of ~ 15 are detected. Pixels for which the luminance value conversion unit 52 does not detect that the luminance value Y ′ is in the range of 1 to 15 in the luminance value detection unit 32 are converted into luminance values Y ″ = Y ′, Value Y '
Is converted to the luminance value of a pixel that is adjacent in the horizontal direction and has a luminance value in the range of 16 to 235 for a pixel in which is detected that is in the range of 1 to 15. replace.

In the .alpha.-value generating section 62, the opacity .alpha. = 1 is set for pixels for which the luminance Y 'is not detected by the luminance detecting section 32 to be within the range of 0 to 15.
The opacity α is set for the pixel for which the luminance value Y ′ is detected to be in the range of 0 to 15 using the table of FIG. The image synthesizing unit 42 performs the α blending of the upper image and the lower image using the image data and the α value data input from the luminance value converting unit 52 and the α value generating unit 62. This allows not only a predetermined object in the upper image to be cut out and displayed superimposed on the lower image, but also the outline of the upper image to be blended with the lower image in 16 steps. .

As a modification of the configuration shown in FIG. 4, as shown in FIG. 9 as a conversion table of the luminance Y ', an arbitrary luminance Y and an arbitrary opacity It is also possible to use a table corresponding to α as a set and use it in the data conversion unit, the luminance value generation unit, and the α value generation unit. In this case, the value of α can be set freely, and α blending can be performed on any pixel in the image data.

Although the invention made by the present inventor has been specifically described above, the present invention is not limited to this, and it goes without saying that various modifications can be made without departing from the gist of the invention.

For example, in the above example, the method of setting the luminance value to a value outside the band limit is used, but the present invention can be applied to a pixel value having another band limit such as a color difference. Also,
The input image data is not limited to image data after MPEG decoding, but can be applied to digital image data in general.

In the above example, the case has been described where the synthesis processing circuit 202 is realized using dedicated hardware. However, the present invention is not limited to this, and can be realized using software. .

For example, by executing a predetermined program in the CPU 110, the following processing is realized.

As shown in FIG. 5, it is determined whether or not the luminance value of the input image data is equal to 0 (step S121). In this determination, the luminance value of the image data is equal to 0 (YES). Is determined, the lower image data is output to the display means (step S12).
2) If it is determined that the luminance value of the image data is not 0 (NO), the upper image data is output to the display means (step S123). Even if you do this,
Since images can be synthesized, the same operation and effect as in the above example can be obtained.

The processing can be performed according to the procedure shown in FIG.

First, the luminance value Y ′ is calculated from the input upper image.
Is within the range of 1 to 15 (step S111). From the input upper image, the luminance value Y ′
Are detected in a range of 1 to 15 (step S111). In this determination, the luminance value Y ′ is 1 to 15
Are determined to be out of the range (NO), the converted luminance value Y ″ = Y ′ (step S112), and the luminance value Y ′ is determined to be in the range of 1 to 15 (YES). The pixel replaces the converted luminance value Y ″ with the luminance value of a pixel that is horizontally adjacent and has a luminance value in the range of 16 to 235 (step S113). Next, it is determined whether or not the luminance Y 'is in the range of 0 to 15.

The luminance Y 'is not in the range of 0 to 15 (N
O), the opacity α = 1 is set (step S115), and if it is determined that the luminance Y ′ is within the range of 0 to 15 (YES), the table of FIG. Is used to set the opacity α (step S116). After the opacity α is obtained as described above, an image combining process is performed based on the opacity α (step S117).

In the above description, the invention made mainly by the present inventor has been described by using the MPE which is the application field in which the background was made.
The case where the present invention is applied to a G4 codec LSI has been described, but the present invention is not limited to this, and can be widely applied to image processing apparatuses.

The present invention can be applied on condition that at least an image synthesizing means for synthesizing an image is included.

[0055]

The effects obtained by typical ones of the inventions disclosed in the present application will be briefly described as follows.

That is, since the control parameter embedded in the image data is detected and the image based on the first image data and the image based on the second image data can be synthesized based on the detected control parameter, the control is performed separately from the image data. Transparency processing such as clipping of an image can be performed without transmitting parameters.

Further, it is determined whether or not the luminance value of the input first image data belongs to a predetermined range. Pixels having a luminance value not belonging to the predetermined range are not converted, and the luminance value is not converted. Pixels belonging to a predetermined range are converted to luminance values of adjacent pixels, and for pixels whose luminance values do not belong to the predetermined range, the α value indicating opacity is set to 1, and the luminance value falls within the predetermined range. For the pixel to which the pixel belongs, an α value corresponding to the pixel is obtained by referring to a predetermined table, and a translucent process can be performed based on the output signal of the luminance value conversion unit and the output value of the α value generation unit. Therefore, the α blending process for each pixel can be performed without transmitting a control parameter separately from the image data.

[Brief description of the drawings]

FIG. 1 is an example of an image processing apparatus according to the present invention;
FIG. 3 is a block diagram illustrating a configuration example of a synthesis processing circuit included in an EG4 codec LSI.

FIG. 2 is a block diagram showing an overall configuration example of the MPEG4 codec LSI.

FIG. 3 is a block diagram illustrating a configuration example of a video interface in the MPEG4 codec LSI.

FIG. 4 is a block diagram illustrating another configuration example of a synthesis processing circuit included in the MPEG4 codec LSI.

FIG. 5 is a flowchart illustrating processing when the above-described synthesis processing circuit is implemented by software.

FIG. 6 is an explanatory diagram of an image synthesis process.

FIG. 7 is an explanatory diagram of an image synthesis process.

FIG. 8 is an explanatory diagram of a conversion table referred to in embedding control parameters in the synthesis processing circuit.

FIG. 9 is an explanatory diagram of a conversion table referred to in embedding control parameters in the synthesis processing circuit.

FIG. 10 is an explanatory diagram of image synthesis using a mask pattern.

FIG. 11 is a flowchart showing processing when the above-described synthesis processing circuit is realized by software.

[Explanation of symbols]

 31 luminance value detection unit 32 luminance value detection unit 41 image synthesis unit 42 image synthesis unit 52 luminance value conversion unit 62 α value generation unit 100 MPEG4 codec LSI 101 input / output interface unit 102 video interface 103 audio interface 104 serial port and parallel Port 105 Video / Graphics Processor 106 Video Signal Processing Unit 107 Controller 108 DSP 109 DCT / Inverse DCT Circuit 110 CPU 111 Program ROM 112 Program RAM 113 Memory Interface 114 Frame Memory 119 Data Conversion Unit 201 Format Conversion Circuit 202 Synthesis Processing Circuit 203 Synchronization circuit 204 Synchronization signal generation circuit

Continued on the front page (72) Inventor Masaru Hase 5-2-1, Josuihoncho, Kodaira-shi, Tokyo Inside System LSI Development Center, Hitachi, Ltd. (72) Inventor Takashi Nakamoto Josuihoncho, Kodaira-shi, Tokyo Go-Chome No. 20-1, Hitachi Ltd. System LSI Development Center (72) Inventor Hiroki Watanabe 5--20-1, Kamimizu Honmachi, Kodaira-shi, Tokyo F-term Hitachi System LSI Development Center Inc. (Reference) 5B057 BA02 CA08 CA12 CA16 CB08 CB12 CB16 CC01 CE08 CE11 5C023 AA16 BA11 CA03 DA02 DA04 EA03 EA06 5C076 AA14 AA19 AA40 BA06 BA09 5C082 AA01 AA02 BA12 BB22 BB26 CA54 CA56 CB05 DA63 DA86 MM

Claims (5)

[Claims] 1. An image processing apparatus comprising an image synthesizing unit for synthesizing an image based on first image data and an image based on second image data, wherein the image synthesizing unit converts the image into first image data in a predetermined format. Detecting means for detecting an embedded control parameter; and synthesizing means for synthesizing an image based on the first image data and an image based on the second image data based on the detected control parameter. Characteristic image processing device. 2. The data conversion means for embedding the control parameter in the first image data using a range of pixel values not used due to band limitation, and using the range of pixel values. The image processing apparatus according to claim 1, further comprising: 3. An image processing apparatus including an image synthesizing unit for synthesizing an image based on first image data and an image based on second image data, wherein the image synthesizing unit is configured to output the first image data. A luminance value detecting means for determining whether or not the luminance value belongs to a predetermined range; and, based on a detection result of the luminance value detecting means, a pixel whose luminance value does not belong to the predetermined range is not converted. A pixel whose luminance value does not belong to a predetermined range, based on a detection result of the luminance value detecting means, for a pixel whose luminance value belongs to a predetermined range; Is an α value indicating an opacity, and for a pixel whose luminance value belongs to a predetermined range, an α value generation means for obtaining an α value corresponding to the pixel by referring to a predetermined table; The image processing apparatus characterized by comprising a synthesizing means for performing semi-transparent processing on the basis of the output value of the output signal and the α value generation means. 4. The image according to claim 3, further comprising data conversion means for converting a luminance value of a pixel to be subjected to a translucent process to a value obtained according to a predetermined table with reference to an α value indicating opacity. Processing equipment. 5. The image processing apparatus according to claim 2, further comprising means for decoding MPEG4 object-based encoded image data to obtain data to be converted by said data conversion unit.
The image processing apparatus according to any one of the preceding claims.