MXPA98004831A

MXPA98004831A - Method and apparatus for generating a composite image and information processing system

Info

Publication number: MXPA98004831A
Application number: MXPA/A/1998/004831A
Authority: MX
Inventors: Oka Masaaki
Original assignee: Sony Computer Entertainment Inc
Priority date: 1997-06-16
Filing date: 1998-06-16
Publication date: 1999-06-01

Abstract

A key signal extracted from a first original image and the first original image is subjected to an image processing, the first processed original image and a second original image are mixed using the key signal processed to provide a new composite image of which prevent quality deterioration in an effective way

Description

"METHOD AND APPARATUS FOR GENERATING A COMPOSITE IMAGE, AND INFORMATION PROCESSING SYSTEM" BACKGROUND OF THE INVENTION FIELD OF THE INVENTION The present invention relates to a method and apparatus for mixing a plurality of images to compose a new image, and an information processing system incorporating the method and apparatus.

DESCRIPTION OF THE RELATED TECHNIQUE In a graphic computer, entertainment device, video device, etc. a plurality of images are mixed using a mixing ratio information to compose a new image. For the composition of a new image of a mixture of a plurality of images, it has been proposed to prepare a key signal using information called "alpha information" supplied separately from the plurality of images and also use as a key signal, a value of specific pixel included in an image. Alpha information is a two-dimensional information that has the same resolution as an image and a value that varies from 0 to 1. For example, when the resolution of an image is 640 x 480, the alpha information has a resolution of 640 x 480. Likewise, when the bits indicating an alpha account information of two in number, for example, the alpha information can adopt four classes of values 0, 0.33, 0.66 and 1.0. Assuming here that a first image includes a portion that has a pixel value (Rl, Gl, Bl), and a second image includes a portion that has a pixel value (R2, G2, B2) that corresponds to that portion of the first image . Suppose also that an alpha information for this first and second images adopts a value A. Then, a pixel value of an image composed of a mixture of the first and second images is calculated as follows: A * (R1, Gl, Bl ) + (1 - A) * (R2, G2, B2) =. { (A * R1 + (1-A) * R2), (A * G1 + (1- A) * G2), (A * B1 + (1 - A) * B2)} (1) A * (R1, Gl, Bl) + (R2, G2, B2) =. { (A * Rl + R2), (A * G1 + G2), (A * B1 + B2)} (2) (Rl, Gl, Bl) + (1 - A) * (R2, G2, B2) =. { (Rl + (1 - A) * R2), (Gl + (1-A) * G2 (Bl + (1-A) * B2).} (3) Next, a method for mixing will be discussed below. two images using the aforementioned alpha information to compose a new image: Figure 1 shows the manner in which a first image 11 and a second image 12 are mixed using an alpha information 13 supplied separately from these images 11 and 12, so as to generate a composite image 14. In this example, the alpha information 13 consists of a left half 13a having a value of 0.0, and a right half 13b having a value of 1.0.Therefore, the composite image 14 generated by mixing the first and second images 11 and 12 using the information alpha 13 as a key signal, consists of a left half 14 a corresponding to the left half of the first image 11 and a right half 14 b corresponding to the right half of the second image 12 A pixel value of the composite image 14 is calculated using formula (1) above. A pixel value of the left half 14a is expressed by a following relation (4a) while one of the right half 14b is expressed by a following relation (4b). 1 * (R1, Gl, Bl) + 0 * (R2, G2, B2) = (Rl, Gl, Bl) (4a) 0 * (R1, Gl, Bl) + 1 * (R2, G2, B2) = (R2, G2, B2) (4b) Figure 2 shows the manner in which a first image 21 and a second image 22 are mixed using an alpha information 23 different from the information alpha 13 in the previous example, in order to produce a composite image 24. In this example, the alpha information 23 consists of a portion 23a of a left third party having a value of 0.0, a portion 23b of a right third party having a value of 1, and a portion 23c of a third central part that has a value of 0.5. The composite image 24 generated from the first image 21 and the second image 22 mixed together using the 23 alpha information as a key signal, consists of a portion 24a of a left third portion resulting from a portion of a third left portion of the first image 21, a portion 24b of a right third portion resulting from a portion of a right third portion of the second image 22, and a portion 24c of a third central part resulting from an overlap of the portion of a third central part of the first image 21 and that of the second image 22. A pixel value of this composite image 24 is calculated using the formula (1) previous. A pixel value of the portion 24a of a left third part of the composite image 24 is expressed by a following relation (5a), a pixel value of the portion 24b of a right third part is by a following relation (5b), and a pixel value of the portion 24c of a third central part is by means of a following relation (5c): 1 * (R1, Gl, Bl) + 0 * (R2, G2, B2) = (Rl, Gl, Bl) (5a) 0 * (R1, Gl, Bl) + 1 * (R2, G2, B2) = (R2, G2, B2) (5b) 0.5 * (Rl, Gl, Bl) + 0.5 * (R2, G2, B2) ( 5c) Figure 3 shows an example of the configuration of an image mixing circuit adapted to mix two images using an alpha information to compose a new image. The image mixing circuit is indicated with a reference 31. It is supplied with a first image and a second image, and an alpha information which will be a key signal defining the portions of the first and second images to be mixed and a ratio of which mix the first and second images. In accordance with the alpha information, the first and second images are mixed together to produce a composite image. The mixing of these two images using an alpha information in this image mixer circuit 31 will be further described below. Then, the manner in which a key signal is generated from an image portion having a specific color to mix two images to produce a composite image thereof, will be described below.

Figure 4 shows schematically the manner in which a key signal 43 is generated from a specific color included in a first image 41 and used to mix the first image 41 and a second image 42 together, to produce a composite image 44. The first image 41 consists of a triangular portion 41a having a certain color and a portion 41b surrounding the triangular portion 41a and having a specific color, and it will be a front part of the composite image 44, and the second image 42 will form a background of composite image 44. It includes a square portion 42b that has a certain color. The key signal 43 is a signal acquired through the extraction, from the first image 41, of only the portion 41b of which the value of the pixel provides the specific color. The composite image 44 formed from the mixing of the first and second images 41 and 42 using the key signal 43 is a superposition of the triangular portion 41a of the first image 41 on the second image 42. Figure 5 shows an example of the configuration of the image mixing circuit wherein the key signal 43 extracted from the specific color included in the first image 41, is used to mix the first and second images 41 and 42 to form the composite image 44.

In a specific color identification circuit 51, it is judged that the pixel of the first image (original image) 41 provides a specific color, and only a portion of the image of which the value of the pixel provides the specific color is extracted for generate the 43 key signal. The key signal 43 produced in this way is supplied to an image mixer circuit 53. In the image mixer circuit 53, the portions of the first (original) image 41 and the second image 42 to be mixed and a mixing ratio are determined based on the key signal 43 supplied from the specific color identifier circuit 51 for produce the composite image 44 that is delivered to the outside. By any of the methods for composing previously mentioned images, it is possible to produce a so-called "clear hollow image" by superimposing two images on top of each other with a portion of the original image processed to have a light color. However, the method for composing images using an alpha information is not advantageous since a separate information from the original images must also be used. Also, the method for composing images using a key signal extracted from the image with a light color taken as a specific color is disadvantageous since the quality of a composite image formed of a plurality of images mixed using this key signal deteriorates, when the The original image is subjected to image processing such as low pass filtering.

COMPENDIUM OF THE INVENTION Accordingly, an object of the present invention is to overcome the aforementioned inconveniences of the prior art by providing a method and apparatus for composing a new image by mixing a plurality of images together, and an information processing system that incorporates the method and apparatus, which does not need additional information such as alpha information and where a composite image produced in this way does not deteriorate in quality even when an original image for the composite image undergo an image processing. The aforementioned object can be achieved by providing the method for composing images in which the first image and a second image are mixed using key information to compose a new image, comprising a step generating key information to generate a key information of a color specific to the first original image; a first image processing step of submitting the first original image to a predetermined image processing; a second image processing step of submitting the generated key information to the image processing; and an image mixer weight of mixing the first processed image and the second original image together using the key information processed. Also, the aforementioned object can be achieved by providing an apparatus for composing images in which a first image and a second image are mixed using key information to compose a new image, comprising a means to generate a key information of a specific color of the first original image, a means for submitting the first original image and the key information generated to a predetermined image processing; and a means for mixing the first processed image and the second original image together using the key information processed. The aforementioned object is also achieved by providing an information processing sm in which an image data is generated based on the control information supplied from an input device and presented as an image in a visual presentation unit, comprising a means for generating a key information of a specific color of a first original image; means for submitting the first original image and the generated key information to a predetermined image processing; and a means for mixing the first processed original image and a second original image together using the key information processed. In accordance with the present invention, a method and apparatus for composing images and an information processing sm including the method and apparatus are provided which do not need additional special information such as alpha information and where the composite image does not deteriorate in quality even when the original image undergoes an image processing.

BRIEF DESCRIPTION OF THE DRAWINGS These objects and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken together with the co-enclosing drawings, of which: Figure 1 shows schematically the manner in which a first image and a second image are mixed using an alpha information supplied separately from the images to compose a new image; Figure 2 shows schematically the manner in which a first image and a second image are mixed together using an alpha information different from that in Figure 1 to produce a composite image; Figure 3 is a schematic functional diagram of an example of the configuration of an image mixing circuit adapted to mix two images using an alpha information to form a composite image. Figure 4 shows schematically the manner in which a key signal extracted from a Specific color included in a first image is used to mix the first image and the second image in order to compose a new image; Figure 5 is a schematic functional diagram of an example configuration of the image mixing circuit in which a key signal extracted from a specific color included in a first image is used to mix the first image and the second image to form the image composed Figure 6 shows schematically the manner in which a light color is used as a specific color to generate a "clear hollow image"; Figure 7 shows schematically the manner in which a key signal extracted from a first image subjected to image processing is used to mix a first image and a second image to compose a new image; Figure 8 is a schematic functional diagram of an example of the basic configuration of the image composing apparatus where a "clear hollow image" is produced. Figure 9 shows an embodiment of the method for composing images according to the present invention; Figure 10A is a schematic functional diagram of one embodiment of the apparatus for composing images in accordance with the present invention; and Figure 10B is a schematic functional diagram of an embodiment of an information processing system (entertainment system) in accordance with the present invention.

DETAILED DESCRIPTION OF THE PREFERRED MODALITIES Before continuing the description of the embodiments of the present invention, the generation of a key signal with a light color taken as a specific color that is the basis for the method and apparatus for composing images, and the information processing system will be described. , in accordance with the present invention. Figure 6 shows schematically the manner in which a light color is used as a specific color to produce a "clear hollow image". To simplify the illustration and description, the following description will be provided using a unidirectional graphic image for convenience of explanation. A first image 61 is shown which will form a front part of a composite image 64, and a second image 62 is also shown to be the background of the composite image 64. A key signal 63 is also shown which provides a shift between an image which is the front part of the composite image 64 and an image which is the background when a portion of the first image 61 having a specific color (of which the value of pixel is 0, for example) is taken as a light color. The key signal 63 has a value of 0.0 in the portion of the first image 64 that has the specific color, and a value of 1.0 in the rest of the first image 61 that does not have the specific color, as will be seen in Figure 6. The composite image 64 is formed by mixing the first and second images 61 and 62, using the key signal 63.

However, the quality of a composite image using the first original image, if subjected to an image processing, it may possibly deteriorate as described above. Figure 7 shows schematically the manner in which a composite image is produced as in Figure 6 after the first original image 61 in Figure 6 is subjected to a low pass filtering (to which reference will be made below simply as "filtration"). In Figure 7, there is shown a first processed image 71 that will form a front part of the composite image 74, and a second image 72 also shown to form the background of a composite image 74. The first image 71 processed is a first original image 61 that has been subjected to a filtering. A key signal 73 is also shown to provide a shift between the image 71 of the front of the composite image 74 and the background image 72. The key signal 73 is generated from the first image 71 processed. In this manner, the key signal 73 defines a portion of the first processed image 71 that is at the front of the composite image 74, and from which the pixel value provides a specific color, and the definition scale by the signal key 73 is broader than that by means of the aforementioned key signal 63 generated directly from the first original image 61. The composite image '74 is an image produced by mixing the first filtered image 71 and the second original image 72 using the key signal 73. Due to filtering, the first image 71 has a portion showing an intermediate pixel value almost equal to that for the specific color but not exactly any pixel value for the specific color. Therefore, a portion of the composite image 74 will have a color almost equal to the specific color (light color) that does not appear in the above-mentioned composite image 64. That is, a key signal generated from an original image that has been filtered to mix the original images, will define an erroneous displacement point for the images to be mixed together, which will result in a deterioration of the quality of the image such as a color shift, etc. Figure 8 is a schematic functional diagram of an example of the basic configuration of the apparatus for image composition wherein a "clear hollow image" is produced in accordance with the present invention. This apparatus for composing images uses the light color mentioned above as the specific color.

On the assumption that the specific color is (RO, GO, BO) and the first original image has another color (Rl, Gl, Bl). Then, the first filtered original image is expressed by the following formula: p * (RO, GO, BO) + q * (Rl, Gl, Bl) (p, q? 0) (6) A composite image produced from a mixture of the first processed original image expressed by the formula (6) and a second original image having a color (R2, G2, B2) is expressed by the following formula: S *. { p * (R0, GO, BO) + q * (Rl, Gl, Bl)} + t * (R2, G2, B2) = s * p * (RO, GO, BO) + s * q * (Rl, Gl, Bl) + t * (R2, G2, B2) s t? 0) (7) As you will see from the relationship (7), when the specific color (RO, GO, BO) is other than (0, 0, 0), namely, when it is not black, the specific color it will run towards the front and the bottom of a composite image. Then, the preferred embodiments of the method and apparatus for composing images in accordance with the present invention, based on the method and apparatus for composing images described above, will be described with reference to Figure 9. Figure 9 shows the manner for producing a "clear hollow image" in a preferred embodiment of the method for composing images in accordance with the present invention. A first (original) image 91 is displayed which is processed to make a first image 91 processed. For this image processing, filtering or the like is used for a uniform union between the images. Also, an alpha 93 information is displayed which is a key signal extracted from the first original image 91 on the basis of a specific color. The value of this alpha information 93 is 0 in a portion of the first original image 91 having the specific color, and 1 in the other portion of the first image 91. The information 93 alpha undergoes the same image processing as the which has been carried out for the first original image 91 in order to provide processed alpha information 93a. A composite image 94 is also shown which is formed by mixing the first processed original image 91 and a second original image 92, using the processed alpha information 43a. This image composition is carried out using the formula (3) mentioned above. Figure 10A is a schematic functional diagram of one embodiment of the image compositing apparatus adapted to carry out the aforementioned method for composing images in accordance with the present invention. It should be appreciated that the apparatus for composing images of the present invention can be encompassed using semiconductors, integrated circuits, discrete circuits using a plurality of transistors, or a CPU board. A first (original) image 91 is supplied to a specific color identification circuit 101, and an image processing circuit 102. The specific color identifier circuit 101 identifies a specific color of the first original image 91 to generate a key signal 93. The image processing circuit 102 submits the first original image 91 and the key signal 93 to the same image processing. The first original image 91 processed in this manner is taken as a first image 91a processed, and the key signal 93 thus subjected to signal processing is taken as a processed key signal 93a. This image processing performed here is a filtration or similar. When each of the components R, G and B of a pixel color of the first original image 91 is expressed with 8 bits, the processed key signal 93 is also extended to 8 bits. Also, the 8-bit key signal 93 is subjected, like an alpha information, to the same image processing as for each component (R, G, B) of the first original image 91, in order to generate the key information processed (information alpha) 93a as mentioned above. The first processed image 91a and the processed key signal 93a are supplied to an image mixing circuit 103. The image mixer circuit 103 mixes the first processed image 91a and the second original image 92 together, using the key signal 93a processed to form a composite image. During this time, if a pixel value (RO, GO, BO) = (0, 0, 0) is selected for a specific color, namely, if black is selected as the specific color, the mentioned relation (4) previously it can be expressed in the form (8) as will be shown below. In addition, when it is assumed that s = l and t = l-q to use formula (3), the relation (8) presented below can be expressed as the following formula (9). That is, only the front and the bottom can be mixed. s *. { p * (0, 0, 0) + q * (Rl, Gl, Bl)} + t * (R2, G2, B2) = s * q * (Rl, Gl, Bl) + t * (R2, G2, B2) (8) = q * (Rl, Gl, Bl) + (1 - q) * (R2, G2, B2) (9) Figure 10B is a schematic functional diagram of an embodiment of the information processing system (entertainment system) according to the present invention, incorporating the apparatus for composing images that has been described above. This entertainment system can present a three-dimensional image with high accuracy and high speed. As shown in Figure 10B, the information processing system or the entertainment system comprises a CPU (central processing unit) 101 that calculates a vortex coordinate of a triangular area (polygon) of an image capable of being visually displayed. will occur, and calculates an internal product of a normal vector of a light source vector of an attribute of an object and the data of a light source, respectively. From CPU 101, the vortex coordinate of the triangular area has color information (Rg, Gg, Bg, Ag) of three triangle vortices are supplied to a graphics processor (GPU) 106 as the apparatus for composing images. In addition, the CPU 101 acquires control information from an input device 104 such as an input device, game lever, etc. through an interface (I / F) 103 and the main bus collector 109. Based on the control information acquired in this way, the CPU 101 supplies a three-dimensional image information stored in a main memory 102 to the graphics processor ( GPU) 106 through the main bus collector 109.

The graphics processor 106 (GPU) is provided to transform the information of the three-dimensional image supplied from the CPU 101 into an image data. It incorporates the apparatus for composing images previously mentioned. Therefore, a three-dimensional image produced from the image data generated in a graphics processor (GPU) 106 is written to a video memory 105. The data of the three-dimensional image written in the video memory 105 is read during the scanning of a video signal, and is presented visually as a three-dimensional image on a TV monitor (not shown) or the like. On the other hand, an audio or sound information corresponding to the visually presented three-dimensional image and included in a swept information extracted by the CPU 101 is supplied to an audio processor 107 which will supply, based on the audio information supplied from the CPU 101, an audio data stored in an audio memory 108 as a sound of a loudspeaker (not shown). In accordance with the present invention, a method and apparatus comprising an image and an information processing system are provided wherein a portion of an original image having a light color is extracted as the key signal, the key signal is also submits to the same image processing as the original image, and the key signal is used to mix the original images to form a composite image so that no additional special information is required such as the alpha information and the composite image obtained in this way it will not deteriorate in quality even when the original image is subjected to image processing.

Claims

CLAIMS:

1. A method for composing images in which a first image and a second image are mixed using a key information to compose a new image, comprising: a step to generate a key information that generates a key information of a specific color of the first image original; a first image processing step of submitting the first original image to a predetermined image processing; a second step of image processing to submit the generated key information to image processing; and an image mixing step for mixing the first processed image and the second original image together using the key information processed.

2. The method of claim 1, wherein the specific color is black.

3. The method according to claim 1, wherein the image processing is a low pass filtering.

4. An apparatus for composing images in which the first image and a second image are mixed using key information to compose a new image, comprising: a means for generating a key information of a specific color of the first original image; means for submitting the first original image and the generated key information to a predetermined image processing; and a means for mixing the first processed image and the second original image together, using the key information processed. The apparatus according to claim 4, wherein the image processing means submits the first original image and the key information generated to a low pass filtering. The apparatus according to claim 4, wherein the key information generating means takes the color black as the specific color. 7. An information processing system in which the image data is generated based on the control information supplied from an input device and presented visually as an image and not a visual presentation unit comprising: a means for generating a key information of a specific color of a first original image; means for submitting the first original image and the generated key information to a predetermined image processing; means for mixing the first processed image and a second original image together, using the key information processed. The system according to claim 1, wherein the image processing means submits the first original image and the key information generated to a low pass filtering. The system according to claim 7, wherein the key information generating means takes the color black as the specific color.