WO2001054400A1

WO2001054400A1 - Image synthesizing device, recorded medium, and program

Info

Publication number: WO2001054400A1
Application number: PCT/JP2001/000462
Authority: WO
Inventors: Yasushi Waki; Takakazu Shiomi
Original assignee: Matsushita Electric Industrial Co., Ltd.
Priority date: 2000-01-24
Filing date: 2001-01-24
Publication date: 2001-07-26
Also published as: CN1365571A; EP1170942A1; EP1170942A4; CN1197351C; AU2710201A; US6888577B2; US20020171765A1

Abstract

An image synthesizing device (100) for synthesizing an image by combining a time-varying image and still image, comprising first acquiring means (103) for acquiring synthesization information including the image combining order for determining the combination ratios of the images before the synthesization to the synthesized image and acquiring the still images, first synthesizing means (108) for combining the still images according to the synthesization information to create a synthesized still image, calculating means (109) for determining the combination ratio of the time-varying image to the synthesized image according to the synthesization information, second acquiring means (105) for acquiring each frame constituting the time-varying image, and second synthesizing means (110) for combining the frames and the synthesized still image at the combination ratio of the time-varying image, whereby a synthesized image is created in real time in accordance with the reproduction rate of the time-varying image.

Description

Specification

Image synthesis device, recording medium and program

The present invention relates to an image display device that combines and outputs a plurality of images. Background art

In recent years, digital broadcasting has started. In digital television, a technology has been realized in which a still image is generated based on data broadcasted from a broadcasting station, and the still image is superimposed on a moving image of a broadcast program. I have.

Digital television is equipped with an image synthesizing device as a mechanism for realizing this technology.

The image synthesizing device performs a predetermined inter-pixel operation on the image F and the image G by using a pixel f (x, y) of the image F and a pixel g (x, y) in the image G corresponding to the pixel. To output an image H with pixel h (x, y).

One of the methods of this inter-pixel operation is an α-combining operation. The α synthesis operation is an operation in which the weighted average of the pixel f (x, y) and the pixel g (x, y) is set to the pixel h (x, y), and the weight coefficient for the pixel g (x, y) is calculated. When α

(Equation 1) h (x, y) = a * g (x, y) + (1-α) * f (x, y)

It is expressed as In (Equation 1), 'water' represents the product. The weighting coefficient α is called an α value, transparency, or a rendering coefficient, and takes a value from 0 to].

When the value is 0, the pixel g (x, y) becomes completely transparent and the pixel f (x, y) becomes the pixel h of the synthesized result. When the α value is 1, the pixel g (x, y) is completely opaque, and the pixel g (x, y) becomes the pixel ίι (X, y) as the synthesis result. When the value is 0.5, pixel h (x, y) is a value obtained by mixing pixel g (x, y) and pixel f (x, y) at half the ratio. By adjusting these values, The overlapping of images can be expressed. In the actual hardware configuration, each pixel is represented by RGB (red, green, blue) components of color display, so the above formula is performed for each component.

In order to synthesize a still image and a moving image in real time, the image synthesizing device has a 0SD plane which is a memory area for developing a still image and a video plane which is a memory area for expanding a moving image in frame units. Each time the frame of the video plane is updated, the images of these two areas are synthesized and output by the α synthesis operation. FIG. 1A is a conceptual diagram illustrating a state in which an image of the 0SD plane 2502 and an image of the video plane 2501 are combined and a combined image 2503 is output. By the way, in recent digital broadcasting, for example, as shown in FIG. 1 (b), while reproducing a moving image 2512 as a broadcast program, a plurality of still images such as a title of a broadcast program and a TV column are displayed on the same screen. When 1, 2513, 2514, and 2515 are superimposed and displayed, more diverse expressions are required.

To combine a moving image and a plurality of still images, theoretically, it is sufficient to perform the (composition 1) a combining operation in order from the lower layer. However, in reality, the amount of calculation of the α synthesis operation required for synthesizing a plurality of images is enormous, and it is extremely difficult to perform real-time synthesis every time a frame of a moving image is updated. By preparing a plane corresponding to each image and a plurality of hardware that performs the α synthesis operation, the speed of the synthesis process can be increased, but the cost of hardware is increased and many planes for image development are used. There is a problem that it becomes necessary. DISCLOSURE OF THE INVENTION.

In order to achieve the above object, a first object of the present invention is to provide an image synthesizing apparatus, a recording medium, and a program for synthesizing in real time in accordance with a reproduction rate of a moving image, that is, performing a synthesizing process at high speed. I do. Again Akira has a second object to provide an image synthesizing apparatus, a recording medium, and a program with a reduced memory for image development.

In order to achieve the object, an image synthesizing apparatus according to the present invention is an image synthesizing apparatus that generates a synthesized image by synthesizing a moving image and a plurality of still images, wherein the synthesis information includes an image synthesis order, First acquisition means for acquiring the combination information for obtaining a combination ratio of each image before combination with the combination image and the plurality of still images; and combining the plurality of still images based on the combination information. First combining means for generating one combined still image, calculating means for calculating a combining ratio of the moving image to the combined image based on the combining information, and acquiring each frame constituting the moving image It is characterized by comprising a second acquisition unit, and a second combination unit that combines the frames and the combined still image using the combination ratio of the moving image.

According to this configuration, the image synthesizing apparatus of the present invention first synthesizes a plurality of still images and calculates the synthesizing ratio of the moving image, and then synthesizes the moving image with each frame. Unlike the conventional method, it is not necessary to combine a plurality of still images and moving images for each frame each time, and the calculation load is light and the processing speed is high. As a result of the higher processing speed, images can be combined and displayed in real time according to the playback rate of moving images.

The synthesis information further includes, for each of the images, a coefficient corresponding to the image and operation information indicating a synthesis operation using the coefficient.

According to this configuration, in addition to the above-described effects, the image synthesizing apparatus can synthesize a plurality of still images and moving images in which coefficients and calculation information are specified. The image synthesizing device includes a first frame buffer for storing an image, and a second frame buffer for storing each frame constituting the moving image, wherein the first synthesizing unit includes: The plurality of still images obtained by the first obtaining unit are read out in accordance with the image synthesis order, and the stored contents of the first frame buffer are read out using the coefficients and the operation information. The first frame buffer replaces the storage contents of the first frame buffer with the result of the synthesis.The second acquisition unit stores the acquired frames in the second frame buffer, (2) The synthesizing means synthesizes each frame stored in the second frame buffer with the content stored in the first frame buffer using a synthesizing ratio of the moving image, and stores the synthesized image in the second frame buffer.

According to this configuration, the image synthesizing apparatus can synthesize a plurality of still images and moving images using only two frame buffers, that is, the first frame buffer and the second frame buffer.

The first synthesizing means includes a coefficient and an operation corresponding to the moving image, after synthesizing the still image immediately before the moving image in the image synthesizing order and before synthesizing the still image immediately after the moving image. Using the information, a combining operation is performed on the image stored in the first frame buffer, and the result of the combining operation replaces the contents of the first frame buffer.

According to this configuration, it is possible to combine a plurality of still images and a moving image including a moving image therebetween with an accurate combining ratio.

The image synthesizing apparatus has a screen for image display, and the synthesizing by the first synthesizing unit, the obtaining by the second obtaining unit, and the synthesizing by the second synthesizing unit are performed in parallel, and The two-frame buffer outputs the result to the screen at the same time as storing the result of synthesis by the second synthesis means.

According to this configuration, the state during the synthesis is displayed on the screen, so that a state in which nothing is displayed until the synthesis is completed can be eliminated.

The combination information is characterized by further including, for each of the plurality of images, a combination coefficient indicating a combination ratio with respect to a combination result of the image and an image other than the image.

According to this configuration, the image synthesizing device can synthesize a plurality of still images and moving images in which the synthesis ratio of each of the plurality of images and another image and the overlapping order are specified. The image compositing order represents an order in which images are superimposed, and the compositing coefficient represents, for each of the plurality of images, a compositing ratio of the image to a composite result from the rearmost image to the image in the image compositing order. A calculating unit that calculates a combining ratio of the moving image with respect to the synthesized image from the alpha value corresponding to the moving image and all the images positioned in front of the moving image.

According to this configuration, the image synthesizing device can synthesize a still image and a moving image in which the alpha value and the superposition order are defined. An image in which the alpha value is specified has the advantage that the layout can be flexibly changed compared to an image in which the composition ratio of the entire image is specified in advance. Therefore, an image synthesizing apparatus that performs image synthesizing using an alpha value has an effect that it can support EPG display of various layouts.

The image synthesizing apparatus may further include: an exchanging means for exchanging the order of two images adjacent to each other in the superimposing order; and a synthesizing result when the synthesizing is performed before and after the exchanging is the same. Updating means for obtaining and updating an alpha value corresponding to the two images after the replacement, wherein the first combining means, the calculating means, and the second combining means are provided after the replacement by the replacement means. Each process is performed using the order and the alpha value updated by the updating unit.

According to this configuration, even if the order of a plurality of images is changed, an accurate composition ratio can be obtained.

Enables image synthesis. '

Also, by using this configuration, if the moving image located in the middle layer is replaced so as to be located in the upper layer, a plurality of still images are synthesized in order from the lowest layer, and the moving image is synthesized last. Compositing can be performed by the procedure, and the calculation amount is small and the load is light.

The image synthesizing apparatus has a storage unit for storing the plurality of still images acquired by the first acquiring unit, and each of the plurality of still images has the same or less number of pixels as the synthesized image. Image data and the corresponding pixel data Wherein the first synthesizing means, the calculating means and the second synthesizing means are superimposed portions of the respective images determined by the ray information. Perform each processing for.

The image synthesizing apparatus has a storage unit for storing the plurality of still images acquired by the first acquiring unit, wherein each of the plurality of still images is represented in a vector data format, and The means converts the vector data into pixel data and then synthesizes. .

According to this configuration, since the data of the still image is vector data having a smaller amount of data than the pixel data, the memory capacity is further reduced. An image synthesizing apparatus for synthesizing a still image and a synthetic image to generate a synthetic image, the synthesizing information including an image synthesizing order, wherein the synthesizing information for obtaining a synthesizing ratio of each image before synthesizing to the synthetic image. A first obtaining unit that obtains the plurality of still images; a first obtaining unit that generates the one combined still image by combining the plurality of still images based on the combining information; and Calculating means for obtaining a synthesis ratio of each of the plurality of moving images with respect to the synthesized image, a second obtaining means for obtaining each frame constituting each of the plurality of moving images, A second synthesizing unit that synthesizes each frame of each of the plurality of moving images and the synthesized still image by using a synthesis ratio of each image.

According to this configuration, the amount of synthesis of a plurality of moving images and a plurality of still images can be reduced.

Also, the image synthesizing device of the present invention is an image synthesizing device for synthesizing a moving image and a plurality of still images to generate a synthesized image, wherein: a first obtaining means for obtaining the plurality of still images; First synthesizing means for synthesizing the still images to generate one synthesized still image, second obtaining means for obtaining each frame forming the moving image, and each frame forming the moving image and the synthesized still image. Second combining means for combining the still image and the still image.

According to this configuration, it is not necessary to prepare the memory for the still image corresponding to the same number of pixels as the display image by the number of still images, so that the memory capacity can be reduced.

Further, according to this configuration, since a plurality of still images are first synthesized and a moving image is synthesized later, it is not necessary to synthesize a plurality of still images and moving images every frame unit of the moving image, and the amount of calculation is reduced. Less. BRIEF DESCRIPTION OF THE FIGURES

FIG. 1A is a conceptual diagram illustrating a state in which images of the 0SD plane 2502 and the video plane 2501 are combined and a combined image 2503 is output.

FIG. 1B shows a state in which components of a still image and a moving image are superimposed.

FIG. 2 is a block diagram showing a configuration of the image synthesizing apparatus according to Embodiment 1 of the present invention.

FIG. 3A shows a composite image 201.

FIG. 3B shows the components that make up the composite image 201.

FIG. 4 is a diagram for explaining the structure of an image file. 'Figure 5 shows an example of an index file.

FIG. 6 shows the structure of moving image data.

FIG. 7 shows the pixel data of the moving image data 501 in an image. FIG. 8 shows the data structure of the first composite image data.

FIG. 9 is a flowchart showing a processing procedure of the first forming process.

FIG. 10 is a flowchart illustrating a processing procedure of the second synthesis.

FIG. 11 is a diagram illustrating the flow of the operation of the image synthesizing apparatus 100 according to the present embodiment when displaying an EPG.

FIG. 12 shows a component group including N + 1 moving image components or still image components. FIG. 13 is a program describing the operation of the present invention in a C language style. FIG. 14 shows a program obtained by modifying FIG.

FIG. 15 is a block diagram illustrating a configuration of an image composition device according to Embodiment 2 of the present invention.

FIG. 16 is a flowchart illustrating a processing procedure of the fourth synthesis processing. FIG. 17 is a diagram illustrating the flow of the operation of the image synthesizing apparatus 200 of the present embodiment when displaying an EPG.

FIG. 18 is a diagram for explaining the replacement of the overlapping order of two adjacent components.

Figure 19 shows how a moving image component is replaced with a still image component so that it is at the top layer.

FIG. 20 is a block diagram illustrating a configuration of an image composition device according to Embodiment 3 of the present invention.

FIG. 21 is a flowchart showing the processing procedure of the seventh synthesis.

FIG. 22 is a block diagram illustrating a configuration of an image composition device according to Embodiment 4 of the present invention.

Figure 23 (a) shows an example of an index file

FIG. 23 (b) shows the correspondence between operation types and numbers.

FIG. 24 is a flowchart showing the procedure of combining by the ninth combining unit 4108.

FIG. 25 shows the Porter Duff α combination operation according to the operation type. -Figure 26 shows the Porter Duff synthesis operation according to the operation type.

FIG. 27 shows Porter Duff a combination operation according to the operation type. .

FIG. 28 is a flowchart showing a procedure of synthesis by the tenth synthesis unit 4109.

FIG. 29 shows Porter-Duff synthesis operation according to the operation type. .

FIG. 30 shows a Porter-Duff synthesis operation according to the operation type.

FIG. 31 is a program describing the operation of the present embodiment in a C language style. BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

In the figure, the image synthesizing apparatus 100 includes an input unit 101, a control unit 102, an EPG generation unit 103, an image holding unit 104, a video reproduction unit 105, a video plane 106, a 0SD plane 107, a first synthesis unit 108, and a second synthesis unit. It comprises a unit 109, a third combining unit 110, and an output unit 111.

The input unit 101 has a remote controller, front panel buttons, and the like, and receives instructions from the user when these are operated. Specific instructions include an on / off instruction for the image synthesizing apparatus 100, a channel switching instruction, and an on / off instruction for an EPG (Electronic Program Guide) display. Here, the instruction to turn on / off the image synthesizing apparatus 100 indicates that the power of the image synthesizing apparatus 100 is turned on when it is on, and that the power is turned off when it is off, and that the EPG display is on. When off, indicates that the EPG screen is not displayed. · '.

Here, the EPG is a system that displays a program table of a broadcast program, information on the content of the broadcast program, and the like on a television screen. EPG display refers to the state in which a program guide or the like is displayed on the TV screen. As an application example of EPG, for example, a user can search for a program genre and a performer name on an EPG display screen, or easily make a recording reservation using an EPG-compatible VCR.

When the EPG display is on, the image synthesizing apparatus 100 displays a synthetic image 201 shown in FIG. 3A as an example.

As shown in FIG. 3 (b), the composite image 201 of FIG. It consists of 203 and 204. The component 202 is a still image representing a program guide, the component 203 is a moving image representing the content of the broadcast program of one channel itself, and the component 204 is a still image representing the program name of the component 203. The components 202 and 204 are generated by the EPG generation unit 103 based on the EPG information transmitted from the broadcasting station.

As described above, while the EPG display is on, the image synthesizing apparatus 100 displays the component 202, displays the component 203 at the lower right position of the component 202 at a rate of several tens of frames per second, and outputs the component 203. The component 204 is displayed in the upper left position of.

As described above, the image synthesizing apparatus 100 of the present embodiment has a function of generating and displaying a synthesized image in which a plurality of components including a plurality of still images and one moving image are combined.

Hereafter, explanation will be made with reference to Figs. 3 (a) and (b) as necessary. Note that a still image component and a moving image component are simply referred to as components unless it is necessary to distinguish them.

The control unit 102 controls all the components of the image synthesizing device 100. More specifically, when the image synthesizing apparatus 10.0 is instructed to be turned on, the video reproducing unit 105 reproduces a broadcast program. When the channel switching is instructed, the video reproduction unit 105 reproduces a broadcast program different from the one currently being reproduced. When the ON of the EPG display is instructed, the EPG generation unit 103, the video reproduction unit 105, the first synthesis unit 108, the second synthesis unit 109, and the third synthesis unit 11, 10, and the like are controlled to generate the synthesized image 201. Generate. The processing related to the EPG display will be described later. .

EPG generation section 103 acquires and holds EPG information broadcast from a broadcasting station. The EPG information includes layout information such as the display size and display position of the component, the overlapping order of the components, and the still image information of the still image component. The still image information corresponds to the image content of the still image component, and includes text, graphics data, and the like. EPG raw The generating unit 103 generates a plurality of image files and one index file based on the EPG information, and stores them in the image holding unit 104. Further, the EPG generation unit 103 extracts the display size and the display position of the component of the moving image constituting the EPG display screen from the EPG information, and outputs the same to the video reproduction unit 105. The image file corresponds to the components 202, 203, and 204 in FIG. 3B, and is composed of the display size, display position, and image data of the component. In the case of a still image component, the image data is generated based on the still image information.

The index file is for managing a plurality of image files, and is composed of a plurality of pieces of image information. One image information corresponds to one image file. The plurality of pieces of image information are arranged in the index file in the same order as the components in FIG. 3A. The image information includes an image type and a storage position. The image type indicates whether the image of the image file is a still image or a moving image. The storage position indicates the position of the head of the image file in the image holding unit 104.

Next, specific examples of the image file and the index file will be described with reference to FIGS.

Figure 4 is a diagram for explaining the structure of an image file.

In the image file 301 shown in the figure, the first line indicates the display position, the second line indicates the display size, and the third and subsequent lines indicate the image data. The image data consists of a set of pixel data corresponding to all pixels of the component. The pixel data is composed of an RGB component indicating the color of the pixel and an α value indicating the transparency of the pixel. That is, the α value indicates the synthesis ratio of the image to the synthesis result from the rearmost image to the image. The storage position 302 described on the left side of the image file 301 indicates the storage position of each data with reference to the case where the head position of the image file 301 is “0000”.

In the image file 301, the third line is the pixel data corresponding to the pixel at the coordinate position (0, 0), and the fourth line is the pixel data corresponding to the pixel at the coordinate position (1, 0). The cell data, the fifth row is the pixel data corresponding to the pixel at the coordinate position (2, 0). Here, the coordinate position (0, 0) indicates the coordinate position of the upper left corner of the component. Thus, pixel data corresponds to pixels from left to right and from top to bottom of the component.

The display position is the coordinate position of the component on the composite image 201, and is represented by Χ and Υ coordinates of the upper left corner of the component with the origin at the upper left corner of the composite image 201.

The display size consists of the height Η and width W of the component rectangle, where height Η and width W are expressed in pixels. .

Each of the RGB components takes a value from 0 to 255. If all three components are 0, the color of the pixel is black, and if all three are 255, the color is white. The α value takes a value from 0 to 255, and indicates the transparency when that pixel is superimposed on another pixel, that is, how much the pixel in the lower layer transmits through the pixel in the upper layer.

More specifically, when pixel Α is combined with pixel 値 by value, pixel C of the combined result is C = (o; A + (255−cB) / 255. Α value is 0

It takes a value of ~ 255, and when the α value is 0, pixel Α is transparent and 100% transparent to pixel B. A value of 255 makes pixel A opaque, does not transmit pixel B, and 'fills' 100% of pixel A. If the value is 128, pixel A transmits pixel B approximately 50%, that is, pixel A and pixel B are mixed by approximately 50%.

In practice, pixels A, B, and C are represented by RGB components, and the composition result is calculated for each component.

The EPG generation unit 103 sets the values of the image files of the components 202, 203, and 204 as follows. That is, the EPG generation unit 103 sets the α values of all the pixels in the component 202 to 255. This is because the component 202 is the lowest layer component in the composite image 201. Also, the EPG generation unit 103 Set the α value of all pixels at to 192. For the value of component 204, set the character part to 255 and the other parts to 64. '

Note that a detailed method of determining the RGB component and the value of the value is a well-known technique and is not a characteristic part of the present invention, and thus the description thereof is omitted.

Figure 4 shows the correspondence between the content of image data and one component. As shown in the figure, starting from the upper left of the component 303, it corresponds to pixel data from left to right and from top to bottom one pixel at a time.

The EPG generation unit 103 generates the above-described image file. However, for moving image components, an image file is generated by setting all RGB components to 0. That is, the EPG generation unit 103 determines the display position, the display size, and the α value for the image file of the moving image component, and generates an image file in which the RGB components are not determined. This is because the moving image components correspond to broadcast programs and are sent in real time from the broadcasting station, so the RGB components are not determined when the image file is generated. The RGB components of the image file corresponding to the components of the moving image are determined by a video playback unit 105 described later.

Figure 5 is Indekkusufu 'aisle 410 of _ό drawing showing an example of an index file is composed of the image information 421, 422 and 423 represent the Align order superimposed on the same order. That is, the image information 421 corresponds to the lowest layer component 202 shown in FIG. 3 (b), the image information 422 corresponds to the next layer component 203, and the image information 423 corresponds to the highest layer component 204. I will respond. The value on the side of column 41 1 indicates the image type of the component, “0” indicates a still image, and “1” indicates a moving image. The value in column 412 indicates the storage location of the image file.

The images 430, 440, and 450 correspond to the components 202, 203, and 204, respectively, and are images of the image data in the respective image files. Here, image 440 corresponds to component 203. All the RGB components of the image data are 0.

The image holding unit 104 is configured by a memory, a hard disk, or the like, and holds the image file and the index file generated by the EPG generation unit 103.

The video reproduction unit 105 receives a broadcast program from a broadcast station, performs decoding and the like, reproduces a moving image of several tens of frames / second, and sequentially stores the moving image data in the video plane 106. At the time of storage, the video playback unit 105 determines the layout of the video for the composite image 201 based on the display position and display size input from the EPG playback unit 103, and corresponds to the layout on the video plane 106. The R, G, and B components of the video are stored in the area where

The video plane 106 is configured by a memory or the like, and holds moving image data stored by the video reproduction unit 105 and a synthesized α value stored by a second synthesis unit 109 described later. The composite α value indicates the transparency of one pixel among all pixels when a plurality of pixels are overlapped and synthesized, and is calculated using the α value of each pixel. That is, the composite α value held by the video playback unit 105 represents the transparency of the pixels of the moving image component among all the pixels to be superimposed when all the components are superimposed and synthesized.

FIG. 6 shows the structure of moving image data. As shown in the figure, the moving image data 501 is a set of pixel data, and the pixel data includes an RGB component and a combined α value. An image 502 is a diagram for illustrating a correspondence relationship between pixel data and a pixel position on a screen.

FIG. 7 illustrates pixel data of the moving image data 501 in an image. As shown in the figure, an image 600 corresponding to the moving image data 501 includes an area 601 corresponding to the component 203 and an area 602 other than the area 601. Image 600 has the same height and width as composite image 201. The video plane 106 receives the RGB components of the area 601 from the video playback unit 105 and the combined α value from the second combining unit 109. The video plane 106 holds 0 as the RGB components and the composite value of the area 602 in advance. RGB component of area 601 Is updated at several tens of frames / second in accordance with the playback rate of the video playback unit 105.

The 0SD plane 107 is configured by a memory or the like, and holds the first combined image data output by the first combining unit 108.

FIG. 8 shows the data structure of the first composite image data. The first composite image data 701 is composed of a set of RGB components, and corresponds to a result obtained by combining components 202 and 204, that is, a result obtained by combining still image components. The composition will be described later.

An image 702 shows the correspondence between the RGB components of the first combined image data 701 and the pixel positions on the screen. As shown in the figure, the RGB components of the first composite image data 701 are arranged in order from left to right and from top to bottom of the image 702. Image 702 has the same height and width in pixels as composite image 201. The first combining unit 108 combines the image data of the plurality of image files held in the image holding unit 104 to generate first combined image data, and stores the first combined image data in the 0SD plane 107. This process is referred to as the first synthesis process.

FIG. 9 is a flowchart illustrating a processing procedure of the first synthesis processing.

First, the first synthesizing unit 108 initializes the 0SD plane 107 (Step S800). Specifically, all the RGB component areas of the 0SD plane 107 are set to 0. '

Next, the first synthesizing unit 108 repeats the processing of steps S801 to S807, and performs processing of synthesizing the components of the still image in order from the lower layer to the component of the upper layer.

First, the first combining unit 108 reads out image information i from the index file of the image holding unit 104 (Step S801). Here, i indicates a number assigned to components in ascending order, such as 0, 1, 2, 2, ... in order from the lower layer to the component of the upper layer for convenience in this flow chart. The image information and the image file corresponding to the component i, which are variables, are called the image information i and the image file i, respectively. Book float In the yat, the initial value of i is 0, and the increment is 1.

Next, the first synthesizing unit 108 extracts the image file i stored in the storage position indicated by the image information i from the image holding unit 104 (Step S802). · The first synthesizing unit 108 reads the display size and the display position of the image file i, and sets an overlapping range of the component i of the image file on the screen (step S803).

The first synthesizing unit 108 determines whether the image type indicated by the image information i is a moving image or a still image (step S804).

If the result of the determination is that the image is a still image, the first synthesizing unit 108 performs an α synthesis operation on the RGB components of the image file i and the RGB components in the overlapping range of the 0SD plane 107 (step S805). The formula for this combination operation is shown below.

(Equation 2)

R (x, y) = a (x, y) * Ri (x, y) + (1-a (x, y)) * R (x, y)

G (x, y) = ai (x, y) * Gi (x, y) + (1-«i (x, y)) * G (x, y)

B (x, y) = "i (x, y) * Bi (x, y) + (1-(x, y)) * B (x, y)

In this equation, R (x, y :), G (x, y), B (x, y) on the left side are the RGB components to be newly obtained, and Ri (x, y), Gi (x, y) , (X, y) and α; (x, y) are the RGB components and values of the image file i ', and R (x, y;), G (x, y), B (x, y) on the right side Are the RGB components held within the overlap range of the OSD plane 107. In other words, R (x, y) on the left side is obtained by adding the value obtained by weighting,) with £ ^ and the value obtained by weighting R (x, y) held in the 0SD plane 107 with 11 <i. Is obtained. The same applies to G (x, y) and B (x, y). The first combining unit 108 stores the newly obtained R (x, y :), G (x, y), and B (x, y) in the OSD plane 107. On the other hand, if it is determined in step S804 that the image type indicated by the image information i is a moving image, the first synthesizing unit 108 calculates the α value of the image file i and the RGB component of the overlapping range of the 0SD plane 107. Then, the following calculation is performed (step S806). The formula is shown below. (Equation 3)

R (x, y) = (l- «i (x, y)) * R (x, y)

G (x, y) = (l- "i (x, y)) * G (x, y)

B (x, y) = (l- «i (x, y)) * B (x, y)

In this equation, R (x, y :), G (x, y), and B (x, y) on the left side are the newly obtained RGB components, a xj) is the α value of the image file i, and R (x, y), G (x, y), and B (x, y) are RGB components held within the overlapping range of the 0SD plane 107. In other words, the newly obtained R (x, y) is obtained by weighting R (x, y) obtained so far with 1-i (x, y). The same applies to G (x, y) and B (x, y).

(Equation 3) is the same as (Equation 2) except that the first term on the right side of the equation is set to 0. Since the first term on the right side of each expression in step S805 weights the RGB component of the image file with a, it means that in the case of each expression in S806, no weighting is applied to the RGB component of the moving image. .

The first combining unit 108 stores the R (x, y), G (x, y), and B (x, y) calculated in step S805 or S806 in the 0SD plane 107. 'In this way, the first synthesizing unit 108 finishes the first synthesizing process after processing all image files (step S807).

By the above processing, the result of synthesizing the components of the still image is stored in the 0SD plane 107.

The second combining unit 109 performs a second combining process of calculating a combined value for the moving image component and storing the combined value in the video plane 106.

First, the second synthesizing unit 109 initializes the area of the synthesized α value of the video plane 106 (Step S900). Specifically, the area for the composite value of all pixels of the video plane 106 is set to zero. As a result of the processing in FIG. 10, this composite value area holds the composite value for the moving image component. Things.

Next, the second synthesis unit 109 repeats the processing of steps S901 to S907. First, the second synthesizing unit 109 reads out image information i from the index file of the image holding unit 104 (Step S901). Where i is, for convenience, sequentially from the lower layer to the upper layer components in this flowchart.

A variable indicating the number assigned to each component in ascending order, such as 0, 1, 2,..., And the image information and image file corresponding to component i are referred to as image information i and image file i, respectively. I do. In this flowchart, the initial value of i is 0, and the increment is 1.

Next, the second synthesizing unit 109 extracts the image file stored in the storage position indicated by the image information i from the image holding unit 104 (Step S902).

The second synthesizing unit 109 reads the display size and the display position of the image file i, and sets the overlapping range of the component i on the screen (step S903).

The second combining unit 109 determines whether the image type indicated by the image information i is a moving image or a still image (step S904).

If the result of the determination is that the image is a still image, the second combining unit 109 calculates a new combined α value using the oi i of the image file i and the combined α value in the overlapping range of the video plane 106 (step S905). . This equation is shown below. (Equation 4) a (x, y) = (1-\ (x, y)) * α (χ, y) In this equation, the left-hand side α (x, y) is a newly obtained composition i (x, y) is the α value of the image file i, and a (x, y) on the right side is the composite α value held in the overlapping range of the video plane 106.

The second synthesizing unit 109 stores the newly obtained a (x, y) in the video plane 106. On the other hand, if it is determined in step S904 that the image type indicated by the image information i is a moving image, the second combining unit 109 performs the following calculation (step S906).

(Equation 5)

In the equation a (x, y) = ai (x, y), (x, y) on the left side is the newly calculated composite value, and a i (x, y) on the right side is the α value of the image layer i. In other words, tt i (x, y) is directly used as (x, y). The first combining unit 109 stores the α (x, y) calculated in step S905 or step S906 in the video plane 106.

In this way, the second synthesizing unit 109 finishes the second synthesizing process after processing all image files (step S907). As a result, the composite value for the video component is finally stored in the video plane 106.

Upon receiving a normal reproduction instruction from the control unit 102, the third synthesis unit 110 outputs the R, G, and B components held in the video plane 106 to the output unit 111. Further, when receiving an EPG display instruction from the control unit 102, the third synthesis unit 110 receives the R, G, B components of the moving image data stored in the video plane 106 and the first synthesized image stored in the 0SD plane 107. The third synthesis is performed to synthesize the RGB components of the data, and the synthesized image of the synthesis result is output to the output unit 111.

The third synthesis processing is represented by the following equation.

(Equation 6)

R (x, y) = (x, y) * _Rv (x, y) + R. (x, y)

G (x, y) = a (x, y) * _Gv (x, y) + G. (x, y)

B (x, y) = «(x, y) * Bv (x, y) + B ₀ (x, y) Here, R (x, y :), G (x, y), and B (x, y) are the R, G, and B components of each pixel output to the output unit 111 as a result of the third synthesis. a (x, y), _Rv (x, y), _Gv (x, y), _Bv (x, y) are the composite values of the moving image data stored in the video plane 106 and R, G , B components, and Ro (x, y), Go (x, y), and Bo (x, y) are the R, G, and B components of the first composite image data.

That is, the third synthesis unit 110 calculates the value obtained by multiplying the R, G, and B components of the image data held in the video plane 106 by the synthesis a value, and the first synthesized image data held in the 0SD plane 107. The processing of adding the R, G, and B components is performed for each frame of video playback.

The output unit 111 is configured by a CRT or the like, and receives the R, G, and B components output by the third synthesis unit 110 and displays them on a screen.

Operation>

The operation of the image synthesizing apparatus 100 configured as described above will be described below.

In the figure, the contents of the rectangle indicate the operation of each component, and the arrows indicate the flow of data.

When the input unit 101 receives an instruction to display an EPG (step S1001), the instruction is transmitted to each unit via the control unit 102. Upon receiving the instruction, the EPG generation unit 103 generates an image file and an index file based on the broadcast EPG information, and stores the image file and the index file in the image holding unit 104. Further, the EPG generation unit 103 notifies the video reproduction unit 105 of the display position and display size of the moving image component acquired at the time of generating the image file (Step S1002). The first synthesizing unit 108 synthesizes the RGB components within the overlapping range of the still image components stored in the image holding unit 104, and stores the synthesized RGB components in the 0SD plane 107 (step S1003).

The second synthesizing unit 109 calculates the value of the image file stored in the image holding unit 104 Then, a composite α value for the moving image component is calculated based on the video and stored in the video plane 106 (step S1004).

The video playback unit 105 plays back the moving image, and stores the moving image data in the video plane 106 in the layout indicated by the display size and display position notified from the EPG generation unit 103 (step S1005).

The third combining unit 110 weights the RGB components of the moving image component with the combined α value, adds the RGB components of the first combined image data thereto, and outputs the RGB components of the combined image obtained after the addition. (Step S1006).

The output unit 111 displays the RGB components of the synthesized image from the third synthesis unit 110 (Step S1007)

In the figure, the processes of step S1003, step S1004, step S1005, and step S1006 are performed in parallel. The processing in step S1006 is performed at the same rate as the reproduction of the moving image in step S1005.

The following describes capture of the feature of the present invention, that is, the synthesis of a moving image and a still image.

FIG. 12 shows a component group composed of +1 moving image components or still image components. Each component has the R, G, B components and values shown in FIG. However, the α value of the lowermost component is << 0 = 1.0. The β, G, and Β components of the image obtained as a result of superimposing these component groups are given by the following equations (Equation 7).

(Equation 7)

R = ∑ i * Ri)

i = 0to N

G = ∑ (i * Gi)

i = 0to N

B = ∑ i * Bi)

i = 0to N

However,

βί = ί ①… ①

i = i + lto N

Here, β _; represents the composition ratio of the component i to the image that is the final composition result, and is also referred to as contribution. That is, j3i is the composite α value.

According to the present invention, when one of these components is a moving image, the operation of combining the components of the still image other than the component of the moving image using values and adding the component of the moving image to the result is performed. Has been realized. The still image component exists as an image file with the display size and display position, and is synthesized on the 0SD plane at the time of synthesis. Therefore, the still image component is efficiently synthesized with a small memory capacity of 107-SD planes. be able to.

This enables efficient processing with a small memory of one 0SD plane for still images and one video plane for moving images.

If the composition of these components is performed sequentially in the order of superimposition, including moving images and still images, it is necessary to calculate the composition ratio for each pixel of all components and store it in memory. is there. This requires a lot of memory and is inefficient.

Also, the formula for calculating the composition ratio requires a large number of multiplications if performed independently one by one. In the N + 1 of the Component for, 1) / 2 multiplications are required, the Ν Λ ² of the order of calculation.

In the present invention, these problems are solved and the sequential calculation that does not require memory is performed. And realize that the number of multiplications is on the order of N. Regarding the relationship between the components and the operation of the present embodiment, the first synthesizing unit 108 is in charge of adding the terms of components other than the moving image, and the second synthesizing unit 109 is in charge of calculating the degree of contribution of the moving image. The third combining unit 110 is responsible for adding the last moving image.

FIG. 13 is a program describing the operation of the present invention in a C language style. This program focuses on one pixel. R, G, and B represent the RGB components of one pixel on the OSD plane, and represents the composite α value of the moving image corresponding to this pixel. , Gi, Bi, << i are the RGB component and a value of the i-th component added to the OSD plane. R _v , G _v , and B _v are the RGB components of the moving image.

The first to fourth lines indicate the initialization process. Lines 5 to 17 show the addition of still images, and the rest show addition of moving images. Lines 7, 8, 9, 12, 13, and 14 indicate the operations performed by the first synthesis unit 108. Here, the composition formula of the moving image is equivalent to calculating assuming that the .RGB component of the moving image is 0. The tenth and fifteenth lines are calculations performed by the second synthesizing unit 109 to calculate the synthesized value of the moving image. The fifth, sixth, eleventh, sixteenth, and seventeenth lines are processes common to the first combining unit 108 and the second combining unit 109. The 18th, 19th, and 20th rows are operations performed by the third synthesis unit 110. Here, lines 18, 19, and 20 are executed in parallel in different threads, and may be an endless loop.

Figure 14 shows a program that achieves this. programl and progr-am2 split the program in Figure 13, and program2 makes the output a permanent loop. These two programs can be processed in parallel. When parallel processing is performed, it is possible to display the state in the middle of superposition and the process of superimposition.

Embodiment 2>

Configuration>

FIG. 15 is a block diagram showing the configuration of the image composition device according to the second embodiment of the present invention. FIG.

In the figure, the image synthesizing apparatus 200 includes an input unit 101, a control unit 1500, an EPG generation unit 103, an image holding unit 104, a video reproduction unit 105, a video plane 106, a 0SD plane 1501, a fourth synthesis unit 1502, and a fifth It consists of a synthesis unit 1503 and an output unit 111. Among these constituent elements, those having the same reference numerals as those of the image synthesizing apparatus 100 have the same functions, and therefore the description thereof will be omitted.

The control unit 1500 controls all the components of the image composition device 200. More specifically, when the image synthesis device 200 is instructed to be turned on, the video reproduction unit 105 reproduces a broadcast program. When the channel switching is instructed, the video reproduction unit 105 reproduces a broadcast program different from the one currently being reproduced. When the ON of the EPG display is instructed, it controls the EPG generating unit 103, the video reproducing unit 105, the fourth synthesizing unit 1502, the fifth synthesizing unit 1503, and the like to generate a synthesized image.

The 0SD plane 1501 is composed of a memory or the like, and holds RGB components and values output by the fourth synthesis unit 1502. The 0SD plane 1501 holds RGB components and values corresponding to pixels on the screen.

Upon receiving an instruction from the control unit 1500, the fourth combining unit 1502 combines the image data of the plurality of image files held in the image holding unit 104, and finally generates the fourth combined image data. , Stored in the 0DS plane 1501. This processing is called the fourth synthesis processing.

FIG. 16 is a flowchart showing a processing procedure of the fourth synthesis processing. First, the fourth synthesizing unit 1502 sets 0 to all areas of the RGB component of the 0SD plane 1501 and sets 0 to all areas of the synthesized value of the video plane 106 (steps S1600 and S1601). ).

Next, the fourth compositing unit 1502 repeats the processing of steps S1602 to S1612, composites the components of the still image in order from the lower layer to the component of the upper layer, and generates the 0SD plane 1501. Processing to calculate the composite value for the video component and store it in the video plane 106 I do.

First, the fourth synthesizing unit 1502 reads out image information i from the index file of the image holding unit 104 (Step S1602). Here, i indicates the number assigned to each component in ascending order from 0 to 1 in the order from the lower layer to the upper layer for convenience in this flow chart. The image information and image file corresponding to component i, which are variables, are called image information i and image file i, respectively. In this flowchart, the initial value of i is 0, and the increment is 1.

Then, the fourth synthesizing unit 1502 extracts the image file i stored in the storage position indicated by the image information i from the image holding unit 104 (Step S1603). The fourth synthesizing unit 1502 reads the display size and display position of the image file i, and sets a range of superimposition on the screen of the component i of the image file i (step S1604).

The fourth synthesizing unit 1502 determines whether the image type indicated by the image information i is a moving image or a still image (step S1605).

As a result of the determination, when it is determined that the image type indicated by the image information i is a moving image, the fourth combining unit 1502 copies the α value of the image file i to the video plane 106 (step S1611).

If it is determined in step S1605 that the image type indicated by the image information i is a still image, the fourth combining unit 1502 determines whether or not all the combined α values held in the video plane 106 are 0. (Step S1606). That is, in this case, the process branches depending on whether or not there is a moving image in a lower layer than the still image.

As a result of the determination in step S1606, when it is determined that the combined α values held in the video plane 106 are all 0, the fourth combining unit 1502 copies the value of the image file i to the 0SD plane 1501 (step S1609). ). As a result of the determination in step S1606, if the composite values held in the video plane 106 are not all 0, that is, at least one is If not, the fourth synthesizing unit 1502 obtains a new α value by the calculation of (Equation 8) using the synthesized value held in the video plane 106 and the value of the image file i, and stores it in the 0SD plane 1501 ( Step S1607).

(Equation 8)

«I (x, y)

"osd (X, y) =

la _v (x, y) * (l- «i (x, y))

In this equation, aosd (x, y) on the left side is the newly obtained α value, α i (x, y) on the right side is the α value of the image file i, and a _v (x, y) is the video plane 106. This is the retained composite α value.

· Further, the fourth combining unit 1502 calculates the combined α value for the moving image by (Equation 9), and stores it in the area of the combined α value of the video plane 106 (Step S1608) _c (Equation 9)

«V (x, y) =" _v (x, y) * (l- "i (x, y))

In this equation, a _v (x, y) on the left side is a composite value newly stored in the video plane 106, and α _ν (χ _; γ) on the right side is a composite value stored in the video plane 106 before this storage. ai (x, y) is the α value of image file i.

The fourth combining unit 1502 performs a combining operation of the RGB component of the image file i and the RGB component in the overlapping range of the .0SD plane 1501, using the α value set in the 0SD plane 1501 in step S1607 or step S1609. And stores the result in the 0SD plane 1501 (step S1610). The equation is shown in (Equation 10).

(Number 10)

R (x, y) = a _0S d (χ, y) * Ri (x, y) + (l- flr _os d (x, y)) * R (x, y)

G (x, y) = _0S d (x, y) * Gi (x, y) + (l- a ₀ sd (x, y)) * G (x, y)

B (x, y) = «osd (x, y) * Bi (x, y) + (1-or ₀ sd (x, y)) * B (x, y) In this equation, R (x, y), G (x, y), and B (x, y) on the left side are RGB components to be newly obtained, and! ¾), 0,), 8 ^) and 03 (1 are the RGB components and α values held in the 0SD plane 1501, and Ri (x, y), Gi (x, y), Bi ( x, y) is the RGB component of image file i.

In this way, the fourth synthesis unit 1502 performs the processing from step S1602 to step S1612 on all the image files, and ends the fourth synthesis processing (step S1612).

After the fourth compositing process is completed, the result of compositing all the components of the still image in the 0SD plane 1501 is retained, and the composite α value of the moving image component is retained in the composite α value area of the video plane 106. The Rukoto. The fifth synthesizing unit 1503 outputs the R, G, and B components held in the video plane 106 to the output unit 111 when receiving a normal reproduction instruction from the control unit 1500. Further, when receiving the EPG display instruction from the control unit 1500, the fifth synthesizing unit 1503 stores the H, G, B components and the synthesized α value of the moving image data stored in the video plane 106, and the fifth stored in the 0SD plane 1501. (4) A fifth synthesis for synthesizing the RGB components of the synthesized image data and is performed, and the resultant synthesized image is output to the output unit 111. The fifth synthesis processing is represented by the following equation.

(Equation 11)

R (x, y) = α (χ, y) * RV (x, y) + R ₀ (x, y)

G (x, y) = a (x, y) * G _v (x, y) + G ₀ (x, y).

B (x, y) = a (x, y) * B _v (x, y) + B ₀ (x, y)

Here, R (x, y), G (x, y), and B (x, y) are the R, G, and B components of each pixel output to the output unit Ill as a result of the fifth synthesis. (X, y), R _v (x, y), G _v (x, y), B _v (x, y) are the combined α value of the moving image data stored in the video plane 106 and R, G , B components, and Ro (x, y), Go (x, y), .Bo (x, y) are the R, G, B components of the fourth composite image data. In this way, the fifth combining unit 1503 combines with the fourth combined image data every time a frame is updated.

The operation of the image synthesizing apparatus 200 configured as described above will be described below.

FIG. 17 is a diagram illustrating the flow of the operation of the image synthesizing apparatus 200 of the present embodiment when displaying an EPG. In the figure, the contents of the rectangles indicate the operation of each component, and the arrows indicate the flow of data. Note that, in this figure, the operation at the same step number as in FIG. 11 means the same operation as that in FIG. When the input unit 101 receives an EPG display instruction (step S1001), the instruction is transmitted to each unit through the control unit 1500. Upon receiving the instruction, the EPG generation unit 103 generates an image file and an index file based on the broadcast EPG information, and stores the generated image file and the index file in the image holding unit 104. Further, the EPG generation unit 103 notifies the video reproduction unit 105 of the display position and display size of the moving image component acquired at the time of generating the image file (step S1002). The fourth synthesizing unit # 502 synthesizes the RGB components within the overlapping range of the still image components stored in the image holding unit 104, and stores them in the 0SD plane 1501. Further, the fourth combining unit 150 calculates a combined value for a moving image and stores the combined value in the video plane 106 (step S1701).

The video reproducing unit 105 reproduces the moving image, and stores the moving image data in the video plane 106 in the layout indicated by the display size and the display position notified from the EPG generating unit 103 (Step S1005).

The fifth synthesizing unit 1503 weights the RGB components of the moving image component with the synthesized α value, adds the RGB components of the fourth synthesized image data thereto, and outputs the RGB components of the synthesized image obtained after the addition. (Step S1702). The output unit 111 displays the RGB components of the synthesized image from the fifth synthesis unit 1503 (Step S1007)

In the figure, the processing in step S1702 is performed in step S1005 and step S1701. Is performed in parallel.

As shown in FIG. 18, the synthesizing method according to the present embodiment is performed by changing the overlapping order of two adjacent components. At this time, the values of the two components after the replacement are updated by the equations shown in steps S1607 and S1608 of FIG. 16 so that the synthesis result before the replacement is the same as the synthesis result after the replacement.

That is, assuming that the a value of component i in the lower layer of the two replaced components is i +1 and the value of component i +1 in the upper layer is a i ', the α value of each component is Is expressed by the following equation (Equation 12).

(Number 12)

'ai + i

= αι = «i (l-ai + i)

Here, (Equation 12) is obtained as follows.

Assuming that the values of component i and component i + 1 before replacement are ai and ai + 1, respectively, the contribution of component i and component i + 1 to the final composite image | 3 i and | 3 i + 1 are obtained from (Equation 7) (Equation 13)

Becomes

On the other hand, when the contributions j3 _{i + 1} and | 3 i of the component i + 1 and the component i after replacement with respect to the final composite image are expressed using (Equation 7), ¾ ₊₁ '= no- ^ j)… ③

j = i + 2 to N

i 'two n-^ j)… ④

j = i + 2toN.

In order for the combined result to be the same before and after the exchange, it is sufficient that (Equation 13) and (Equation 14) have the same values for (1), (2), (2) and (3). That is,

(Number 15)

i + i * n (i- «j) = n (i-« j)

j = i + 2 to N j = i + 2to N Solving these equations for a _{i +1} 'gives (Equation 12).

By using this rule, all the still image components are synthesized before the moving image components by replacing the components of the moving image with the components of the still image so that the components of the moving image are at the topmost layer as shown in Fig. 19. -Finally, it is possible to combine moving images.

The fourth synthesizing unit 1502 efficiently calculates values necessary for replacing adjacent components and synthesizes them into the 0SD plane 1501, as shown in the flowchart of FIG. The fifth combining unit 1503 combines the 0SD plane 1501 and the −video plane 106.

In the figure, the image synthesizing device 300 includes an input unit 101, a control unit 2000, an EPG generation unit 2003, an image holding unit 104, a video reproduction unit 2001, a video plane 2002, It comprises an OSD plane 107, a sixth synthesis unit 2004, a seventh synthesis unit 2Q05, an eighth synthesis unit 2006, and an output unit 111. With this configuration, the image synthesizing device 300 synthesizes a plurality of moving image components and a plurality of still image components. Among these constituent elements, those having the same reference numerals as those of the image synthesizing apparatus 100 and the image synthesizing apparatus 200 have the same functions, and thus the description thereof will be omitted, and the explanation will be focused on the elements having different numbers.

The control unit 2000 controls all the components of the image synthesizing device 300. More specifically, when the image synthesizing device 300 is turned on, the video reproducing unit 2001 reproduces a broadcast program. When the channel switching is instructed, the video reproducing unit 2001 reproduces a broadcast program different from the one currently being reproduced. When the ON of the EPG display is instructed, the EPG generating unit 2003, the video reproducing unit 2001, the sixth synthesizing unit 2004, the #th synthesizing unit 2005, and the eighth synthesizing unit 2006 are controlled to generate a synthetic image.

The video playback unit 2001 includes a plurality of playback units, that is, a first playback unit, a second playback unit,..., And an N-th playback unit. Each playback unit receives a broadcast program from a broadcast station. Then, decoding is performed, and a moving image of several tens of frames / second is reproduced, and the moving image data is stored in the video plane 2002 while being overwritten and updated in frame units. At the time of storage, the video playback unit 2001 determines the layout of the moving image with respect to the final composite image based on the display position and display size input from the EPG playback unit 2.003, and determines the layout on the video plane 2002. The .R, G, and B components of the video are stored in the area corresponding to.

The video plane 2002 has a plurality of planes inside, that is, a first plane, a second plane, and an N-th plane. Each plane includes a memory and the like, and each plane corresponds to a plurality of reproduction units of the video reproduction unit 2001. Then, the moving image data stored by the playback unit and the composite α value stored by the seventh synthesis unit 2005 are retained.

The EPG generation unit 2003 acquires EPG information for displaying an EPG, which includes a plurality of moving image components and still image components, from a broadcast station. A plurality of image files and one index file are generated based on the information, and stored in the image holding unit 104. In addition, the display size and display position of a plurality of moving image components constituting the EPG display screen are extracted from the EPG information and output to the video playback unit 2001.

The EPG generation unit 2003 is the same as the EPG generation unit 103 except that it supports a plurality of moving image components.

The sixth combining unit 2004 combines the image data of the plurality of image files held in the image holding unit 104 to generate sixth combined image data, and stores the sixth combined image data in the 0SD plane 107. This process is called a sixth synthesis process. This process is almost the same as the first combining process shown in FIG. 9 of the first embodiment, except that the process of step S806 is performed a plurality of times corresponding to a plurality of moving images.

Upon receiving an instruction from the control unit 2000, the seventh combining unit 2005 calculates a combined α value for each of the plurality of video components, and performs a seventh combining process of storing the combined α values in the corresponding planes of the video plane 2002. .

FIG. 21 is a flowchart illustrating the procedure of the seventh combining process.

First, the seventh synthesizing unit 2005 initializes the area of the synthesized α value of the video plane 2002 (step S2100). Specifically, 0 is set in the area for the combined α value of all pixels in each plane of the video plane 2002. This composite value area eventually holds the composite a value.

Next, the seventh combining unit 2005 repeats the processing of steps S2101 to S2107.

First, the seventh synthesizing unit 2005 reads image information i from the index file of the image holding unit 104 (Step S2101). Here, i is a number assigned to components in ascending order, such as 0, 1, 2, 2,... From the lower layer to the upper layer for convenience in this flowchart. The image information and the image file corresponding to the component i, which are variables, are called the image information i and the image file i, respectively. Book float In the yat, the initial value of i is 0, and the increment is 1.

Next, the seventh synthesizing unit 2005 retrieves the image file stored in the storage position indicated by the image information i from the image holding unit 104 (Step S2102).

The seventh synthesizing unit 2005 reads the display size and the display position of the image file i, and sets an overlapping range of the component i of the image file on the screen (step S2103).

The seventh synthesizing unit 2005 determines whether the image type indicated by the image information i is a moving image or a still image (step S2104).

If the result of the determination is that the video is a moving image, the seventh synthesizing unit 2005 copies the α value of the image file i into the α value area of the k-th plane of the video plane 2002 (step S2105). When this is made into the formula, it becomes (Formula 16)

(Number 16)

In k (x, y) = o¾ (x, y) (Equation 16), | 3 _k (x, y) indicates the value stored in the area of the composite α value of the k-th plane of the video plane 2002. ; (χ, γ) is the α value of image file i. The value of k takes a value from 1 to Ν, and is incremented by 1 each time the process of step S2105 is repeated.

After that, the seventh combining unit 2005 calculates a combined 'α value region of the m-th plane other than the k-th plane,

(Number 17)

? m (x, y) = (1-«i (x, y)) * Perform the calculation of P (x, y) and update the value of / S _n (x, y). Here, iS x. Y) indicates a value stored in the area of the composite value of the m-th plane, and m takes a value from 1 to N excluding k (step S2106).

That is, the operations in step S2105 and step S2106 are equivalent to the operation of ① in (Equation 7), and each moving image component for the final synthesized result image is calculated. The composition ratio of the components, that is, the composition α value, is calculated.

In this way, the seventh combining unit 2005 ends the seventh combining process after performing the process for all the image files (step S2107). As a result, each plane of the video plane 2002 stores a composite value for each moving image component.

Upon receiving an EPG display instruction from the control unit 2000, the eighth synthesis unit 2006 performs an eighth synthesis process of synthesizing each RGB component of each plane of the video plane 2002 and each of the 0SD planes 107, and generates a resultant synthesized image. Output to the output unit 111. When there are N moving image components, the composition by the eighth composition unit 2006 is expressed by the following equation.

(Number 18)

R (x, y) = (x, y) * Rvi (x, y)

+ ^ ₂ (x, y) * Rv2 (x, y) +-+? N (x, y) * RvN (x, y) + R ₀ (x, y)

G (x, y) = ^ i (x, y) * Gvi (x, y)

+ ^ 2 (x, y) * G _V 2 (x, y) + '+ N (X, y) * GvN (x, y) + G ₀ (x, y)

B (x, y) = (x ₅ y) * B _v i (x, y)

+ ^ ₂ (x, y) * Bv2 (x, y) +-+ N (X, y) * B _V N (x, y) + B ₀ (x, y)

Here, R (x, y), Ci (x, y), and B (x, y) are the G and B components of each pixel output to the output unit 111, and! ^), R _vl (x, y), G _vl (x, y), and 'B _vl (x, y) are the combined alpha value and RGB of the video data stored in the first plane of the video plane 2002. I8 ₂ (x, y), R _v2 (x, y), G _v2 (x, y), B _v2 (x, y) are the moving images stored in the second plane of the video plane 2002. a synthetic α Ne及Pi RGB components of the image _{data, i3 N (x, y)} , R vN (x, y), G vN (x, y), B vN (x, y) is. video plane 2002 R. (x, y), G. (x, y), B. (x, y) are the OSD planes. This is the RGB component stored in 107.

<Operation> The operation of the image synthesizing device 300 configured as described above is almost the same as the flow of the operation of the image synthesizing device 100 shown in FIG.

However, they differ in the following points.

That is, the EPG generation unit 2003 performs the process of step S1002, and the EPG generation unit 2003 notifies the video reproduction unit 2001 of the display positions and display sizes of a plurality of moving image components instead of one moving image component.

The process of step S1003 is performed by the sixth synthesis unit 2004.

The process of step S1004 is performed by the seventh synthesizing unit 2005, and the seventh synthesizing unit 2005 calculates a synthesized value corresponding to each of the plurality of moving image components and stores the calculated value in the video plane 2002.

The video playback unit 2001 performs the process of step S1005, and the video playback unit 2001 plays back a plurality of moving images and stores them in the video plane 2002. Further, the processing of step S1006 is performed by the eighth compositing unit 2006, and the eighth compositing unit 2006 computes the weighted RGB components of the plurality of moving image components with their composite values and the RGB components of the sixth composite image data. Are added, and the RGB components of the composite image obtained after the addition are output to the output unit 111 (step S1006).

In this way, the eighth synthesizing unit 2006 continues synthesizing and outputting each plane of the video plane '2002 and the 0SD plane 107 until the stop instruction is given, and outputs the video reproducing unit 2001, the sixth synthesizing unit 2004, and Processing is performed in parallel with the processing of the seventh combining unit 2005.

Embodiment 4>

Configuration>

In the figure, an image synthesizing device 400 includes an input unit 101, a control unit 4102, an EPG generation unit 4103, an image holding unit 4104, a video playback unit 105, a video plane 106, a 0SD plane 107, a ninth synthesis unit 4108, and a tenth synthesis unit. Part 4109, third synthesis part 110, It is composed of an output unit 111 and performs image synthesis using Porter-Duff operation. The Porter-Duff operation is described in, for example, r Composing ltng Digital Images J (SI GGRAPH 84, 253-259.), Co-authored by τ. Porter and T. Duff. Note that, also in the present embodiment, description will be given with reference to FIGS. 3A and 3B as necessary.

Here, the porter-duff operation will be briefly described.

The Porter-Duff operation specifies one or two types of operation rules for combining a source pixel and a destination pixel. The source and destination pixels have RGB components and 'values. However, the definition of the α value is different from the α value of the first to third embodiments. The values used in the first to third embodiments are defined between two images, whereas the α value used in the porter-duff operation of the present embodiment is the α value for each image. Is defined. In the Porter-Duff operation, the image resulting from the composite operation of two images with α values also has a value. When the image resulting from the synthesis is actually displayed on the screen, a value obtained by multiplying the RGB value of the image by the α value is output.

The image synthesizing apparatus 400 of the present embodiment performs image synthesis using eight types of useful calculations out of the 12 types of calculation rules. The eight calculation rules are "CLEAR", "SRG", "SRC-0VER", "DST-0VER", "SRC-1 IN", "DST-1 IN", "SRC_0UT" and "DST-0UT". is there.

“CLEAR” clears both the RGB component and α value of the destination. Neither the source nor the destination is used as input. In the present embodiment, the destination corresponds to the image held in the 0SD plane 107, and the source corresponds to the image of the image file i.

"SBC" copies source to destination

"SRC-0VERJ overlays the source on the destination. DST-0VERJ overlays the destination on the source and replaces the destination with the resulting RGB components. r SRC_INJ replaces the destination with the source part that overlaps the destination.

“DST_IN” replaces the destination with the destination part that overlaps the source.

“SRC_0UT” replaces the destination with a source part that does not overlap with the destination.

"DST-0UTJ replaces the destination with a destination that does not overlap the source.

Hereinafter, components having reference numerals different from those of the other embodiments will be described.

The control unit 4102 controls all the components of the image composition device 400. More specifically, when the image synthesis device 400 is instructed to be turned on, the video reproduction unit 105 reproduces a broadcast program. When the channel switching is instructed, the video reproducing unit 105 reproduces a broadcast program different from the one currently being reproduced. When the ON of the EPG display is instructed, the EPG generation unit 4103, the video reproduction unit 105, the ninth synthesis unit 4108, the tenth synthesis unit 4109, the third synthesis unit 110, and the like are controlled to generate a synthesized image.

As in the first embodiment, the EPG generation unit 4103 acquires and holds EPG information broadcasted from a broadcasting station, generates a plurality of image files and one index file based on the EPG information, and stores the image files. Stored in section 4104. Further, the EPG generation unit 4103 extracts the display size and the display position of the component of the moving image constituting the EPG display screen from the EPG information, and outputs the same to the video reproduction unit 105.

The image holding unit 4104 holds a plurality of image files and one index file.

The index file is for managing a plurality of image files and is composed of a plurality of image information. One image information corresponds to one image file. The plurality of pieces of image information are arranged in the index file in the same order as the components are superimposed. Image information The report includes an image type, a calculation type, and a storage location. The image type indicates whether the image of the image file is a still image or a moving image. The calculation type indicates which of the 12 types of calculation specified in the Pouffer Duff is used. The storage position indicates the head position of the image file in the image holding unit 4104.

FIG. 23 (a) shows an example of the index file. The index file 4410 shown in the figure is composed of image information 4421, 442, and 4423, and indicates the order of superposition of components in the same order. That is, the image information 4421 corresponds to the component of the lowest layer, the image information 4422 corresponds to the component of the next layer, and the image information 4423 corresponds to the component of the highest layer. The value on the column 41 1 side indicates the image type of the component, “0” indicates a still image, and “1” indicates a moving image. The value in column 413 indicates the type of operation. Figure 23 (b) shows the relationship between the values that can be entered in column 413 and the operation type. For example, “3” stored in row 44 2 2 column 4 1 3 calculates the operation “SRCOVER”, and “2” stored in row 44 2 3 column 4 13 and row 44 2 1 column 4 13 Represents the operation "SRC". The value in column 412 indicates the storage location of the image file.

The images 430, 440, and 450 correspond to the components 202, 203, and 204, respectively, and represent the image data in the respective image files. Here, the image 440 indicates that all the RGB components of the image data corresponding to the component 203 are 0. In the Porter-Duff operation, the value is set for all images, so image 430 is the bottom component, but any alpha value is set for each pixel .

Upon receiving an instruction from the control unit 4102, the ninth combining unit 4108 combines the image data of the plurality of image files held in the image holding unit 4104 to generate ninth combined image data, and stores it in the 0SD plane 107. I do. This processing is performed in the ninth synthesis Called processing.

FIG. 24 is a flowchart showing the procedure of the synthesis by the ninth synthesis unit 4108.

In the inquiry diagram, the same process is performed in steps having the same step numbers as those in FIG. Hereinafter, processing of step numbers different from those in FIG. 9 will be described. Step S4802: The ninth combining section 4108 reads out the image file i stored in the storage location indicated by the image information i and the operation type indicated by the image information i.

Step S4805: According to the operation type read in step S4802, the porter duff alpha synthesis operation shown in FIG. 25 is performed for each pixel.

Step S4806: The RGB components of the moving image are set to 0, and the calculation shown in FIG. 26 is performed in pixel units according to the type of calculation.

Step S4807: The α value shown in FIG. 27 is calculated for each pixel according to the operation type.

Note that FIGS. 25, 26, and 27 show equations for only eight operations that are said to be significant in the Porter-Duff operation.

Upon receiving an instruction from the control unit 4102, the tenth combining unit 4109 generates an α value of the video plane 106 for the OSD plane 107 from the image data stored in the image holding unit 4104, and Store in 6.

FIG. 28 is a flowchart showing the procedure of combining by the tenth combining unit 4109.

In the figure, the same processing is performed in steps having the same step numbers as those in FIG. Therefore, the process of step numbers different from those in FIG. 10 will be described below.

Step S1202: The tenth combining unit 4109 reads the image file i stored in the storage location indicated by the image information i and the operation type indicated by the image information i. Step S4905: The tenth combining section 4109 performs the calculation shown in FIG. 29 on a pixel basis according to the calculation type. Basically, since a still image is drawn on the moving image, the components of the moving image are weakened. In the table of Fig. 29, is the composite value of each pixel of the video plane 106, and << is the value of each pixel superimposed on the video plane 106 of the extracted image file, and these values are theoretical values. Top Take between 0 and 1. As an actual implementation method, the value is represented by a value such as 0 to 255 or 0 to 15.

Step S4906: The tenth combining section 4109 performs the calculation shown in FIG. 30 on a pixel basis according to the calculation type. The ninth combining section 4108 and the tenth combining section 4109 operate in synchronization. Steps S1202, S903, S904, and S907 may be synchronized using the processing results of steps S4802, S803, S804, and S807 as they are. Steps S4905 and S4906 must be completed before step S807.

With the above-described configuration, the image synthesizing apparatus 400 can perform image synthesizing by Powell-Duff operation, and can reduce the memory and the number of times of multiplication similarly to the first embodiment.

FIG. 31 is a program describing the operation of the present embodiment in a C language style. This program focuses on only one pixel. R, G, B, and a represent the R, G, B, and value of one pixel on the OSD plane, and _av represents the composite a value of the video plane corresponding to this pixel.

1. Lines 5 to 5 are the initialization process. The eighth, 1-1, and 14th lines are operations performed by the ninth combining unit 4108. The 9th and 12th lines are the operations assigned to the tenth combining unit 4109, which calculates the contribution (| 8 value) of the moving image, that is, the combined α value. The 16th, 17th, and 18th rows are the operations performed by the third synthesis unit 110. Here, the 16th, 17th, and 18th lines are performed in parallel by another thread, and may be in a permanent loop. Here, the specific calculation on the eighth line is shown in FIG. 26, and the specific calculation on the 11th line is shown in FIG. Further, the specific operation on the ninth line is shown in FIG. 30, and the specific operation on the 12th line is shown in FIG. here 29 and 29 correspond to _v in FIG. The specific calculation on the 14th line is shown in FIG.

As described above, the image synthesizing apparatus 400 according to the fourth embodiment performs the synthesis of a plurality of still images and the calculation of the synthesis ratio of a moving image in accordance with the calculation type of the Porter-duff calculation determined for each image. Each frame of the moving image is combined with the combined still image according to the type. This eliminates the need to combine multiple still images and moving image frames each time a moving image frame is updated, reducing the computational load and increasing the processing speed. As a result of the higher processing speed, images can be combined and displayed in real time according to the playback rate of the moving image. In addition, since the image can be synthesized in real time, each time an image of one frame is expanded in the frame buffer, the image of the frame is synthesized with the synthesized still image. The effect is that the capacity of the frame buffer can be reduced.

As described above, the image synthesizing apparatus of the present invention has been described based on the embodiments, but the present invention is not limited to the above embodiments. That is,

(1) In the first and second embodiments, the combined α value of the moving image component is held in the video plane 106, 1501, but when there is one moving image component, it is held in the 0SD plane 107, 1501. It may be configured as follows. (2) In the first to fourth embodiments, pixel data in a memory or the like is R,

G, B, are stored in this order, but are not limited to this order, and may be stored in, for example, the order of B, G, B or the order of .alpha., B, G, R. Also, values for all pixels may be stored for each component of R, G, Β, and not for each pixel.

(3) The data length of R, G, Β, and α is 1 byte each, that is, R, G, and で are 256 gradations from 0 to 255, and a total of about 1.670 million colors Can be expressed. However

The data length of R, G, B, may be expressed in 4 bits each, so-called high color.

(4) In the first to fourth embodiments, the YUV component may be used instead of the RGB component. Good.

(5) The image holding units 104 and 4104 hold pixel data as image data of the still image component, but hold vector data, that is, data composed of graphic elements such as mathematical expressions, straight lines, points, and circles, When the first combining unit 108 combines image data, the first combining unit 108 may develop the pixel data. With this configuration, the memory capacity of the image holding units 104 and 4104 can be further reduced.

(6) A computer program for causing a general-purpose computer or a device having a program execution function or the like to execute the operation procedure of each component of the image synthesis devices 100, 200, 300, and 400 may be used.

Particularly, the computer program corresponding to the image synthesizing apparatus 400 of the fourth embodiment eliminates the need for eight types of arithmetic units corresponding to the eight types of operations of Porter Duff, and all eight types are provided in one processor that executes the computer program. Has the effect that the calculation of (1) can be performed.

Further, this computer program can be recorded on a recording medium or distributed and distributed via various communication channels. Such recording media include an IC card, an optical disk, a flexible disk, and a ROM.

(7) The third synthesizing unit 110 is configured to perform the above-described addition processing for each frame of video playback, that is, to perform the addition processing in synchronization with frame playback. It is not necessary, and the configuration may be such that the addition process is performed asynchronously. The same applies to the fifth synthesis unit 1503 and the eighth synthesis unit 2006. Industrial applicability

An image display device that combines and outputs a plurality of images, and is particularly used for a television that receives digital broadcasts.

Claims

The scope of the claims

1.

An image synthesizing apparatus for synthesizing a moving image and a plurality of still images to generate a synthetic image,

First obtaining means for obtaining synthesis information for obtaining a synthesis ratio of each image before synthesis with respect to the synthesis image and the plurality of still images, the synthesis information including an image synthesis order;

First combining means for combining the plurality of still images based on the combining information to generate one combined still image;

Calculating means for calculating a synthesis ratio of the moving image to the synthesized image based on the synthesis information;

A second acquisition unit for acquiring each frame constituting the moving image; and a second combining unit for combining each frame and the combined still image using a combination ratio of the moving image.

An image synthesizing apparatus comprising:

2.

The synthesis information further includes, for each image, a coefficient corresponding to the image and operation information indicating a synthesis operation using the coefficient.

2. The image synthesizing device according to claim 1, wherein:

3.

The image composition device,

A first frame buffer for storing images,

A second frame buffer for storing each frame constituting the moving image,

The first combining means includes:

The plurality of still images acquired by the first acquiring means are combined in the image synthesis order. The stored contents of the first frame buffer are combined with the read contents of the first frame buffer using the coefficients and the calculation information, and the combined contents are stored in the first frame buffer. Replace

The second acquisition means,

Each of the obtained frames is stored in the second frame buffer, and the second synthesizing means includes:

Each frame stored in the second frame buffer and the content stored in the first frame buffer are synthesized using the synthesis ratio of the moving image, and the resultant is stored in the second frame buffer.

3. The image synthesizing device according to claim 2, wherein:

Four.

4. The image synthesizing device according to claim 3, wherein: Five.

The image synthesizing device has a screen for image display, and the synthesizing by the first synthesizing unit, the obtaining by the second obtaining unit, and the synthesizing by the second synthesizing unit are performed in parallel.

The second frame buffer outputs to the screen at the same time as storing the synthesis result by the second synthesis means.

4. The image synthesizing device according to claim 3, wherein: The synthesis information further includes, for each of the plurality of images, a synthesis coefficient indicating a synthesis ratio with respect to a synthesis result of the image and an image other than the image.

2. The image synthesizing device according to claim 1, wherein:

7.

The image composition device,

A first frame buffer for storing images,

A second frame buffer for storing each frame constituting the moving image,

The first combining means includes:

Reading the plurality of still images obtained by the first obtaining means in accordance with the image synthesis order; synthesizing the storage contents of the first frame buffer with each read image using the synthesis coefficient; The result of the synthesis replaces the stored contents of the first frame buffer,

The second acquisition means,

Combining each frame stored in the second frame buffer with the stored contents of the first frame buffer using the combination ratio of the moving image, and storing the synthesized frame in the second frame buffer.

7. The image synthesizing device according to claim 6, wherein:

8.

The first synthesizing unit, after synthesizing the still image immediately before the moving image in the image synthesizing order, and before synthesizing the still image immediately after the moving image, calculates a synthesizing coefficient corresponding to the moving image. To perform a compositing operation on the image stored in the first frame buffer, and use the result of the compositing operation to generate the first frame. Replace the contents of the frame buffer

8. The image synthesizing device according to claim 7, wherein:

9.

The image combining device has a screen for displaying an image,

The combining by the first combining unit, the acquisition by the second acquiring unit, and the combining by the second combining unit are performed in parallel,

The second frame buffer outputs to the screen simultaneously with storage of the synthesis result by the second synthesis means.

8. The image synthesizing device according to claim 7, wherein:

Ten.

The image combining order represents an order of superimposing images, and the combining coefficient represents, for each of the plurality of images, a combining ratio of the image to a combining result from the rearmost image to the image in the image combining order. Lufa value,

The calculating means calculates a composition ratio of the moving image with respect to the composite image from alpha values corresponding to the moving image and all images located in front of the moving image.

7. The image synthesizing device according to claim 6, wherein:

1 1.

The image.

A first frame buffer for storing images,

A second frame buffer for storing each frame constituting the moving image,

The first combining means includes:

The image synthesis from a plurality of still images acquired by the first acquisition means; One still image is read in order from the foreground to the foreground indicated by the order, and the stored contents of the first frame buffer are combined with each read image using the alpha value. Replacing the stored contents of the first frame buffer with the result of the synthesis;

The second acquisition means,

Each of the acquired frames is stored in the second frame buffer, and the second synthesizing unit includes:

The respective frames stored in the second frame buffer and the contents stored in the first frame buffer are synthesized using the synthesis ratio of the moving image, and the resultant is stored in the second frame buffer.

11. The image synthesizing device according to claim 10, wherein:

12.

The first synthesizing unit uses the alpha value of the moving image after synthesizing the still image immediately before the moving image in the image synthesizing order and before synthesizing the still image immediately after the moving image in the image synthesizing order. Perform a combining operation on the image stored in the first frame buffer, and replace the contents of the first frame buffer with the result of the combining operation.

12. The image synthesizing device according to claim 11, wherein:

13.

The image synthesizing device has a screen for displaying an image, and includes:

The second frame buffer stores the result of synthesis by the second synthesis means and outputs the result to the screen at the same time.

12. The image synthesizing device according to claim 11, wherein:

14.

The image synthesizing device further comprises:

Replacement means for replacing the order of two adjacent images in the superposition order;

An updating means for obtaining and updating an alpha value corresponding to the two images after the replacement so that the combined result when the composition is performed before and after the replacement is the same as each other.

With

The first synthesizing unit, the calculating unit, and the second synthesizing unit perform each process using the order after the replacement by the replacing unit and the alpha value after the updating by the updating unit.

11. The image synthesizing device according to claim 10, wherein: 15 ..

The replacement means,

The order of the i-th image i whose alpha value is [i] and the order is counted from the lower order, and the order of the i-th image i +1 whose alpha value is and the order is counted from the lower order are interchanged.

The updating means sets an alpha value for the i-th image i + 1 after the replacement as a [i] *-a [i + 1]) and an alpha value for the i + 1-th image i after the replacement. Is α [1] ノ (1-aa [i + 1]))

15. The image synthesizing device according to claim 14, wherein:

16.

The image composition device,

A storage unit for storing the plurality of still images acquired by the first acquisition unit,

Each of the plurality of still images is the same or less than the composite image. It is composed of image data of prime numbers and layout information indicating a layout of the pixel data on the composite image,

The first synthesizing unit, the calculating unit, and the second synthesizing unit perform respective processes on a superimposed portion of the respective images determined from the ray information.

2. The image synthesizing device according to claim 1, wherein:

17.

The image composition device,

Each of the plurality of still images is represented in a vector data format, and the first synthesizing unit converts the vector data into pixel data and then synthesizes

2. The image synthesizing device according to claim 1, wherein:

18.

An image synthesizing device that generates a synthesized image by synthesizing a plurality of moving images and a plurality of still images,

Calculating means for calculating a combination ratio of the plurality of moving images with respect to the combined image based on the combination information;

A second acquisition for acquiring each frame constituting each of the plurality of moving images Means,

A second synthesizing unit for synthesizing each of the plurality of moving images and each frame thereof with the synthesized still image using a synthesizing ratio of each of the plurality of moving images.

An image synthesizing device, characterized in that:

19.

An image synthesizing apparatus for synthesizing a moving image and a plurality of still images to generate a synthesized image,

First acquisition means for acquiring the plurality of still images,

First synthesizing means for synthesizing the plurality of still images to generate one synthesized still image;

A second acquisition unit that acquires each frame that constitutes the moving image; and a second combination unit that combines each frame that constitutes the moving image with the combined still image.

An image synthesizing device, characterized in that:

20.

A computer-readable recording medium that stores a program for causing a computer to perform a process by combining a moving image and a plurality of still images to generate a composite image.

A first obtaining step of obtaining synthesis information for obtaining a synthesis ratio of each image before synthesis with respect to the synthesis image and the plurality of still images, which is synthesis information including an image synthesis order;

A first combining step of combining the plurality of still images based on the combining information to generate one combined still image;

A calculating step of calculating a synthesis ratio of the moving image to the synthesized image based on the synthesis information; A second obtaining step of obtaining each frame constituting the moving image; and synthesizing the respective frames and the synthesized still image using a synthesizing ratio of the moving image.

A recording medium characterized by the above-mentioned.

twenty one.

A program for causing a computer to execute a process of combining a moving image and a plurality of still images to generate a combined image,

A first obtaining step of obtaining synthesis information including an image synthesis order and obtaining the synthesis information and the plurality of still images for obtaining a synthesis ratio of each image before synthesis with respect to the synthesis image;

A calculating step of calculating a composite ratio of the moving image to the composite image based on the composite information;

A second acquisition step of acquiring each frame constituting the moving image, a second combining step of combining the said synthetic still image and the respective frame with the mixing ratio of the moving image - program of _"eta,