JP2001285749A - Image synthesizer, recording medium and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image synthesizer or the like for synthesizing plural moving images and still images at a high speed. SOLUTION: In this image synthesizer 100 for synthesizing the moving image and the plural still images and generating synthetic images, an EPG generation part 103 acquires synthesis information including an image synthesis order for obtaining the synthesis ratio of the respective images before synthesis to the synthetic image and the plural still images and stores them in an image holding part 104. A first synthesis part 108 generates one synthetic still image by synthesizing the plural still images on the basis of the synthesis information and stores it in an OSD plane 107, and a second synthesis part obtains the synthesis ratio of the moving image to the synthetic image on the basis of the synthesis information. A video reproducing part 105 stores the respective frames of the moving image in a video plane 106 and a third synthesis part 110 synthesizes the synthetic still image and the respective frames of the moving image and outputs them to an output part 111.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image display device, a recording medium and a program for synthesizing a plurality of images, and more particularly to a technique for synthesizing a plurality of moving images and still images.

[0002]

2. Description of the Related Art In recent years, digital broadcasting has been started. In digital television, a technology for generating a still image based on data broadcasted from a broadcasting station and superimposing the still image on a moving image of a broadcast program is displayed. Has been realized. The digital television includes an image synthesizing device as a mechanism for realizing this technology.

[0003] The image synthesizing apparatus, for the image F and the image G,
Pixel f (x, y) of image F and corresponding pixel in image G
g (x, y) to perform a predetermined inter-pixel operation using pixel h (x, y)
Output the image H with. One of the methods of this inter-pixel operation is an α-combining operation. The α synthesis operation is performed for the pixel f (x, y) and the pixel
g (x, y) is an operation to calculate the weighted average with the pixel h (x, y), and when the weight coefficient for the pixel g (x, y) is α,

[0004]

(Equation 1)

[0005] In (Equation 1), '*' represents a product. The weighting coefficient α is called an α value, transparency, or a blending coefficient, and takes a value of 0 to 1. When the α value is 0, the pixel g (x, y) becomes completely transparent, and the pixel f (x, y) becomes the pixel h as a synthesis result as it is. When the α value is 1, the pixel g (x, y) becomes completely opaque, and the pixel g (x, y) becomes a pixel h (x, y) as a synthesis result as it is. When the α value is 0.5, the pixel h (x, y)
Is a value obtained by mixing the pixel g (x, y) and the pixel f (x, y) at a ratio of half each. Thus, by adjusting the α value,
The overlapping of images can be expressed. In an actual hardware configuration, each pixel is represented by an RGB (red-green-blue) component for color display, so the above equation is performed for each component.

The image synthesizing apparatus includes an OSD plane as a memory area for expanding a still image and a memory area for expanding a moving image in units of frames in order to synthesize a still image and a moving image in real time. Each time a frame of the video plane is updated, an image of these two areas is synthesized and output by an α synthesis operation. FIG. 31A is a conceptual diagram illustrating a state in which images of the OSD plane 2502 and the video plane 2501 are combined and a combined image 2503 is output.

[0007]

In recent digital broadcasting, for example, as shown in FIG. 31 (b), while reproducing a moving image 2512 as a broadcast program, the title of the broadcast program and the television program are displayed on the same screen. A plurality of still images 2511 such as columns,
More diverse expressions are required, such as superimposing and displaying 2513, 2514, and 2515.

To combine a moving image and a plurality of still images,
Theoretically, the α-combining operation of (Equation 1) may be performed sequentially from the lower layer. However, in reality, the amount of calculation of the α combining operation for combining a plurality of images is enormous, and it is extremely difficult to perform real-time combining every time a frame of a moving image is updated. By preparing a plane corresponding to each image and a plurality of hardware that performs the α synthesis operation, the synthesis processing can be performed at high speed. However, hardware costs are increased, and many planes for image development are used. There is a problem that it becomes necessary.

In order to achieve the above object, the present invention provides an image synthesizing apparatus, a recording medium, and a program for synthesizing in real time in accordance with a reproduction rate of a moving image, that is, for performing a synthesizing process at high speed. Aim. A second object of the present invention is to provide an image synthesizing apparatus, a recording medium, and a program in which the memory for image expansion is reduced.

[0010]

In order to achieve the above object, an image synthesizing apparatus according to the present invention is an image synthesizing apparatus for synthesizing a moving image and a plurality of still images to generate a synthetic image. First obtaining means for obtaining the synthesis information and the plurality of still images for obtaining a synthesis ratio of each image before synthesis with respect to the synthesis image, the synthesis information including an order, and based on the synthesis information. A first combining unit that combines the plurality of still images to generate one combined still image; a calculating unit that determines a combining ratio of the moving image to the combined image based on the combining information; And a second synthesizing unit for synthesizing each of the frames and the synthesized still image using a synthesizing ratio of the moving image.

[0011]

Embodiments of the present invention will be described below with reference to the drawings. First Embodiment <Configuration> FIG. 1 is a block diagram showing a configuration of an image synthesizing apparatus according to a first embodiment of the present invention.

In FIG. 1, an image synthesizing apparatus 100 includes an input unit 1.
01, control unit 102, EPG generation unit 103, image holding unit 104, video playback unit 105, video plane 106, OSD plane 107, first
It comprises a combining unit 108, a second combining unit 109, a third combining unit 110, and an output unit 111. The input unit 101 has a remote controller, front panel buttons, and the like, and receives instructions from the user when these are operated. Specific instructions include an on / off instruction for the image synthesizing apparatus 100, a channel switching instruction, and an on / off instruction for EPG (Electronic Program Guide) display. Here, the on / off instruction of the image synthesizing apparatus 100 indicates that the power of the image synthesizing apparatus 100 is turned on when it is on, and that the power is turned off when it is off.
The display on / off indicates that the EPG screen is displayed when the display is on, and the EPG screen is not displayed when the display is off.

The EPG is a system for displaying a program table of a broadcast program, information on the contents of the broadcast program, and the like on a television screen. The EPG display means a state in which a program guide or the like is displayed on a television screen. As an application example of the EPG, for example, a user can perform a search by a program genre or a performer name on an EPG display screen, or can easily make a recording reservation using a video deck compatible with the EPG.

When the EPG display is on, the image synthesizing apparatus 100 displays a synthetic image 201 shown in FIG. 2A as an example.
The composite image 201 in FIG. 2A includes components 202, 203, and 204, as shown in FIG. The component 202 is a still image representing a program table, the component 203 is a moving image representing the content itself of a broadcast program of one channel, and the component 204 is a still image representing the program name of the component 203. The components 202 and 204 are generated by the EPG generation unit 103 based on the EPG information transmitted from the broadcasting station.

As described above, the image synthesizing apparatus 100 displays the component 202 while the EPG display is on, displays the component 203 at a lower right position of the component 202 at a rate of several tens of frames per second, and Is displayed at the position of. As described above, the image synthesizing apparatus 100 of the present embodiment has a function of generating and displaying a synthesized image in which a plurality of components including a plurality of still images and one moving image are combined.

Hereinafter, description will be made with reference to FIGS. 2A and 2B as needed. In addition, a still image component and a moving image component will be simply referred to as components unless it is necessary to distinguish them. The control unit 102 controls all the components of the image composition device 100.
More specifically, when the image synthesizing apparatus 100 is instructed to be turned on, the video reproducing unit 105 reproduces a broadcast program. When the channel switching is instructed, the video reproducing unit 105 reproduces a broadcast program different from the one currently being reproduced. When instructed to turn on the EPG display, the EPG generation unit 103 and the video playback unit 10
5. Control the first combining unit 108, the second combining unit 109, the third combining unit 110, and the like to generate the combined image 201. The processing related to the EPG display will be described later.

[0017] The EPG generation unit 103 outputs an EP broadcast from a broadcast station.
Acquire and retain G information. The EPG information includes layout information such as the display size and display position of the component, the order of overlapping the components, and still image information of the still image component. The still image information corresponds to the image content of the still image component, and includes text, graphics data, and the like. The EPG generation unit 103 generates a plurality of image files and one index file based on the EPG information, and stores them in the image holding unit 104. Further, the EPG generation unit 103 extracts the display size and the display position of the component of the moving image constituting the EPG display screen from the EPG information, and outputs the same to the video reproduction unit 105.

The image file is composed of the components shown in FIG.
It corresponds to 202, 203, and 204, and is composed of the display size, display position, and image data of the component. In the case of a still image component, image data is generated based on still image information. The index file is for managing a plurality of image files and includes a plurality of pieces of image information. One piece of image information corresponds to one image file. The plurality of pieces of image information are arranged in the index file in the same order as the overlapping order of the components in FIG. The image information includes an image type and a storage position. The image type indicates whether the image of the image file is a still image or a moving image. The storage position indicates the position of the head of the image file in the image holding unit 104.

Next, specific examples of the image file and the index file will be described with reference to FIGS. FIG. 3 is a diagram for explaining the structure of an image file. In the image file 301 shown in the figure, the first line indicates the display position, the second line indicates the display size, and the third and subsequent lines indicate the image data. The image data is composed of a set of pixel data corresponding to all the pixels of the component, and the pixel data indicates a color of the pixel.
It is composed of an RGB value and an α value representing the transparency of the pixel. That is, the α value indicates a composition ratio of the image to the composition result from the rearmost image to the image. The storage position 302 described on the left side of the image file 301 indicates the storage position of each data based on the case where the start position of the image file 301 is “0000”.

In the image file 301, the third line is pixel data corresponding to the pixel at the coordinate position (0,0), the fourth line is the pixel data corresponding to the pixel at the coordinate position (1,0),
The row is pixel data corresponding to the pixel at the coordinate position (2,0). Here, the coordinate position (0,0) indicates the coordinate position of the upper left corner of the component. As described above, the pixel data corresponds to pixels in order from left to right and from top to bottom of the component.

The display position is the coordinate position of the component on the composite image 201, and is represented by the X and Y coordinates of the upper left corner of the component with the origin at the upper left corner of the composite image 201. The display size is made up of the height H and width W of the rectangle of the component, and the height H and width W are expressed in pixel units. Each of the RGB components takes a value from 0 to 255. When all three components are 0, the color of the pixel is black, and when all three are 255, the pixel is white.

The α value takes a value from 0 to 255, and represents the transparency when the pixel is superimposed on another pixel, that is, how much the pixel in the lower layer transmits through the pixel in the upper layer. . More specifically, pixel B
When the pixel A is synthesized with the α value, the pixel C of the synthesis result is C = (αA + (255−α) B) / 255. α value is 0
The value of ~ 255, and if the α value is 0, pixel A is transparent and 100% transparent to pixel B. When the α value is 255, pixel A is opaque, does not transmit pixel B, and is 100% of pixel A. When the α value is 128, pixel A transmits pixel B approximately 50%, that is, pixel A
And the pixel B are mixed by about 50%.

In practice, the pixels A, B and C are represented by RGB components, and the composition result is calculated for each component. EPG generator 103
Sets the α value of the image file of each of the components 202, 203, and 204 as follows. That is, the EPG generation unit 103 sets the α values of all pixels in the component 202 to 255. This is component 2
This is because 02 is the lowest layer component in the composite image 201. In addition, the EPG generation unit 103
Is set to 192 for all pixels at. As for the α value of the component 204, 255 is set for the character portion, and 64 for the other portions.

The method of determining the RGB component and the value of the α value in detail is a well-known technique and is not a characteristic part of the present invention, so that the description is omitted. FIG. 3 shows the correspondence between the content of the image data and one component. As shown in the drawing, starting from the upper left of the component 303, it corresponds to pixel data from left to right and from top to bottom one pixel at a time.

The EPG generator 103 generates the image file described above. However, for the components of the moving image, an image file is generated by setting all RGB components to 0. That is, the EPG generation unit 103 determines the display position, the display size, and the α value for the image file of the moving image component, and generates an image file in which the RGB components are not determined. This is because the components of a moving image correspond to a broadcast program and are transmitted in real time from a broadcasting station, so that the RGB components are not determined when an image file is generated. The RGB components of the image file corresponding to the components of the video are
Determined by 05.

FIG. 4 shows an example of the index file. The index file 410 in FIG.
422 and 423, and represent the overlapping order in the same order. That is, the image information 421 corresponds to the component 202 of the lowest layer shown in FIG. 2B, the image information 422 corresponds to the component 203 of the next layer, and the image information 423 corresponds to the component 204 of the highest layer. . The value in the column 411 indicates the image type of the component, “0” is a still image, “1”
Indicates a moving image. The value in column 412 indicates the storage location of the image file.

The images 430, 440, and 450 correspond to the components 202, 203, and 204, respectively, and represent the image data in the respective image files. Here, the image 440 indicates that all the RGB components of the image data corresponding to the component 203 are 0. The image holding unit 104 includes a memory, a hard disk, and the like, and holds the image file and the index file generated by the EPG generation unit 103.

The video reproducing unit 105 receives and decodes a broadcast program from a broadcast station, reproduces a moving image of several tens of frames / second, and sequentially stores the moving image data in the video plane 106. At the time of storage, the video playback unit 105 determines the layout of the video for the composite image 201 based on the display position and the display size input from the EPG playback unit 103, and stores the video in an area on the video plane 106 corresponding to the layout. of
Stores R, G, B components.

The video plane 106 is composed of a memory or the like, and includes moving image data stored by the video reproducing unit 105,
It holds the combined α value stored by the second combining unit 109 described later. The composite α value represents the transparency of one pixel among all pixels when a plurality of pixels are overlapped and composited, and is calculated using the α value of each pixel. That is, the composite α value held by the video reproduction unit 105 represents the transparency of the pixels of the moving image component among all the pixels to be superimposed when all the components are superimposed and synthesized.

FIG. 5 shows the structure of moving image data. As shown in the figure, the moving image data 501 is a set of pixel data, and the pixel data includes an RGB component and a combined α value. An image 502 is a diagram for illustrating a correspondence relationship between pixel data and a pixel position on a screen. FIG.
The pixel data of the moving image data 501 is shown as an image. As shown in the figure, an image 6 corresponding to the moving image data 501
00 includes an area 601 corresponding to the component 203 and an area 602 other than the area 601. Image 600 is composite image 20
Has the same height and width as 1. The video plane 106 is
The RGB component of 601 is given from the video playback unit 105,
The value is given from the second combining unit 109. Also video plane
106 holds 0 in advance as the RGB components and the combined α value of the area 602. The RGB components of the area 601 are updated at several tens of frames / second in accordance with the playback rate of the video playback unit 105.

The OSD plane 107 is composed of a memory or the like,
The first synthesis image data output by the first synthesis unit 108 is held. FIG. 7 shows the data structure of the first combined image data. The first combined image data 701 is composed of a set of RGB components, and corresponds to a result of combining the components 202 and 204, that is, a result of combining still image components. The composition will be described later.

An image 702 shows the correspondence between the RGB components of the first combined image data 701 and the pixel positions on the screen. As shown in the figure, the RGB components of the first combined image data 701 are arranged in order from left to right and from top to bottom of the image 702. image
702 has the same height and width in pixel units as the composite image 201. The first combining unit 108 combines the image data of the plurality of image files held in the image holding unit 104 to generate first combined image data, and stores the first combined image data in the OSD plane 107. This processing is referred to as first synthesis processing.

FIG. 8 is a flowchart showing the procedure of the first synthesizing process. First, the first combining unit 108 initializes the OSD plane 107 (Step S800). Specifically, OSD
All the RGB component areas of the plane 107 are set to 0.
Next, the first combining unit 108 repeats the processing of steps S801 to S807, and performs processing of combining the components of the still image in order from the lower layer to the component of the upper layer.

First, the first synthesizing unit 108 reads out image information i from the index file of the image holding unit 104 (step S801). Here, i is 0, 1, in order from the lower layer to the component of the upper layer for convenience in this flowchart.
Are variables indicating the numbers assigned to the components in ascending order, such as 2,..., And the image information and the image file corresponding to the component i are called image information i and image file i, respectively. In this flowchart, the initial value of i is 0, and the increment is 1.

Next, the first synthesizing unit 108 stores the image file i stored in the storage position indicated by the image information i in the image holding unit 104
Is taken out (step S802). The first synthesizing unit 108 reads the display size and the display position of the image file i, and sets an overlapping range of the component i of the image file on the screen (step S803).

The first combining unit 108 determines whether the image type indicated by the image information i is a moving image or a still image (step S804). If the result of the determination is that the image is a still image,
The first synthesis unit 108 performs an α synthesis operation on the RGB components of the image file i and the RGB components in the overlapping range of the OSD plane 107 (step S805). The equation for the α synthesis operation is shown below.

[0037]

(Equation 2)

In this equation, R (x, y), G (x, y), B
(x, y) is a newly obtained RGB component, and R _i (x, y) and G _i (x, y
y), B _i (x, y) and α _i (x, y) are the RGB components and α value of the image file i, and R (x, y), G (x, y), B (x , y) are the RGB components held within the overlapping range of the OSD plane 107. That is, R (x, y) on the left side is a value obtained by weighting R _i (x, y) with α _i and R (x, y) held in the OSD plane 107.
It is obtained by adding a value weighted by 1−α _i . G (x,
The same applies to y) and B (x, y). The first synthesis unit 108 converts the newly obtained R (x, y), G (x, y), B (x, y) into the OSD plane 107
To be stored.

On the other hand, in step S804, image information i
If it is determined that the image type shown in
1 The synthesizing unit 108 calculates the α value of the image file i and the OSD plane 107
The following calculation is performed using the RGB components in the overlapping range (step S806). The formula is shown below.

[0040]

(Equation 3)

In this equation, R (x, y), G (x, y), B
(x, y) is a newly obtained RGB component, α _i (x, y) is an α value of the image file i, and the right side R (x, y), G (x, y), B (x, y ) Is O
RGB held within the overlapping range of SD plane 107
Component. That is, R (x, y) newly obtained is obtained by weighting R (x, y) obtained so far by 1−α _i (x, y). The same applies to G (x, y) and B (x, y).

(Equation 3) is the same as (Equation 2) except that the first term on the right side is set to 0. The right side of each expression in step S805
The first term is to weight α to the RGB component of the image file, and in the case of each expression of S806, it means that the weighting of α to the RGB component of the moving image is not added. The first combining unit 108 calculates the R calculated in step S805 or S806.
(x, y), G (x, y), and B (x, y) are stored in the OSD plane 107.

As described above, the first synthesizing unit 108 ends the first synthesizing process after processing all image files (step S807). By the above processing, the result of synthesizing the components of the still image is stored in the OSD plane 107. The second combining unit 109 calculates a combined α value for the moving image component, and
Performing Second Combining Process Stored in 6 FIG. 9 is a flowchart showing a procedure of the second combining process.

First, the second synthesizing unit 109 sets the video plane 10
The area of the combined α value of 6 is initialized (step S900). Specifically, the area for the combined α value of all the pixels of the video plane 106 is set to 0. The composite α value area holds the composite α value for the moving image component as a result of the processing in FIG. Next, the second synthesis unit 109 repeats the processing of steps S901 to S907.

First, the second synthesizing unit 109 reads out image information i from the index file of the image holding unit 104 (step S901). Here, i is 0, 1, in order from the lower layer to the component of the upper layer for convenience in this flowchart.
Are variables indicating the numbers assigned to the components in ascending order, such as 2,..., And the image information and the image file corresponding to the component i are called image information i and image file i, respectively. In this flowchart, the initial value of i is 0, and the increment is 1.

Next, the second synthesizing unit 109 stores the image file stored in the storage position indicated by the image information i in the image holding unit 104
Is taken out (step S902). The second synthesizing unit 109 reads the display size and the display position of the image file i, and sets the overlapping range of the component i on the screen (Step S903). The second synthesizing unit 109 determines whether the image type indicated in the image information i is a moving image or a still image (step S904).

If the result of the determination is that the image is a still image, the second combining unit 109 calculates a new combined α value using α _{i of the} image file i and the combined α value in the overlapping range of the video plane 106 ( Step S905). This equation is shown below

[0048]

(Equation 4)

In this equation, α (x, y) on the left side is a newly obtained composite α value, α _i (x, y) is the α value of the image file i, and α (x, y) on the right side is The composite α value held in the overlapping range of the video plane 106. 2nd synthesis unit 1
09 stores the newly obtained α (x, y) in the video plane 106. On the other hand, in step S904, when it is determined that the image type indicated by the image information i is a moving image,
The second combining unit 109 performs the following calculation (step S906).

[0050]

(Equation 5)

In this equation, α (x, y) on the left side is a newly obtained composite α value, and α _i (x, y) on the right side is an image file i
Is the α value of That is, α _i (x, y) is directly used as α (x, y) to be newly obtained. The second synthesizing unit 109 determines in step S905
Alternatively, α (x, y) calculated in step S906 is stored in video plane 106.

In this way, the second synthesizing unit 109 finishes the second synthesizing process after processing all image files (step S907). As a result, the video plane
In 106, finally the composite α for the moving image component
The value is stored. The third synthesis unit 110 is held in the video plane 106 upon receiving a normal reproduction instruction from the control unit 102
The R, G, and B components are output to the output unit 111. Also, the third synthesis unit 110
When an EPG display instruction is received from the control unit 102, the R, G, B components of the moving image data stored in the video plane 106 and the OS
A third combination is performed to combine the RGB components of the first combined image data stored in the D plane 107, and the combined image as a combined result is output to the output unit 111.

The third synthesis processing is represented by the following equation.

[0054]

(Equation 6)

Where R (x, y), G (x, y) and B (x, y) are
As a result of the third synthesis, each pixel output to the output unit 111
R, G, B components, α (x, y), R _v (x, y), G _v (x, y), B
_v (x, y) is a composite α value of moving image data stored in the video plane 106 and R, G, B components, and Ro (x, y), Go
(x, y) and Bo (x, y) are the R, G, B components of the first combined image data.

That is, the third synthesizing unit 110 outputs the video plane 10
Combined with R, G, B components of moving image data held in 6
The value multiplied by the value and the first value held in the OSD plane 107
The processing of adding the R, G, and B components of the composite image data is performed for each frame of video reproduction. The output unit 111 includes a CRT or the like, and receives the R, G, and B components output by the third combining unit 110 and displays the components on a screen. <Operation> The operation of the image synthesizing apparatus 100 configured as described above will be described below.

FIG. 10 shows the E of the image synthesizing apparatus 100 of this embodiment.
FIG. 9 is a diagram for explaining the flow of operation during PG display. In the figure, the contents of the rectangles indicate the operation of each component, and the arrows indicate the flow of data. When the input unit 101 receives an EPG display instruction (step S1001), the instruction is transmitted to each unit through the control unit 102. EPG generation unit 103 receiving the instruction
Generates an image file and an index file based on the broadcasted EPG information, and stores them in the image holding unit 104. Further, the EPG generation unit 103 notifies the video reproduction unit 105 of the display position and display size of the moving image component acquired at the time of generating the image file (Step S1002).

The first synthesizing unit 108 synthesizes the RGB components within the overlapping range of the still image components stored in the image holding unit 104, and stores them in the OSD plane 107 (step S1003). The second combining unit 109 calculates a combined α value for the moving image component based on the α value of the image file stored in the image holding unit 104, and stores the combined α value in the video plane 106 (step S1004).

The video reproducing unit 105 reproduces a moving image,
The moving image data is stored in the video plane 106 with the layout indicated by the display size and display position notified from the generation unit 103 (step S1005). The third combining unit 110 weights the RGB components of the moving image component with the combined α value, adds the RGB components of the first combined image data thereto, and outputs the RGB components of the combined image obtained after the addition to the output unit 111 ( Step S
1006).

The output unit 111 displays the RGB components of the composite image from the third compositing unit 110 (step S1007).
Steps S1005 and S1006 are performed in parallel. The processing in step S1006 is performed at the same rate as the reproduction of the moving image in step S1005. <Capture Explanation> The capture of a moving image and a still image, which is a feature of the present invention, will be described below.

FIG. 11 shows a component group consisting of N + 1 moving image components or still image components. Each component has the R, G, B components and α values shown in FIG. However, the α value of the lowermost component is α0 = 1.0. The R, G, and B components of the image obtained as a result of superimposing these component groups are
It is given by (Equation 7).

[0062]

(Equation 7)

Here, β _i represents the composition ratio of component i to the image that is the final composition result,
Also called contribution. That is, β _i is a combined α value. The present invention realizes an operation in which, when one of these components is a moving image, the components of the still image other than the components of the moving image are combined using the α value, and the component of the moving image is added to the result. ing. The still image component exists as an image file with the display size and display position.
Since synthesis is performed on a plane, a still image component can be efficiently synthesized with a small memory capacity of one OSD plane 107.

Thus, the processing can be efficiently performed with a small memory of one OSD plane for a still image and one video plane for a moving image. If the composition of these components is performed sequentially in the order of superimposition including both moving images and still images, it is necessary to calculate the composition ratio for each pixel of all components and store it in the memory. . This requires a lot of memory and is inefficient.

Also, the formula for calculating the synthesis ratio requires a large number of multiplications if performed independently one by one. For N + 1 components, N * (N-1) / 2 multiplications are required, which is a calculation on the order of N ^ 2. The present invention solves these problems and realizes a sequential calculation that does not require a memory and the number of times of multiplication is reduced to the order of N. The relationship between the components and the operation of the present embodiment is that the first combining unit 108 is in charge of adding the terms of the components other than the moving image, the second combining unit 109 is in charge of calculating the contribution of the moving image, 3 The synthesizing unit 110 is responsible for adding the last moving image.

FIG. 12 shows the operation of the present invention described in C language style.
Program. This program has one pixel
Focus on R, G and B are one on the OSD plane
Represents the RGB component of a pixel, and α corresponds to this pixel
Represents the combined α value of the moving image. R _i, G_i, B_i, Α_iOSD
RGB component of i-th component added to lane and
And α value. R_v, G_v, B_vIs the RGB component of the moving image.

The first to fourth lines indicate the initialization processing. The 5th to 17th lines are for addition of still images, and the rest are for addition of moving images. The seventh, eighth, ninth, twelfth, thirteenth, and fourteenth rows indicate the operations performed by the first combining unit 108. here,
The formula for synthesizing a moving image is equivalent to calculating assuming that the RGB components of the moving image are 0. The tenth and fifteenth lines are calculations performed by the second synthesizing unit 109 to calculate the synthesized α value of the moving image. The fifth, sixth, eleventh, sixteenth, and seventeenth rows are processes common to the first combining unit 108 and the second combining unit 109. 1
Lines 8, 19, and 20 are operations performed by the third synthesis unit 110. Here, lines 18, 19, and 20 are performed in parallel by another thread, and may be a permanent loop.

FIG. 13 shows a program that realizes this. program1 and program2 divide the program of FIG. 12, and program2 makes the output a permanent loop. These two programs can be processed in parallel. In the case of performing the parallel processing, it is possible to display a state in the middle of the superposition and a process of the superposition. <Embodiment 2><Configuration> FIG. 14 is a block diagram showing a configuration of an image synthesizing apparatus according to Embodiment 2 of the present invention.

In the figure, the image synthesizing device 200 is the input unit 1
01, control unit 1500, EPG generation unit 103, image holding unit 104, video reproduction unit 105, video plane 106, OSD plane 1501,
It comprises a fourth synthesis unit 1502, a fifth synthesis unit 1503, and an output unit 111. Among these constituent elements, those having the same reference numerals as those of the image synthesizing apparatus 100 have the same functions, and therefore the description thereof will be omitted.

The control section 1500 controls all the components of the image synthesizing apparatus 200. More specifically, the image synthesizing device 200
Is turned on, the video playback unit 105 plays back the broadcast program. When the channel switching is instructed, the video reproducing unit 105 reproduces a broadcast program different from the one currently being reproduced. When instructed to turn on the EPG display, the EPG generation unit 10
3.Video playback unit 105, fourth synthesis unit 1502, and fifth synthesis unit 1503
And the like to generate a composite image.

The OSD plane 1501 is composed of a memory or the like, and holds the RGB components and α values output by the fourth synthesizing unit 1502. The OSD plane 1501 holds RGB components and α values corresponding to pixels on the screen. Upon receiving the instruction from the control unit 1500, the fourth combining unit 1502 combines the image data of the plurality of image files held in the image holding unit 104, and finally generates fourth combined image data, and Plain 1501
To be stored. This processing is called a fourth synthesis processing.

FIG. 15 is a flowchart showing the procedure of the fourth synthesizing process. First, the fourth synthesizing unit 1502 sets 0 to all the RGB component areas of the OSD plane 1501 and sets 0 to all of the synthesized α value areas of the video plane 106 (Step S1600, Step S1601). Next, the fourth synthesis unit 1
502 repeats the processing of steps S1602 to S1612, synthesizes the components of the still image in order from the lower layer to the component of the upper layer, stores it in the OSD plane 1501, and also sets the synthesized α value for the moving image component. Is calculated and stored in the video plane 106.

First, the fourth synthesizing unit 1502 reads out image information i from the index file of the image holding unit 104 (step S1602). Here, i is 0, 1, in order from the lower layer to the component of the upper layer for convenience in this flowchart.
Are variables indicating the numbers assigned to the components in ascending order, such as 2,..., And the image information and the image file corresponding to the component i are called image information i and image file i, respectively. In this flowchart, the initial value of i is 0, and the increment is 1.

The fourth synthesizing unit 1502 stores the image file i stored in the storage location indicated by the image information i in the image holding unit 1
Take it out from 04 (step S1603). The fourth synthesis unit 1502
Read the display size and display position of image file i,
The overlapping range of the component i of the image file on the screen is set (step S1604).

The fourth synthesizing unit 1502 determines whether the image type indicated by the image information i is a moving image or a still image (step S1605). As a result of the determination, when it is determined that the image type indicated by the image information i is a moving image, the fourth combining unit 150
2 copies the α value of the image file i to the video plane 106 (step S1611).

As a result of the determination in step S1605, if it is determined that the image type indicated by the image information i is a still image, the fourth synthesizing unit 1502 determines that the synthesized α value held in the video plane 106 is all 0 and It is determined whether or not there is (step S1606). That is, here, the processing branches depending on whether or not there is a moving image in a lower layer than the still image.

If it is determined in step S1606 that the combined α values held in the video plane 106 are all 0, the fourth combining unit 1502 copies the α value of the image file i to the OSD plane 1501. (Step S1609).
As a result of the determination in step S1606, the video plane 106
If all the combined α values held in the video plane 106 are not 0, that is, at least one is not 0, the fourth combining unit 1502 uses the combined α value held in the video plane 106 and the α value of the image file i. A new α value is obtained by the calculation of (Equation 8) and stored in the OSD plane 1501 (step S160)
7).

[0078]

(Equation 8)

In this equation, αosd (x, y) on the left side is a newly obtained α value, and αi (x, y) on the right side is the α value of the image file i.
Α _v (x, y) is the composite α value held in the video plane 106. Further, the fourth synthesis unit 1502 calculates
The composite α value for the moving image is calculated according to 9) and stored in the composite α value area of the video plane 106 (step S1608).

[0080]

[Equation 9]

[0081] α _v (x, y) of the left side in this equation is a synthetic alpha value to be stored in the new video plane 106, the right side alpha _v
(x, y) is the composite α value held in the video plane 106 before this storage, and αi (x, y) is the α of the image file i.
Value. The fourth synthesis unit 1502 performs an α synthesis operation on the RGB components of the image file i and the RGB components in the overlapping range of the OSD plane 1501, using the α value set in the OSD plane 1501 in step S1607 or step S1609. The result is stored in the OSD plane 1501 (step S161
0). The equation is shown in (Equation 10).

[0082]

[Equation 10]

In this equation, R (x, y), G (x, y), B
(x, y) is a newly obtained RGB component, and R (x, y), G on the right side
(x, y), B (x, y) and αosd are the RGB components and α value held in the OSD plane 1501, and Ri (x, y), Gi (x, y), Bi
(x, y) is an RGB component of the image file i. In this way, the fourth synthesis unit 1502 performs the processing from step S1602 to step S1612 on all the image files, and ends the fourth synthesis processing (step S1612). After the end of the fourth synthesis processing, the result obtained by synthesizing all the components of the still image in the OSD plane 1501 is held, and the video plane 106
The composite α value of the component of the moving image is held in the area of the composite α value of.

Upon receiving a normal reproduction instruction from the control unit 1500, the fifth synthesizing unit 1503
The G and B components are output to the output unit 111. Fifth synthesis unit 1503
When an EPG display instruction is received from the control unit 1500, the R, G, B components and the combined α value of the moving image data stored in the video plane 106 and the RGB of the fourth combined image data stored in the OSD plane 1501 A fifth synthesis for synthesizing the components is performed, and the resultant synthesized image is output to the output unit 111.

The fifth synthesis processing is represented by the following equation.

[0086]

[Equation 11]

Here, R (x, y), G (x, y) and B (x, y) are
As a result of the fifth synthesis, each pixel output to the output unit 111
R, G, B components, α (x, y), R _v (x, y), G _v (x, y), B
_v (x, y) is a composite α value of moving image data stored in the video plane 106 and R, G, B components, and Ro (x, y), Go
(x, y) and Bo (x, y) are the R, G, B components of the fourth combined image data.

As described above, the fifth synthesizing unit 1503 performs synthesis with the fourth synthesized image data every time a frame is updated. <Operation> The operation of the image synthesizing apparatus 200 configured as described above will be described below.

FIG. 16 shows the E of the image synthesizing apparatus 200 of this embodiment.
FIG. 9 is a diagram for explaining the flow of operation during PG display. In the figure, the contents of the rectangles indicate the operation of each component, and the arrows indicate the flow of data. Note that, in this figure, the operations at the same step numbers as those in FIG. 10 mean the same operations as those in FIG. When the input unit 101 receives an EPG display instruction (step S1001), the instruction is transmitted to each unit through the control unit 1500. EPG generation unit 103 receiving the instruction
Generates an image file and an index file based on the broadcasted EPG information, and stores them in the image holding unit 104. Further, the EPG generation unit 103 notifies the video reproduction unit 105 of the display position and display size of the moving image component acquired at the time of generating the image file (Step S1002).

The fourth synthesizing unit 1502 synthesizes the RGB components within the overlapping range of the still image components stored in the image holding unit 104, and stores them in the OSD plane 1501. Further, the fourth combining unit 1502 calculates a combined α value for a moving image and stores the combined α value in the video plane 106 (step S1701). The video reproduction unit 105 reproduces the moving image, and stores the moving image data in the video plane 106 in the layout indicated by the display size and the display position notified from the EPG generation unit 103 (Step S100)
Five).

The fifth synthesizing unit 1503 weights the RGB components of the moving image component with the synthesized α value, adds the RGB components of the fourth synthesized image data thereto, and calculates the R of the synthesized image obtained after the addition.
The GB component is output to the output unit 111 (step S1702). Output section
111 displays the RGB components of the composite image from the fifth compositing unit 1503 (step S1007).
This is performed in parallel with step S1701. <Supplementary explanation> As shown in FIG. 17, the synthesizing method according to the present embodiment performs synthesizing by changing the overlapping order of two adjacent components. At this time, the α values of the two components after the replacement are set in steps S1607 and S1607 in FIG. 15 so that the synthesis result before the replacement is the same as the synthesis result after the replacement.
Update with the formula shown in 1608.

That is, if the α value of the component i in the lower layer of the two components after replacement is αi + 1 ′ and the α value of the component i + 1 in the upper layer is αi ′, the respective α values are as follows: It is represented by the equation (Equation 12).

[0093]

(Equation 12)

Here, (Equation 12) is obtained as follows. Component i and component i before replacement
Assuming that the α value of +1 is αi and αi + 1, respectively, the contributions βi and βi + 1 of component i and component i + 1 to the final composite image are given by (Equation 7).

[0095]

(Equation 13)

## EQU10 ## On the other hand, when the contributions β _{i + 1} ′ and β _i ′ of the component i + 1 and the component i after the replacement for the final composite image are expressed by using (Equation 7),

[0097]

[Equation 14]

Is obtained. In order for the combined result to be the same before and after the replacement, it is only necessary that (Equation 13) and (Equation 14) have the same value. Ie

[0099]

(Equation 15)

By solving these equations for α _i ′ and α _{i + 1} ′, (Equation 12) is obtained. By utilizing this rule and replacing the components of the moving image with the components of the still image so that the components of the moving image are at the top layer as shown in FIG. 18, all the still image components are synthesized before the moving image components, Finally, it realizes composing a moving image.

The fourth synthesizing unit 1502 efficiently calculates the α value required for the replacement of adjacent components and synthesizes it with the OSD plane 1501 as shown in the flowchart of FIG. The fifth synthesizing unit 1503 synthesizes the OSD plane 1501 with the video plane 106. Third Embodiment <Structure> FIG. 19 is a block diagram showing a structure of an image synthesizing apparatus according to a third embodiment of the present invention.

In the same figure, the image synthesizing device 300 has an input unit 1
01, control unit 2000, EPG generation unit 2003, image holding unit 104, video reproduction unit 2001, video plane 2002, OSD plane 107,
It comprises a sixth synthesis unit 2004, a seventh synthesis unit 2005, an eighth synthesis unit 2006, and an output unit 111. With this configuration, the image synthesizing device 3
00 synthesizes a plurality of moving image components and a plurality of still image components. Among these constituent elements, those having the same reference numerals as those of the image synthesizing apparatus 100 and the image synthesizing apparatus 200 have the same functions, and thus the description thereof will be omitted, and the explanation will be focused on those having different reference numerals.

The control unit 2000 controls all the components of the image synthesizing apparatus 300. More specifically, the image synthesizing device 300
Is turned on, the video reproducing unit 2001 reproduces a broadcast program. When the channel switching is instructed, the video reproducing unit 2001 reproduces a broadcast program different from the one currently being reproduced. When instructed to turn on the EPG display, the EPG generation unit 200
3, video playback unit 2001, sixth synthesis unit 2004, seventh synthesis unit 2005
And the eighth combining unit 2006 and the like are controlled to generate a combined image.

The video playback unit 2001 has a plurality of playback units, ie, a first playback unit, a second playback unit,..., And an N-th playback unit. And performs decoding and the like, reproduces a moving image of several tens of frames / second, and stores the moving image data in the video plane 2002 while overwriting and updating it in frame units. At the time of storage, the video playback unit 2001 determines the layout of the moving image with respect to the final composite image based on the display position and display size input from the EPG playback unit 2003, and corresponds to the layout on the video plane 2002. The R, G, and B components of the video are stored in the area.

The video plane 2002 has a plurality of planes therein, that is, a first plane, a second plane,..., An N-th plane. Each plane is constituted by a memory or the like. The moving image data corresponding to the plurality of playback units and stored by the playback unit and the combined α value stored by the seventh combining unit 2005 are held. EPG generator 2003
Obtains EPG information for EPG display including a plurality of moving image components and still image components from a broadcast station, generates a plurality of image files and one index file based on the EPG information, Store in 104. Further, the display size and display position of a plurality of moving image components constituting the EPG display screen are extracted from the EPG information, and output to the video playback unit 2001.

The EPG generation unit 2003 is the same as the EPG generation unit 103 except that it supports a plurality of moving image components. The sixth combining unit 2004 combines the image data of the plurality of image files held in the image holding unit 104 to generate sixth combined image data, and stores the sixth combined image data in the OSD plane 107. This processing is called a sixth synthesis processing. This processing is almost the same as the first synthesis processing shown in FIG.
The difference from the first embodiment is that the process of S806 is performed a plurality of times corresponding to a plurality of moving images.

Upon receiving an instruction from the control unit 2000, the seventh combining unit 2005 calculates a combined α value for each of a plurality of moving image components, and stores the combined α value in the corresponding plane of the video plane 2002. I do. Fig. 20
15 is a flowchart showing a seventh synthesis processing procedure.
First, the seventh synthesis unit 2005 generates a synthesis α of the video plane 2002.
The value area is initialized (step S2100). Specifically, 0 is set in the area for the combined α value of all the pixels in each plane of the video plane 2002. This combined α value area finally holds the combined α value.

Next, the seventh combining section 2005 repeats the processing from steps S2101 to S2107. First, the seventh combining unit 2005 reads out image information i from the index file of the image holding unit 104 (Step S2101). Here, i is a variable indicating a number assigned to each component in ascending order such as 0, 1, 2, ... in order from the lower layer to the component of the upper layer for convenience in this flowchart.
Are referred to as image information i and image file i, respectively. In this flowchart, the initial value of i is 0, and the increment is 1.

Next, the seventh synthesizing unit 2005 stores the image file stored in the storage position indicated by the image information i in the image holding unit 104
Is taken out (step S2102). The seventh synthesizing unit 2005 reads the display size and the display position of the image file i, and sets an overlapping range of the component i of the image file on the screen (step S2103).

The seventh synthesizing unit 2005 determines whether the image type indicated by the image information i is a moving image or a still image (step S2104). If the result of the determination is a moving image,
The seventh synthesizing unit 2005 copies the α value of the image file i into the area of the α value of the k-th plane of the video plane 2002 (Step S2105). When this is made into an equation, it becomes (Formula 16).

[0111]

(Equation 16)

In equation (16), β _k (x, y) indicates a value stored in the area of the composite α value of the k-th plane of the video plane 2002, and α _i (x, y) indicates α of the image file i. Value. This k
Takes a value from 1 to N, and is incremented by one each time the process of step S2105 is repeated. Then
7 The synthesizing unit 2005 calculates the synthesized α value area of the m-th plane other than the k-th plane.

[0113]

[Equation 17]

Is calculated to update the value of β _m (x, y).
Here, β _m (x, y) indicates a value stored in the composite α value area of the m-th plane, and m takes a value from 1 to N excluding k (step S2106). That is, steps S2105 and S210
The calculation of 6 is equivalent to the calculation of (Equation 7), and calculates the combination ratio of each moving image component to the final combined result image, that is, the combined α value.

As described above, the seventh synthesizing unit 2005 ends the seventh synthesizing process when all the image files have been processed (step S2107). As a result, in each plane of the video plane 2002, the composite α value for each moving image component is stored. The eighth synthesis unit 2006
When receiving an EPG display instruction from the control unit 2000, the control unit 2000 performs an eighth synthesis process of synthesizing each RGB component between each plane of the video plane 2002 and the OSD plane 107, and outputs the resultant synthesized image to the output unit 111.

If there are N moving image components, the eighth
The synthesis by the synthesis unit 2006 is represented by the following equation.

[0117]

(Equation 18)

Here, R (x, y), G (x, y) and B (x, y) are
R, G, and B components of each pixel output to the output unit 111.
, Β₁(x, y), R_v1(x, y), G_v1(x, y), B_v1(x, y) is the bidet
Moving image stored in the first plane of Oplane 2002
The combined alpha value and RGB component of the data, β_Two(x, y), R
_v2(x, y), G_v2(x, y), B_v2(x, y) is video plane 2002
Of moving image data stored in the second plane of
Value and RGB components, β_N(x, y), R_vN(x, y), G_vN(x,
y), B_vN(x, y) is the Nth plane of video plane 2002
With the combined alpha value and RGB component of the stored moving image data
Yes, R_o(x, y), G_o(x, y), B _o(x, y) on OSD plane 107
The stored RGB components. <Operation> The image synthesizing apparatus 300 configured as described above
The operation flow of the image synthesizing apparatus 100 shown in FIG.
is there.

However, they differ in the following points. That is, the EPG generation unit 2003 performs the process of step S1002, and the EPG generation unit 2003 notifies the video reproduction unit 2001 of the display positions and display sizes of a plurality of moving image components instead of one moving image component. The sixth combining unit 2004 performs the process of step S1003.

Further, the processing in step S1004 is performed by the seventh synthesizing unit 2005.
The seventh combining unit 2005 calculates a combined α value corresponding to each of the plurality of moving image components and stores the combined α value in the video plane 2002. Also, the video playback unit 2001 performs the process of step S1005, and the video playback unit 2001 plays back a plurality of moving images and stores them in the video plane 2002.

Further, the processing in step S1006 is performed by the eighth synthesizing unit 20.
06, the eighth synthesis unit 2006 adds the RGB components of the sixth synthesized image data and the values obtained by weighting the RGB components of the plurality of moving image components with the respective synthesized α values, and obtains a synthesized image obtained after the addition. Are output to the output unit 111 (step S1006). In this way, the eighth synthesizing unit 2006 checks each plane of the video plane 2002 and the OS until the stop instruction is given.
Continue to synthesize and output with the D plane 107,
The processing is performed in parallel with the processing of 2001, the sixth synthesis unit 2004, and the seventh synthesis unit 2005. Fourth Embodiment <Structure> FIG. 21 is a block diagram showing a structure of an image synthesizing apparatus according to a fourth embodiment of the present invention.

In the same figure, the image synthesizing device 400 includes an input unit 1
01, control unit 4102, EPG generation unit 4103, image holding unit 4104, video reproduction unit 105, video plane 106, OSD plane 107,
It is composed of a ninth combining unit 4108, a tenth combining unit 4109, a third combining unit 110, and an output unit 111, and performs image combining using Porter-Duff operation. For Porter Duff operation, for example, T.Po
rter and T. Duff, Compositing Digital Im
ages "(SIGGRAPH 84, 253-259). In this embodiment, as shown in FIG.
This will be explained using (b).

Here, the porter-duff operation will be briefly described. The Porter-Duff operation is a 12-bit operation for combining the source and destination pixels.
It defines the types of operation rules. The source pixel and the destination pixel have an RGB component and an α value. However, the definition of the α value is different from the α values of the first to third embodiments. The α value used in the first to third embodiments is
In contrast to the definition between two images, the α value used in the porter-duff operation of the present embodiment is defined for each image. In the Porter-Duff operation, an image obtained as a result of combining two images having an α value also has an α value. When an image resulting from the synthesis is actually displayed on the screen, a value obtained by multiplying the RGB value of the image by the α value is output.

The image synthesizing apparatus 400 of this embodiment performs image synthesizing by using eight types of useful operations out of twelve types of operation rules. The eight calculation rules are "CLEAR", "SRC",
"SRC # OVER", "DST # OVER", "SRC # IN", "DST # I"
N "," SRC # OUT "and" DST # OUT ". "CLEAR"
Clears both the RGB components and the α value of the destination. Neither the source nor the destination is used as input. In the present embodiment, the destination corresponds to the image held in the OSD plane 107, and the source corresponds to the image of the image file i.

“SRC” copies a source to a destination. "SRC # OVER" overlays the source on the destination. “DST_OVER” overlays the destination on the source and replaces the destination with the resulting RGB components.
"SRC # IN" replaces the destination by the source part overlapping the destination.

"DST # IN" replaces a destination with a destination portion overlapping with the source. “SRC # OUT” replaces the destination with a source part that does not overlap with the destination.
“DST # OUT” replaces the destination with a destination portion that does not overlap with the source. Hereinafter, components having reference numerals different from those of the other embodiments will be described.

The control unit 4102 controls all the components of the image synthesizing apparatus 400. More specifically, the image synthesizing device 400
Is turned on, the video playback unit 105 plays back the broadcast program. When the channel switching is instructed, the video reproducing unit 105 reproduces a broadcast program different from the one currently being reproduced. When instructed to turn on the EPG display, the EPG generation unit 410
3, video playback unit 105, ninth combining unit 4108, tenth combining unit 4109
And the third combining unit 110 and the like are controlled to generate a combined image.

The EPG generation unit 4103 acquires and holds EPG information broadcasted from a broadcast station, generates a plurality of image files and one index file based on the EPG information, as in the first embodiment. The image is stored in the image holding unit 4104. further
The EPG generation unit 4103 extracts the display size and the display position of the component of the moving image constituting the EPG display screen from the EPG information, and outputs it to the video reproduction unit 105.

The image holding unit 4104 holds a plurality of image files and one index file. The index file is for managing a plurality of image files and includes a plurality of pieces of image information. One piece of image information corresponds to one image file. The plurality of pieces of image information are arranged in the index file in the same order as the superposition order of the components. Image information is
It is composed of an image type, a calculation type, and a storage position. The image type indicates whether the image of the image file is a still image or a moving image. The calculation type indicates which of the twelve types of calculation specified by Porta Duff is used. The storage position indicates the head position of the image file in the image holding unit 4104.

FIG. 22A shows an example of an index file. An index file 4410 shown in FIG. 11 is composed of image information 4421, 4422, and 4423, and indicates the order of superposition of components in the same order. That is, the image information 4421 corresponds to the component of the lowest layer, the image information 4422 corresponds to the component of the next layer, and the image information 4423 corresponds to the component of the highest layer. The value in column 411 is
Indicates the image type of the component, "0" is a still image,
“1” indicates a moving image. The value in column 413 indicates the type of operation. Column
FIG. 13B shows the relationship between the values that can enter 413 and the operation type. For example, “3” stored in row 4422, column 413
Computes the operation "SRC OVER" in row 4423, column 413,
“2” stored in row 4421 and column 413 is the operation “S
RC ". The value in column 412 indicates the storage location of the image file.

The images 430, 440, and 450 correspond to the components 202, 203, and 204, respectively, and represent the image data in each image file. Here, the image 440 indicates that all the RGB components of the image data corresponding to the component 203 are 0. In the Porter-Duff operation, since the α value is set for all the images, the image 430 is the lowest component, but an arbitrary α value is set for each pixel.

The image holding unit 4104 holds a plurality of image files and one index file. 9th synthesis section 4
108 receives the instruction from the control unit 4102, the image holding unit 410
The ninth combined image data is generated by combining the image data of the plurality of image files held in 4 and stored in the OSD plane 107. This process is called a ninth combining process.

FIG. 23 is a flowchart showing the procedure of combining by the ninth combining section 4108. In the figure, the same processing is performed in steps having the same step numbers as those in FIG. Hereinafter, the processing of the step numbers different from those in FIG. 8 will be described. Step S4802: The ninth combining section 4108 reads the image file i stored in the storage position indicated by the image information i and the operation type indicated by the image information i.

Step S4805: The Porter Duff alpha synthesis operation shown in FIG. 24 is performed for each pixel in accordance with the operation type read in step S4802. Step S4806: With the RGB components of the moving image set to 0, the calculation shown in FIG. 25 is performed for each pixel according to the type of calculation. Step S4807: FIG. 2 in pixel units according to the operation type
The α value shown in 6 is calculated.

FIGS. 24, 25 and 26 show equations for only eight operations which are said to be significant in the Porter-Duff operation. Upon receiving an instruction from the control unit 4102, the tenth combining unit 4109 generates an α value of the video plane 106 for the OSD plane 107 from the image data stored in the image holding unit 4104, and stores the α value in the video plane 106.

FIG. 27 is a flowchart showing the procedure of the combining by the tenth combining section 4109. In the figure, the same processing is performed in steps having the same step numbers as those in FIG. Therefore, the process of step numbers different from those in FIG. 9 will be described below. Step S1202: The tenth combining section 4109 reads out the image file i stored in the storage location indicated by the image information i and the operation type indicated by the image information i.

Step S4905: The tenth combining section 4109 performs the calculation shown in FIG. 28 for each pixel in accordance with the type of calculation.
Basically, since a still image is drawn on a moving image, the components of the moving image are weakened. In the table of FIG. 28, α is a composite α value of each pixel of the video plane 106, and α _i is
The α value of each pixel superimposed on the video plane 106 of the extracted image file, and these α values theoretically take a value between 0 and 1. As a realization method,
The α value is represented by a value such as 0 to 255 or 0 to 15.

Step S4906: The tenth combining section 4109 performs the calculation shown in FIG. 29 for each pixel according to the type of calculation.
The ninth combining unit 4108 and the tenth combining unit 4109 operate in synchronization. Steps S1202, S903, S904, and S907 may be synchronized using the processing results of steps S4802, S803, S804, and S807 as they are. Steps S4905 and S4906 must be completed before step S807.

The image synthesizing apparatus 400 having the above configuration
Can perform image synthesis by Porter-Duff operation, and can reduce the memory and the number of times of multiplication as in the first embodiment. FIG. 30 is a program in which the calculation of the present embodiment is described in a C language style. This program focuses on only one pixel. R, G,
B and α are R, G, and R of one pixel on the OSD plane.
B, α value, and α _v represents the composite α value of the video plane corresponding to this pixel.

The first to fifth lines are initialization processing. The eighth, eleventh, and fourteenth rows are calculations performed by the ninth combining unit 4108. The ninth and twelfth lines are calculations performed by the tenth combining unit 4109, and the contribution (β value) of the moving image, that is, the combined α value, is calculated. The 16th, 17th, and 18th rows are the third combining unit 110
Is the operation that is responsible for. Here, the 16th, 17th, and 18th lines may be performed in parallel by another thread, and may be in a permanent loop. Here, the specific calculation on the eighth line is shown in FIG.
The specific calculation in the row is shown in FIG. The specific calculation on the ninth row is shown in FIG. 29, and the specific calculation on the twelfth row is shown in FIG. Here, α in FIG. 28 corresponds to α _v in FIG. The specific calculation on the 14th line is shown in FIG.

As described above, the image synthesizing device 4 of the fourth embodiment
00 performs synthesis of a plurality of still images and calculation of a synthesis ratio of a moving image according to the operation type of the porter duff operation determined for each image, and then synthesizes with each frame of the moving image according to the operation type. Combined with the still image. This eliminates the need to combine a plurality of still images and moving image frames each time a moving image frame is updated, reducing the calculation load and increasing the processing speed. As a result of the higher processing speed, images can be combined and displayed in real time in accordance with the playback rate of the moving image. Further, by performing the synthesis in real time, each time an image of one frame is expanded in the frame buffer, the image of the frame and the synthesized still image are synthesized. There is an effect that the capacity can be reduced.

Although the image synthesizing apparatus of the present invention has been described based on the embodiments, the present invention is not limited to the above embodiments. That is, (1) In the first and second embodiments, the composite α value of the moving image component is held in the video planes 106 and 1501, but when there is one moving image component, the OSD plane 1
07, 1501. (2) In the first to fourth embodiments, pixel data is stored in a memory or the like in the order of R, G, B, and α.
Not limited to this order, for example, the order of α, R, G, B or α, B, G, R
May be stored in this order. Also, not pixel by pixel,
A value for all pixels may be stored for each of the R, G, B, and α components. (3) The data length of R, G, B, α is 1 byte each, that is, R,
G and B are 256 gradations from 0 to 255, and a total of about 16
700,000 colors can be expressed. However, the data length of R, G, B, α may be expressed by 4 bits, so-called high color. (4) In the first to fourth embodiments, a YUV component may be used instead of the RGB component. (5) The image holding units 104 and 4104 hold pixel data as image data of a still image component.
Vector data, that is, data composed of mathematical elements and graphic elements such as straight lines, points, and circles,
May be configured to be developed into pixel data when combining image data. With this configuration, the memory capacity of the image holding units 104 and 4104 can be further reduced. (6) The operation procedure of each component of the image synthesizing apparatuses 100, 200, 300, and 400 may be a computer program for causing a general-purpose computer or a device having a program execution function to execute.

In particular, the computer program corresponding to the image synthesizing apparatus 400 of the fourth embodiment eliminates the need for eight types of arithmetic units corresponding to the eight types of operations of Porter Duff, and all eight types are provided in one processor that executes the computer program. Has the effect that the calculation of (1) can be performed.
The computer program may be recorded on a recording medium or distributed and distributed via various communication paths. Such recording media include IC cards, optical disks,
There are a flexible disk, a ROM, and the like. (7) The third combining unit 110 is configured to perform the above-described addition processing for each frame of video playback, that is, to perform the addition processing in synchronization with the playback of the frame. However, the addition processing may be performed asynchronously. Fifth synthesis unit 1503 and eighth synthesis unit
The same is true for 2006.

[0144]

The image synthesizing apparatus according to the present invention is an image synthesizing apparatus for generating a synthesized image by synthesizing a moving image and a plurality of still images, wherein the synthesizing information includes an image synthesizing order. First acquisition means for acquiring the combination information and the plurality of still images for obtaining the combination ratio of each image before combination with the image, and combining the plurality of still images based on the combination information to form one image A first combining unit that generates a combined still image, a calculating unit that calculates a combining ratio of the moving image to the combined image based on the combining information, and a second obtaining unit that acquires each frame that forms the moving image. And a second synthesizing unit for synthesizing each of the frames and the synthesized still image by using the synthesis ratio of the moving image.

According to this configuration, the image synthesizing apparatus according to the present invention first synthesizes a plurality of still images and calculates the synthesizing ratio of the moving image, and then synthesizes the moving image with each frame. In addition, unlike the related art, it is not necessary to combine a plurality of still images and moving images for each frame every time, so that the calculation load is light and the processing speed is high. As a result of the higher processing speed, images can be combined and displayed in real time in accordance with the playback rate of the moving image.

The composition information further includes, for each image, a coefficient corresponding to the image and operation information indicating a composition operation using the coefficient. According to this configuration, the image synthesizing apparatus can synthesize a plurality of still images and moving images in which coefficients and calculation information are defined, in addition to the above-described effects. The image synthesizing device has a first frame buffer for storing an image, and a second frame buffer for storing each frame constituting the moving image, wherein the first synthesizing unit includes the first frame buffer. The plurality of still images obtained by the obtaining unit are read out according to the image synthesis order, and the plurality of still images are read out using the coefficient and the calculation information.
The storage contents of the one frame buffer and the read images are combined, and the storage result of the first frame buffer is replaced with the result of the combination.The second acquisition unit replaces each of the acquired frames with the second Stored in a 2 frame buffer, the second combining means combines the respective frames stored in the second frame buffer and the storage contents of the first frame buffer using a combination ratio of the moving image, and
Store in 2 frame buffer.

According to this configuration, the image synthesizing device can
A plurality of still images and moving images can be synthesized using only two frame buffers, the frame buffer and the second frame buffer. The first synthesizing means may include a coefficient and an operation corresponding to the moving image after synthesizing the still image immediately before the moving image in the image synthesizing order and before synthesizing the still image immediately after the moving image. A combining operation is performed on the image stored in the first frame buffer using the information, and the contents of the first frame buffer are replaced with the result of the combining operation.

According to this configuration, it is possible to combine a plurality of still images and moving images with moving images between them at an accurate combining ratio. The image synthesizing device has a screen for image display, the synthesis by the first synthesizing unit, the acquisition by the second obtaining unit, and the synthesizing by the second synthesizing unit are performed in parallel, and the second frame The buffer outputs the result to the screen at the same time as storing the result of the synthesis by the second synthesis means.

According to this configuration, a state in the middle of combining is displayed on the screen, so that a state in which nothing is displayed until the combining is completed can be eliminated. The synthesis information includes:
Further, each of the plurality of images includes a synthesis coefficient indicating a synthesis ratio with respect to a synthesis result of the image and an image other than the image.

According to this configuration, the image synthesizing apparatus can synthesize a plurality of still images and moving images in which the synthesizing ratio of each of the plurality of images and another image and the superposition order are specified. The image synthesis order represents an order in which images are superimposed, and the synthesis coefficient is an alpha that indicates, for each of the plurality of images, a synthesis ratio of the image to a synthesis result from the rearmost image to the image in the image synthesis order. A calculating unit that calculates a combination ratio of the moving image to the synthesized image from the moving image and an alpha value corresponding to all the images positioned in front of the moving image.

According to this configuration, the image synthesizing apparatus can synthesize a still image and a moving image in which the alpha value and the overlapping order are specified. An image in which the alpha value is specified has an advantage that the layout can be flexibly changed as compared with an image in which the composition ratio for the entire image is specified in advance. Therefore, an image synthesizing apparatus that performs image synthesizing using an alpha value has an effect that it can support EPG display of various layouts.

The image synthesizing apparatus further includes a swapping means for changing the order of two images adjacent to each other in the superimposing order, and a synthesizing result when the synthesizing is performed before and after the swapping. Updating means for obtaining and updating the alpha value corresponding to the two images after the replacement, wherein the first combining means, the calculating means, and the second combining means are provided after the replacement by the replacing means. , And the respective alpha values after the updating by the updating means.

According to this configuration, even if the order of a plurality of images is changed, it is possible to perform image composition with an accurate composition ratio.
Also, by using this configuration, if the moving image located in the intermediate layer is replaced so as to be located in the upper layer, a plurality of still images are synthesized in order from the lowest layer, and finally the moving image is synthesized. Can be combined, the amount of calculation is small, and the load is light.

The image synthesizing apparatus has a storage unit for storing the plurality of still images acquired by the first acquiring means, and each of the plurality of still images has the same or less pixels as the synthesized image. Number of image data and layout information indicating the layout of the pixel data on the composite image, wherein the first synthesizing unit, the calculating unit, and the second synthesizing unit are each of the images determined by the layout information. Each processing is performed for the overlapping portion between the two.

The image synthesizing apparatus has a storage unit for storing the plurality of still images acquired by the first acquiring means, wherein each of the plurality of still images is represented in a vector data format. (1) The synthesizing unit converts the vector data into the pixel data and then synthesizes the pixel data. According to this configuration,
Since the still image data is vector data having a smaller data amount than the pixel data, the memory capacity is further reduced.

An image synthesizing apparatus according to the present invention is an image synthesizing apparatus for synthesizing a plurality of moving images and a plurality of still images to generate a synthesized image, wherein the synthesizing information includes an image synthesizing order. First acquisition means for acquiring the combination information and the plurality of still images for obtaining the combination ratio of each image before combination with the image, and combining the plurality of still images based on the combination information to form one image First combining means for generating a combined still image, calculating means for calculating a combining ratio of each of the plurality of moving images with respect to the combined image based on the combining information, and obtaining each frame constituting each of the plurality of moving images And a second synthesizing unit for synthesizing each frame of each of the plurality of moving images and the synthesized still image using a synthesizing ratio of each of the plurality of moving images.

According to this configuration, the amount of calculation of combining a plurality of moving images and a plurality of still images can be reduced. Further, the image synthesizing device of the present invention is an image synthesizing device that synthesizes a moving image and a plurality of still images to generate a synthesized image, wherein a first obtaining unit that obtains the plurality of still images, First synthesizing means for synthesizing the still images of the first and second frames to generate one synthesized still image, second obtaining means for obtaining each frame forming the moving image, and each frame forming the moving image and the synthesized still image. Second combining means for combining the image with the image.

According to this configuration, it is not necessary to prepare still image memories for the number of still images corresponding to the same number of pixels as the display screen, so that the memory capacity can be reduced. Further, according to this configuration, since a plurality of still images are combined first and a moving image is combined later, it is not necessary to combine a plurality of still images and a moving image every frame unit of the moving image, and the amount of calculation is reduced. Less.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image composition device according to a first embodiment of the present invention.

FIG. 2A shows a composite image 201. (b) is a composite image 20
Shows the components that make up 1.

FIG. 3 is a diagram for explaining the structure of an image file.

FIG. 4 shows an example of an index file.

FIG. 5 shows a structure of moving image data.

FIG. 6 illustrates pixel data of moving image data 501 in an image.

FIG. 7 shows a data structure of first combined image data.

FIG. 8 is a flowchart illustrating a processing procedure of a first combining process.

FIG. 9 is a flowchart illustrating a processing procedure of a second synthesis.

FIG. 10 is a diagram illustrating a flow of an operation of the image synthesizing apparatus 100 according to the present embodiment when displaying an EPG.

FIG. 11 illustrates a component group including N + 1 moving image components or still image components.

FIG. 12 is a program describing the operation of the present invention in a C language style.

FIG. 13 shows a program obtained by modifying FIG. 12.

FIG. 14 is a block diagram illustrating a configuration of an image composition device according to Embodiment 2 of the present invention.

FIG. 15 is a flowchart illustrating a processing procedure of a fourth combining process.

FIG. 16 is a diagram illustrating a flow of an operation of the image combining device 200 according to the present embodiment when displaying an EPG.

FIG. 17 is a diagram for explaining replacement of the overlapping order of two adjacent components.

FIG. 18 illustrates a situation in which components of a moving image are replaced with components of a still image so as to be at the highest layer.

FIG. 19 is a block diagram illustrating a configuration of an image composition device according to Embodiment 3 of the present invention.

FIG. 20 is a flowchart illustrating a procedure of a seventh combining process.

FIG. 21 is a block diagram illustrating a configuration of an image composition device according to Embodiment 4 of the present invention.

FIG. 22 (a) shows an example of an index file (b)
Indicates the correspondence between the operation type and the number.

[Fig. 23] Fig. 23 is a flowchart illustrating a procedure of the combining by a ninth combining unit 4108.

FIG. 24 illustrates a Porter Duff α synthesis operation according to the operation type.

FIG. 25 illustrates a Porter Duff α combination operation according to the operation type.

FIG. 26 illustrates a Porter Duff α combination operation according to the operation type.

[Fig. 27] Fig. 27 is a flowchart illustrating a procedure of combining by a tenth combining unit 4109.

FIG. 28 illustrates a Porter Duff α combination operation according to the operation type.

FIG. 29 illustrates a Porter Duff α combination operation according to the operation type.

FIG. 30 is a program describing the computation of the present embodiment in a C language style.

FIG. 31A shows an OSD plane 2502 and a video plane 2501.
FIG. 5 is a conceptual diagram illustrating a state in which the images of FIG. (b) shows a state in which components of a still image and a moving image are superimposed.

[Explanation of symbols]

 100 Image synthesis device 101 Input unit 102 Control unit 103 EPG generation unit 104 Image holding unit 105 Video playback unit 106 Video plane 107 OSD plane 108 First synthesis unit 109 Second synthesis unit 110 Third synthesis unit 111 Output unit

Claims (21)

[Claims] 1. An image synthesizing apparatus for synthesizing a moving image and a plurality of still images to generate a synthetic image, the synthesizing information including an image synthesizing order, and First obtaining means for obtaining the combined information and the plurality of still images for obtaining a combined ratio; and a first obtaining unit for combining the plurality of still images based on the combined information to generate one combined still image Synthesizing means, calculating means for calculating a synthesizing ratio of the moving image to the synthetic image based on the synthesizing information, second obtaining means for obtaining each frame constituting the moving image, and synthesizing ratio of the moving image An image synthesizing apparatus, comprising: second synthesizing means for synthesizing each of the frames and the synthesized still image by using the second synthesizing unit. 2. The composite information further includes, for each image, a coefficient corresponding to the image and operation information indicating a synthesis operation using the coefficient.
2. The image synthesizing device according to 1. 3. The image synthesizing device, comprising: a first frame buffer for storing an image, and a second frame buffer for storing each frame constituting the moving image.
A frame buffer, wherein the first synthesizing unit reads the plurality of still images obtained by the first obtaining unit in the image synthesizing order, and uses the coefficient and the calculation information to form the first frame buffer. The storage contents of the first frame buffer are replaced with the storage contents of the first frame buffer with the result of the combination, and the second acquisition unit replaces each of the acquired frames with the second image.
Stored in a frame buffer, the second synthesizing unit synthesizes each of the frames stored in the second frame buffer and the storage content of the first frame buffer using a synthesizing ratio of the moving image, and 3. The image synthesizing device according to claim 2, wherein the image is stored in a frame buffer. 4. The first synthesizing unit, after synthesizing the still image immediately before the moving image in the image synthesizing order, and before synthesizing the still image immediately after the moving image, Using the corresponding coefficient and operation information, performing a synthesis operation on the image stored in the first frame buffer, replacing the contents of the first frame buffer with the result of the synthesis operation, wherein The image synthesizing device according to the above. 5. The image synthesizing apparatus has a screen for displaying an image, and the synthesis by the first synthesis unit, the acquisition by the second acquisition unit, and the synthesis by the second synthesis unit are performed in parallel. 4. The apparatus according to claim 3, wherein the second frame buffer outputs the result of synthesis to the screen at the same time as storing the synthesis result by the second synthesis unit. 6. The combination information according to claim 1, wherein the combination information further includes, for each of the plurality of images, a combination coefficient indicating a combination ratio of a combination result of the image and an image other than the image. Image synthesis device. 7. The image synthesizing device, comprising: a first frame buffer for storing an image, and a second frame buffer for storing each frame constituting the moving image.
A frame buffer, the first combining unit reads the plurality of still images acquired by the first acquiring unit in accordance with the image combining order, and reads the stored contents of the first frame buffer using the combination coefficient. Performing composition with each of the output images, replacing the storage content of the first frame buffer with the result of the composition, and the second acquisition unit replaces each of the acquired frames with the second frame.
Stored in a frame buffer, the second synthesizing unit synthesizes each of the frames stored in the second frame buffer and the storage content of the first frame buffer using a synthesizing ratio of the moving image, and 7. The image synthesizing device according to claim 6, wherein the image is stored in a frame buffer. 8. The first synthesizing unit, after synthesizing the still image immediately before the moving image in the image synthesizing order, and before synthesizing the still image immediately after the moving image, The method according to claim 7, wherein a combining operation is performed on an image stored in the first frame buffer using a corresponding combining coefficient, and the content of the first frame buffer is replaced with a result of the combining operation. Image synthesis device. 9. The image synthesizing device has a screen for displaying an image, and the synthesizing by the first synthesizing unit, the obtaining by the second obtaining unit, and the synthesizing by the second synthesizing unit are performed in parallel. 8. The image synthesizing apparatus according to claim 7, wherein the second frame buffer outputs to the screen at the same time as storing the synthesis result by the second synthesizing unit. 10. The image compositing order represents an order in which images are superimposed, and the compositing coefficient is, for each of the plurality of images, the image of the image relative to a composite result from the rearmost image to the image in the image compositing order. An alpha value representing a combination ratio, wherein the calculation unit calculates a combination ratio of the moving image with respect to the combined image from alpha values corresponding to the moving image and all images located in front of the moving image. 7. The image synthesizing device according to claim 6, wherein: 11. The image synthesizing device, comprising: a first frame buffer for storing an image, and a second frame buffer for storing each frame constituting the moving image.
And a frame buffer, wherein the first synthesizing unit sequentially converts one still image from a plurality of still images obtained by the first obtaining unit in a direction from the most background to the foreground indicated by the image synthesis order. Reading, synthesizing the stored contents of the first frame buffer and each read image using the alpha value, replacing the stored contents of the first frame buffer with the result of the synthesis, Replaces each of the acquired frames with the second
Stored in a frame buffer, the second synthesizing unit synthesizes each of the frames stored in the second frame buffer and the storage content of the first frame buffer using a synthesizing ratio of the moving image, and 11. The image synthesizing device according to claim 10, wherein the image synthesizing device is stored in a frame buffer. 12. The first synthesizing unit, after synthesizing the still image immediately before the moving image in the image synthesizing order, and before synthesizing the still image immediately after the moving image, The image synthesis method according to claim 11, wherein a synthesis operation is performed on an image stored in the first frame buffer using an alpha value, and the content of the first frame buffer is replaced with a result of the synthesis operation. apparatus. 13. The image synthesizing device has a screen for displaying an image, and the synthesizing by the first synthesizing unit, the obtaining by the second obtaining unit, and the synthesizing by the second synthesizing unit are performed in parallel. 12. The image synthesizing apparatus according to claim 11, wherein the second frame buffer outputs to the screen at the same time as storing the synthesis result by the second synthesizing unit. 14. The image synthesizing device, further comprising: a swap unit that swaps the order of two images adjacent to each other in the superimposition order; and a synthesis result when the synthesis is performed before and after the swap. Updating means for obtaining and updating an alpha value corresponding to the two images after the replacement so as to be the same as each other, wherein the first combining means, the calculating means, and the second combining means 11. The image synthesizing apparatus according to claim 10, wherein each processing is performed using the order after the replacement and the alpha value updated by the updating unit. 15. The replacement means, wherein an alpha value is α [i] and the order is from the lower order, i-th image i, and an alpha value is α [i + 1] and the order is from the lower order. The order of the i + 1-th image i + 1 is replaced, and the updating unit sets the alpha value for the i-th image i + 1 after the replacement to α [i] * (1-α [i + 1]). , After replacement i + 1
The alpha value for the i-th image i is α [i + 1] / (1-α
15. The image synthesizing device according to claim 14, wherein [i] * (1−α [i + 1])). 16. The image synthesizing device, further comprising: a storage unit for storing the plurality of still images acquired by the first acquiring unit, wherein each of the plurality of still images is the same as the synthesized image or The first combining means, the calculating means, and the second combining means are determined from the layout information, the image data having a small number of pixels, and layout information indicating a layout of the pixel data on the composite image. 2. The image synthesizing apparatus according to claim 1, wherein each processing is performed on a superimposed portion of each image. 17. The image synthesizing apparatus, further comprising: a storage unit for storing the plurality of still images acquired by the first acquisition unit, each of the plurality of still images is represented in a vector data format, 2. The image synthesizing apparatus according to claim 1, wherein the first synthesizing unit performs the synthesizing after converting the vector data into pixel data. 18. An image synthesizing apparatus for synthesizing a plurality of moving images and a plurality of still images to generate a synthesized image, the synthesizing information including an image synthesizing order, and A first acquisition unit configured to acquire the combined information and the plurality of still images for obtaining a combined ratio of images, and to combine the plurality of still images based on the combined information to generate one combined still image A first combining unit, a calculating unit that calculates a combining ratio of each of the plurality of moving images with respect to the combined image based on the combining information, and a second acquiring unit that acquires each frame configuring each of the plurality of moving images. An image synthesizing apparatus, comprising: a second synthesizing unit that synthesizes each frame of each of the plurality of moving images and the synthesized still image by using a synthesis ratio of each of the plurality of moving images. 19. An image synthesizing apparatus for synthesizing a moving image and a plurality of still images to generate a synthesized image, a first obtaining unit obtaining the plurality of still images, and synthesizing the plurality of still images. A first synthesizing unit that generates one synthesized still image, a second obtaining unit that obtains each frame that forms the moving image, and synthesizes each frame that forms the moving image with the synthesized still image An image synthesizing apparatus, comprising: a second synthesizing unit. 20. A computer-readable recording medium recording a program for causing a computer to perform a process of generating a composite image by combining a moving image and a plurality of still images, the program including: A first obtaining step of obtaining synthesis information and the plurality of still images for obtaining a synthesis ratio of each image before synthesis with respect to the synthesis image, the synthesis information including an image synthesis order; and A first combining step of combining the plurality of still images on the basis of the plurality of still images to generate one combined still image; a calculating step of calculating a combining ratio of the moving image to the combined image based on the combining information; A second obtaining step of obtaining each frame constituting the image, and synthesizing the respective frames and the synthesized still image using a synthesis ratio of the moving image. Recording medium, characterized in that to execute a second synthesis step. 21. A program for causing a computer to execute a process of generating a combined image by combining a moving image and a plurality of still images, the combining information including an image combining order, and A first obtaining step of obtaining the synthesis information for obtaining a synthesis ratio of each image before synthesis and the plurality of still images; and synthesizing the plurality of still images based on the synthesis information to form one synthesized still image A first combining step of generating an image; a calculating step of calculating a combining ratio of the moving image to the combined image based on the combining information; a second acquiring step of acquiring each frame constituting the moving image; A second synthesizing step of synthesizing each of the frames and the synthetic still image using a synthesizing ratio of a moving image.