JPH11296692A - Image processing method and its device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent overall blurring of a synthesized image by comparing the object sizes of original images with each other based on correspondence data, discriminating an image that is projected with the highest definition among those objects, and generating a synthesized image only from discriminated images. SOLUTION: A synthesized data generation processing including a largest image discrimination process 1900, a color mixture rate decision processing 1310 and a redundant data discrimination processing 1360 is performed as a pre-process 1300 before a synthetic image generation process 1700. That is, the process 1900 discriminates an object that is projected in the largest size among those objects which are projected on the original images 1100. The process 1310 reduces discontinuous color changes caused among images which are synthesized from different images 1100 and suppresses the conspicuous joints of images. Then the process 1360 deletes the images 1100 and the initial deformation data 1200 and defines the image 1100 that is projected in the largest size as a synthetic image 1800 based on the deletion output results of those images 1100 and data 1200.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image processing for generating a new image by performing image processing such as image deformation and image mixing on the image.

[0002]

2. Description of the Related Art Conventionally, there has been known an image processing method for generating a composite video viewed from another viewpoint based on a video shot or drawn from one viewpoint. This image processing method is effective for producing a video effect of changing a viewpoint for a real-life video or a painting in which it is difficult to create a three-dimensional model.

[0003] One of the techniques for generating a composite image viewed from another viewpoint is a warping technique ("interpolation between images based on projective transformation" IEICE IEICE, IEICE, IE94-14 (1994-0
5) pp.29-36) is known. The warping technique is an image processing for deforming an original image, which is a digital image, based on deformation data composed of movement data for each pixel or a plurality of pixels of the original image to generate a new composite image. By using a video from a certain viewpoint as an original image and deforming the original image using deformation data, an approximate video viewed from another viewpoint can be synthesized. However, in the warping technique, when the image of the object in the composite image is larger than the image of the object in the original image, it is necessary to enlarge the image of the original image, and the image quality of the composite image is greatly deteriorated.

A morphing technique ("Feature-Based Image Met") is another technique for generating a composite image viewed from another viewpoint.
amorphosis, "Computer Graphics Proc. Vol. 29, pp. 35
-42, (1992)) is also known. The morphing technique transforms a plurality of original images using corresponding data indicating which part of each original image shows the same object among the plurality of original images, and mixes the deformed images with each other. , A technique for generating a composite image having intermediate features of the plurality of original images. In morphing technology, the image of the object in the composite image is generated by mixing the images of multiple original images, so if the image of the object is large in any of the original images, the case of using the warping technology As a result, the deterioration of the image quality can be suppressed.
However, in the morphing technique, as a result of mixing a plurality of images, the composite image often becomes a blurred image as a whole.

[0005]

In a method of generating a composite image from only one original image, such as a warping technique, even if an image of an object is shown in detail in another original image, the image is reproduced. Not available effectively.

On the other hand, in the morphing technique, since a video of another original image is mixed in addition to a video of an original image which is the largest image of a certain object, the composite image is totally blurred.

[0007] A first problem of the present invention is that, when a certain object appears in a plurality of original images, in each of the original images,
The purpose is to generate a video of the object in the composite image from only the original image determined to be the largest. A second problem of the present invention is that, in a composite image created from different original images for each object, when the color changes discontinuously at a connection portion between videos generated from different original images, the color change is reduced. By making it continuous, the connection part is made inconspicuous.

A third object of the present invention is to generate a composite image from an original image which is determined to be the largest in the original image and a corresponding data portion which are unnecessary by generating a composite image. Sometimes, the amount of calculation for generating a composite image is reduced by not performing the image deformation process and the image mixing process. A fourth object of the present invention is to delete a part of the original image that is no longer necessary from the original image data and a part of the corresponding data that is no longer necessary from the corresponding data, thereby generating a composite image. Is to reduce the amount of important data.

[0009]

Means for Solving the Problems In order to solve the first problem, the size of each object in each original image is compared by using corresponding data, and the size of each object is sharpest. Provided is an image processing method for determining an existing video and generating a composite image only from the determined video.

[0010] Here, the sharpest image includes the largest image and the first characteristic of the shape of the object. An object refers to an object projected in an image. For this reason, if it is left as an image in an image, a plant is also an object on humans and animals. Also, a part of them can be an object (of course, a part other than a human can also be an object). In order to solve the second problem, at the connection portion of images generated from different original images, the color mixing ratio of one original image is gradually reduced and the color of the other original image is mixed at the same time. An image processing method for gradually increasing the ratio is provided.

[0011] In order to solve the third problem, as pre-processing of a composite image generation process, discrimination data for discriminating an unnecessary part of an original image and an unnecessary part of corresponding data is created, and An image processing method is provided that skips image deformation processing and image mixing processing for unnecessary original image portions and unnecessary corresponding data portions using discrimination data at the time of generation. In order to solve the fourth problem, as a pre-process of the composite image generation processing, an image processing method for deleting unnecessary original image portions from the original image and an unnecessary corresponding data portion are deleted from the corresponding data. An image processing method is provided.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a first embodiment of the present invention will be described in detail. FIG. 1 is a diagram showing a processing procedure of the present invention and its input / output data. FIG. 2 is a diagram illustrating a system configuration according to the first embodiment. FIGS. 3 and 4 are flowcharts showing the maximum image discrimination processing 1900. FIG. 5 is a diagram showing a flowchart of the color mixing ratio determination processing 1310. FIG. 6 shows a redundant data determination process 13.
FIG. 60 is a diagram showing a flowchart of 60. FIG. 7 is a diagram showing a flowchart of the composite image generation processing 1700. FIG. 9 is a diagram illustrating the color mixing ratio determination processing 1310 using a specific example.

The system configuration of the first embodiment is, as shown in FIG. 2, a display device 2100, an input device 2200, an external storage device 2300, a processing device 2400, and a storage device 250.
0 is connected to a bus 2600 which is a data transfer path. The display device 2100 is, for example, a CRT display or a liquid crystal display device, and displays the composite image 1800 and instruction guidance to the user.

The input device 2200 is, for example, a keyboard,
It is a mouse and inputs instructions from the user. The external storage device 2300 stores the original image 1100, the corresponding data 120
0, the original image '1400, the corresponding data' 1500, and the combined data 1600. Original image 1100, corresponding data 1200, original image '1400, corresponding data' 1500,
The details of the composite data 1600 will be described later. The processing device 2400 executes the pre-processing 1300 and the composite image generation processing 1700. Details of the preprocessing 1300 and the composite image generation processing 1700 will be described later. Storage device 25
00 is, for example, a memory, which is an original image 1100, corresponding data 1200, an original image '1400,
The composite data 1600, the composite image 1800, the count data 2700, and the weight data 2800 are temporarily stored. Details of the composite image 1800, the count data 2700, and the weight data 2800 will be described later.

Hereinafter, the processing procedure of this embodiment will be described with reference to FIG.
It is described using. The input data of the present invention includes a plurality of original images.
[m] 1110 (1 ≦ m ≦ M) original image 1100 and a plurality of corresponding data [m, n] 1210 (n ≠ m, 1 ≦ n ≦ M)
The corresponding data 1200 is composed of Characters in [] in the description shall represent subscripts. Here, the constant M indicates that the number of original images 1100 is M. Hereinafter, when the original image, the corresponding data, the original image ', the corresponding data', and the composite data are represented by subscripts, they indicate a specific one of the plurality of data. Shall be inserted. Parameters in uppercase letters represent constants. Also, the parameters expressed in lowercase letters represent variables, and are appropriately changed in the definition area. However, the value of a variable shall not change between the range in which the value of a variable is explicitly declared to be immutable in the specification and the value in a sentence in the specification. Original image [m] 111
0 is a digital image. The digital image is obtained by dividing the image into small discrete points called pixels, and expressing the density value of each pixel by a discrete value, for example, a bit map.

Each pixel of the digital image is denoted by a pixel [s, t] (1 ≦ s ≦ S, 1 ≦ t ≦ T) with a subscript. Here, the constant S and the constant T represent that the digital image has a size of S pixels on the horizontal axis and T pixels on the vertical axis. Pixel [s, t] represents the s-th pixel from the left and the t-th pixel from the top of the digital image. Hereinafter, when the pixel is described as [m] [s, t], the original image [m] 111
It represents a pixel [s, t] of 0. Corresponding data [m, n] 12
Reference numeral 10 denotes data for deforming the original image [m] 1100, which is an original image [n] 1 corresponding to each pixel of the original image [m] 1110.
This is data indicating 110 pixels. However, when there is no corresponding pixel, information that “there is no corresponding pixel” is registered. The pixel corresponding to the pixel [s, t]
[u, v] (1 ≦ u ≦ S, 1 ≦ v ≦ T) means that an original image [n] 1110 including a pixel [u, v] is an original image [m] including a pixel [s, t]. 11
Unlike FIG. 10, the object represented by the pixel [u, v] is the pixel [s,
t] is a pixel similar to the object represented by t].

Hereinafter, when the corresponding data is expressed as [m, n] [s, t], it represents data indicating the pixel of the original image [n] 1110 corresponding to the pixel [m] [s, t]. I do. The pre-processing 1300 includes:
Using the original image 1100 and the corresponding data 1200 as input data, the original image '1400, the corresponding data' which is intermediate data
1500, and generate combined data 1600. Preprocessing 13
00 includes a maximum image discrimination process 1900, a color mixing ratio decision process 1310, and a redundant data discrimination process 1360. The maximum image discrimination process 1900 is a process of discriminating which original image [m] 1110 of the object shown in the original image 1100 contains the largest image of the object. The details of the maximum image determination processing 1900 will be described later. The color mixture ratio determination processing 1310 is performed for the composite image 1
In 800, the adjacent video is a different original image [m] 11
10. When synthesized from the original image [n] 1110, a video synthesized from the original image [m] 1110 and the original image [n] 1110
The original image [m] 11 near the boundary line of the image synthesized from the image so that the color does not change discontinuously at the boundary line.
This is a process of mixing the color of the video synthesized from 10 with the color of the video synthesized from the original image [n] 1110. The details of the color mixture ratio determination processing 1310 will be described later.

The redundant data discriminating process 1360 is unnecessary for the original image 1 which is unnecessary in the synthetic image generating process 1700 to be described later.
This is a process of determining the portion 100 and the portion of the corresponding data 1200 and deleting those unnecessary data. Details of the redundant data determination processing 1360 will be described later. The original image '1400 is composed of a plurality of original images' [m] 1410. The original image '[m] 1410 is derived from the original image [m] 1110.
This is the remaining data from which data not used in the composite video generation processing 1700 is removed.

The correspondence data '1500 is composed of a plurality of correspondence data' [m, n] 1510. Corresponding data '[m, n] 151
0 is the remaining data obtained by removing data not used in the composite video generation processing 1700 from the corresponding data [m, n] 1200. The synthesized data 1600 is a plurality of synthesized data [m]
1610. The combined data [m] 1610 determines which part of the original image [m] 1110 and the corresponding data [mn] 1210 to use in the combined image generation processing 1700 (in other words, which part of the original image [m] 1110 is the original image [m] 1110). Image '[m] 1
Information indicating which part of the corresponding data [m] 1200 remains in the corresponding data '[m] 1500), and the color of the original image' [m] 1410 to the original image '[m] 141
This is data having information indicating how much the color of the original image '1400 other than 0 is mixed with the color. Hereinafter, when the combined data is expressed as [m] [s, t], the original image '[m] 1410
Represents the combined data of the pixel [s, t].

The composite image generation processing 1700 is performed in the original image '1
A composite image 1800 is generated using the input data 400, the corresponding data '1500, and the composite data 1600 as input data. The details of the composite image generation processing 1700 will be described later.

Hereinafter, the maximum image discriminating process 1900 will be described in detail with reference to FIGS. Processing step 19
At 05, a storage area for the count data 2700 is secured in the storage device 2500 and its value is cleared to zero. The count data 2700 is composed of a plurality of count data [m] assigned to each of the original images [m] 1110,
Further, the count data [m] is composed of count data [m] [s, t] assigned to all pixels [s, t] of the original image [m] 1110.

The count data [m] [s, t] is a pixel [m] [s, t]
Is the original image 1 other than the original image [m] 1110
Data used to count how many are in 100. In processing step 1910, the original image [m] 1110
From the external storage device 2300 to the storage device 1800. In each loop of the processing control performed by the processing step 1945, the value of the variable m does not change. In processing step 1915, the original image [n] 1110 is read from the external storage device 2300 to the storage device 2500. The value of the variable n does not change in each processing control loop performed by the processing step 1940. In processing step 1920, the corresponding data [m, n] 1210 is stored in the external storage device 230.
0 is read into the storage device 2500. Processing step 19
At 25, it is checked whether the pixel [n] [u, v] corresponding to the pixel [m] [s, t] exists using the corresponding data [m, n] [s, t]. In each processing control loop formed by the processing step 1935, the values of the variables s, t, u, and v do not change.

If the corresponding pixel [n] [u, v] exists, the control is transferred to the processing step 1930. Corresponding pixel
If [n] [u, v] does not exist, processing step 1935
Transfer processing control to. In processing step 1930, the value of count data [n] [u, v] is increased by one. Processing Step 1
At 935, all pixels [s, t] of the original image [m] 1110
From processing step 1925 to processing step 193
Repeat up to 5. In processing step 1940, processing steps 1915 to 1940 are repeated for all original images [n] 1110 other than original image [m] 1110. In processing step 1945, processing steps 1910 to 1945 are repeated for all original images [m] 1110. In processing step 1948, the storage area of combined data 1600 is stored in storage device 2500.
To secure. The synthesized data 1600 is the synthesized data [m] 1610 allocated to each original image [m] 1110.
, And the combined data [m] 1610 is composed data [m] [s, t] assigned to all pixels [m] [s, t].
It consists of.

The value of the synthesized data [m] [s, t] is Xmin (Xmin = 0
) To Xmax (Xmax> 0), and especially when the value of the composite data [m] [s, t] is Xmin, the pixel [m] [s, t] is subjected to the composite image generation processing 1700. Indicates that it is unnecessary. In processing step 1950, it is determined whether a pixel [n] [u, v] corresponding to the pixel [m] [s, t] exists or not by corresponding data [m, n] [s,
Check using [t]. The value of the variable m does not change in each loop of the processing control performed in the processing step 1980. In each loop of the process control performed by the process step 1975, the values of the variables s and t do not change. In each loop of the processing control performed by the processing step 1960, the values of the variables n, u, and v do not change. Processing Step 195
In step 5, it is checked whether the value of the count data [m] [s, t] is equal to or less than the value of the count data [n] [u, v].
If it is equal to or less than the value of [u, v], the process control is shifted to 1960. Otherwise, the process control is transferred to 1970. Processing control 1
At 960, processing steps 1950 to 1960 are repeated for all original images [n] 1110 other than original image [m] 1110. Processing Step 196
5, the value of the composite data of the pixel [m] [s, t] is Xmax (Xm
ax> 0) is set. In processing step 1970, Xmin (Xmin = 0) is set to the value of the composite data of the pixel [m] [s, t]. In processing step 1975, the original image [m]
Processing step 19 for all pixels [s, t] of 1110
Steps 50 to 1975 are repeated. In processing step 1980, processing steps 1950 to 1980 are repeated for all original images [m] 1110.

The processing procedure of the color mixture ratio determination processing 1310 will be described below in detail with reference to FIG. In processing step 1315, a storage area for storing the combined data [O] (O = M + 1) is secured in the storage device 2500. The composite data [O] is data that temporarily holds the composite data [m] 1610. In processing step 1317, a smoothing process (introduction to computer image processing, supervised by Hideyuki Tamura, edited by Japan Industrial Technology Center) is performed near the synthesized data [m] [s, t], and the result is converted to synthesized data [O] [s, t]. The smoothing process means, for example, the value of the synthesized data [m] [s, t] and the synthesized data adjacent to the synthesized data [m] [s, t] (the synthesized data [m] [s-1,
t-1] ... This is a process for calculating the average of the combined data [m] [s + 1, t + 1]). The value of the variable m does not change in each loop of the processing control performed in the processing step 1327.

The values of the variables s and t do not change in each loop of the processing control performed in the processing step 1320. In processing step 1320, the original image [m] 111
Processing steps 1317 to 1320 are repeated for all pixels [s, t] of 0. In processing step 1323, all the values of the composite data [O] are copied to the composite data [m]. In processing step 1325, processing steps 1317 to 1325 are repeated I times. The constant I is an arbitrary constant. When the value of the constant I is increased, a different original image 1
The range in which colors are mixed in the connection portion of the image generated from the image 100 is widened.

In processing step 1327, processing steps 1317 to 1327 are repeated for all original images [m] 1110. The synthesized data 1600 generated by the above processing steps is used to generate a synthesized image 180
The principle that the mixing ratio of colors changes continuously at the connection portion between images synthesized from 0 different original images 1100 will be described with reference to FIG.

Hereinafter, in order to simplify this example, the number of original images is two (M = 2) and the size of the original image is 6 × 1 (S
= 6, T = 1), the pixels at the same position correspond to each other in each original image, and three pixels in the left half of the original image [0] It is assumed that it is determined that the video is larger and the video of the original image [1] 1110 is larger for the three pixels in the right half. The value of Xmax is set to 9. Synthetic data [0] before performing color mixing ratio determination processing 1310
The value of 1610 is a value 9210, and the composite data [1] 16
The value of 10 is the value 9710. Here, it is assumed that the mixing ratio of the color of the pixel [0] [s, t] (referred to as a ratio [0] [s, t]) is obtained by the following equation, and the ratio [0] [s, t] = synthesized data [ 0] [s, t] ÷ (combined data [0] [s, t] + combined data [n] [s, t]) Pixel [1] [s, t] mixture ratio (ratio [1] [s, t]) is obtained by the following equation, the ratio [1] [s, t] = synthesized data [1] [s, t] ÷
(Synthesized data [0] [s, t] + Synthesized data [n] [s, t]) The ratio of mixing the color of each pixel of the original image [0] 1110 before the color mixing ratio determination processing 1310 is 9100. And the original image [1]
The ratio of mixing the colors of the pixels of 1110 is 9600. If the colors of the pixels [0] [3,1] and the pixels [1] [4,1] are different, the colors are discontinuous at the connection part. Change. Here, the display of the ratio in the figure is a percentage display.

Next, in the color mixture ratio determination processing 1310, when the smoothing processing of the processing step 1317 is repeated twice for all the values of the composite data [0] 1610 and the composite data [1] 1610 (I = 2) Is described). First, as a result of applying the smoothing processing once to all the values of the composite data [0] 1610, the value of the composite data [0] 1610 becomes 9
220 (the new value of each composite data [0] [s, t] is the average of the value of the composite data and the value of the left and right composite data). Similarly, the value of synthesized data [1] 1610 is 97
It becomes 20. Further, by repeating the smoothing process again, the value of the combined data [0] becomes 9230, and the value of the combined data [1] becomes 9730. As a result, color mixture ratio determination processing 13
The ratio of mixing the color of each pixel of the original image [0] 1110 after 10 is 9300, and the ratio of mixing the color of each pixel of the original image [1] 1110 is 9800. In the connection part, the ratio of mixing colors is from the original image [0] 1110 to the original image
[1] It gradually changes to 1110, and it can be seen that the discontinuity of the connection can be reduced.

Hereinafter, the processing procedure of the redundant data determination processing 1360 will be described in detail with reference to FIG. In processing step 1635, it is checked whether or not the value of the composite data [m] [s, t] is Xmin. In the case of Xmin, the processing control is performed in processing step 1
Transfer to 367. If it is not Xmin, the process control moves to a process step 1373. In each loop formed by the processing step 1375, the value of the variable m does not change. In each loop formed by the processing step 1373, the values of the variables s and t do not change. In processing step 1367, the pixel [m] [s, t] is deleted.

At processing step 1370, the corresponding data
Delete [m, n] [s, t]. In processing step 1371, all original images [n] 1110 other than original image [m] 1110
From processing step 1370 to processing step 137
Repeat up to 1. In processing step 1373, processing step 136 is performed on all pixels [s, t] of the original image [m].
5 to processing step 1373 are repeated. In processing step 1375, processing steps 1365 to 1375 are repeated for all original images [m] 1110. In processing step 1377, the original image 1
100 as an original image '1400 and an external storage device 2300
Write out. In processing step 1380, correspondence data 1200 is written to external storage device 2300 as deformed data '1500. In processing step 1383,
The combined data 1600 is written to the external storage device 2300.

The procedure of the composite image generation process will now be described with reference to FIG.
This will be described in detail. In processing step 1705, a viewpoint position parameter indicating the viewpoint position at which the composite image 1800 is generated is received from outside. Processing step 171
0, the storage area of the weight data is stored in the storage device 250.
Reserved to 0 and initialized. The weight data is the synthesized image 1800
And weight data [w, x] assigned to each pixel [w, x] (1 ≦ w ≦ S, 1 ≦ x ≦ T). The weight data [w, x] is
Original image 1100 already mixed with pixel [w, x] of the composite image
Is the data that holds the sum of the pixel weights. The values of the weight data [w, x] are all set to 0 as initial values. In processing step 1713, the original image '[m] 1410 is read from the external storage device 2300 to the storage device 2500.

In each loop formed by the processing step 1770, the value of the variable m does not change. In processing step 1715, the combined data [m] 1610 is read from the external storage device 2300 to the storage device 2500. In processing step 1718, the corresponding data '[m, n] 1510
From the external storage device 2300 to the storage device 2500. In processing step 1719, the original image '[m] 14
Processing steps 1718 to 1719 are repeated for all original images' [n] 1410 other than 10.

At processing step 1720, it is checked whether or not the value of the composite data [m] [s, t] is larger than Xmin. If it is larger than Xmin, the processing control is shifted to the processing step 1730. In the case of Xmin, the processing control is shifted to the processing step 1760.

In each loop formed by the processing step 1760, the values of the variables s and t do not change. In processing step 1730, pixel [s, t] of original image '[m] 1410
The position of the pixel [w, x] in the composite image 1800 is calculated. The method of obtaining the position of the pixel [w, x] is the same as that in the case of the normal morphing technique, and is determined by the viewpoint position parameter and the corresponding data '1500. Processing step 17
From 30 to processing step 1750, the values of the variables w and x do not change.

In processing step 1740, the color of pixel [w, x] of composite image 1800 and the pixel of original image '[m] 1410
Mix the colors of [s, t] and add new pixels
Calculate the color of [w, x]. The color of the pixel [w, x] of the composite image 1800 is C1, the color of the pixel [s, t] of the original image '[m] 1410 is C2, and the value of the weight data [w, x] is W1 composite data [m]. Assuming that the value of [s, t] is W2, the color C3 of the pixel [w, x] of the new composite image 1800
Is obtained by C3 = (C1 × W1 + C2 × W2) ÷ (W1 + W2). In processing step 1750, the value of the weight data [w, x] is
The value is updated by adding the value of the weight data [w, x] and the value of the composite data [m] [s, t].

In processing step 1760, the original image [m]
Processing step 1 for all pixels [s, t] of 1110
Steps 720 to 1760 are repeated. In processing step 1770, all original images [m] 1110
From processing step 1120 to processing step 177
Repeat until 0.

Hereinafter, the first embodiment will be summarized.
In the first embodiment, as the preprocessing 1300 of the composite image generation processing 1700, the maximum image determination processing 1900, the color mixture ratio determination processing 1310, and the redundant data determination processing 1310
A composite data generation process 1300 comprising 60 is performed. By performing the maximum image discrimination processing 1900, the original image 1100
Then, it is determined which original image 1100 has the largest image of each object shown in FIG. By performing the color mixture ratio determination processing 1310, discontinuous color changes between videos synthesized from different original images 1100 in the synthesized image 1800 are reduced, and the connection is made inconspicuous. By performing the redundant data determination processing 1360, the synthesized image 18
The portion of the original image 1100 and the portion of the initial deformation data 1200 that are unnecessary for synthesizing 00 are deleted. In the composite image generation processing 1700, a composite image 1600 is generated using the original image '1400, the corresponding data' 1500, and the composite data 1600, which are the output results of the preprocessing 1300.

Hereinafter, a second embodiment of the present invention will be described in detail. As shown in FIG. 7, the system configuration of the second embodiment includes a server 8100 and a client 8200.
Are connected using a network 8300 which is a wide area data transfer path. The server 8100 is connected to the network 830
0, a communication device 8110 that performs communication with the client 8200 via the communication device 0, a processing device 8120 that performs the preprocessing 1300 (a), the original image 1100, and the corresponding data 1200.
And a storage device 8130 for temporarily storing the synthesized data 1600 and the count data 2700 and an external storage device 2300 using a bus 8140 which is a data transfer path. The client 8200 is connected to the network 830
Communication device 821 that communicates with server 8100 using 0
0, display device 2100, and composite image generation processing 1700
A processing device 8220 for executing (a), and an input device 220
0 and the storage device 2500 are connected to the bus 8 which is a data transfer path.
Connect using 220.

The server 8100 executes the synthetic data generation processing 1300 (a). Synthetic data generation processing 1
300 (a) is a modified version of the synthetic data generation process 1300.
In 0, the storage device was the storage device 1800, whereas
In the combined data generation processing 1300 (a), the storage device 812
0. Synthetic image generation processing 1700 (a)
Is a modification of the composite image generation processing 1700. The difference is that the storage device 2
The data communication between the external storage device 500 and the external storage device 2300 is performed via the bus 2600, whereas the processed image generation processing 1700 (a) uses the bus 8140 and the communication device 811
0, network 8300, communication device 8210, bus 8
The point is that data communication is performed via 220.

Hereinafter, the second embodiment will be summarized.
In the second embodiment, the server 8100 stores the original image '1400, the corresponding data' 1500, and the combined data 1600.
Create and hold in. In the client 8200,
When there is a request to create the composite image 1800, the server 8100
Original image '1400 and corresponding data' 150 created in
0 and the combined data 1600 from the external storage device 2300 of the server 8100 to the storage device 2800 of the client 8200, and
At step 220, a composite image generation process 1700 (a) is executed to generate a composite image 1800.

[0042]

As described above, according to the present invention, in image processing for generating a composite image viewed from another viewpoint based on an original image photographed or drawn from a plurality of viewpoints, one object is processed. The largest original image is determined, and only the largest original image of each object is used to generate an image of that object in the composite image. An image processing method capable of generating a detailed and clear composite image can be provided as compared with an image processing method for generating a composite image.

Further, in the connection portion between the images generated from different original images in the composite image, the ratio of mixing the color of one original image to the color of the other original image is continuously changed. Therefore, it is possible to provide an image processing method that makes connection portions inconspicuous. Further, since data indicating a portion of the original image that is unnecessary for synthesizing the composite image and a portion of the corresponding data are prepared, the transformation process is performed on the original image that is unnecessary in the process of generating the composite image. It is not necessary to perform the mixing process, and as a result, the calculation amount of the process for generating the composite image can be reduced.

Further, for each object, the data other than the image which is shown in the largest size and the corresponding data for deforming the image are deleted, so that the amount of data required to generate the composite image can be reduced. .

[Brief description of the drawings]

FIG. 1 is a diagram showing a processing procedure and input / output data according to a first embodiment.

FIG. 2 is a configuration diagram of a system according to a first embodiment.

FIG. 3 is a flowchart illustrating a processing procedure (first half) of a maximum image determination processing according to the first embodiment.

FIG. 4 is a flowchart illustrating a processing procedure (second half) of a maximum image determination processing according to the first embodiment.

FIG. 5 is a flowchart illustrating a processing procedure of a color mixing ratio determination process according to the first embodiment.

FIG. 6 is a flowchart illustrating a processing procedure of redundant data determination processing according to the first embodiment.

FIG. 7 is a flowchart illustrating a processing procedure of a synthetic image generation process according to the first embodiment.

FIG. 8 is a system configuration diagram of a second embodiment.

FIG. 9 is a diagram showing the effect of a color right ratio determination process using a specific example.

Claims (6)

[Claims] 1. A plurality of original images photographed or drawn from a plurality of viewpoints, and a correspondence between the plurality of original images to indicate in which portion of each of the original images the same object is shown or written. In an image processing method for generating a composite image viewed from another viewpoint from data, a processing step of specifying the original image that is most clearly reflected or written for each of the objects, An image processing method, comprising a processing step of combining a video of each of the objects in the composite image from an image. 2. The method according to claim 1, wherein the correspondence data is used to identify the original image that is most clearly captured for each of the objects. An image processing method comprising a processing step of specifying an image. 3. The processing method according to claim 1, wherein in order to reduce a discontinuous change in a color of a connection portion of a video synthesized from a different original image in the video of the synthesized image, a processing step of determining the connection portion is performed. And a processing step of mixing the colors of the original images to be connected in the determined connection portion. 4. The method according to claim 1, wherein, prior to the process of generating the composite image, a portion of the original image and a portion of the corresponding data that are unnecessary for generating the composite image are determined in advance. A processing step of discriminating the unnecessary part of the original image and the unnecessary deformation data in order to reduce the calculation amount of the processing of generating the composite image; and the discriminated part of the unnecessary original image. And a processing step of generating composite data that enables the unnecessary corresponding data to be searched with a small amount of calculation as pre-processing of the processing of generating the composite image. And a processing step of generating. 5. The data amount required for synthesizing the composite image by deleting a portion of the original image and a portion of the corresponding data which are unnecessary for synthesizing the composite image according to claim 1. A processing step of determining a portion of the original image that becomes unnecessary and the deformed data that becomes unnecessary, and a process of deleting the part of the original image that becomes unnecessary and the deformed data that becomes unnecessary. And a step as a pre-process of a process of generating the image. 6. A means for storing the plurality of original images and the corresponding data; a processing step for identifying the original image most clearly captured for each of the objects; An image processing apparatus comprising: means for generating a composite image; and means for displaying the generated composite image.