JP2003087553A - Device and method for compositing images and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image compositing device for obtaining an image equivalent to an image obtained by zooming during the exposure by segmenting a plurality of images with different magnification from one piece of image data and compositing the plurality of images. SOLUTION: In a recording mode, an image fetched by a CCD 1 is sent to a YUV processor 3 to be subjected to color process processing, and image data are sent to an image memory 4 via a memory controller 5. A through image is simultaneously displayed. If it is in a composite zoom photographing mode here, a plurality of images are prepared by zooming the image data obtained at the moment a shutter is pressed down and stored in the image memory 4 as an original image to a plurality of stages. A composite zoom image equivalent to an image obtained be zooming during the exposure is prepared by compositing the plurality of the images.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image synthesizing apparatus, an image synthesizing method, and a program capable of obtaining an image having an effect equivalent to that of zooming during exposure which is one of the photographing methods of a silver salt camera. is there.

[0002]

2. Description of the Related Art Conventionally, as a photographing method of a silver halide camera, there is an inter-exposure zoom which obtains a moving light path by performing optical zoom with a shutter open. To zoom between exposures, fix the silver-salt camera on a tripod, set the shutter speed longer, and change the optical zoom while the shutter is open to shoot.

A digital camera is widely known as a type of electronic image pickup device. In recent years, the number of pixels in digital cameras has been increasing, and it has become possible to obtain images at a level comparable to that of silver halide cameras. The digital camera has a function such as a recording mode in which an image is captured and stored and a reproduction mode in which the stored image is reproduced. When taking an image in the recording mode of this digital camera, the same optical zoom effect can be obtained by using the optical zoom as in the silver halide camera.

[0004]

However, in the conventional silver halide camera or digital camera, in order to perform the zoom between exposures, the shutter speed must be set long, and therefore the camera body must be firmly fixed to the tripod. In addition, it was very time-consuming to change the optical zoom in that state, and it was not possible to shoot easily. Moreover, without a zoom lens, it was impossible to zoom between exposures with a silver halide camera or a digital camera.

The present invention has been made in view of such a conventional problem, and a plurality of image data having different enlargement ratios in which the enlargement ratio is continuously changed for one image data is created, An object of the present invention is to provide an image synthesizing apparatus, an image synthesizing method, and a program that obtain an image having the same effect as zooming during exposure by synthesizing them.

[0006]

In order to solve the above-mentioned problems, the invention according to claim 1 uses an image as an original image, and an image cutting-out means for cutting out images of different sizes from the original image, An image enlarging unit for enlarging each of the plurality of images cut out by the image cutting unit to the same size as the original image, and an image synthesizing unit for synthesizing the plurality of images enlarged by the image enlarging unit. There is.

As described above, according to the first aspect of the present invention, images of a plurality of sizes are cut out from one image, enlarged to the same size as the image before the cutout, and then these images are combined. Since this is done, it is possible to obtain an image equivalent to the inter-exposure zoom using the optical zoom without trouble.

According to a second aspect of the present invention, an image pickup means for picking up an image of a subject, a recording means for recording the image of the subject generated by the image pickup means as image data, and an information recording in which information of a cutout size is stored. And means.

As described above, according to the second aspect of the present invention, since the photographed image is recorded and a plurality of images are cut out based on the information of the size to be cut out, it is not necessary to fix the camera body on a tripod, and the handheld device The same effect as during-exposure zoom can be obtained in shooting.

According to a third aspect of the present invention, the image enlarging means in the image synthesizing device is executed based on a rectangular enlarging algorithm, and one pixel is selected from the pixels forming the image before enlarging and the value is selected. Is characterized by replicating.

As described above, according to the third aspect of the present invention, by using the rectangle enlargement algorithm which can enlarge the rectangle only by a simple integer operation, the image can be displayed in a short time without burdening the CPU. You can enlarge and combine.

Further, the invention according to claim 4 is characterized in that the image enlarging means is carried out based on a rectangular enlarging algorithm, weights a plurality of pixels constituting an image before enlarging, and duplicates the value. I am trying.

As described above, according to the fourth aspect of the invention, a sharp image can be obtained by performing interpolation processing on a plurality of pixels by weighting in the rectangular enlargement algorithm.

Further, the invention according to claim 5 is characterized in that the original image is a through image captured from an image pickup device.

As described above, according to the fifth aspect of the invention, the user can shoot while viewing the actual zoom composite image.

Further, in the invention according to claim 6, an image cutting process for cutting out an image of a plurality of different sizes from the original image, and a plurality of images cut out by the image cutting means are used as an original image. The image synthesizing method is characterized by an image synthesizing process that enlarges the image to the same size as the original image and an image synthesizing process that synthesizes a plurality of images enlarged by the image enlarging process.

As described above, according to the sixth aspect of the present invention, images of a plurality of sizes are cut out from one image, enlarged to the same size as the image before cutting, and then these images are combined. Since the method is adopted, it is possible to obtain an image equivalent to the zoom during exposure using the optical zoom without any trouble.

Further, in the invention as set forth in claim 7, an image cutting means for cutting out an image of a plurality of different sizes from the original image and a plurality of images cut out by the image cutting means are used as an original image. It is characterized by an image synthesizing program for realizing an image synthesizing unit for enlarging each image to the same size as the original image and an image synthesizing unit for synthesizing a plurality of images enlarged by the image enlarging unit.

As described above, according to the seventh aspect of the present invention, images of a plurality of sizes are cut out from one image, enlarged to the same size as the image before cutting, and then these images are combined. Since the program is used, it is possible to obtain an image equivalent to the in-exposure zoom using the optical zoom without trouble.

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention in which a digital camera 100 is used as an image synthesizing apparatus will be described below with reference to the drawings.

(First Embodiment) FIG. 1 shows a block diagram of a digital camera 100. Digital camera 100
Has a recording mode and a reproduction mode. In record mode,
Image data captured by the CCD 1 arranged behind the lens is converted into digital data by an A / D converter (not shown) provided together with the CCD control unit 2 controlling the CCD 1 and sent to the YUV processor 3. . YUV
Color processing is performed by the processor 3, and the digital luminance and color difference multiplex signals (YUV data) are transferred to the image memory 4 via the memory controller 5. The image memory 4 is composed of a memory such as a RAM.

At the same time, in order to display a through image, YU
The V data is also sent to the video encoder 6 via the memory controller 5, and the video encoder 6 periodically reads the YUV data, generates a video signal based on these data, outputs the video signal to the video output unit 7, and simultaneously outputs the video signal. The signal is also output to the display unit 14. The display unit 14 is composed of a liquid crystal display device or the like. As a result, an image based on the image information currently captured from the CCD 1 is displayed on the display unit 14.

When the shutter key 12 is operated at a timing at which recording and storage are desired while the image is currently displayed on the display unit 14 as described above, the key processing unit 11 processes the input of the shutter key 12 and the CPU 9 , Current C
Immediately after transferring the YUV data loaded from the CD 1 to the image memory 4 via the memory controller 5, the CCD
The path from 1 to the image memory 4 is stopped, and the state transits to the record storage state. In this recording and saving state, the CPU 9 compresses the YUV data written in the image memory 4,
Write to the recording medium 10. The recording medium 10 is a non-volatile memory such as a flash memory. The non-volatile memory may be removable. Then, when the compression processing of the YUV data and the writing of the compressed data to the flash memory are completed, the CPU 9 activates the path from the CCD 1 to the image memory 4 again.

In the reproduction mode, the CPU 9 stops the path from the CCD 1 to the image memory 4. When selecting from the images recorded by operating the various key units 13, the key processing unit 11 performs an input process, the CPU 9 reads the compressed data from the flash memory, performs the decompression process, and stores the YUV data in the image memory 4. expand. Then, the video encoder 6 generates a video signal based on the YUV data,
The video is output to the video output unit 7 and also output to the display unit 14 for display.

The data communication unit 8 transmits an image photographed and recorded or an image already recorded on the recording medium 10 to an external device, such as a personal computer, or receives an image from the external device. The communication may be wireless or wired (USB or the like).

In the operation of the various key portions 13, when the mode is the one for obtaining an image equivalent to the zoom during exposure, that is, the zoom composite photographing mode, first, at the moment when the shutter key 12 stored in the image memory 4 is pressed. A plurality of images that are zoomed in a plurality of steps are created using the image data as the original image.
Then, by combining these images, a zoom composite image equivalent to the image obtained by the zoom during exposure is created. The combined image is output to the video output unit 7 through the video encoder 6 and also to the display unit 14,
It is also recorded on the recording medium 10. The original image is CCD1
In addition to the image captured from the image, an image already recorded on the recording medium 10 or an image sent through the data communication unit 8 may be used, and a zoom composite image can be created for these images.

FIG. 2 shows the principle of zoom composition. Image 12
Is the original image captured from CCD 1. The image 13 is a one-step zoom-up image obtained by enlarging a part of the original image. The image 14 is a two-step zoom-up image obtained by enlarging the one-step zoom-up image. Image 15 is 2
It is a 3-step zoom-up image obtained by enlarging a step-zoom image. By combining these images, a zoom combined image like the image 16 can be obtained. In order to obtain a smooth zoom composite image in the image 16, it is necessary to set the zoom ratios of the zoomed-up images of the images 12 to 15 so as to change as finely as possible without taking discrete values.

Next, a rectangle enlargement algorithm which is an algorithm required for zoom composition for obtaining a smooth zoom ratio will be described with reference to FIGS. 3 (A) to 5 (B). Figure 3
FIG. 3A and FIG. 3B are line segment generation algorithms that are the basis of the rectangle enlargement algorithm required for zoom composition.
The line segment generation algorithm is from (X0, Y0) to (XN, Y
When drawing a line segment in N), it is determined at which coordinate the dots forming the line segment are placed.

As shown in FIG. 3A, the ideal straight line is a straight line that simply connects (X0, Y0) and (XN, YN), but in the digital image, the coordinates where points can be arranged are infinite. Does not exist, so place points on the coordinates closest to the ideal straight line. The starting point is (X0, Y0), but since the coordinates originally exist here, the Y coordinate at X0 becomes Y0. Hereafter, each time X increases by 1, an ideal straight line Y
Will increase by (YN-Y0) / (XN-X0). Ideally, the Y coordinate when X1 is Y0 + (YN-Y
0) / (XN-X0), but the closest Y1 is selected as indicated by the cross mark in FIG. Next X0 = 0, XN
= 16, Y0 = 0, YN = 10, showing how the coordinates are determined. When X increases by 1, the amount of increase in Y is (YN-
Y0) / (XN-X0) = 10/16 = 0.625. Further, at (X0, Y0), the error from the Y coordinate (Y error) is 0. The error of Y when X1 is 0+
0.625 = 0.625. The Y coordinate is determined by whether the Y error is greater than 0.5. In this case, since it is larger than 0.5, the Y coordinate is incremented by 1 and becomes 1 (Y1). Since the Y coordinate has increased by 1, the Y error is 0.625-1 = − by subtracting 1 from the Y error at this time.
It becomes 0.375. The error of Y when X2 is -0.37
5 + 0.625 = 0.25. Since the error of Y is 0.5 or less, the Y coordinate remains 1 (Y1). The Y error remains the same. Similarly, if 0.625 is added to the error of Y and it is determined whether or not it is larger than 0.5, a straight line (marked with X) as shown in FIG. 3A is obtained.

The flow chart of FIG. 3B is realized by only integer arithmetic in order to facilitate the processing of the above-described operation by a computer. In the above description,
The initial value of the error of Y was set to 0, and it was determined whether or not it was larger than 0.5 when the increment of Y was added. First, the initial value of the error was set to -0.5, and the increase of Y was increased. It was determined whether or not it was greater than 0 when the minutes were added. ΔY
= YN−Y0 and ΔX = XN−X0, the initial value of the error is −0.5 and the error is E _{i + 1} = E _i + ΔY / ΔX> 0.
Determine with. (If this condition is satisfied, E _{i + 1} = E
_i −1) Next, change the error to be performed by multiplying the error by 2ΔX. Then, the initial value of the error is −ΔX, and the error is E _{i + 1} * 2ΔX = E _i * 2ΔX + 2ΔY> 0.
Will be determined by. (E if this condition is met
_{i + 1} = E _i −2ΔX)

The operation of the above algorithm will be described with reference to the flowchart of FIG. First, the initial coordinate of X is set to X0 and the initial coordinate of Y is set to Y0 (S1). Next, the initial value of the error E is set to -ΔX =-(XN-X0) (S2). Draw a point on the coordinates (X, Y) (S3), error E
2ΔY = 2 (YN−Y0) is added to (S4), and the error E
Is determined to be greater than 0. If the error E is 0 or less, the process is skipped. If the error E is greater than 0, (S
5) 2ΔX is subtracted from the error E (S6), and the Y coordinate is incremented (S7). Next, the X coordinate is incremented (S8). The above process is repeated until the X coordinate exceeds XN (S9).

4 (A) to 4 (C) show a line segment expansion algorithm capable of expanding a line segment using the line segment generation algorithm shown in FIGS. 3 (A) and 3 (B). Show. As shown in FIG. 4A, a line segment S composed of dots S0 to SN is extended to a line segment D composed of dots D0 to DN. The principle is to replace the X coordinate of FIG. 3 (A) with the decompression destination line segment D and the Y coordinate with the original line segment S (FIG. 4).
(B)).

The line segment expansion algorithm will be described with reference to the flowchart of FIG. First, the initial pixel of the decompression destination line segment D is set to D0, and the initial pixel of the decompression source line segment S is set to S0 (S10). The initial value of the error E of the line segment S is -ΔD =
It is set to (DN-D0) (S11). Dot S of line segment S (initial state is S0.) Is copied to dot D of line segment D (initial state is D0.) (S12). The error E is 2ΔS =
2 (SN-S0) is added (S13) to determine whether the error E is larger than 0. If the error E is 0 or less, it is skipped. If the error E is larger than 0 (S14), the error is generated. 2ΔD is subtracted from E (S15), and at the same time, the line segment S of the extension source
Is incremented (the next dot is selected) (S16). Next, the D dot of the extension source is incremented (the next dot is selected) (S17). The above processing is repeated until the dots of D exceed DN (S18) until the processing of all the dots is completed.

FIGS. 4A to 4 are shown in FIGS. 5A and 5B.
Applying the line segment expansion algorithm shown in FIG.
We show a rectangle enlargement algorithm that can enlarge a rectangle by expanding it to dimensions. As shown in FIG. 5A, horizontal XS0 to XS
A rectangle S consisting of N and vertical YS0 to YSN is set to horizontal XD0 to XD
Enlarge to a rectangle D consisting of N and vertical YD0 to YDN.

In order to inflate the lines YS0 to YSN to the lines YD0 to YDN in the vertical direction, FIGS.
The line segment expansion algorithm shown in (C) is used. Line Y
The line YS0 is on D0, and the lines YD1 and YD2 are
The line YS1 is copied to. When the line is determined, the dots in the line can be expanded using the line segment expansion algorithm shown in FIGS. 4 (A) to 4 (C).

The rectangle enlarging algorithm shown above is shown in FIG.
This will be described with reference to the flowchart of (B). The portions other than the shaded areas (S19, S20, S30 to S35) are the processing for padding the line, and the shaded portions (S21 to S29) are the processing for extending the dots in the line. First, in order to pad the lines, the initial line of the rectangle S is set to YS0 and the initial line of the rectangle D is set to YD0 (S19). The initial value of the error EY for padding the line is -ΔYD =-(YD
(N-YD0) (S20). After this, the dots in the line are expanded (hatched portion). The shaded portions (S21 to S29) correspond to the flowcharts S10 to S18 described with reference to FIG. Next, for the error EY, 2ΔYS = 2 (YSN-YS
0) is added (S30). It is determined whether or not the error EY is greater than 0. If the error EY is 0 or less, the process is skipped. If the error EY is greater than 0 (S31), the error EY is determined.
Is subtracted from 2ΔYD (S32), and at the same time, the transfer source line YS is incremented (the next line is selected) (S33). Next, the transfer destination line YD is incremented (the next line is selected) (S34). The above process is repeated until the line of YD exceeds YDN (S35) and the process of all lines is completed.

By using this rectangle enlargement algorithm, it is possible to enlarge a rectangle of any size to a rectangle of any size. Therefore, prepare a plurality of images in which the zoom ratio changes minutely with respect to the original image. You can In addition, the rectangle can be expanded only by a simple integer operation.
It is possible to obtain a zoom composite image in a short time without imposing a burden on 9.

Next, FIG. 6A shows a cut-out image information table for cutting out images of a plurality of sizes. It is a table that summarizes the upper left coordinates and the image size of the original image that is cut out when the zoom ratio is continuously changed. Figure 6
In (A), the size of the original image is 1280 × 960.
The upper left coordinates of the image cut out when the original image is zoomed up by one step are (4, 3), and the image size is 1272 ×
954. Further, when zooming up in two steps, the upper left coordinates of the image to be cut out are (8, 6). Similarly, the upper left coordinates and image sizes for a plurality of stages of zoom-up are in a table. The cut-out images are enlarged to 1280 × 960 images by the rectangle enlargement algorithm shown in FIGS. 5A and 5B.

FIG. 6B shows an image obtained by enlarging a rectangle of the cut out image. If the upper left coordinates of the clipped image are (320, 240) and the image size is 640 × 480, then 1280 × 96 is obtained by the rectangle enlargement algorithm.
The case where the image is enlarged to 0 is shown.

As shown in FIG. 6B, the center of the image is zoomed up to obtain a plurality of zoomed images. However, the zooming center is zoomed while moving the zoom-up center in any direction from the center of the image. It is also possible to obtain an image. Further, although a plurality of zoom images are created from the original image, a zoom image may be created from an image that was previously cut out and rectangularly enlarged. In this case, since the upper left coordinates and the image size of the cutout image can be made constant, the information table of the cutout image as shown in FIG. 6A becomes unnecessary.

Further, the cut-out image size of the cut-out image information table is set finely in order to obtain a smooth zoom composite image, but by setting the cut-out image size roughly, it is possible to obtain a completely new image which cannot be obtained by the optical zoom. A zoom composite image can be obtained.

A method of synthesizing enlarged images will be described with reference to FIG. The image data has three RGB components and is composed of a plurality of pixels, and each RGB component has a value of 0 to 255 (8 bits). In FIG. 7, the synthesis of one of the RGB components will be described. The images 17 and 18 are images of a certain component to be combined, and the image 19 is an image of a certain component after combination. Focusing on the upper left pixel, the image 17 is 11 and the image 18 is 8, and when simply added, the image 19 becomes 19. In the pixel on the right of the image 17, the image 17 is 9 and the image 18 is 7, and when simply added, the image 19 is 16. Although not shown in FIG. 7, if the added result exceeds 255, the value of 255 or more is deleted. The respective RGB components can be similarly synthesized.

FIG. 8 shows a flowchart of the zoom composition process. Although not shown in the flowchart, first, the desired zoom-up size is set to x5, x10,
Designate as × 20. When the shutter key 12 is pressed in the zoom composite photographing mode and photographing is performed, an image captured from the CCD 1 is first used by using a part of the image memory 4 as an original image buffer and stored therein (S36). In the original image buffer, during zoom composition processing,
When shooting is performed, the image captured from the CCD 1 is always stored. A plurality of zoom images are created using these images as original images. At the same time, a part of the image memory 4 is used as a composite image buffer, and the captured image is also stored in the composite image buffer (setting of initial image) (S37). The composite image buffer stores composite images up to the previous zoom ratio during zoom combining processing, and the composite image buffer is composed with the image in the composite image buffer every time an image with a new zoom ratio is created. When the initial image is set in the composite image buffer, the image in the original image buffer is cut out according to the table described in FIG. 6A (S38). The cut-out image is subjected to rectangular enlargement processing to enlarge it to the same size as the original image (S3).
9). The rectangularly enlarged image is combined with the image in the combined image buffer and stored again in the combined image buffer (S4).
0). The processing from S38 is repeated until the zoom up to the target size is completed (S41). When the zoom up to the target size is completed, the image is stored in the recording medium 10 as a zoom composite image.

A plurality of rectangles of continuously changing size are cut out from one original image photographed as described above, the plurality of cut out rectangles are enlarged to the same size as the original image, and then these are combined. Since the image is obtained, it is possible to easily obtain an image equivalent to the image taken by the inter-exposure zoom using the optical zoom. Also, since it is sufficient to take one image normally, it is not necessary to fix the camera on a tripod as in the case of using the optical zoom, and the camera can be held by hand and taken.

(Second Embodiment) In the rectangular enlargement algorithm of the zoom composition process described so far, an arbitrary pixel forming the image after enlargement is located at the ideal position among the pixels forming the image before enlargement. A method of selecting one pixel close to the pixel and copying the value as it is was used. Next, a method will be described in which, as for any pixel forming the image after enlargement, a plurality of pixels around the ideal position forming the image before enlargement are weighted and then the value is copied. 9A to 10
In (B), a rectangle enlargement algorithm using weighting is shown. The operation of the digital camera 100 other than the rectangular enlargement algorithm in the zoom combining process, the information table of the cut-out image, the zoom combining process, and the like are the same as those in the first embodiment already described, and thus the description thereof is omitted.

9A to 9C show a line segment expansion algorithm using weighting. As shown in FIG. 9A, a line segment S composed of (n + 1) dots from S0 to SN is
The weighting is used to expand from D0 to a line segment D consisting of DM (m + 1) dots. As shown in FIG. 9B, first, the starting point S0 of the line segment S is directly copied to the starting point D0 of the line segment D. Next, when the line segment D increases by 1 from D0 to D1, the pixel of the ideal line segment S becomes a position (circle mark) from S0 between S0 (Δ mark) and S1 (▽ mark) to n / m. . However, since no pixel actually exists at this position, pixel S1 and pixel S
It is calculated from the value of 2. Since the position is n / m from S0 and is (m−n) / m from S1, the value of the pixel D1 is D1.
= S0 * (m−n) / m + S1 * n / m.

Further, when the line segment D is increased by 1 from D1 to D2, the pixel of the ideal line segment S is located at a position of 2 * n / m from S0. At this time, as shown in FIG.
When * n / m is 1 or more, pixels of the line segment S used for interpolation are S1 (marked with Δ) and S2 (marked with ▽), and ideal pixels of the line segment S are from S1 to (2 * n / m) -1 position (○
It can be said that it is in (). The value of pixel D2 is D2 = S1 * 2
It is calculated by the formula of (m−n) / m + S2 * (2n−m) / m.

When these are expressed by general formulas, if the pixels of the line segment S used for interpolation are S (Δ mark) and S ′ (∇ mark), the pixel S (Δ mark) and the ideal position (◯ mark) are obtained. The initial value of the error is 0, and the error is determined by E _{i + 1} = E _i + n / m ≧ 1. (If this condition is satisfied, E _{i + 1} = E _i −1)
If E _{i + 1} = E _i + n / m ≧ 1 is satisfied, 1 is subtracted from E and the pixels S and S ′ used for interpolation are updated. Further, the value of the pixel D is S = (S * (1-E) + S '* E) from S (Δ mark) and S ′ (∇ mark).

In order to make the above-described operation easy for the computer to process, the flowchart of FIG.
The error discrimination part is realized by integer arithmetic. In the above description, the initial value of the error is set to 0, and it is determined whether or not it is 1 or more when n / m is added. However, first, the initial value of the error is set to -1, and n / m is added. When it is, it is determined whether or not it is 0 or more. The initial value of the error is −1, and the error is determined by E _{i + 1} = E _i + n / m ≧ 0. (E _{i + 1} = E _i -1 if this condition is satisfied) and D =
S * (-E) + S '* (E + 1). Next, the error is determined so that the error is multiplied by m. Then, the initial value of the error is −m, and the error is E _{i +1} * m = E
It will be determined by _i * m + n ≧ 0. (If they meet the conditions _{E i + 1 = E i -m} ) and D = S * (-
E) / m + S '* (E + m) / m.

The operation of the above algorithm is shown in detail in FIG.
Description will be given along the flowchart of (C). First, the pixel D of the decompression destination line segment D is initialized to D0, the first pixel S used for interpolation of the decompression source line segment S is initialized to S0, and the second pixel S ′ used for interpolation is initialized to S1. Yes (S42). Next, the line segment S
The initial value of the error E of is set to -m (S43). D = S
* (-E) / m + S '* (E + m) / m is used for each RGB component to obtain the RGB value of the pixel D of the transfer destination (S4).
4). N is added to the error E (S45), it is determined whether or not the error E is larger than 0. If the error E is 0 or less, it is skipped. If the error E is larger than 0 (S46), the error E is obtained. Is subtracted from m (S47), and at the same time, the line segment S of the extension source
The pixels S and S'of are incremented (the next pixel is selected) (S48). Next, the pixel D of the decompression destination line segment D is incremented (the next pixel is selected) (S49). The above processing is repeated until the pixels of D exceed DM (S50) until the processing of all the pixels D is completed.

Next, in FIG. 10 (A) and FIG. 10 (B),
A rectangle enlargement algorithm that can enlarge a rectangle by using the line segment extension algorithm using the weighting shown in FIGS. 9A to 9C is shown. As shown in FIG. 10A, a rectangle composed of horizontal (nx + 1) dots and vertical (ny + 1) dots is expanded to a rectangle composed of horizontal (mx + 1) dots and vertical (my + 1) dots. In both the horizontal (X) direction and the vertical (Y) direction, the line segment decompression algorithm described with reference to FIGS. 9A to 9C is used to change from the four pixels of the decompression source rectangle S to the decompression destination rectangle D. Calculate the value of one pixel.

The above rectangle enlarging algorithm is shown in FIG.
A description will be given along the flowchart of (B). First, the Y coordinate YD of the pixel D of the decompression destination rectangle D is YD0, the first Y coordinate YS used for interpolation of the decompression source rectangle S is YS0, and the second Y coordinate YS 'used for interpolation is Initially set to YS1 (S51). The initial value of the error EY in the Y direction is set to -my (S52). After this, processing in the X direction is performed.
(Shaded portion) This shaded portion (S53 to S61) is Y
Only the X coordinate is updated while the coordinates remain fixed. First, the X coordinate XD of the image D of the decompression destination rectangle D is XD0, the first X coordinate XS used for interpolation of the decompression source rectangle S is XS0,
The second X coordinate XS 'used for interpolation is initialized to XS1 (S53). Next, the initial value of the error EX in the X direction is −
mx is set (S54). Two Y coordinates Y at this point
Four pixels (XS, YS) (XS ', YS) of a rectangle S used for interpolation from S, YS' and two X coordinates XS, XS '
(XS, YS ') (XS', YS ') is determined. The interpolation method will be described with reference to FIG. 11, but the RGB values of the pixel at one point (XD, YD) on the decompression destination rectangle D are obtained from the values of these four points (S55). Next, nx is added to the error EX (S56). It is determined whether the error EX is larger than 0, and if the error EX is 0 or less, the error EX is skipped and the error EX
Is larger than 0 (S57), mx is subtracted from the error EX (S58), and at the same time, the X-coordinates XS and XS 'of the decompression source are incremented (S59). Next, the X coordinate XD of the decompression destination
Is incremented (the next pixel is selected) (S60).
Until XD pixels exceed XDM (S61), all X
The above process is repeated until the coordinate process is completed. Next, ny is added to the error EY (S62). It is determined whether or not the error EY is larger than 0. If the error EY is 0 or less, the process is skipped. If the error EY is larger than 0 (S63).
My is subtracted from the error EY (S64), and at the same time, the Y of the extension
The coordinates YS and YS 'are incremented (S65). Next, the Y coordinate YD of the extension destination is incremented (the next line is selected) (S66). Until the YD line exceeds YDM (S67), the above processing is repeated until the processing of all Y coordinates is completed.

FIG. 11 illustrates a method of image interpolation by weighting and interpolating one pixel at the decompression source from four pixels at the decompression source. By the rectangle enlargement algorithm shown in FIGS. 10 (A) and 10 (B), the four pixels S1 (XS, Y) of the extension source are used.
S), S2 (XS ', YS), S3 (XS, YS'),
S4 (XS ', YS') is determined. Furthermore, X direction, Y
The value of the weighting in both directions is S * (-EX) / m in the X direction.
x + S '* (EX + mx) / mx, and Y direction is S *
Since (−EY) / my + S ′ * (EY + my) / my, the calculated RGB value is ((S1 * (− EX) / mx + S
2 * (EX + mx) / mx) * (-EY) / my) +
((S3 * (-EX) / mx + S4 * (EX + mx) /
mx) * (EY + my) / my). RGB
It may be calculated from this formula for each.

In this way, since the four pixels of the original image to be expanded are weighted to determine the pixel values of the expanded image, a clear image can be obtained. As a method of enlarging a rectangle, weighted interpolation is performed in both the vertical and horizontal directions. However, only one of the vertical direction and the horizontal direction is weighted, and the remaining one is shown in FIG. The normal rectangle enlargement algorithm shown in FIG. 5 (B) may be used. This makes it possible to reduce the load on the CPU 9 while maintaining a certain level of image quality.

(Third Embodiment) Next, referring to FIG.
A zoom combining process will be described in which the rectangular enlargement algorithm shown in (A) and FIG. 5 (B) and the rectangular enlargement algorithm by weighting shown in FIGS. 10 (A) and 10 (B) are separately used.

Although not shown in the flowchart of FIG. 12, first, the target zoom-up size is designated by various key portions 13 such as × 5, × 10, and × 20. When the shutter key 12 is pressed in the zoom composite photographing mode to perform photographing, first, an image captured from the CCD 1 is used by using a part of the image memory 4 as an original image buffer and stored therein (S68). When shooting is performed during the zoom composition process, the original image buffer always stores the image captured from the CCD 1. A plurality of zoom images are created using these images as original images. At the same time, a part of the image memory 4 is used as a composite image buffer, and the captured image is also stored in the composite image buffer (S69). Next, the image in the original image buffer is cut out according to the table of FIG. 6 (A) (S70). If the image is the image with the smallest zoom ratio (first image) or the image with the largest zoom ratio (last image) (S71), FIG. 10A and FIG.
The rectangular enlargement algorithm by weighting shown in (B) is used for processing (S73), and for other images, the rectangular enlargement algorithm shown in FIG. 5A and FIG. (S72). The rectangular enlarged image is combined with the image in the combined image buffer (S74). The processing from S70 is repeated until the zoom-up to the target size is completed (S75). When the zoom up to the target size is completed, the image is stored in the recording medium 10 as a zoom composite image.

As described above, by properly using the two types of rectangle enlargement algorithms, the images at the both ends of the zoom start and the zoom end become clear, and the CPU load can be lightened in the dynamic image filling between them. It will be possible.
In addition to the images at the start and end of zooming, it is possible to improve the image quality of the zoom composite image by inserting a zoom image using a rectangular enlargement algorithm that performs weighting at regular intervals among the zoom images that make up the space. It is possible.

(Fourth Embodiment) In the first to third embodiments described above, the zoom compositing process is a compositing process using the image captured by pressing the shutter key 12 as the original image. However, as the fourth embodiment, an example in which the zoom compositing process is performed using the through image before the shutter key 12 is pressed as the original image will be described. An information table of the operation of the digital camera 100 and the cutout image,
The rectangle enlargement algorithm is the same as that of the first to third embodiments already described, and therefore the description thereof is omitted.

In this embodiment, the desired zoom-up size is first set to x5, x10,
Designate as × 20. When in the zoom composite shooting mode, the zoom composite processing is performed while the image data taken in from the CCD 1 is displayed as a through image on the display unit 14. That is, after the shutter key 12 is operated in the above-described first to third embodiments, the same process as the zoom combining process performed in the stage of the process of capturing and storing the image is performed before the shutter key 12 is operated. This is performed in the ready state at any time, and as a result, the zoom-combined image is output to the video output unit 7 and the display unit 14 at the same time. As a result, the zoom composite image is always displayed on the display unit 14 in the imaging preparation state, and the user presses the shutter key 12 while confirming on the display unit 14 the zoom-combined image to be captured. Shutter key 12
When is pressed, the zoom composite image displayed on the display unit 14 at that time is stored in the recording medium 10. Of course, in this case, as in the first to third embodiments described above, the combining process may be performed using the image captured by pressing the shutter key 12 as the original image.

As described above, before the shutter key 12 is pressed, the zoom composition processing is performed and displayed on the display unit 14 at the time of displaying the through image, so that the user can take a picture while viewing the zoom composition image. Therefore, since it is possible to select whether or not to shoot at the time of viewing the zoom composite image, useless shooting and recording can be avoided, and an intended image can be easily obtained.

[0061]

As described above, according to the first aspect of the present invention, images of a plurality of sizes are cut out from one image,
Since these images are combined after being enlarged to the same size as the image before being cut out, it is possible to obtain an image equivalent to the in-exposure zoom using the optical zoom without trouble.

According to the second aspect of the present invention, since the photographed image is recorded and a plurality of images are cut out based on the information on the size to be cut out, it is not necessary to fix the camera body to a tripod, and it is possible to take a handheld photograph. The same effect as the zoom between exposures can be obtained.

According to the third aspect of the invention, by using the rectangle enlargement algorithm which can enlarge the rectangle only by a simple integer operation, the image can be enlarged in a short time without burdening the CPU. Can be synthesized.

According to the fourth aspect of the invention, a sharp image can be obtained by performing interpolation processing on a plurality of pixels by weighting in the rectangular enlargement algorithm.

According to the fifth aspect of the invention, since the original image is the through image captured from the image pickup element, it is possible to shoot while viewing the actual zoom composite image.

According to the sixth aspect of the present invention, there is provided a method of cutting out images of a plurality of sizes from one image, enlarging the image to the same size as the image before cutting, and then synthesizing these images. Therefore, it is possible to obtain an image equivalent to the during-exposure zoom using the optical zoom without trouble.

Further, according to the invention described in claim 7, an image having a plurality of sizes is cut out from one image, enlarged to the same size as the image before the cutout, and then a program for synthesizing these images is provided. Therefore, it is possible to obtain an image equivalent to the during-exposure zoom using the optical zoom without trouble.

[Brief description of drawings]

FIG. 1 is a block diagram of a device of the present invention.

FIG. 2 is a diagram showing a principle of zoom combination.

FIG. 3A is a coordinate diagram showing a line segment generation algorithm. (B) is a flowchart showing a line segment generation algorithm.

FIG. 4A is a structural diagram showing a line segment expansion algorithm. (B) is a coordinate diagram showing a line segment expansion algorithm. (C) is a flow chart showing the minute extension algorithm.

FIG. 5A is a structural diagram showing a rectangle enlargement algorithm. (B) is a flowchart showing a rectangle enlargement algorithm.

FIG. 6A is a diagram showing an information table of a cutout image. (B) is a diagram showing an image of a cutout image.

FIG. 7 is a diagram showing a method of synthesizing an enlarged image.

FIG. 8 is a flowchart of zoom synthesis processing.

FIG. 9A is a structural diagram showing a weighted line segment expansion algorithm. (B) is a coordinate diagram showing a line segment expansion algorithm by weighting. (C) is a flowchart showing a weighted line segment expansion algorithm.

FIG. 10A is a structural diagram showing a rectangle enlargement algorithm by weighting. (B) is a flowchart showing a rectangle enlargement algorithm by weighting.

FIG. 11 is a diagram showing image interpolation by weighting.

FIG. 12 is a flowchart of zoom synthesis processing by two types of rectangle enlargement algorithms.

[Explanation of symbols]

1 CCD 2 CCD control unit 3 YUV processor 4 image memory 5 Memory controller 6 video encoder 7 Video output section 8 Data communication section 9 CPU 10 recording media 11 Key processing part 100 digital camera

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 5/232 H04N 5/232 Z 5/262 5/262 // H04N 101: 00 101: 00 F term ( Reference) 5B057 BA02 BA11 CA08 CA12 CA16 CB08 CB12 CB16 CD05 CD06 CE08 CE09 CH09 5C022 AA13 AB36 AB68 AC01 AC42 AC69 5C023 AA11 AA37 AA38 BA11 CA01 DA04 5C076 AA02 AA11 AA19 AA21 BA06 BB03 BB25 CB25 BB25 CA10

Claims (7)

[Claims] 1. An image cutting-out unit that cuts out a plurality of images of different sizes from one original image as an original image, and a plurality of images cut out by the image cutting-out unit have the same size as the original image, respectively. An image synthesizing apparatus comprising: an image enlarging means for enlarging; and an image synthesizing means for synthesizing a plurality of images enlarged by the image enlarging means. 2. An image pickup unit for picking up an image of a subject, a recording unit for recording an image of the subject generated by the image pickup unit as image data, and an information recording unit for storing information on a size to be cut out. The image synthesizing apparatus described. 3. The image enlarging means is executed based on a rectangle enlarging algorithm, and selects one pixel from pixels constituting an image before enlarging and duplicates the value. Image synthesizer. 4. The image enlargement means according to claim 1, wherein the image enlargement means is executed based on a rectangle enlargement algorithm, weights a plurality of pixels constituting an image before enlargement, and duplicates the value. Synthesizer. 5. The image synthesizing apparatus according to claim 1, wherein the original image is a through image captured from an image sensor. 6. An image cropping process of cropping an image of a plurality of different sizes from the original image using one image as an original image, and the plurality of images cropped by the image cropping unit having the same size as the original image, respectively. An image combining method comprising an image expanding process for expanding and an image combining process for combining a plurality of images expanded by the image expanding process. 7. An image cutting means for cutting out a plurality of images of different sizes from the original image using one image as an original image, and a plurality of images cut out by the image cutting means to have the same size as the original image, respectively. An image composition program for realizing an image enlarging means for enlarging and an image synthesizing means for synthesizing a plurality of images enlarged by the image enlarging means.