JP3861641B2 - Image composition apparatus, image composition method, and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image synthesizing apparatus, an image synthesizing method, and a program capable of obtaining an image having an effect equivalent to that of an inter-exposure zoom which is one of photographing methods of a silver salt camera.
[0002]
[Prior art]
Conventionally, as a photographing method of a silver salt camera, there is an inter-exposure zoom in which an optical zoom is performed with a shutter opened to obtain a flowing light trajectory. To zoom between exposures, fix a silver salt camera with a tripod, set a long shutter speed, and change the optical zoom while the shutter is open.
[0003]
A digital camera is widely known as a kind of electronic imaging apparatus. In recent years, the number of pixels of a digital camera has been increased, and an image at a level comparable to that of a silver halide camera can be obtained. A digital camera has functions such as a recording mode for capturing and storing images, and a playback mode for reproducing stored images. When taking an image in the recording mode of the digital camera, the same effect of zooming between exposures can be obtained by using an optical zoom in the same manner as a silver halide camera.
[0004]
[Problems to be solved by the invention]
However, in conventional silver halide cameras and digital cameras, zooming between exposures requires a long shutter speed, so the camera body must be firmly fixed to a tripod and optical It was very time consuming to change the zoom, and it was not possible to shoot easily. Also, without a zoom lens, it was impossible to zoom between exposures with a silver salt camera or a digital camera.
[0005]
The present invention has been made in view of such a conventional problem. A plurality of pieces of image data having different enlargement ratios are generated by continuously changing the enlargement ratio for one piece of image data, and these are synthesized. Accordingly, an object of the present invention is to provide an image composition device, an image composition method, and a program for obtaining an image that has the same effect as the zoom between exposures.
[0006]
[Means for Solving the Problems]
In order to solve the above problems, the invention according to claim 1 is characterized in that an image cutout unit that cuts out images of a plurality of different sizes from the original image, and a plurality of pieces cut out by the image cutout unit. An image enlarging means for enlarging each image to the same size as the original image, and an image enlarged by the image enlarging means Add pixel values of corresponding pixels of multiple images of the same size And image synthesizing means.
[0007]
As described above, according to the first aspect of the present invention, an image having a plurality of sizes is cut out from a single image, enlarged to the same size as the image before cutting, and then combined. Therefore, an image equivalent to the zoom between exposures using the optical zoom can be obtained without taking time and effort.
[0008]
According to a second aspect of the present invention, there is provided an imaging unit that images a subject, a recording unit that records a subject image generated by the imaging unit as image data, and an information recording unit that stores information about a size to be cut out. It is characterized by providing.
[0009]
Thus, according to the second aspect of the present invention, since the photographed image is recorded and a plurality of images are cut out based on the size information to be cut out, there is no need to fix the camera body to a tripod, and exposure in hand-held shooting The same effect as zoom can be obtained.
[0010]
According to a third aspect of the present invention, the image enlargement means in the image composition apparatus is performed based on a rectangular enlargement algorithm, and selects one pixel from pixels constituting the image before enlargement and duplicates the value. It is characterized by that.
[0011]
Thus, according to the third aspect of the present invention, by using a rectangular enlargement algorithm that can enlarge a rectangle with only a simple integer operation, it is possible to enlarge and synthesize an image in a short time without imposing a burden on the CPU. Can be done.
[0012]
According to a fourth aspect of the present invention, the image enlarging means is performed based on a rectangular enlarging algorithm, weights a plurality of pixels constituting the image before enlargement, and duplicates the values.
[0013]
Thus, according to the fourth aspect of the present invention, a clear image can be obtained by interpolating a plurality of pixels by weighting in the rectangular enlargement algorithm.
[0014]
The invention according to claim 5 is characterized in that the original image is a through image taken from an image sensor.
[0015]
Thus, according to the fifth aspect of the invention, the user can take a picture while viewing the actual zoom composite image.
[0016]
According to a sixth aspect of the present invention, there is provided an image cutout process for cutting out images of a plurality of different sizes from the original image, and a plurality of images cut out by the image cutout means as the original image. The image was enlarged by the image enlargement process and the image enlargement process. Add pixel values of corresponding pixels of multiple images of the same size It is characterized by an image composition method comprising image composition processing.
[0017]
As described above, according to the invention described in claim 6, after a plurality of sizes of images are cut out from one image and enlarged to the same size as the image before cutting out, these images are combined. Therefore, an image equivalent to the exposure zoom using the optical zoom can be obtained without taking time and effort.
[0018]
According to the seventh aspect of the present invention, there is provided an image cutout unit that cuts out images of a plurality of different sizes from the original image, and a plurality of images cut out by the image cutout unit as the original image. And image enlargement means for enlarging each to the same size, and enlarged by the image enlargement means Add pixel values of corresponding pixels of multiple images of the same size It is characterized by an image composition program for realizing an image composition means.
[0019]
Thus, according to the seventh aspect of the present invention, after cutting out images of a plurality of sizes from a single image and enlarging them to the same size as the image before cutting out, a program for combining these images is made. Therefore, an image equivalent to the exposure zoom using the optical zoom can be obtained without taking time and effort.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention when a digital camera 100 is used as an image composition apparatus will be described with reference to the drawings.
[0021]
(First embodiment)
FIG. 1 shows a block diagram of the digital camera 100. The digital camera 100 has a recording mode and a playback mode. In the recording mode, the image data captured by the CCD 1 disposed behind the lens is converted into digital data by an A / D converter (not shown) provided with the CCD control unit 2 that controls the CCD 1, and is converted into a YUV processor. Sent to 3. A color process is performed by the YUV processor 3, and digital luminance and color difference multiplexed signals (YUV data) are transferred to the image memory 4 via the memory controller 5. The image memory 4 is composed of a memory such as a RAM.
[0022]
At the same time, the YUV data is also sent to the video encoder 6 via the memory controller 5 to display the through image, and the video encoder 6 periodically reads the YUV data and generates a video signal based on these data. Simultaneously with the output to the video output unit 7, the video signal is also output to the display unit 14. The display unit 14 includes a liquid crystal display device. As a result, an image based on the image information captured from the current CCD 1 is displayed on the display unit 14.
[0023]
In this way, when the shutter key 12 is operated at the timing at which recording and storage is desired while the image is currently displayed on the display unit 14, the key processing unit 11 processes the input of the shutter key 12, and the CPU 9 After the YUV data fetched from the CCD 1 is transferred to the image memory 4 via the memory controller 5, the path from the CCD 1 to the image memory 4 is immediately stopped and the recording and storage state is entered. In this recording / saving state, the CPU 9 compresses the YUV data written in the image memory 4 and writes it in the recording medium 10. The recording medium 10 is a flash memory that is a nonvolatile memory. The nonvolatile memory may be removable. Then, along with the compression processing of the YUV data and the writing of the compressed data to the flash memory, the CPU 9 activates the path from the CCD 1 to the image memory 4 again.
[0024]
In the reproduction mode, the CPU 9 stops the path from the CCD 1 to the image memory 4. When a selection is made from images recorded by operating the various key sections 13, the key processing section 11 performs input processing, the CPU 9 reads compressed data from the flash memory, performs decompression processing, and stores YUV data in the image memory 4. expand. Then, the video encoder 6 generates a video signal based on the YUV data and outputs it to the video output unit 7 and also to the display unit 14 for display.
[0025]
The data communication unit 8 transmits a captured and recorded image or an image already recorded on the recording medium 10 to an external device, such as a personal computer, or receives an image from the external device. Communication may be either wireless or wired (such as USB).
[0026]
When the operation of various key units 13 is in a mode for obtaining an image equivalent to the zoom during exposure, that is, in the zoom composite shooting mode, first, image data at the moment when the shutter key 12 stored in the image memory 4 is pressed is obtained. A plurality of images zoomed in a plurality of stages are created as original images. Then, by combining these images, a zoom composite image equivalent to the image obtained by the zoom between exposures is created. The synthesized image is output to the video output unit 7 through the video encoder 6, is output to the display unit 14, and is also recorded on the recording medium 10. Further, the original image is not limited to the image captured from the CCD 1 but may be an image already recorded on the recording medium 10 or an image sent through the data communication unit 8. Can be created.
[0027]
FIG. 2 shows the principle of zoom composition. An image 12 is an original image taken from the CCD 1. The image 13 is a one-stage zoom-up image obtained by enlarging a part of the original image. The image 14 is a two-stage zoom-up image obtained by enlarging the one-stage zoom-up image. The image 15 is a three-stage zoom-up image obtained by enlarging the two-stage zoom-up image. When these are combined, a zoom combined image like the image 16 can be obtained. In order to obtain a smooth zoom composite image with the image 16, it is necessary to take values that change as finely as possible without taking the values of the zoom ratios of the zoomed-up images of the images 12 to 15.
[0028]
Next, a rectangular enlargement algorithm, which is an algorithm necessary for zoom composition for obtaining a smooth zoom ratio, will be described with reference to FIGS. 3 (A) to 5 (B). FIGS. 3A and 3B are line segment generation algorithms that are the basis of a rectangular enlargement algorithm necessary for zoom composition. The line segment generation algorithm determines the coordinates at which the dots constituting the line segment are placed when the line segment is drawn from (X0, Y0) to (XN, YN).
[0029]
As shown in FIG. 3A, an ideal straight line is a straight line simply connecting (X0, Y0) and (XN, YN). However, in a digital image, there are infinite coordinates where points can be arranged. Therefore, the points are arranged on the coordinates closest to the ideal straight line. The starting point is (X0, Y0), but since the coordinates originally exist here, the Y coordinate at X0 is Y0. Hereinafter, Y increases by (YN−Y0) / (XN−X0) by an ideal straight line every time X increases by 1. The Y coordinate at X1 is ideally Y0 + (YN-Y0) / (XN-X0), but the closest Y1 is selected as shown by the x mark in FIG. Next, how the coordinates are determined with X0 = 0, XN = 16, Y0 = 0, and YN = 10 is shown. The increment of Y when X increases by 1 is (YN−Y0) / (XN−X0) = 10/16 = 0.625. Further, the error (Y error) from the Y coordinate is 0 at (X0, Y0). The error of Y at X1 is 0 + 0.625 = 0.625. The Y coordinate is determined by whether or not the error of Y is greater than 0.5. In this case, since it is larger than 0.5, the Y coordinate increases by 1 to 1 (Y1). Since the Y coordinate has increased by 1, by subtracting 1 from the Y error at this time, the Y error becomes 0.625-1 = −0.375. The error of Y at X2 is −0.375 + 0.625 = 0.25. Since the error of Y is 0.5 or less, the Y coordinate remains 1 (Y1). The error of Y remains as it is. Similarly, when 0.625 is added to the error of Y to determine whether it is larger than 0.5, a straight line (x mark) as shown in FIG. 3A is obtained.
[0030]
The flowchart in FIG. 3B is realized only by integer arithmetic so that the operation described above can be easily processed by a computer. In the above description, the initial value of the error of Y is set to 0, and it is determined whether or not it is larger than 0.5 when the increment of Y is added. Thus, the determination is made based on whether or not the increase in Y is greater than 0. If ΔY = YN−Y0 and ΔX = XN−X0, the initial value of the error is −0.5 and the error is E _{i + 1} = E _i It is determined by + ΔY / ΔX> 0. (If this condition is met, E _{i + 1} = E _i -1) Next, the error is determined so that the error is determined by multiplying the error by 2ΔX. Then, the initial value of the error is −ΔX, and the error is E _{i + 1} * 2ΔX = E _i * 2 ΔX + 2ΔY> 0. (If this condition is met, E _{i + 1} = E _i -2ΔX)
[0031]
The operation of the above algorithm will be described with reference to the flowchart of FIG. First, the initial coordinate of X is set to X0, and the initial coordinate of Y is set to Y0 (S1). Next, an initial value of the error E is set to −ΔX = − (XN−X0) (S2). A point is drawn at the coordinates (X, Y) (S3), 2ΔY = 2 (YN−Y0) is added to the error E (S4), and it is determined whether or not the error E is greater than 0, and the error E is 0. In the following cases, the process is left as it is, and if the error E is larger than 0 (S5), 2ΔX is subtracted from the error E (S6), and the Y coordinate is incremented (S7). Next, the X coordinate is incremented (S8). The above processing is repeated until the X coordinate exceeds XN (S9).
[0032]
4A to 4C show a line segment expansion algorithm that can expand a line segment using the line segment generation algorithm shown in FIGS. 3A and 3B. As shown in FIG. 4A, the line segment S composed of dots from S0 to SN is expanded to a line segment D composed of dots from D0 to DN. The principle is that the X-coordinate in FIG. 3A may be replaced with the stretched line segment D and the Y-coordinate is replaced with the original line segment S (FIG. 4B).
[0033]
The line segment extension algorithm will be described with reference to the flowchart of FIG. First, the initial pixel of the extension line segment D is set to D0, and the initial pixel of the extension line segment S is set to S0 (S10). The initial value of the error E of the line segment S is set to −ΔD = (DN−D0) (S11). The dot S of the line segment S (initial state is S0) is copied to the dot D of the line segment D (initial state is D0) (S12). 2ΔS = 2 (SN−S0) is added to the error E (S13), and it is determined whether or not the error E is greater than 0. If the error E is equal to or less than 0, the error E is lost, and the error E is greater than 0. (S14) 2ΔD is subtracted from the error E (S15), and at the same time, the dots of the line segment S as the expansion source are incremented (the next dot is selected) (S16). Next, the extension dot D is incremented (the next dot is selected) (S17). Until the dot of D exceeds DN (S18), the above processing is repeated until the processing of all dots is completed.
[0034]
FIGS. 5A and 5B show a rectangular enlargement algorithm that can enlarge a rectangle when it is expanded in two dimensions by applying the line segment extension algorithm shown in FIGS. 4A to 4C. As shown in FIG. 5A, a rectangle S composed of horizontal XS0 to XSN and vertical YS0 to YSN is expanded to a rectangle D composed of horizontal XD0 to XDN and vertical YD0 to YDN.
[0035]
In the vertical direction, the line segment expansion algorithm shown in FIGS. 4A to 4C is used to pad the lines YS0 to YSN to the lines YD0 to YDN. The line YS0 is copied to the line YD0, and the line YS1 is copied to the lines YD1 and YD2. When the line is determined, the dots in the line can be expanded using the line segment expansion algorithm shown in FIGS. 4 (A) to 4 (C).
[0036]
The rectangular enlargement algorithm described above will be described with reference to the flowchart of FIG. Portions other than the shaded areas (S19, S20, S30 to S35) are processes for adding water to the line, and shaded areas (S21 to S29) are processes for expanding the dots in the line. First, in order to pad the line, the initial line of the rectangle S is set to YS0, and the initial line of the rectangle D is set to YD0 (S19). The initial value of the error EY for inflating the line is set to −ΔYD = − (YDN−YD0) (S20). Thereafter, the dots in the line are expanded (shaded portion). The shaded portions (S21 to S29) correspond to the flowcharts S10 to S18 described with reference to FIG. Next, 2ΔYS = 2 (YSN−YS0) is added to the error EY (S30). It is determined whether or not the error EY is greater than 0. If the error EY is less than or equal to 0, the process exits. If the error EY is greater than 0 (S31), 2ΔYD is subtracted from the error EY (S32). The line YS is incremented (the next line is selected) (S33). Next, the transfer destination line YD is incremented (the next line is selected) (S34). Until the YD line exceeds YDN (S35), the above process is repeated until all the lines have been processed.
[0037]
If this rectangular enlargement algorithm is used, a rectangle having an arbitrary size can be enlarged to a rectangle having an arbitrary size, so that a plurality of images whose zoom ratios change finely with respect to the original image can be prepared. Further, since the enlargement of the rectangle can be performed only by a simple integer calculation, a zoom composite image can be obtained in a short time without imposing a burden on the CPU 9.
[0038]
Next, FIG. 6A shows a cut-out image information table for cutting out images of a plurality of sizes. It is the table which put together the upper left coordinate and image size of the original image cut out when changing a zoom ratio continuously. In FIG. 6A, the size of the original image is 1280 × 960. The upper left coordinate of the image cut out when the original image is zoomed up by one step is (4, 3), and the image size is 1272 × 954. Further, when zooming up in two steps, the upper left coordinate of the image to be cut out is (8, 6). Similarly, upper left coordinates and image sizes for a plurality of stages of zoom-up are provided as a table. Each cut-out image is enlarged to an image of 1280 × 960 by the rectangular enlargement algorithm shown in FIGS. 5 (A) and 5 (B).
[0039]
FIG. 6B shows an enlarged image of the cut out image. When the upper left coordinate of the cut out image is (320, 240) and the image size is 640 × 480, the image is enlarged to a 1280 × 960 image by the rectangular enlargement algorithm.
[0040]
As shown in FIG. 6B, the center of the image is zoomed up to obtain a plurality of zoom images, but a zoom image is obtained while moving the zoom-up center at a predetermined interval from the center of the image in an arbitrary direction. It is also possible. In addition, a plurality of zoom images have been created from the original image, but a zoom image may be created from an image that was previously cut out and enlarged in a rectangular shape. In this case, since the upper left coordinate and the image size of the clipped image can be made constant, an information table of the clipped image as shown in FIG. 6A becomes unnecessary.
[0041]
Furthermore, in order to obtain a smooth zoom composite image, the cut-out image size of the cut-out image information table was finely set, but by setting the cut-out image size coarsely, a completely new zoom composite image that cannot be obtained with optical zoom Can be obtained.
[0042]
FIG. 7 illustrates a method for combining enlarged images. The image data has three RGB components and is composed of a plurality of pixels. Each RGB component has a value of 0 to 255 (8 bits). FIG. 7 illustrates the synthesis of one of the RGB components. An image 17 and an image 18 are images of a certain component to be synthesized, and an image 19 is an image of a certain component after synthesis. Focusing on the upper left pixel, image 17 is 11 and image 18 is 8, and simply adding it becomes 19 in image 19. In the pixel on the right side, the image 17 is 9 and the image 18 is 7, and when simply added, the image 19 becomes 16. Although not shown in FIG. 7, if the result of addition exceeds 255, a value of 255 or more is reduced to 255. The RGB components can be similarly synthesized.
[0043]
FIG. 8 shows a flowchart of zoom composition processing. Although not shown in the flowchart, first, the target zoom-up size is designated as x5, x10, and x20 by the various key units 13. When shooting is performed by pressing the shutter key 12 in the zoom composite shooting mode, first, an image captured from the CCD 1 is used by using a part of the image memory 4 as an original image buffer and stored therein (S36). When an image is taken during zoom composition processing, the original image buffer always stores an image captured from the CCD 1. A plurality of zoom images are created using these images as original images. At the same time, a part of the image memory 4 is used as a composite image buffer, and the captured image is also stored in the composite image buffer (setting of an initial image) (S37). The composite image buffer stores a composite image up to the previous zoom ratio during zoom composition processing, and is composed with the image in the composite image buffer each time an image with a new zoom ratio is created. When the initial image is set in the composite image buffer, the image in the original image buffer is cut out according to the table described with reference to FIG. 6A (S38). The cut-out image is enlarged to the same size as the original image by performing a rectangle enlargement process (S39). The rectangularly enlarged image is combined with the image in the combined image buffer and stored again in the combined image buffer (S40). The processing from S38 is repeated until zooming up to the target size is completed (S41). When zooming up to the target size is completed, the image is stored in the recording medium 10 as a zoom composite image.
[0044]
A plurality of rectangles that continuously change in size are cut out from one original image shot as described above, and after the plurality of cut out rectangles are enlarged to the same size as the original image, these combined images are obtained. As a result, it is possible to easily obtain an image equivalent to an image photographed by the zoom between exposures using the optical zoom. Further, since it is sufficient to shoot one image normally, there is no need to fix the camera with a tripod as in the case of using the optical zoom, and it is possible to shoot by holding the camera by hand.
[0045]
(Second Embodiment)
The rectangular enlargement algorithm of the zoom composition processing described so far uses an arbitrary pixel constituting the enlarged image by selecting one pixel closest to the ideal position from the pixels constituting the image before enlargement and leaving the value as it is. The method of copying was used. Next, a description will be given of a method for copying arbitrary values of arbitrary pixels constituting an image after enlargement after weighting a plurality of pixels around the ideal position constituting the image before enlargement. 9A to 10B show a rectangular enlargement algorithm using weighting. Since the operation of the digital camera 100 other than the rectangular enlargement algorithm in the zoom composition processing, the information table of the cut-out image, the zoom composition processing, and the like are the same as those in the first embodiment already described, the description thereof is omitted.
[0046]
FIG. 9A to FIG. 9C are line segment extension algorithms using weighting. As shown in FIG. 9A, a line segment S composed of (n + 1) dots from S0 to SN is extended to a line segment D composed of (m + 1) dots from D0 to DM using weighting. As shown in FIG. 9B, first, the start point S0 of the line segment S is copied as it is to the start point D0 of the line segment D. Next, when the line segment D increases by 1 from D0 to D1, the ideal pixel of the line segment S is located at a position (circle mark) from S0 between S0 (△ mark) and S1 (▽ mark). . However, in actuality, there is no pixel at this position, so the calculation is performed from the values of the pixels S1 and S2. Since the position from S1 to (mn) / m is from S0 to n / m, the value of the pixel D1 can be obtained by the equation D1 = S0 * (mn) / m + S1 * n / m.
[0047]
Further, when the line segment D increases by 1 from D1 to D2, the ideal pixel of the line segment S is at a position of 2 * n / m from S0. At this time, as shown in FIG. 9B, when 2 * n / m is 1 or more, the pixels of the line segment S used for interpolation are S1 (Δ mark) and S2 (▽ mark), and an ideal line It can be said that the pixel of the minute S is located at the position (◯ mark) from S1 to (2 * n / m) -1. The value of the pixel D2 is obtained by the following equation: D2 = S1 * 2 (mn) / m + S2 * (2n-m) / m.
[0048]
When these are represented by general formulas, if the pixel of the line segment S used for interpolation is S (Δ mark) and S ′ (▽ mark), the error between the pixel S (Δ mark) and the ideal position (◯ mark) is The initial value of the error is 0 and the error is E _{i + 1} = E _i It is determined by + n / m ≧ 1. (If this condition is met, E _{i + 1} = E _i -1) E _{i + 1} = E _i If + n / m ≧ 1, 1 is subtracted from E and the pixels S and S ′ used for interpolation are updated. From S (Δ mark) and S ′ (▽ mark), the value of the pixel D is D = S * (1−E) + S ′ * E.
[0049]
In the flowchart of FIG. 9C, in order to make it easy for the computer to process the operation described above, the error determination part is realized by integer arithmetic. In the above description, the initial value of the error is set to 0, and it is determined whether or not it is 1 or more when n / m is added. First, the initial value of the error is set to −1 and n / m is added. It was made to discriminate depending on whether it was 0 or more. The initial value of the error is -1 and the error is E _{i + 1} = E _i It is determined by + n / m ≧ 0. (If this condition is met, E _{i + 1} = E _i -1) and D = S * (-E) + S '* (E + 1). Next, a change is made so that the error is determined by multiplying the error by m. Then, the initial value of the error is −m, and the error is E _{i + 1} * M = E _i * M + n ≧ 0. (If this condition is met, E _{i + 1} = E _i −m) and D = S * (− E) / m + S ′ * (E + m) / m.
[0050]
The operation of the above algorithm will be described in detail with reference to the flowchart of FIG. First, the pixel D of the extension line segment D is set to D0, the first pixel S used for interpolation of the extension line segment S is set to S0, and the second pixel S ′ used for interpolation is set to S1. (S42). Next, the initial value of the error E of the line segment S is set to -m (S43). The RGB value of the transfer destination pixel D is obtained using D = S * (− E) / m + S ′ * (E + m) / m for each RGB component (S44). N is added to the error E (S45) to determine whether or not the error E is greater than 0. If the error E is less than or equal to 0, the error E is left as it is. If the error E is greater than 0 (S46), the error E Then, m is subtracted from S (S47), and at the same time, the pixels S and S 'of the line segment S as the extension source are incremented (next pixel is selected) (S48). Next, the pixel D of the extension line segment D is incremented (the next pixel is selected) (S49). The above processing is repeated until the processing of all the pixels D is completed until the pixels of D exceed DM (S50).
[0051]
Next, in FIG. 10A and FIG. 10B, a rectangular enlargement algorithm capable of enlarging a rectangle using the line segment expansion algorithm using the weighting shown in FIGS. 9A to 9C. Indicates. As shown in FIG. 10A, a rectangle composed of horizontal (nx + 1) dots and vertical (ny + 1) dots is expanded into a rectangle composed of horizontal (mx + 1) dots and vertical (my + 1) dots. In both the horizontal (X) direction and the vertical (Y) direction, the line segment expansion algorithm described in FIGS. 9A to 9C is used to change the expansion destination rectangle D from the four pixels of the expansion source rectangle S. One pixel value is calculated.
[0052]
The rectangle enlargement algorithm will be described with reference to the flowchart of FIG. First, the Y coordinate YD of the pixel D of the extension destination rectangle D is YD0, the first Y coordinate YS used for interpolation of the extension source rectangle S is YS0, and the second Y coordinate YS ′ used for interpolation is set. Initial setting is made to YS1 (S51). The initial value of the error EY in the Y direction is set to -my (S52). Thereafter, processing in the X direction is performed. (Shaded portion) In this shaded portion (S53 to S61), only the X coordinate is updated while the Y coordinate is fixed. First, the X coordinate XD of the image D of the extension destination rectangle D is XD0, the first X coordinate XS used for interpolation of the extension source rectangle S is XS0, and the second X coordinate XS ′ used for interpolation is set. Initial setting is made to XS1 (S53). Next, the initial value of the error EX in the X direction is set to -mx (S54). Four pixels (XS, YS) (XS ', YS) (XS, YS') (XS) of the rectangle S used for interpolation from the two Y coordinates YS, YS 'and the two X coordinates XS, XS' at this time ', YS') is determined. The interpolation method will be described with reference to FIG. 11. From these four values, the RGB values of one point (XD, YD) on the extension destination rectangle D are obtained (S55). Next, nx is added to the error EX (S56).
It is determined whether or not the error EX is greater than 0. If the error EX is 0 or less, the process exits. If the error EX is greater than 0 (S57), mx is subtracted from the error EX (S58). The coordinates XS and XS ′ are incremented (S59). Next, the X coordinate XD of the expansion destination is incremented (next pixel is selected) (S60). The above processing is repeated until the processing of all the X coordinates is completed until the pixel of XD exceeds XDM (S61). Next, ny is added to the error EY (S62). It is determined whether or not the error EY is greater than 0. If the error EY is less than or equal to 0, the error EY is left. If the error EY is greater than 0 (S63), my is subtracted from the error EY (S64). The coordinates YS and YS ′ are incremented (S65). Next, the Y coordinate YD of the extension destination is incremented (the next line is selected) (S66). Until the YD line exceeds YDM (S67), the above process is repeated until all the Y coordinate processes are completed.
[0053]
FIG. 11 illustrates an image interpolation method using weighting and a method of interpolating one pixel of the expansion destination from four pixels of the expansion source. By the rectangular enlargement algorithm shown in FIGS. 10A and 10B, the four pixels S1 (XS, YS), S2 (XS ′, YS), S3 (XS, YS ′), S4 (XS) of the expansion source ', YS') is determined. Further, the weighting values in both the X direction and the Y direction are S * (− EX) / mx + S ′ * (EX + mx) / mx in the X direction, and S * (− EY) / my + S ′ * (EY + my) in the Y direction. Therefore, the RGB value to be calculated is ((S1 * (− EX) / mx + S2 * (EX + mx) / mx) * (− EY) / my) + ((S3 * (− EX) / mx + S4 * (EX + mx) / mx) ) * (EY + my) / my). What is necessary is just to calculate from this formula for each of RGB.
[0054]
In this way, interpolation by weighting is performed on four pixels of the decompression source image to determine the pixel value of the decompressed image, so that a clear image can be obtained. In addition, as a method of enlarging the rectangle, interpolation by weighting is performed in both the vertical direction and the horizontal direction, but interpolation by weighting is performed only in either the vertical direction or the horizontal direction, and the other one is shown in FIG. The normal rectangular enlargement algorithm shown in 5 (B) may be used. As a result, the burden on the CPU 9 can be reduced while maintaining a certain level of image quality.
[0055]
(Third embodiment)
Next, in FIG. 12, zoom composition is performed using the rectangular enlargement algorithm shown in FIGS. 5A and 5B and the rectangular enlargement algorithm based on weighting shown in FIGS. 10A and 10B. Processing will be described.
[0056]
Although not shown in the flowchart of FIG. 12, first, the target zoom-up size is designated by various key units 13 such as x5, x10, and x20. When shooting is performed by pressing the shutter key 12 in the zoom composite shooting mode, first, an image captured from the CCD 1 is used as a part of the image memory 4 as an original image buffer and stored therein (S68). When an image is taken during zoom composition processing, the original image buffer always stores an image captured from the CCD 1. A plurality of zoom images are created using these images as original images. At the same time, a part of the image memory 4 is used as a composite image buffer, and the captured image is also stored in the composite image buffer (S69). Next, the image in the original image buffer is cut out according to the table of FIG. 6A (S70). If the image has the smallest zoom ratio (first image) or the largest zoom ratio (last image) (S71), rectangular enlargement by weighting shown in FIGS. 10A and 10B is performed. Processing is performed using an algorithm (S73), and if the image is any other image, processing is performed using the rectangular enlargement algorithm shown in FIGS. 5A and 5B in which weighting is not performed (S72). The rectangularly enlarged image is synthesized with the synthesized image buffer image (S74). The processing from S70 is repeated until the zoom-in to the target size is completed (S75). When zooming up to the target size is completed, the image is stored in the recording medium 10 as a zoom composite image.
[0057]
As described above, by properly using the two types of rectangular enlargement algorithms, the images at both ends of the zoom start and the zoom end become clear, and it becomes possible to reduce the load on the CPU in a dynamic image that fills between them. . In addition to the zoom start and zoom end images, the zoom composite image can be improved in quality by inserting a zoom image using a rectangular enlargement algorithm that performs weighting at regular intervals from among the zoom images constituting the gap. Is possible.
[0058]
(Fourth embodiment)
In the first to third embodiments described above, the zoom composition processing is performed using the image captured when the shutter key 12 is pressed as the original image, but the fourth embodiment. As an example, zoom composition processing is performed using a through image before the shutter key 12 is pressed as an original image. Since the operation of the digital camera 100, the information table of the cut-out image, and the rectangular enlargement algorithm are the same as those in the first to third embodiments already described, description thereof is omitted.
[0059]
In this embodiment, first, the target zoom-up size is designated by the various key sections 13 as x5, x10, and x20. In the zoom composition shooting mode, zoom composition processing is performed in a state where the image data taken from the CCD 1 is displayed as a through image on the display unit 14. That is, after the shutter key 12 is operated in the first to third embodiments, the same processing as the zoom composition processing performed at the stage of processing for capturing and storing is performed before the shutter key 12 is operated. This is performed at any time in the preparation state, and as a result, the zoomed image is output to the video output unit 7 and output to the display unit 14 at any time. Thus, the zoom composite image is always displayed on the display unit 14 in the imaging preparation state, and the user presses the shutter key 12 while confirming the zoom composite image to be captured on the display unit 14. When the shutter key 12 is pressed, the zoom composite image displayed on the display unit 14 is stored in the recording medium 10 at that time. Needless to say, at this time, as in the first to third embodiments, an image captured by pressing the shutter key 12 may be used as the original image for the synthesis process.
[0060]
As described above, since zoom composition processing is performed and displayed on the display unit 14 at the time of through image display before the shutter key 12 is pressed, the user can take a picture while viewing the zoom composite image. Accordingly, since it is possible to select whether or not to shoot when the zoom composite image is viewed, it is possible to avoid useless shooting and recording and easily obtain an intended image.
[0061]
【The invention's effect】
As described above, according to the first aspect of the present invention, an image having a plurality of sizes is cut out from a single image, enlarged to the same size as the image before cutting, and then combined. Therefore, an image equivalent to the zoom between exposures using the optical zoom can be obtained without taking time and effort.
[0062]
According to the second aspect of the present invention, since the captured image is recorded and a plurality of images are cut out based on the information of the cut-out size, there is no need to fix the camera body to a tripod, and zoom between exposures in hand-held shooting Equivalent effects can be obtained.
[0063]
According to the third aspect of the present invention, it is possible to enlarge and synthesize an image in a short time without imposing a burden on the CPU by using a rectangle enlargement algorithm capable of enlarging a rectangle with only a simple integer operation. .
[0064]
According to the fourth aspect of the present invention, a clear image can be obtained by interpolating a plurality of pixels by weighting in the rectangular enlargement algorithm.
[0065]
According to the invention of claim 5, the original image is taken from the image sensor. Because it is a through image You can shoot while viewing the actual zoom composite image.
[0066]
Further, according to the invention of claim 6, since an image having a plurality of sizes is cut out from a single image and enlarged to the same size as the image before the cut out, these images are combined. An image equivalent to the exposure zoom using the optical zoom can be obtained without taking time and effort.
[0067]
Further, according to the invention of claim 7, since a plurality of sizes of images are cut out from one image and enlarged to the same size as the image before cutting out, the program is composed to synthesize these images. An image equivalent to the exposure zoom using the optical zoom can be obtained without taking time and effort.
[Brief description of the drawings]
FIG. 1 is a block diagram of an apparatus according to the present invention.
FIG. 2 is a diagram illustrating the principle of zoom composition.
FIG. 3A is a coordinate diagram showing a line segment generation algorithm.
(B) is a flowchart showing a line segment generation algorithm.
FIG. 4A is a structural diagram showing a line segment decompression algorithm.
(B) is a coordinate diagram showing a line segment extension algorithm.
(C) is a flowchart showing a minute extension algorithm.
FIG. 5A is a structural diagram showing a rectangular enlargement algorithm.
(B) is a flowchart showing a rectangular enlargement algorithm.
FIG. 6A is a diagram illustrating an information table of clipped images.
(B) is the figure which showed the image of the cut-out image.
FIG. 7 is a diagram illustrating a method for synthesizing an enlarged image.
FIG. 8 is a flowchart of zoom composition processing.
FIG. 9A is a structural diagram showing a line segment expansion algorithm by weighting.
(B) is a coordinate diagram showing a line segment extension algorithm by weighting.
(C) is a flowchart showing a line segment expansion algorithm by weighting.
FIG. 10A is a structural diagram showing a rectangular enlargement algorithm by weighting.
(B) is a flowchart showing a rectangular enlargement algorithm by weighting.
FIG. 11 is a diagram showing image interpolation by weighting.
FIG. 12 is a flowchart of zoom composition processing by two types of rectangular enlargement algorithms.
[Explanation of symbols]
1 CCD
2 CCD controller
3 YUV processor
4 Image memory
5 Memory controller
6 Video encoder
7 Video output section
8 Data communication department
9 CPU
10 Recording media
11 Key processing part
100 digital camera

Claims (7)

Image cutting means for cutting out images of a plurality of different sizes from the original image, with one image as the original image,
Image enlargement means for enlarging each of the plurality of images cut out by the image cutout means to the same size as the original image;
Image combining means for adding pixel values of corresponding pixels of a plurality of images of the same size enlarged by the image enlargement means;
An image synthesizing apparatus. Imaging means for imaging a subject;
Recording means for recording the subject image generated by the imaging means as image data;
Information recording means for storing information on the size to be cut out;
The image synthesizing device according to claim 1.   2. The image synthesizing apparatus according to claim 1, wherein the image enlarging means is performed based on a rectangular enlarging algorithm, and selects one pixel from pixels constituting the image before enlarging and duplicates the value.   2. The image synthesizing apparatus according to claim 1, wherein the image enlarging means is performed based on a rectangular enlarging algorithm, weights a plurality of pixels constituting an image before enlargement, and duplicates the values.   The image composition apparatus according to claim 1, wherein the original image is a through image captured from an image sensor. Image cutout processing for cutting out images of a plurality of different sizes from the original image using one image as the original image;
Image enlargement processing for enlarging each of the plurality of images cut out by the image cutout means to the same size as the original image;
An image composition method comprising image composition processing for adding pixel values of corresponding pixels of a plurality of images of the same size enlarged by the image enlargement processing. Image cutting means for cutting out images of a plurality of different sizes from the original image, with one image as the original image,
Image enlargement means for enlarging each of the plurality of images cut out by the image cutout means to the same size as the original image;
An image composition program for realizing image composition means for adding pixel values of corresponding pixels of a plurality of images of the same size enlarged by the image enlargement means.

Images