JP2010034964A - Image composition apparatus, image composition method and image composition program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image composition apparatus capable of reducing the deterioration of image quality caused by variation in a plurality of imaging devices, and composing reference image data into a high-resolution image at high speed. <P>SOLUTION: The image composition apparatus comprises: an image input means for inputting a plurality of image data items obtained by imaging the same subject; an image composition means for composing the plurality of image data items inputted by the image input means to produce composite image data; a composition processing range-selecting means for selecting one of the plurality of image data items as reference image data, comparing image quality of each divided region between the reference image data and the composite image data and selecting a composition processing range including the divided region where the image quality of the composite image data is improved; and a high-resolution image producing means for producing a resolution-improved image by composing the reference image data outside the composition processing range to the composite image data within the composition processing range selected by the composition processing range selecting means. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image composition device, an image composition method, and an image composition program.

  Recently, the appearance of a high-definition camera is desired in order to obtain a high-definition image. However, a high-resolution imaging device used for a high-definition camera has a problem that it is expensive because of its difficulty in manufacturing. Therefore, an attempt has been made to obtain a high-resolution image by using a multi-lens camera using a plurality of inexpensive low-resolution imaging devices and synthesizing a plurality of images obtained by the respective imaging devices. FIG. 8 is a diagram illustrating a procedure for synthesizing a high-resolution image using a multi-lens camera. In FIG. 8, the multi-lens camera constituted by the three cameras 51, 52, 53 is directed in substantially the same direction, and images the subject 4 that is sufficiently separated from each other to obtain three images P1, P2, P3. Output. The three cameras 51, 52 and 53 have the same angle of view, but the optical axes are adjusted so as to be shifted from each other at each pixel level. The image synthesizing unit 6 generates a new pixel between the pixels using the difference in pixel level of each of the three captured images P1, P2, and P3, and increases the apparent number of pixels to increase the number of pixels. By increasing the resolution, the synthesized image 7 having a higher resolution is output.

As shown in FIG. 8, a technique described in Non-Patent Document 1 or the like is known as a prior art for obtaining a high-resolution image by combining a plurality of images having a pixel level shift and combining the plurality of images. It has been.
Superresolution images reconstructed from aliased images (P.Vandewalle et al., 2003)

  By the way, since a plurality of imaging devices constituting the multi-lens camera have variations in attachment of optical lenses and photoelectric conversion devices, there is a difference in distortion generated in an image captured by each imaging device. FIG. 9 is an explanatory diagram showing the state of distortion generated in images obtained by two imaging devices. FIG. 9A shows an image with no distortion, and FIG. 9B shows an image with distortion around. As described above, when there is a difference in the distortion generated in the images obtained by the two imaging devices, when the synthesis process is executed using a high resolution technique such as Non-Patent Document 1, as illustrated in FIG. Furthermore, the composite image is distorted. In general, the distortion caused by the lens increases as the distance from the center (optical axis) increases, and the variation increases. Therefore, as shown in FIG. A shift of one pixel or more occurs in the portion, and not only high resolution cannot be achieved but also the image quality deteriorates. In particular, in the peripheral portion of the image (shown enlarged in FIG. 9 (d)), a mismatch in the coordinate system occurs, and the image quality of the image obtained by the image synthesis performed for achieving high resolution is deteriorated conversely. There is a problem of end up.

  The present invention has been made in view of such circumstances, and an image composition apparatus and an image composition method capable of reducing high-quality images at high speed while reducing image quality deterioration due to variations in distortion of a plurality of imaging devices. An object of the present invention is to provide an image composition program.

  The present invention includes an image input unit that inputs a plurality of image data obtained by imaging the same subject, an image combining unit that generates combined image data by combining a plurality of image data input by the image input unit, One of the plurality of image data is selected as the reference image data, and the image quality of the composite image data is improved by comparing the image quality of the divided areas with respect to each of the reference image data and the composite image data. A synthesis processing range selection unit that selects a synthesis processing range that includes the divided area, and the synthesis image data in the synthesis processing range selected by the synthesis processing range selection unit is out of the synthesis processing range. And a high-resolution image generation means for generating a high-resolution image by synthesizing the reference image data.

  The present invention is characterized in that the divided area is an area excluding the periphery of the image.

  According to the present invention, the composition of the plurality of image data is performed based on a pixel shift amount, and a new pixel is generated by multiplying a weight value by a distance between pixels on the superimposed coordinate system. Features.

  The present invention is characterized in that the image is a monochrome image or a color image.

  The present invention is an image composition method in an image composition apparatus comprising an image input means, an image composition means, a composition processing range selection means, and a high resolution image generation means, wherein the image input means detects the same subject. An image input step for inputting a plurality of captured image data; an image combining step for generating combined image data by combining the plurality of image data input by the image combining unit; and the combining process A range selection unit selects one of the plurality of image data as reference image data, compares the image quality of divided areas with respect to each of the reference image data and the composite image data, and outputs the composite image A synthesis processing range selection step for selecting a synthesis processing range including a divided area in which the image quality of data is improved, and a high-resolution image generation means Generating a high-resolution image by synthesizing the reference image data outside the synthesis processing range with the synthesized image data in the synthesis processing range selected by the synthesis processing range selection step. And a step.

  The present invention is an image composition program for causing a computer on an image composition apparatus that outputs a high resolution image by combining a plurality of image data to perform image composition processing, and a plurality of images obtained by capturing the same subject. An image input step for inputting data; an image combining step for generating composite image data by combining a plurality of image data input in the image input step; and one of the plurality of image data as reference image data Comparing the image quality of the divided areas with respect to each of the reference image data and the composite image data, and selecting a composite processing range including a divided area in which the image quality of the composite image data is improved. The synthesis processing range selection step to be selected, and the synthesis of the synthesis processing range selected by the synthesis processing range selection step To the image data, characterized in that to perform a high-resolution image generating step of generating a high resolution image by synthesizing the reference image data outside the range the composition processing in a computer.

  According to the present invention, in an image composition device that combines a plurality of images to obtain a high-resolution image, when combining the images, the pixels in the region that contributes to the high resolution are selected and image composition is performed. Since pixels that do not contribute to higher resolution are not subjected to image synthesis, an effect of reducing image quality degradation due to variations in imaging devices can be obtained. In addition, when compositing images, only the pixels in the region that contribute to higher resolution are selected and image compositing is performed, so the amount of image compositing processing can be reduced, and high-resolution images can be created at high speed. The effect that it can produce | generate can also be acquired.

  Hereinafter, an image composition device according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the embodiment. In this figure, reference numerals 11 and 12 denote cameras provided with an imaging device constituted by an optical lens, a photoelectric conversion element, or the like. Here, an example in which a multi-lens camera is configured by two cameras 11 and 12 is illustrated, but three or more cameras may be included in the multi-lens camera. Reference numeral 2 denotes an image synthesizing apparatus that inputs two sets of image data output from a plurality of cameras 11 and 12, synthesizes two pieces of image data, and outputs high-resolution image data.

  Reference numeral 21 denotes an image input unit that inputs image data output from the two cameras 11 and 12 and holds the image data of each of the two systems. Reference numeral 22 denotes an image combining unit that combines two pieces of image data held in the image input unit 21 and outputs one piece of combined image data. Reference numeral 23 denotes a synthesis processing range selection unit that selects a range (region) to be synthesized in the synthesis target image data based on the synthesized image data synthesized by the image synthesis unit 22. Reference numeral 24 synthesizes two sets of image data for only the range to be subjected to the synthesis process selected by the synthesis process range selection unit 23, and uses the input image data except for the range to be subjected to the synthesis process. This is a high-resolution image generation unit that generates and outputs a high-resolution image.

  In the following description, it is assumed that two images captured by the two cameras 11 and 12 are combined, and the size of the image obtained by the cameras 11 and 12 is VGA size (width 640 pixels, height 480 pixels). In the following description, the width and height of the combined high-resolution image are quad VGA sizes (width 1280 pixels, height 960 pixels).

  Next, the operation of the image composition apparatus 2 shown in FIG. 1 will be described with reference to FIG. First, the image input unit 21 inputs image data (data composed of a pixel value sequence of each pixel) of two systems captured by the cameras 11 and 12 with substantially the same angle of view (step S1), and two pieces of image data Is stored inside (stored in an internal storage device). Subsequently, the image composition unit 22 reads the two pieces of image data held in the image input unit 21 and sets one of the two pieces of image data as the image data of the reference image. As an example, one of the cameras 11 and 12 is determined in advance as a camera that captures a reference image, and image data output by the camera that captures the reference image is used as a reference image. Here, description will be made assuming that the camera 11 is a camera that captures a reference image.

  Next, the image composition unit 22 performs an XY shift that minimizes the sum of differences between the pixel values of the reference image data (camera 11 image data) and the other image data (camera 12 image data). An amount is obtained (step S2). Here, the XY shift amount is a value (unit: pixel) indicating how much the other image has moved in the XY (vertical and horizontal) directions with respect to the reference image. For example, the amount of XY shift is changed by 0.1 pixel in the range of (−10, −10) to (+10, +10), the total value of the difference (absolute value) of the pixel values is obtained, and this total value is minimized. The XY shift amount is calculated as follows. Here, the 0.1 pixel is a pixel located at a point obtained by dividing an actual pixel into 10 equal parts, and the pixel value is interpolated by a bilinear method, a bicubic method, or the like from the surrounding actual pixel value. Can be obtained.

  Next, the image composition unit 22 converts the coordinate system into a quad VGA size coordinate system by doubling the coordinate system of the image data of two sheets (the reference image and the other image). Then, the image composition unit 22 superimposes the enlarged coordinate systems according to the previously obtained XY shift amount (step S3). A coordinate system obtained by enlarging the reference image data is used as a coordinate system for a high-resolution image to be newly generated.

  Next, the image composition unit 22 multiplies the weight value based on the distance from the pixel value of the pixel on the coordinate system superimposed in step S3 to obtain each pixel value of the coordinate system of the high-resolution image to be generated ( Step S4). Through the processing operations in steps S2 to S4, the combined processing is performed on the entire range of the image data of two sheets (the reference image and the other image) to generate combined image data.

  Here, the processing operation of step S4 shown in FIG. 2 will be described in detail with reference to FIG. FIG. 3 is a diagram illustrating an operation of obtaining each pixel value of the coordinate system of the high-resolution image to be generated by multiplying the weight by the distance with the pixel value of the pixel on the coordinate system superimposed in step S3. Here, it is assumed that the image 1 is a reference image and the image 2 is the other image. FIG. 3A shows a state in which the two coordinate systems of the image 1 and the image 2 are overlapped according to the XY shift amount (X shift amount and Y shift amount). The circles in the figure are the pixels of the image 1, and the triangles in the figure are the pixels of the image 2. The resolution is increased by generating new pixels between adjacent pixels of the image 1.

  FIG. 3B shows a processing operation for generating a new pixel (● mark). The pixel value of the pixel to be newly generated is obtained by multiplying the pixel value of the image 1 and the pixel of the image value 2 around the target pixel by the weight value wi (i is 0 to 8), respectively. The weight value wi is a value determined by the distance between the reference pixel and the generated pixel, and is equivalent to an interpolation calculation method such as a bilinear method or a bicubic method. By combining the pixel of the image 1 and the newly generated pixel, an image with a high resolution of twice the resolution can be obtained. When the number of images used for composition is three or more, it is only necessary to calculate wi for each image pixel and perform interpolation calculation to obtain a pixel value. This processing requires that the pixel shift of each image used for composition is smaller than one pixel. This is because when the shift is one pixel or more, completely different pixels are used for synthesis, and an incorrect pixel value is obtained by performing interpolation, resulting in degradation of image quality.

  Returning to FIG. 2, next, the composition processing range selection unit 23 inputs the composite image generated by the image composition unit 22 in steps S <b> 2 to S <b> 4, and selects the composition processing range according to the image quality deterioration degree of the composite image. (Step S5). The composition processing range here is a rectangular area having a predetermined size, and the center of which coincides with the center of the image. With reference to FIG. 4 and FIG. 5, how to obtain this rectangular area will be described. As shown in FIG. 4, the reference image before composition in (A) and the composite image after composition in (B) are divided into the same number of divisions (in the example shown in FIG. 4, 8 horizontal divisions × 6 vertical divisions) Divided into grid-like blocks (divided regions), the spatial frequency of the pixel value is determined for each grid block.

  In FIG. 4, (a) to (d) represent graphs of the spatial frequency distribution of the lattice blocks, where the horizontal axis is the spatial frequency and the vertical axis is the energy. This high-definition lattice block has a higher frequency component than the pre-combination lattice block, and the non-high-definition block has no higher frequency component than the pre-synthesis lattice block. Based on the analysis result, it is possible to determine whether or not each lattice block has been refined. A determination is made in this way to mark whether or not each lattice block has been refined. In FIG. 4C, a high-definition lattice block is indicated by a circle, and a non-high-definition lattice block is indicated by a cross.

  FIG. 5 is a diagram showing a procedure for selecting a rectangular area that is a synthesis processing range after all lattice blocks are marked with ○ (high definition) or × (not high definition). is there. FIG. 5A is a diagram showing the determination result. The rectangular area, which is the compositing range, is the size of a grid block unit so as to include the center grid block, is a rectangle that is as large as possible with a size greater than or equal to a predetermined size, and the ratio of the ○ mark in the rectangle is as high as possible A rectangular area (a divided area composed of a plurality of lattice blocks) is selected. As an example, an arbitrary number of various rectangular areas excluding the peripheral portion are prepared, and among them, the one having the largest ratio of the mark “◯” in the rectangular area is selected. In the example shown in FIG. 5, the area shown in (b) is higher in the area shown in (b) than the area shown in (a) or the area shown in (c). select. The composition processing range selection unit 23 outputs information on the rectangular area that is the composition processing range selected here to the high-resolution image generation unit 24.

  Returning to FIG. 2, the high resolution image generation unit 24 then reads out the new two pieces of image data held in the image input unit 21, performs synthesis processing, and generates and outputs a high resolution composite image. (Step S6). At this time, the high-resolution image generation unit 24 performs image data synthesis processing using the information on the synthesis processing range output from the synthesis processing range selection unit 23. Here, with reference to FIG. 6, an operation of performing image data composition processing using information of the composition processing range will be described. FIG. 6 is a diagram illustrating an operation of performing image data combining processing using information on a combining processing range.

  6A and 6B are image data obtained by imaging with the cameras 11 and 12, respectively, FIG. 6A is a reference image captured with the camera 11, and FIG. It is the other image imaged by. In the two pieces of image data (a) and (b) shown in FIG. 6, the hatched portion is an area used for image composition. In other words, the reference image (camera 11 image) is the entire region to be combined, and the other image (camera 12 image) is the region specified by the combination processing range output from the combination processing range selection unit 23. It becomes the synthesis target area. The high resolution image generation unit 24 performs image composition processing using the pixel values in the two composition target areas. At this time, an image (partial composite image data) obtained by combining the two images of the reference image and the other image is applied to the pixels within the composition processing range, and the reference image is enlarged for the pixels outside the composition processing range. Apply the processed image. That is, interpolation calculation is performed on the reference image, and the pixel value of the pixel required is obtained by enlarging the VGA size to the Quad VGA size. Alternatively, the same pixel value as that of the adjacent pixel is set as the pixel value of the required pixel.

  Thereby, the composite image data shown in FIG. 6C is obtained. The synthesized image data is increased in resolution by synthesizing two pieces of image data within the synthesis processing range selected by the synthesis processing range selection unit 23, and is synthesized outside the synthesis processing range. The image is obtained using the pixels of the reference image without any problem. As a result, it is possible to prevent the image quality from deteriorating by synthesizing two images having a difference in image distortion. Compared to the conventional image synthesizing method, the synthesized image to be output can be prevented. Image quality can be improved.

  Returning to FIG. 2, next, the high resolution image generation unit 24 determines whether or not to continue the synthesis process (step S7). If the process is continued, the process returns to step S6 to repeat the synthesis process. For example, the image composition device 2 includes an imaging button for turning on / off the operation, and the determination result of step S7 is YES (continuation) while the imaging button is on. If the imaging button is OFF, NO (end) is determined. As a result, since the image composition processing after selection of the composition processing range is performed only in the composition processing range within the selected composition processing range, the conventional method of repeating the composition processing of two pieces of image data every time is used. In comparison, the processing amount can be greatly reduced, the processing time can be shortened, and the processing load on the high-resolution image generation unit 24 can be reduced.

Next, a modified example of the image composition apparatus according to the above-described embodiment of the present invention will be described.
The image composition unit 22 and the high resolution image generation unit 24 shown in FIG. 1 may be any means that synthesizes a high resolution image using a plurality of images, and is not limited to the image composition method shown in Non-Patent Document 1. Absent. Any so-called super-resolution technique can be used. For example, a Lanchos method may be used as a weight calculation method for pixel interpolation, or an estimation operation called a normalized convolution may be used instead of interpolation. Many of the methods tend to increase the processing amount as the image quality improves, and an optimum method of high image quality or low processing amount may be used depending on the application.

  In addition, the imaging devices included in the cameras 11 and 12 may be monochrome or color. For example, in the case of a color camera equipped with an RGB color filter, each input image data is decomposed into R (red), G (green), and B (blue) components, and a plurality of image data is used for each. In addition to the basic method of combining resolution images, only the G component is combined using a plurality of images. For the R and B components, the reference image is used as it is, and each component is combined again to generate a high resolution image. You may make it obtain. Even when color image data is decomposed into Y (luminance), Cb (color difference), and Cr (color difference) components, similarly, only the Y component is combined using a plurality of images, and the Cb and Cr components are used as reference images. May be used as they are. This is because the human eye can distinguish luminance information better than color information, and an image obtained by this method is recognized as a high-resolution image as a whole without particularly increasing the resolution of R, B, Cr, and Cb. Because it is done. With this configuration, a color image can be obtained while reducing the processing amount.

  In the above description, the synthesis processing range selected by the synthesis processing range selection unit 23 has been described as a rectangle having a predetermined size from the center, but is not limited thereto. That is, it may be a rectangle starting from other than the center, may be a figure other than a rectangle, for example, a circle, or all the lattice blocks of the ○ mark used in the above description may be set as the synthesis processing range.

  Further, the composition processing range selection unit 23 is provided with means for detecting a characteristic image region such as a human face, and when a characteristic image region can be detected in the image, that region is used as the composition processing range. Also good. With reference to FIG. 7, for example, a process for detecting a face will be described. (A) shown in FIG. 7 is a binarized image in which an image including a human face is expressed in white and black. (B) is a matching pattern, which is also a binary image of white and black. The absolute value of the pixel value difference is obtained while moving the coordinates of the matching pattern of (b) vertically and horizontally in the image of (a). When the sum of the differences in the matching pattern falls below a predetermined value, it is recognized that a human face exists at the coordinates, and the human face can be detected. In addition to this method, the pattern can be detected with higher accuracy by enlarging / rotating / reducing the pattern or using matching using a multi-valued image. With this configuration, it is possible to select a range in which fine image quality is improved according to the application.

  The composition processing range selected by the composition processing range selection unit 23 may be a range outside an area having a certain size from the center. This synthesis processing range selection method takes into consideration the characteristics of the imaging device. An image obtained by imaging using a general optical lens has a relatively good image quality at the central portion (near the optical axis), and the image quality becomes worse at the periphery. For this reason, it is not necessary to improve the image quality in the central portion, and it can be applied to an imaging system in which the image quality needs to be improved in the peripheral portion.

  In the above description, the selection of the synthesis processing range performed by the synthesis processing range selection unit 23 is a processing operation performed every time the image synthesis apparatus 2 is activated. Information on the selected composition processing range is stored in the composition processing range storage means, and when the image composition apparatus 2 is activated next time, the information on the composition processing range stored in the composition processing range storage means is read. The high-resolution image generation unit 24 may use the image composition process. With this configuration, the image composition process can be started promptly when the image composition apparatus 2 is activated, so that the operability can be improved.

  Further, the information on the synthesis processing range once selected may be output as synthesis processing range data to a device outside the apparatus, and when used, the information on the synthesis processing range may be read from the external device and used. With this configuration, it is possible to save a plurality of composite processing range data that matches the imaging conditions, and select and reuse them according to the conditions, so that the composite processing can be performed even if the conditions change. In addition to being able to omit the range selection process, these data can be selected intentionally regardless of the conditions, and the result of the synthesis process using each data can be confirmed.

  In the above description, the method of examining the spatial frequency component of the synthesized image is used as the image quality degradation determination method used for selecting the synthesis processing range. However, the synthesis processing range selection unit 23 is further prepared in advance. Deterioration may be determined by inputting a reference image and comparing the reference image with the composite image obtained in steps S2 to S4. Specifically, the two images are divided into fine regions, and PSNR values (Peak Signal to Noise Ratio; a value representing the distortion ratio of the reference image and the image to be evaluated in logarithm) are obtained in each region. Then, a threshold value is determined using the average value of the obtained PSNR values, or an area having a PSNR value equal to or greater than a predetermined threshold value may be set as the synthesis processing range. With such a configuration, the determination of image quality deterioration is less affected by the image to be captured, and a more effective synthesis processing range can be selected.

  Further, the composition processing range selection unit 23 may be configured to process an image obtained by capturing a predetermined chart. The predetermined chart is, for example, a resolution chart or a mass chart. A resolution chart is a chart created so that the resolution capability of an imaging system can be measured by imaging the resolution chart. By taking a resolution chart under predetermined conditions and examining the resolution of the composite image, it is possible to examine in detail the deterioration of the image quality. A mass chart is a lattice or checkered chart. Image degradation can be examined by imaging a mass chart under predetermined conditions and examining the positions and distortions of intersections and lines of the lattice. In this way, with the method using a predetermined chart, since an actual subject is not imaged, the selection process of the synthesis processing range can be stabilized regardless of the conditions.

  Further, the synthesis processing range selection unit 23 may select a synthesis processing range from three or more images. In this case, the combination processing range selection process described above is performed for all image combinations, and the combination of images having the highest effect is set as the combination processing range. In this way, it is possible to select an appropriate synthesis processing range even if there are images that contribute significantly or do not contribute to the synthesis.

  Further, the synthesis processing range selection unit 23 may include a counter (counting unit) (not shown), counts each time processing is performed, and determines the synthesis processing range according to the count value. Specifically, the counter is a repetitive counter such as 0, 1 and the like. When the count value is 1, all the images other than the reference image are not selected for synthesis processing, that is, the reference image may be output as it is. Good. With this configuration, for example, when a plurality of moving images can be generated per second in the image composition device 2, the image quality is improved every other frame. When a sufficient number of such moving images are displayed per second, it is observed that the overall image quality is improved by the illusion. With this configuration, the processing amount can be reduced when the image composition device 2 is used as a moving image camera.

  Note that the parameters of the imaging devices provided in the cameras 11 and 12 may be changed in order to generate a higher quality image. The parameters here are in-focus, aperture, angle of view, etc., and when a variable lens is used, the parameters are also included. As a method of changing these, there is a method of performing imaging using all or a part of various combinations of parameters, combining each of them, and selecting the best parameter from them. In addition, if it is known how the parameter affects the image to be captured, for example, the aperture affects the brightness of the image, and if there is a means for measuring the effect from the image, it will be improved. Alternatively, a feedback control method of sequentially adjusting parameters may be used.

  As described above, in an image composition device that combines a plurality of images to obtain a high resolution image, when compositing the images, the pixels in the region that contributes to high resolution are selected and image composition is performed. Since pixels that do not contribute to higher resolution are not subjected to image synthesis, image quality degradation caused by variations in imaging devices can be reduced, image synthesis processing amount can be reduced, and high-resolution images can be generated at high speed. it can.

  A program for realizing the functions of the image composition unit 22, the composition processing range selection unit 23, and the high-resolution image generation unit 24 shown in FIG. 1 is recorded on a computer-readable recording medium and recorded on the recording medium. The image composition processing may be performed by causing the computer system incorporated in the image composition apparatus 2 to read and execute the program. Here, the “computer system” includes software such as an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  The image synthesizing apparatus according to the present invention can be applied to a multi-lens electronic camera that captures moving images and still images, and particularly, to an apparatus that synthesizes images captured by a plurality of imaging devices and synthesizes a high-definition image at high speed. It can be used.

It is a block diagram which shows the structure of one Embodiment of this invention. 3 is a flowchart illustrating an operation of the image composition device 2 illustrated in FIG. 1. It is explanatory drawing which shows the operation | movement which produces | generates a high resolution image by synthesize | combining a some image. It is explanatory drawing which shows the operation | movement which determines whether the synthesized image was highly defined. It is explanatory drawing which shows the operation | movement which selects a synthetic | combination process range. It is explanatory drawing which shows the operation | movement which synthesize | combines an image using a synthetic | combination process range. It is explanatory drawing which shows an example of the matching pattern used for a face detection process. It is explanatory drawing which shows the processing operation which produces | generates a high-resolution image using a multiview camera. It is explanatory drawing which shows an example of the image quality degradation by the image composition using the high resolution technique.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11, 12 ... Camera, 2 ... Image composition apparatus, 21 ... Image input part, 22 ... Image composition part, 23 ... Composition processing range selection part, 24 ... High-resolution image generation Part

Claims (6)

Image input means for inputting a plurality of image data obtained by imaging the same subject;
Image combining means for generating composite image data by combining a plurality of image data input by the image input means;
One of the plurality of image data is selected as reference image data, and the image quality of the composite image data is improved by comparing the image quality of divided areas with respect to each of the reference image data and the composite image data. A synthesis processing range selection means for selecting a synthesis processing range including the divided area,
High-resolution image generation means for generating a high-resolution image by combining the reference image data outside the synthesis processing range with the synthesized image data in the synthesis processing range selected by the synthesis processing range selection means An image composition device comprising:   The image synthesis apparatus according to claim 1, wherein the divided area is an area excluding the periphery of the image.   The synthesis of the plurality of image data is performed based on a shift amount of pixels, and a new pixel is generated by multiplying a weight value according to a distance between pixels on the superimposed coordinate system. Item 3. The image composition device according to Item 1 or 2.   The image synthesizing apparatus according to claim 1, wherein the image is a monochrome image or a color image. An image composition method in an image composition device comprising an image input means, an image composition means, a composition processing range selection means, and a high resolution image generation means,
An image input step in which the image input means inputs a plurality of image data obtained by imaging the same subject;
An image compositing step in which the image compositing means generates composite image data by compositing a plurality of image data input in the image input step;
The synthesis processing range selection means selects one of the plurality of image data as reference image data, compares the image quality of the divided areas with respect to each of the reference image data and the synthesized image data, A synthesis processing range selection step for selecting a synthesis processing range including a divided area where the image quality of the composite image data is improved;
High-resolution image generation means combines the reference image data outside the synthesis processing range with the synthesized image data in the synthesis processing range selected in the synthesis processing range selection step, thereby generating a high-resolution image. And a high-resolution image generation step for generating the image. An image composition program for causing a computer on an image composition apparatus that outputs a high resolution image by combining a plurality of image data to perform image composition processing,
An image input step of inputting a plurality of image data obtained by imaging the same subject;
An image compositing step for generating composite image data by compositing a plurality of image data input in the image input step;
One of the plurality of image data is selected as reference image data, and the image quality of the composite image data is improved by comparing the image quality of divided areas with respect to each of the reference image data and the composite image data. A synthesis processing range selection step for selecting a synthesis processing range including the divided area,
A high-resolution image generation step of generating a high-resolution image by combining the reference image data outside the combination processing range with the combined image data in the combination processing range selected by the combination processing range selection step. An image composition program characterized by causing a computer to perform.

Images