JP2010213166A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus and the like creating a high-resolution image with higher image quality from less low-resolution images. <P>SOLUTION: The image processing apparatus includes: a pixel vector generating means for calculating a pixel vector with respect to all pixels of a plurality of low-resolution images; an image overlapping means for overlapping the plurality of low-resolution images to create an accumulation image and deciding whether or not the pixel values of all the pixels has been determined; a pixel vector aligning means for aligning the pixel vectors by adding or deleting them for each of all pixels of the accumulation image; a first pixel value estimating means for estimating a pixel value of a blank pixel, that is decided as non-determined, using the pixel value determined pixels whose pixel values have been determined and the aligned pixel vectors; and a second pixel value estimating means for determining one of two diagonal directions of vertical and horizontal directions, in which the number of blank pixels disposed continuously in the above two diagonal directions is the most, for each of the pixel values of the blank pixels that can not be estimated by the first pixel value estimating means, and estimating the pixel values of the blank pixels based on the pixel value-determined pixels in both the directions vertical to the determined direction. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for performing super-resolution processing for creating a high-resolution image based on a plurality of low-resolution images.

  An image processing technique for reconstructing a plurality of low resolution images to create a high resolution image is called super-resolution processing, and many techniques have been developed conventionally. For example, typical super-resolution processing methods such as an ML (Maximum-likelihood) method, a MAP (Maximum A Posterior) method, and a POCS (Projection On to Convex Sets) method have been proposed. Further, as described in Patent Document 1, a high-resolution image generation method for generating one high-resolution image from a plurality of low-resolution images including positional deviation has been proposed.

JP 2008-109375 A

  In such super-resolution processing, the image quality of a high-resolution image to be created varies greatly according to the number of low-resolution images that are original images. The higher the number of low-resolution images to be used, such as 50, 60, and 100, the more high-quality high-resolution images that reproduce the subject more faithfully can be created. However, since a large number of images are handled, the amount of calculation increases enormously. On the other hand, if the number of low-resolution images to be used is as small as 5 or 10, the percentage of blank pixels remaining after superposition increases, and the error of the estimated pixel value increases, so The problem that it cannot be restored occurs.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing apparatus, an image processing method, and a program that can create a higher-quality high-resolution image from a smaller number of low-resolution images. .

  In order to achieve such an object, an image processing apparatus according to the present invention provides, as a first aspect, an image processing apparatus that creates a high resolution image based on pixel values in a plurality of low resolution images. For all pixels, a pixel vector generation means for calculating a pixel vector and a plurality of low resolution images are superimposed to create an accumulation image, and pixel values are determined for all the pixels in the accumulation image, or pixel values are set. The image superimposing means to be undetermined, the pixel vector organizing means for organizing the pixel vectors by addition or deletion based on the setting made in advance for every pixel in the accumulation image, and the pixel value by the image superimposing means Based on a preset condition using a pixel value determined pixel for which the pixel value is determined and a pixel vector organized by the pixel vector organizing means. A first pixel value estimating unit that estimates a pixel value of a blank pixel whose pixel value has not been determined by the image superimposing unit, and a blank pixel value that cannot be estimated by the first pixel value estimating unit. The direction with the largest number of pixels is determined from among the number of blank pixels lined up continuously in the vertical direction, the horizontal direction, and the two diagonal directions, and the pixel value determination pixels existing in both directions perpendicular to the determined direction are determined. And a second pixel value estimating means for estimating the pixel value of the blank pixel.

  In addition, as a second aspect, the image processing apparatus of the present invention is a pixel vector generation unit that calculates a pixel vector of a pixel value fixed pixel for an image in which pixel value fixed pixels with fixed pixel values exist at random. A first pixel value estimating unit that estimates a pixel value of a blank pixel whose pixel value is undefined based on a preset condition using a pixel value determined pixel and a pixel vector; For each pixel value of the blank pixel that could not be estimated by the pixel value estimation means, determine the direction with the largest number of pixels among the number of blank pixels lined up continuously in the vertical direction, the horizontal direction, and the two diagonal directions, And second pixel value estimating means for estimating a pixel value of a blank pixel based on pixel value determined pixels existing in both directions perpendicular to the determined direction.

  As a first aspect, the image processing method of the present invention is an image processing method performed by an apparatus that creates a high resolution image based on pixel values in a plurality of low resolution images. An image in which a pixel vector generation step for calculating a pixel vector and an accumulation image are created by superimposing a plurality of low resolution images, and pixel values are determined for all the pixels in the accumulation image, or pixel values are not determined yet Overlaying step, pixel vector organizing step for organizing pixel vectors by addition or deletion based on preset settings for every pixel in the accumulation image, and pixels for which pixel values have been determined in the image superimposing step Using the value-determined pixel and the pixel vector organized in the pixel vector organizing step, Accordingly, a first pixel value estimating step for estimating a pixel value of a blank pixel whose pixel value has not been determined in the image superimposing step, and each blank pixel value that could not be estimated in the first pixel value estimating step. In addition, the pixel number determination pixel that determines the direction with the largest number of pixels among the number of blank pixels arranged continuously in the vertical direction, the horizontal direction, and the two diagonal directions, and exists in both directions perpendicular to the determined direction. And a second pixel value estimating step for estimating the pixel value of the blank pixel.

  According to a second aspect of the image processing method of the present invention, in the image processing method performed by an apparatus that performs image processing, a pixel value is determined for an image in which pixel value fixed pixels with fixed pixel values exist at random. A pixel vector generation step for calculating a pixel vector of a definite pixel, a pixel value definite pixel, and a pixel vector are used to estimate a pixel value of a blank pixel whose pixel value is undetermined based on a preset condition. For each pixel value of the blank pixel that could not be estimated in the first pixel value estimation step and the first pixel value estimation step, the number of blank pixels lined up continuously in the vertical direction, the horizontal direction, and the two diagonal directions The second pixel value estimation step for determining the pixel value of the blank pixel based on the pixel value determined pixels existing in both directions perpendicular to the determined direction. And having a, the.

  According to a first aspect of the program of the present invention, in a program that is executed by a device that creates a high-resolution image based on pixel values in a plurality of low-resolution images, pixel vectors are calculated for all the pixels in the plurality of low-resolution images. Pixel vector generation processing to be calculated and image overlay processing that creates an accumulation image by superimposing multiple low-resolution images and determines pixel values for all the pixels in the accumulation image or sets pixel values to be undefined A pixel vector arrangement process that arranges pixel vectors by addition or deletion based on a predetermined setting for every pixel in the accumulation image, and a pixel value determination pixel in which the pixel value is determined by the image superposition process And the pixel vector arranged by the pixel vector arrangement process, based on the preset condition, For each pixel value of the blank pixel that could not be estimated by the first pixel value estimation process and the first pixel value estimation process that estimates the pixel value of the blank pixel in which the pixel value is undetermined by the adjustment process, The direction in which the number of pixels is the largest among the number of blank pixels lined up continuously in the horizontal direction and two diagonal directions is determined, and the blank is determined based on the pixel value determination pixels existing in both directions perpendicular to the determined direction. And a second pixel value estimating process for estimating the pixel value of the pixel.

  In addition, as a second aspect, the program of the present invention is a program that is executed by an apparatus that performs image processing. In an image in which pixel value fixed pixels with fixed pixel values exist at random, the pixel value fixed pixels are A pixel vector generation process for calculating a pixel vector, a pixel value fixed pixel, and a pixel vector are used to estimate a pixel value of a blank pixel whose pixel value is undetermined based on a preset condition. For each pixel value of the blank pixel that could not be estimated by the pixel value estimation process and the first pixel value estimation process, the highest number of blank pixels lined up continuously in the vertical direction, the horizontal direction, and the two diagonal directions. A second pixel value estimation process for determining a direction in which the number of pixels is large and estimating a pixel value of a blank pixel based on pixel value fixed pixels that exist in both directions perpendicular to the determined direction. And wherein the Rukoto.

  According to the present invention, it is possible to create a high-resolution image with higher image quality from a smaller number of low-resolution images.

FIG. 2 is a block diagram illustrating functions of an image processing apparatus according to an embodiment of the present invention, and a flowchart illustrating operations of the image processing apparatus. It is a figure explaining a pixel vector generation determination area. It is a figure explaining the magnitude | size of a pixel vector. It is a figure explaining the magnitude | size of the pixel vector expanded n times. It is a figure explaining the pixel vector addition method in a pixel vector rearrangement process. It is a figure explaining the pixel vector deletion method in a pixel vector rearrangement process. It is a figure explaining the condition 1 which selects the pixel value fixed pixel used for pixel value estimation of a blank pixel. It is a figure explaining the condition 2 which selects the pixel value definite pixel used for pixel value estimation of a blank pixel. It is a figure explaining the example of execution of pixel value presumption of a blank pixel. It is a figure explaining the distance between the pixels of a blank pixel. It is a figure explaining the method of determining the directionality of a blank pixel. It is a flowchart which shows operation | movement of the image processing apparatus which concerns on one Embodiment of this invention. It is a figure explaining the image which has the pixel value fixed pixel scattered at random, overlapping with a to-be-photographed object. It is a figure explaining the image which has the pixel value fixed pixel scattered at random. It is a figure explaining a pixel vector generation determination area. It is a figure explaining a pixel vector.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments (embodiments) for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiment of the present invention described below does not unduly limit the content of the present invention described in the claims. In addition, all of the configurations described in the following embodiments are not essential configuration requirements of the present invention.

  An image processing apparatus according to an embodiment of the present invention will be described. The image processing apparatus according to this embodiment may be any apparatus that performs super-resolution processing for creating a high-resolution image based on a plurality of low-resolution images. As an application example of the image processing apparatus of the present embodiment, an MFP (Multi Function Peripheral), a facsimile apparatus, a printer apparatus, a personal computer, a game machine, a mobile phone, a car navigation system having composite functions such as copying, facsimile, scanner, and printing Examples thereof include a device, a digital camera, a television, and a DVD (Digital Versatile Disc) player.

  The image processing apparatus according to the present embodiment has a hardware configuration such as a CPU (Central Processing Unit) that controls processing of the entire image processing apparatus and a ROM (Read Only Memory) that stores fixed data such as a program for controlling the CPU in advance. ), And a RAM (Random Access Memory) that forms an area for temporarily storing various data. The CPU controls the operation of the entire image processing apparatus according to the control program read from the ROM, and implements the functions shown in FIG.

  Next, functions of the image processing apparatus according to the present embodiment will be described with reference to FIG. FIG. 1A is a functional block diagram of the image processing apparatus of the present embodiment. As illustrated in FIG. 1A, the image processing apparatus includes a pixel vector generation unit 11, an image superimposition unit 12, a pixel vector arrangement unit 13, and a pixel value estimation unit 14. By these parts, a five-step process shown in FIG. 1B is executed.

  Next, details of the operation of the image processing apparatus according to the present embodiment described with reference to FIG. 1B (one embodiment of the image processing method according to the present invention) will be described below.

  In the present embodiment, the pixel values of blank pixels remaining in the superimposed image are adjusted in the final process of generating a high-resolution image by aligning and superimposing a plurality of low-resolution images with misalignment. Efficient estimation using pixel vectors.

  First, the pixel vector generation unit 11 calculates pixel vectors of all pixels of a plurality of low-resolution images as an in-image pixel vector generation process (first stage) (step S1). Details of this step will be described below.

  In the low-resolution image, for example, when the pixel located at the center is a calculation target (hereinafter, this pixel is referred to as a target pixel) in the low-resolution image, the pixel vector generation unit 11 uses the target pixel as a reference for vertical / A total of eight pixels (determination region pixels) including the four pixels adjacent in the horizontal direction and the four pixels in contact with the two diagonal directions are defined as a determination region. In FIG. 2, each pixel indicated by a thick line is a determination area. Then, the pixel vector generation unit 11 calculates a difference in pixel values between the target pixel and the eight determination region pixels, and obtains a direction from a pixel having a smaller difference than a preset threshold value.

  Further, for example, as shown in FIG. 3, the pixel vector generation unit 11 sets the distance to a pixel adjacent in the vertical / horizontal direction to 1.0 and the distance to a pixel in contact with the diagonal two directions to 1.5. To determine the pixel vector of the target pixel.

  For example, when the difference between the adjacent pixels in the horizontal right direction is equal to or smaller than the threshold value, the target pixel has a pixel vector having a size of 1.0 in the horizontal direction rightward.

  In this manner, the pixel vector generation unit 11 obtains pixel vectors for all pixels in other low resolution images.

  Next, the image superposition unit 12 superimposes the images after aligning the low-resolution images as an image superposition process (second stage). That is, the image superimposing unit 12 first calculates a coordinate conversion vector from the positional deviation of each low-resolution image, then multiplies all the low-resolution images by n, and finally performs coordinate conversion and superimposes (step S2). ). The superimposed images are hereinafter referred to as “accumulate images”. Details of this step will be described below.

  In this step, alignment processing generally used in mosaicing processing or the like for creating a large image by integrating a plurality of images is used.

  The image superimposing unit 12 designates one low-resolution image as a reference image in order to align a plurality of low-resolution images.

  The image superimposing unit 12 obtains a coordinate conversion vector of the frame image in order to superimpose a low-resolution image (hereinafter referred to as a frame image) other than the reference image on the reference image.

  The image superimposing unit 12 deforms the image of the frame image based on the conversion vector (hereinafter referred to as a predicted frame image). At the same time, the image superimposing unit 12 also changes the pixel vector of the predicted frame image based on the conversion vector.

  By setting the distance between the pixels of the reference image and the plurality of prediction frame images to n (n is a number larger than 1), the size of the pixel vector is also 1 → n, 1.5 →, as shown in FIG. 1.5n.

  The image superimposing unit 12 superimposes the n-fold reference image and a plurality of predicted frame images to create an accumulation image. At this time, when a plurality of pixels are overlapped by overlapping the images, the image overlapping unit 12 determines the pixel value by calculating an average value of the pixel values. If no pixel is present, the pixel is not complemented and left blank (pixel value undefined).

  Next, the pixel vector rearrangement unit 13 rearranges (adds / deletes) the pixel vectors defined in the respective low resolution images as the inter-image pixel vector rearrangement process (third stage) (step S3). ). Details of this step will be described below.

  A pixel (target) that is located in the center of a square of 2n on one side surrounded by 8 pixels (judgment area pixels) that are in contact with the vertical / horizontal / diagonal 2 directions at the time of the reference image or frame image before enlarging n times Pay attention to the positional relationship and pixel value between the pixel vector of (pixel) and the pixels of the other image existing in the same region.

  The pixel vector organizing unit 13 adds a pixel vector when there is a pixel value having a difference within a preset threshold value and the pixel value of the target pixel in the region.

  For example, in FIG. 5, when there is a pixel 52 having a difference within a preset threshold value with respect to the target pixel 51 in the region of one side 2n centered on the target pixel 51, the pixel vector organizing unit 13 Adds a pixel vector 53 to the target pixel 51.

  In addition, when there is a pixel having a difference between the pixel value of the target pixel and a preset threshold value on the pixel vector of the target pixel and / or in the vicinity of the pixel vector, the pixel vector organizing unit 13 Is deleted.

  For example, in FIG. 6, when there is a pixel 62 having a difference exceeding the pixel value of the target pixel 61 and a preset threshold on the pixel vector 63 of the target pixel 61, the pixel vector organizing unit 13 sets the pixel vector 63. delete.

  The pixel vector organizing unit 13 performs the above-described operation for all the pixels of the accumulation image.

  In this way, the pixel value estimation of the blank pixel described below is performed based on the reference image and frame image information obtained up to this step.

  Next, the pixel value estimation unit 14 uses the pixel vector finally determined by the pixel vector organizing unit 13 as the pixel value estimation A process (fourth stage) of the blank pixel, and calculates the pixel value of the blank pixel. Estimate (step S4). Details of this step will be described below.

  The pixel value estimation unit 14 performs pixel value estimation of blank pixels from pixels that satisfy the following conditions.

Condition 1: The pixel vector penetrates the target blank pixel.
Condition 1 refers to a case where the pixel vector 73 from the pixel value determination pixel 71 passes through the blank pixel 72 as shown in FIG. In the example of FIG. 7, when estimating the pixel value of the blank pixel 72, the pixel value estimation unit 14 uses the pixel value of the pixel value determination pixel 71.

Condition 2: The target blank pixel is included inside a triangle formed by a pixel vector coming out of one pixel.
For example, as shown in FIG. 8, the condition 2 is that when estimating the pixel value of the blank pixel 82 included in the triangle formed by the pixel vector 83 and the pixel vector 84 from the pixel value determination pixel 81, the pixel value estimation unit 14 Uses the pixel value of the pixel value determination pixel 81. However, if the distance between the pixel value determined pixel and the target blank pixel is short, the effective range of the angle formed by the two pixel vectors is increased, and if the distance is far, the effective range is decreased.

  The pixel value estimation unit 14 weights the pixel value determined pixels that satisfy the above conditions according to the distance of the blank pixel, and estimates the pixel value of the blank pixel.

  For example, in FIG. 9, when the pixel value estimation unit 14 estimates the pixel value of the blank pixel 95, the following is performed.

Pixel value determination pixel 91: A blank pixel 95 is included inside a triangle formed by two pixel vectors.
Pixel value determination pixel 92: one pixel vector penetrates the blank pixel 95.
Pixel value determination pixel 93: A blank pixel 95 is included inside a triangle formed by two pixel vectors.
Pixel value determination pixel 94: A blank pixel 95 is not included in a triangle formed by two pixel vectors.

  As described above, when estimating the pixel value of the blank pixel 95, the pixel value estimation unit 14 uses the pixel value determined pixel 91, the pixel value determined pixel 92, and the pixel value determined pixel 93. For example, the pixel value estimation unit 14 calculates the average value of the pixel values of the pixel value determined pixels 91, 92, and 93 and sets it as the pixel value of the blank pixel 95.

  At this time, the pixel value estimation unit 14 considers the distance (value) between the blank pixel 95 and the pixel value determined pixel 91, the pixel value determined pixel 92, and the pixel value determined pixel 93 as a parameter, and determines the pixel value of the blank pixel 95 as a parameter. You may make it estimate.

  As described above, when the pixel values of all the blank pixels can be estimated in the stage of this step, a high-resolution image can be obtained.

  However, for edge portions where the color changes rapidly, the difference in pixel values is large, so that a pixel vector necessary for pixel value estimation is not generated, and there is a high possibility that blank pixels remain as they are. Therefore, pixel estimation described below is performed.

  Next, as the pixel value estimation B process (fifth stage) of the blank pixel, the pixel value estimation unit 14 estimates the direction of the blank pixel that cannot be estimated by the pixel value estimation process A of the blank pixel and estimates the pixel value. (Step S5). Details of this step will be described below.

  First, the pixel value estimation unit 14 obtains the directionality of the blank pixel.

  The pixel value estimation unit 14 counts the number of blank pixels arranged in a straight line in a total of four directions of vertical / horizontal / diagonal directions for each blank pixel.

  At this time, as shown in FIG. 10, the pixel value estimation unit 14 sets the distance to the pixels adjacent in the vertical / horizontal direction to 1.0 and the distance to the pixel in contact with the two diagonal directions to 1.5. .

For example, in FIG. 11, the blank pixels 111 include four in the horizontal direction including the blank pixels 111, three in the vertical direction, five in the diagonal upper right and lower left direction, and the diagonal left upper right. There are two consecutive downwards. In this case, the distance in each direction is
Horizontal direction: 5.0,
Vertical direction: 4.0,
Diagonal upper right lower left direction: 9.0
Diagonal left upper right lower direction: 4.5
Therefore, the pixel value estimation unit 14 determines the diagonal right upper right lower left direction as the longest distance as the directionality of the blank pixel 111.

  Next, the pixel value estimation unit 14 determines a pixel to be used for pixel value estimation of the blank pixel.

  For the pixel value estimation of the blank pixel, a pixel having a fixed pixel value that is present in both directions perpendicular to the directionality of the blank pixel is used.

  For example, in the case of the blank pixel 111 in FIG. 11, since the directionality is the diagonal upper right and lower left direction, the pixel value estimation unit 14 and the pixel value determination pixel 112 and the pixel value that exist in both the diagonal upper left upper right and lower directions across the blank pixel 111. The pixel value of the definite pixel 113 is used. At this time, the pixel value estimation unit 14 weights the pixel value determined pixel by the distance of the blank pixel, and estimates the pixel value of the blank pixel.

  As described above, according to this embodiment, in the creation of a super-resolution image, the pixel value of the pixel value undefined pixel (blank pixel) remaining after superposing a plurality of low-resolution images The pixel vector of the plurality of low resolution images generated by the vector generation unit 11 and the pixel vector of the image superimposed by the image superimposing unit 12 are organized by the pixel vector organizing unit 13, and the pixel value estimating unit The directionality is estimated and the pixel value is estimated.

  Therefore, according to the present embodiment, a high-quality high-resolution image can be created from a smaller number of low-resolution images by estimating the pixel value using the directionality of the pixel vector and the blank pixel.

  Further, according to the present embodiment, the calculation cost can be reduced by reducing the number of low-resolution images to be used.

  Further, according to the present embodiment, by estimating the pixel value using the directionality of the pixel vector and the blank pixel, it is possible to restore an image having pixel value definite pixels scattered randomly.

  Next, an embodiment different from the above-described embodiment (hereinafter referred to as the first embodiment) will be described in detail with reference to the drawings.

  Although the configuration of the image processing apparatus according to the present embodiment is not illustrated, the configuration includes the pixel vector generation unit 11 and the pixel value estimation unit 14 in FIG. FIG. 12 is a flowchart showing the operation (blank pixel value estimation method) of the image processing apparatus according to this embodiment. In the first embodiment described above, the blank pixel value estimation method in the multiple super-resolution image creation step is used. However, in this embodiment, for example, as shown in FIG. It will be described that the present invention is also effective for a pixel value estimation method for blank pixels that can be obtained only in a certain state.

  The pixel vector generation unit 11 generates pixel vectors of all pixel value fixed pixels for an image having pixel value fixed pixels that exist randomly as shown in FIG. 14 (step S131).

  In the present embodiment, assuming that the pixel vector determination region is 7 pixels × 7 pixels shown in FIG. 15, the pixel vectors are as shown in FIG.

  Thereafter, similarly to the first embodiment (steps S4 and S5 in FIG. 1B), the pixel value estimation unit 14 performs pixel value estimation using a pixel vector (step S132), and then blank pixels. The pixel value is estimated using the directionality (step S133).

  As described above, according to the present embodiment, the pixel value of the blank pixel is estimated from the image including the pixel value fixed pixels that are randomly scattered. Therefore, the present embodiment is also effective for a pixel value estimation method for blank pixels in a case where pixel value determined pixels are obtained only in a random state with respect to the subject.

  As mentioned above, although each embodiment of this invention was described, it is not limited to said each embodiment, A various deformation | transformation is possible in the range which does not deviate from the summary.

  For example, the operations in the above-described embodiments can be executed by hardware, software, or a combined configuration of both.

  When executing processing by software, a program in which a processing sequence is recorded may be installed and executed in a memory in a computer incorporated in dedicated hardware. Or you may install and run a program in the general purpose computer which can perform various processes.

  For example, the program can be recorded in advance on a hard disk or a ROM (Read Only Memory) as a recording medium. Alternatively, the program is temporarily or permanently stored on a removable recording medium such as a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disc, a DVD (Digital Versatile Disc), a magnetic disc, or a semiconductor memory. It can be stored (recorded). Such a removable recording medium can be provided as so-called package software.

  The program may be wirelessly transferred from the download site to the computer in addition to being installed on the computer from the removable recording medium as described above. Or you may wire-transfer to a computer via networks, such as LAN (Local Area Network) and the internet. The computer can receive the transferred program and install it on a recording medium such as a built-in hard disk.

  In addition to being executed in time series in accordance with the processing operations described in the above embodiment, the processing capability of the apparatus that executes the processing, or a configuration to execute in parallel or individually as necessary Is also possible.

  The present invention can be applied to a technique for performing image processing, and in particular, can be applied to apparatuses / devices, systems, methods, and programs in general that perform super-resolution processing.

DESCRIPTION OF SYMBOLS 11 Pixel vector production | generation part 12 Image superimposition part 13 Pixel vector rearrangement part 14 Pixel value estimation part 51, 61 Target pixel 52, 62 Pixel 53, 63 Pixel vector 71, 81, 91, 92, 93, 94, 112, 113 pixel Value definite pixel 72, 82, 95, 111 Blank pixel 73, 83, 84 Pixel vector

Claims (11)

In an image processing apparatus that creates a high-resolution image based on pixel values in a plurality of low-resolution images,
Pixel vector generation means for calculating a pixel vector for all pixels in the plurality of low resolution images;
Creating an accumulation image by superimposing the plurality of low-resolution images, and confirming a pixel value for all the pixels in the accumulation image, or an image superimposing means for undecided pixel value;
Pixel vector organizing means for organizing the pixel vectors by addition or deletion based on a preset setting for every pixel in the accumulating image;
A pixel value determined by the image superimposing unit and a pixel vector determined by the pixel vector organizing unit are used, and a pixel value is determined by the image superimposing unit based on a preset condition. First pixel value estimation means for estimating a pixel value of a blank pixel that has not been determined;
For each pixel value of the blank pixel that could not be estimated by the first pixel value estimating means, the direction having the largest number of pixels among the number of blank pixels arranged continuously in the vertical direction, the horizontal direction, and the two diagonal directions And a second pixel value estimating means for estimating a pixel value of the blank pixel based on a pixel value determined pixel existing in both directions perpendicular to the determined direction;
An image processing apparatus comprising: The pixel vector generation means includes
For every pixel in the plurality of low resolution images, a pixel existing in the vertical direction, the horizontal direction and two diagonal directions is defined as a determination region pixel, and a difference in pixel value from each determination region pixel is calculated.
Finding the direction of the determination area pixel and the difference is smaller than a preset threshold,
The image processing apparatus according to claim 1, wherein the pixel vector is set by setting a distance set in advance in the direction. The image superimposing means includes
One of the plurality of low resolution images is designated as a reference image,
In order to superimpose a low-resolution image other than the reference image on the reference image as a frame image, a coordinate conversion vector of the frame image is obtained,
The frame image is deformed based on the conversion vector to obtain a predicted frame image, and the pixel vector of the predicted frame image is changed based on the conversion vector,
The distance between pixels of the reference image and the plurality of prediction frame images is set to a predetermined value n (n> 1), the reference image multiplied by n and the plurality of prediction frame images are superimposed, and the accumulation image The image processing apparatus according to claim 1, wherein the image processing apparatus generates an image. The pixel vector organizing means includes:
For every pixel in the accumulative image, the pixel value difference with each determination region pixel existing within a predetermined range in the vertical direction, the horizontal direction and the two diagonal directions is calculated, and the difference is preset. While adding a new pixel vector between the determination area pixels within the threshold,
For every pixel in the accumulative image, a difference in pixel value from a pixel existing on or in the vicinity of the pixel vector generated by the pixel vector generation means is calculated, and the difference is set in advance. 4. The image processing apparatus according to claim 1, wherein a pixel vector existing between the pixels exceeding the threshold is deleted. 5. The first pixel value estimating means includes:
The pixel value of the blank pixel is estimated using the pixel value determined pixel when a pixel vector having the pixel value determined pixel as a base point passes over the blank pixel. 5. The image processing apparatus according to any one of 4 above. The first pixel value estimating means includes:
Estimating the pixel value of the blank pixel using the pixel value determined pixel when the blank pixel exists within a triangular range formed by two pixel vectors having the pixel value determined pixel as a base point. The image processing apparatus according to claim 1, wherein the image processing apparatus is characterized. A pixel vector generation means for calculating a pixel vector of the pixel value determination pixel for an image in which pixel value determination pixels in which a pixel value is fixed exist at random;
First pixel value estimating means for estimating a pixel value of a blank pixel whose pixel value is undetermined based on a preset condition using the pixel value determined pixel and the pixel vector;
For each pixel value of the blank pixel that could not be estimated by the first pixel value estimating means, the direction having the largest number of pixels among the number of blank pixels arranged continuously in the vertical direction, the horizontal direction, and the two diagonal directions And a second pixel value estimating means for estimating a pixel value of the blank pixel based on a pixel value determined pixel existing in both directions perpendicular to the determined direction;
An image processing apparatus comprising: In an image processing method performed by an apparatus that creates a high-resolution image based on pixel values in a plurality of low-resolution images,
A pixel vector generation step of calculating a pixel vector for all pixels in the plurality of low resolution images;
An image overlay step of creating an accumulation image by superimposing the plurality of low-resolution images, and determining pixel values for all the pixels in the accumulation image or setting pixel values to be undefined,
A pixel vector organizing step for organizing the pixel vector by addition or deletion based on a preset setting for every pixel in the accumulating image;
The pixel value is determined in the image superimposing step based on a preset condition using the pixel value determined pixel in which the pixel value is determined in the image superimposing step and the pixel vector rearranged in the pixel vector organizing step. A first pixel value estimating step for estimating a pixel value of an undefined blank pixel;
For each pixel value of the blank pixel that could not be estimated in the first pixel value estimation step, the direction having the largest number of pixels among the number of blank pixels arranged continuously in the vertical direction, the horizontal direction, and the two diagonal directions A second pixel value estimating step for estimating a pixel value of the blank pixel based on pixel value fixed pixels existing in both directions perpendicular to the determined direction;
An image processing method comprising: In an image processing method performed by an apparatus that performs image processing,
A pixel vector generation step of calculating a pixel vector of the pixel value determination pixel for an image in which pixel value determination pixels in which a pixel value is fixed exist at random;
A first pixel value estimating step for estimating a pixel value of a blank pixel whose pixel value is undefined based on a preset condition using the pixel value determined pixel and the pixel vector;
For each pixel value of the blank pixel that could not be estimated in the first pixel value estimation step, the direction having the largest number of pixels among the number of blank pixels arranged continuously in the vertical direction, the horizontal direction, and the two diagonal directions A second pixel value estimating step for estimating a pixel value of the blank pixel based on pixel value fixed pixels existing in both directions perpendicular to the determined direction;
An image processing method comprising: In a program to be executed by a device that creates a high-resolution image based on pixel values in a plurality of low-resolution images,
Pixel vector generation processing for calculating a pixel vector for all the pixels in the plurality of low resolution images;
Creating an accumulation image by superimposing the plurality of low-resolution images, and confirming a pixel value for all the pixels in the accumulation image, or an image overlapping process in which the pixel value is undetermined;
A pixel vector organizing process for organizing the pixel vector by addition or deletion based on a predetermined setting for every pixel in the accumulating image;
The pixel value is determined in the image superimposition process based on a preset condition using the pixel value determined pixel in which the pixel value is determined in the image superimposition process and the pixel vector rearranged in the pixel vector rearrangement process. A first pixel value estimation process for estimating a pixel value of a blank pixel that has not been determined;
For each pixel value of the blank pixel that could not be estimated by the first pixel value estimation process, the direction with the largest number of pixels among the number of blank pixels arranged continuously in the vertical direction, the horizontal direction, and the two diagonal directions A second pixel value estimation process for estimating a pixel value of the blank pixel based on a pixel value determination pixel existing in both directions perpendicular to the determined direction;
A program characterized by having executed. In a program to be executed by an apparatus that performs image processing,
A pixel vector generation process for calculating a pixel vector of the pixel value determination pixel for an image in which pixel value determination pixels in which a pixel value is fixed exist at random;
A first pixel value estimation process for estimating a pixel value of a blank pixel whose pixel value is undetermined based on a preset condition using the pixel value determined pixel and the pixel vector;
For each pixel value of the blank pixel that could not be estimated by the first pixel value estimation process, the direction with the largest number of pixels among the number of blank pixels arranged continuously in the vertical direction, the horizontal direction, and the two diagonal directions A second pixel value estimation process for estimating a pixel value of the blank pixel based on a pixel value determination pixel existing in both directions perpendicular to the determined direction;
A program characterized by having executed.

Images