JP6041649B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing program capable of reducing processing time by performing parallel processing when, for example, aligning two images taken in a close range. Is.

With the recent development of sensors for image acquisition, various high-resolution images can be taken.
As an image acquisition sensor, an area sensor that acquires an image at once with pixels arranged in two dimensions, or a line sensor that forms an image by acquiring signals while moving elements arranged in one dimension and so on.
In general, a line sensor can capture a higher resolution image than an area sensor.

In addition, techniques for generating data using an image acquired by an image acquisition sensor have been developed.
For example, there is a technique for generating a super-resolution image using a plurality of images taken of similar places (for example, see Non-Patent Document 1).
In a generally used super-resolution image generation process, it is necessary to obtain a positional relationship between pixels in a plurality of images in order to increase the resolution (reconstruction).
However, since processing needs to be performed in units of pixels, the processing time increases as the number of pixels in the image increases. However, as described above, due to recent development of image acquisition sensors, high-resolution images can be captured. Since it is possible (capturing in units of several thousand pixels in the vertical and horizontal directions), the number of pixels per image is increasing.
When a super-resolution image is generated using the high-resolution image as described above, it is assumed that the processing time is enormous for obtaining the positional relationship between the pixels.

Here, parallel processing is an example of a method for shortening the processing time.
In image processing, a target image is divided, processing (for example, calculation of positional relationship) is performed in parallel on a plurality of divided images, and the processing results are combined.
For example, Patent Document 1 below discloses an image processing apparatus that divides a target image and performs image processing.
In this image processing apparatus, when halftone processing is performed on an image, the image is divided, the plurality of divided images are processed in parallel, and the processing results are combined.
Since halftone processing requires surrounding information, an overlapping portion is provided between a plurality of divided images, and only one processing result is employed in the overlapping portion. This is because halftone processing depends only on neighborhood values.

In this image processing apparatus, an image is divided when halftone processing is performed on the image. However, in the positional deviation calculation processing and positional relationship acquisition processing of a plurality of images, only the neighborhood value is not always used.
For this reason, in processing that uses information of the entire image, the processing result may differ between when the image is divided and parallel processing is performed, and when processing without parallel processing is performed.
For example, when using information on the images themselves (such as color distribution information of the images of each other), if the images are divided and parallel processing is performed, the processing results may be different from those when processing is performed without parallel processing.
Even when information including an error is used when obtaining a positional relationship, when an image is divided and parallel processing is performed, the processing result may be different from that when processing without parallel processing is performed.

JP 2010-244098 A (paragraph numbers [0007] to [0011])

"Keyword super-resolution technology you should know" http://www.ite.or.jp/data/a_j_keyword/data/FILE-20120103131407.pdf

  Since the conventional image processing apparatus is configured as described above, even when a super-resolution image is generated using a high-resolution image, the high-resolution image is divided and paralleled to a plurality of divided images. If the processing is performed at the same time, the processing time can be shortened. However, there has been a problem that a processing result when processing is performed in parallel on a plurality of divided images may be different from a processing result when processing is performed without performing parallel processing.

  The present invention has been made to solve the above-described problems, and an image processing apparatus and an image processing capable of acquiring in a short time a processing result corresponding to a processing result when processing is performed without performing parallel processing. The purpose is to obtain a program.

  An image processing apparatus according to the present invention includes an image dividing unit that divides an overlapping divided image generated by the overlapping divided image generating unit into a plurality of blocks, and the reference image corresponding to each pixel of the block divided by the image dividing unit. The position deviation of the block relative to the reference image is calculated, a plurality of position deviation calculation means for correcting the position deviation, and the block after the position deviation correction by the position deviation calculation means are overlapped by one If it belongs only to the divided image, or if the block after misalignment correction belongs to a plurality of overlapping divided images but is not a block at the four corners of the overlapping divided image, the pixels in the block after misalignment correction are Move and perform smoothing processing to increase the continuity between the block and the adjacent block. If a corner of the block, and a smoothing processing unit does not perform smoothing processing, in which so as to constitute a plurality of positional deviation correcting means.

  According to the present invention, the image dividing unit that divides the overlapping divided image generated by the overlapping divided image generating unit into a plurality of blocks, and the pixel of the reference image corresponding to each pixel of the block divided by the image dividing unit. Specifically, the positional deviation of the block with respect to the reference image is calculated, a plurality of positional deviation calculation means for correcting the positional deviation, and the block after the positional deviation correction by the positional deviation calculation means is only one overlapping divided image If the block after misalignment correction belongs to a plurality of overlapping divided images but is not a block at the four corners of the overlapping divided image, the pixels in the block after misalignment correction are moved. The smoothing process that increases the continuity between the block and the adjacent block is performed, while the block after the misalignment correction has four corner blocks. In this case, since a plurality of misalignment correction units are configured from the smoothing processing unit that does not perform the smoothing processing, the processing result corresponding to the processing result when processing without parallel processing is performed in a short time. There is an effect that can be acquired.

1 is a configuration diagram illustrating an image processing apparatus according to Embodiment 1 of the present invention; It is a block diagram which shows position shift correction | amendment part 12-n (n = 1, 2, ..., N) of the image processing apparatus by Embodiment 1 of this invention. By overlapping the divided image generating unit 11 is an explanatory diagram showing a plurality of overlapping divided images DP _OBJn the target image P _OBJ is generated is divided. It is explanatory drawing which shows the example of a division _| segmentation of the overlapping division image _DPOBJn by the image division part 21. FIG. It is explanatory drawing explaining the position calculation of a pixel. It is explanatory drawing which shows the smoothing of each pixel by the smoothing process part.

Embodiment 1 FIG.
1 is a block diagram showing an image processing apparatus according to Embodiment 1 of the present invention.
In FIG. 1, the reference image storage unit 1 is composed of a storage device such as a RAM or a hard disk, and stores a reference image _PSTD .
The target image storage unit 2 is composed of a storage device such as a RAM or a hard disk, for example, and is an image (hereinafter referred to as “target image P _OBJ ”) that _aligns with the reference image _PSTD stored in the reference image storage unit 1. Is stored).

Dividing condition storing section 3 is constituted by a storage device such as RAM or a hard disk, a division condition of the subject image P _OBJ, for example, the target image P _OBJ dividing number (lateral division number nx, longitudinal division number ny ) And overlapping ranges between divided images (horizontal overlapping pixel number dx, vertical overlapping pixel number dy), and the like.
The vector information storage unit 4 includes a storage device such as a RAM or a hard disk, and includes a line-of-sight vector corresponding to each pixel in the reference image P _STD and the target image P _OBJ , shooting position information, and the like. I remember information.

The overlapping divided image generation unit 11 is constituted by, for example, a semiconductor integrated circuit mounted with a CPU or a one-chip microcomputer, and the target image storage unit 2 according to the division conditions stored in the division condition storage unit 3. by dividing the object image _{P OBJ} stored in a plurality of overlapping divided images _DP OBJn that overlap partially with each other (n = 1,2, ···, n ) carrying out the process of generating. N is the number of divisions. The overlapping divided image generating unit 11 constitutes an overlapping divided image generating unit.

The misregistration correction units 12-1 to 12 -N are configured by, for example, a semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like, and the overlapping divided image DP _OBJn generated by the overlapping divided image generation unit 11. It identifies the pixels of the corresponding reference image _{P STD} in each pixel, and calculates the positional deviation Δr overlapping divided images _{DP OBJn} for the reference image _{P STD,} overlapping the divided image DP with respect to the reference image _{P STD} according to the calculation result A process of correcting the positional deviation Δr of _OBJn is performed. The misregistration correction units 12-1 to 12-N constitute misregistration correction means.

FIG. 1 shows an example in which the same number of misalignment correction units 12 as the overlapping divided images DP _OBJn generated by the overlapping divided image generating unit 11 are mounted. However, the same number as the generated overlapping divided images DP _OBJn is shown. Need not be.
However, _when the number of misalignment correction units 12 is smaller than the number of the overlapping segment images DP _OBJn generated by the overlapping segment image generation unit 11, some misalignment correction units 12 _perform processing on a plurality of overlapping segment images. Since it is necessary to carry out in order, it is assumed that the processing time is somewhat longer.

The overlapping divided image joining unit 13 is configured by, for example, a semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like, and the overlapping divided image after the positional deviation correction by the positional deviation correcting units 12-1 to 12-N. DP ′ _OBJ1 , DP ′ _OBJ2 ,..., DP ′ _OBJN are joined. The overlapping divided image joining unit 13 constitutes overlapping divided image joining means.

FIG. 2 is a block diagram showing the misregistration correction unit 12-n (n = 1, 2,..., N) of the image processing apparatus according to Embodiment 1 of the present invention.
In FIG. 2, the image dividing unit 21 performs a process of dividing the overlapping divided image _DPOBJn generated by the overlapping divided image generating unit 11 into a plurality of blocks B _{i, j} . The image dividing unit 21 constitutes an image dividing unit. i = 1, 2,..., I, j = 1, 2,.

Positional deviation calculation unit 22-1 to 22-M are block _{B i} divided by the image dividing unit _21, to identify the reference image _{P STD} of pixels corresponding to each pixel of the _j, the block B with respect to the reference image _{P STD i,} position shift [Delta] r _i of _j, with calculating a _j, carries out a process of correcting the positional displacement [Delta] r _{i, j.} The misregistration calculation units 22-1 to 22-M constitute misregistration calculation means.
In the example of FIG. 2, it is assumed that the same number of misregistration calculation units 22 as the blocks B _{i, j} divided by the image division unit 21 are mounted (M = I × J). The number of blocks B _{i, j} need not be the same.
However, when the number of misalignment calculation units 22 is smaller than the number of blocks B _{i, j} divided by the image dividing unit 21, some misregistration calculation units 22 need to sequentially perform processing on a plurality of blocks. Therefore, it is assumed that the processing time is somewhat longer.

The smoothing processing unit 23 performs processing when the block B ′ _{i, j} after the positional deviation correction by the positional deviation calculation units 22-1 to 22-M belongs to only one overlapping divided image, or the block B ′ _{i, j.} Belongs to a plurality of overlapping divided images, but is not a block at the four corners of the overlapping divided image, the pixel in the block B ′ _{i, j} after the positional deviation correction is moved, and the block B ′ _{i, j} On the other hand, when the smoothing process for increasing the continuity with the adjacent blocks is performed, and the block B ′ _{i, j} after the misalignment correction is a four-corner block, the smoothing process is not performed.
Further, the smoothing processing unit 23 joins the blocks B ″ _{1,1 to} B ″ _{I, J} (including the four corner blocks that are not subjected to the smoothing processing) after the smoothing processing, and the overlapping divided image DP _OBJn Similarly, a process of generating overlapping divided images DP ′ _OBJn (n = 1, 2,..., N) after positional deviation correction that partially overlap each other is performed.
Note that the smoothing processing unit 23 constitutes a smoothing processing means.

In the example of FIG. 1, the reference image storage unit 1, the target image storage unit 2, the division condition storage unit 3, the vector information storage unit 4, the overlapping divided image generation unit 11, and the misregistration correction unit 12 that are components of the image processing apparatus. Although it is assumed that each of the −1 to 12-N and the overlap-divided image joining unit 13 is configured by dedicated hardware, the image processing apparatus may be configured by a computer.
When the image processing apparatus is configured by a computer, the reference image storage unit 1, the target image storage unit 2, the division condition storage unit 3 and the vector information storage unit 4 are configured on the internal memory or external memory of the computer. An image processing program describing the processing contents of the divided image generation unit 11, the misregistration correction units 12-1 to 12-N, and the overlapping divided image joining unit 13 is stored in a memory of a computer, and the CPU of the computer stores the memory. The image processing program stored in the computer may be executed.

Next, the operation will be described.
The division condition storage unit 3, as the division conditions of the target image P _OBJ, for example, the target image P _OBJ dividing number (lateral division number nx, longitudinal division number ny) and, overlapping range between the divided image (horizontal direction The number of overlapping pixels dx, the number of vertical overlapping pixels dy), and the like are stored.
In the first embodiment, it is assumed that the same number of divisions as the number of the positional deviation correction units 12-1 to 12-N are stored in the division condition storage unit 3.
Here, FIG. 3 is an explanatory view showing a plurality of overlapping divided images DP _OBJn (n = 1, 2,..., N) generated by dividing the target image P _OBJ by the overlapping divided image generation unit 11. . In the example of FIG. 3, N = 12.

The overlapping divided image generation unit 11 acquires the reference image _PSTD from the reference image storage unit 1 and acquires the target image _POBJ from the target image storage unit 2. When a plurality of target images P _OBJ are stored in the target image storage unit 2, the target images P _OBJ are acquired one by one in order, and a process of generating an overlapping divided image DP _OBJn described later is performed.
Next, the overlapping divided image generation unit 11 acquires the division condition stored in the division condition storage unit 3.

The overlapping divided image generation unit 11 includes, as dividing conditions, the number of divisions of the target image _POBJ (the number of horizontal divisions nx, the number of vertical divisions ny) and the overlapping range between the divided images (the number of horizontal overlapping pixels dx, the vertical direction). If the number of overlapping pixels dy) is stored, the horizontal width (number of horizontal pixels) w1 and the vertical width (number of vertical pixels) of the overlapping divided image DP _OBJn are _calculated from the number of divisions of the target image P _OBJ and the overlapping range between the divided images. h1 is calculated.
w1 = dx + [(w− (nx−1) × dx) / nx] (1)
h1 = dy + [(h− (ny−1) × dy) / ny] (2)
In the formula (1) (2), w is the width (the number of horizontal pixels) of the target image _{P OBJ,} h is the target image _{P OBJ} of the vertical width (the number of vertical pixels).
In addition, [] represents a Gaussian symbol, and a portion that cannot be divided is separately added to any one of the overlapping divided images _DPOBJn so that there is no remainder.

When the overlapping divided image generation unit 11 _calculates the horizontal width (the number of horizontal pixels) w1 and the vertical width (the number of vertical pixels) h1 of the overlapping divided image _DPOBJn , as shown in FIG. 3, the overlapping divided image generation unit 11 stores it in the target image storage unit 2. The target image P _OBJ is divided to generate overlapping divided images DP _OBJn (n = 1, 2,..., N) having a horizontal width of w1 and a vertical width of h1.
At this time, the overlap between the overlapped divided images adjacent in the horizontal direction is the number of pixels dx, and the overlap between the overlapped divided images adjacent in the vertical direction is the number of pixels dy.
In the example of FIG. 3, because there is much in the lateral direction alpha, and adding the alpha remainder to the right end of the overlap divided images DP _OBJn, the right end of the overlap divided images DP _OBJn the width (number of horizontal pixels) is turned w1 + alpha Yes.
Further, since the vertical direction is very beta, and adding the remainder beta Duplicate divided images DP _OBJn of lower, longitudinal width of the overlap divided images DP _OBJn of lower (the number of vertical pixels) is set to h1 + beta.

When the overlapping divided image generation unit 11 generates a plurality of overlapping divided images DP _{OBJ1 to} DP _OBJN , the overlapping divided image DP _OBJ1 is _converted into the misalignment correction unit 12-1, the overlapping divided image DP _OBJ2 is _converted into the misalignment correction unit 12-2, _.. , And outputs the overlapping divided image _DPOBJN to the misalignment correction unit 12-N.
Incidentally, overlapping the divided image generation unit 11 is output to the position deviation correction unit 12-1 to 12-N for vector information about the reference image _{P STD} stored in the vector information storage unit 4, the target image _{P OBJ} The vector information corresponding to each pixel is divided in the same manner as the target image P _OBJ, and the vector information corresponding to each pixel of the overlapping divided image DP _OBJ1 is each pixel of the misalignment correction unit 12-1 and the overlapping divided image DP _OBJ2 . ., And vector information corresponding to each pixel of the overlapped divided image DP _OBJN is output to the misalignment correction unit 12-N.

However, _when the horizontal width w1 of the overlapping divided image DP _OBJn is not 3 × dx or more and the vertical width h1 of the overlapping divided image DP _OBJn is not 3 × dy or more, the overlapping divided image generation unit 11 _selects the target image P _OBJ . Without division, the target image _POBJ and the vector information are output to one misregistration correction unit (for example, misregistration correction unit 12-1).

Positional deviation correcting unit 12-1 to 12-N receives the overlap divided images _{DP OBJn} and vector information from the overlap divided image generating unit 11, the reference image _{P STD} of pixels corresponding to each pixel of the overlap divided images _{DP OBJn} identify and to its positional deviation Δr overlapping divided images _{DP OBJn} relative to the reference image _{P STD} is calculated to correct the positional displacement Δr overlapping divided images _{DP OBJn} relative to the reference image _{P STD} in accordance with the calculation result.
Hereinafter, the processing content of the position shift correction unit 12-n (n = 1, 2,..., N) will be specifically described.

When receiving the overlapped divided image _DPOBJn and the vector information from the overlapped divided image generating unit 11, the image dividing unit 21 of the misalignment correcting unit 12-n divides the overlapped divided image _DPOBJn into a plurality of blocks B _{i, j} . .
Here, FIG. 4 is an explanatory diagram illustrating an example of division of the overlapping divided image _DPOBJn by the image dividing unit 21.
The procedure of dividing the overlapping divided image DP _OBJn by the image dividing unit 21 is as follows. First, the left and right dx pixel ranges and the upper and lower dy pixel ranges (the hatched range in FIG. 4) of the overlapping divided image DP _OBJn are divided. To do. The blocks in the hatched range are the four corner blocks of the overlapping divided image _DPOBJn , the right or left adjacent overlapping divided image, the upper adjacent or lower adjacent overlapping divided image, the upper right, the upper left, This is a portion that may overlap the lower right or lower left overlapping divided image.

Next, in the outer peripheral portion of the overlapped divided image _DPOBJn (excluding the four corner blocks), the range adjacent to the upper and lower sides (the range between the upper left corner block and the upper right corner block, the lower left corner block) And the lower right corner block) is divided so that the horizontal pixel is not less than dx and the vertical pixel is dy.
_{Further, in} the outer peripheral portion (excluding the four corner blocks) of the overlapped divided image _DPOBJn, the range adjacent to the left side and the right side (the range between the upper left corner block and the lower left corner block, the upper right corner block, Is divided so that the vertical pixel is not less than dy and the horizontal pixel is dx.
Finally, the central portion (the shaded range in FIG. 4) of the overlapping divided image _DPOBJn is divided so that the horizontal pixel is dx or more and the vertical pixel is dy or more.
In the blocks B _{i, j} divided by the image dividing unit 21, there is no overlap with adjacent blocks. In the example of FIG. 4, the overlapping divided image DP _OBJn is divided into 20 blocks B _{i, j} .

When the image dividing unit 21 divides the overlapping divided image _DPOBJn into a plurality of blocks B _{i, j} , the block B _1,1 is divided into the position deviation calculating unit 22-1 and the block B _2,1 is divided into the position deviation calculating unit 22-2. ,..., And outputs the blocks _{BI and J} to the positional deviation calculator 22-M.
Incidentally, the image dividing unit 21 is the vector information about the reference image _{P STD} outputs the positional displacement calculating section 22-1 to 22-M, the vector information corresponding to each pixel of the overlap divided images _{DP OBJn} is duplicated The vector information corresponding to each pixel of the block B _2,1 is divided as in the divided image DP _OBJn , the vector information corresponding to each pixel of the block B _2,2 is vector information corresponding to each pixel of the block B _2,2 22-2,..., Vector information corresponding to each pixel of the blocks BI, _J is output to the misregistration calculator 22-M.

When receiving the block B _{i, j} and the vector information from the image dividing unit 21, the position deviation calculating units 22-1 to 22-M of the position deviation correcting unit 12-n use the vector information to generate the block B _i, A pixel of the reference image P _STD corresponding to each pixel of _j is specified, and a positional deviation Δr _{i, j} (i = 1, 2,..., I, j) of the block B _{i, j with} respect to the reference image P _STD = 1, 2,..., J).
Hereinafter, the processing content of the misregistration calculation unit 22-m (m = 1, 2,..., M) will be specifically described.
Here, FIG. 5 is an explanatory diagram for explaining pixel position calculation.

The position shift calculation unit 22-m converts, for each pixel of the block B _{i, j} , the line-of-sight vector (shooting direction) and the shooting position included in the vector information corresponding to the pixel into the earth center coordinate system. The gaze vector (shooting direction) based on the shooting position and the ellipsoidal surface that serves as a reference for altitude information (in a general digital elevation model (DEM), the height from the earth ellipsoidal surface is used as altitude information. The intersection point P is calculated.
In the middle of the line-of-sight vector with the shooting position as the base point, there is a shooting location Q (location indicating a pixel in the image) of the image, and the altitude at that location is altitude information (altitude information indicating the height from the earth ellipsoidal plane) May be given from the outside, or may be held by the misregistration calculation unit 22-m), and if there is no error, it matches the altitude indicated by the altitude information.
The position that coincides with the altitude indicated by the altitude information is the position Q of the pixel. However, since the position Q cannot be obtained directly, Q is calculated using the intersection point P and the line-of-sight vector (imaging direction) while performing iterative calculation. It becomes the form to ask for '.

Positional deviation calculation unit 22-m, a block _{B i,} when obtaining the position Q of each pixel of the _j, similarly to the block _{B i, j-pixels,} with reference to the vector information about the reference image _{P STD,} the reference image P The position of each pixel of the _STD is obtained, and the position Q of each pixel of the block B _{i, j} corresponds to which position of the reference image P _STD .
The misregistration calculation unit 22-m has the coordinates of the pixel of the block B _{i, j} on the target image P _OBJ (x1, y1), and the reference image whose position matches the pixel of the block B _{i, j Assuming} that the coordinates of the pixel of P _STD on the image are (x0, y0), as shown in the following equations (3) and (4), the lateral displacement Δr _x of the pixel of the block B _{i, j} Then, the positional deviation Δr _{x, y} of each pixel is calculated by calculating the positional deviation Δr _y in the vertical direction.
Δr _x = x0−x1 (3)
Δr _y = y0−y1 (4)
Δr _{x, y} = (Δr _x , Δr _y ) (5)

When the positional deviation calculation unit 22-m calculates the positional deviation Δr _{x, y} of each pixel, as shown in the following equation (6), the positional deviation Δr _{i, j} of the block B _{i, j with} respect to the reference image P _STD Calculate
Δr _{i, j} = (Δr _1,1 Δr _2,1 ... Δr _{x, y} ... Δr _{a, b} ) (6)
In the formula (6), a block _{B i,} lateral number of pixels of _j, b is a block _{B i,} the number of pixels longitudinal _j.

When the position deviation calculation unit 22-m calculates the position deviation Δr _{i, j} of the block B _{i, j with} respect to the reference image P _STD , the position deviation calculation unit 22-m corrects the position deviation Δr _{i, j} of the block B _{i, j with} respect to the reference image P _STD . .
That is, each pixel in the block B _{i, j} is moved so that the positional deviation Δr _{i, j} of the block B _{i, j with} respect to the reference image P _STD becomes zero.
The misregistration calculation unit 22-m outputs the block B ′ _{i, j} after the misregistration correction to the smoothing processing unit 23.

When the smoothing processing unit 23 of the misregistration correction unit 12-n receives the misregistration correction blocks B ′ _{1,1 to} B ′ _{I, J} from the misregistration calculation units 22-1 to 22-M, the misregistration processing unit 23-n receives the misregistration. Smoothing processing is performed on the corrected blocks B ′ _{1,1 to} B ′ _{I, J.}
When the block B ′ _{i, j} after the misalignment correction belongs to only one overlapping divided image (in FIG. 4, when the block is an inner peripheral block that is shaded), and after the misalignment correction When the block B ′ _{i, j} belongs to a plurality of overlapping divided images but is not a block at the four corners of the overlapping divided image (when it is an outer peripheral block other than the four corner blocks in the overlapping divided image), the positional deviation correction The processing content of the smoothing processing unit 23 differs between the case where the subsequent block B ′ _{i, j} is a block at the four corners of the overlapping divided image (the case where the block is shaded in FIG. 4).
Hereinafter, the processing content of the smoothing process part 23 is demonstrated concretely.
FIG. 6 is an explanatory diagram showing the smoothing of each pixel by the smoothing processing unit 23.

[When block B ′ _{i, j} belongs to only one overlapping divided image]
As shown in FIG. 6, the smoothing processing unit 23 smoothes each pixel in the block B ′ _{i, j} after the positional deviation correction in four directions.
That is, in the vicinity of the boundary between adjacent blocks, the continuity may be deteriorated due to the parallel processing of the misalignment correction. Therefore, the pixels in the block B ′ _{i, j} after the misalignment correction are moved. Then, the smoothing process for increasing the continuity between the block B ′ _{i, j} and the adjacent block is performed.
However, when performing smoothing, the pixels in the block B ′ _{i, j} are moved in the vicinity of the boundary between the inner peripheral blocks in consideration of the positions of the pixels of the adjacent inner peripheral blocks in order to improve continuity. Like that.
On the other hand, as will be described later, smoothing in the direction in which the inner peripheral block and the outer peripheral block are aligned is not performed in the outer peripheral block, so the pixel position of the outer peripheral block is considered near the boundary between the inner peripheral block and the outer peripheral block. Without moving, the pixels in the inner block B _{i, j} are moved.

[In the case of an outer peripheral block (upper or lower outer peripheral block) other than the four corner blocks in the overlapping divided image]
Since the upper or lower peripheral block overlaps with the upper or lower adjacent overlapping divided image, if smoothing is performed in the vertical direction, parallel processing is performed when multiple overlapping divided images are joined in the subsequent processing. A problem that is different from the processing result when processing is performed without performing the processing occurs.
For example, the outer peripheral block at the upper end (referred to as “the upper outer peripheral block in the overlapping divided image” as necessary) overlaps with the lower outer peripheral block in the upper adjacent overlapping divided image. When smoothing each pixel in the vertical direction, each pixel is moved in consideration of the pixel position of the inner peripheral block one step below the upper peripheral block. Since the correction unit 12 does not know the position of the pixel in the inner peripheral block one step above the lower peripheral block in the upper adjacent divided image, the inner portion of the lower upper peripheral block in the upper adjacent divided image Each pixel is moved without considering the position of the pixel in the peripheral block.

On the other hand, when performing smoothing in the vertical direction on the outer peripheral block at the lower end in the upper adjacent overlapping divided image, each pixel is moved in consideration of the position of the pixel one step above the lower peripheral block. In the misalignment correction unit 12 that handles the adjacent overlapping divided image, the position of the pixel in the inner peripheral block one step below the upper outer peripheral block in the overlapping divided image is unknown, and therefore the upper outer peripheral block in the overlapping divided image is not known. Each pixel is moved without considering the pixel position of the inner peripheral block one step below.
As described above, the pixel block of the adjacent block to be considered when performing smoothing in the vertical direction is different between the upper peripheral block having an overlapping relationship and the lower peripheral block in the upper adjacent overlapping divided image. If a plurality of overlap-divided images are joined in this process, there arises a problem that is different from the processing result when the parallel processing is not performed.

Therefore, as shown in FIG. 6, the smoothing processing unit 23 performs smoothing only in the left-right direction on the outer peripheral block at the upper end or the lower end, and does not perform smoothing in the up-down direction.
However, when performing smoothing, in the vicinity of the boundary between the outer peripheral blocks other than the four corner blocks, in order to increase continuity, the positions of the pixels of the adjacent outer peripheral blocks are taken into account, and the block B ′ _{i, j} Move the pixel.
On the other hand, since smoothing is not performed in the four corner blocks as described later, the outer peripheral block B ′ _{i is} not considered in the vicinity of the boundary between the outer peripheral block and the four corner blocks without considering the pixel positions of the four corner blocks. _{, J} are moved.

[In the case of an outer peripheral block (right outer edge or left outer peripheral block) other than the four corner blocks in the overlapping divided image]
Since the right or left outer peripheral block overlaps with the right or left adjacent overlapping divided image, if smoothing is performed in the left-right direction, parallel processing is performed when multiple overlapping divided images are joined in the subsequent processing. A problem that is different from the processing result when processing is performed without performing the processing occurs.
For example, the rightmost outer peripheral block (referred to as “the rightmost outer peripheral block in the overlapping divided image” as necessary) overlaps with the leftmost outer peripheral block in the right adjacent overlapping divided image. When smoothing each pixel in the horizontal direction, each pixel is moved in consideration of the position of the pixel in the inner peripheral block one row left of the outermost block at the right end. Since the correction unit 12 does not know the position of the pixel in the inner peripheral block on the right of the left outer peripheral block in the right adjacent overlapping divided image, the corrector 12 does not know the position of the right inner line in the left outer peripheral block in the right adjacent overlapping divided image. Each pixel is moved without considering the position of the pixel in the peripheral block.

On the other hand, when performing smoothing in the left-right direction on the leftmost outer peripheral block in the adjacent overlapped image on the right, each pixel is moved in consideration of the position of the pixel on the right of one line of the leftmost outer peripheral block. In the misalignment correction unit 12 that handles the adjacent overlapping divided image, the position of the pixel in the inner peripheral block on the left of the outermost block on the right end in the overlapping divided image is not known. Each pixel is moved without considering the position of the pixel in the inner peripheral block on the left one line.
As described above, since the pixel block of the adjacent block to be considered when performing smoothing in the left-right direction is different between the rightmost outer peripheral block having an overlapping relationship and the leftmost outer peripheral block in the right adjacent overlapping divided image, the latter stage If a plurality of overlap-divided images are joined in this process, there arises a problem that is different from the processing result when the parallel processing is not performed.

Therefore, as shown in FIG. 6, the smoothing processing unit 23 performs smoothing only in the vertical direction on the outer peripheral block at the right end or the left end, and does not perform smoothing in the horizontal direction.
However, when performing smoothing, in the vicinity of the boundary between the outer peripheral blocks other than the four corner blocks, in order to increase continuity, the positions of the pixels of the adjacent outer peripheral blocks are taken into account, and the block B ′ _{i, j} Move the pixel.
On the other hand, since smoothing is not performed in the four corner blocks as described later, the outer peripheral block B ′ _{i is} not considered in the vicinity of the boundary between the outer peripheral block and the four corner blocks without considering the pixel positions of the four corner blocks. _{, J} are moved.

[When Block B ′ _{i, j} is a Block at Four Corners of Overlapping Division Image]
Since the four corner blocks overlap with the right or left adjacent overlapping divided image, the upper adjacent or lower adjacent overlapping divided image, the upper right adjacent, the upper left adjacent, the lower right adjacent or the lower left adjacent overlapping divided image, etc. When smoothing is performed, if a plurality of overlapping divided images are joined in subsequent processing, a problem that is different from the processing result in the case of processing without parallel processing occurs.
The reason why this defect occurs is that, as with the outer peripheral blocks other than the four corner blocks, the pixel positions of adjacent blocks to be considered when performing smoothing are different.
Therefore, the smoothing processing unit 23 does not perform the smoothing process on the block B ′ _{i, j} after the positional deviation correction.

When the smoothing processing unit 23 performs the smoothing processing on the blocks B ′ _{1,1 to} B ′ _{I, J} after the positional deviation correction, the smoothing processing block 23 ″, _{1 to} B ″ _{I, J} (the smoothing processing is performed). In the same manner as the overlapping divided image DP _OBJn , the overlapping divided images DP ′ _OBJn (n = 1, 2, after the misalignment correction) are partially overlapped with each other. _.. , N) are generated, and the overlapped divided image DP ′ _OBJn after the positional deviation correction is output to the overlapped divided image joining unit 13.

The overlapping divided image joining unit 13 receives the overlapping divided images DP ′ _OBJ1 , DP ′ _OBJ2 ,..., DP ′ _OBJN after the positional deviation correction from the smoothing processing unit 23 of the positional deviation correcting units 12-1 to 12-N. Then, the overlapping divided images DP ′ _OBJ1 , DP ′ _OBJ2 ,..., DP ′ _OBJN after the positional deviation correction are joined while overlapping the mutually overlapping portions, and correspond to the original target image P _OBJ . An image P ′ _OBJ to be generated is generated.
The overlapping divided image joining unit 13 refers to the division result of the overlapping divided image in the overlapping divided image generation unit 11 and simply arranges the processing results of the misalignment correction units 12-1 to 12-N. An image P ′ _OBJ corresponding to the target image P _OBJ can be generated.

As apparent from the above, according to the first embodiment, the image dividing unit 21 that divides the overlapping divided image _DPOBJn generated by the overlapping divided image generating unit 11 into a plurality of blocks B _{i, j} , and the image dividing The pixel of the reference image P _STD corresponding to each pixel of the block B _{i, j} divided by the unit 21 is specified, and the positional deviation Δr _{i, j} of the block B _{i, j with} respect to the reference image P _STD is calculated. The positional deviation calculation units 22-1 to 22-M for correcting the positional deviation Δr _{i, j} and the block B ′ _{i, j} after the positional deviation correction by the positional deviation calculation units 22-1 to 22-M are one. If you belong only to the overlap divided image, 'smoothing _i, each pixel in the _j in four directions, the block B of the displacement-corrected' blocks B of the displacement-corrected _{i, j} are a plurality of overlapping divided image Although belonging to, overlapping if not four corners of the block of the divided image, and smoothing in the direction 90 ° from the direction are arranged a plurality of overlapping image segment blocks B _'i of the _{displacement-corrected, j} is 4 corners In the case of the block, since the misalignment correction units 12-1 to 12-N are configured from the smoothing processing unit 23 that does not perform the smoothing processing, the processing result when processing is performed without performing parallel processing There is an effect that the processing result corresponding to can be acquired in a short time.
That is, the high-speed processing by parallel processing can be obtained, an effect of the case of performing the alignment of the reference image P _STD not break target image P _OBJ and the same processing result is obtained.
A super-resolution image or the like can be generated using the target image _P′OBJ and the reference image _PSTD that have been aligned by the image processing apparatus of FIG.

Embodiment 2. FIG.
In the first embodiment, the position shift calculation unit 22-m refers to the vector information and obtains the position Q of each pixel of the block B _{i, j} and the position of each pixel of the reference image P _STD. B _i, showed that identifies the pixels of the reference image _{P STD} corresponding to each pixel of the _j, block _{B i,} while shifting the position of the _j (or block _{B i,} while changing the resolution _j), By calculating the image correlation between the block B _{i, j} and the reference image P _STD and specifying the position (or resolution) at which the image correlation is highest, the reference image P corresponding to each pixel of the block B _{i, j} You may make it specify the pixel of _STD .

Embodiment 3 FIG.
In the first embodiment, the position shift calculation unit 22-m calculates the position shift Δr _{i, j} of the block B _{i, j with} respect to the reference image P _STD and corrects the position shift Δr _{i, j.} Although shown, even if the correction is performed, the positional deviation Δri _{, j} does not become zero, and a slight positional deviation may remain.
Therefore, in the third embodiment, positional displacement calculating unit 22-m is the block _{B i} relative to the reference image _{P STD,} positional displacement [Delta] r _i of _j, by calculating the _j, and correct the positional deviation [Delta] r _{i, j} later, the block B of the displacement-corrected with respect to the reference image P _{STD 'i,} to calculate the positional deviation of the _j again, the block B of the displacement-corrected with respect to the reference image P _STD in accordance with the calculation result' _i, position shift of the _j Will be corrected again.
As a result, the correction accuracy of the positional deviation can be increased.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  1 reference image storage unit, 2 target image storage unit, 3 division condition storage unit, 4 vector information storage unit, 11 overlapping divided image generation unit (overlapping divided image generation unit), 12-1 to 12-N misregistration correction unit ( Misalignment correcting means), 13 overlapping divided image joining section (overlapping divided image joining means), 22-1 to 22-M misregistration calculating section (positional deviation calculating means), 23 smoothing processing section (smoothing processing means).

Claims (7)

An overlapping divided image generating means for dividing the target image according to a dividing condition of the target image to be aligned with the reference image and generating a plurality of overlapping divided images partially overlapping each other;
The pixel of the reference image corresponding to each pixel of the overlapping divided image generated by the overlapping divided image generating means is specified, the positional deviation of the overlapping divided image with respect to the reference image is calculated, and the position of the overlapping image is calculated according to the calculation result. A plurality of misalignment correcting means for correcting misalignment of the overlapped divided image with respect to the reference image;
The overlapping divided image joining means for joining the overlapping divided images after the positional deviation correction by the plurality of positional deviation correcting means,
The plurality of misregistration correction means includes
Image dividing means for dividing the overlapping divided image generated by the overlapping divided image generating means into a plurality of blocks;
A plurality of misregistration calculations for identifying the pixel of the reference image corresponding to each pixel of the block divided by the image dividing means and calculating the misregistration of the block with respect to the reference image and correcting the misregistration. Means,
If the block after the positional deviation correction by the positional deviation calculating means belongs to only one overlapping divided image, or the block after the positional deviation correction belongs to a plurality of overlapping divided images, If the block is not a four-corner block, the pixels in the block after the misalignment correction are moved to perform a smoothing process that increases the continuity between the block and the adjacent block. An image processing apparatus comprising: smoothing processing means that does not perform the smoothing processing when the block is a corner block. The smoothing processing means is
When the block after the positional deviation correction by the positional deviation calculating means belongs to only one overlapping divided image, the pixels in the block after the positional deviation correction are smoothed in four directions,
If the block after misalignment correction belongs to a plurality of overlapped divided images, but is not the block at the four corners of the overlapped divided image, the plurality of overlapped divided images are arranged in the pixels in the block after misalignment correction. The image processing apparatus according to claim 1, wherein smoothing is performed in a direction different from the direction by 90 degrees.   The misregistration calculating means calculates the image correlation between the block divided by the image dividing means and the reference image, and specifies the pixel of the reference image corresponding to each pixel of the block based on the image correlation. The image processing apparatus according to claim 1 or 2.   The misregistration calculation means calculates the image correlation while changing the resolution of the block, and specifies the pixel of the reference image corresponding to each pixel of the block by specifying the resolution with the highest image correlation. The image processing apparatus according to claim 3, wherein:   The positional deviation calculation means calculates the positional deviation of the block with respect to the reference image, corrects the positional deviation, calculates again the positional deviation of the block after the positional deviation correction with respect to the reference image, and according to the calculation result, calculates the positional deviation of the block. The image processing apparatus according to claim 1, wherein the positional deviation of the block after the positional deviation correction with respect to the reference image is recorrected.   When there are a plurality of target images to be aligned with the reference image, the overlapping divided image generating means selects and divides the target images to be divided one by one from the plurality of target images. The image processing apparatus according to any one of claims 1 to 5. An overlapping divided image generation processing procedure for dividing the target image according to a dividing condition of the target image to be aligned with the reference image and generating a plurality of overlapping divided images partially overlapping each other;
A pixel of the reference image corresponding to each pixel of the overlapping divided image generated by the overlapping divided image generation processing procedure is specified, and a positional deviation of the overlapping divided image with respect to the reference image is calculated, and according to the calculation result A plurality of misalignment correction processing procedures for correcting misalignment of the overlapping divided image with respect to the reference image;
An image processing program for causing a computer to execute an overlapping divided image joining processing procedure for joining overlapping divided images after positional deviation correction by the plurality of misregistration correction processing procedures,
The plurality of misregistration correction processing procedures are as follows:
An image division processing procedure for dividing the overlapping divided image generated by the overlapping divided image generation processing procedure into a plurality of blocks;
A plurality of misregistrations that specify the pixel of the reference image corresponding to each pixel of the block divided by the image segmentation processing procedure, calculate the misregistration of the block with respect to the reference image, and correct the misregistration Calculation procedure,
When the block after the positional deviation correction by the positional deviation calculation processing procedure belongs to only one overlapping divided image, or the block after the positional deviation correction belongs to a plurality of overlapping divided images. If the block is not a block at the four corners, the pixels in the block after the positional deviation correction are moved to perform a smoothing process for increasing the continuity between the block and the adjacent block, while the block after the positional deviation correction is performed. An image processing program comprising: a smoothing process procedure that does not perform the smoothing process when the block is a block at four corners.

Images