JP2006227895A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor capable of performing alignment of a plurality of photographic images with high accuracy regardless of a light emission characteristic of an object that is a reference of the alignment, and a photographic condition of the object. <P>SOLUTION: An area pixel data integration part 130 relatively shifts an arithmetic area outputted from an arithmetic area setting part 102 to a filter image 101, and sequentially calculates area integration data wherein all luminance values of the arithmetic area are added up in each shift position. A shift amount calculation part 104 finds a maximum value from the area integration data of each the shift position, and calculates the shift position corresponding to the maximum value as a shift amount for the alignment. A position correction part 105 shifts the filter image 101 on the basis of the calculated shift amount, and outputs it as an output image 106. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus that performs alignment processing of a plurality of images, and more particularly to a technique for correcting a positional deviation between images obtained by imaging the same imaging object a plurality of times.

  As an alignment method for correcting misalignment between a plurality of images, a method for evaluating the correlation between images and aligning the image at a position having the largest correlation is generally well known. The residual is used for evaluating the correlation between images. The residual is a value obtained by accumulatively adding the absolute value of the difference between the luminance values of the overlapped pixels when the two images are overlapped, and indicates the degree of mismatch between the two images. Therefore, it is possible to obtain the position of alignment by calculating the residual at each position while changing the overlapping position and detecting the position where the residual is minimum.

  Patent Document 1 is a document describing a technique related to such alignment processing. Patent Document 1 discloses an image alignment method for obtaining a position where an image to be aligned can be best superimposed on a reference image, and the alignment between the reference image and the target image is good. As a measure of similarity, the similarity in each overlap position candidate with respect to the reference image of the target image is obtained by using a similarity measure obtained by normalizing the residual in the overlap region where the reference image and the target image overlap with the number of elements included in the overlap region. Among the measures, an alignment method is described in which the candidate for the overlapping position where the best similarity measure is obtained is used as the alignment position of the target image with respect to the reference image.

  FIG. 20 is a flowchart showing this alignment process. In FIG. 20, first, the target image to be subjected to the alignment process and the image data of the reference image to be the reference for the alignment process are read (step A0). Also, a set of position parameter candidates (position parameter candidate group) indicating positions that can be taken by the target image with respect to the reference image is read (step A1). Then, the best similarity measure memory for storing the position parameter candidate to which the best similarity measure is given, the number of elements of the overlapping region obtained by the position parameter candidate, and the similarity measure is initialized (step A2).

  Subsequently, one position parameter candidate is selected from the read position parameter candidate group (step A3), and when the selected position parameter candidate is applied to the target image, the target image and the reference image overlap. An overlapping area is obtained and set (step A4). Then, the number of elements included in the overlapping area is calculated (step A5) and stored. Further, a threshold value is obtained and set from the stored number of elements in the overlapping area and the best similarity measure calculated so far (step A6). This threshold value is a value obtained by multiplying the number of elements in the overlapping region by the best similarity measure, and is used to determine whether or not to abort the procedure in the following procedure for calculating the residual in the overlapping region.

  Next, when obtaining the residual in the overlapping region, a residual memory for storing the intermediate progress of the residual calculation (partial residual) is initialized (step A7). Then, a residual is obtained from the luminance values of the pixels of the reference image and the target image corresponding to the elements in the overlapping area, and the residual memory is updated (steps A8 and A9). The updated value in the residual memory is compared with the threshold value set in step A6 (step A10), and if the value in the residual memory exceeds the threshold value, a procedure for obtaining a residual for the overlapping region is performed. The process is interrupted, the process returns to Step A3, and the same processing as that after Step A3 is performed on the next position parameter candidate in the position parameter candidate group.

  On the other hand, if the value in the residual memory is less than the threshold value, the next element in the overlap area is selected, and the process of updating the residual memory value is continued. When the process of updating the values in the residual memory for all the elements in the overlap region is completed, the position parameter in the best similarity measure memory and the position parameter are applied to the target image based on the obtained result. Each of the residual of the overlapping area obtained in this case and the number of elements included in the overlapping area are updated (step A11).

  Then, returning to step A3, the next position parameter candidate in the position parameter candidate group is selected, and the processes after step A3 are repeated. When all the position parameter candidates in the position parameter candidate group are selected and the above processing is applied to them, the best similarity measure memory stores the position parameter that gives the best similarity measure in the position parameter candidate group. And the similarity measure of the overlapping area obtained by the position parameter and the number of elements of the overlapping area are stored. These values are used as alignment results.

  As described above, by using the similarity measure obtained by normalizing the residual in the overlapping area between the reference image and the target image by the number of elements included in the overlapping area as an index for evaluating the goodness of overlapping between the images, Even when the size or shape of the overlapping portion changes depending on the positional relationship between them, accurate alignment can be performed. In addition, when calculating the similarity measure at a certain overlapping position of the target image with respect to the reference image, a threshold value used in the calculation of the similarity measure this time is a value obtained by multiplying the number of elements in the overlapping portion of both images by the best similarity measure so far If the residual during calculation in the overlap region exceeds the threshold, it turns out that the best value up to that point cannot be updated. Is reduced, enabling high-speed alignment. Furthermore, since the similarity measure, which is a normalized residual, is a value that is not affected by the size or shape of the overlapping portion of the image, it holds information indicating whether each pixel of the image is a valid pixel or an invalid pixel. By preparing an image mask in advance and calculating the similarity measure, the image mask is referred to, and only the elements that are both effective pixels in the overlapping area of the reference image and the target image are used, thereby enabling a specific range in the image. It is also possible to align only the image processing target.

Japanese Patent Laid-Open No. 10-49681 FIG. 2

  However, the above alignment processing is based on the premise that the reference image and the target image are obtained by shooting the same target under the same shooting conditions, and one shooting target is passed through filters having different passbands. Therefore, it is difficult to apply to the correction of misalignment between a plurality of images taken in this manner. That is, generally, an object has various emission characteristics, and images taken through filters having different pass bands have different brightness of the same object. Therefore, it becomes difficult to evaluate the correlation between images, and in some cases, the position of the minimum value of the residual obtained by superimposing a pair of images is not necessarily the matching position.

  The present invention has been made in view of the above circumstances, and can accurately align a plurality of captured images regardless of the light emission characteristics of an object serving as a reference for alignment and the imaging conditions of the object. An object is to provide a processing apparatus.

  An image processing apparatus according to the present invention is an image processing apparatus that performs alignment processing of a plurality of images, and includes an arithmetic area that is a pixel area that performs arithmetic processing for alignment for each image to be aligned. The calculation area setting unit to be set and area integration data obtained by adding the luminance values of the pixels of the calculation area in the processing target image at a plurality of shift positions where the calculation area is moved relative to the processing target image. A region pixel data integration unit to calculate, and a shift amount calculation unit to calculate a shift amount indicating the shift position corresponding to the maximum value or the minimum value among the plurality of region integration data calculated in the region pixel data integration unit; A position correction unit that generates a corrected image of the processing target image based on the shift amount calculated by the shift amount calculation unit.

  According to the present invention, it is possible to perform alignment of a plurality of captured images with high accuracy regardless of the light emission characteristics of an object serving as a reference for alignment and the imaging conditions of the object.

  The image processing apparatus of the present invention includes an image processing apparatus in which the calculation area set by the calculation area setting unit is an area of an object serving as a reference for alignment in the processing target image.

  In the image processing apparatus of the present invention, the calculation area set by the calculation area setting unit is one of image data obtained by binarizing a calculation area calculation image that is image data for obtaining the calculation area. This includes areas occupied by pixel values.

  In the image processing apparatus of the present invention, the calculation area calculation image includes an image obtained by photographing a shadow when light is irradiated to an object serving as a reference for alignment in the processing target image.

  The image processing apparatus according to the present invention is an image processing apparatus in which the calculation area calculation image is an image in which all objects that are used as alignment references in the processing target image are captured.

  In the image processing apparatus of the present invention, the image to be aligned is a captured image that is captured by switching a plurality of filters having different pass bands, and the calculation region calculation image is an all pass band of the plurality of filters. Including an image taken using a filter passing through or without using a filter.

  In the image processing device of the present invention, the image to be aligned is a captured image that is captured by switching a plurality of filters having different passbands, and the calculation area setting unit is configured to capture an image pattern of an object to be captured. Based on a threshold value given in advance in the image data, creating a mask pattern indicating the shape of the image, superimposing a plurality of images of the captured image and adding the luminance values of the corresponding pixels of each image The mask pattern is arranged so that the centroid position of the extracted area matches the centroid position of the mask pattern, and the area of the mask pattern is used as the calculation area.

  In the image processing apparatus according to the aspect of the invention, the luminance value of the calculation area determined by the area pixel data integration unit according to the relatively moved range of the calculation areas set by the calculation area setting unit. Including those that add.

  The image processing apparatus of the present invention includes an apparatus in which the region pixel data integration unit adds the luminance values of the calculation region corresponding to a portion excluding a peripheral portion of the processing target image.

  The image processing apparatus of the present invention includes an apparatus in which the calculation area setting unit sets the calculation area in a range determined according to the relatively moved range.

  In the image processing apparatus according to the present invention, the region in which the calculation region is set is a region corresponding to a portion other than a peripheral portion of the processing target image.

  The image processing apparatus of the present invention includes an apparatus in which the region pixel data integration unit moves the calculation region relative to the processing target image at an interval larger than one pixel.

  In the image processing apparatus of the present invention, the area pixel data accumulation unit includes a table having a storage address corresponding to the shift position, and stores area accumulation data at the shift position at the address position. .

  In the image processing apparatus of the present invention, the region pixel data integration unit includes, as the region integration data, a value obtained by subtracting a predetermined value from a result of adding the luminance values of the pixels in the calculation region.

  In the image processing apparatus of the present invention, the region pixel data integration unit calculates an average luminance value for each region connected to the calculation region in the processing target image, and adds all the calculated average luminance values of each region. Including those whose values are area integration data.

  In the image processing apparatus of the present invention, when the shift amount calculation unit has a plurality of maximum values or minimum values of the region integration data calculated by the region pixel data integration unit, a plurality of shift amounts corresponding to the maximum value or the minimum value Including an average value of the shift amount.

  In the image processing apparatus of the present invention, the shift amount calculation unit calculates a maximum value or a minimum value of the shift amount when the calculation area is moved relative to the processing target image at an interval larger than one pixel. What is output to the region pixel data integration unit is included.

  In the image processing apparatus according to the present invention, the region pixel data integration unit has a narrower range and smaller intervals near the movement position corresponding to the maximum value or the minimum value of the shift amount input from the shift amount calculation unit. Including calculating again the area integration data when the calculation area is moved relative to the processing target image. According to the present invention, alignment can be performed at high speed with a small amount of calculation.

  The image processing apparatus of the present invention includes an apparatus that omits the calculation of the region integration data at the already calculated movement position when the region pixel data integration unit calculates the region integration data again.

  The image processing apparatus of the present invention includes an image processing apparatus that further includes a determination unit that determines whether or not the processing target image can be aligned based on the region integration data calculated by the region pixel data integration unit. According to the present invention, when a filter image that may cause a malfunction in the alignment process is input, it is possible to prevent alignment.

  In the image processing apparatus of the present invention, the determination unit obtains a maximum value and a minimum value among the region integration data calculated by the region pixel data integration unit, and based on a ratio between the obtained maximum value and minimum value. Including determining whether the processing target image can be aligned.

  In the image processing apparatus of the present invention, the determination unit obtains a maximum value or a minimum value among the region integration data calculated by the region pixel data integration unit, and a predetermined ratio with respect to the calculated maximum value or minimum value. And determining whether or not the processing target image can be aligned based on the extracted distribution of shift positions.

  The image processing apparatus according to the aspect of the invention includes an apparatus in which the determination unit determines whether or not the processing target image can be aligned based on the size of the extracted range of the shift position.

  The image processing apparatus according to the aspect of the invention includes the determination unit that determines whether or not the processing target image is aligned based on at least one of a vertical width and a horizontal width of the extracted range of the shift position. .

  The image processing apparatus according to the aspect of the invention includes an apparatus in which the determination unit determines whether the alignment of the processing target image is possible based on a sum of a vertical width and a horizontal width of the extracted shift position existence range.

  The image processing apparatus according to the aspect of the invention includes an apparatus in which the determination unit determines whether the alignment of the processing target image is possible based on a product of a vertical width and a horizontal width of the extracted range of the shift position.

  The image processing apparatus according to the aspect of the invention includes an apparatus in which the determination unit determines whether the alignment of the processing target image is possible based on a difference between a vertical width and a horizontal width of the extracted shift position existence range.

  The image processing apparatus according to the aspect of the invention includes an apparatus in which the determination unit determines whether or not the processing target image can be aligned based on a ratio between a vertical width and a horizontal width of the extracted range of the shift position.

  In the image processing apparatus of the present invention, when the image to be aligned is a captured image captured by switching a plurality of filters having different pass bands, and the determination unit determines that alignment is not possible, the same filter , And changing the photographing position to a photographed image and repeating the alignment process.

  In the image processing device of the present invention, the region pixel data integration unit outputs the region integration data at a plurality of shift positions where the calculation region is moved relative to the processing target image in a second movement range. The shift amount calculation unit calculates a maximum value or a minimum value of the region integration data in the second movement range, and a shift position corresponding to the calculated maximum value or minimum value is the second value. If it is within the first movement range smaller than the movement range, the shift amount indicating the shift position corresponding to the maximum value or the minimum value is output as the shift amount, and if it is outside the first movement range , And those that output the shift amount as zero. According to the present invention, it is possible to prevent erroneous alignment with respect to an image that is shifted more than the first movement range.

  In the image processing device according to the aspect of the invention, the region pixel data integration unit may determine that the shift position corresponding to the maximum value or the minimum value of the region integration data in the second movement range is the maximum shift position of the second movement range. In some cases, the second moving range is expanded by a certain amount.

  In the image processing device according to the aspect of the invention, the region pixel data integration unit may determine that the shift position corresponding to the maximum value or the minimum value of the region integration data in the second movement range is the maximum shift position of the second movement range. In some cases, the second movement range is expanded by a certain amount only in the direction of the shift amount corresponding to the shift position.

  In the image processing apparatus of the present invention, the image to be aligned is a captured image obtained by switching a plurality of filters having different passbands, and the second moving range obtained by the region pixel data integrating unit When the shift position corresponding to the maximum value or the minimum value of the area integration data at is outside the first movement range, the same filter is used to change the shooting position to the shot image and repeat the alignment process. Including things.

  In the image processing device of the present invention, the shift amount calculation unit may be configured such that a shift position corresponding to the maximum value or the minimum value of the region integration data in the second movement range is outside the first movement range. When the shift position is not the maximum shift position of the second movement range, the shift amount indicating the shift position is output as the shift amount.

  In the image processing apparatus of the present invention, the image to be aligned includes an image captured by switching a plurality of filters having different pass bands. According to the present invention, even if the alignment target image is an image obtained by photographing an object having various light emission characteristics by switching a plurality of filters having different pass bands, alignment can be reliably performed.

  In the image processing apparatus of the present invention, the image to be aligned includes an image photographed using the same filter at a plurality of different photographing positions, and the region pixel data integrating unit includes the region of the plurality of images. The region pixel data integration unit has a function of storing integration data, the region pixel data integration unit relatively shifts the calculation region with respect to each filter image, obtains region integration data at each shifted position, and each filter image The area integration data is stored, and the shift amount calculation unit obtains a maximum value and a minimum value from the area integration data stored for each image captured using the same filter. This includes selecting a specific image based on the ratio between the maximum value and the minimum value and calculating the shift amount of the image. According to the present invention, alignment processing is performed using a filter image that can perform alignment with the highest accuracy, and alignment can be performed with higher accuracy.

  According to the present invention, it is possible to perform alignment of a plurality of captured images with high accuracy regardless of the light emission characteristics of an object serving as a reference for alignment and the imaging conditions of the object.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Here, in a fluorescence microscope that measures fluorescent substances having different emission wavelengths using a band-pass filter that transmits light in a plurality of specific wavelength bands, an image captured using each band-pass filter is aligned. A case will be described as an example.

(First embodiment)
FIG. 1 is a block diagram showing a schematic configuration of a main part of the image processing apparatus according to the first embodiment of the present invention. The image processing apparatus in FIG. 1 includes a calculation region setting unit 102, a region pixel data integration unit 103, a shift amount calculation unit 104, and a position correction unit 105. Each of these elements can be realized using a computer in which a predetermined program is installed.

  The filter image 101 input to the image processing apparatus is image data obtained by photographing fluorescent materials having different emission wavelengths with a CCD area sensor through a band-pass filter. Here, it is assumed that a fluorescent substance that emits various colors by excitation light is a spherical microbead (hereinafter referred to as a bead) having a diameter of 10 μm, and one or a plurality of beads are present in the filter image 101. . The filter image 101 is digitized with 256 gradations, and the value of each pixel increases as the luminance increases. FIG. 2 shows an example of the filter image 101. As shown in FIG. 2, the luminance value of beads emitting various colors is large (bright), and the luminance value of the background is small (dark). Also, the brightness value of the beads emitting more light of the wavelength that the band pass filter used for photographing passes is larger.

  The calculation area setting unit 102 includes an area setting data storage unit 1021 and a calculation area calculation unit 1022, and sets a specific area (hereinafter referred to as a calculation area) for performing alignment calculation on the filter image 101. The calculation area is output to the area pixel data integration unit 103. The calculation area is a bead area that is an object serving as a reference for alignment in the filter image 101 that is an image to be processed.

  The area setting data storage unit 1021 stores area setting data indicating the shape and position information of beads existing in the filter image in advance, and the stored area setting data is output to the calculation area calculation unit 1022. The calculation area calculation unit 1022 calculates a calculation area based on the area setting data and outputs the calculation area to the area pixel data integration unit 103.

  The area pixel data integration unit 103 performs the process of integrating the luminance values of pixels in the calculation area of the filter image 101 using the calculation areas output from the filter image 101 and the calculation area calculation unit 1022. The luminance value integration process is performed a plurality of times with the operation area relatively shifted with respect to the filter image 101, and the integrated value of luminance values (hereinafter referred to as area integration data) at each shifted position (hereinafter referred to as a shift position). Is output to the shift amount calculation unit 104.

  The shift amount calculation unit 104 obtains the maximum value among the region integration data at each shift position output from the region pixel data integration unit 103, and outputs the shift amount indicating the shift position corresponding to the maximum value to the position correction unit 105. To do. In this example, since the processing target image is an object that serves as a reference for aligning the light-emitting beads, the maximum value is obtained in the area integration data, but depending on the type of processing target image and the type of the target object. May find the minimum value. For example, in the processing target image, the luminance value of the reference object area is small (dark) and the luminance value of the background area is large (bright).

  The position correction unit 105 outputs, as an output image 106, a corrected image obtained by shifting the entire filter image 101 in accordance with the shift amount output from the shift amount calculation unit 104. Since the output image 106 is an image in which the position of the photographing target is shifted to a certain position, it is possible to obtain a superimposed image without positional deviation by directly superimposing output images corresponding to a plurality of input images. it can.

  Next, the operation of the image processing apparatus according to the first embodiment of the present invention configured as described above will be described in detail with reference to FIGS.

  FIG. 3 is a flowchart for explaining image processing performed by the image processing apparatus according to the first embodiment of the present invention. The flowchart in FIG. 3 includes steps S101 to S106. Step S101 includes a calculation region setting step, step S1020 including steps S102 to S104 is a region pixel data integration step, step S105 is a shift amount calculation step, and step S106. Indicates a position correction step.

(Step S101)
First, the calculation area calculation unit 1022 calculates a calculation area for performing alignment calculation on the input filter image 101 based on information stored in the area setting data storage unit 1021. The calculation area is an area where an object serving as a reference for alignment is imaged, and here is a position where a bead is imaged.

The area setting data storage unit 1021 stores bead shape information and position information in advance. The bead shape information is binary digital image data (hereinafter referred to as a pattern image) in which only the beads exist in the image. The bead area is 1 and the other areas are 0. The size of the pattern image may be ensured so that the entire bead can be accommodated. When the bead diameter is 10 pixels, an image memory having 10 pixels in both vertical and horizontal directions is prepared. The position information of the beads is the coordinates on the filter image corresponding to the center position of the pattern image when the bead position in the filter image matches the pattern image, and the X and Y coordinates for the number of beads in the filter image are stored. A memory is prepared to store the coordinates of each bead.

  The calculation area calculation unit 1022 prepares binary digital image data initialized to 0 of the same size as the filter image 101 (hereinafter referred to as calculation area image), and is stored in the area setting data storage unit 1021. The pattern image is overlaid at the coordinate position, and the logical sum operation is performed on each of the overlapped pixels, and the process of writing the operation result in the operation area image is repeated.

  FIG. 4 is a calculation area in the filter image shown in FIG. When referring to the calculation area, an area where the pixel value in the calculation area image is 1 may be examined. The calculation area image created in this way is set as a calculation area and output to the area pixel data integration unit 103.

  In addition, although the example using a spherical bead was shown here, by creating a pattern image according to the shape of the target object, it is not limited to a circular shape, and an object having a complicated shape or a target having a different size is used. In contrast, it is possible to set a calculation area. In addition, the rotation angle information is stored in addition to the shape information and the position information even when the object is rotating, and the pattern image is rotated and arranged when creating the calculation area image. Good. Furthermore, a calculation region image may be created in advance and stored in the region setting data storage unit 1021, and the calculation region calculation unit 1022 may output the stored calculation region image to the region pixel data integration unit 103 as it is. .

(Step S102)
Next, in the region pixel data integration unit 103, the calculation region output from the calculation region calculation unit 1022 is relative to the filter image 101 within a predetermined movement range (hereinafter referred to as a first movement range). Thus, a shift process for shifting one pixel at a time is performed. The initial position is the reference position (shift amount is 0). Further, the order of shifting within the first movement range is arbitrary, but here it is assumed that the position is shifted spirally from the reference position.

  The first movement range may be set to a range including the maximum shift amount that can be taken when the filter images are aligned, and the position where the calculation region image is directly superimposed on the filter image is set. The reference position is set to a range in which the maximum shift amount is taken up, down, left and right from the reference position. Here, it is assumed that the maximum shift amount is 10, and a square range having a spread of 10 pixels vertically and horizontally from the reference position is set as the first movement range. As a result, if the deviation from the reference position is within 10 pixels, alignment is possible. The shift position is represented by the X axis on the horizontal axis and the Y axis on the vertical axis, and the value increases in the right and downward directions, and the reference position is (0, 0). Therefore, the four corners of the square range to be shifted are respectively (-10, -10) for the upper left position, (10, -10) for the upper right position, (-10, 10) for the lower left position, and (10, 10) for the lower right position. 10).

(Step S103)
Next, in the region pixel data integration unit 103, the filter image 101 and the calculation region image at the reference position (shift amount 0) or the shifted calculation region image are superimposed, and the luminance value of the pixel of the filter image 101 that overlaps the calculation region The area integration data is calculated by adding all of the above.

  In this case, if the calculation area exists near the periphery of the calculation area image, the calculation area may protrude from the filter image when the filter image and the shifted calculation area are overlapped. This is illustrated in FIG. As shown in FIG. 5, in the calculation region that protrudes from the filter image, the luminance value cannot be integrated because there is no luminance value, and the total number of pixels to be integrated differs depending on the shift position. In order to avoid such a state, the luminance values may be integrated only in the calculation area that exists in a range that does not protrude from the filter image due to the shift of the calculation area. It is possible to determine whether or not the calculation area in the range that does not protrude exists in the area narrowed to the inner side of the maximum shift amount from the edge of the calculation area image.

  In addition, when the calculation area is set in step S101, such a state can be avoided by generating the calculation area image except for the calculation area outside the range that does not protrude by the shift process. Specifically, the pixel value 1 indicating the calculation area outside the range in the calculation area image may be changed to 0. The range in which the calculation area is set may be a range narrowed to the inside more than the maximum shift amount. In such a range, if the periphery of the image to be processed is distorted due to lens characteristics or other reasons and is not optimal when performing alignment, the reference objects such as beads existing in these non-optimal areas are excluded from the calculation. And more accurate alignment can be performed. The range at this time may be a range having a certain distance from the center of the image.

(Step S104)
The steps S102 and S103 are repeated at each shift position of the first movement range, the area integration data at each shift position of the first movement range is sequentially calculated, and the calculated area integration data is used as the shift amount calculation unit 104. Output to.

  At this time, the memory for storing the area integration data at each shift position is a table having a storage address corresponding to the shifted shift amount, and the area integration data at each shift position shifted to that address position is stored. . In this way, when it is desired to know at which shift position the area integration data stored in the table is calculated, it can be obtained by referring to the address.

  Hereinafter, a specific storage operation will be described for the case where the maximum shift amount is 10 and the first movement range is the entire shift range in step S102. Assuming that the maximum shift amount is 10, the number of area integration data to be calculated is (10 + 1 + 10) × (10 + 1 + 10), which is 441, and the address of the table to be stored is prepared from 0 to 440. The storage address Ad1 when the shift position is (x1, y1) can be obtained by Ad1 = (y1 + 10) × 21 + (x1 + 10). Further, when obtaining the shift position (x2, y2) of the area integration data stored in the storage address Ad2, x2 is a value obtained by subtracting 10 from the remainder obtained by dividing Ad2 by (10 + 1 + 10), and y2 is: A value obtained by subtracting 10 from the integer value of the quotient obtained by dividing Ad2 by (10 + 1 + 10) may be used.

(Step S105)
Next, the shift amount calculation unit 104 obtains the maximum value among the region integration data of each shift position output from the region pixel data integration unit 103, and indicates the shift amount indicating the shift position corresponding to the maximum value (in this example, Since the shift position indicates the amount of movement from the reference position, the shift position matches the shift amount.) Is output to the position correction unit 105 as a shift amount for alignment.

  Hereinafter, the principle of this process will be described. As shown in FIG. 2, since the beads emit light, the bead area in the filter image has a higher luminance value than the background area. Therefore, when the bead area is set as the calculation area and the luminance values are integrated in the calculation area in the filter image, the integrated value becomes larger when the luminance value is integrated in the area including many bead areas. FIG. 6 shows luminance values of pixels on a line parallel to the X axis including the beads B and D in the filter image shown in FIG. From FIG. 6, it can be said that when the calculation area best overlaps the bead area in the filter image, the integrated value becomes the largest, and the smaller the overlap, the smaller the luminance value is mixed and the smaller the integrated value becomes. I understand. As a result, it is possible to obtain an optimum alignment shift amount by obtaining the maximum value in the area integration data of each shift position and obtaining the shift position corresponding to the maximum value.

  Also, even in a filter image taken by changing the bandpass filter, the luminance value of the bead region is higher than the luminance value of the background region, so by obtaining the shift position corresponding to the maximum value of the region integration data, An alignment shift amount can be obtained without adjusting parameters.

  Furthermore, even when the object has various light emission characteristics, it is possible to obtain the shift amount in the same manner. At this time, beads that do not have a wavelength that passes through the band pass filter used for imaging may be present in the filter image. However, in the calculation area where such beads do not emit light, the luminance value of each shift position is Since all integrated values are obtained by integrating the luminance values of the background area, the integrated values of the shift positions in the calculation area where the beads are not emitting light are the same, and the integrated values of the calculation areas of other emitting beads are the same. Since there is no influence, it is not necessary to prepare a separate determination process for checking the presence of beads, and alignment using only effective beads can be performed.

  In addition, since integration is performed in the local calculation area around the bead, there are places where light hits strongly and places where it hits weakly depending on the position of the light source, luminance unevenness occurs in the captured filter image, and the light is strong The brightness value of the hitting background area will be higher than the brightness value of the bead that is hit by light, and even in the same background area, the degree of brightness unevenness will change for each bunt pass filter. Even in this case, if the periphery of the bead is viewed locally, the bead area always has a luminance value larger than that of the background area, so that it is not affected by the luminance unevenness, and an optimum alignment shift amount can be obtained.

  Further, using the integrated value of the calculation area has the following advantages. In general, when performing alignment while looking at the filter image, humans should place more emphasis on bright beads with a difference in brightness from the background area where the deviation is well known than dark beads where the boundary between the beads and the background is not clear. However, the same thing can be said when performing calculations with a computer. The larger the difference in the brightness value between the bead area and the background area, the more the position does not match. In this case, the difference in the calculation result is large, and generally better results can be obtained by performing the positioning by weighting the region where the difference in the luminance value is large.

  And in the image processing of the present invention, in calculating the region integration data used for the alignment evaluation, a larger value is integrated into the region integration data as the pixel having a larger luminance value, so without adding new processing, Similarly, a process in which bright beads are weighted is realized. This weight has an upper limit on the luminance value to be integrated, and in the case of a luminance value greater than or equal to the upper limit, it can be adjusted by integrating the upper limit value as the luminance value of the pixel. What is necessary is just to set with the value larger than the luminance value of a background area | region, and smaller than the maximum luminance value of a filter image. The smaller the upper limit value, the more the brightness value of the dark bead region is reflected.

  When there are a plurality of maximum values in the calculation area data, a position obtained by averaging a plurality of shift positions corresponding to the maximum value is set as the shift amount. This occurs when the calculation area is smaller or larger than the bead area, and occurs when the integrated luminance value does not change even if the calculation area is shifted. FIG. 7 is a diagram illustrating a case where the calculation area is larger than the bead area. In FIG. 7, when the luminance value of the background area is uniform, the calculation integrated data becomes the same value even if the calculation area is shifted so that the bead area is included in the shaded area between the calculation area and the bead area. In such a case, the center of the region having a plurality of maximum values may be shifted, and an average position at a plurality of shift positions may be obtained.

(Step S106)
Next, the position correction unit 105 shifts the filter image 101 based on the shift amount input from the shift amount calculation unit 104 to generate a correction filter image. Since the shift amount input from the shift amount calculation unit 104 is a shift position obtained by shifting the calculation area with respect to the input filter image 101, the filter position is a value obtained by reversing the sign of the shift position with plus or minus. The image 101 is shifted. For example, when the shift amount output from the shift amount calculation unit 104 is (4, 5), each pixel (x, y) of the filter image 101 is shifted to (x-4, y-5). It will be. Then, the result of shifting in all pixels is output as an output image 106. The output image 106 is an image in which the position of the photographing target is shifted to a certain position. Therefore, an image obtained by superimposing a plurality of output images obtained by performing the processes of steps S101 to S106 on all the filter images is aligned so that the emitted beads are overlapped. Image.

  As described above, according to the image processing apparatus of the first embodiment of the present invention, for each filter image having a relative positional deviation photographed through a plurality of bandpass filters, a filter image and a calculation area The area integration data at each position is sequentially calculated while relatively shifting the position of the position, the maximum value is obtained from the calculated area integration data, the position corresponding to the maximum value is set as the shift amount, and the filter is based on the shift amount. By aligning the images, it is possible to reliably align even an image obtained by photographing an object having various emission characteristics through a plurality of bandpass filters.

  In the above description, in step S101, the calculation area calculation unit 1022 calculates the calculation area based on the bead shape information and the position information stored in the area setting data storage unit 1021, but the area setting data An image obtained by photographing the beads is stored in the storage unit 1021 as a calculation region calculation image, and the calculation region calculation unit 1022 performs binarization processing so that the bead position can be specified for the calculation region calculation image. Alternatively, a bead pixel of binary data may be set as the calculation area. In this way, by setting the calculation area from the image obtained by photographing the beads, it is possible to flexibly cope with the change in the position of the bead, which is an object serving as a reference for alignment.

  Hereinafter, specific processing in the case of using the calculation area calculation image will be described. As a calculation area | region calculation image, what image | photographed the shadow produced by irradiating light to a bead can be used, for example. By photographing the shadow of the beads in this manner, all the beads can be photographed regardless of the light emission characteristics of the beads. FIG. 8 is an example of an image captured in a state where light is irradiated from the back side of the paper surface. Since the bead is in a state of capturing a shadow, the bead region is darker than the background region.

  The processing for obtaining the calculation area from the calculation area calculation image is performed as follows. A threshold value that can separate the bead area and the background area is set in advance, and a binarization process is performed on the calculation area calculation image based on the threshold value to obtain a binarized image. That is, an image obtained by binarization is obtained with 0 being a pixel equal to or greater than the threshold and 1 being the other pixel. The created calculation area image is set as a calculation area. In this case, since the bead area is darker than the background area, the pixel value is 1.

  The threshold for separating the bead area and the background area can also be obtained from the calculation area calculation image. FIG. 9 shows the luminance distribution of the calculation area calculation image shown in FIG. The horizontal axis represents the luminance value, the left represents low luminance, and the right represents high luminance. The vertical axis represents the total number of pixels at each luminance. Since the background region is bright, it is distributed on the high luminance side, and the bead region is darker than the background, so it is distributed on the low luminance side. Therefore, in order to separate the bead area and the background area, the gradation of the valley formed by the two distributions indicated by the broken lines in the figure may be set as the threshold value. If the brightness of the calculation area calculation image changes locally, the calculation area calculation image is divided into a plurality of areas, a luminance distribution is created for each of the divided areas, and the valley gradation is obtained as a threshold value. The binarization process may be performed with the obtained threshold value. The size to be divided may be set according to the size of the calculation area calculation image. In the case of an image of about 1360 × 1024 pixels, the size may be divided by 100 × 100 pixels.

  The binarization process described above is an example. For example, a bead position may be detected by a template matching process, and the detected bead position may be set as a calculation region. In the template matching process, an image obtained by cutting only an object to be detected from the calculation area calculation image is used as a template image, the calculation area calculation image and the template image are overlapped, a difference value is obtained for each overlapped pixel, and the entire template image is obtained. All the difference results are squared individually and then added. This added value indicates the correlation between both images, and is a small value when they are the same image, and a large value when they are different. Therefore, an addition value that can be determined to be matched in advance is used as a threshold value. If the obtained addition value is smaller than the threshold value, it is determined that a pattern has been detected, and if it is greater than the threshold value, it can be determined that no pattern exists. Then, this determination process is performed for the entire calculation area calculation image by repeating the same determination while shifting the template image pixel by pixel.

  FIG. 10 is a diagram showing the template matching process described above. In the template matching process, a template image having a bead region is superimposed on the calculation region calculation image at the initial position 1001, and the determination is sequentially performed in the direction of the arrow from that position. Then, beads are detected at the position 1002 and the like.

  In this way, by performing the template matching process, even if dust is mixed in the calculation area calculation image, the dust is ignored and a calculation area in which only the bead area is extracted can be obtained.

  Note that, as the calculation region calculation image, an image taken using a bunt pass filter that transmits all wavelengths of a plurality of bunt pass filters used for measurement of the fluorescent material, or an image taken without using the bunt pass filter may be used. Good. By doing so, it is possible to obtain an image in which all the light-emitting beads are photographed regardless of the light emission characteristics of the beads. In this calculation area calculation image, since the light emitted by the beads is captured, the bead area becomes brighter than the background area. This is because the brightness of the bead area and the background area is only the reverse of the calculation area calculation image obtained by photographing the shadow that is generated when light strikes the bead shown in FIG. A calculation area can be similarly obtained using a template matching process.

  The calculation area calculation image is not limited to an image obtained by photographing a shadow formed by shining light on a bead, and an image created from a plurality of photographed filter images can also be used. In this case, the calculation area setting unit 102 has a mask pattern indicating the shape of the beads in advance, creates a plurality of filter images and creates image data in which the luminance values of the corresponding pixels of each filter image are added, This is the calculation area calculation image. Then, an area having a luminance value equal to or higher than a predetermined threshold is extracted from the calculated calculation area calculation image, and the mask pattern is arranged and arranged so that the centroid position of the extracted area matches the centroid position of the mask pattern. The mask pattern area is set as the calculation area. This mask pattern is stored as an image in the same manner as the pattern image indicating the shape information of the beads used in step S101.

  The operation of the calculation area calculation image creation process will be described with reference to FIG. FIG. 11 is a view in which three filter images are overlaid and a region where beads are present is cut out. When there is a positional shift between the filter images, the beads are overlapped at the shifted position as shown in FIG. Since the bead area is brighter than the background area, when these three filter images are overlapped and each pixel is added, the area where the three bead areas overlap together is the area with the highest luminance among the overlapping images. Become. This overlapping area is extracted by obtaining a pixel having a luminance value equal to or higher than the threshold in the calculation area calculation image created by overlapping the filter images, and the barycentric position of this area is obtained. Then, the mask pattern is arranged so that the obtained center-of-gravity position matches the center of gravity of the mask pattern, and this area is set as a calculation area. By doing so, it is possible to bring the calculation area to the center of the filter image having each positional deviation, and it is possible to perform alignment at the shortest position in all the filter images in the alignment processing.

  Note that the threshold value described above is obtained by dividing the calculation area calculation image into a plurality of areas, creating a luminance distribution in each divided area, and adding the number of pixels of the luminance distribution from the top to a predetermined value of the total number of pixels of the luminance distribution. A gradation is set such that the ratio is, for example, 2%. The size to be divided may be set in accordance with the size of the calculation area calculation image and the size of the beads. In the case of an image of about 1360 × 1024 pixels, for example, the size is divided by 50 × 50 pixels. By doing so, it is possible to extract the overlapping region with the highest luminance. Note that 2% is an example, and the ratio of the number of pixels in the overlapping region to the total number of pixels in the region divided in accordance with the calculation region calculation image may be set.

  In the first embodiment of the present invention, the region pixel data integration unit 103 adds the luminance values of the calculation regions in the filter image 101 to obtain the region integration data, but the integration is performed by adding the luminance values. A function of subtracting a constant value from the result may be added, and a value obtained by reducing the value of the area integration data may be output to the shift amount calculation unit 104. This subtracting process may be performed by subtracting the luminance value to be added when the luminance values in the calculation area are integrated, or may be subtracted from the area integration data obtained by integrating the luminance values of all the pixels in the calculation area. By doing so, the value of the area integration data can be reduced, and the memory for storing the area integration data prepared in advance can be reduced. The fixed value to be subtracted may be set to a value that can make the memory the smallest, and in the former case, the minimum value of the luminance value of each pixel in the filter image 101 may be set. In the latter case, a value obtained by multiplying the minimum luminance value of the filter image 101 by the number of pixels in the calculation area may be set.

  Further, the area pixel data integration unit 103 calculates an average luminance value for each area where the pixels of the calculation area in the filter image 101 are connected, and adds the calculated average luminance value of each area to the area integration data. It is good. The area where the pixels of this calculation area are connected is each bead area, which is equivalent to calculating the average luminance value of each bead. By this processing, the value of the area integration data can be reduced, and the memory for storing the area integration data prepared in advance can be reduced. Further, by using together with the above-described function of subtracting a constant value from the integration result obtained by adding the luminance values, it is possible to further save memory.

  In the above description, the region pixel data integration unit 103 sets the calculation region output from the calculation region calculation unit 1022 relative to the filter image 101 within the first movement range given in advance as one pixel. Although the shift processing is performed by shifting each step, the shift processing may be performed with an interval larger than one pixel. In this case, the unit of the alignment shift amount is larger than one pixel and the alignment accuracy is lowered, but the calculation amount is reduced, so that the processing can be performed at high speed. For example, when the shift amount is 2 pixels, the number of times of obtaining the region integration data in the vertical direction and the horizontal direction is halved compared to the case of 1 pixel, so the calculation amount for obtaining the entire region integration data is 4 minutes. 1 This processing is effective when the accuracy of one pixel unit is not required and the processing time is more weighted than the accuracy.

(Second Embodiment)
FIG. 12 is a block diagram illustrating a schematic configuration of a main part of the image processing apparatus according to the second embodiment of the present invention. The image processing apparatus in FIG. 12 includes a calculation region setting unit 102, a region pixel data integration unit 103, a shift amount calculation unit 104, and a position correction unit 105. The same components as those described in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. The difference from the configuration of the image processing apparatus according to the first embodiment shown in FIG. 1 is that the shift amount calculated by the shift amount calculation unit 104 is output to the region pixel data integration unit 103.

  In this image processing apparatus, first, when the region integration data is calculated by the region pixel data integration unit 103, the calculation region is moved relative to the processing target image at an interval larger than one pixel. That is, the area integration data is calculated by roughly setting the shift position. Then, after obtaining the shift position where the area integration data shows the maximum value from the shift amount calculation unit 104, the calculation area is moved at a narrower interval near the shift position, and the area integration data is calculated again. Therefore, it is possible to reduce the processing for obtaining the shift position where the area integration data is maximized.

  Next, the operation of the image processing apparatus according to the second embodiment of the present invention will be described in detail with reference to FIG. FIG. 13 is a flowchart for explaining image processing performed by the image processing apparatus according to the second embodiment of the present invention. The flowchart of FIG. 13 includes steps S201 to S207, where step S201 is a calculation region setting step, step S2020 consisting of steps S202 to S204 is a region pixel data integrating step, and step S2050 consisting of steps S205 and S206 is step S2050. Step S207 shows a shift amount calculation step, and step S207 shows a position correction step.

(Step S201)
First, the calculation area calculation unit 1022 calculates a calculation area for performing alignment calculation on the input filter image 101 based on information stored in the area setting data storage unit 1021. The calculation method of the calculation area is the same as that described in step S101 in FIG.

(Step S202)
Next, in the region pixel data integration unit 103, the calculation region output from the calculation region calculation unit 1022 with respect to the filter image 101 is relatively larger than one pixel in the first movement range given in advance. Shift processing is performed with a gap. Here, the description will be made assuming that this interval is 4 pixels.

(Step S203)
Next, in the region pixel data integration unit 103, the filter image 101 and the calculation region image at the reference position or the shifted calculation region image are superimposed, and all the luminance values of the pixels of the filter image 101 that overlap the calculation region are added. Area integration data is calculated. Note that the method for calculating the area integration data is the same as that described in step S103 in FIG.

(Step S204)
These steps S202 and S203 are repeated at each shift position in the first movement range, the area integration data at each shift position in the first movement range is sequentially calculated, and the area integration data at each calculated shift position is shifted. The data is output to the quantity calculation unit 104.

(Step S205)
Next, the shift amount calculation unit 104 obtains the maximum value among the region integration data of each shift position output from the region pixel data integration unit 103, and obtains the shift position (shift amount) corresponding to the maximum value.

(Step S206)
Then, it is determined whether or not the shift interval (the movement interval of the calculation area with respect to the filter image 101) is one pixel when the shift amount calculation unit 104 obtains the shift position. In the case of one pixel, the shift position obtained by the shift amount calculation unit 104 is output as it is to the position correction unit 105 as an alignment shift amount. On the other hand, in the case of other than one pixel, the shift position obtained by the shift amount calculation unit 104 is output to the region pixel data integration unit 103, and the steps S202 to S205 are repeated again.

  When step S202 is performed again, the shift interval is set to half of the previous shift interval. In this example, since the first shift interval is 4 pixels, the shift interval is set to 2 pixels for the second time, and 1 pixel is set for the third time. The first movement range in which the calculation area is shifted relative to the filter image 101 is a range having the previous shift interval in the vertical and horizontal directions with the shift position output from the shift amount calculation unit 104 as the center. Therefore, in this example, the second time is a square range with 4 pixels vertically and horizontally from the shift position, and the third time is a square range with 2 pixels vertically and horizontally from the shift position.

(Step S207)
Next, the position correction unit 105 shifts the filter image 101 based on the shift amount output from the shift amount calculation unit 104 to generate a correction filter image. The method for shifting the filter image is the same as that described in step S106 in FIG.

  As described above, according to the image processing apparatus of the second embodiment of the present invention, the calculation area is shifted with respect to the filter image by an interval larger than one pixel, and the shift amount of the filter image at that time is changed. After roughly obtaining the indicated shift position, in order to obtain a more accurate shift position within an area limited to the vicinity of the shift position obtained in the previous processing while sequentially reducing the interval of shifting the calculation area with respect to the filter image By doing so, it is not necessary to obtain area integration data at all shift positions of the first movement range in which the calculation area is shifted with respect to the filter image, the calculation amount can be greatly reduced, and the speed can be increased.

  Further, when the process from step S202 to step S205 is repeated, if the area pixel data integration unit 103 already has the area integration data of the shift position to be obtained on the memory, the integration process is not performed and the integration process is not performed. The area integration data may be used. By doing this, it is possible to prevent calculation area data at the same position from being repeatedly obtained and to increase the speed when repeating the process of calculating the calculation area in order while reducing the interval at which the calculation area is shifted with respect to the filter image. .

(Third embodiment)
FIG. 14 is a block diagram illustrating a schematic configuration of main parts of an image processing apparatus according to the third embodiment of the present invention. The image processing apparatus in FIG. 14 includes a calculation region setting unit 102, a region pixel data integration unit 103, a shift amount calculation unit 104, a position correction unit 105, and a determination unit 307. The same components as those described in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. The difference from the configuration of the image processing apparatus according to the first embodiment shown in FIG. 1 is that a determination unit 307 is newly added. The determination unit 307 determines whether or not the input filter image 101 can be aligned based on the region integration data output from the region pixel data integration unit 103, and outputs the determination result to the shift amount calculation unit 104. It is.

  Next, the operation of the image processing apparatus according to the third embodiment of the present invention will be described in detail with reference to FIG. FIG. 15 is a flowchart for explaining image processing performed by the image processing apparatus according to the third embodiment of the present invention. The flowchart of FIG. 15 includes steps S301 to S308, where step S301 is a calculation area setting step, step S3020 including steps S302 to S304 is an area pixel data integrating step, and step S3050 including steps S305 to S307 is step S3050. Step S308 shows a shift amount calculation step, and step S308 shows a position correction step.

(Step S301)
First, the calculation area calculation unit 1022 calculates a calculation area for performing alignment calculation on the input filter image 101 based on information stored in the area setting data storage unit 1021. The calculation method of the calculation area is the same as that described in step S101 in FIG.

(Step S302)
Next, the region pixel data integration unit 103 shifts the calculation region input from the calculation region calculation unit 1022 relative to the filter image 101 one pixel at a time within a first movement range given in advance. I do. Since this shift process is the same as that described in step S102 in FIG. 3, the description thereof is omitted here.

(Step S303)
Next, in the region pixel data integration unit 103, the filter image 101 and the calculation region image at the reference position or the shifted calculation region image are superimposed, and all the luminance values of the pixels of the filter image 101 that overlap the calculation region are added. Area integration data is calculated. The calculation method of the calculation area is the same as that described in step S103 in FIG.

(Step S304)
These steps S302 and S303 are repeated at each shift position of the first movement range given in advance, and the area integration data of each shift position in the first movement range is sequentially calculated. The area integration data is output to the shift amount calculation unit 104 and the determination unit 307.

(Step S305)
Next, the determination unit 307 determines whether or not the input filter image 101 can be aligned based on the region integration data output from the region pixel data integration unit 103.

(Step S306)
As a result of the determination by the determination unit 307, when the input filter image 101 can be aligned, the shift amount calculation unit 104 determines whether the shift amount calculation unit 104 outputs the region integration data at each shift position. Then, the maximum value is obtained, and the shift position corresponding to the maximum value is output to the position correction unit 105 as the shift amount.

(Step S307)
On the other hand, as a result of the determination by the determination unit 307, if the input filter image 101 cannot be aligned, the shift amount calculation unit 104 sets the shift amount to 0 so that the alignment is not performed at all. To 105.

(Step S308)
Next, the position correction unit 105 shifts the filter image 101 based on the shift amount input from the shift amount calculation unit 104 to generate a correction filter image. Note that this shift method is the same as that described in step S106 in FIG.

  As described above, according to the image processing apparatus of the third embodiment of the present invention, in the determination unit 307, the input filter image 101 is input based on the region integration data output from the region pixel data integration unit 103. In the case where alignment is possible, the shift amount calculation unit 104 calculates the shift amount from the area integration data, and the position correction unit 105 performs shift processing for alignment. On the other hand, when position correction is impossible, the shift amount calculation unit 104 sets the shift amount to 0 and the position correction unit 105 does not perform shift processing, thereby preventing erroneous alignment processing.

  Hereinafter, the determination process in the determination unit 307 will be described in detail.

  As a cause of erroneous alignment, the input filter image 101 does not have a bead serving as a reference for alignment in the filter image, or even if a bead exists, light is hardly emitted and the luminance level of the bead May be almost the same as the background brightness level. This occurs when the bandpass filter used for imaging does not pass light of the wavelength emitted by the beads. In such a case, it is not possible to determine where the position is matched, so it is better not to perform the positioning process.

  Therefore, the determination unit 307 obtains the maximum value and the minimum value among the region integration data of each shift position output from the region pixel data integration unit 103, and performs alignment based on the ratio of the obtained maximum value and minimum value. Judgment is made.

  For example, when the value obtained by dividing the maximum value by the minimum value (hereinafter referred to as the maximum / minimum division value) is smaller than a predetermined threshold value, it is determined that the position correction of the filter image 101 is impossible, In other cases, it is determined that the position of the filter image 101 can be corrected. As shown in FIG. 2, in the filter image, the bead area has a larger luminance value than the background area. Therefore, the larger the difference in the luminance level between the bead area and the background area, the larger the maximum and minimum division value. The maximum / minimum division value decreases as the difference in luminance level decreases. When the luminance levels of the bead area and the background area are the same, the maximum and minimum division value is 1.0. Therefore, the difference between the luminance levels of the bead area and the background area can be evaluated by obtaining the maximum / minimum division value. When the maximum / minimum division value is smaller than the threshold value, the alignment of the filter image 101 is impossible. Can be determined.

  Note that the threshold value given in advance here is a decimal value of 1.0 or more, and may be set according to the difference in luminance level between the bead area and the background area of the filter image, and a plurality of filter images are used in advance. The maximum / minimum division value is calculated, and the smallest value among the maximum / minimum division values when the alignment is successful may be set.

  The determination using the maximum / minimum division value is the same as the bead region described above even when the reference beads for alignment do not exist in the filter image, or when beads are present but hardly emit light. This is effective when the difference in the brightness level of the background area becomes small. For example, even when the exposure time is made too long and the brightness value of the entire image is saturated, erroneous alignment processing can be prevented. .

  Furthermore, when the ratio of the entire calculation region is small even if the beads are intensely emitted, for example, when only one of the 100 beads is emitted, alignment is performed by evaluating the entire image. Since the reliability is poor in the case of one, it is better not to perform alignment in general. However, even in such a case, the maximum / minimum division value becomes small, so that the alignment of the filter image is impossible. Can be determined. Conversely, even if the light emission amount of the entire bead is small and there is not much difference in brightness between the bead area and the background area, reliability is improved when the ratio of the entire calculation area is large. However, in such a case, since the maximum / minimum division value becomes large, it can be determined that the position of the filter image can be corrected.

  Another possible cause of misalignment is when the apparatus vibrates during photographing and the shape of the beads in the filter image is deformed. FIG. 16 shows a filter image when vibration occurs. In FIG. 16, rolling occurs and the beads are horizontally long. In such a case, since it is not possible to specify which shift position is the matching position, it is better not to perform the alignment process.

  Therefore, the determination unit 307 shifts the region integration data at each shift position output from the region pixel data integration unit 103 with a value greater than or equal to a predetermined ratio with respect to the value indicating the maximum value in the region integration data. A position is extracted, and whether or not the processing target image is aligned is determined based on the extracted distribution of shift positions. For example, if a feature amount is calculated from an area occupied by the extracted shift position (hereinafter referred to as a predetermined ratio area), and the calculated feature amount satisfies a predetermined condition, the filter image 101 cannot be aligned. Otherwise, it is determined that the position of the filter image 101 can be corrected.

  FIG. 17 is a diagram illustrating a predetermined ratio area when there is no vibration that can be aligned. FIG. 17 (a) displays area integration data at each shift position on a two-dimensional plane. The shift amount of the calculation area with respect to the filter image is indicated by the horizontal axis indicating the X-direction shift amount and the vertical axis indicating the Y-direction shift. The predetermined ratio area in the case where the position can be aligned is shown. FIG. 17B shows the area integration data on the X axis in FIG. 17A, where the horizontal axis represents the shift amount in the X direction and the vertical axis represents the value of the area integration data. A shift position having a maximum value and a position having a value greater than a predetermined ratio are shown. The predetermined ratio may be set to a value at which the characteristic of the predetermined ratio area appears, and is set to 80% here.

  In this way, by obtaining a shift position having a value greater than or equal to a predetermined ratio with respect to the value indicating the maximum value in the area integration data as a predetermined ratio area, the brightness of the object changes and the maximum of the area integration data is obtained. Even if the value changes, the shape of the predetermined ratio area is exactly the same. In FIG. 17B, the value of the region integration data decreases as the overlap between the calculation region and the bead region in the filter image decreases, and thus the value decreases as the distance from the maximum value position increases. Therefore, when the predetermined ratio area is extracted, the circle is centered on the position of the maximum value as shown in FIG.

  However, in the case of a filtered filter image, the beads are elliptical. Therefore, when a predetermined ratio area at a position having a value equal to or higher than a predetermined ratio is extracted, the predetermined ratio area becomes an ellipse similarly to the shape of the beads. This is because the calculation area is included in the bead area, and even if the calculation area is shifted in the major axis direction, all the pixels in the calculation area become the bead area. This is because it becomes longer than when there is no vibration that can be matched. FIG. 18 shows a predetermined ratio area of the filter image photographed during the vibration shown in FIG.

  As can be seen from the above description, it is possible to determine whether or not alignment is possible by obtaining a difference from the shape when alignment is possible in the predetermined ratio area. The difference in shape is determined by calculating a feature amount indicating the shape of the predetermined ratio region and using the calculated feature amount.

  As the feature amount indicating the shape of the predetermined ratio area, the size of the area of the predetermined ratio area can be used. Then, a range that can be taken by the area value when alignment is possible is obtained in advance, and if it is outside the range, it is determined that alignment of the filter image is impossible. The area value of the predetermined ratio area can be obtained by counting the number of shift positions in the predetermined ratio area.

  Further, the vertical and horizontal widths of the predetermined ratio area can be used as the feature amount. Then, the range of the vertical width and the horizontal width when alignment is possible is obtained in advance, and if one or both of the vertical width and horizontal width are outside the range, it is determined that the filter image cannot be aligned. To do. The vertical width and the horizontal width are obtained as the maximum value and the minimum value for the X and Y coordinates of the shift position in the predetermined ratio area, respectively, the vertical width is the absolute value obtained by subtracting the minimum value from the maximum value of the Y coordinate, The absolute value can be obtained by subtracting the minimum value from the maximum coordinate value.

  Furthermore, a value obtained by calculating the vertical width and the horizontal width of the predetermined ratio area can also be used as the feature amount. For example, the sum of the vertical width and the horizontal width or the product of the vertical width and the horizontal width is used. With these values, the size of the predetermined ratio area can be evaluated.

  Further, a difference between the vertical width and the horizontal width, or a ratio between the vertical width and the horizontal width can be used. These values can be used to evaluate the roundness of a given area, the roundness can be evaluated by the absolute amount of the shift position based on the difference between the vertical width and the horizontal width, and the roundness can be evaluated based on the ratio of the vertical width to the horizontal width. The degree can be evaluated as a ratio.

  Note that the determination may be performed by simultaneously using a plurality of values obtained by calculating the area value, the vertical width, the horizontal width, or the vertical width and the horizontal width as the feature amount indicating the shape of the predetermined ratio area. Further, the determination unit 307 may perform the determination using the above-described method using the maximum / minimum division value and the method using the feature amount indicating the shape of the predetermined ratio area at the same time. In this case, alignment is possible as an output of the determination unit 307 only when it is determined that both methods can be aligned.

  Further, in the image processing apparatus according to the third embodiment, when it is determined that the alignment is impossible by the determination unit 307 and the alignment process is not performed, the filter image is re-taken by changing the shooting position, and the image is re-taken. The alignment process may be repeated on the filter image a plurality of times until the alignment of the filter image is performed or the designated number of times is reached. The designated number of times is a limit number of repetitions of the alignment process, and is set to end the process when the alignment cannot be performed no matter how many times the image is taken. By doing so, it is possible to automate the retry in the alignment process.

(Fourth embodiment)
Next, an image processing apparatus according to a fourth embodiment of the present invention will be described. The block diagram showing the configuration of the image processing apparatus according to the fourth embodiment of the present invention is the same as that shown in FIG. 1, but the operations of the area pixel data integrating unit 103 and the shift amount calculating unit 104 are different. In other words, when the input filter image 101 is shifted larger than the first movement range given in advance, incorrect alignment is not performed.

  The area pixel data integration unit 103 of the image processing apparatus according to the fourth embodiment of the present invention performs filtering within a predetermined range (hereinafter referred to as a second movement range) that is greater than a first movement range given in advance. A function of shifting the calculation area relative to the image 101 is provided, and area integration data at each shift position within the second movement range is sequentially calculated in the same manner as in steps S102 and S105 shown in FIG.

  FIG. 19 illustrates a first movement range and a second movement range given in advance. The size of the second movement range may be larger than the first movement range, and is set so that the centers of the first movement range and the second movement range coincide. However, if the second movement range is too large, the amount of calculation increases according to the size, so here the length of one side is set to be 1.2 times the first movement range. . This magnification is an example, and may be set according to the amount of deviation in the filter image. The larger the amount of deviation, the larger the magnification is set.

  In the shift amount calculation unit 104 according to the fourth embodiment of the present invention, the maximum value among the region integration data of each shift position output from the region pixel data integration unit 103 is obtained in the same manner as in step S105 shown in FIG. The obtained shift position corresponding to the maximum value is output to the position correction unit 105 as a shift amount for alignment. However, if the shift position corresponding to the maximum value is outside the first movement range, this indicates that there is no shift position for matching the filter image 101 within the first movement range. Therefore, the shift amount is set to 0 so that the filter image 101 is not aligned, and is output to the position correction unit 105.

  As described above, according to the image processing apparatus of the fourth embodiment of the present invention, it is possible to prevent erroneous alignment with respect to a filter image that is shifted more than the first movement range. be able to. Furthermore, it is possible to obtain information on how much the filter image is shifted from the calculated shift amount to be larger than the first movement range. And it is also possible to perform adjustment when setting the first movement range in advance using this information.

  In the shift amount calculation unit 104 of the image processing apparatus according to the fourth embodiment of the present invention, when the obtained shift position is the maximum shift position of the second movement range, the predetermined second movement range. A process of enlarging a certain amount may be performed. More specifically, when the shift position corresponding to the maximum value is the outer peripheral position of the second movement range shown in FIG. 19, the second movement range is uniformly expanded from the center by a certain amount. The second movement range after the enlargement may be set again so that the obtained shift position does not become the maximum shift position of the second movement range. As an example, one side of the original second movement range is set as one side. .2 times shall be set.

  Further, in the area integration data obtained at each shift position, as shown in FIG. 17B, the area integration data decreases as the distance from the maximum shift position increases, so that the maximum shift position is the second movement range. In the case of the outer peripheral position, it can be said that there is a possibility that there is a region integration data having a larger value outside the outer peripheral position. Conversely, the maximum shift position is not the outer peripheral position of the second movement range. In this case, the maximum shift position can be said to be the most consistent position. Therefore, when the obtained shift amount is the maximum shift amount of the second movement range, the shift amount of the filter image that is further shifted is obtained by enlarging the predetermined second movement range by a certain amount. Is possible.

  Further, in the shift amount calculation unit 104, when the obtained shift position is the maximum shift position of the second movement range, the predetermined second movement range is shifted when the obtained shift position is used as the shift amount. A certain amount may be enlarged only in the direction. More specifically, when the shift position corresponding to the maximum value is the outer peripheral position on the right side of the second movement range shown in FIG. 19, the second movement range is expanded by a certain amount only in the right direction. Although the amount of calculation increases as the second movement range becomes larger, the second movement range can be enlarged with a necessary minimum size by limiting the direction of expansion.

  Further, as described above, when the maximum value shift position is not the outer peripheral position of the second movement range, it can be said that the maximum value shift position is the best match position, and thus the shift amount calculation unit 104 obtains it. If the shift position is not the maximum shift position of the second movement range, the obtained shift position is positioned even if the obtained shift position is outside the first movement range given in advance to the area pixel data integrating unit 103. Positioning may be performed using a shift amount of alignment. By doing so, alignment is possible even if the first movement range given in advance to the area pixel data integration unit 103 is set narrow.

  Furthermore, in the image processing apparatus according to the fourth embodiment of the present invention, when the alignment processing is not performed, the filter position is changed and the filter image is re-taken. The alignment process may be repeated a plurality of times until the filter image is aligned or until the designated number of times is reached. The designated number of times is a limit number of repetitions of the alignment process, and is set to end the process when the alignment cannot be performed no matter how many times the image is taken. By doing so, it is possible to automate the retry in the alignment process.

(Fifth embodiment)
Next, an image processing apparatus according to a fourth embodiment of the present invention will be described. The block diagram showing the configuration of the image processing apparatus according to the fifth embodiment of the present invention is the same as that shown in FIG. 1, but the operations of the region pixel data integrating unit 103 and the shift amount calculating unit 104 are different. That is, a plurality of filter images are photographed using the same bunt pass filter at a plurality of different photographing positions, and alignment is performed using a filter image with the best result among the photographed filter images. .

  First, a plurality of filter images are photographed using the same bunt pass filter at a plurality of different photographing positions. In this case, about three filter images are sufficient to be photographed.

  The area pixel data integration unit 103 of the image processing apparatus according to the fifth embodiment of the present invention further has a function of storing area integration data for a plurality of filter images, and for each filter image. The calculation area is relatively shifted, area integration data is obtained at each shifted shift position, and area integration data obtained at each shift position is stored for each filter image.

  Then, in the shift amount calculation unit 104 of the image processing apparatus according to the fifth embodiment of the present invention, the maximum value and the minimum value are obtained from the area integration data stored for each filter image, and the maximum value is minimized. The shift amount is calculated from the area integration data of the filter image having the largest value divided by the value, and the calculated shift amount is output to the position correction unit 105. Since the method for calculating the shift amount from the area integration data is the same as that described in step S105 in FIG. 3, the description thereof is omitted here.

  As described above, according to the image processing apparatus of the fifth embodiment of the present invention, as described above, the value obtained by dividing the maximum value and the minimum value in the area integration data is the luminance level of the bead area and the background area. The larger the difference is, the larger the difference is. The larger the brightness level difference between the bead area and the background area is, the more accurate the alignment is. Therefore, the area integrated data of the filter image having the largest value obtained by dividing the maximum by the minimum By calculating the shift amount from the above, it is possible to perform alignment with higher accuracy.

  The image processing apparatus of the present invention accurately aligns a positional shift in each image captured through a bandpass filter that transmits light in a plurality of specific wavelength bands without being affected by the light emission characteristics of the object. The present invention is applicable to various processing apparatuses that measure fluorescent substances having different emission wavelengths using a plurality of bandpass filters.

1 is a block diagram showing a schematic configuration of a main part of an image processing apparatus according to a first embodiment of the present invention. The figure which shows an example of the filter image input into the image processing apparatus of the 1st Embodiment of this invention The flowchart explaining the image processing which the image processing apparatus of the 1st Embodiment of this invention performs The figure which shows the calculation area | region in the filter image input into the image processing apparatus of the 1st Embodiment of this invention The figure which shows the calculation area | region protruded from the filter image input into the image processing apparatus of the 1st Embodiment of this invention The figure which shows the luminance value of the pixel on the line parallel to the X-axis of the filter image input into the image processing apparatus of the 1st Embodiment of this invention The figure which shows an example when a calculation area | region is large with respect to the bead area | region of the filter image input into the image processing apparatus of the 1st Embodiment of this invention The figure which shows an example of the calculation area | region calculation image which image | photographed the shadow produced by irradiating light to the bead memorize | stored in the calculation area | region setting part of the image processing apparatus of the 1st Embodiment of this invention The figure which shows the luminance distribution of an example of the calculation area calculation image which image | photographed the shadow produced by irradiating the light memorize | stored in the bead memorize | stored in the calculation area setting part of the image processing apparatus of the 1st Embodiment of this invention The figure which shows the template matching process for the calculation area setting of the image processing apparatus of the 1st Embodiment of this invention The figure explaining the creation process of the calculation area calculation image for the calculation area setting of the image processing apparatus of the 1st Embodiment of this invention The block diagram which shows the principal part schematic structure of the image processing apparatus of the 2nd Embodiment of this invention. The flowchart explaining the image processing which the image processing apparatus of the 2nd Embodiment of this invention performs The block diagram which shows the principal part schematic structure of the image processing apparatus of the 3rd Embodiment of this invention. The flowchart explaining the image processing which the image processing apparatus of the 3rd Embodiment of this invention performs The figure which shows an example of the image when a vibration occurs at the time of imaging | photography of the filter image which performs alignment The figure explaining the distribution of the shift area which the filter image which can be aligned extracted The figure which shows the range of the shift area | region which the filter image extracted when the vibration shown in FIG. 16 occurred The figure which shows the relationship between the 1st movement range which relatively moves a calculation area | region in order to perform the process which calculates | requires area | region integrated data, and a 2nd movement range. Flowchart of registration processing in conventional image registration method

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Filter image 102 Calculation area | region setting part 103 Area pixel data integration part 104 Shift amount calculation part 105 Position correction part 106 Output image 307 Judgment part 1001 Initial pattern matching position 1002 Position 1021 where pattern was detected Area setting data storage part 1022 Calculation area Calculation unit

Claims (36)

An image processing apparatus that performs alignment processing of a plurality of images,
For each image to be aligned, a calculation area setting unit that sets a calculation area that is a pixel area for performing calculation processing for alignment;
A region pixel data integration unit that calculates region integration data obtained by adding luminance values of pixels in the calculation region in the processing target image at a plurality of shift positions that are moved relative to the processing target image; ,
A shift amount calculation unit that calculates a shift amount indicating the shift position corresponding to the maximum value or the minimum value among the plurality of region integration data calculated in the region pixel data integration unit;
An image processing apparatus comprising: a position correction unit that generates a corrected image of the processing target image based on the shift amount calculated by the shift amount calculation unit. The image processing apparatus according to claim 1,
The image processing apparatus is an image processing device in which the calculation area set by the calculation area setting unit is an area of an object serving as a reference for alignment in the processing target image. The image processing apparatus according to claim 1 or 2,
The calculation area set by the calculation area setting unit is an area occupied by one pixel value of image data obtained by binarizing a calculation area calculation image that is image data for obtaining the calculation area. Processing equipment. The image processing apparatus according to claim 3,
The calculation region calculation image is an image processing device that is an image obtained by photographing a shadow when an object that is a reference for alignment in the processing target image is irradiated with light. The image processing apparatus according to claim 3,
The image processing device is an image processing device in which the calculation area calculation image is an image in which all the objects serving as the alignment reference in the processing target image are captured. The image processing apparatus according to claim 5, wherein
The image to be aligned is a photographed image photographed by switching a plurality of filters having different passbands,
The calculation region calculation image is an image processing device that is an image captured using a filter that passes through all the passbands of the plurality of filters or without using a filter. The image processing apparatus according to claim 1,
The image to be aligned is a photographed image photographed by switching a plurality of filters having different passbands,
The calculation area setting unit has a mask pattern indicating the shape of the image pattern of the object to be photographed, and image data obtained by superimposing a plurality of images of the photographed image and adding luminance values of corresponding pixels of the images. An image in which the mask pattern is arranged so that the centroid position of the region extracted based on the threshold given in advance in the image data and the centroid position of the mask pattern coincide with each other, and the mask pattern region is used as the calculation region Processing equipment. The image processing apparatus according to any one of claims 1 to 7,
The area pixel data integration unit is an image processing device that adds the luminance value of the calculation area determined according to the range to be relatively moved among the calculation areas set by the calculation area setting unit. The image processing apparatus according to claim 8, wherein
The area pixel data integrating unit is an image processing apparatus that adds the luminance values of the calculation area corresponding to a portion excluding a peripheral portion of the processing target image. The image processing apparatus according to any one of claims 1 to 7,
The calculation region setting unit is an image processing device that sets the calculation region in a range determined according to the relatively moved range. The image processing apparatus according to claim 10,
The area for setting the calculation area is an image processing apparatus corresponding to a portion excluding a peripheral portion of the processing target image. The image processing apparatus according to any one of claims 1 to 11,
The area pixel data integration unit is an image processing apparatus that moves the calculation area relative to the processing target image at an interval larger than one pixel. The image processing apparatus according to any one of claims 1 to 12,
The area pixel data integration unit includes an image processing apparatus having a table having a storage address corresponding to the shift position, and storing area integration data at the shift position at the address position. An image processing apparatus according to any one of claims 1 to 13,
The area pixel data integration unit is an image processing apparatus in which a value obtained by subtracting a predetermined value from a result of adding luminance values of pixels in the calculation area is used as the area integration data. 15. The image processing apparatus according to claim 1, wherein
The region pixel data integration unit calculates an average luminance value for each region connected to the calculation region in the processing target image, and uses the value obtained by adding all the calculated average luminance values of the regions as region integration data. Processing equipment. The image processing apparatus according to any one of claims 1 to 15,
The shift amount calculation unit uses an average value of a plurality of shift amounts corresponding to the maximum value or the minimum value as a shift amount when there are a plurality of maximum values or minimum values of the region integration data calculated by the region pixel data integration unit. Image processing device. The image processing apparatus according to any one of claims 1 to 16,
The shift amount calculation unit outputs a maximum value or a minimum value of a shift amount when the calculation region is moved relative to the processing target image at an interval larger than one pixel to the region pixel data integration unit. An image processing apparatus. The image processing device according to claim 17,
The region pixel data integration unit is configured to display the processing region in a narrower range and at a smaller interval in the vicinity of the movement position corresponding to the maximum value or the minimum value of the shift amount input from the shift amount calculation unit. An image processing apparatus that recalculates the area integration data when moved relative to the image. The image processing device according to claim 18, wherein the region pixel data integration unit omits the calculation of the region integration data at the already calculated movement position when calculating the region integration data again. The image processing apparatus according to any one of claims 1 to 19,
An image processing apparatus further comprising: a determination unit that determines whether or not the processing target image can be aligned based on the region integration data calculated by the region pixel data integration unit. The image processing apparatus according to claim 20, wherein
The determination unit obtains a maximum value and a minimum value in the region integration data calculated by the region pixel data integration unit, and performs alignment of the processing target image based on a ratio between the calculated maximum value and minimum value. An image processing apparatus that determines whether or not it is possible. The image processing apparatus according to claim 20, wherein
The determination unit obtains a maximum value or a minimum value from the region integration data calculated by the region pixel data integration unit, and extracts a shift position having a predetermined ratio with respect to the calculated maximum value or minimum value. An image processing apparatus that determines whether or not the processing target image can be aligned based on the extracted distribution of shift positions. The image processing apparatus according to claim 22, wherein
The determination unit is an image processing apparatus that determines whether or not alignment of the processing target image is possible based on a size of the extracted range of shift positions. The image processing apparatus according to claim 22, wherein
The determination unit is an image processing apparatus that determines whether or not the processing target image can be aligned based on at least one of a vertical width and a horizontal width of the extracted shift position existence range. The image processing apparatus according to claim 22, wherein
The determination unit is an image processing apparatus that determines whether or not alignment of the processing target image is possible based on a sum of a vertical width and a horizontal width of the extracted shift position existence range. The image processing apparatus according to claim 22, wherein
The determination unit is an image processing apparatus that determines whether or not the processing target image can be aligned based on a product of a vertical width and a horizontal width of the extracted range of shift positions. The image processing apparatus according to claim 22, wherein
The determination unit is an image processing apparatus that determines whether or not alignment of the processing target image is possible based on a difference between a vertical width and a horizontal width of the extracted shift position existence range. The image processing apparatus according to claim 22, wherein
The determination unit is an image processing apparatus that determines whether or not the processing target image can be aligned based on a ratio of a vertical width and a horizontal width of the extracted range of shift positions. The image processing apparatus according to any one of claims 20 to 28, wherein:
The image to be aligned is a photographed image photographed by switching a plurality of filters having different passbands,
When the determination unit determines that positioning is impossible, the image processing apparatus uses the same filter, changes the shooting position, changes to a shot image, and repeats the positioning process. An image processing apparatus according to any one of claims 1 to 29,
The region pixel data integration unit calculates the region integration data at a plurality of shift positions in which the calculation region is moved relative to the processing target image in a second movement range.
The shift amount calculation unit obtains a maximum value or a minimum value of the area integration data in the second movement range, and a shift position corresponding to the obtained maximum value or minimum value is smaller than the second movement range. If it is within the movement range of 1, the shift amount indicating the shift position corresponding to the maximum value or the minimum value is output as the shift amount, and if it is outside the first movement range, the shift amount is set to 0. As an image processing apparatus. The image processing apparatus according to claim 30, wherein
When the shift position corresponding to the maximum value or the minimum value of the region integration data in the second movement range is the maximum shift position of the second movement range, the region pixel data integration unit is configured to perform the second movement. An image processing device that expands the range by a certain amount. The image processing apparatus according to claim 31, wherein
The area pixel data integration unit corresponds to the shift position when the shift position corresponding to the maximum value or the minimum value of the area integration data in the second movement range is the maximum shift position of the second movement range. An image processing apparatus that expands the second movement range by a certain amount only in the direction of the shift amount to be performed. The image processing apparatus according to any one of claims 30 to 32, wherein:
The image to be aligned is a photographed image photographed by switching a plurality of filters having different passbands,
When the shift position corresponding to the maximum value or the minimum value of the area integration data in the second movement range obtained in the area pixel data integration unit is outside the first movement range, the same filter is used, An image processing apparatus that changes the image to a photographed image and repeats the alignment process. The image processing apparatus according to claim 30, wherein
The shift amount calculation unit is a case where a shift position corresponding to the maximum value or the minimum value of the area integration data in the second movement range is outside the first movement range, and in the second movement range. An image processing apparatus that outputs a shift amount indicating the shift position as the shift amount when it is not the maximum shift position. An image processing apparatus according to any one of claims 1 to 34, wherein
The image processing apparatus, wherein the image to be aligned is a photographed image photographed by switching a plurality of filters having different pass bands. 36. The image processing device according to claim 35, wherein
The image to be aligned includes images taken using the same filter at a plurality of different shooting positions,
The region pixel data integration unit has a function of storing the region integration data of a plurality of images,
The region pixel data integration unit shifts the calculation region relative to each filter image, obtains region integration data at each shifted position, stores the region integration data of each filter image,
The shift amount calculation unit obtains a maximum value and a minimum value in the area integration data stored for each image photographed using the same filter, and based on a ratio between the obtained maximum value and minimum value An image processing apparatus that selects a specific image and calculates the shift amount of the image.