CN114648443A

CN114648443A - Image processing method, image processing apparatus, and computer-readable storage medium

Info

Publication number: CN114648443A
Application number: CN202011521229.1A
Authority: CN
Inventors: 卢坚; 赵红梅; 温朝凯; 赵陆伟; 赵腾; 潘茗茗
Original assignee: Guangyuan Technology Shenzhen Co ltd
Current assignee: Guangyuan Technology Shenzhen Co ltd
Priority date: 2020-12-21
Filing date: 2020-12-21
Publication date: 2022-06-21

Abstract

An image processing method, an image processing apparatus, and a storage medium are disclosed. The method comprises the following steps: intercepting an overlapping region for an N-dimensional image pair which is adjacent to the object in position in a plurality of N-dimensional images of each part of the object; respectively blocking the overlapped areas of the adjacent N-dimensional image pairs in N directions to obtain two groups of N-dimensional block images in each direction; projecting the two groups of N-dimensional block images in corresponding directions to obtain two groups of two-dimensional block images; matching two groups of two-dimensional block images to obtain an offset with the highest confidence in the corresponding direction; if the offset is larger than the preset threshold, repeating the steps according to the offset until the offset is smaller than the preset threshold; repeating the above steps for other adjacent N-dimensional image pairs of the plurality of N-dimensional images; the size of the plurality of N-dimensional images after being stitched together is recalculated based on the offset of all the N-dimensional image pairs, and the overlapping regions between the N-dimensional image pairs are fused.

Description

Image processing method, image processing apparatus, and computer-readable storage medium

Technical Field

The present disclosure relates to the field of image processing, and in particular to image stitching techniques.

Background

Obtaining high resolution microscopic images of large samples has wide application requirements in the biomedical field. In the case of a relatively large sample size, there are two problems. The first problem is that the microscope cannot capture the entire sample in the X-Y direction of view. A second problem is that the laser cannot penetrate the object in the Z-direction due to the relatively thick sample. The first problem is solved by translating the sample stage of the slide microscope in the X-Y-Z directions, thereby performing a multi-point scan or a tiled scan of multiple locations. The second problem is solved by rotating the sample stage to perform multi-angle scanning. Finally, the images of the multi-point or tiled scan and the multi-angle scan are stitched and fused to form a larger field of view of the sample image.

However, due to the complexity of the optical imaging system of the light sheet microscope and the sample, the partial images of the overlapped area of different images collected at different positions have difference, so that the stitching effect is not ideal. On the other hand, automatic stitching techniques for rotated images are lacking.

Disclosure of Invention

The following presents a simplified summary of the disclosure in order to provide a basic understanding of some aspects of the disclosure. It should be understood that this summary is not an exhaustive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure or to delineate the scope of the disclosure. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later.

According to an aspect of the present invention, there is provided an image processing method including: intercepting an overlapping area for an N-dimensional image pair that is adjacent in position on an object among a plurality of N-dimensional images of portions of the object being photographed, where N is an integer greater than or equal to three; partitioning the overlapped areas of the adjacent N-dimensional image pairs in N directions respectively to obtain two groups of N-dimensional partitioned images aiming at each direction; projecting the two groups of N-dimensional block images in the corresponding directions to obtain two groups of two-dimensional block images; performing pairwise matching of the two-dimensional block images between the two groups of two-dimensional block images to obtain the offset of the two groups of two-dimensional block images with the highest confidence degrees in the corresponding directions; if the offset is larger than a preset threshold, repeating the steps according to the offset until the offset is smaller than the preset threshold; iteratively performing the above steps for other adjacent N-dimensional image pairs of the plurality of N-dimensional images; recalculating the image size of the plurality of N-dimensional images spliced together for image splicing based on the obtained offset of all the N-dimensional image pairs, and fusing the overlapping regions between all the N-dimensional image pairs.

Preferably, the image processing method according to the present invention further includes adjusting the plurality of N-dimensional images to the same viewing angle before intercepting the overlap region.

Preferably, the fusion of the overlapping regions uses a fusion method based on gaussian filtering.

Preferably, the blocking the overlapping regions of the adjacent N-dimensional image pairs in N directions respectively includes: and determining the block number of the overlapping area in each direction according to the ratio of the length of the overlapping area in each direction to the difference between the block thickness and the coincidence degree.

Preferably, the projection is based on a maximum intensity projection method.

Preferably, performing pairwise matching of the two-dimensional block images between the two sets of two-dimensional block images to obtain the offset amount of the two sets of two-dimensional block images with the highest confidence in the corresponding direction thereof comprises: matching two-dimensional block images in one group of the two groups of two-dimensional block images with two-dimensional block images in the other group in pairs to obtain offset in other N-1 directions except the corresponding directions of the two groups of two-dimensional block images and confidence degrees under the offset; and taking the offset with the highest confidence as the offset along the corresponding direction.

Preferably, the matching is based on a normalized cross-correlation algorithm.

Preferably, recalculating the image size after stitching the plurality of N-dimensional images together comprises: calculating the maximum absolute position of each N-dimensional image in N directions based on the obtained respective offsets; and determining the size of the image obtained after the plurality of N-dimensional images are spliced together according to all the obtained maximum absolute positions.

Preferably, the predetermined threshold is less than or equal to 5 pixels.

According to another aspect of the present invention, there is provided an image processing apparatus including: a clipping device configured to clip an overlapping region for an N-dimensional image pair located adjacent on an object among a plurality of N-dimensional images of respective portions of the object to be photographed, where N is an integer greater than or equal to three; a blocking device configured to block overlapping regions of the adjacent N-dimensional image pairs in N directions, respectively, to obtain two sets of N-dimensional block images for each direction; a projection device configured to project the two sets of N-dimensional block images in their corresponding directions to obtain two sets of two-dimensional block images; a matching device configured to perform pairwise matching of two-dimensional patch images between the two sets of two-dimensional patch images to obtain an offset amount of the two sets of two-dimensional patch images in a corresponding direction thereof with a highest confidence; and determining means configured to determine whether the offset amount is greater than a predetermined threshold, and if the offset amount is greater than the predetermined threshold, re-intercept the overlapping region according to the offset amount and re-determine the offset amount with the highest confidence until the offset amount is less than the predetermined threshold. The image processing apparatus is configured to: determining offsets of other adjacent N-dimensional image pairs of the plurality of N-dimensional images, and recalculating image sizes of the plurality of N-dimensional images stitched together for image stitching based on the obtained offsets of all N-dimensional image pairs, and fusing overlapping regions between all N-dimensional image pairs.

According to other aspects of the invention, corresponding computer program code, computer readable storage medium and computer program product are also provided.

By the image processing method and the image processing equipment, the calculation complexity can be reduced, the splicing effect can be improved, and the automatic splicing of the rotated images can be supported.

These and other advantages of the present invention will become more apparent from the following detailed description of the preferred embodiments of the present invention when taken in conjunction with the accompanying drawings.

Drawings

To further clarify the above and other advantages and features of the present disclosure, a more particular description of embodiments of the present disclosure will be rendered by reference to the appended drawings. Which are incorporated in and form a part of this specification, along with the detailed description that follows. Elements having the same function and structure are denoted by the same reference numerals. It is appreciated that these drawings depict only typical examples of the disclosure and are therefore not to be considered limiting of its scope. In the drawings:

FIG. 1 is a flow diagram of an image processing method according to one embodiment of the invention;

FIG. 2 is a flow diagram for recalculating the size of a stitched image, according to an embodiment of the present invention;

FIG. 3A schematically illustrates a MIP projection of a three-dimensional image blocked in the Z-axis;

FIG. 3B schematically illustrates pairwise matching between two sets of two-dimensional images;

FIG. 4 schematically illustrates the offset of a template image relative to an image to be matched;

FIG. 5 shows a comparison of the stitching effect of the method according to the invention with the prior art method;

FIG. 6 is a block diagram of an image processing apparatus according to an embodiment of the present invention;

FIG. 7 is a block diagram of an exemplary architecture of a general purpose personal computer in which methods and/or apparatus according to embodiments of the invention may be implemented.

Detailed Description

Exemplary embodiments of the present disclosure will be described hereinafter with reference to the accompanying drawings. In the interest of clarity and conciseness, not all features of an actual implementation are described in the specification. It will of course be appreciated that in the development of any such actual embodiment, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which will vary from one implementation to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the art having the benefit of this disclosure.

Here, it should be further noted that, in order to avoid obscuring the present disclosure with unnecessary details, only the device structures and/or processing steps closely related to the scheme according to the present disclosure are shown in the drawings, and other details not so relevant to the present disclosure are omitted.

Currently, in biomedical image processing, a plug-in locking module in Imagej is often used for image splicing. The plug-in is based on PCM (phase correlation method) to register images, calculate the offset and then perform image fusion. An image stitching plug-in iStitch module commonly used in Vaa3d registers images based on PCM and NCC (normalized cross-correlation), calculates an offset, and then performs image fusion. These methods have poor registration effect on images with differences in overlapping regions, resulting in unsatisfactory overall stitching effect and do not support automatic stitching of rotated images.

In view of this, the present invention provides a fine-grained image stitching method for a multi-dimensional image acquired by translational and rotational scanning of a light-sheet microscope. An image processing method 100 according to an embodiment of the present invention is described below with reference to fig. 1.

First, in step 101, an overlap region is cut out for N-dimensional images that are positioned adjacent on an object among a plurality of N-dimensional images of respective portions of an object to be photographed, where N is an integer greater than or equal to three. Specifically, in the present embodiment, the N-dimensional image is, for example, a three-dimensional image, and the pair of three-dimensional images positioned adjacent to each other on the object means that the pair of three-dimensional images can be stitched together at the overlapping region to constitute an image containing a part of the object.

It should be understood that the three-dimensional images are examples only and are not intended to limit the present invention. The method according to this embodiment is also applicable to four-dimensional or multi-dimensional images.

Specifically, in the present embodiment, the overlapping area of the pair of images is calculated from the translation size and the image size of the pair of three-dimensional images, and then the overlapping areas of the two three-dimensional images are respectively clipped to obtain two overlapping area three-dimensional images.

Preferably, the image processing method 100 according to the present embodiment further includes a step 101' of adjusting the plurality of N-dimensional images to the same viewing angle before intercepting the overlap region. Specifically, in the present embodiment, if it is determined that there is a rotation of the plurality of three-dimensional images, they are rotated back to zero degrees around the rotation axis so that all of the N three-dimensional images return to the same viewing angle.

It should be understood that it is known in the art how to determine whether there is a rotation of the three-dimensional image and how to rotate the three-dimensional image back to zero degrees, and therefore is not described in detail here.

Next, in step 102, the overlapping regions of adjacent N-dimensional image pairs are respectively blocked in N directions to obtain two sets of N-dimensional block images for each direction. Specifically, in the present embodiment, the two overlapping area three-dimensional images obtained in step 101 are respectively blocked in X, Y, Z three directions to obtain two sets of three-dimensional images in each direction.

Preferably, it can be according to a function

To block the overlap region three-dimensional image in each direction, wherein c denotes a thickness of each block in the corresponding direction, p denotes a length of two adjacent blocks overlapping each other in the corresponding direction, and m denotes an overall length of the overlap region three-dimensional image in the corresponding direction.

Preferably, the value of c may be, for example, one tenth of the length of the three-dimensional image in the direction X, Y, Z, respectively, and the value of p may be 20% of c.

It should be understood that the above blocking approach is merely an example and is not intended to limit the present invention. Other suitable partitioning schemes may also be used.

It should also be understood that the values of c and p are not limited to the above manner, but may also be set appropriately according to the accuracy requirement of image stitching.

Next, in step 103, two sets of N-dimensional block images in each direction are projected in their corresponding directions to obtain two sets of two-dimensional block images. Specifically, in the present embodiment, for example, MIP (maximum intensity projection) projection is performed on the two sets of three-dimensional images obtained in step 102 to obtain two sets of two-dimensional images in each direction.

It should be understood that MIP projection is merely an example and is not intended to limit the present invention. Other suitable projection methods may also be used.

Fig. 3A schematically shows the blocking of the three-dimensional image of the overlap region in the Z-axis direction and the MIP projection. After the MIP projection is performed, as shown in FIG. 3A, it is obtained

Two-dimensional image with a patch thickness of 1.

Note that the step 102 of blocking and the step of projectingStep 103 causes the amount of data to be reduced

Thereby drastically reducing memory overhead and computational load and making it possible to splice data of the 1000GB level on a common workstation such as 128GB memory.

Next, in step 104, two-by-two matching of the two-dimensional block images is performed between the two sets of two-dimensional block images in each direction to obtain an offset amount of the two sets of two-dimensional block images with the highest confidence in their corresponding directions.

Specifically, as shown in fig. 3B, one of the two sets of two-dimensional images in each direction obtained in step 103 is used as a template set, and the other is used as an image set to be matched, with each set having

A two-dimensional image. And matching each two-dimensional image in the template group and each two-dimensional image in the image group to be matched two by two based on an NCC algorithm, so as to obtain the offset of the two groups of two-dimensional images in one direction in the other two directions and the confidence coefficient under the offset. Each direction is common

And (5) group matching results. In each direction

The offset of the pair of two-dimensional images in the set with the highest confidence serves as a pair of offsets along the slice in that direction. Thus, there are three pairs of offsets for X, Y and the Z direction.

Fig. 4 schematically shows the shift amount of the template image with respect to the image to be matched, in which the left front cube within the outermost peripheral cube is the image to be matched and the right rear cube is the template image. In the example shown in fig. 4, the amount of offset in the X direction is greater than zero, the amount of offset in the Y direction is less than zero, and the amount of offset in the Z direction is greater than zero.

It should be understood that how to calculate the offset and confidence based on the NCC algorithm is known in the art and therefore will not be described in detail here.

It should also be understood that the NCC algorithm is only one example and is not intended to limit the present invention. Other suitable algorithms may also be used to calculate the offset and confidence.

Next, in step 105, it is determined whether the resulting three pairs of offsets are greater than a predetermined threshold, and if so, steps 101 to 104 are repeated according to the calculated offsets until the final offset is less than or equal to the predetermined threshold. In the present embodiment, the predetermined threshold is, for example, less than or equal to 5 pixels.

It is to be understood that the present invention is not limited thereto, but the predetermined threshold may be set as needed.

It will also be appreciated that re-performing steps 101 to 104 in dependence on the calculated offset amount refers to re-truncating the overlapping area based on the calculated offset amount.

If it is determined in step 105 that the offset is less than or equal to the predetermined threshold, then proceed to step 106. In step 106, steps 101 to 105 are iteratively performed for other adjacent N-dimensional image pairs of the plurality of N-dimensional images. Specifically, in the present embodiment, steps 101 to 105 are performed for all other three-dimensional images that are positioned adjacent on the object to determine the shift amount of all the adjacent three-dimensional images.

Finally, in step 107, the image size after stitching the plurality of N-dimensional images together is recalculated for image stitching based on the obtained offset of all the N-dimensional image pairs, and the overlapping regions between all the N-dimensional image pairs are fused.

One embodiment of recalculating the size of the stitched image is described below in conjunction with FIG. 2.

First, in step 1071, the maximum absolute position in the N directions of each N-dimensional image is calculated based on the obtained respective amounts of shift. Specifically, in the present embodiment, the maximum absolute positions of all the adjacent three-dimensional images in the directions X, Y and Z are calculated based on the amounts of shift in the directions X, Y and Z of these three-dimensional images obtained in step 106. For example, the absolute positions of the sequentially adjacent images may be calculated from the relative offsets by traversing all the sequentially adjacent images with the position of the first image as the absolute zero position according to the order in which the images are adjacent.

Next, in step 1072, the size of the image after stitching together the plurality of N-dimensional images is determined based on all the obtained maximum absolute positions. Specifically, in the present embodiment, the size of the three-dimensional image displaying the complete object after stitching together the three-dimensional images is determined based on the sum of the maximum absolute positions of all the three-dimensional images in the X, Y and Z directions.

Finally, in step 1072, the overlap regions between all the N-dimensional image pairs are fused. Specifically, in the present embodiment, the overlap region may be fused by, for example, a fusion method based on gaussian filtering. It is to be understood that the present invention is not so limited, and that any suitable fusion method may be used, such as, but not limited to, linear fusion, minimum fusion, maximum fusion, mean fusion, and the like. In the case of rotated image fusion, a fusion method based on gaussian filtering is preferably used.

By the method, the calculation complexity is reduced, the splicing effect is improved, and the rotated images can be automatically spliced. Table 1 below illustrates the experimental effect of the method according to the invention compared to the Imagej method and Vaa3d method, respectively.

TABLE 1

In table 1, Imagej denotes a method of the plug-in stopping module in Imagej, and Vaa3d denotes a method of the plug-in iStitch module in Vaa3 d. Each image data represents the width X height X depth color channel position of the image by X Y C P, and each pixel is 16 bits. Fli1:2.92GB (1024 × 150 × 1 × 10) in table 1 indicates that the data is 2.92GB, 1024 pixels wide, 1024 pixels high, 150 layers deep, 1 color channel, and 10 positions.

Fig. 5 shows the stitching effect of the method according to the invention compared to the Imagej method and the Vaa3d method. In fig. 5, (a-1) to (a-4) are MIPs of images acquired by a light sheet microscope in the Z direction, (b) MIPs of the resulting images stitched with Imagej in the Z direction, (c) MIPs of the resulting images stitched with Vaa3d in the Z direction, and (d) MIPs of the resulting images stitched with the method of the present invention in the Z direction.

As can be seen from table 1 and fig. 5 above, the method of the present invention achieves a better splicing effect than the prior art method.

The methods discussed above may be implemented entirely by computer-executable programs, or may be implemented partially or entirely using hardware and/or firmware. When it is implemented in hardware and/or firmware, or when a computer-executable program is loaded into a hardware device that can execute the program, an image processing device to be described below is implemented. In the following, a summary of these devices is given without repeating some details that have been discussed above, but it should be noted that, although these devices may perform the methods described in the foregoing, the methods do not necessarily employ or be performed by those components of the described devices.

Fig. 6 shows an image processing apparatus 600 according to an embodiment, comprising a clipping means 601, a blocking means 602, a projection means 603, a matching means 604, a determination means 605 and a stitching means 606. The clipping means 601 is configured to clip an overlapping area for an N-dimensional image pair positioned adjacent to an object among a plurality of N-dimensional images of respective portions of a subject, where N is an integer greater than or equal to three. The blocking device 602 is configured to block the overlapping regions of adjacent N-dimensional image pairs in N directions, respectively, to obtain two sets of N-dimensional block images for each direction. The projection device 603 is configured to project the two sets of N-dimensional block images in the corresponding directions to obtain two sets of two-dimensional block images. The matching device 604 is configured to perform pairwise matching of the two-dimensional block images between the two-dimensional block images to obtain an offset amount of the two-dimensional block images in a corresponding direction thereof with the highest confidence. The determining means 605 is used to determine whether the offset amount is larger than a predetermined threshold, and if the offset amount is larger than the predetermined threshold, re-intercept the overlapping area according to the offset amount and re-determine the offset amount with the highest confidence until the offset amount is smaller than the predetermined threshold. The stitching means 606 is used to recalculate the image size after stitching the plurality of N-dimensional images together for image stitching based on the offset of all N-dimensional image pairs obtained by the determining means 605, and to fuse the overlapping regions between all N-dimensional image pairs.

Preferably, the image processing apparatus 600 according to the present embodiment further includes an adjusting device 601' for adjusting the plurality of N-dimensional images to the same viewing angle before the overlapping region is cut out.

The image processing apparatus 600 shown in fig. 6 corresponds to the image processing method 100 shown in fig. 1. Therefore, details related to each device in the image processing apparatus 600 are given in detail in the description of the image processing method 100 of fig. 1, and are not repeated here.

Each constituent module and unit in the above-described apparatus may be configured by software, firmware, hardware, or a combination thereof. The specific means or manner in which the configuration can be used is well known to those skilled in the art and will not be described further herein. When the software or firmware is implemented, a program constituting the software is installed from a storage medium or a network to a computer (for example, a general-purpose computer 700 shown in fig. 7) having a dedicated hardware configuration, and the computer can execute various functions and the like when various programs are installed.

FIG. 7 is a block diagram of an exemplary architecture of a general purpose personal computer in which methods and/or apparatus according to embodiments of the invention may be implemented. As shown in fig. 7, a Central Processing Unit (CPU)701 performs various processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from a storage section 708 to a Random Access Memory (RAM) 703. In the RAM 703, data necessary when the CPU 701 executes various processes and the like is also stored as necessary. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input/output interface 705 is also connected to the bus 704.

The following components are connected to the input/output interface 705: an input section 706 (including a keyboard, a mouse, and the like), an output section 707 (including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker and the like), a storage section 708 (including a hard disk and the like), a communication section 709 (including a network interface card such as a LAN card, a modem, and the like). The communication section 709 performs communication processing via a network such as the internet. A driver 710 may also be connected to the input/output interface 705, as desired. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as needed, so that the computer program read out therefrom is installed in the storage section 708 as needed.

In the case where the above-described series of processes is realized by software, a program constituting the software is installed from a network such as the internet or a storage medium such as the removable medium 711.

It will be understood by those skilled in the art that such a storage medium is not limited to the removable medium 711 shown in fig. 7 in which the program is stored, distributed separately from the apparatus to provide the program to the user. Examples of the removable medium 711 include a magnetic disk (including a floppy disk (registered trademark)), an optical disk (including a compact disc read only memory (CD-ROM) and a Digital Versatile Disc (DVD)), a magneto-optical disk (including a Mini Disk (MD) (registered trademark)), and a semiconductor memory. Alternatively, the storage medium may be the ROM 702, a hard disk included in the storage section 708, or the like, in which programs are stored and which are distributed to users together with the apparatus including them.

The invention also provides a corresponding computer program code and a computer program product with a machine readable instruction code stored. The instruction codes are read by a machine and can execute the method according to the embodiment of the invention when being executed.

Accordingly, storage media configured to carry the above-described program product having machine-readable instruction code stored thereon are also included in the present disclosure. Including, but not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, and the like.

Through the above description, the embodiments of the present disclosure provide the following technical solutions, but are not limited thereto.

Supplementary note 1. an image processing method, comprising:

intercepting an overlapping region for an N-dimensional image pair located adjacent on an object among a plurality of N-dimensional images of respective portions of the object to be photographed, where N is an integer greater than or equal to three;

partitioning the overlapped areas of the adjacent N-dimensional image pairs in N directions respectively to obtain two groups of N-dimensional partitioned images aiming at each direction;

projecting the two groups of N-dimensional block images in the corresponding directions to obtain two groups of two-dimensional block images;

performing pairwise matching of the two-dimensional block images between the two groups of two-dimensional block images to obtain the offset of the two groups of two-dimensional block images with the highest confidence degrees in the corresponding directions;

if the offset is larger than a preset threshold, repeating the steps according to the offset until the offset is smaller than the preset threshold;

iteratively performing the above steps for other adjacent N-dimensional image pairs of the plurality of N-dimensional images; and

recalculating an image size after stitching the plurality of N-dimensional images together for image stitching based on the obtained offset of all the N-dimensional image pairs, and fusing overlapping regions between all the N-dimensional image pairs.

Supplementary note 2. the image processing method according to supplementary note 1, further comprising adjusting the plurality of N-dimensional images to the same viewing angle before intercepting the overlapping region.

Supplementary note 3. the image processing method according to supplementary note 2, wherein a fusion method based on gaussian filtering is used for fusing the overlapping region.

Note 4. the image processing method according to any one of notes 1 to 3, wherein the blocking of the overlapping regions of the adjacent N-dimensional image pairs in the N directions, respectively, includes:

and determining the block number of the overlapping area in each direction according to the ratio of the length of the overlapping area in each direction to the difference between the block thickness and the coincidence degree.

Supplementary note 5. the image processing method according to any one of supplementary notes 1 to 3, wherein the projection is based on a maximum intensity projection method.

Supplementary notes 6. the image processing method according to any of supplementary notes 1 to 3, wherein pairwise matching of two-dimensional patch images between the two sets of two-dimensional patch images to obtain an offset amount of the two sets of two-dimensional patch images having a highest confidence in their corresponding directions comprises:

matching two-dimensional block images in one group of the two groups of two-dimensional block images with two-dimensional block images in the other group in pairs to obtain offset in other N-1 directions except the corresponding directions of the two groups of two-dimensional block images and confidence degrees under the offset; and

and taking the offset with the highest confidence as the offset along the corresponding direction.

Supplementary note 7. the image processing method according to supplementary note 6, wherein the matching is based on a normalized cross-correlation algorithm.

Note 8. the image processing method according to note 6, wherein the recalculating the image size after stitching the plurality of N-dimensional images together includes:

calculating the maximum absolute position of each N-dimensional image in N directions based on the obtained respective offsets; and

and determining the size of the image obtained after the plurality of N-dimensional images are spliced together according to all the obtained maximum absolute positions.

Supplementary note 9. the image processing method according to any one of supplementary notes 1 to 3, wherein the predetermined threshold value is less than or equal to 5 pixels.

Supplementary note 10. an image processing apparatus, comprising:

a clipping device configured to clip an overlapping region for an N-dimensional image pair located adjacent on an object among a plurality of N-dimensional images of respective portions of the object to be photographed, where N is an integer greater than or equal to three;

a blocking device configured to block the overlapping regions of the adjacent N-dimensional image pairs in N directions, respectively, to obtain two sets of N-dimensional block images for each direction;

a projection device configured to project the two sets of N-dimensional block images in their corresponding directions to obtain two sets of two-dimensional block images;

a matching device configured to perform pairwise matching of two-dimensional patch images between the two sets of two-dimensional patch images to obtain an offset amount of the two sets of two-dimensional patch images in a corresponding direction thereof with a highest confidence; and

determining means configured to determine whether the offset amount is greater than a predetermined threshold, and if the offset amount is greater than the predetermined threshold, re-truncating the overlapping region according to the offset amount and re-determining the offset amount with the highest confidence until the offset amount is less than the predetermined threshold; and

a stitching means configured to recalculate an image size after stitching the plurality of N-dimensional images together for image stitching based on the shift amounts of all the N-dimensional image pairs obtained by the determination means, and to fuse overlapping regions between all the N-dimensional image pairs.

Note 11. the image processing apparatus according to note 10, further comprising angle adjusting means configured to adjust the plurality of N-dimensional images to the same viewing angle before the overlap region is cut out.

Supplementary note 12. the image processing apparatus according to supplementary note 11, wherein a fusion method based on gaussian filtering is used for fusing the overlapping region.

Note 13 the image processing apparatus according to any one of notes 10 to 12, wherein the blocking means is further configured to:

Supplementary notes 14. the image processing apparatus according to any of supplementary notes 10 to 12, wherein the projection is based on a maximum intensity projection method.

Supplementary note 15 the image processing apparatus according to any one of supplementary notes 10 to 12, wherein the matching means is further configured to:

Supplementary note 16. the image processing apparatus according to supplementary note 15, wherein the matching is based on a normalized cross-correlation algorithm.

Supplementary note 17 the image processing apparatus according to supplementary note 15, wherein recalculating the image size after stitching the plurality of N-dimensional images together comprises:

Supplementary note 18. the image processing apparatus according to any one of supplementary notes 10 to 12, wherein the predetermined threshold value is less than or equal to 5 pixels.

Supplementary note 19 a computer-readable storage medium storing a program executable by a processor to perform the operations of:

recalculating the image size of the plurality of N-dimensional images spliced together for image splicing based on the obtained offset of all the N-dimensional image pairs, and fusing the overlapping regions between all the N-dimensional image pairs.

Finally, it should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Furthermore, without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

Although the embodiments of the present invention have been described in detail with reference to the accompanying drawings, it should be understood that the above described embodiments are only configured to illustrate the present invention and do not constitute a limitation of the present invention. It will be apparent to those skilled in the art that various modifications and variations can be made in the above-described embodiments without departing from the spirit and scope of the invention. Accordingly, the scope of the invention is to be defined only by the claims appended hereto, and by their equivalents.

Claims

1. An image processing method comprising:

performing pairwise matching of the two-dimensional block images between the two-dimensional block images to obtain an offset of the two-dimensional block images with the highest confidence in the corresponding directions;

2. The image processing method according to claim 1, further comprising adjusting the plurality of N-dimensional images to a same viewing angle before intercepting the overlapping region.

3. The image processing method according to claim 2, wherein fusing the overlapping region uses a gaussian filter-based fusion method.

4. The image processing method according to any one of claims 1 to 3, wherein blocking overlapping regions of the adjacent N-dimensional image pairs in N directions, respectively, comprises:

5. The image processing method according to any one of claims 1 to 3, wherein the projection is based on a maximum intensity projection method.

6. The image processing method according to any one of claims 1 to 3, wherein performing pairwise matching of two-dimensional patch images between the two sets of two-dimensional patch images to obtain an offset amount of the two sets of two-dimensional patch images in their corresponding directions with a highest confidence includes:

7. The image processing method of claim 6, wherein the matching is based on a normalized cross-correlation algorithm.

8. The image processing method according to claim 6, wherein recalculating the image size after stitching the plurality of N-dimensional images together comprises:

9. The image processing method according to any one of claims 1 to 3, wherein the predetermined threshold is less than or equal to 5 pixels.

10. An image processing apparatus comprising:

an intercepting means configured to intercept an overlapping area for an N-dimensional image pair that is adjacent in position on the subject, among a plurality of N-dimensional images of respective portions of the subject being photographed, where N is an integer greater than or equal to three;

a blocking device configured to block overlapping regions of the adjacent N-dimensional image pairs in N directions, respectively, to obtain two sets of N-dimensional block images for each direction;

a matching device configured to perform pairwise matching of two-dimensional patch images between the two sets of two-dimensional patch images to obtain an offset amount of the two sets of two-dimensional patch images in a corresponding direction thereof with a highest confidence;

11. A computer-readable storage medium storing a program executable by a processor to: