JPH03182976A - Joining method for digital picture - Google Patents

Abstract

PURPOSE:To attain the automatic joint of pictures by extracting two particle parts formed by means of the specified number of picture elements as characteristic particles and joining two pictures by means of a straight line connecting the centroid coordinates of two characteristic particles in two pictures. CONSTITUTION:Two digital pictures 1MA and 1MB which are overlapped and image-picked up are density-converted so that density becomes equal, and are binarized. In respective pictures, two particle parts composed of the specified number of the picture elements are automatically extracted as the characteristic particles (a), (b), a' and b'. Then the pictures are joined with the straight line as a joining line. For generating what is called a panoramic photograph by joining plural digital pictures which are overlapped and are continuously image- picked up, the difference of gradation, the incontinuity of a joining part and the like can be excluded and automatic joint is attained.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to image joining, that is, a method for creating a so-called panoramic photograph by joining a plurality of overlapping and continuously captured digital images. This invention relates to a method for joining crystal photographs or slab cross-sectional photographs for inspecting metal Mi textures while eliminating gradation differences, discontinuities in joints, etc.

[Conventional technology]

Traditionally, when creating a so-called panoramic photograph by joining multiple photographs, whether landscape photographs or microscopic photographs, an optical camera and silver halide film are used to image the subject, and each The images were enlarged onto photographic paper, printed, developed, and the resulting photographs were joined together by cutting and pasting.

On the other hand, with the development of so-called digital image processing technology, television cameras and scanning electron microscopes (SEM)
canning Electron Microsco
Imaging devices that can obtain images obtained with pe) and the like as electronic information are at odds.

In such a digital imaging device, for example, a metal M
The quality of the photo used for i-weave inspection is 1024 x 1024 pixels (5cm square size) with 256 gradations (
It has been experimentally confirmed that there are no practical problems if an original image of approximately 20 pixels per square inch is obtained. Furthermore, when the image information digitally processed as described above is finally printed on photographic paper, there is no practical problem as long as the photographic image has 256 gradations and 24 million pixels or more on A4 size photographic paper. It is known that.

By the way, in digital image processing, an object is imaged by a television camera, an electron microscope, etc., and the relationship between the imaging magnification and the size of the image must be maintained until it is printed on photographic paper. Each captured image has differences in brightness due to differences in exposure amount, non-uniformity in brightness on the same image due to lack of peripheral light in the optical system of the imaging device, image shape due to aberration of the optical system, and It has size distortion, etc. For this reason, even when creating a panoramic photograph using conventional ordinary photographs, it is common to trim the taken photographs and join them together using only the central portion.

As mentioned above, for example, conventional ordinary photographs that are spliced together for the purpose of inspecting metal abrasion fabrics have to be photographed and printed again due to exposure of the inspection target area and poor image quality. There are many things. Furthermore, in the conventional method, skilled techniques are required for film development, enlargement, printing, etc., and furthermore, the work of joining photographs requires quite detailed manual work. Therefore, if it becomes possible to join images having sufficient gradation and resolution through digital processing, it becomes easy to improve gradation, sharpness, image distortion, brightness, etc. using affine transformation Laplacian or the like.

From this point of view, the inventors of the present application first published Japanese Patent Application No.
The invention of No. 85939 is proposed.

The invention of this patent application No. Fl-85939 can be summarized as “
In a method of joining a plurality of digital images that have gradation and are taken in an overlapping manner, two common points in each of the two images to be joined are arbitrarily extracted, and the two points are combined in each image. The two images are joined using a straight line connecting them as a joining line, and the gradation of each image is corrected by calculating the average of the brightness distribution of each image to be joined.

[Problem to be Solved by the Invention] By the way, when carrying out the invention of the above-mentioned Japanese Patent Application No. 1-85939, if the work of "arbitrarily extracting two common points in each of two images to be joined" is carried out, The following problems were found.

(1) If the density difference between two images is large, 21
Extracting points is extremely difficult. In particular, automatic extraction by an image processing device is practically impossible.

(2) Since the operator arbitrarily selects a common point and operates it on screen 2, the coordinate values of the common point are not detected.

Due to the existence of such problems, automatic joining is virtually impossible.

The present invention was made in view of these circumstances, and
The purpose of this invention is to provide a digital image joining method that enables automatic joining of images.

[Means to solve the problem]

The digital image joining method of the present invention converts image information obtained by a television camera or an electron microscope into digital data, and the inventors of the present invention convert the digital image into digital data by, for example, making the density of the digital images uniform. For the purpose of converting the density of both images or one of the images so that the density of the two images becomes equal, the density between the images is changed using an image processing method previously proposed in the invention of Japanese Patent Application No. 1-110615. is equalized, further binarized, two particulate parts formed by a specific number of pixels in the two images that are the direct targets of joining are extracted as characteristic particles, and the barycentric coordinates of these two characteristic particles are calculated. It joins two images with a connecting straight line.

[Effect]

In the method of the present invention, two digital images taken in an optical system are converted into two digital images so that their densities are equal, and then binarized. Two particulate parts are automatically extracted as characteristic particles, and the images are joined using a straight line connecting the two as a joining line.

[Principle of the invention]

Hereinafter, the principle of the present invention will first be explained.

For example, suppose two digital images IMA and IMB are to be joined as shown in FIG. In this case, both parts overlap each other, for example, 2
The image is captured with 56 gradations.

First, one image IMA is input to an image processing apparatus, and a density equalization process, that is, a process for making the densities of both images equal, is performed, for example, according to the method of the invention of Japanese Patent Application No. 1-110615-q. Next, within the range overlapping with the image IFn of the image IMA, the density level is 10.
Binarization processing is performed using the 0 gradation (=near 1) as a threshold.

After this, particle cutting is performed so that two particles are extracted within the overlamp range. What is this particle cut?
Particles are extracted by limiting the number of pixels in a portion expressed in a particle form on a binarized image to a specific range, and ultimately two particles are extracted.

For example, assume that the image IMA is as shown in FIG. 2 [al]. This image IMA has 256 gradations, so if it is binarized at around 100 gradations, an image in which only relatively high-density parts remain, as shown in Figure 2 (bl), is obtained. (The number of pixels forming each particulate part scattered on the image as shown in bl is counted, and only parts within a specific pixel number range are extracted. If the number of extracted particulate parts is more than 2, the number of pixels The number range is narrowed and iterated until the last two particles are extracted (Fig. 2 fc).
shows a state in which two particles are extracted as characteristic particles A and B. In addition, in Figure 2 (al and (bl), the particles to be extracted as characteristic particles A and B are shown as (A), (I
It is shown as I).

Then, the barycenter coordinates of the two extracted particles are determined, stored as image completion information, and transmitted to the master CPU.

Similar processing is performed on the other image IMB.

The barycentric coordinates of two particles extracted from two images IMA and IMB are respectively a, b and a', b''.

Next, the center of gravity coordinates of one of the images IMA and IMB, for example, a and ao, are made to match and the images IMA and IMB are superimposed. And both images IMA and IMB
Either image IMA or IMB is rotated around point a in order to match the other barycentric coordinates and b'. Here, since ab-a'b'', triangle abb' is an isosceles triangle.

Furthermore, from the relationship between the orthogonal projection position of image rMA onto the display screen and the orthogonal projection position of image TMB onto the display screen, bb
Since the distance between ' is known, the rotation angle θ necessary to make the point and point b coincide with each other can be found as shown in Figure 3. However, e: length of ab: r: length of bb' .

After images IMA and I[3 are superimposed in this way, it is determined which part of both images TMA and TMB, with line segment ab as the boundary, should be selected and joined.

That is, as shown in FIG. 4, (The shaded area of the 11th image MA is used to join both images TMA and IMB. (2) The shaded area of image rMB is used to join both images IMA and IMB. , (3) Discard the shaded parts of both images IMA and IMB and join them.

As described above, both images IMA and IMB are joined to obtain one image.

[Embodiments of the invention]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below based on drawings showing embodiments thereof.

FIG. 5 is a schematic diagram showing the system configuration of an image processing apparatus used to implement the digital image joining method according to the present invention. Note that this embodiment adopts a configuration that is applied to metal tests.

In FIG. 5, reference numeral 1 is a host cPU, which is used as photographing information input means. The old HOS+-CP has a 32-bit wide central processing unit. Also, the host l-CPU
I registers photographing information of a photographing request to an operator who photographs a subject using an image input means 4, which will be described later, and processes information from a master CPU 2, which will be described later.

The photographing information from the CPU is given to the mask CPU 2, which is the main processing section, according to, for example, the NEC standard HEL 2B procedure.

The mask CPU 2 controls each section, and sends photographing information via the optical cable network 5 to seven sets of image human power units 41 to 47, which are image human power means.

The image input unit 41 takes a large macro photograph of the subject using a CCD camera, 42 takes an external photograph of the subject using a TTV camera, 43 and 44 take a micro photograph of the subject obtained by taking a microscope image with an ITV camera, and 45 is SEM
The photograph was taken using an EPMA (Electron r
'robeχ-Ray Microanalyzer:
The electron beam microanalyzer (electron beam microanalyzer) is used for imaging, and the microanalyzer (47) is used for analytical electron microscopy.

The mask CPU 2 receives images photographed by each of the image processing units 41 to 47, and stores them in the optical disk device 6.

Further, the mask CPU 2 assembles an image based on the assembly information stored in the output queue, and sends it to the personal computer 7, which is the control CPU of the photo printer 8.

The photo printer 8 converts the image information into laser output to form a latent image on photosensitive paper, and the developing device 9 develops this latent image into a visible image.

FIGS. 6, 7, and 8 are system flowcharts I showing the processing procedure of the digital image joining method of the present invention. Hereinafter, the procedure of the method of the present invention will be explained with reference to this flowchart.

FIG. 6 is a flowchart showing the procedure for inputting a request form for giving work instructions to the apparatus.

First, by operating the post CPUI, the operator manually fills out a request form that includes items on whether or not an image joining request is required (
Step Sl).

Next, it is determined whether it is necessary to input character string information (step S
2) If it is determined that it is necessary, the character string information is entered manually (step S3). Then, information regarding the automatic editing of the manually entered character string information and the request form is written to the automatic editing file in the mask CPU 2 (step S4).

If it is determined in step S2 that the input of character string information is unnecessary, only the header information, sample management information, and instruction information are written to the respective files in the master CPU 2 (step 35), and the request form is input. The work is finished.

FIG. 7 is a flowchart showing the procedure of format registration work.

First, it is determined whether the formant is to be modified or newly registered (step 511).

When the format is to be modified, the registered format to be modified is sent from the mask CI'l+2 to the image editing device 3 via the optical cable network 5 (step 512), and in the case of new manual editing, a new formant is created. (Step 514).

The modified formant or newly registered format is sent to the mask CPU via the optical cable network 5.
2 and registered in its format management information file (step 515).

FIG. 8 is a flowchart showing the actual procedure of image joining. This processing is performed in the image input units 42 to 45 and 47.

First, the image processing units 42 to 45.47 receive a schedule and shooting instruction information regarding shooting from the mask CPU 2 (step 521). Each of the image input units 42 to 45 and 47 captures an image according to the received information and manually sends the image to the image processing device (step 522) to perform image density uniformization processing (
Step 523).

The image density uniformization process in step S23 is shown in FIG.
Japanese Patent Application No. 1-11061 previously filed by the present inventors, the flowchart of which is shown in FIGS. 10 and 11.
Utilize the procedure of invention No. 5.

The procedure shown in FIG. 9 is called a region-fixed density correction method.

First, the first raw image data that has not been subjected to density correction is input (step 5ill). This raw image data is one image format! This is the first image among a plurality of images to be placed in the second position, and is hereinafter referred to as a reference image.

Next, the density distribution of this entire reference image is measured and converted to an optimal image density (step 5112).

Next, an area for density comparison (hereinafter referred to as a window) is set at an arbitrary predetermined position of the reference image, and the average density value Ao within this window is determined (step S113).
.

The average density value Ao of the reference image found within the window is stored in the storage means as the average density value of the reference image (step S114). With the above steps, the processing for the reference image is completed.

Next, raw images other than the reference image (hereinafter referred to as converted images)
is processed. That is, similarly to steps 5111 and 5113, the image data is input manually (step 5115), a window is set in the same area as the reference image, and the average density value AC within the window is determined (step 5116). Then, the difference S between the average density values within the window of both images is determined (step 5117), and the density of all pixels of the converted image is corrected (step 3118).

By repeating the processing from steps 5115 to 5119 until the density correction of all five images to be incorporated into the same image format is completed, image density uniformization processing is performed.

FIG. 1O is a flowchart showing the procedure of a method called a fixed magnification density correction method.

First, one piece of raw image data is input (step 5ll).
l), it is determined whether this is the same enlargement ratio as the image data that has already been input (step 5121).

If image data with the same magnification rate has not been input in the past, that image data is used as the reference image of the image group with that magnification rate and is processed in steps 5112, 5112, and 5114 in the same way as in the case of the area fixed density correction method described above. is performed, and the average density value Ao within the reference image window is determined.

In step 5ill, if the magnification rate of the image data manually inputted is the same as that of image data manually inputted in the past, in other words, the average density value of the image data with the same magnification rate as the magnification rate of the image data has already been calculated. If so, steps 5116, 5117, and 5118 of the area fixed density correction 6 normal cleaning described above are performed, and density correction is performed for all pixels.

As described above, density uniformization is performed for a group of images having the same magnification.

FIG. 11 is a flowchart 1 showing the procedure of a method called a fixed density concentration correction method.

In this method, steps 5ill, 5112.5113 are performed similarly to the area fixed density correction method shown in FIG.
The process is performed in the order of S114 to obtain the average density value Ao of the reference image.

Next, similarly to the area-fixed density correction method, the transformed image is manually input in step 5115, and the window setting coefficients for this image are determined (step S131). This is a process for setting a window corresponding to the enlargement ratio on the converted image when the reference image manually input in step 5111 and the converted image inputted in step 5115 have different enlargement ratios.

When a window is set on the converted image in this way,
Thereafter, step 5116 is performed as in the area fixed density correction method.
5117.5118.5119 are processed in the order of density correction of the converted image.

When the image density uniformization process is completed using any of the above-described appropriate methods, each density-uniformed image and completion information indicating that the process has been completed are transferred to each image human-powered unit 4 as an original image.
2 to 45.47 are temporarily stored on the hard disk (step 524).

Next, it is determined whether the manually created image is an image to be joined (step 525), and if it is not an image to be joined, steps S26 to S31 are omitted.

If the input image is an image to be spliced, the image is binarized with the density gradation of around 100 levels as the dark value (step 326), and the image is binarized with a specific size, that is, a specific number of pixels. In order to extract only particles that are composed of pixels, particles that are composed of a number of pixels other than a specific number of pixels, that is, particles that are outside the range, are canted (step 527). In step S29, it is determined whether the number of remaining particles is two, and if it is not two, the cutting range is expanded (step 828).

Then, the processes of steps S27 to S29 are repeated until the number of particles after cutting becomes two, and when the number of particles reaches two, the centroid coordinates of the extracted particles are calculated and included in the completion information. The image data is stored as one in the hard disk of each image input unit 42 to 4547 (step 531).

After this, the original image and completion information are transmitted to the mass 501 port 2 via the optical cable network 5 (step 5
32). When the mask CI'12 receives the original image and the completion information (step 533), it stores them in the optical disk device 6 (step 534), and updates the completion information and automatic editing information (step 535).

Next, it is determined whether or not all images in the same format have been captured (step 336). If not, the process returns to step S22, and if it has been completed, the process is automatically Image assembly information is created by the editing task (step 537) and registered in the output queue within the mask CPU 2 (step 338).

9 In the next step S39, an image is assembled based on the image assembly-C information. As part of the process of step S39, step S40. In S41, the two images are joined together. The image joining process performed here is
Using a straight line passing through the barycenter coordinates of the 211 particles extracted in steps 326 to 331 as a joining line, both images are joined by performing linear movement and four-wheel movement if necessary.

The image assembled in this way is transferred to the photo printer 8 and printed on photographic paper (step 54).
2), automatically transported to the developing machine 9 and processed for development
(Step 543).

[Effects of the Invention] As described in detail below, according to the present invention, two images of the number of joining rows are automatically joined, so that the fill J1. It has excellent effects such as eliminating the need for consumables for processing chemical crystals and not requiring much skill in the work.

[Brief explanation of drawings]

1 to 4 are detailed explanatory diagrams of the present invention. FIG. 1 is a schematic diagram showing two images to be joined and two characteristic particles existing thereon, and FIG. Figure 3 is a schematic diagram showing the procedure for extracting two characteristic particles from each image, and Figure 3 shows a state where the two images are superimposed on one of the two characteristic particles on each image. FIG. 4 is a schematic diagram showing which region is selected when joining two images, and FIG. 5 is a block diagram showing the configuration of an apparatus for implementing the method of the present invention. , FIGS. 6 to 11 are flowcharts 1 showing the steps of the method of the present invention. IMA, IMB...Images to be joined a, a',
b, h'--characteristic particles

Claims (1)

[Claims] 1. In a method for joining a plurality of digital images having gradations and captured mutually overlapping, the density is converted so that the density of the two images to be joined becomes equal. , Binarize the density of each image after density conversion, and extract two characteristic particles by extracting a particulate part formed by a specific number of pixels in each image after the binarization process. A method for joining two images to be joined, the method comprising connecting two images to be joined using a straight line connecting two characteristic particles selected in each image as a joining line.