JPH10108003A - Image compositing device and image compositing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the image compositing device with high matching precision and a small processing amount by having only to overlap divided images to be composited manually and roughly in the case of compositing the division images and to provide the image compositing method. SOLUTION: A comparison range limit section 7 is provided with a divided image display section 8 for a plurality of divided images, a dividing image moving section 9 that moves displayed divided images to overlap them, a compositing image display section 11 to display the overlapped divided images, and a comparison range extract section 10 that sets a retrieval range T resulting from a specific limit part in a reference block S of the one divided image to the other divided image. A composite information extract section 3 conducts pattern matching based on the reference block S, and the retrieval range T extracted by the comparison range extract section 10 to extract composite information. An image composited section 4 composites a plurality of division images from a divided image storage section 2, based on the composite information from the composites information extract section 3 to store the composited image to a composite image storage section 5.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an image synthesizing apparatus and an image synthesizing method, and more particularly, to an image synthesizing apparatus and an image synthesizing method for synthesizing a high-definition image with a wide viewing angle from a plurality of still images. About.

[0002]

2. Description of the Related Art In general, an image pickup apparatus having a solid-state image pickup device such as a CCD has a feature of small size and low power consumption. ing.

[0003] Solid-state imaging devices are generally used for consumer use.
About 400,000 pixels for area sensor, 2 for line sensor
Although about 000 pixels are widely used, in recent years there has been an increasing demand for obtaining high-resolution images. For this reason, area sensors have several million pixels, and line sensors have 5,000 to 10,000. A solid-state image pickup device of about a pixel has been manufactured.

[0004] However, such a high-resolution solid-state image pickup device is very expensive, and there is no prospect for a reduction in price at present.

In the prior art, instead of using such an expensive solid-state image sensor, an inexpensive solid-state image sensor having a small number of pixels is used to divide an imaging region of a subject into a plurality of regions. There has been proposed an image synthesizing apparatus which achieves high definition by synthesizing a plurality of still images obtained by imaging each area (for example, Japanese Patent Application Laid-Open No. H5-25-2).
No. 60264).

FIG. 2 shows the configuration of a conventional image synthesizing apparatus of this kind.
4 will be described in more detail. In FIG. 24, 1
Is a divided image input unit, 2 is a divided image storage unit, 3 is a combined information extracting unit, 4 is an image combining unit, 5 is a combined image storage unit, and 6 is a combined image output unit.

[0007] The divided image input unit 1 receives a divided image obtained by dividing an imaging range of a subject into a plurality of parts. As means for obtaining the divided image, for example, Japanese Patent Laid-Open No. 3-2
As shown in Japanese Patent No. 40372, the number of imaging devices is two.
Although there is a method of moving by a mechanical mechanism in a three-dimensional manner, a camera equipped with a special mechanism is required to realize this, and a general-purpose digital still camera or a personal computer camera cannot be used. A method of capturing a divided image using a general-purpose camera to obtain a high-definition image is disclosed in, for example, Japanese Patent Application Laid-Open No. 5-260264.

Each divided image obtained by the divided image input unit 1 is sequentially stored in the divided image storage unit 2. The composite information extraction unit 3 compares the divided images stored in the divided image storage unit 2 and extracts the composite information necessary for the composition. The information necessary for the synthesis includes, for example, motion vector information between divided images, information such as an enlargement ratio and rotation, and luminance correction information between each divided image. When the combined image is extracted, the image combining unit 4 reads the two or more divided images from the divided image storage unit 2, takes in the combined information extracted by the combined information extracting unit 3, and adds the combined information to the combined information. The divided images are synthesized based on the divided images.

Here, for example, an operation of synthesizing each divided image based on motion vector information as synthesis information will be described with reference to FIG.

As shown in FIG. 1A, when an image pickup area of a subject is picked up while being overlapped as six divided images indicated by reference numerals # 1 to # 6 (W indicates an overlapped area), division is performed. The image storage unit 2 stores six divided images # 1 to # 6 as shown in FIG. It is a whole image 61 of a specific rectangular area of a subject having a gray part with halftone dots.

When the camera is manually moved in the case of such overlapped imaging, usually, the amount of movement of the imaging region of the divided image is not constant. Therefore, if the motion vector is not extracted by the synthesis information extraction unit 3, it becomes impossible to maintain the alignment consistency between the divided images, and if the image synthesis unit 4 synthesizes the divided images, As shown in FIG. 9C, the whole image 61 'is divided image # 1.
~ # 6, and the continuity of the image is lost.

On the other hand, as shown in FIG. 26A, FIG. 26B shows a case where the imaging region of the subject is imaged while overlapping as six divided images indicated by reference numerals # 1 to # 6. ) Of the six divided images # 1 to # 6,
A so-called block for setting a reference block S of a certain area to be a mark for joining adjacent divided images at the time of synthesis in each overlap area W by the synthesis information extracting unit 3 and detecting a motion of each reference block S If a motion vector is extracted by the matching method, an index (parameter) for maintaining good alignment consistency between the divided images can be obtained. If the images # 1 to # 6 are combined, the entire image 61a is completely identical to the entire image 61 of the original subject, as shown in FIG. Is reconstructed as one continuous image as it is.

The image 61a of the entire image pickup area synthesized by the image synthesizing unit 4 is sent to the synthesized image storage unit 5 and stored therein. This image is output to the composite image output unit 6.
Output to a printer as needed, or transferred to another device via a network.

[0014]

However, FIG.
In the conventional image synthesizing apparatus having the configuration shown in (1), when an image is hand-held using a general-purpose digital still camera or video camera, there is a great possibility that a large variation occurs in the amount of overlap between images. Because of the large size, it is necessary to set a considerably large search range for block matching for comparing between divided images.

For example, the overlap between the divided image # 1 and the divided image # 2 due to hand-held imaging is shown in FIG.
Let us consider a case where the values fluctuate as shown in 1), (b1), and (c1). The reference block S is shown in FIG.
As shown in (2) and (c2), when a specific position of the divided image # 1 is set at the center of the right end as an example, (a) in FIG.
3), (b3), (as shown in c3), as the search range T ₁ of the for block matching, a possibility that the position of the corresponding block S ₁ corresponding to the reference block S of the divided image # 2 may vary Need to be set to cover all ranges with This variation range is statistically obtained by analyzing the data of the actually divided and captured images. In order to correspond to all types of divided images, in the worst case, all the regions of the divided image # 2 are searched. And must be.

As described above, in the conventional technique, the search range of the divided image for extracting the image synthesis information becomes very large, and as a result, the amount of calculation for block matching becomes enormous, and the calculation processing becomes large. There is a problem that the load on the device and the memory becomes excessive, the processing time is long, and the probability of occurrence of the same pattern in the search range is increased, thereby lowering the matching accuracy.

As a method of solving the above problem, for example, a method of detecting a moving amount of a camera in real time based on a correlation between image pickup devices or divided images and automatically releasing a shutter can be considered. However, a special camera is required, and realization with a general-purpose camera is difficult.

SUMMARY OF THE INVENTION The present invention has been made to solve the problems which cannot be realized by the conventional techniques as described above. It is an object of the present invention to provide an image synthesizing apparatus and an image synthesizing method which have high matching accuracy and a small amount of processing only by roughly superimposing the images manually.

[0019]

According to a first aspect of the present invention, there is provided an image synthesizing apparatus for comparing a divided image storing means for storing an input divided image with adjacent divided images in the divided image storing means. Means for extracting combined information necessary for image synthesis, and combining adjacent divided images input from the divided image storage means based on the combined information from the combined information extracting means to generate a combined image. In an image synthesizing apparatus including an image synthesizing unit that performs image synthesis and an image synthesizing unit that accumulates a synthetic image synthesized by the image synthesizing unit, a range in which adjacent divided images in the divided image accumulating unit are to be compared is narrowly limited. And the combined information extracting means includes an adjacent divided image in the image comparison range limited by the comparison range limiting means. It is characterized by being configured to extract synthetic information required compared to the synthesis. Since the range to be compared between adjacent divided images for obtaining the synthesis information necessary for image synthesis is limited to a narrow range, the amount of calculation when extracting the synthesis information in the narrow image comparison range is reduced, and the arithmetic processing unit is reduced. And the load on the memory can be reduced, and the processing time can be shortened. further,
Since the image comparison range is narrow, the probability of occurrence of a plurality of the same patterns during pattern matching is reduced, and the matching accuracy can be improved. Further, the present invention can be applied to a general-purpose camera.

According to a second aspect of the present invention, there is provided an image synthesizing apparatus.
2. The apparatus according to claim 1, wherein the comparison range limiting unit includes: a divided image display unit that displays a plurality of divided images; a divided image moving unit that moves and overlaps the divided images displayed by the divided image display unit; A composite image display means for displaying the divided images obtained in a superimposed state, and a reference block set to one of the divided images in an overlap area of the adjacent divided images displayed in the superimposed state in the composite image display means. A comparison range extracting means for obtaining a limited image comparison range by setting a search range surrounding the periphery with a narrow width in the other divided image, wherein the synthesis information extracting means comprises: In the other divided image, a pattern matching is performed on the reference block of one divided image to obtain a corresponding image region in the other divided image. It is characterized by being configured to acquire synthesized information based on the comparison of the response image area. By simply superimposing the desired divided images that the user wants to combine roughly, a reference block is set in one of the divided images, and a search range surrounding the reference block in a narrow range is set in the other divided image. The image comparison range can be narrowed. Then, a corresponding image area for pattern matching with the reference block in the narrow image comparison range is obtained in the other divided image. However, since the matching range at this time is narrow, the amount of calculation for extracting the combined information is reduced, and the calculation processing is reduced. The load on the device and the memory can be reduced, and the processing time can be shortened. Further, since the image comparison range is narrow, the probability of occurrence of a plurality of the same patterns during pattern matching is reduced, and the matching accuracy can be improved.

According to a third aspect of the present invention, an image synthesizing apparatus is provided.
In the above-mentioned claim 2, the comparison range limiting means includes divided image designating means for designating two adjacent divided images to be recombined when the synthesis state of the synthesized image displayed by the synthesized image display means is defective, The composite image display means,
The designated two divided images are enlarged and displayed, and one enlarged divided image is superimposed on the other enlarged divided image, and the comparison range extracting means is configured to perform the comparison range extracting means on the superimposed adjacent enlarged divided image. Is configured to obtain a limited image comparison range by setting a search range surrounding the reference block set in one enlarged divided image in the overlap region with a narrow width in the other enlarged divided image. It is characterized by: Even if the composite state of the composite image once created is defective, it is possible to retry image extraction range extraction, composite information extraction, and image composition by narrowing down to the defective image.
Efficiency is better than when starting over completely from the beginning.
Moreover, at the time of retry, since two adjacent divided images having a defect are displayed in an enlarged manner and the image comparison range is extracted and the synthesized information is extracted, the matching accuracy is high,
Retry is not required again, and workability can be improved.

According to a fourth aspect of the present invention, there is provided an image combining apparatus comprising:
In any one of the first to third aspects, the divided image moving means is configured to perform display control in a state in which adjacent divided images are translucently combined in an overlap region of the divided images. Features. According to a fifth aspect of the present invention, there is provided the image synthesizing apparatus according to any one of the first to third aspects, wherein the divided image moving means shifts adjacent divided images for a predetermined period of time in the overlap region of the divided images. It is characterized in that display control is performed in a state of switching between the front surface and the back surface at intervals. When moving the next divided image with respect to the previously moved divided image, the user satisfies the moving condition that the image comparison range including the reference block and the search range is included in the overlap region of both divided images. , Rough alignment can be facilitated.

According to a sixth aspect of the present invention, there is provided the image synthesizing apparatus,
In any one of the first to third aspects, the divided image moving means may divide an adjacent divided image into a saw-like shape at a boundary between the divided images and arrange the divided images on the front surface and the rear surface. It is characterized by being constituted. If the overlap becomes large at the time of the divided imaging, when the adjacent two divided images to be combined are roughly overlapped, there is a possibility that the boundary becomes blank and the position of the overlap cannot be specified.
When both divided images are arranged on the front surface and the back surface with a sawtooth boundary, rough alignment can be easily performed.

According to a seventh aspect of the present invention, there is provided an image combining method comprising:
A step of inputting and storing the divided images, a step of reducing and displaying the plurality of divided images, a step of roughly superimposing the reduced divided images to be synthesized among the displayed plurality of reduced divided images, and Setting a reference block at a predetermined position based on the origin of the divided image in one of the superimposed adjacent divided images, and narrowing the periphery of the reference block in the other divided image. Setting a search range surrounded by a width; extracting a corresponding image region of pattern matching in the other divided image with respect to a reference block of the one divided image within the search range; and Extracting synthesis information required for image synthesis based on comparison with the region; A step of combining the two divided images by performing the correction of alignment relative to the one of the divided images of the other divided images on the basis of the distribution, is characterized by comprising the step of storing the composite image. By simply superimposing the desired divided images that the user wants to combine roughly, a reference block is set in one of the divided images, and a search range surrounding the reference block in a narrow range is set in the other divided image. The image comparison range can be narrowed. Then, a corresponding image area for pattern matching with the reference block in the narrow image comparison range is obtained in the other divided image. However, since the matching range at this time is narrow, the amount of calculation for extracting the combined information is reduced, and The load on the device and the memory can be reduced, and the processing time can be shortened. Further, since the image comparison range is narrow, the probability of occurrence of a plurality of the same patterns during pattern matching is reduced, and the matching accuracy can be improved.

[0025]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, specific embodiments of an image synthesizing apparatus according to the present invention will be described in detail with reference to the drawings.

[First Embodiment] FIG. 1 is a block diagram showing a configuration of an image synthesizing apparatus according to a first embodiment of the present invention. In FIG. 1, reference numeral 1 denotes a divided image input unit for sequentially inputting divided images obtained by imaging each part of a subject one by one by a digital imaging device such as a digital still camera, and 2 denotes a divided image input unit 1 A divided image storage unit for storing a plurality of divided images input by the CPU, information on motion vector information between divided images necessary for synthesizing the divided images, information on enlargement ratio, rotation, etc., and brightness correction between each divided image A combined information extraction unit 4 for extracting combined information such as information from the divided images reads a plurality of divided images from the divided image storage unit 2 and reads combined information from the combined information extraction unit 3 to convert the plurality of divided images into combined information. An image synthesizing unit 5 for synthesizing based on the synthesized image stored in the synthesized image accumulating unit 5 is a printer or a network. A composite image output unit that outputs the over-click, or topology is similar to that of the prior art. Reference numeral 7 denotes a comparison range limiting unit interposed between the divided image storage unit 2 and the combined information extraction unit 3. Comparative range limitation section 7
Is to limit the range to be compared between adjacent divided images stored in the divided image storage unit 2 according to a certain condition. The feature of this embodiment is that this comparison range limiting unit 7 is provided. is there. The synthesis information extracting unit 3 is configured to extract synthesis information from the adjacent divided image in the comparison range of the adjacent divided image limited by the comparison range limiting unit 7. The configuration shown in FIG. 1 corresponds to claim 1.

FIG. 2 is a block diagram showing a more specific configuration of the comparison range limiting section 7 in the first embodiment together with peripheral portions. The comparison range limiting unit 7 includes a divided image display unit 8
And a divided image moving unit 9, a comparison range extracting unit 10, a combined image display unit 11, and a divided image designating unit 12.
The divided image display unit 8 reduces and displays the plurality of divided images in the divided image storage unit 2, and the divided image moving unit 9 converts the displayed reduced divided image into the combined image display unit 1.
1 to overlap adjacent reduced divided images while securing a certain overlap area,
The comparison range extraction unit 10 obtains a reference block S and a search range T of the divided image from the display position of the reduced divided image displayed on the composite image display unit 11.
1 displays a reduced divided image moved from the divided image display unit 8, and the divided image designation unit 12 performs extraction of combined information again and image combining when image combining is not normally performed. In order to do so, the abnormal divided image is designated. The function of the comparison range limiting unit 7 can be realized by a mouse (or a keyboard), a CPU (Central Processing Unit), a display, and the like. Therefore, it means that the image synthesizing apparatus according to the present embodiment can be realized by various information processing apparatuses having a GUI (graphical user interface) such as a personal computer. The configuration shown in FIG. 2 corresponds to claims 2 and 3.

FIG. 3 is a configuration diagram of a correspondence table 8a of the reduced image and the divided image incorporated in the divided image display unit 8.
The divided image display unit 8 has a built-in video memory (not shown), and is configured to reduce the divided image input from the divided image storage unit 2 and store the reduced image in the video memory. The reduced image and divided image correspondence table 8a stores identification codes of the reduced divided images and identification codes of the divided images stored in the divided image storage unit 2 in association with each other.

Next, the operation of the image synthesizing apparatus configured as described above will be described. FIG. 4 is a flowchart for explaining the operation. 5 to 8 are screen display state diagrams for explaining the operation. 9 and 10 are display state diagrams of the overlap display. 11 to 18 are explanatory diagrams for explaining a specific processing situation.

When the system is started, a menu bar 21, a divided image window 22, and a composite image window 23 are displayed on the display unit 20, as shown in FIG. Display 2
0 corresponds to the combination of the divided image display unit 8 and the composite image display unit 11 in FIG. In FIG. 5, 24
Is a start button, and 25 is a cancel button. In the initial state, only the outer frame of the window is displayed and no image is displayed, but the image can be searched by operating the menu bar 21. If there is no divided image in the divided image storage unit 2, the divided image is input from the divided image input unit 1 based on the operation of the menu bar 21, and
It is stored in the divided image storage unit 2. When the image synthesizing apparatus is a camera integrated with the imaging unit, the means for obtaining the divided image may be obtained by moving the camera as described in the related art and taking an image. When the image synthesizing unit is separated from the image synthesizing unit, it may be obtained from a peripheral device such as a memory card or a hard disk connected to the image synthesizing device, or may be obtained from another device through a network. The selection of which image to select is made by the user operating the menu bar 21 of the display unit 20 and selecting from a pull-down menu.

When the divided images are stored in the divided image storage unit 2, the divided images are displayed on the divided image display unit 8 in the comparison range limiting unit 7. The display screen of the divided image display unit 8 corresponds to the divided image window 22 of the display unit 20. At the time of display, a reduced image is displayed so that about eight divided images can be displayed in a list. For example, if the original image is an image of about 640 × 480 pixels, the image is reduced to と も に both vertically and horizontally to obtain a reduced image of 160 × 120 pixels.
Display of about eight sheets is sufficiently possible. Of course, the reduction ratio is not limited to this, but if the image is too large, it will be inefficient because it lacks in listing, and if it is too small, the image will be difficult to see and a large error may occur in the synthesis. is there. The method for reducing the image may be simple thinning or averaging. As described above, the reduced divided images 31a to 31h are displayed in the divided image window 22 of the display unit 20, as shown in FIG. This is step S in the flowchart of FIG.
Equivalent to 1.

Next, in preparation for the synthesis of the divided images,
The divided image moving unit 9 in the comparison range limiting unit 7 synthesizes an arbitrary divided image to be connected among the reduced divided images 31a to 31h displayed in the divided image window 22, as indicated by an arrow in FIG. The image is moved to the image window 23, and is roughly arranged so that the joint consistency is maintained to some extent. In the example shown in the figure, the reduced divided images 31a, 31b, 31c, 31g, and 31h having halftone dots are selected as the divided images to be combined. The movement of the divided image is performed, for example, by using the mouse to move the divided image window 22.
Move the mouse cursor over the reduced image above, click the mouse button, and drag as it is. Of course, the movement by keyboard operation is also acceptable. This corresponds to step S2 in the flowchart of FIG.

FIG. 9 shows a method of displaying adjacent superimposed reduced divided images. For example, as shown in FIG. 9A, a method of displaying an overlap area W ′ of an adjacent reduced divided image (a dash “′” is added to a code W because it is an overlap area of a reduced divided image) The divided image 31y placed later may be displayed in front of the divided image 31x placed earlier, or the divided image 31y placed later may be placed first as shown in FIG. 9B. Split image 3
It may be hidden behind the back of the 1x. FIG. 9 (c)
As shown in the figure, the average of the density values of the respective pixels in the overlap area W 'of the two divided images 31x and 31y is taken, or the divided images 31x and 31y are displayed separately in odd and even fields. Depending on the method, so-called translucent synthesis may be performed. Also, arrow 3 in FIG.
As shown by 2, both divided images 31x and 31y may be alternately switched in a time division manner so that one is displayed on the front and the other is displayed on the back. The movement of the next divided image 31y with respect to the previously moved divided image 31x and the approximate superposition are as follows:
An image comparison range including a reference block S and a search range T, which will be described later with reference to FIG. 12 and the like, is included in the overlap region W 'of the two divided images 31x and 31y. Note that the relative position of the reference block S in the divided image 31x that has been moved earlier is predetermined (around the upper and lower central portions near the right side), and since the user knows it, roughly, It is relatively easy to move the divided image so as to satisfy the requirement. Details will be described later. FIG.
As shown in FIG. 9C, translucent composition is displayed.
By alternately displaying the front surface and the back surface as shown in (d), it is possible to easily superimpose both divided images 31x and 31y in the overlap area W 'in a state satisfying the above conditions.

By the way, as shown in FIG.
When two adjacent divided images 31x and 31y are photographed with a relatively large overlap at the time of photographing, as shown in FIG. 10B, both divided images are overlapped by any of the methods of FIG. However, the overlap area W 'becomes large, and simply performing rough alignment may result in a case where the position cannot be specified because the boundary becomes blank. As a countermeasure for this problem, as shown in FIGS. 10C and 10D, the divided image moving unit 9
In the overlap area W 'of 1y, the adjacent divided image may be divided into a saw-like shape at the boundary and arranged on the front surface and the rear surface. Adjacent divided image 31
Since the superimposed boundaries of x and 31y can be compared and contrasted in the upper, lower, left, and right directions, rough positioning can be easily performed. Although the saw shape in the case of FIGS. 10C and 10D is rectangular, other shapes such as a triangular saw shape as shown in FIG. 10E may be used.

There are two or more divided images in the composite image window 23, and the overlap is larger than a specified value (for example,
If there are 33 × 80 pixels or more in the reduced divided image (the original divided image which is not reduced corresponds to 130 × 320 image or more as will be described later), it means that the preparation for synthesis is complete, and the start button 24 Is clicked on with a mouse cursor, and the flow proceeds to the process of extracting the image comparison range in step S4. If not, the flow returns to step S2 to continue moving the divided image. This determination corresponds to step S3 in the flowchart of FIG. The determination as to whether or not there is an overlap exceeding the specified value is made by a determination unit (not shown) built in the divided image moving unit 9. However, the number of images to be moved to the composite image window 23 is arbitrary, and the mouse is operated when the user determines that the necessary number of images has been moved.

When the required divided images are arranged in the composite image window 23 and the start button 24 is clicked as described above, the comparison range extraction unit 10 in the comparison range limiting unit 7 extracts the composite information. Extract the image comparison range. This image comparison range includes a range of the reference block S and a search range T. The extraction of the image comparison range corresponds to step S4 in the flowchart of FIG. 4, and the extraction of the next combined information corresponds to step S5.

Now, as shown in FIG. 8, the reduced divided images A to H (31a to 31) in the divided image window 22 are displayed.
It is assumed that five divided images A, B, C, G, and H are selected from h) and arranged in the composite image window 23. The image comparison range extraction and the synthesis information extraction are first performed in order from the two adjacent reduced divided images A and B on the left.

In extracting the image comparison range, first, FIG.
As shown in FIG. 1, an address on the video memory for display in the divided image display section 8 is detected. The origin is set at the upper left of the reduced divided images A, and the reduced divided images A,
The divided images # 1 and # 2 corresponding to B are selected from the divided image storage unit 2, and a virtual composite image 34 corresponding to the reduced composite image 33 in FIG. 11 is created as shown in FIG.
This virtual composite image 34 is actually stored in the divided image storage unit 2.
May be created in the image synthesizing unit 4 using the divided images # 1 and # 2 stored in the storage unit. However, since it is not the final synthesized image, the corresponding address is simply calculated by the comparison range extracting unit 10. Good.

Here, in order to precisely connect the divided image # 1 and the divided image # 2, a reference block S is set on the reference divided image (here, referred to as # 1) and synthesized. It is necessary to set a search range T in the divided image (here, # 2) and perform matching. An image comparison range for performing this matching is extracted from the divided image. First, in FIG. 12, a rectangular reference block S painted black is set at a predetermined position of the divided image # 1. In the present embodiment, the positions are the upper and lower central portions of the right side of the divided image # 1, and the divided image # 1 and the divided image # 1
2 and around the center of the overlap area W. However, if the rectangular search range T indicated by halftone dots does not exist in the overlap area W, the true position may not be calculated. The position of is determined. The position of the reference block S is determined by the distance from the origin and the vertical and horizontal dimensions of the reference divided image, and the coordinates of the origin of the reference divided image are divided with respect to the origin of the composite image window 23 itself. Since it differs for each image, it is necessary to calculate the position of the reference block S for each reference divided image, and to calculate the position of the search range T in the adjacent divided image based on the position of the reference block S. There is. That is,
The calculation of the position of the reference block S and the position of the search range T is obtained by calculating each time for the two adjacent divided images that have moved, but the reference block S with respect to the reference coordinates (origin) of one arbitrary divided image. Are determined in advance. Further, although the search range T is for another divided image, the point of calculation so as to surround the reference block S with a predetermined width is predetermined. In an example described later, the search range T is calculated so as to surround the reference block S with a width of 40 pixels in both the upper, lower, left, and right directions.

Now, the synthesis accuracy of the image obtained by the user moving the reduced divided image and roughly aligning it is ± 40 pixels in both the vertical and horizontal directions (± 10 pixels on the actual screen because the divided image is reduced to ４). If the reference block S falls within the range of (pixels), the size and position of the reference block S must be selected so as not to deviate from the overlap area W even if the reference block S moves by ± 40 pixels. In FIG. 12, as an example, if the origin coordinates of the reference divided image are (0, 0) so as to satisfy these conditions, (540, 120),
(589, 120), (540, 359), (589,
A rectangular area of 50 × 240 having a vertex at 359) is defined as the position and size of the reference block S. The search range T is programmed to have a size obtained by expanding the reference block S by ± 40 pixels. That is,
The search range T on the divided image to be superimposed in the virtual composite image 34 is (500, 80), (62)
9,80), (500,399), (629,399)
Is a rectangular area having a size of 130 × 320 having a vertex as a vertex. In FIG. 12, the upper side of the search range T and the divided image #
2 coincides with each other, which is a coincidence, and the upper left coordinate of the divided image # 2 is, for example, (400, 7).
0) and (400, 60), the coordinates of the four vertices of the search range T are the same as above. FIG. 17 is an enlarged view of the overlap area W in FIG.
It is rewritten to show the coordinate values together. In addition,
FIG. 1 shows the result of image synthesis 35 based on synthesis information described later.
FIG. 18 is an enlarged view of the overlap region W in FIG. 15 rewritten to show a detailed X coordinate value and Y coordinate value. The state at the time of rough alignment by the user's hand (Fig. 1
2, FIG. 17) and the state (FIGS. 15 and 18) at the time when the image combining based on the combining information is performed (the details will be described later).

The above-described reference block S and search range T are actually illustrated on the divided image storage unit 2 (the divided memory in the correspondence table 8a between reduced images and divided images in FIG. 3) in FIG. 13 (a) and 13 (b).

That is, the divided image # 1 is (x1, y1) to (x1 + 63) at the coordinates of the divided image storage unit 2.
9, y1 + 479) (the size of the divided image is 640 × 480 pixels), the coordinates of the four vertices of the reference block S are both the X coordinate and the Y coordinate. [Reference block on virtual composite image 34 Coordinates) −
[Reference coordinates of divided image # 1 on virtual composite image 34]
+ [Reference coordinates of the divided image # 1 on the divided image storage unit 2]. Reference coordinates are (x1, y1)
It is.

For example, regarding the X coordinate at the upper left of the reference block S, the coordinates on the virtual composite image 34 are “540” and the divided image # 1 on the virtual composite image 34 from FIG.
Is "0", the divided image # on the divided image storage unit 2
Since the reference coordinate of No. 1 is “x1”, 540−0 + x1
= X1 + 540. Regarding the upper left Y coordinate, the coordinate on the virtual composite image 34 is “120”, the reference coordinate of the divided image # 1 on the virtual composite image 34 is “0”, and the divided image on the divided image storage unit 2 Since the reference coordinate of # 1 is “y1”, 120−0 + y1 = y1 + 120. The X coordinate at the lower right of the reference block S is shown in FIG.
Thus, the coordinates on the virtual composite image 34 are “589”, the reference coordinates of the divided image # 1 on the virtual composite image 34 are “0”, and the reference coordinates of the divided image # 1 on the divided image storage unit 2. Is "x1", so 589-0 + x1 = x1 + 58
It becomes 9. Regarding the lower right Y coordinate, the coordinate on the virtual composite image 34 is “359”, the reference coordinate of the divided image # 1 on the virtual composite image 34 is “0”, Since the reference coordinates of image # 1 are “y1”, 35
9-0 + y1 = y1 + 359.

As a result, as shown in FIG. 13A, the reference block S on the divided image # 1 has (x1 + 540, y1 + 120), (x
1 + 589, y1 + 120), (x1 + 540, y1 +
359) and (x1 + 589, y1 + 359) as vertices, and becomes a 50 × 240 rectangular area (see FIG. 17 for the size).

The divided image # 2 is (x2, y2) to (x2 + 639, y) in the coordinates of the divided image storage unit 2.
2 + 479), the coordinates of the four vertices of the search range T are X coordinate and Y coordinate, [the coordinates of the search range on the virtual synthetic image 34] − [the coordinate of the virtual synthetic image 34 The reference coordinates of the divided image # 2] + [the reference coordinates of the divided image # 2 on the divided image storage unit 2] can be calculated. Reference coordinates are (x2, y2)
It is.

For example, as for the X coordinate at the upper left of the search range T, from FIG. 12, the coordinate on the virtual composite image 34 is “500”, and the reference coordinate of the divided image # 2 on the virtual composite image 34 is Since the reference coordinate of the divided image # 2 on the divided image storage unit 2 is “x2”, “500” is used.
+ X2 = x2 + 100. Regarding the upper left Y coordinate, the coordinate on the virtual composite image 34 is “80”, and the reference coordinate of the divided image # 2 on the virtual composite image 34 is “8”.
0 ", and the reference coordinate of the divided image # 2 on the divided image storage unit 2 is" y2 ", so that 80-80 + y2 = y2.
As for the X coordinate at the lower right of the search range T, from FIG. 12, the coordinates on the virtual composite image 34 are “629”, and the reference coordinates of the divided image # 2 on the virtual composite image 34 are “40”.
0 ”, and the reference coordinates of the divided image # 2 on the divided image storage unit 2 are“ x2 ”, so that 629−400 + x2 = x2 + 2
29. Regarding the lower right Y coordinate, the coordinate on the virtual composite image 34 is “399”, the reference coordinate of the divided image # 2 on the virtual composite image 34 is “80”, Since the reference coordinates of image # 2 are “y2”,
399−80 + y2 = y2 + 319.

As a result, the search range T on the divided image # 2
Is (x2 + 100, y, as shown in FIG.
2), (x2 + 229, y2), (x2 + 100, y2)
+319) and (x2 + 229, y2 + 319) as the vertices and become a 130 × 320 rectangular area (see FIG. 17 for the size).

Here, the fact that there is a correlation between the coordinates of the search range T and the coordinates of the reference block S, that is, the fact that the coordinates of the search range T are calculated based on the coordinates of the reference block S will be described. I do. The X coordinate at the upper left of the reference block S is “x1 + 540”, and the X coordinate at the upper left of the search range T is “x2 + 100”. In FIG. 12, the X coordinate of the upper left of the reference block S in the divided image # 1 is “54”.
0 ". Since the search range T has a width in the X direction surrounding the reference block S of 40 pixels, the X coordinate at the upper left of the search range T is" 500 "from 540-40 = 500. The X coordinate of the reference coordinate of the divided image # 2 including the range T is “400.” The X coordinate “50” at the upper left of the search range T
The difference between “0” and the X coordinate “400” of the reference coordinate of the divided image # 2 is 500−400 = 100.
00 ”is the X coordinate“ x2 + 100 ”at the upper left of the search range T
Is "100". The upper left Y coordinate of the reference block S is “y1 + 120”, and the upper left Y coordinate of the search range T is “y2”. In FIG. 12, the upper left Y coordinate of the reference block S in the divided image # 1 is “12”.
0 ". Since the search range T has a width of 40 pixels in the Y direction surrounding the reference block S, the upper left Y coordinate of the search range T is" 80 "from 120-40 = 80. The Y coordinate of the reference coordinates of the divided image # 2 including the range T is “8”.
0. The difference between the upper left Y coordinate “80” of the search range T and the Y coordinate “80” of the reference coordinate of the divided image # 2 is 80−
Although 80 = 0, the difference “0” is “0” of the Y coordinate “y2” = “y2 + 0” at the upper left of the search range T. Similarly, the lower right X coordinate of the reference block S is “x1 + 589”, and the lower right X coordinate of the search range T is “x2 + 229”. In FIG. 12, the divided image #
The X coordinate at the lower right of the reference block S at 1 is “589”. The search range T is 4 in the X direction surrounding the reference block S.
Since the number of pixels is 0, the X coordinate at the lower right of the search range T is 589.
From + 40 = 629, it is “629”. The X coordinate of the reference coordinates of the divided image # 2 including the search range T is “40”.
0. The difference between the X coordinate “629” at the lower right of the search range T and the X coordinate “400” of the reference coordinate of the divided image # 2 is 6
29−400 = 229, and the difference “229”
Is “2” of the X coordinate “x2 + 229” at the lower right of the search range T.
29 ". The lower right Y coordinate of the reference block S is" y1 + 359 "and the lower right Y coordinate of the search range T is" y2 + 319. "In FIG. The lower right Y coordinate of the reference block S is “35”.
9 ". Since the search range T has a width in the Y direction surrounding the reference block S of 40 pixels, the Y coordinate at the lower right of the search range T is" 399 "from 359 + 40 = 399. The Y coordinate of the reference coordinate of the divided image # 2 including T is “80.” The Y coordinate “39” at the lower right of the search range T
The difference between “9” and the Y coordinate “80” of the reference coordinates of the divided image # 2 is 399−80 = 319, and this difference is “31”.
9 ”is“ 319 ”of the Y coordinate“ y2 + 319 ”at the lower right of the search range T. As described above, the coordinates of the search range T are calculated based on the coordinates of the reference block S. become.

As described above, the image comparison range for extracting the synthesis information necessary for synthesizing the two adjacent divided images # 1 and # 2 is limited. The image comparison range is a general term for the range of the reference block S and the search range T. This corresponds to step S4 in the flowchart of FIG.

It is to be noted that the user operates the divided image window 2
2 in the composite image window 2
When moving the next reduced divided image B after moving to 3, the divided image B partially overlaps the reference block S at a predetermined position on the divided image A and surrounds the reference block S. The divided image B is moved so that the search range T is secured. However, such a moving operation can be realized by a rough operation. At this time, when the display of the translucent composition is performed as shown in FIG. 9C, or the alternate display of the front surface and the back surface is performed as shown in FIG. A superposition in a state satisfying the above conditions of A and B can be easily performed. If the overlap area W 'of the two divided images A and B is relatively large and a blank is generated at the boundary, a reference for alignment cannot be found in FIG.
The movement may be performed in a mode having a saw-like boundary as shown in (c) to (e).

Next, in step S4, synthesis information necessary for synthesis is extracted using the reference block S and the search range T defined by the comparison range extraction unit 10 in the comparison range limitation unit 7. As shown by the arrows in FIGS. 13A and 13B, the coordinate information of the reference block S is extracted from the coordinate information of the divided image # 1 in the divided image storage unit 2, and
The coordinate information of the search range T is extracted from the coordinate information of the divided image # 2 in the divided image storage unit 2. This reference block S
The coordinate information of the search range T and the coordinate information of the search range T
From 0, it is transferred to the synthesis information extraction unit 3. The extracted reference block S and search range T are also shown in FIG. 14 for explaining the operation of block matching. this is,
This indicates that the reference block S and the search range T extracted as described above are used for extracting synthetic information. In FIGS. 13B and 14, a rectangle within the search range T is a reference block equivalent range S ′ indicating a range corresponding to the reference block S.

The synthetic information necessary for synthesizing the divided images includes information such as a motion vector between the divided images, an enlargement ratio, rotation, and the like, and synthesized information such as luminance correction information between the divided images.
For example, in the case of matching information, the synthetic information extracting unit 3
Performs block matching as shown in FIG. 14 in its internal working memory. As a result, the divided image #
Reference block S of divided image # 1 inside search range T
And the block having the highest correlation with the motion vector M
V is output. That is, the search range T of the divided image # 2
, A divided image # 2 having a pixel density pattern that best matches the pixel density pattern of the reference block S
The corresponding block S ″ at the top (within the search range T) is found, and the motion vector MV from the reference block equivalent range S ′ toward the corresponding block S ″ is calculated. In the case of FIG. 14, the motion vector MV is +20 pixels in the X direction and Y
It has a vector component for -20 pixels in the direction (the Y-coordinate axis is positive downward). That is, the motion vector MV is
It is represented by (+20, -20). If necessary,
At the time of this block matching, other combined information such as information such as an enlargement ratio and rotation is also obtained. This corresponds to step S5 in the flowchart of FIG.

The image synthesizing section 4 is based on the synthesizing information obtained from the synthesizing information extracting section 3 and is stored in the split image accumulating section 2 and displayed in the synthetic image window 23 as an object of synthesis. Are synthesized. When the motion vector MV (= (+ 20, -20)) which is the synthetic information obtained from the synthetic information extracting unit 3 is subtracted from the virtual synthetic image 34 shown in FIG. 12, the synthetic image shown in FIG. 35 is generated. This corresponds to step S6 in the flowchart of FIG.

FIG. 15 shows the case of the divided image storage section 2. FIG. 16 shows a reduced composite image 36 in the composite image window 23 corresponding to FIG. FIG. 18 is a diagram in which the portion of the overlap area W in FIG. 15 is enlarged and rewritten to show detailed X coordinate values and Y coordinate values. Virtual composite image 34 at the time of rough alignment by the user shown in FIGS. 12 and 17
And the synthetic information (motion vector M
Synthesized image 3 at the time when image synthesis based on V) is performed
5, the motion vector MV (= (+ 20,-
The X component for canceling the X component “+20” of 20)) is “−2”.
0 ”, and the X-direction cancellation vector component is represented by V _{X. The} X component that cancels the Y component“ −20 ”is“ +
20 ", and the Y-direction cancellation vector component is V _Y
Expressed by A vector having a vector component of (V _X , V _Y ) = (− 20, +20) is the correction vector CV. In the above description, the motion vector MV is subtracted, but the correction vector CV may be added. Original split image #
2 is indicated by a two-dot chain line 37, and the original search range T ′ is indicated by a two-dot chain line 38. The reference coordinates of the divided image # 2 are (40) in FIG.
(0, 80), the operations of 400−20, 80 + 20 are performed, and are shifted to (380, 100) as shown in FIG. Similarly, regarding the vertex coordinates of the search range T in FIG. 17, (500, 80) is (480, 1) in FIG.
00), and (629, 80) in FIG.
(609, 100) and (500, 100) in FIG.
399) is shifted to (480, 419) in FIG. 18, and (629, 399) in FIG. 17 is (609, 41) in FIG.
It has shifted to 9). In each case, the original divided image # 2,
Original search range T 'coordinates correction vector X cancellation direction vector component V _X = -20 the CV of shifted by the Y-direction canceling vector components V _Y = + 20. It goes without saying that the coordinates of the four vertices of the reference block S do not change.

The data of the synthesized image 35 obtained by image synthesis is transferred to the synthesized image storage unit 5 and stored. Further, the coordinate information of the divided image # 2 after the image synthesis is performed is stored in the built-in memory of the image synthesis unit 4. The coordinate information stored in the built-in memory is transferred to the comparison range extracting unit 10 at the time of extracting the image comparison range and combining information in the next cycle.

For synthesizing the above divided images,
The processing of step S4 for extracting the image comparison range, step S5 for extracting the combined information, and step S6 for combining the images is repeated until it is determined that the combining of all the divided images is completed.
This determination corresponds to step S7 in the flowchart of FIG. In the case of the example shown in FIG. 8, symbolically describing, after combining # 1 (A) and # 2 (B) with # 12 (AB), # 12 (AB) and # 3 (C) To # 123 (A
BC), and # 1237 (ABCG) from # 123 (ABC) and # 7 (G), and # 1237 (ABC
G) and # 8 (H) to # 12378 (ABCGH) are combined in order to combine the divided images, and five divided images are combined. Of course, any number of sheets can be combined as long as the memory allows. By expanding the area of the synthesized image window 23 in the upward direction, the synthesis of the divided images in the two-dimensional direction can be similarly processed. The synthesis order is (HG) → (HGC) → (HGCB) → (H
GCBA) or a completely random order. Further, on the contrary, the reference block S may be set to the divided image # 2, and the search range T may be set to the divided image # 1.

The processing from step S4 to step S7 is executed on a CPU (Central Processing Unit), and the processing status is not displayed on the screen.

After the synthesis of all the divided images is completed, the data of the synthesized image stored in the synthesized image storage unit 5 is transferred to the synthesized image display unit 11, and the data is stored in the video memory built in the synthesized image display unit 11. In both the vertical and horizontal directions, the image is reduced to 1/4 and stored. As shown in FIG. 6, a composite image 39 in a reduced state in which five reduced divided images are composited is displayed in the composite image window 23. This corresponds to step S8 in the flowchart of FIG.

The user visually checks whether or not the composite image 39 in the reduced state displayed in the composite image window 23 has been properly combined. If it is determined that the combined image 39 has been combined normally, an OK instruction is issued by operating the mouse. give. In this case, the determination of the success of the combination in step S9 is affirmative, and the process proceeds to step S13. Conversely, when it is determined that the images have not been properly synthesized, as shown in FIG. 6, a defective portion a in the reduced synthesized image 39 displayed in the synthesized image window 23 is clicked. In this case, the determination in step S9 is negative, and two divided images corresponding to the clicked location a are specified. This designation is performed by the divided image designation unit 12. This is,
This corresponds to step S10 in the flowchart of FIG.

The identification codes of the two divided images specified by the divided image specifying unit 12 based on the click of the defective part are given to the composite image display unit 11. Then, as shown in FIG. 6, the composite image display unit 11 newly develops the fine composite image window 40 in a state of being overwritten on the divided image window 22, and the new composite image window 40 corresponds to the click position a. Two adjacent divided images are read from the divided image storage unit 2 and displayed in the original size that is not reduced. In the case of the illustrated example, the two divided images displayed in the fine composite image window 40 are reduced divided images C,
Since it is G, the divided image # 3 and the divided image # 7 are determined based on the correspondence table 8a between the reduced image and the divided image. The resolution of the divided images # 3 and # 7 is 16 times that of the reduced divided images C and G. The enlarged divided images # 3 and # 7 are indicated by reference numerals 41 and 42. This enlarged display corresponds to step S11 in the flowchart of FIG.

Next, as shown in FIG. 7, the button b in the enlarged divided image 42 corresponding to the divided image # 7 is clicked with the mouse cursor, and the enlarged divided image 42 is dragged. Divided image # 3
Is moved toward the enlarged divided image 41 corresponding to and the positioning is performed again. If the position cannot be accurately adjusted by simple movement alone, the enlarged divided image 42 is rotated by dragging the button c with the mouse, or the enlarged divided image 42 is enlarged or reduced by clicking the button d. The enlargement / reduction may be performed on the other enlarged divided image 41. The operation is repeated until the alignment of the two enlarged divided images 41 and 42 is maintained. This is step S in the flowchart of FIG.
Equivalent to 12.

Next, proceeding to step S4, for the two adjacent enlarged divided images 41 and 42, the image comparison range extraction (step S4), the synthesis information extraction (step S5), and the image synthesis (step S6) are performed as described above. ) To display the combined image in the combined image window 23 (step S8).

Image synthesis on the fine synthesized image window 40 is repeated until all the divided images are successfully synthesized.

The user visually checks whether or not the composite image 39 in the reduced state displayed in the composite image window 23 has been properly combined. When it is determined that the combined image 39 has been combined normally, an OK instruction is given by operating the mouse. give. In this case, the determination of the success of the combination in step S9 is affirmative, and the process proceeds to step S13. The data of the synthesized image of the plurality of divided images that have been successfully synthesized is stored in the synthesized image storage unit 5 in step S6.

When outputting a normally synthesized image, the user gives an output instruction using a mouse.
When the output is not performed, an instruction to end the work is given. The composite image output unit 6 determines whether or not an output instruction has been given (step S13), and if not, proceeds to step S14 to determine whether or not there has been an instruction to end the work. If there is no instruction to end the work, the process returns to step S13. When an output instruction is given, the composite image output unit 6 outputs the composite image storage unit 5.
, Reads out the data of the composite image and outputs it to a printer or transfers it to another device via a network. These operations are performed using, for example, the menu bar 21 on the display. When there is an instruction to end the work, all operations are ended.

By synthesizing the divided images by the method described above, the amount of calculation at the time of matching can be greatly reduced as compared with the conventional technique. For example, (a) of FIG.
3), (b3), whereas was search range T ₁ is 320 × 480 pixels in the case of matching the conventional technique shown in (c3), in the present embodiment, in FIG. 19 As shown, the search range T can be sufficiently small (it can be seen from FIG. 17 that the search range T requires only 130 × 320 images), and the amount of calculation for matching can be reduced to about 27% in total. If the size of the reference block S is reduced to, for example, 30 × 60 pixels, and the search range T surrounds the reference block S with a width of ± 40 pixels both vertically and horizontally, the size becomes 110 × 140 pixels. In this case, The calculation amount can be reduced to about 10% as compared with the conventional technology.

[Second Embodiment] FIG. 20 is a block diagram showing a configuration of an image synthesizing apparatus according to a second embodiment of the present invention. The image synthesizing apparatus according to the second embodiment is similar to the image synthesizing apparatus according to the first embodiment.
The configuration of the image synthesizing apparatus shown in FIG.
Is removed, and it is assumed that the combination in step S6 of FIG. 4 in the case of the first embodiment always succeeds once. The function of each unit is the same as that of the first embodiment, and the operation is as shown in the flowchart of FIG.

[Embodiment 3] FIG. 22 is a block diagram showing a configuration of an image synthesizing apparatus according to Embodiment 3 of the present invention. The image synthesizing apparatus according to the third embodiment corresponds to the first embodiment.
The configuration of the image synthesizing apparatus shown in FIG.
And a new image synthesis interrupting unit 13 is provided. The image synthesis suspension unit 13 corresponds to the cancel button 25. When the cancel button 25 is clicked on with a mouse cursor, the image composition suspending unit 13 is activated, and gives an interruption command to the composite information extracting unit 3, the image compositing unit 4, the composite image accumulating unit 5, and the comparison range limiting unit 7, and the register of each unit is registered. And reset the value of the flag to the initial value.
In particular, the data of the composite image stored in the composite image storage unit 5 is deleted. In the case of the first embodiment, when there is a defect in the image composition as determined in step S9, the process is started again from the beginning. The function of each unit is the same as that of the first embodiment, and the operation is as shown in the flowchart of FIG.

[0069]

According to the image synthesizing apparatus of the first aspect of the present invention, there is provided a comparison range limiting means for narrowing a range to be compared between adjacent divided images in the divided image storage means, and a synthetic information extracting means. Is configured to compare adjacent divided images in the limited image comparison range by the comparison range limiting means and extract synthesis information necessary for synthesis, so that the user roughly aligns the divided images to be synthesized. Then, since the comparison information is extracted after narrowing the comparison range of two adjacent divided images narrowly, the amount of calculation when extracting the synthesis information in the narrow image comparison range is reduced. The load on the memory can be reduced, and the processing time can be shortened. Further, since the image comparison range is narrow, the probability of occurrence of a plurality of the same patterns during pattern matching is reduced, and the matching accuracy can be improved. Further, the present invention can be applied to a general-purpose camera.

According to the image synthesizing device of the second aspect of the present invention, the periphery of the reference block set for one of the divided images in the overlap region of the adjacent divided images displayed in a superimposed state on the synthesized image display means. And a comparison range extracting means for obtaining a limited image comparison range by setting a search range enclosing a narrow width in the other divided image, and combining information extracting means for referring to one of the divided images within the search range. Since the corresponding image area is obtained by pattern matching with the block in the other divided image and the synthesis information is obtained based on the comparison between the reference block and the corresponding image area, the desired divided image to be synthesized by the user is desired. By simply roughly superimposing, the reference block is set in one of the divided images, and the reference block around the reference block is set in the other divided image. Is set search range surrounded by a narrow range, thereby, the image comparison range can be narrow. Then, a corresponding image area for pattern matching with the reference block in the narrow image comparison range is obtained in the other divided image. However, since the matching range at this time is narrow, the amount of calculation for extracting the synthetic information is reduced, and the calculation processing is reduced. The load on the device and the memory can be reduced, and the processing time can be shortened. Further, since the image comparison range is narrow, the probability of occurrence of a plurality of the same patterns during pattern matching is reduced, and the matching accuracy can be improved.

According to the third aspect of the present invention, there is provided a divided image designating means for designating two adjacent divided images to be recombined when the composite state of the displayed composite image is defective. In addition, the composite image display means is configured to enlarge and display the two designated divided images and to superimpose one enlarged divided image on the other enlarged divided image. For the adjacent enlarged divided image, a limited image comparison range is determined by setting a search range surrounding the reference block set in one enlarged divided image with a narrow width in the overlap region in the other enlarged divided image in the overlap area. Therefore, even if the composite state of the composite image once created is defective, the image comparison range extraction is performed by narrowing down to the defective divided image. Since Synthetic information extraction Compose can retry a better efficiency as compared with the case where the first from totally redone. Moreover, when retrying,
Since two adjacent divided images having a defect are enlarged and displayed to perform image comparison range extraction and synthesis information extraction, matching accuracy is high, retry is not required again, and workability can be improved.

According to the image synthesizing apparatus of the fourth aspect of the present invention, the display control is performed in a state where the adjacent divided images are translucently synthesized in the overlap region of the divided images. According to the image synthesizing apparatus of the fifth aspect of the present invention, in the overlap region of the divided images, the display control is performed such that the adjacent divided images are switched between the front and the back at a fixed time interval. When moving the next divided image with respect to the previously moved divided image, in a state where the moving condition that the image comparison range including the reference block and the search range is included in the overlap region of both divided images is satisfied, Rough positioning by the user can be facilitated.

According to the image synthesizing apparatus of the sixth aspect of the present invention, in the overlapping area of the divided images, the adjacent divided images are divided into a saw-like shape at the boundary and arranged on the front and the back. Since the overlap is large at the time of the divided imaging, the boundary becomes blank when two adjacent divided images to be combined are roughly superimposed, so that the position of the superimposition cannot be specified. Even in such a case, rough alignment can be easily performed by arranging both divided images on the front surface and the rear surface with a sawtooth boundary.

According to the image synthesizing method of the seventh aspect of the present invention, of the plurality of displayed reduced divided images, roughly overlapping the reduced divided images to be synthesized,
Setting a reference block located at a predetermined position with respect to the origin of an arbitrary divided image in one of the superimposed adjacent divided images, and surrounding the reference block in the other divided image. Setting a search range enclosing with a narrow width; extracting a corresponding image area of pattern matching in the other divided image with respect to a reference block of the one divided image within the search range; Extracting synthesis information necessary for image synthesis based on comparison with the image area; and correcting the alignment of the other divided image with respect to the one divided image based on the extracted synthesized information to perform both splitting. And synthesizing an image. Just fit it, the reference block is set to one of the divided images, a search range is set to surround the reference block to the other split image in a narrow range,
Thereby, the image comparison range can be narrowed. Then, a corresponding image area for pattern matching with the reference block in the narrow image comparison range is obtained in the other divided image. However, since the matching range at this time is narrow, the amount of calculation when extracting synthetic information is reduced,
The load on the processing unit and the memory can be reduced, and the processing time can be shortened. Further, since the image comparison range is narrow, the probability of occurrence of a plurality of the same patterns during pattern matching is reduced, and the matching accuracy can be improved.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an image synthesizing apparatus according to Embodiment 1 of the present invention.

FIG. 2 is a block diagram showing a specific configuration of a comparison range limiting unit of the image synthesizing apparatus according to the first embodiment together with peripheral parts;

FIG. 3 is a configuration diagram of a correspondence table of reduced images and divided images incorporated in the divided image display unit of the image synthesizing apparatus according to the first embodiment.

FIG. 4 is a flowchart illustrating an operation of the image synthesizing apparatus according to the first embodiment;

FIG. 5 is a screen display state diagram (multiple display of reduced divided images and movement and superposition of reduced divided images) for explaining the operation of the image synthesizing apparatus according to the first embodiment;

FIG. 6 is a screen display state diagram for explaining the operation of the image synthesizing apparatus according to the first embodiment (a target reduced divided image is enlarged and displayed when there is a defect in image synthesis).

FIG. 7 is a screen display state diagram (movement of an enlarged divided image) for explaining the operation of the image synthesizing apparatus according to the first embodiment;

FIG. 8 is a screen display state diagram (multiple display of reduced divided images and movement / superposition of reduced divided images) for explaining the operation of the image synthesizing apparatus according to the first embodiment;

FIG. 9 is an explanatory diagram of a method of displaying adjacent superimposed reduced divided images in the first embodiment.

FIG. 10 is an explanatory diagram of another method of displaying adjacent superimposed reduced divided images in the first embodiment.

FIG. 11 is a display state diagram of a reduced composite image according to the first embodiment.

FIG. 12 is a diagram illustrating a state of a virtual composite image of the divided images in the divided image storage unit according to the first embodiment.

FIG. 13 is a diagram illustrating the relationship between two actual divided images in the divided image storage unit according to the first embodiment, a reference block in each divided image, and a search range.

FIG. 14 is an explanatory diagram (motion vector calculation) of block matching operation in the first embodiment.

FIG. 15 is a diagram illustrating a second example in the divided image storage unit in the first embodiment.
FIG. 10 is a diagram illustrating a state of a combined image obtained by combining two divided images.

FIG. 16 is a display state diagram of a reduced composite image corresponding to FIG. 15 in the first embodiment.

FIG. 17 is an enlarged view of the overlap region in which the overlap region of FIG. 12 in the rough alignment state according to the first embodiment is enlarged and rewritten to show a detailed X coordinate value and Y coordinate value together; is there.

FIG. 18 is a diagram illustrating a screen synthesis state according to the first embodiment;
5 is an enlarged view of an overlap area in which a portion of an overlap area of No. 5 is enlarged and rewritten to show detailed X coordinate values and Y coordinate values together.

FIG. 19 is an explanatory diagram showing a size of a search range to cope with a case where the overlap between two divided images fluctuates in the first embodiment.

FIG. 20 is a block diagram showing a configuration of an image synthesizing apparatus according to Embodiment 2 of the present invention.

FIG. 21 is a flowchart for explaining the operation of the image synthesizing apparatus according to the second embodiment;

FIG. 22 is a block diagram showing a configuration of an image synthesizing apparatus according to Embodiment 3 of the present invention.

FIG. 23 is a flowchart illustrating an operation of the image composition device according to the third embodiment;

FIG. 24 is a block diagram showing a configuration of an image synthesizing apparatus according to a conventional technique.

FIG. 25 is an explanatory diagram of an image synthesizing operation of an image synthesizing apparatus according to a conventional technique (when image continuity is lost).

FIG. 26 is an explanatory diagram of an image synthesizing operation of an image synthesizing apparatus according to a conventional technique (when image continuity is maintained).

FIG. 27 is an explanatory diagram showing the size of a search range for coping with a case where the overlap between two divided images fluctuates in image synthesis by an image synthesis apparatus according to a conventional technique.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Divided image input part 2 ... Divided image storage part 3 ... Synthetic information extraction part 4 ... Image synthesis part 5 ... Synthetic image storage part 6 ... Synthetic image output part 7 ... Comparison range limitation part 8 ... ... Divided image display unit 9... Divided image moving unit 10... Comparison range extraction unit 11... Composite image display unit 12... Divided image designation unit 13. 22 ... divided image window 23 ... composite image window 24 ... start button 25 ... cancel button 31a to 31h, 31x, 31y ... reduced divided image 33 ... reduced composite image 34 ... virtual composite image 35: Composite image 36: Reduced composite image 39: Reduced composite image 40: Fine composite image window 41, 42: Enlarged divided image W: Overlap area S: Reference block H S ': Reference block equivalent range S ": Corresponding block to reference block T: Search range MV: Motion vector CV: Correction vector

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Kazuyuki Nago 22-22 Nagaikecho, Abeno-ku, Osaka-shi, Osaka Inside Sharp Corporation

Claims (7)

[Claims] 1. A divided image accumulating means for accumulating an inputted divided image, and synthetic information extracting means for comparing adjacent divided images in the divided image accumulating means and extracting synthetic information necessary for image synthesis. Image synthesizing means for synthesizing adjacent divided images input from the divided image accumulating means based on the synthesizing information from the synthesizing information extracting means to generate a synthesized image; and synthesizing the synthesized image synthesized by the image synthesizing means. An image synthesizing apparatus comprising: a synthesized image accumulating means for accumulating; and a comparison range limiting means for narrowing a range in which adjacent divided images in the divided image accumulating means are to be compared with each other. The image processing apparatus is configured to compare adjacent divided images in an image comparison range limited by the range limiting unit and extract synthesis information necessary for synthesis. An image synthesizing apparatus characterized by being. 2. A divided image display means for displaying a plurality of divided images, a divided image moving means for moving and overlapping the divided images displayed by the divided image display means, Composite image display means for displaying the obtained divided images in a superimposed state, and a periphery of a reference block set to one of the divided images in an overlapping area of adjacent divided images displayed in a superimposed state in the composite image display means And a comparison range extracting unit that obtains a limited image comparison range by setting a search range that encloses a narrow width in the other divided image. Then, an image area corresponding to the reference block of one of the divided images is obtained by pattern matching in the other divided image, and the reference block and the corresponding image area are obtained. 2. The apparatus according to claim 1, wherein the composition information is obtained based on a comparison with the area.
An image synthesizing apparatus according to claim 1. 3. The comparison range limiting unit includes a divided image designating unit that designates two adjacent divided images to be recombined when the combined state of the combined image displayed by the combined image display unit is defective. The composite image display means is configured to enlarge and display the two designated divided images and to superimpose one enlarged divided image on the other enlarged divided image. For the adjacent enlarged divided image, a limited image comparison range is determined by setting a search range surrounding the reference block set in one enlarged divided image with a narrow width in the overlap region in the other enlarged divided image in the overlap area. The image synthesizing device according to claim 2, wherein the image synthesizing device is configured as follows. 4. The divided image moving means is configured to perform display control in a state in which adjacent divided images are translucently synthesized in an overlap area of the divided images. 3. The image synthesizing apparatus according to any one of 3). 5. The divided image moving means is configured to control display of an adjacent divided image in a state of switching between a front surface and a rear surface at predetermined time intervals in an overlap region of the divided images. Claims 1 to 3
The image synthesizing device according to any one of the above. 6. The divided image moving means is configured to divide an adjacent divided image into a saw-like shape at the boundary in the overlap region of the divided images and to arrange the divided images on the front surface and the rear surface. The image synthesizing device according to any one of claims 1 to 3. 7. A step of inputting and accumulating divided images, a step of reducing and displaying a plurality of divided images, and roughly superimposing reduced divided images to be synthesized from among the displayed plurality of reduced divided images. Aligning, setting a reference block at a predetermined position with respect to the origin of an arbitrary divided image in one of the superimposed adjacent divided images, and setting the reference block in the other divided image. Setting a search range surrounding a block with a narrow width; extracting a corresponding image area of pattern matching in the other divided image with respect to a reference block of the one divided image within the search range; Extracting synthesis information necessary for image synthesis based on a comparison between the block and the corresponding image area; An image correcting step of correcting the registration of the other divided image with respect to the one divided image based on the extracted combined information to combine the two divided images, and accumulating the combined image. Synthesis method.