JPH07273977A - Image handling device - Google Patents

Abstract

PURPOSE:To obtain a connected image which has no shift by performing switching control on the basis of the forms of images of overlapping areas by utilizing one of the image parts that are put one over the other when the images are connected. CONSTITUTION:An input image is made into partial images at a cumulative addition part 115 and they are stored as contrast data in a memory. A CPU 2 controls a lens 103 on the basis of the data, a coordinate transformation part 110 performs coordinate transformation, and a binarization circuit 119 performs binarization; while an output part 114 outputs the resulting data, they are held in a border buffer 111, and a position shift between both the images and their motion vectors are found by a detection part 112 and a calculation part 113. Then the two images outputted from the output part 114 and border buffer 111 are pasted while switched by a switching control means 5 dexterously at good timing. Consequently, a shift between feature parts which are present in the overlap area between partial images can be suppressed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention performs a process of combining partial images by a plurality of image pickup devices, which are used in a commercial camera (hereinafter referred to as an OA camera) for forming a high definition image. The present invention relates to an image handling device.

[0002]

2. Description of the Related Art Hitherto, the following OA camera has been proposed by the applicant of the present invention as an image handling apparatus using a predetermined method for obtaining an image with a large number of pixel configurations with a small number of pixel configurations.

That is, the predetermined method is to scan an image to be picked up by a mirror for each partial image and pick up each captured partial image corresponding to each of a plurality of image pickup devices. This is a method in which exposure is performed for each element and the partial images obtained in this manner are sequentially re-stitched to input a multi-pixel imaged target image in a large area. O as an example of its use
FIG. 19 is a block diagram for explaining the configuration of the A camera.

The construction and operation of the OA camera as the image handling apparatus will be described below with reference to FIGS.

In FIG. 19, the total reflection mirror 101 is
Incident light 102 is reflected in a predetermined direction to change the reflection angle with respect to the image to be captured (hereinafter, total reflection mirror 101
The total reflection mirror 1 is configured to be movable in order to change the angle of
On the optical axis of 01, from the side closer to the total reflection mirror 101,
A lens unit 10 including a diaphragm and a focusing lens
3, the half mirror 104 is provided in this order, and the first CCD 105 and the second CCD 106 are arranged as two imagers. First CCD 105
Of the incident light 102 receives the passing light 102 a that has passed through the half mirror 104, and the second CCD 106 reflects the reflected light 10 of the incident light 102 reflected by the half mirror 104.
2b can be received. Further, as shown in FIG. 20, in each position of the total reflection mirror 101, the first CCD 105 has a region which is a part of the image 200 to be captured and which is a substantially upper half of the partial images A, B, C, and D. Upper half images A1, B1, C1, and D1 corresponding to
Is exposed to the second CCD 106, and the same partial image A,
In order that the lower half images A2, B2, C2, and D2 corresponding to the substantially lower half regions of B, C, and D are exposed at the same time, the positional relationship between the two imagers is such that the optical axes of the two imagers are different from each other. It is configured to be shifted by a predetermined amount with respect to.
Here, FIG. 20 is an explanatory diagram for explaining an overlapping region and the like of each partial image.

Further, the incident light 102 taken in by the total reflection mirror 101 at the position of the initial angle is output from each imager and the corresponding first process circuit 107 or the second process circuit 108 is output. Then, it is converted into a video signal. When the pixels of the two CCDs as imagers are both 752 pixels × 580 pixels, the upper half image A1 consisting of 752 pixels × 580 pixels
(A region shaded by a solid line in FIG. 20),
The lower half image A2 of the same pixel size (the area shaded by dotted lines in FIG. 20) is the overlap margin A12 of 100 pixels × 580 pixels (the overlapping area of the areas shaded by solid lines and dotted lines in FIG. 20). ), The data is input to the vertical bonding unit 109 for uniform bonding. That is, for each video signal input to the vertical stitching unit 109, the upper half image A1 and the lower half image A2 are uniformly stitched together and captured by one imager of 1404 pixels × 580 pixels. It is converted into a video signal as if it were high (corresponding to the partial image A of the imaging target image 200 shown in FIG. 20).

Such operation is repeated every time the total reflection mirror 101 is scanned at a predetermined position.

Next, the upper half image A corresponding to the partial image A
1 and the lower half image A2 are captured, and then the CPU 116
By the action of the mirror driving unit 110 based on the instruction from, the total reflection mirror 101 is set at a position moved by a predetermined angle from the initial position. At the mirror-scanned position in this way, the total reflection mirror 101 is again subjected to the same processing as described above, and this time, the upper half image B1 corresponding to the substantially upper half region of the partial image B of the image to be captured 200, A video signal as if the lower half image B2 corresponding to the lower half area was evenly bonded and captured by one imager (corresponding to the partial image B of the image 200 to be captured shown in FIG. 20). Is converted to.

Further, in order to capture the partial images C and D, the area to be captured is changed each time the total reflection mirror 101 is sequentially moved by a predetermined angle (mirror scan), and the same processing as described above is repeated, respectively, and as a whole. You can get a big picture.

However, what is important here is that the partial images (the partial images A,
To further attach B, C, and D), see FIG.
As shown in FIG.
However, the overlapping partial images, such as the partial image A that was captured the first time and vertically joined and the partial image B that was captured the second time and formed in the same way, are determined in advance in each image. In addition, it is configured to have a predetermined overlapping portion, that is, an overlapping region of 1404 pixels × 72 pixels as shown in FIG.

That is, in order to perform such superimposition, as shown in FIG. 19, the overlapping area of the previously captured partial images (for example, as shown in FIG. 20, for example, the overlapping area Ab belonging to the partial image A). ) For temporarily holding), the overlapping area of the contents of the boundary buffer 111 and the partial image of the next captured partial image (for example, the overlapping region belonging to the partial image A). The content of the overlapping area (for example, the overlapping area Ba belonging to the partial image B) corresponding to the area Ab) is compared to find the characteristic portion of the image, that is, the same picture portion, and the shift between the pictures is detected. Motion vector detection unit 11 for
2. A deviation amount calculation unit 113 for calculating a deviation amount using the detection result, and a calculated deviation amount according to a partial image (for example, partial image A) previously fetched, and then Partial image captured (eg, partial image B)
The image data joined to each other is output from the output unit 114 by the coordinate conversion unit 110 or the like for correcting the position of ∘ and correctly overlapping the overlapping images.
The coordinate conversion referred to here is parallel movement or rotational movement, and includes conversion processing such as enlargement / reduction when target images have different sizes. In this way
Each time a mirror scan is performed, the partial images that overlap are successively stitched together.

Here, the following factors can be considered as factors that cause a shift in the mutual positional relationship of the captured overlapping partial images.

For example, the overlapping partial images captured by the mirror scan are configured to have a predetermined overlapping portion, that is, an overlapping region of 1404 pixels × 72 pixels as shown in FIG. That is, as described above, there is an error or variation in the scan amount (predetermined angular movement amount) of the total reflection mirror 101, or an image pickup target while sequentially capturing partial images at a predetermined timing. (Subject) changes, or conversely, a position shift on the capturing device side such as camera shake caused by an operator.

On the other hand, the lens unit 103 has the diaphragm, the focus lens, and the like as described above, and in order to control these at the time of image pickup, the first CC as an imager
The output signals of D105 and the second CCD 106 are sent to the cumulative addition unit 115.

The data that has been subjected to so-called cumulative addition processing in the cumulative addition unit 115 is sent to the CPU 116, and a control signal from the CPU 116 based on these data causes the AE
(Automatic exposure adjustment) drive and AF (automatic focusing)
Is driven.

Here, for example, the cumulative addition process in AF is performed by dividing a group of pixels forming a CCD as an imager into a predetermined area into one area, and each area is divided by each pixel. Focusing on the high-frequency components of the obtained image data and attempting to obtain the contrast data (the larger the cumulatively added data, the more the contrast is determined) by cumulatively adding these waveforms. Is.

The respective contrast data thus obtained are stored in a predetermined memory (not shown), each contrast data is read from this memory as needed, and the AF is controlled based on the data. The system is a system for sending a control signal for controlling the focus lens of the lens unit 103.

By the way, in the above-mentioned OA camera and the like, the present applicant has proposed the following processing when the joining processing is performed by bonding the partial images to each other.

The configuration of this OA camera is shown in FIG.
, And the contents of FIG. 19 and the binarization unit 11 already described.
Since it is almost the same except 9, the description thereof will be omitted.

That is, the main difference from FIG. 19 is that of FIG.
In No. 1, the image data coordinate-converted by the coordinate conversion unit 110 is binarized by the binarization unit 119, output through the output unit 114, and is held in the boundary buffer 111 and held therein. The point is to detect a motion vector based on the binarized data.

Usually, it is conceivable that the amount of positional deviation between partial images in the overlapping area is calculated by the amount of deviation calculating section 113, and the positions at which the images are switched are uniformly fixed.

For example, as shown in FIG. 23 (a), when the character "1" is straddled at a predetermined switching position at the left end of the partial image A, the partial image A is moved from the predetermined switching position. When the image is switched to the partial image B, the mutual images are joined in the middle of the character "1".
Here, FIG. 23A is a diagram illustrating a case where no deviation occurs in the process of combining the partial image A and the partial image B. FIG. 23 (a) is a diagram for explaining a case where no deviation occurs in the process of combining the partial image A and the partial image B, and FIG. 23 (b) shows the partial image A and the partial image B.
It is a figure explaining the case where a gap has arisen in the process of pasting with and.

FIG. 22A is a time chart for explaining the outline of the output state in the joining process, and FIG. 22B is an explanatory diagram for showing the image area. The processing operation of the OA camera shown in FIG. 21 will be described below with reference to FIGS.

First, a mirror scan is performed at the time of image pickup, and partial images A, B, C and D are obtained as shown in FIG.
As described above, the data of each partial image is A, as shown in the output 141 of FIG.
B, C and D are output in this order.

After adjusting the mutual positional deviations for the respective partial images output in this way, the position at which the partial images are switched at the time of joining may be switched at any position within a predetermined overlapping area. Basically, they can be joined.

Therefore, in this example, as shown in FIG. 22B, assuming that the start point of the partial image B, the start point of the partial image C, and the start point of the partial image D are the switching positions of the partial images, respectively. Areas of partial images are Ax, Bx, C
x, D. Therefore, as a time chart of the X output 142, A, B, C, D output from the output 141
Ax, Bx, Cx, D for the data
As shown in FIG. 22 (b), the bonding can be performed completely,
Joining of all partial images is completed.

[0027]

However, in such a configuration, since the position or timing at which the partial images are switched is always fixed, tentatively, there is a character at the position at which the partial images are switched, and misregistration correction is performed. If there is a slight deviation, as shown in FIG. 23 (b), there is a drawback that a misaligned image is output. That is, by detecting the motion vector, calculating the amount of displacement between the partial images, performing coordinate conversion, and thereby causing some misalignment in the process of returning the position of the displaced image to the original, For example, when a shift of about 1 pixel occurs, the above problem is unavoidable, and particularly when the image is a character, it is remarkable.

In the present invention, in consideration of such a problem of the image handling apparatus described with reference to FIG. 21, when the partial images are pasted together, the feature portion existing in the overlapping area of the partial images is displaced. An object is to provide an image handling device that does not occur.

[0029]

The present invention provides an image,
Regarding another image that partially overlaps with the one image, the one image and the other image are overlapped with each other, and a bonding image forming unit for performing a bonding process, and as a result of the bonding process, When forming the combined image of, the switching control is performed based on the form of the image of the overlapping region, whether to use the one image or the other image for the overlapping image portion. The image handling apparatus is provided with a switching control means for performing the operation.

[0030]

According to the present invention, the combined image forming means superimposes the one image and the other image on each other and partially overlaps the one image, and performs a combining process. And the switching control means uses the one image as the image portion to be superimposed when forming the joined image as a result of the pasting processing,
Alternatively, whether to use the other image is controlled based on the form of the image of the overlapping area.

[0031]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an OA camera as an example of the image handling apparatus of this example, and FIG. 2 is a block diagram showing the configuration of a cumulative addition unit 115 of this example. The configuration of this example will be described with reference to FIG.

The same parts as those in FIG. 19 are designated by the same reference numerals and the description thereof will be omitted. The main difference from the configuration of FIG. 19 is that in addition to the conventional method in which the contrast data obtained by the cumulative addition unit 115 is used for AF control,
The point is that it is determined whether or not there is a characteristic portion in the overlapping area of the partial images to be joined, and the result can be output from the joining information output terminal unit 1 of the CPU 2.

Here, the determination means of the present invention is included in the CPU 2, and the predetermined characteristic evaluation standard of the present invention is a standard determined with respect to the above contrast data, and the joined image formation of the present invention is performed. The means is the coordinate conversion unit 110.
A boundary buffer 111, a movement belt detector 112, and a shift amount calculator 113 are included.

Further, in FIG. 2, a band pass filter (BPF) 115a is for extracting only a predetermined high frequency component from the input image data, and the absolute value calculator 115b is a minus waveform of the input signal. Is a circuit for converting into a plus waveform to obtain an absolute value, and the cumulative calculator 115c shows each input waveform obtained by the absolute value calculator 115b in a predetermined cumulative addition area 31 (shown in FIG. 3A). , The first CCD 105 and the second CCD 106 are each 6
It is divided into 4 areas, and the divided 94 pixels x 72
One area of pixels is hereinafter referred to as a cumulative addition area 31), and the contrast data memory 115d stores the contrast data obtained by the cumulative addition for one CCD. Since 64 pieces are created, it is a storage means for storing a total of 128 pieces of contrast data as two CCDs. Here, FIG. 3A is an explanatory diagram for explaining the configuration of the partial image, and FIG. 3B is an explanatory diagram for explaining the cumulative addition area.

The operation of this embodiment will be described mainly with reference to FIG. 2 in the above configuration.

In the figure, the input image data 21 is B
Only the high-frequency component waveform 22 is taken out by the PF 115a, and this is passed through the absolute value calculator 115b to become the absolute value converted waveform 23, which is the cumulative calculator 115.
The shaded portion 24 of the waveform whose absolute value has been converted through c is cumulatively added in each cumulative addition area 31, and a total of 128 contrast data are stored in the contrast data memory 115d as one partial image. .

Then, the CPU 2 reads out the respective contrast data from the contrast data memory 115d as necessary, and sends a control signal for controlling AF based on the data to the AE / AF drive section 117, and the lens The focus lens of the unit 103 is controlled.

Further, the CPU 2 determines whether or not a characteristic portion exists in the overlapping area of the partial images to be joined, and outputs the result from the joining information output terminal unit 1.

That is, the CPU 2, for example, as the partial image to be joined, in the overlapping region of the partial image A and the partial image B, belonging to the partial image A (see FIG. 3).
It is determined in advance by paying attention to 16 pieces of contrast data (corresponding to portions of 16 cumulative addition areas, which are numbered 1 to 16 at the right end of the partial image A shown in (b)). And a predetermined contrast value (threshold value) as a predetermined characteristic evaluation criterion. If all 16 values are smaller than the predetermined contrast value, the overlapping area A
It is determined that there is no contrast in b, that is, there is no characteristic portion.

As described above, the CPU 2 pastes because no contrast data exceeding a predetermined threshold value exists in any of the cumulative addition areas in, for example, the overlapping area Ab (the same applies to other overlapping areas). When it is determined that the matching is impossible, the joining information output terminal unit 1 outputs information to that effect.

By the output signal from the joining information output terminal portion 1, information indicating that the bonding is impossible is output to the operator as an alarm. Further, the output signal from the joining information output terminal unit 1 is also used as a control signal for a shutter operation for automatically guarding the shutter of the OA camera of this embodiment so as not to operate. Here, there are various methods of the alarm, for example, by causing an LED to emit light in the finder, or making a sound for calling attention.

On the other hand, for example, when the CPU 2 determines that the contrast data exceeding the threshold value exists in the cumulative addition area in the overlapping area Ab and the bonding is possible, the coordinate conversion unit 110 and the deviation amount calculation unit. 113
The same is the same as that of FIG. 19 for performing the partial image joining and joining process.

In this manner, the one image and the other image are provided on the one image side and / or the other image side in the region where the one image and the other image partially overlap with each other. Whether or not there is a characteristic portion required to superimpose, the determination means for determining based on a predetermined predetermined characteristic evaluation criteria, by utilizing the determination result by the determination means, It is possible to realize an image handling apparatus that includes one image and the other image, and a joining image forming unit for performing a joining process.

Therefore, according to this example, it is possible to determine whether or not there is a characteristic portion of the image to be used for superposition and joining in the overlapping area of the partial images.

More specifically, if there is no characteristic portion in the overlapping area, the fact is judged without adding a special circuit or the like, and the bonding is impossible. Information such as an alarm for transmitting the message to the operator is output.

In the present embodiment, the judging means focuses on the contrast data as the AF control data and does not use a special dedicated circuit.
Although the case of using PU2 has been described, the present invention is not limited to this, and may be realized by configuring a dedicated circuit for the above determination, and the predetermined feature evaluation criterion is not limited to that relating to contrast data and is realized. In short, in short, as long as it is possible to determine whether or not there is a feature part of the image to be used for pasting and joining in the overlapping area of the partial images, the contents of the realization circuit and the predetermined feature evaluation criteria The content of does not matter.

Further, in the above-mentioned example, the case where the presence of the characteristic portion is determined is explained by targeting the overlapping area belonging to the first partial image of the two overlapping partial images. However, the present invention is not limited to this, and for example, the overlap area belonging to the partial image captured later may be targeted, or both of them may be targeted.

Further, in the above example, the joining information output terminal section 1
From the output of the information from the
The case where it is used for locking the operation of the shutter and for issuing an alarm for the purpose of calling attention to the operator has been described, but the invention is not limited to this, and the output from the joining information output terminal unit 1 is used. Regarding the way of doing, it does not matter the condition, scene or purpose.

Further, the present invention may be realized by hardware using each component, or not limited to this, and may be realized by software.

The contents to be described below are the same as those described above.
An example in which the OA camera side positively creates contrast with respect to an image pickup target when there is no contrast data exceeding the predetermined threshold value when the partial images are pasted and joined using the A camera. It is about.

In the above embodiment, when the contrast data exceeding the predetermined threshold does not exist in the overlapping area, the bonding cannot be performed, so the shutter operation of the OA camera is automatically locked, and the operator mistakenly operates the shutter operation. It is to prevent shooting. Therefore, there is a great advantage that it is possible to prevent the misaligned images from being stuck together, but it is the same as the conventional case in that the misaligned images cannot be correctly pasted together.

Therefore, in this example, considering the above points,
It is possible to determine whether or not there is a characteristic part of the image to be used for overlapping and joining in the overlapping area of the partial images, and even if the characteristic part does not exist, the partial images It is an object of the present invention to provide an image handling device that can correct misalignment and correct the bonding.

FIG. 4 is a block diagram showing the structure of an OA camera as an example of the image handling apparatus, and the structure of this example will be described with reference to FIG.

The same parts as those in FIG. 1 are designated by the same reference numerals and the description thereof will be omitted. Further, the main difference from the configuration of FIG. 19 is that the CPU 2 or the like can perform the bonding process of the bonding correctly even when the CPU 2 or the like determines that the characteristic part does not exist in the overlapping area. On the other hand, for the purpose of positively creating a pseudo contrast as a characteristic portion, the LED 11 as the auxiliary light element is used.
(Which is a part of the marker projection means) and its LED 11
Auxiliary light drive unit 1 for controlling the timing of light emission of the
2 (which is a part of the marker projection means) and the like.

The LED 11 is the first imager
The CCD 105 (or the second CCD 106) projects light having a distinguishable wavelength. Also, LED11
Is provided inside the OA camera so that the light projected by itself hits the portion corresponding to the overlapping area of the partial images with respect to the imaging target. The auxiliary light drive unit 12 is a CP
The light emitting operation of the LED 11 as the auxiliary light element is controlled on the basis of an instruction from U2 indicating that the characteristic portion does not exist in the overlapping area.

The operation of this embodiment will be described with reference to FIGS. 5 and 6 in the above-mentioned configuration.

FIG. 5 is a time chart for explaining the operation of this example.

First, when the power of the OA camera (ON / OFF state of the power of the OA camera is indicated by the POWER signal 51 in FIG. 5) is input, the mirror scan is started regardless of whether or not the image is taken. Here, in FIG. 5, the operation status of the mirror scan is indicated by A, B, in order to show the correspondence with each partial image (partial images A, B, C, D, ...) Taken in at a predetermined timing. As shown in the mirror scan time chart 52 labeled C and D, A, B,
The scan is repeated in the order of C, D, A, B, C, D, .... In the mirror scan time chart 52, the “bow line” connecting the alphabetic characters A to D indicates that the total reflection mirror 101 is moving, and each alphabetic character is surrounded by a “square”. For example, in the area A, the picture in the area of the partial image A is displayed by the imager (first CCD 105 and second CCD 10).
It means that it is reflected in 6). Light is accumulated in the imager at the timing A in the mirror scan time chart 52.

By repeating this in sequence, scanning from A to B and scanning from B to C, the total reflection mirror 101
Is sequentially moved, a predetermined partial image is displayed on the imager. By performing the mirror scan in this way, the output 53 of the CCD (the first CCD 105 and the second CCD 106) is partially output such that the output of A is delayed by a predetermined timing after the partial image A is exposed. The output is delayed by a predetermined timing from the exposure timing of the image.
The order of B, C, D basically does not change.

Here, if it is determined by the CPU 2 that the contrast data exceeding the predetermined threshold value, which is a characteristic portion of the images necessary for superimposing, does not exist in the overlapping area of the partial images, As the alarm signal 54 from the joining information output terminal section 1 (see FIG. 4),
A High signal is output. Here, there are various methods of the alarm, for example, by causing an LED to emit light in the finder, or making a sound for calling attention.

As the alarm signal 54, a High signal is output, and the search for the characteristic portion is repeated several times as necessary. If it is still not found, it is determined that there is no characteristic portion, that is, there is no contrast. . In this case, when the release button is pressed (in FIG. 5,
This corresponds to the case where the release signal 55 indicates the high level). Since the CPU 2 has determined at this point that there is no contrast in the bonding area, the LED 11 as auxiliary light is caused to emit light, and a pseudo light is emitted. Creates a high contrast and uses this to stitch partial images together.

The timing for causing the LED 11 to emit light is as shown in the mirror scan time chart 52.
LED1 is set at the timing 51a when the partial image A is exposed and at the timing 51b when the partial image B is exposed.
1 is emitted twice (the timing at which the LED 11 emits light corresponds to the case where the auxiliary light signal 56 in FIG. 5 indicates the high level).

The place where the auxiliary light (LED11) is projected is
Since the partial images must overlap each other, as shown in FIG. 6, for example, the partial image A has a right end 61 of the image plane, and the partial image B has a left end 62 of the image plane.

In this way, the auxiliary light is projected at each position of the partial image.

Originally, the overlapping area between the partial images is
There was no feature for pasting and there was no contrast, but it was a “sticky image”, but due to the projection of this auxiliary light, there was a pseudo feature and the misregistration was detected. Therefore, the partial images can be correctly pasted together without causing positional displacement. Therefore, for example, even when the camera is displaced due to camera shake during operation and the partial image A and the partial image B are vertically displaced from each other by Y in the vertical direction, the partial image A and the partial image B are still captured based on the information of the auxiliary light. If you stick them together, they can be bonded correctly.

In this way, the one image and the other image are provided on the one image side and / or the other image side in the region where the one image and the other image partially overlap with each other. Whether or not there is a characteristic portion required to superimpose, the determination means for determining based on a predetermined predetermined characteristic evaluation criteria, and the imaging target corresponding to the overlapping region, A marker projecting unit for projecting a marker used for the superimposing and a result of the determination by the determining unit are used to superimpose the one image and the other image to perform a laminating process. It is possible to realize an image handling device provided with a joining image forming means.

Therefore, according to this example, it is possible to determine whether or not there is a characteristic part of the image to be used for superposition and joining in the overlapping area of the partial images, and the characteristic part exists. Even if it is not performed, the positional deviation between the partial images can be corrected and the images can be correctly combined.

Although the marker in the present invention has been described as LED light in the above embodiment, the present invention is not limited to this.
In short, it can be anything as long as the imager can be identified.

When light is used as a marker, it may be light having any wavelength as long as it can be identified by the imager.
It doesn't matter.

In the above example, when the characteristic portion does not exist in the overlapping area of the partial images, the light of the LED is projected from the device side to the object to be imaged to positively make the contrast, and the partial image is obtained. Detects mutual positional deviation,
It is possible to perform a correct bonding process without any deviation.

However, with such a configuration, there is a possibility that the contrast which originally did not exist in the image pickup target is output to the output image.

Therefore, as an output image, which is necessary for the bonding process for bonding, unnecessary contrast,
That is, an example in which an obstructive image that should not originally exist can be erased in advance for the purpose of preventing the obstructive image from appearing in the output image will be described below.

When the overlapping region of the partial images is determined as a region having no characteristic portion, that is, a region having no contrast, the region is considered to be a "bluish image" with little change, and Replace the disturbing image with the surrounding image and erase the shining part,
This is to provide an interpolating unit 71 (see FIG. 7) for returning an image that is extremely close to the original image and then outputting the image. Here, FIG. 7 is a block diagram for showing the configuration of the interpolation unit 71, in which the part marked with corresponds to the part marked as the output side of the output unit 114 in FIG. It is shown that they are mutually connected in a circuit manner.

As another example, the case of an OA camera that handles a binarized signal will be described below with reference to FIG. Here, FIG. 8 is a block diagram for showing the configuration of the binarization unit 75. In FIG. 8, the portions marked with and are on the output side of the output unit 114 in FIG.
It corresponds to the portion marked with as the output side of the joining information output terminal portion 1 and indicates that they are in a circuit-like connected state with each other.

In this case, the output signal is binarized,
When the light of the LED 11 is placed on the binarized output image signal, only the part exposed to the light becomes “white” even though the surroundings are “black”, resulting in an image different from the original one. Become.

In this case, as explained in the above example,
There is no contrast in the overlapping area. For example, if only the overlapping area at the right end of the partial image A and the left end of the partial image B is forced to be “black”, it becomes 2
Even in the case of digitization, it can be handled in the same manner as the above example.

Next, in order to record the images obtained by pasting the partial images and joining them together at a high density, for example, JPEG
Regarding a device that performs band compression based on the format of (Joint photography expert group) and outputs, a case where there is no characteristic part in the overlapping region of partial images will be described.

FIG. 9 is a block diagram showing the configuration of the image handling apparatus of this example. The configuration of this example will be described with reference to FIG. The same components as those in FIG. 4 are designated by the same reference numerals,
The description is omitted. Further, the main difference from the configuration shown in FIG. 4 is that in the case of this example, when obtaining contrast data, J
DCT (Discrete cosin) based on PEG format
e transfer) processing (herein, DCT processing mainly means processing for converting the arrangement of images into the arrangement of frequencies) and using the DCT coefficient obtained in the process of band-compressing the image data. is there.

In FIG. 9, a DCT processing unit 81 is for performing DCT processing on the blocked image data, and a zigzag scanning unit 82 is for transforming the sequence in the frequency domain into low frequency components. From the high frequency component to serially rearrange, and the cumulative addition unit 83 cumulatively adds only a necessary portion of the signal after the zigzag scanning, and thereby, the data for AE and the data for AF are added. Contrast data is obtained. The Huffman encoder 84 is for encoding the data after the zigzag scan.

The operation of this example will be described with reference to FIG. 10 in the above-mentioned configuration.

First, based on the JPEG format,
DCT processing is performed on the blocked data corresponding to the block 91 of 64 pixels of 8 pixels × 8 pixels, and the numbers 1 to 64 are individually assigned to the blocked data for each frequency. Be disassembled. Here, FIG. 10A is an explanatory diagram showing a block 91 of 8 pixels × 8 pixels of 64 pixels based on the JPEG format, and FIG. 10B shows a DCT process for the blocked data. It is an explanatory view for explaining.

The data represented by the number 1 has a lower frequency, and the data having a larger number has a higher frequency component. When this is zigzag scanned by the zigzag scanning unit 82, numbers 1 to 6
The data of No. 4 are serially rearranged in order (see FIG. 10B). Here, the number 1 is 8 × 8
The DC component in the block 91 of pixels, numbered 2 and 3
The frequency component becomes higher as the number increases in order.

Therefore, for example, in order to obtain the necessary data for performing the AE control in which the low frequency component of the signal is used, cumulative addition is performed using only relatively low frequency components such as numbers 1 to 4. Data for AE control is obtained by cumulative addition in the unit 83.

On the other hand, in the case of AF control, a comparatively large contrast is required, so the cumulative addition unit 83 performs cumulative addition using only the region of relatively high frequency to obtain AF control data.

In this way, the zigzag scanning unit 82
As a result of the zigzag scanning by, the cumulative addition unit 83 cumulatively adds only the portion necessary for the predetermined control, the information is sent to the CPU 2, and the AE and AF are driven.

As is clear from the above, the CPU 2
By this, when it is determined that the contrast does not exist in the overlapping area of the partial images, alarm information is output, or auxiliary light is projected toward the imaging target,
By actively forming pseudo contrast,
It is possible to create a correct spliced image without displacement.

In this way, for one image and another image partially overlapping the one image, the image signal representing the one image and / or the image signal representing the other image is used. Orthogonal transformation means for obtaining orthogonal transformation data and necessary for superimposing the one image and the other image on the one image side and / or the other image side of the overlapping area. Using the determination result by the determination means for determining whether or not there is a characteristic portion based on the criterion determined for the orthogonal transformation data, the determination result by the determination means, the one image and the other It is possible to realize an image handling apparatus including a combined image forming unit for superposing images and performing a combining process.

Therefore, according to this example, for example, the above DCT
It is possible to determine whether or not there is a characteristic part of an image to be used for superposition joining in the overlapping area of the partial images based on a criterion defined with respect to the orthogonal transformation data such as a coefficient. Even if the characteristic portion does not exist, the positional deviation between the partial images can be corrected and the images can be correctly combined.

Further, the present invention is applicable not only to a camera but also to other image handling devices such as a copying machine, a printer, a facsimile and a film scanner.

In the above embodiment, the case where the DCT coefficient or the like is used as the reference defined for the orthogonal transform data of this example has been described. However, the present invention is not limited to this, and the DST (Discrete sine transfer) process is used. Of course, it may be realized by using a DST coefficient or a Fourier transform coefficient.

By the way, FIG. 11 is a block diagram showing the construction of an OA camera as an embodiment of the image handling apparatus according to the present invention.

FIG. 13 is a diagram showing a circuit configuration of the switching control means 5 which will be described later, and the configuration of the OA camera of this embodiment will be described with reference to FIG.

The same components as those shown in FIG. 4 already described are designated by the same reference numerals and the description thereof will be omitted. The main difference from the configuration shown in FIG. 4 is that in the configuration shown in FIG.
The point is that the switching control means 5 is provided to control the switching of the partial images to be used based on the form of the characteristic portion in the overlapping area in the joining process by bonding the partial images. Further, in the configuration shown in FIG. 11, the coordinate conversion unit 11
The image data coordinate-converted at 0 is binarized by the binarization unit 119 and is output through the output unit 114.
It is also different in that it is held in the boundary buffer 111 and a motion vector is detected based on the held binarized data. Further, the joined image forming means of the present invention includes the coordinate conversion unit 110 and the like.

In FIG. 13, the portions marked with and are
It corresponds to the portion marked with and as an input portion to the switching control means 5 in FIG. 11, and shows that they are in a circuit connection state. Further, with respect to the figure, the SW 121 for switching the signal has a portion 125 and a signal inputted from the portion, that is, the data 125 and the data 12
6 (see FIG. 15) is a switch unit for switching and outputting, and the SW control circuit 122 includes an AND circuit 123.
And a horizontal synchronizing signal (HSYNC) 128 (see FIG. 15) input signal for switching control of the connection state of the SW 121.
9 (see FIG. 15) is output. The AND circuit 123 has a select signal (SEL) which is output by its own input side in an attempt to switch signals of partial images at the same timing.
The output side of the ECT) 127 (see FIG. 15) and the output unit 11
4 is connected to the output side (corresponding to the part).

In the above-mentioned structure, mainly in FIG.
The operation of this embodiment will be described with reference to FIG. 12A is a time chart for explaining the outline of the output state in the joining process of the present embodiment, and FIG. 12B is an explanatory diagram for explaining the image area. In addition, in FIG.
Since the output 126 (corresponding to the output of the data in the boundary buffer 111) corresponds to the above-mentioned data 125 and the data 126 (see FIG. 15), the same reference numerals are given and the images pasted together using these The data is the Y output 124. Here, the main difference from FIG. 22 is the output 126.
Is the point to utilize.

First, the output 126, that is, the output in the boundary buffer 111 is the right end of the partial image A in order to detect the motion vector with the partial image B when the partial image A is mirror-scanned once, for example. Ar as image information
Must be stored in the boundary buffer 111 as binarized data in case the partial image B is scanned. Next, when the partial image B is scanned and comes out, the left edge of the partial image B and the right edge Ar of the partial image A stored in the boundary buffer 111 are used respectively to cause misalignment between them, Since it is configured to detect the motion vector, when the partial image B is output from the output unit 114, since Ar is contained in the boundary buffer 111, the output 125 shown in FIG. As indicated by the output 126, Ar is output from the boundary buffer 111 at the timing when the partial image B is output from the output unit 114.

Thereafter, the right edge Br of the partial image B is sequentially output at the timing of outputting the partial image C, and Cr is output at the timing of outputting the partial image D.

Here, an important point is that the output of the output unit 114 and the output of the boundary buffer 111 are skillfully switched by the SW 121 so that the images of the joint portion do not shift when the partial images are pasted as in the conventional case. It is configured.

That is, as shown in FIG. 12B, in short, in the overlapping region of the partial image A and the partial image B,
Instead of switching partial images at uniform timing,
That's what it means.

Further, regarding the timing and the like, FIG.
3, FIG. 15 and FIG. Here, FIG.
FIG. 13 is a time chart for explaining the output state in the joining process of the present embodiment, and FIG. 13 is an explanatory diagram for explaining the circuit configuration of the switching control means 5. Further, FIG. 16 is an explanatory diagram for explaining the timing of switching between the partial image A and the partial image B.

In FIG. 15, the area of one partial image is captured by one mirror scan. For example, the timing at which the area of partial image B is scanned and output will be described.

The image signal is scanned in the horizontal direction, and for example, the data of the partial image B is changed to the horizontal synchronizing signal (HSYN).
C) The data 125 is output at the time indicated by the falling timing 154 of 128. Where data 1
As the signal waveform of 25, it is assumed that the character indicates High, and the white background indicates Low. The data indicated by the data 126 is the data of the boundary of the partial image A held in the boundary buffer 111, and is partially the data 12
It becomes the same as the signal waveform of 5.

Next, referring to FIG. 16, select signal 12
Operations such as 7 will be described.

As shown in the figure, the partial image 151 (see FIG.
2 (b) corresponding to partial image A) and partial image 152 (FIG. 1).
2 (b) (corresponding to partial image B) overlaps in a predetermined area. Here, assume that the horizontal synchronization signal (HSYNC) 12
Even if the partial image 151 is uniformly switched to the partial image 152 at the time point indicated by the trailing edge timing 154 of FIG.

Therefore, in the overlapping area 153 of the partial images, basically, there is no change whether the partial image 151 or the partial image 152 is output, so that the horizontal synchronizing signal (HSYNC) 128 falls. When the timing 154 is used to generate a switching signal at a time point indicated by the rising timing 155 of the select signal (SELECT) 127 at a time point sufficiently earlier than the time point when the output of the partial image 151 ends, basically, , The correct bonding is done.

However, the horizontal synchronization signal (HSYNC) 12
L of select signal (SELECT) 127 synchronized with 8
If the pulse width of ow is fixed in time with this select signal according to each H, the boundary line is always constant as a result, and if only that boundary line is used as a switching signal, it is described above. For this reason, since the joining portion of the partial images becomes linear due to the switching, there is a possibility that a character shift or a step may occur.

In this embodiment, in order to avoid this, control is further performed as follows. Returning to FIG. 15 again, description will be made.

At the first rising timing of the horizontal synchronizing signal (HSYNC) 128, the select signal (SELEC
(T) Timing 129a at which 127 switches from Low to High and switches from partial image A to partial image B
Then, since the data 125 is Low, that is, a portion of “white” where there is no character, no problem can occur even if it is switched unconditionally at this timing.

Next, select signal (SELECT) 12
Timing 129 when 7 switches from Low to High
In b, the data 125 indicates that High, that is, that a character exists, and if it is switched at this timing, there is a possibility that a shift may occur in the joined images due to the above factors. Have

Therefore, it is waited until the data 125 changes from High to Low, that is, until the timing 129c, and then the next partial image B is switched.

By implementing the above-described operation of the timing of switching the partial images by the switching control means 5 shown in FIG. 13, there is no shift or step. According to the circuit of the switching control means 5,
As shown in FIG. 17, the switching location (in FIG. 17, the switching location is indicated by the broken line 201. Also, all the images of “1” are reproduced based on the image data of the partial image A) means the right side of the character. Will change along. Here, FIG. 17 is an explanatory diagram showing the switching location of the image data when the circuit of the switching control means 5 having the configuration shown in FIG. 13 is used.

In the above embodiment, the case of a simple shape such as "1" is used. However, in the case of a complicated character such as "2", the above method does not make a good decision. It may not be possible, and a corresponding example will be described next.

That is, in such a case, for example, so-called projection processing for each character is performed. Here, the projection process is for a predetermined character,
This is a process of performing projection in each of the X direction and the Y direction and detecting the width thereof.

For example, when the character is "2", as shown in FIG. 18, a rectangular area circumscribing "2" (in FIG. 18,
This circumscribing rectangular area 202 is represented by a chain line) and is recognized and obtained as data. The circumscribing rectangle data 203 thus obtained is used as the AND circuit 123 shown in FIG.
Is used as the input data (when the circumscribed rectangular data 203 is present, the select signal is not switched and is made to stand by), and other than that, it is the same as the configuration of FIG. 13 described in the above embodiment and is the same. By processing by the method, it is possible to more accurately realize the prevention of image shift. Here, FIG. 14 is an explanatory diagram for explaining another circuit configuration of the switching control means 5 in FIG. Further, FIG. 18 is an explanatory diagram showing the switching location of the image data when the circuit of the switching control means 5 having the configuration shown in FIG. 14 is used.

That is, with the above configuration, by determining the switching timing, as shown in FIG. 18, the switching is always performed along the circumscribing rectangular area 202 of the character, so that the character never crosses over and the position is temporarily changed. Even if a deviation detection error occurs, it does not affect the joining state of images such as characters.
The position shift can be prevented more accurately than the configuration of 3 and the like.

In the above embodiment, the case where the output data is binary data has been described. However, the output data is not limited to this, and may be multi-valued data, and the same effect can be obtained.

Further, in the above embodiment, when the partial images are pasted together, the CPU 2 determines whether or not the characteristic portion exists in the overlapping area of the partial images, and if the characteristic portion does not exist. Has described the case of the configuration including the LED 11 and the auxiliary light driving unit 12 for actively producing the characteristic portion, but the present invention is not limited to this, and the CPU 2 does not include the LED 11 and the auxiliary light driving unit 12, and the CPU 2 makes the above determination. Needless to say, the configuration may not be performed.

Further, the present embodiment may be realized by hardware using each component, or not limited to this, and may be realized by software.

[0120]

As is apparent from the above description,
The present invention has an advantage that when the partial images are pasted together, the displacement of the characteristic portion existing in the overlapping region of the partial images is further less likely to occur than in the conventional case.

[Brief description of drawings]

FIG. 1 is an OA as an example of an image handling apparatus of this example.
Block diagram to show camera configuration

FIG. 2 is a block diagram showing the configuration of a cumulative addition unit of the embodiment.

FIG. 3 (a); an explanatory diagram for explaining the configuration of a partial image of the same embodiment. FIG. 3 (b); an explanatory diagram for explaining a cumulative addition area of the same embodiment.

FIG. 4 is an OA as an example of the image handling apparatus of this example.
Block diagram to show camera configuration

FIG. 5 is a time chart for explaining the operation of this example.

FIG. 6 is an explanatory diagram for explaining a place where auxiliary light is projected in FIG.

FIG. 7 is a block diagram showing the configuration of an example in which an interpolation unit is added in FIG.

FIG. 8 is a block diagram showing a configuration of a binarizing unit of another example.

FIG. 9 is a block diagram showing the configuration of an image handling apparatus of this example.

10 (a); an explanatory diagram showing a block of 8 pixels × 8 pixels of 64 pixels based on the JPEG format. FIG. 10 (b); DCT for blocked data.
Explanatory diagram for explaining the process

FIG. 11 is a block diagram showing a configuration of an OA camera as an example of an image handling apparatus according to the present invention.

12A is a time chart for explaining the outline of the output state in the joining process of the present embodiment. FIG. 12B is an explanatory diagram for explaining the image area.

13 is an explanatory diagram for explaining a circuit configuration of a switching control unit in FIG.

FIG. 14 is an explanatory diagram for explaining another circuit configuration of the switching control means in FIG.

15 is a time chart for explaining an output state in the joining process with the configuration of FIG.

16 is a partial image A and a partial image B having the configuration of FIG.
Explanatory diagram for explaining the timing of switching

FIG. 17 is an explanatory diagram showing a switching location of image data when the circuit of the switching control means having the configuration shown in FIG. 13 is used.

FIG. 18 is an explanatory diagram showing a switching location of image data when the circuit of the switching control means having the configuration shown in FIG. 14 is used.

FIG. 19 is a block diagram for explaining a configuration of an already proposed OA camera.

20 is an explanatory diagram for explaining an overlapping area and the like of each partial image in the conventional example shown in FIG.

FIG. 21 is a block diagram for explaining the configuration of another proposed OA camera.

22 (a); another conventional OA shown in FIG. 21.
FIG. 22B is a time chart for explaining the outline of the output state in the joining process of the camera. FIG. 22B shows the region of the image when explaining the outline of the output state in the joining process of the other conventional OA camera shown in FIG. Figure

FIG. 23 (a); another conventional OA shown in FIG.
FIG. 23B is a diagram for explaining a case where no deviation occurs in the process of combining the partial image A and the partial image B in the camera. FIG. 23B; the partial image A and the partial image in another conventional OA camera shown in FIG. In the process of bonding with B,
Diagram explaining the case where misalignment has occurred

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Bonding information output terminal section 2 CPU 5 Switching control means 101 Total reflection mirror 102 Incident light 102a Passed light 102b Reflected light 103 Lens section 104 Half mirror 105 First CCD 106 Second CCD 110 Coordinate conversion section 200 Image to be captured

Claims (1)

[Claims] 1. A combined image forming means for performing a laminating process on one image and another image that partially overlaps the one image by superposing the one image and the other image. And, when forming a joined image as a result of the pasting process, whether to use the one image or the other image for the image portion to be overlapped, An image handling device, comprising: a switching control means for performing switching control based on a form of an image.