JPH06259557A - Picture input device - Google Patents

Abstract

PURPOSE:To provide a light-weight and inexpensive picture input device capable of obtaining high resolution and having a practical and wide visual angle range. CONSTITUTION:While scanning an original, a partial picture is picked up by an image pickup part 1, partial picture sequence picked up by the image pickup part 1 is fetched in a pickup image synthesizing part 3 and the parallelism and rotational angle of a partial picture fetched this time with respect to a partial picture fetched just before this are detected by an image distortion detecting part 31. An image distortion correcting part 32 corrects relation between the partial picture fetched this time and the partial picture fetched just before this in accordance with the parallelism and rotational angle detected by an image distortion detecting part 31, a partial picture inserting position calculating part 33 calculates the inserting position of the partial picture fetched this time based upon a corrected output and a synthesized picture storing part 4 generates a synthesized picture of respective partial pictures based upon an output from the synthesizing part 3.

Description

[Detailed description of the invention]

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image input device for inputting image information.

[0002]

2. Description of the Related Art Conventionally, as an image input device, there is known a stationary scanner for inputting image data by converting light reflected from a drawing into an electric signal by a photoelectric conversion sensor.

[0003] Such stationary scanners are widely used because they can read handwritten drawings as they are at a high resolution of 300 dpi or higher. However, on the other hand, since it requires a mechanical system, it has the problem of being heavy and expensive.

[0004] This is because the high-resolution lens is made of large-diameter glass, which inevitably becomes heavy and expensive. In addition, since the mechanical system is required to have high precision, it is necessary to increase the rigidity, which also causes an increase in weight and cost. Also, the use of a glass platen increases weight and cost. For this reason, it is uneconomical in terms of power consumption in the mechanical system, and is not portable due to its weight and size.

[0005] Therefore, a handy scanner, which is smaller, lighter, and easier to handle than such a stationary scanner, is sometimes used. In addition to being narrow, it is difficult to handle because it is necessary to scan by hand at a constant speed without sideways movement. Moreover, there are problems such as being bound by a cable led out from the handy scanner.

[0006] On the other hand, recently, attention has been paid to taking in image information by photographing a drawing with an electronic still camera. Unlike the scanners described above, such an electronic still camera is excellent in portability, and since it does not come into contact with the original, it is possible to pick up images of billboards and whiteboards in remote locations.

[0007] However, such an electronic still camera
The number of pixels is 640x because a normal TV camera is used.
There is only about 480, and there is a fatal problem of low resolution. This means that at a resolution of 300 dpi, for example, A
Approximately 3600 x 2500 when reading a 4th edition manuscript
Pixels are required, and in an electronic still camera, about 1/25
resolution is obtained.

[0008] It is also conceivable to use a high-definition camera for the electronic still camera, but even if this high-definition camera is used, it is still insufficient because it has only about 2000 x 1000 pixels at most. In addition, if the resolution of the image pickup device is increased in this way, it is meaningless unless the resolution of the lens is also increased accordingly. It will fall into a vicious circle in which it becomes expensive due to the need for

Further, there is an attempt to improve the resolution by using a piezoelectric element such as a piezoelectric element as an imaging device and vibrating the piezoelectric element, but even in this case the problem of increasing the size of the optical system remains. I cannot escape.

Further, it is conceivable to increase the resolution by improving the degree of integration of the image sensor itself due to advances in semiconductor technology.

[0011]

As described above, there is no conventional image input device that satisfies all of high resolution, portability and price. In addition, the inexpensive and simple device has a problem that the visual field range is too narrow.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide an image input device which has a high resolution, a wide practical field of view, and is lightweight and inexpensive.

[0013]

SUMMARY OF THE INVENTION The present invention provides an imaging section for picking up partial images from an object, a series of partial images picked up by the imaging section, and translation of the currently picked up partial image with respect to the previously picked up partial image. and means for detecting a rotation angle, means for correcting the relationship between the previously captured partial image and the currently captured partial image according to the translation and rotation angles detected by the means, and the currently captured portion based on the corrected output of the means. It is composed of captured image synthesizing means having means for calculating an image embedding position, and means for generating a synthesized image of each partial image from the output from the captured image synthesizing means.

[0014]

As a result, according to the present invention, partial images captured by the imaging section of the subject can be combined to generate a single composite image, thus minimizing the imaging range of the imaging section. Therefore, it is possible to obtain an image of the entire object wider than the field of view of the imaging unit, to dramatically improve the resolution, and to realize a lightweight and inexpensive image input device.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will now be described with reference to the drawings.

FIG. 1 shows a schematic configuration of the same embodiment. In the figure, reference numeral 1 denotes an imaging unit, which scans a document 2 and captures partial images as an input image sequence. In this case, the imaging unit 1 includes a TV
A camera, a handy scanner, a CCD image sensor, or the like that continuously inputs image information is used.

An input image sequence input by the imaging unit 1 is sent to the captured image synthesizing unit 3 . This picked-up image synthesizing unit 3 synthesizes the input partial image trains at appropriate positions.

In this case, the picked-up image synthesizing section 3 comprises an image distortion detecting section 31, an image distortion correcting section 32, and a partial image fitting position calculating section 33. FIG. Then, an input image sequence (partial image) taken from the imaging unit 1 is given to an image distortion detection unit 31 and an image distortion correction unit 32 . The image distortion detector 31 detects translation and rotation of a series of input image sequences and sends them to the image distortion corrector 32 . In the image distortion corrector 32,
Based on the information obtained by the image distortion detector 31, image processing such as affine transformation is applied to the input image to correct the distortion of the input image. Also, the input partial image corrected by the image distortion correction unit 32 is given to the partial image fitting position calculation unit 33 . Then, the partial image embedding position calculation unit 33 compares the partial image sequence that has been input, combined, and accumulated with the corrected input partial image, and calculates the position to be connected. This calculated value is the position where the corrected input partial image should be inserted, and by writing the corrected input partial image in the corresponding address area of the composite image storage unit 4, which will be described later, the input image can be synthesized. I am trying to do it.

At this time, if the input partial image is completely included in the synthesized image, no writing is performed. On the other hand, partial images for which the inset position cannot be calculated are temporarily stored in the reserve memory unit, and the inset position is searched for again when the synthesis of the whole is complete.

The input partial images synthesized at appropriate positions by the picked-up image synthesizing section 3 are applied to and stored in the synthetic image storage section 4 . In this case, the composite image stored in the composite image storage unit 4 is sent to the composite image display unit 5 at the same time. The composite image display unit 5 is designed to allow real-time viewing of how the input partial image sequences are being composited.

The composite image stored in the composite image storage section 4 is also sent to the composite image completion determination section 6 and the external interface section 7 . The composite image completion determination unit 6 determines that the composition is completed from the state in which the images are stored in the composite image storage unit 4, and notifies the user of this through the notification unit 8. there is Also, the image for which the synthesis has been completed is transferred to the outside through the external interface section 7 . note that,
Reference numeral 9 denotes an operation instruction section for instructing various operations, and 10 denotes a control section for giving control information to each block. Next, the operation of the embodiment configured as above will be described.

FIG. 2 is a diagram for explaining the state in which the apparatus 11 of this embodiment is actually used. Here, an example in which the imaging unit 1 is a TV camera is shown. In this case, as shown in FIG. 11(a), when the embodiment apparatus 11 is held by hand and the original 2 is scanned, the combined image display section 5 displays the image shown in FIG. You can see in real time how the input image sequence is synthesized. "1, 2, 3, . . . " in the figure indicate the process in which the synthesized image grows on the screen. Then, the input partial image from the document 2 fetched from the imaging section 1 in this way is sent to the image distortion detecting section 31 of the captured image synthesizing section 3 .

FIG. 3 shows a schematic configuration of the image distortion detector 31. The new frame memory 311 and the correlation calculator 31 are shown in FIG.
2. It consists of a control judgment unit 313, a template memory 314, and an old frame memory 315.

[0024] In this case, the current image input by the imaging unit 1 is stored in the new frame memory 311 . On the other hand, the image at the previous time point is stored in the old frame memory 312 . In this state, the control determination unit 313 cuts out a small area near an arbitrary point in the image in the old frame memory 315 and sets it in the template memory 314 . Then, in accordance with the start command from the control determination section 313, the correlation calculation section 312 scans the new frame memory 311 to obtain the coordinates of the location in the new frame memory 311 where the contents of the template memory 314 best match. The result is output to the control determination section 313 .

[0025] In this case, the calculation in the correlation calculator 312 is as follows:
As shown in FIG. 4, the vicinity of the center point P _o of the old frame image 41 stored in the old frame memory 315 is cut out and set in the template memory 314, and the position in the new frame memory 311 is obtained. It will determine in which direction the whole thing has moved.

In this case, since the displacement from the old frame to the new frame has three degrees of _freedom of two-dimensional position and rotation, the new frame memory 3
11, these new frame images 42
and the old frame image 41 is required.

Here, as shown in FIG. 5, a point P _i
are expressed in the coordinate system of the old frame memory 315 as ¹ P _i expressed in the coordinate system of the new frame memory 311 is ² P _i , the vector T of the translation component from the old frame to the new frame is T= ¹ P _i − ²
P _i , and the rotation angle θ between the new frame image 42 and the old frame image 41 is

cos θ=( ^1Pj- _{_} ¹ P _i )・( ^2Pj- _{_} ² P _i )/| ^1Pj- _{_} ¹ P _i || ^2Pj- _{_} ² P _i |

As shown in FIG. 4, a plurality of points (P _o ∼
P ₄ ) is made to correspond between the new and old frames, the accuracy is further improved. In this case, the number of equations increases and becomes redundant compared to the degree of freedom to be obtained, but the least squares method should be applied to obtain the solution. A detection output from the image distortion detector 31 is provided to the image distortion corrector 32 . FIG. 6 shows a schematic configuration of the image distortion correcting section 32. The address generator 321, the rotation transforming section 32, and the like are shown in FIG.
2. It consists of a correction frame memory 323 .

In this case, address generator 321
sequentially generate addresses for raster scanning the new frame memory 311 described above. Rotation converter 322
is that when the device is rotated by θ and there is a rotation of angle θ between the new and old frame images, the content of the new frame image is −θ
Since it is rotated, the contents of this new frame image are corrected and rotated by θ to match the old frame image. That is, as shown in FIG. 6, for the address (x, y) generated by the address generator 321,
An address (x'y') obtained by rotating R(.theta.) by an angle .theta. is generated and supplied to the correction frame memory 323. FIG.

By writing the data read from the address (x, y) of the new frame memory 311 to the address (x', y') of the correction frame memory 323, a rotation-corrected frame image can be obtained. It will be.

FIG. 7 shows (x', ) from address (x, y).
y'), which is composed of multipliers 3221 to 3224, adders (subtractors) 3225 and 3226, and a trigonometric function table 322.
It consists of 7. Note that since such a configuration is already known, a description thereof will be omitted here. Such a corrected output from the image distortion correction section 32 is given to the partial image embedding position calculation section 33 .

FIG. 8 shows the partial image fitting position calculator 33.
shows a schematic configuration of the address generator 33
1, rotation transformation R(θ) calculator 332, base register 3
33 and a translation vector register 334 .

In this case, address generator 331
generates a raster address for the new frame memory 311 . The rotation transform R(θ) calculator 332 is the same as that shown in FIG. 7 described above. base register 333
holds the value P _o in the combined screen coordinates of the origin position of the old frame memory 315 described with reference to FIG. The translation vector register 334 stores the translation vector T between the new and old frames, and this vector T is a value obtained together with the rotation angle θ in the image distortion detector 31 described above. FIG. 9 shows the relationship between the new frame image 42 and the old frame image 41 obtained by the partial image embedding position calculator 33 and the composite screen coordinates.

Then, image data is read out from the new frame memory 311 according to the raster address (x, y) generated by the address generator 331, and rotated and transformed R(θ).
By writing the combined screen coordinate address (X, Y) obtained from the sum of the outputs from the arithmetic unit 332, the base register 333, and the translation vector register 334 into the combined image storage unit 4,
Synthesis is performed in a form in which the rotation of the image is corrected.

When the contents of the new frame memory 311 have been written to the composite image storage section 4, the contents of the correction frame memory 323 of the image distortion correction section 32 are transferred to the old frame memory 315. FIG.

In this case, the synthesized image storage unit 4 has a synthesized image bit plane 401 as shown in FIG.
When 01 is written, 1 is set in the corresponding area. The writing here is random access, and the partial image is performed according to the address generated by the embedding position calculator 33 .

The composite image storage unit 4 is composed of a two-port memory with separate reading and writing functions. The reading is raster scanning of sequential access, and the read data is constantly transferred to the composite image display section 5 and displayed.

The input partial images synthesized at appropriate positions by the picked-up image synthesizing section 3 in this manner are sequentially stored in the synthetic image storage section 4 . The synthesized images stored in the synthesized image storage unit 4 are simultaneously displayed on the synthesized image display unit 5, so that the process of synthesizing the input partial image sequences can be seen in real time.

The composite image bit plane 401 in the composite image storage unit 4 is sent to the composite image completion determination unit 6, where one image is stored in the composite image storage unit 4. It is determined whether or not the combination of the images has been completed, and when the completion of the combination is determined, the notification section 8 notifies the user to that effect. Furthermore, the image from the composite image storage unit 4 for which the completion of composition has been determined is also transferred to the outside via the external interface unit 7 .

Accordingly, in this manner, the partial image picked up by the imaging section 1 is taken into the picked-up image synthesizing section 3, and the translation and rotation angles of the partial image picked up this time with respect to the partial image picked up last time are detected. Based on the detected translation and rotation angles, the relation of the currently fetched partial image with respect to the previously fetched partial image is corrected, and based on this correction, the composite image storage unit 4 calculates the fitting position of the currently fetched partial image. Since an image is generated by synthesizing the partial images, the partial images captured by the imaging unit 1 can be combined with the document 2 to generate a single composite image, and the imaging range of the imaging unit 1 can be expanded. to a minimum,
An image of the entire document with a wide field of view can be obtained.
The resolution can be dramatically increased, and a lightweight and inexpensive image input device can be realized. Also, the synthesized image storage unit 4
are sequentially sent to the composite image display unit 5, so that the growth of the composite image can be seen in real time.

For the sake of convenience, the above description assumes that a TV camera or an electronic camera is used as the imaging means.
It is also effective for imaging means other than V cameras and electronic cameras. For example, when using the handy scanner 111 shown in FIG. 11, a belt-like image 1
12 can be overlapped and picked up, and synthesized to obtain an image wider than the width of the handy scanner. In this case, the capacities of the new and old frame memories are made large enough to accommodate one band-shaped image, and the positional relationship and inclination between the band-shaped images are obtained by performing correlation calculations on the overlapping portion. can be realized.

[0043] Further, as shown in Fig. 12, the sideways movement and blur during one scan of the handy scanner 111 can be corrected by, for example, a line sensor perpendicular to the traveling direction of the handy scanner 111, as shown in Fig. 13. By putting each image 113 into the new/old frame memory and performing the above-described correlation operation, the horizontal displacement can be obtained and the correction can be realized.

In the above description, the inclination and rotation of the entire apparatus are obtained from the input partial image from the imaging unit 1. However, FIG.
4, an angle sensor 141 is provided, and the output from this angle sensor 141 is input to the image distortion detection section 31 of the captured image synthesizing section 3 to detect the tilt and rotation of the entire apparatus and correct the image distortion. You may do so. By doing so, the configuration of the image distortion detection unit 31 can be simplified. Needless to say, the angle sensor 141 includes a combination of an angular velocity sensor and a (time) integrator.

Further, as shown in FIG. 15, a distance sensor 151 for detecting the distance h between the imaging section 1 and the document 2 may be provided. In this way, the image distortion correction unit 32 can be given the distance h from the document 2 in addition to the rotation angle θ, so that it is possible to correct the variation in the size of the image due to the variation in the distance h. .

In this case, as shown in FIG. 16, the image distortion correcting section 32 adds an enlargement/reduction processing section 152 and a second correction frame memory 153, and the content of the correction frame memory 323 is processed by the distance sensor 151. The enlargement/reduction processing unit 152 performs enlargement/reduction M(h) processing according to the detected distance h, and stores this in the second correction frame memory 153 . Here, as the distance sensor 151, for example, an ultrasonic distance sensor that obtains the distance from the time from when the ultrasonic wave is emitted until the reflected wave returns can be used. Of course, if the above-described distance sensor 141 and angle sensor 151 are used together, the accuracy of image distortion correction can be improved and the configuration of the image distortion detector can be simplified.

A calendar/clock unit 171 may be provided as shown in FIG. 17(a). Then, if the date and time information obtained from the calendar/clock unit 171 is given to the control unit 10 and entered in the margins of the composite image generated by the composite image storage unit 5, then the composited image can be organized later. It becomes possible to plan for convenience when doing so. Here, as shown in FIG. 8B, the date and time information can be considered to be blank portions 173 at the upper and lower ends of the composite image 172 .

Also, as shown in FIG. 18, the character recognition section 18
1 and a character code storage unit 182 may be provided, and the synthesized image obtained from the synthesized image storage unit 4 may be converted into character code by the character recognition unit 181 and stored in the character code storage unit 182 . In this way, the composite image itself,
Alternatively, the character code or both of them can be taken out to the outside via the external interface section 7. FIG.

As shown in FIG. 19(a), the composite image obtained from the composite image storage unit 4 is sent to the FAX modem 191 via the external interface 7, converted into a FAX code, and sent to the telephone 192 or You can also make calls to phone lines. In this case, if the calendar/clock section 193 is provided, the date and time information obtained from the calendar/clock section 193 can be written in the margin of the transmission image. Also, if a character recognition unit 194 and a character code storage unit 195 are provided, it is possible to perform character recognition of the destination number and the like from the composite image from the composite image storage unit 4, and use the recognition result to send out an image image. It is also possible to save the trouble of specifying the destination number. FIG. 2(b) shows an example of usage when such a configuration is integrally incorporated into the mobile phone 196. As shown in FIG.

Further, as shown in FIG. 20, a hard copy section 201 is provided, and a composite image is sent to the hard copy section 201 in accordance with an instruction from the operation/instruction section 9 to obtain a hard copy of the composite image. can be done. In this case, by providing the calendar/clock unit 202, date and time information can be written in the margin of the hard copy for the convenience of later organization. For example, if a small, lightweight printer such as a thermal printer or a bubble jet printer is used as the hard copy unit 201, and a small thumb-sized camera is used as the imaging means, the functions of a copier and an electronic whiteboard can be carried around. You will get the same situation.

On the other hand, in the above description, only the two-dimensional rotation shown in FIG. 21(a) was considered as the rotation of the imaging section 1, but three-dimensional rotation can also be considered as shown in FIG. 21(b). can. The deformation of the image in this case becomes an affine transformed image shown in FIG.

[0052]

[Number 1] Here, (x, y) and (X, Y) are the coordinates of the image before and after transformation, and a, b, c, and d are non-zero coefficients (affine coefficients).

In order to obtain these affine coefficients a to d, n on the screen (n.gtoreq.
The points P1 to Pn in 4) can be obtained by substituting the coordinates before and after the transformation into the above equation and making them simultaneous. In this case, it is obtained by the following formula. The following equation can be solved by the method of least squares.

[0054]

[Number 2] In addition, the present invention can be appropriately modified and implemented without changing the gist thereof.

[0055]

According to the present invention, it is possible to synthesize partial images of a subject captured by the imaging unit to generate a single composite image, minimize the imaging range of the imaging unit, An image of the entire subject that is wider than the field of view of the imaging unit can be obtained, and the resolution can be dramatically improved.
A lightweight and inexpensive image input device can be realized.

[Brief description of the drawing]

1 is a diagram showing a schematic configuration of an embodiment of the present invention; FIG.

FIG. 2 is a diagram showing a state in which the device of one embodiment is in use;

FIG. 3 is a diagram showing a schematic configuration of an image distortion detection unit of one embodiment;

4A and 4B are diagrams for explaining the operation of an image distortion detection unit according to an embodiment; FIG.

5A and 5B are diagrams for explaining the operation of an image distortion detection unit according to an embodiment; FIG.

FIG. 6 is a diagram showing a schematic configuration of an image distortion correction unit of one embodiment;

FIG. 7 is a diagram showing a schematic configuration of a rotation converter of one embodiment;

FIG. 8 is a diagram showing a schematic configuration of a partial image fitting position detection unit of one embodiment;

FIG. 9 is a diagram for explaining a partial image fitting position detection unit according to an embodiment;

10A and 10B are diagrams for explaining a composite image storage unit according to an embodiment; FIG.

11A and 11B are diagrams for explaining a case where the imaging unit of another embodiment of the present invention is a handy scanner; FIG.

12A and 12B are diagrams showing horizontal shaking and blurring when using a handy scanner of another embodiment; FIG.

FIG. 13 is a diagram showing a line sensor image by a handy scanner of another embodiment;

FIG. 14 is a diagram showing a schematic configuration of another different embodiment of the present invention;

FIG. 15 is a diagram showing a schematic configuration of another different embodiment of the present invention;

FIG. 16 is a diagram showing a schematic configuration of another different embodiment of the present invention;

FIG. 17 is a diagram showing a schematic configuration of another different embodiment of the present invention;

FIG. 18 is a diagram showing a schematic configuration of another different embodiment of the present invention;

FIG. 19 is a diagram showing a schematic configuration of another different embodiment of the present invention;

FIG. 20 is a diagram showing a schematic configuration of another different embodiment of the present invention;

FIG. 21 is a diagram showing a schematic configuration of another different embodiment of the present invention;

[Description of symbols]

DESCRIPTION OF SYMBOLS 1... Imaging part 2... Document 3... Captured image synthesizing part 4... Synthetic image storage part 5... Synthetic image display part 6... Synthetic image completion determination part 7... External interface part 8... Notification part 9 ... operation instruction section, 10 ... control section, 31 ... image distortion detection section, 32 ...
Image distortion corrector 33 ... Partial image embedding position calculator 3
11... new frame memory, 312... correlation calculator, 313
... control determination unit, 314 ... template memory, 315 ...
old frame memory, 321 ... address generator, 3
22... Rotation converter, 323... Correction frame memory, 33
1 --- address generator, 332 --- rotation conversion R(θ)
Arithmetic unit 333 Base register 334 Translation vector register 141 Angle sensor 151 Distance sensor
152 Enlargement/reduction processing unit 153 Second correction frame memory 171, 193, 202 Calendar/clock unit 181, 194 Character recognition unit 182, 195 Character code storage unit 191 FAX modem 192 ... telephone, 201 ... hard copy.

──────────────────────────────────────────────────── ─── Continuation of the front page (51) ^Int.Cl.

Claims (1)

[Claims] 1. An imaging unit for picking up partial images from a subject; means for capturing a sequence of partial images captured by the imaging unit and for detecting the translation and rotation angles of the currently captured partial image with respect to the previously captured partial image; Means for correcting the relationship of the currently fetched partial image with respect to the previously fetched partial image according to the translation and rotation angles detected by this means, and means for calculating the fitting position of the currently fetched partial image based on the corrected output of this means. and means for generating a synthesized image of partial images from an output from the captured image synthesizing means.