WO2005029408A1 - 画像処理装置、及び、撮像装置 - Google Patents
画像処理装置、及び、撮像装置 Download PDFInfo
- Publication number
- WO2005029408A1 WO2005029408A1 PCT/JP2004/013516 JP2004013516W WO2005029408A1 WO 2005029408 A1 WO2005029408 A1 WO 2005029408A1 JP 2004013516 W JP2004013516 W JP 2004013516W WO 2005029408 A1 WO2005029408 A1 WO 2005029408A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- target object
- plane
- unit
- imaging
- Prior art date
Links
- 238000012545 processing Methods 0.000 title claims abstract description 193
- 238000003384 imaging method Methods 0.000 title claims description 220
- 238000001514 detection method Methods 0.000 claims abstract description 41
- 238000003860 storage Methods 0.000 claims description 99
- 238000006243 chemical reaction Methods 0.000 claims description 85
- 238000004364 calculation method Methods 0.000 claims description 48
- 230000008859 change Effects 0.000 claims description 40
- 238000012937 correction Methods 0.000 claims description 18
- 238000005520 cutting process Methods 0.000 claims description 11
- 239000003086 colorant Substances 0.000 claims description 10
- 239000000284 extract Substances 0.000 claims description 4
- 238000000034 method Methods 0.000 description 80
- 238000006073 displacement reaction Methods 0.000 description 71
- 230000008569 process Effects 0.000 description 55
- 238000010586 diagram Methods 0.000 description 28
- 230000003287 optical effect Effects 0.000 description 12
- 238000003825 pressing Methods 0.000 description 12
- 230000001133 acceleration Effects 0.000 description 10
- 230000004044 response Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- GZPBVLUEICLBOA-UHFFFAOYSA-N 4-(dimethylamino)-3,5-dimethylphenol Chemical compound CN(C)C1=C(C)C=C(O)C=C1C GZPBVLUEICLBOA-UHFFFAOYSA-N 0.000 description 6
- 230000005484 gravity Effects 0.000 description 6
- 238000010191 image analysis Methods 0.000 description 6
- 238000005259 measurement Methods 0.000 description 6
- 238000012790 confirmation Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 239000004973 liquid crystal related substance Substances 0.000 description 4
- 230000009467 reduction Effects 0.000 description 4
- 239000011248 coating agent Substances 0.000 description 3
- 238000000576 coating method Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 239000006117 anti-reflective coating Substances 0.000 description 1
- 230000000994 depressogenic effect Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000001465 metallisation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/63—Control of cameras or camera modules by using electronic viewfinders
- H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
Definitions
- the present invention relates to an image processing apparatus that converts an input image into an image in which the viewpoint of a target object in the image is changed, and an imaging apparatus using the image processing apparatus.
- the document When it is desired to simply record the contents written on a document without using a scanner or the like, the document may be imaged and recorded by a portable imaging device such as a digital camera.
- a portable imaging device such as a digital camera.
- the photographer may not be able to stand up, for example, during a meeting, and may image the document on the desk from an oblique direction.
- Document 1 Japanese Patent Application Laid-Open No. 2002-334327 (hereinafter referred to as Document 1), a character such as a manuscript is obliquely captured in an oblique direction, so that characters are squashed or obliquely oblique, making it difficult to read.
- An image processing apparatus is described in which an image is input and a viewpoint of a target object shown in the image is converted into an image having a viewpoint in which characters are easy to read.
- the image processing apparatus described in Document 1 analyzes an image of a document and identifies a line bundle such as a character arrangement and a document edge. The amount of rotation of the original image is determined. Further, in this image processing apparatus, the document is rotated in a direction in which the character strings written on the document are arranged right beside in the output image (such a direction is called an upright direction), and the document is rotated from the front. It is converted into an image as seen. Disclosure of the invention
- a vertical line bundle based on the left end point or the like of the character string in the rotation-converted image is determined.
- the image written on the document is unclear and it is difficult to identify the character string, and the contrast between the background and the document is low and the outer edge of the document is not clear.
- the line bundle in the image has low identification accuracy, and the accuracy of the image-converted direction is also low.
- the present invention has been made in view of these problems, and it is an object of the present invention to convert an input image into an image in which the viewpoint of a target object in the image is changed with higher accuracy. It is an object of the present invention to provide an image processing device and an imaging device that can perform the image processing.
- an image processing apparatus and a target object image in which a target object having a predetermined shape is captured, and a three-dimensional image of the target object.
- Plane conversion means for inputting shape information relating to a shape, converting the target object image into a plane image in which the target object is viewed from a predetermined direction based on the shape information, and wherein the target object in the plane image is captured;
- An in-plane position detecting unit that identifies a target object region as an area and obtains a position of the target object in the planar image.
- the in-plane position of the target object is obtained based on the planar image obtained by using the three-dimensional shape of the target object.
- the in-plane position of the target object can be obtained more accurately.
- the image of the target object is corrected based on the in-plane position of the target object obtained by the in-plane position detection unit, the image can be accurately corrected.
- an image processing apparatus comprising: a target object image in which a target object having a predetermined shape is captured; Shape conversion means for converting the target object image into a plane image in which the target object is viewed from a predetermined direction based on the shape information; and the target object in the plane image.
- An in-plane angle detecting means for identifying a target object area, which is an area in which is captured, and calculating an angle between a predetermined side of the target object area and a predetermined direction in the plane image plane.
- the in-plane angle of the target object is obtained based on the planar image obtained by using the three-dimensional shape of the target object.
- the in-plane angle of the target object can be obtained more accurately at any time.
- the image can be accurately corrected.
- an imaging apparatus which includes a light projecting unit that projects pattern light, an image of a target object, and an image of the target object.
- Imaging means for acquiring an image; and three-dimensional information for calculating shape information relating to a three-dimensional shape of the target object based on an image obtained by imaging the target object onto which the pattern light has been projected by the imaging means.
- a shape conversion unit configured to convert the captured image into a plane image in which the target object is viewed from a predetermined direction, based on shape information calculated by the three-dimensional shape calculation unit.
- the imaging device includes a display unit that displays the plane image converted by the plane conversion unit, and an input unit that inputs a display mode change parameter for changing a display mode of the plane image displayed on the display unit.
- first changing means for changing the display mode of the planar image displayed on the display device based on the display mode change parameter input by the input device.
- the display mode of the planar image displayed on the display unit is changed based on the display mode change parameter, and the display mode change parameter is changed by the user. Since the input means power is also input, the display mode of the planar image can be edited according to the user's intention. Further, even if the captured image is not converted into the planned plane image for some reason, the plane image can be corrected according to the user's intention.
- FIG. 1 (a) is a perspective view of an entire imaging device according to a first embodiment
- FIG. 1 (b) is a schematic sectional view of the imaging device.
- FIG. 2 is a block diagram illustrating an entire configuration of the imaging device in FIG. 1.
- FIG. 3 is a schematic cross-sectional view of the imaging device in FIG. 1, showing a configuration of a slit light projecting unit.
- FIG. 4 is a view for explaining an image with slit light.
- FIG. 5 is a flowchart showing an operation of a force when a release button is pressed in the imaging apparatus.
- FIG. 6 shows data stored in the difference image storage unit
- FIG. 6 (b) shows data stored in the triangulation calculation result storage unit.
- FIGS. 7 (a), 7 (b) and 7 (c) are diagrams for explaining image data subjected to image processing by the imaging apparatus.
- FIGS. 8 (a) and 8 (b) are diagrams for explaining a three-dimensional spatial position calculation method.
- FIGS. 9 (a), 9 (b) and 9 (c) are diagrams for explaining a coordinate system in the document orientation calculation processing.
- FIG. 10 is a flowchart showing a processing procedure of a plane conversion processing.
- FIG. 11 is a flowchart showing a curvature calculation
- FIG. 11 (b) is a flowchart showing a processing procedure of a tilt distance calculation.
- FIG. 12 (a) is a flowchart showing a processing procedure of Z shift
- FIG. 12 (b) is a flowchart showing a processing procedure of inverse triangulation.
- FIG. 13 is a flowchart illustrating a processing procedure of in-plane displacement rotation recognition processing.
- FIG. 14 is a diagram showing a histogram of a simple luminance image.
- FIG. 15 is a flowchart showing a processing procedure of up / down / left / right displacement calculation.
- FIG. 16 is a flowchart showing a processing procedure of a rotation angle calculation.
- FIG. 17 is a diagram showing a histogram of a secondary differential image related to an image of a document.
- FIG. 18 is a flowchart showing a processing procedure of a rectangular area cutting process.
- FIG. 19 (a) is a perspective view of an imaging device according to a second embodiment
- FIG. 19 (b) is a rear view of the imaging device according to the second embodiment
- FIG. FIG. 9 is an enlarged view of an operation button of the imaging device according to the second embodiment.
- FIG. 20 is a schematic sectional view of an imaging device according to a second embodiment.
- FIG. 21 is a cross-sectional view illustrating a configuration of a slit light projecting unit of the imaging device according to the second embodiment.
- FIG. 22 (a) and FIG. 22 (b) are diagrams for explaining the angular width of slit light.
- FIG. 23 is a block diagram showing an electrical configuration of an imaging device according to a second embodiment.
- FIG. 24 is a flowchart showing a process after pressing a relays button in the imaging device of the second embodiment.
- FIG. 25 (a) and FIG. 25 (b) are views showing slit light image.
- FIG. 26 is a flowchart showing three-dimensional measurement and plane conversion processing in the imaging device according to the second embodiment.
- FIG. 27 (a) is a diagram showing data stored in a difference image storage unit
- FIG. 27 (b) is a diagram showing data stored in a triangulation calculation result storage unit.
- FIG. 28 (a), FIG. 28 (b), and FIG. 28 (c) are diagrams for describing image data subjected to image processing by the imaging device of the second embodiment.
- FIG. 29 is a flowchart showing an editing process in the imaging device of the second embodiment.
- FIGS. 30 (a) to 30 (h) are diagrams for explaining a method of operating the operation buttons.
- FIGS. 31 (a) to 31 (d) are diagrams for explaining an operation method of an operation button, and show a state of an image edited by operation of the operation button.
- FIG. 32 is a diagram showing three-dimensional coordinates set in the imaging device of the third embodiment.
- FIG. 33 is a block diagram showing an electrical configuration of the imaging device according to the third embodiment.
- FIG. 34 (a) to FIG. 34 (f) are diagrams showing an edit pattern of an image displayed on the LCD.
- FIG. 35 is a diagram for explaining how to move and rotate the imaging device according to the third embodiment in order to edit an image.
- FIG. 36 is a flowchart showing editing processing in the imaging device of the third embodiment.
- FIG. 1A is a perspective view of the entire imaging device 1 according to the first embodiment.
- FIG. 1B is a schematic cross-sectional view of the imaging device 1.
- FIG. 2 is a block diagram illustrating a configuration of a control system of the imaging device 1.
- the imaging apparatus 1 includes a rectangular box-shaped main body case 10, an imaging lens 31 provided on the front surface of the main body case 10, and a rear side of the imaging lens 31 (the imaging apparatus 1). And a slit light projecting unit 20 provided below the imaging lens 31.
- the imaging device 1 further includes a processor 40 built in the main body case 10, It has a release button 52 provided on the upper part of the main body case 10, a mode switching switch 59, a card memory 55 built in the main body case 10, and a size input section 56 provided on the back of the main body case 10. These components in the imaging device 1 are connected by signal lines (see FIG. 2).
- the imaging device 1 further includes an LCD (Liquid Crystal Display) 51 provided on the back of the main body case 10 and a finder 53 provided from the back to the front of the main body case 10.
- the LCD 51 and the finder 53 are used when the user determines the imaging range of the imaging device 1.
- the LCD 51 is a liquid crystal display or the like that displays an image, and receives an image signal from the processor 40 and displays an image. From the processor 40 to the LCD 51, a real-time image captured by the CCD image sensor 32, an image stored in the card memory 55, an image for displaying characters of the setting contents of the device, etc., depending on the situation. A signal is sent
- the imaging device 1 when the release button 52 is pressed by the user, external light enters through the imaging lens 31 and is imaged on the CCD image sensor 32 as image data.
- a mode functioning as a so-called digital camera is referred to as a “normal mode”.
- the imaging device 1 has a “corrected imaging mode” function in addition to the imaging function in the normal mode.
- the “corrected imaging mode” is an operation mode in which, when a document P as a subject is imaged from an oblique direction, image data corrected as if the image was captured with a frontal force is created and stored in the card memory 55.
- FIG. 3 shows the configuration of the slit light projecting unit 20 in the imaging device 1 in detail.
- the slit light projecting unit 20 includes a laser diode 21, a collimating lens 22, an aperture 23, a transparent flat plate 24, a cylindrical lens 25, a reflecting mirror 26, and a rod lens 27.
- the laser diode 21 emits a red laser beam.
- the collimating lens 22 focuses the laser beam from the laser diode 21 so that the laser beam is focused on a reference distance VP from the slit light projecting unit 20.
- the aperture 23 is formed of a plate having a rectangular opening. The laser beam incident on the aperture 23 from the collimating lens 22 is shaped into a rectangle by transmitting through the opening, and is emitted from the aperture 23 as a laser beam 70.
- the transparent flat plate 24 is made of a transparent flat plate made of a solid glass material or the like, and has an AR coat (anti-reflection coating) on the back surface.
- the transparent flat plate 24 is disposed at a predetermined angle
- the transparent plate 24 reflects a predetermined percentage (for example, 5%) of the laser beam 70 on the surface and transmits the rest.
- the direction in which the laser light 70 is reflected by the transparent flat plate 24 is called a second direction.
- the reflection mirror 26 is formed of a member such as a mirror that totally reflects the laser beam.
- the reflection mirror 26 is disposed at an angle of 45 degrees on the front side of the main body case 10 downstream of the laser beam 70 transmitted through the transparent flat plate 24, and totally reflects the laser beam 70 transmitted through the transparent flat plate 24 to form an optical path. Turn 90 degrees.
- the direction in which the laser light 70 is reflected by the reflection mirror 26 is referred to as a first direction.
- the rod lens 27 is formed of a cylindrical lens having a short positive focal length.
- the rod lens 27 is disposed downstream of the laser beam 70 reflected by the reflection mirror 26 so that the axial direction of the cylindrical shape is the erect direction. Since the focal length of the rod lens 27 is short, the laser light transmitted through the rod lens 27 immediately spreads beyond the focal position near the rod lens 27, and proceeds in the first direction as slit light having a predetermined spread angle.
- the slit light emitted from the rod lens 27 is hereinafter referred to as first slit light 71.
- the cylindrical lens 25 is a lens having a concave shape in one direction so as to have a negative focal length.
- the cylindrical lens 25 is disposed downstream of the laser beam 70 reflected by the transparent flat plate 24 so that the lens surface is orthogonal to the second direction.
- the cylindrical lens 25 emits the laser beam 70 reflected by the transparent flat plate 24 as slit light traveling in the second direction at a predetermined spread angle.
- the slit light emitted from the cylindrical lens 25 is referred to as a second slit light 72.
- a laser beam is emitted from the laser diode 21 in response to a command from the processor 40, and the first slit light 71 in the first direction and the second direction are emitted.
- the second slit light 72 is below the imaging lens 31 of the main body case 10. The light is emitted from a window 29 provided in the camera.
- FIG. 4 shows a state in which the first and second slit lights 71 and 72 are projected onto a document P which is a sheet-like member.
- a locus 7 la of the first slit light and a locus 72 a of the second slit light are formed on the document P by the first and second slit lights 71 and 72.
- the trajectory 71a of the first slit light and the trajectory 72a of the second slit light vary in length, inclination, and interval depending on the position and orientation of the document P as shown in FIG.
- the imaging lens 31 includes a plurality of lenses.
- the focal length is automatically adjusted by the imaging lens 31, and light of an external force is formed on the CCD image sensor 32.
- the CCD image sensor 32 is configured by arranging photoelectric conversion elements such as CCD (Charge Coupled Device) elements in a matrix.
- the CCD image sensor 32 converts a signal corresponding to the color and intensity of the light of the image formed on the surface into digital data and outputs the digital data to the processor 40.
- the data for one photoelectric conversion element is pixel data of a pixel forming an image, and the image data is composed of pixel data of the number of photoelectric conversion elements.
- the CCD image sensor 32 generates image data having a total number of pixels in the width direction of 1200 pixels and a total number of pixels in the height direction of 1600 pixels.
- the width direction position ccdx and the height direction position ccdy which are the coordinates of the image data, are represented by the number of pixels. That is, the range of the width direction position ccdx is “0”-“1199”, and the range of the height direction position ccdy is “0”-“1599”.
- the pixel interval of the CCD corresponds to a resolution of 100 dpi (dot per inch) in the document.
- the pixel data includes pixel information represented by three colors of red (R), green (G), and blue (B), each having 256 levels of luminance, per pixel.
- the release button 52 is constituted by a push-button switch, and is connected to the processor 40.
- the processor 40 detects that the user presses down the release button 52.
- the card memory 55 is constituted by a nonvolatile and rewritable memory, and is detachable from the main body case 10.
- the mode switching switch 59 is configured by a slide switch or the like that can be switched between two positions, and is connected to the processor 40.
- the processor 40 detects the position of the mode switching switch 59. Mode switch 59 , One position of the switch is detected by the processor 40 as “normal mode”, and the other position of the switch is detected as “corrected imaging mode”.
- the processor 40 includes a CPU (Central Processing Unit) 41, a ROM (Read Only Memory) 42, and a RAM (Random Access Memory) 43.
- the CPU 41 executes various processes based on the program stored in the ROM 42 while using the RAM 43.
- the processing executed by the CPU 41 includes detection of the pressing operation of the release button 52, detection of the state of the mode switching switch 59, capture of image data from the CCD image sensor 32, writing of image data to the card memory 55, and slit light. Switching of emission of slit light by the light emitting unit 20 and the like are included.
- the ROM 42 includes a camera control program 421, a difference extraction program 422, a triangulation calculation program 423, a document attitude calculation program 424, a plane conversion program 425, an in-plane displacement rotation recognition program 426, a rectangular area
- the cutting program 427 is stored.
- the camera control program 421 is a program relating to control of the entire imaging apparatus 1 including the processing of the flowchart shown in FIG. 5 (details will be described later).
- the difference extraction program 422 is a program for generating image data in which the locus of the slit light is extracted from the image of the document P on which the slit light is projected.
- the triangulation calculation program 423 is a program for calculating a three-dimensional spatial position of each trajectory of the slit light with respect to each pixel.
- the document attitude calculation program 424 is a program for estimating and obtaining the position and the three-dimensional shape of the document P.
- the plane conversion program 425 is a program for converting the image data stored in the slit light non-image storage section 432 into image data that also captures the frontal force of the document P.
- the in-plane displacement rotation recognition program 426 is a program for detecting the displacement and rotation angle of the document P in the image data.
- the rectangular area cutting program 427 is a program for converting the image data of the area where the document P is captured in the image data.
- the RAM 43 includes, as storage areas, a slit light image storage unit 431, a slit light non-image storage unit 432, and a difference storage unit having a size for storing data in the form of image data from the CCD image sensor 32.
- the image storage unit 433 is allocated.
- the RAM 43 further stores a triangulation calculation result storage unit 434 having a size for storing the result of calculating the position of each point of the slit light image, and a plane conversion result image having a size for storing the result obtained by converting the image data.
- Department 435 and an in-plane displacement rotation recognition result storage unit 436 for storing a result of an in-plane displacement rotation recognition process described later.
- the RAM 43 has a rectangular area cutout result image storage unit 437 for storing the result of a rectangular area cutout processing described later, and a size used for temporarily storing data for calculation in the CPU 41.
- the parking area 438 is assigned! /
- the finder 53 is configured by an optical lens. When the user looks into the finder 53 from behind the imaging device 1, the user can see a range that substantially matches the range where the image is formed on the CCD image sensor 32 by the imaging lens 31.
- the size input unit 56 is configured by a slide switch or the like that switches to a plurality of positions, and is connected to the processor 40.
- the size of the document P to be imaged is determined by the processor 40 in accordance with the position of the switch of the size input unit 56 operated by the user. For example, in the processor 40, when the position of the switch of the size input section 56 is operated by the user, the leftmost position is "unspecified”, the next position is "A4 size”, and the rightmost position is "A3". Judge as "size”.
- S 110 it is determined whether or not the position of the mode switching switch 59 is in the “corrected imaging mode”. If the result of determination is that the position is the “corrected imaging mode” (S110: YES), the process proceeds to S111. If the position is in the “normal mode” instead of the “corrected imaging mode” (S110: NO), the process proceeds to S120.
- Sl 1 camera control processing for acquiring an image with slit light and an image without slit light is performed. Specifically, after the laser diode 21 is instructed to emit light to the slit light projecting unit 20 and the first slit light 71 and the second slit light 72 are emitted, the CCD image sensor 32 is used as an image with slit light. Thus, image data of the document P on which the first and second slit lights 71 and 72 are projected as shown in FIG. 4 is obtained. Then, the image data is stored in the slit light image storage unit 431 of the RAM 43.
- the light emission of the laser diode 21 is stopped by the finger with respect to the slit light projecting unit 20.
- image data is acquired from the CCD image sensor 32 as an image without slit light. This image data is stored in the slit light non-image storage section 432.
- a difference extraction process is executed. That is, the difference between the image data of the slit light image storage unit 431 and the image data of the slit light non-image storage unit 432 (that is, the trajectory 71a of the first slit light projected on the document P) by the difference extraction program 422 Then, image data from which the locus 72a) of the second slit light is obtained is generated and stored in the difference image storage unit 433.
- the lower half of the image data stored in the difference image storage unit 433 (the height direction position ccdy is between “800” and “1599”) Then, for every 20 pixels between the width direction positions ccdx of “0” and “1199”, the position where the luminance level in one column of pixels in the height direction is the maximum is obtained.
- the index idx numbered in the order in which the positions were determined, the width position ccdx when the positions were determined, and the height position ccdy with the maximum luminance level are shown in Fig. 6 (a).
- the maximum position of the luminance level is the position of the maximum value of the approximate curve connecting the luminance levels of the respective pixels in the height direction.
- the locus 71a of the first slit light is determined to be absent, and is not stored as the slit light image position data 433a.
- the maximum value of the index idx is set to the number N of extracted pixels, which is the total number of pixels extracted as position information of the locus 7 la of the first slit light.
- the difference image storage unit is used as the position information of the second slit light trajectory 72a in the same format as the slit light image position data 433a.
- the difference image storage unit 433 is determined. Is not remembered.
- a document attitude calculation process is executed. That is, using the slit light three-dimensional position data 434a, the three-dimensional spatial position (0, 0, L), inclination ⁇ , and curvature ⁇ of the document P are obtained by the document posture calculation program 424 (details will be described later). ).
- a plane conversion process is executed. That is, the processing shown in the flowchart of FIG. 10 is executed by the plane conversion program 425 (details will be described later), and the image stored in the slit light non-image storage section 432 is obtained by observing the document P from a substantially orthogonal direction.
- the original image P is converted into a flat image (for example, an image in which the document P has a rectangular shape as shown in FIG. 7A), and is stored in the plane conversion result image storage unit 435.
- in-plane displacement rotation recognition processing is executed. That is, the processing shown in the flowchart of FIG. 13 (details will be described later) is performed by the in-plane displacement rotation recognition program 426 on the plane image stored in the plane conversion result image storage unit 435.
- the image area of the document P in the plane image is identified, and the displacement DH in the width direction of the center position of the image area of the document P with respect to the center position of the plane image, and The height direction displacement DV and the rotation angle DROT of a predetermined side of the image area of the original P in the planar image are obtained and stored in the in-plane displacement rotation recognition result storage unit 436.
- a rectangular area cutout process is executed. That is, the processing shown in the flowchart of FIG. 18 (details will be described later) is performed by the rectangular area cutting program 427.
- the slit light non-image stored in the slit light non-image storage section 432 is stored in a state in which the image area of the document P observed from a substantially orthogonal direction is upright as shown in FIG. 7C. Only the image area of the original P is converted to an image that is cut out.
- the cut image is stored in the rectangular region cut result image storage unit 437.
- the processing by the difference extraction program 422 in S112 is specifically the following processing. For each pixel, subtract the RGB value of the corresponding pixel of the image data of the image data of the slit lightless image storage unit 432 from the RGB value of the pixel of the image data of the image file with slit light 431. As a result, a multi-valued image in which only the trajectories 71a and 72a of the first and second slit lights are obtained is obtained.
- the processing by the triangulation calculation program 423 in S113 is specifically the following processing.
- the three-dimensional spatial position for each point determined by the slit light image position data 433a is obtained as follows.
- a coordinate system for imaging a document P curved in the width direction as shown in FIG. 4 is defined as follows. That is, as shown in FIG. 8, the optical axis direction of the imaging lens 31 is defined as the Z axis, a position away from the imaging device 1 by the reference distance VP is defined as the origin position of the X, Y, and Z axes. The horizontal direction is the X axis, and the vertical direction is the Y axis.
- the total number of pixels in the width direction of the CCD image sensor 32 is called ResX, and the total number of pixels in the height direction is called ResY. Furthermore, the upper end of the position where the CCD image sensor 32 is projected on the XY plane through the imaging lens 31 is called Yftop, the lower end is called Yfbottom, the left end is called Xfstart, and the right end is called Xfend. Further, D is the distance from the optical axis of the imaging lens 31 to the optical axis of the first slit light 71 emitted from the slit light projecting unit 20.
- the position in the Y-axis direction where the first slit light 71 intersects the X-Y plane is lasl
- the position in the Y-axis direction where the second slit light 72 intersects the XY plane is las2.
- the three-dimensional spatial position (XI, Yl, Z1) corresponding to the coordinates (ccdxl, ccdyl) on the image of the point determined by the predetermined data of the slit light image position data 433a is stored in the CCD image sensor 32.
- the following five simultaneous equations are established for a triangle formed by a point on the imaging plane of, an emission point of the first slit light 71 and the second slit light 72, and a point intersecting the XY plane. Derive from the solution.
- the three-dimensional spatial position (X2, Y2, Z2) corresponding to the coordinates (ccdx2, ccdy2) on the image of the point determined by the position information of the locus 72a of the second slit light is represented by the following five It is derived from the solution of the simultaneous equations.
- the processing by the document orientation calculation program 424 in SI 14 is specifically the following processing.
- a regression curve for each point of the three-dimensional space position determined by the slit light three-dimensional position data 434a is obtained.
- a straight line is assumed that connects a point where the position of the curve in the X-axis direction is “0” and a point determined by the position information of the locus 72a of the second slit light.
- the point at which this line intersects the Z axis that is, the point at which the optical axis intersects the document P, is determined as the three-dimensional spatial position (0, 0, L) of the document P (hereinafter referred to as position L) (see FIG. 9 (a);;).
- position L three-dimensional spatial position (0, 0, L) of the document P
- FIG. 9 (a) an angle between the straight line and the XY plane is obtained as the inclination ⁇ of the document P.
- the regression curve of the trajectory 71a of the first slit light is rotationally transformed in the reverse direction by the previously obtained inclination ⁇ . That is, consider a state in which the original P is parallel to the XY plane. Then, as shown in FIG. 9 (c), for the cross section of the document P in the X—Z plane, the displacement in the Z-axis direction is obtained at a plurality of points in the X-axis direction. Find the curvature ⁇ (X), which is a function of the tilt in the X-axis direction as a variable.
- the processing by the plane conversion program 425 in S115 is, for example, the processing described in the flowchart of FIG. 10 and described below.
- the working area of RAM43 43 A processing area for the process is allocated to 8, and an initial value of a variable j used for the process, such as a variable for a counter, is set (S121).
- the slit light stored in the slit light non-image storage unit 432 is stored.
- P is rectangular, and the entire image area has a trapezoidal shape. ;)) Is required.
- a rectangular planar image region including this region is set (see FIG. 7A), and the number of pixels Qa included in this region is obtained (S122).
- the pixels in the area of the plane image have a size that is the resolution (25 dpi) of 1Z4 of the resolution (100 dpi) of the image data without slit light stored in the non-slit light image storage section 432. Therefore, the number of pixels Qa is 1Z16 as compared with the case where the resolution is set for the image data without slit light.
- the three-dimensional space position obtained in this manner is converted into coordinates (ccdx, (1)) on the image data by the CCD image sensor 32 using the above-described relational expression of the triangulation calculation processing (3128). If the force is within the range of the image data stored in the slit lightless image storage unit 432 (S129: YES), the pixel information force at the converted coordinates S
- the slit light stored in the slit lightless image storage unit 432 Pixel information power of image data without image-It is obtained by the arrest neighbor method and stored in the plane conversion result image storage unit 435 (S130) .
- the converted coordinates of the image data stored in the slit lightless image storage unit 432 If the value is not within the range (S129: NO), the value of "black" is stored as pixel information in the plane conversion result image storage unit 435 (S131). (S132, S123).
- the nearest neighbor method is to calculate the luminance value of the nearest point from which luminance is to be obtained.
- the curvature calculation in S125 is specifically performed as follows.
- the points ( ⁇ ', ⁇ ', ⁇ ') at which the predetermined point (X, Y, Z) of the three-dimensional coordinates in the area of the plane image is displaced in the Z-axis direction based on the curvature ⁇ (X) are described below. It is obtained by the procedure shown in the flowchart of FIG.
- Y ′ is left as it is (S1252).
- the calculation process of the tilt distance in S126 is specifically performed as follows.
- the points (X “, ⁇ ", ⁇ ") obtained by rotating the points ( ⁇ ', ⁇ ', ⁇ ') subjected to the curvature calculation by 0 around the X-axis are shown in the flowchart shown in Fig. 11 (b). Is required.
- the shift in the Z direction in S127 is specifically performed as follows.
- the point ( ⁇ "', ⁇ '", ⁇ '"), which is obtained by shifting the tilt-corrected point (X", Y ", Z") by the position L in the Z-axis direction, is shown in FIG. ) Is obtained by the procedure shown in the flowchart of FIG.
- the processing by the in-plane displacement rotation recognition program 426 in the SI 16 is, for example, the processing described below shown in the flowchart of FIG.
- the plane image data stored in the plane conversion result image storage unit 435 is converted into monochromatic luminance image data. That is, the plane image data is converted into image data obtained by adding the luminance values of the R, G, and B colors of each pixel (S161).
- a histogram relating to the image luminance distribution of the entire monochromatic planar image data is calculated (for example, as shown in the graph of FIG. 14 in which the horizontal axis represents the luminance and the vertical axis represents the frequency, the region where the luminance is high and the luminance The frequency distribution is divided into a region having a low center of gravity), and the center of gravity value is obtained (S162).
- pixel information of a portion having a luminance higher than the center of gravity of the histogram is “1” (that is, “white”)
- pixel information of a portion having a luminance lower than the center of gravity of the histogram is “0” (that is,
- the image is converted to binary image data (“black”) (S163).
- black binary image data
- the rotation angle DROT of the side in the width direction of the original (the area of the document P) with respect to the width direction of the binarized image is obtained by the procedure shown in the flowchart of FIG. After that, the process ends.
- FIG. 15 is a flowchart showing the vertical / horizontal displacement calculation in S164.
- the vertical / left / right displacement calculation will be described.
- the position of the center of gravity of the area including the original P of the image based on the binary image data (that is, “white”) is obtained (S1641).
- the displacement DH in the width direction and the displacement DV in the height direction are obtained as the displacement of the center of gravity of the “white” region with respect to the center position of the image (S1642).
- the width direction displacement DH and the height direction displacement DV are stored in the in-plane displacement rotation recognition result storage unit 436 (S1643).
- FIG. 16 is a flowchart showing the rotation angle calculation in S165. The rotation angle calculation will be described.
- the binarized image data converted in S163 is read (S1651), and the parameter angle ⁇ and the DMAX value are initialized (S1652). Then, the image based on the binarized image data is rotated so as to have the parameter angle ⁇ (S1655). Next, an image obtained by performing a second derivative in the width direction of the image is generated (S1656). That is, an image in which the outline of the area of the original ⁇ remains is generated.
- a histogram of the number of “white” pixels in the height direction with respect to the width direction of the quadratic differentiated image is calculated (S1657).
- the horizontal axis is the position in the width direction of the image
- the vertical axis is the frequency of the “white” portion in the height direction.
- the value of the histogram increases, and the value at this point is obtained as the maximum value dmax (Y) of the histogram at the parameter angle ⁇ (S1658).
- the value of the maximum value dmax ( ⁇ ) is compared with the maximum value DMAX of the histogram so far (S1659).
- the rotation angle DROT is replaced with the parameter angle 0 (S1661). Then, the maximum value DMAX is replaced with the maximum value dmax (Y) at the parameter angle ⁇ (S1662). If the parameter angle ⁇ is not larger than the maximum value DMAX (S1659: NO), the rotation angle DROT and the maximum value DMAX are left as they are.
- the angle at which the maximum value of the contour histogram becomes the largest when the image based on the binarized image data is rotated from the initial rotation angle ⁇ start to the final rotation angle ⁇ end Can be requested.
- the left or right side of the outline of the document P is parallel to the height direction of the image, as shown by the dotted line in FIG. 17, the outline of the left and right sides is located in the width direction.
- the angle at which the histogram value is maximized by rotating the area of the document P is obtained as the rotation angle DROT of the document P.
- the processing by the rectangular area cutting program 427 in S117 is performed by, for example, the flow of FIG. This is the processing shown in the chart and described below.
- a processing area for the processing is allocated to the working area 438 of the RAM 43, and an initial value of a variable j used for the processing, such as a variable for a counter, is set (S171).
- the four corner points of the image without slit light are rotationally moved around the Y axis by a curved ⁇ (X), and rotated about the X axis by ⁇ , and the position L in the Z axis direction is DH and height displacement DV in the X-Y plane, and rotate about the Z axis by the rotation angle DROT.
- the document P is positioned at the center of the area formed by the four converted points.
- an area formed by the four points is cut out in the size input by the size input unit 56 (that is, the surface of the original P on which the characters and the like are written is observed from a substantially orthogonal direction, and the original P A rectangular image is rotated so that it is parallel to the width and height directions of the image, and an area of a cut image that is an image in which only the area of the document P is cut) is set, and pixels included in this area are set.
- the number Qb is obtained (S172). It should be noted that the pixels in the region of the cut-out image are pixels having the same resolution (100 dpi) as the resolution of the image data without slit light stored in the image storage unit 432 without slit light.
- the three-dimensional spatial position of each pixel included in the set cropped image area is rotated at a rotation angle DROT about the Z axis (S175), and the width direction displacement is applied to the coordinates of this pixel.
- DH and height direction displacement DV are added (S176), displaced in the Z-axis direction based on the curvature ⁇ (X) (S177), and rotated around the X-axis with a slope ⁇ (S178), and Z-axis direction Is shifted by the position L (S179).
- the obtained three-dimensional spatial position is converted into coordinates (ccdcx, (1)) on the CCD image captured by the ideal camera (3180) using the relational expression of triangulation (3180), and the imaging lens used.
- the pixel information at the converted coordinates is obtained from the pixel information of the slit light non-image by the bilinear method. It is obtained and stored in the working area 438 of the RAM 43 as the pixel information of the area set in S172 (S182).
- the above processing is repeatedly executed for each pixel position in the area obtained in S172 by the number of pixels Qb (S183, S173).
- the processing area is released ( S1 74).
- the bilinear method is a method of calculating a force corresponding to an area before a certain point after conversion and calculating an average value of colors of pixels in the area as a color of the point after conversion. .
- pixel information exists at a point at an integer value position, and if the coordinates at which the position before conversion is calculated become (0.2, 0.8), four surrounding points (0, 0) ), (0, 1), (1, 0), (1, 1)
- S177-S180 are the same processing jets as S125-S128.
- the imaging apparatus 1 emits two rows of slit light, the first slit light 71 and the second slit light 72, and outputs the original P on which these slit lights are projected by the CCD image sensor 32. Take an image, and then take an image of the document P on which the slit light is not projected. Further, the imaging device 1 calculates the difference between the two image data, extracts the trajectory of the slit light of the image data, calculates the three-dimensional spatial position of each part of the trajectory of the slit light based on the principle of triangulation, and calculates the document P from these. Find the position L, inclination ⁇ , and curvature ⁇ (X) of.
- the imaging device 1 projects the previously captured slit light, and The image is converted into image data as if the flat original P was viewed from the front in the upright direction, and only the original P was cut out, and recorded in the card memory 55.
- the image stored in the card memory 55 can be displayed on the LCD 51 to check the captured content.
- the image in the card memory 55 can be displayed on an external personal computer or printed. is there.
- the user slides the mode switching switch 59 to the “corrected imaging mode” and sets the size input unit 56 to the size of the document P to be imaged.
- the posture of the document P is obtained by analyzing only the positions of the first and second slit lights 71 and 72 in the image. Therefore, the posture of the document P with high detection accuracy can be easily obtained without performing complicated image analysis. Therefore, the image analysis processing by the processor 40 of the imaging device 1 can be executed in a short time. Thus, the effect that the processor 40 can be simplified can be obtained.
- the color information that is good if the area of the document P can be identified has little relation to the processing content!
- the amount of information used for processing can be reduced, the processing content can be simplified, and the processing speed can be increased.
- each image has the same resolution as the largest image without slit light.
- the images are converted to the same number of colors.
- the camera control processing in S111 by the imaging lens 31, the CCD image sensor 32, the slit light projecting unit 20, and the processor 40 is positioned as an imaging unit.
- a series of processing (S112-S114) from the difference extraction processing to the document attitude calculation processing is positioned as three-dimensional shape detection means.
- the plane conversion processing (S115) in FIG. 5 is positioned as plane conversion means.
- the in-plane displacement rotation recognition processing (S116) is positioned as in-plane position detection means and in-plane angle detection means.
- the rectangular area cutout processing (S117) is positioned as the first and second correction conversion means.
- the size input unit 56 is positioned as a range input unit.
- both the in-plane displacement (in-plane position) and the in-plane rotation angle are detected and used for correction in S116 of FIG. 5, but in S116 There may be a configuration in which only one of the in-plane displacement and the in-plane rotation angle is detected and used for correction.
- the slit light projecting unit 20 detects a three-dimensional shape by using two rows of the first slit light 71 and the second slit light 72. ing .
- a three-dimensional shape may be detected by using three or more rows of slit light.
- the three-dimensional shape may be detected by using a light beam having a specific radiation pattern instead of the slit light.
- the image stored in the card memory 55 which is the final image data
- the image data is cut out at the size of the original P specified by the size input unit 56
- the method of specifying the cutout range is not limited to this.
- the cutout range is determined based on the area of the original P (the “white” area) identified by the binary image data converted in the in-plane displacement rotation recognition processing of S 116. It may be set.
- the area formed by converting the four corner points of the image without slit light in S172 of the rectangular area cutout processing in S117 is input by the size input unit 56 as the range to be the final image data.
- the area is cut out by the size of the force, the area is not limited to this.
- the area force set in S172 of the rectangular area cutout processing in S117 is not cut out in the size input in the size input unit 56, and the four corners of the slit lightless image are not cut out.
- a rectangular range including an area formed by converting points may be used as final image data.
- the range of the final image data may be a region where the rotation is not performed at the rotation angle DROT in S172 and the processing in S175 is not performed, or the width direction displacement DH and The vertical displacement and the vertical displacement are not converted by the DV in the height direction, and the processing in S176 is not performed as an area.
- the four corner points of the image without slit light are converted into the area formed by the points converted by the curvature ⁇ (X), the inclination 0, and the position L, respectively.
- the area that is deformed by the size input by the size input unit 56 may be limited, and the conversion may be performed for each pixel in this area. By doing so, the number of pixels as a planar image is reduced, so that the processing amount in the plane conversion processing can be reduced, and the processing amount of the in-plane displacement rotation recognition processing in S116 can also be reduced. Thus, the entire processing by the processor 40 can be performed at high speed.
- the image data without slit light is converted into a single color and the resolution is 1Z4.
- the resolution may be converted at the resolution of the image without slit light. It is also acceptable that the color information is processed as it is, instead of using a single color.
- the resolution is lower than that of the image without slit light, and the plane image is converted to a plane image in which the number of colors is a single color.
- the same solution as the slit light no image It is possible to reduce the processing amount in this processing and the processing amount in the in-plane displacement rotation recognition processing S116 using this planar image, as compared with an image having the same image number and the same color number.
- the entire processing by the processor 40 can be shortened.
- the target object to be imaged by the imaging apparatus 1 may be other than the object having a rectangular shape like the document P described in the above embodiment.
- the processing of the rotation angle calculation (S165) in the in-plane displacement rotation recognition processing of S116 involves complicated shape determination processing such as searching for a matching shape by pattern matching on a plurality of shape data. Therefore, the above-described embodiment is simpler and better.
- one image capturing device 1 performs the process from capturing an image to converting the image.
- data is transferred to an image processing device such as a personal computer external to the image capturing device 1.
- the processing of S112-S117 may be performed separately for each device so that the processing can be performed.
- the image processing apparatus views the target object image from the predetermined direction based on the shape information and the position of the target object obtained by the in-plane position detecting means.
- the image processing apparatus may further include a first correction conversion unit configured to convert the target image into a corrected image in which the target object is disposed at a substantially central position.
- the image processing apparatus identifies a target object region in a plane image and obtains an angle of a predetermined side of the target object region with respect to a predetermined direction in the plane image plane.
- An in-plane angle detecting means may be further provided.
- the first correction conversion means includes shape information, the position of the target object obtained by the in-plane position detection means, and the in-plane angle detection means. Based on the angle, the target object image is corrected so that the target object can be viewed from a predetermined direction, the target object is arranged at a substantially central position, and a predetermined side of the target object is directed in a predetermined direction in the image. It may be configured to convert to an image.
- the direction in which the predetermined side of the target object is turned in the corrected image is set to the direction in which the characters written on the original document as the target object are parallel to the width direction of the corrected image.
- the first correction conversion means may be configured to convert the corrected image into a corrected image only for a region of the target object image where the target object is captured.
- a portion other than the target object for example, the background
- a corrected image including only the target object can be obtained. Therefore, the corrected image can be more easily viewed.
- image data at unnecessary locations is deleted and the data amount of the entire image data can be reduced, it is possible to reduce the amount of image processing when deriving the data and when using the data. it can.
- the first correction conversion unit may be configured to convert the image into a corrected image having the same number of colors as the number of pixels of the target object image.
- the input color information of the target object image is effectively used, and the converted corrected image can be easily viewed.
- the in-plane position detection means and the in-plane angle detection means detect the position and angle of the target object in the image. These parameters can be detected without using color information.
- processing in order to reduce the amount of data to be processed, processing may be performed using an image in which the number of pixels is smaller than that of the target object image.
- the image processing apparatus may further include an image clipping means for extracting an image in which a portion of the corrected image in which the target object is captured is cut in a predetermined shape. good.
- the image processing device may further include a range input unit for inputting a range of the target object image to be converted by the plane conversion unit.
- the plane conversion unit converts the range of the target object image input by the range input unit into a plane image.
- the amount of image processing can be reduced by limiting the range of conversion processing. This makes it possible to shorten the time for image processing.
- the target object has a rectangular shape when viewed from a direction orthogonal to a predetermined plane.
- the shape of the target object is rectangular, the in-plane position detection means, the in-plane angle detection means, etc.
- the target object region can be easily identified by utilizing the feature that the rectangular shape is composed of four orthogonal sides.
- the plane conversion means may be configured to convert the image into a plane image having a lower resolution than the resolution of the target object image.
- the resolution of the planar image affects the detection accuracy of the in-plane position detecting means and the in-plane angle detecting means. For this reason, it is preferable that the resolution of the planar image be as low as possible so that the influence on the detection accuracy is not increased.
- the plane conversion means is configured to convert the image into a plane image having a single color pixel!
- the amount of information held by the pixels of the planar image is reduced, and the amount of conversion into a planar image by the planar conversion unit and the amount of processing when processing using the planar image can be reduced.
- the imaging device includes the above-described image processing device, a light projecting unit that projects pattern light, and an imaging unit that captures a target object and obtains a target object image. And a three-dimensional shape detecting means for calculating shape information on the three-dimensional shape of the target object based on an image obtained by imaging the target object on which the pattern light is projected by the imaging means.
- the target object image obtained by the imaging means and the shape information calculated by the three-dimensional shape detection means are input to the image processing device.
- the imaging device includes the above-described image processing device, a light projecting unit that emits pattern light, an image of a target object, obtains a target object image, and performs image processing. Based on imaging means input to the apparatus and an image obtained by imaging the target object on which the pattern light is projected by the imaging means, shape information on the three-dimensional shape of the target object is calculated and the shape information is calculated.
- Three-dimensional shape detection means input to the image processing device, display means for displaying the corrected image obtained by the first correction conversion means, and display mode for changing the display mode of the corrected image displayed on the display means Enter the change parameters And means for changing the display mode of the corrected image displayed on the display means based on the display mode change parameter input by the input unit.
- the imaging device includes the above-described image processing device, a light projecting unit that projects pattern light, an image of a target object, obtains a target object image, and performs image processing. Based on imaging means input to the apparatus and an image obtained by imaging the target object on which the pattern light is projected by the imaging means, shape information on the three-dimensional shape of the target object is calculated and the shape information is calculated.
- Three-dimensional shape detection means input to the image processing device, display means for displaying a corrected image obtained by the second correction conversion means, and display mode for changing the display mode of the corrected image displayed on the display means
- Input means for inputting change parameters
- change means for changing the display mode of the corrected image displayed on the display means based on the display mode change parameter input by the input means, Also good,.
- FIG. 19A is a perspective view illustrating an appearance of an imaging device 1B according to the second embodiment.
- FIG. 19 (b) is a rear view of the imaging device 1B
- FIG. 19 (c) is an enlarged view of the operation button 57.
- FIG. 20 shows a schematic cross-sectional view of the imaging device 1B.
- FIG. 20 shows a state where the document P is imaged by the imaging device 1B.
- the imaging device 1B has a normal "normal mode” function and a "corrected imaging mode” function. Have. In the “normal mode”, the imaging device 1B functions as a normal so-called digital camera. In the “corrected imaging mode”, the imaging device 1B automatically generates an image as if the original image was captured from the front side of the original P even when the original P as the subject was also imaged in oblique directions. And a device that can further edit the corrected image according to the operator's intention.
- the imaging device 1B includes a rectangular box-shaped main body case 10, an imaging lens 31 provided in front of the main body case 10, a window 29 provided below the imaging lens 31, A release button 52 and a mode switching switch 59 are provided on the upper part of the camera. Further, the imaging device 1B has an LCD (Liquid Crystal Display) 51, a size input section 56, an operation button 57, an edit parameter selection button 65 provided on the rear of the main body case 10, and a rear force of the main body case 10 also distributed through the front. A finder 53 is provided, and a confirmation button 58 provided on a side surface of the main body case.
- LCD Liquid Crystal Display
- the operation button 57 is a button for editing an image displayed on the LCD 51.
- the operation button 57 is composed of eight independent buttons 57a-57h, and has a circular outer shape. That is, the operation button 57 includes a left rotation button 57a for rotating the image displayed on the LCD 51 to the left, an upward movement button 57b for moving the image upward, and a right rotation button 57c for rotating the image right. , A left button 57d for moving to the left, a right button 57e for moving to the right, a reduction button 57f for reducing, and a down button 5 for moving down. It consists of 7g and an enlargement button 57h for enlargement.
- a CCD image sensor 32 is provided behind the imaging lens 31 (inside the imaging device 1B), and a slit light projecting unit 20 is provided below the imaging lens 31. Is set up. Further, the processor 40 and the memory card 55 are built in the main body case 10. In FIG. 20, the illustration of the finder 53 not connected to the processor 40 and the confirmation button 58 provided on the side surface of the main body case are omitted.
- the imaging lens 31 includes a plurality of lenses.
- the focal length and the aperture are automatically adjusted in the imaging lens 31 by the auto-focus function of the imaging device 1B, and light of an external force is formed on the CCD image sensor 32.
- the CCD image sensor 32 has a configuration in which photoelectric conversion elements such as CCD (Charge Coupled Device) elements are arranged in a matrix.
- the CCD image sensor 32 generates a signal corresponding to the color and intensity of the light of the image formed on the surface, converts the signal into digital data, and outputs the digital data to the processor 40.
- the data for one CCD element is the pixel data of the pixels forming the image, and the image data is composed of the pixel data of the number of CCD elements.
- the CCD image sensor 32 of the present embodiment generates image data having a total number of pixels in the width direction of 1200 pixels and a total number of pixels in the height direction of 1600 pixels.
- the width direction position ccdx and the height direction position ccdy which are the coordinates of the image data, are represented by the number of pixels, and the first pixel is defined as a “0” position with the pixel at the upper left of the image as a base point. That is, the range of the width direction position ccdx is “0”-“1199”, and the range of the height direction position ccdy is “0”-“1599”.
- the pixel interval of the CCD corresponds to a resolution of 100 dpi (dot per inch) in the original.
- the pixel data has pixel information represented by three colors of red (R), green (G), and blue (B), each having 256 levels of luminance.
- the release button 52 is configured by a push button type switch.
- the release button 52 is connected to the processor 40, and the processor 40 detects a user's pressing operation of the release button 52.
- the mode switching switch 59 is configured by a slide switch or the like that can be switched between two positions.
- the switch positions are assigned by the processor 40 so that one switch position of the mode switching switch 59 is detected as "normal mode” and the other switch position is detected as "corrected imaging mode".
- the ⁇ normal mode '' is a mode in which the image of the original P itself is taken as image data
- the ⁇ corrected imaging mode '' is an image of the original P taken from the front when the original P is imaged from an oblique direction. In this mode, the image data is corrected.
- the memory card 55 is composed of a nonvolatile and rewritable memory, and is detachable from the main body case 10.
- the LCD 51 is configured by a liquid crystal display or the like that displays an image, and receives an image signal from the processor 40 and displays an image. From processor 40 to LCD51, In response, an image signal for displaying a real-time image received by the CCD image sensor 32, an image stored in the memory card 55, characters of the setting contents of the apparatus, and the like is transmitted.
- the size input unit 56 is configured by a slide switch or the like that switches to a plurality of positions, and is connected to the processor 40.
- the size of the document P to be imaged is determined by the processor 40 according to the position of the size input section 56 set by the user. For example, in the processor 40, the position of the switch of the size input section 56 is set by the user, and when the leftmost end is “unspecified”, the next position is “A4 size”, and the rightmost end is “A3 size”. ".
- the edit parameter selection button 65 is a button for selecting a specific edit parameter from among the edit parameters stored in an edit parameter storage unit 439 described later.
- the edit parameter selection button 65 is constituted by a push button type switch.
- the editing parameter selection button 65 is connected to the processor 40, and the processor 40 detects a user's pressing operation of the editing parameter selection button 65. When a predetermined editing parameter is selected by the pressing operation of the editing parameter selection button 65, the image displayed on the LCD 51 is edited based on the selected editing parameter.
- Each of the buttons 57a-57h of the operation button 57 is constituted by a push button type switch.
- Each of the buttons 57a-57h is connected to the processor 40, and the processor 40 detects a pressing operation of each of the buttons 57a-57h by the user.
- the amount of movement and enlargement / reduction of the image for each push-down operation of each button 57a-57h are set in advance, and the amount of movement or enlargement of the image according to the number of times each button 57a-57h is pressed down. The reduction ratio is controlled.
- the finder 53 is configured by an optical lens.
- the viewfinder 53 is configured so that when the user also looks into the viewfinder 53 with the rear side force of the imaging device 1B, a range almost coincident with the range where the imaging lens 31 forms an image on the CCD image sensor 32 can be seen. You.
- the confirm button 58 is a button for determining whether or not to save the image displayed on the LCD 51.
- the confirm button 58 is configured in the form of a push button.
- the enter button 58 is connected to the inlet processor 40, and the processor 40 detects a pressing operation by the user. By pressing this confirm button 53 while an image is displayed on the LCD 51, the image is displayed on the LCD 51 and can be saved in a state where the image is displayed.
- FIG. 21 is a diagram showing a configuration of the slit light projecting unit 20.
- FIG. 22 (a) and FIG. 22 (b) are diagrams for explaining the angular width of the slit light.
- the slit light projecting unit 20 includes a laser diode 21, a collimating lens 22, an aperture 23, a transparent flat plate 24, a cylindrical lens 25, a reflecting mirror 26, and a rod lens 27.
- the laser diode 21 emits a red laser beam.
- the emission and the stop of the laser beam in the laser diode 21 are switched.
- the output of the laser diode 21 is adjusted so that a constant output (for example, lmW) can be obtained at the point passing through the aperture 23 in consideration of the individual dispersion of the spread angle of the laser beam with respect to the maximum output rating (for example, 5 mW).
- the rated output has been adjusted.
- the collimating lens 22 focuses the laser beam from the laser diode 21 so as to focus on a reference distance VP (for example, 330 mm) from the slit light projecting unit 20.
- VP for example, 330 mm
- the aperture 23 is formed of a plate having a rectangular opening.
- the laser beam from the collimating lens 22 is shaped into a rectangle by passing through the opening of the aperture 23.
- the transparent plate 24 is made of a transparent plate made of a solid glass material or the like, and has an AR coating on the back surface.
- the transparent flat plate 24 is disposed at a predetermined angle
- the transparent flat plate 24 reflects about 5% (about 50 ⁇ W) of the power of one laser beam incident from the aperture 23 on the surface and transmits about 95% (about 950 W).
- a second direction the direction in which the laser beam is reflected by the transparent flat plate 24 (upward by 33 degrees with respect to the horizontal plane in front of the imaging device 1B) is referred to as a second direction.
- the reflection of one beam of the laser beam entering the transparent plate 24 when exiting from the transparent plate 24 is reduced, and the one beam of the laser beam inside the transparent plate 24 is reduced.
- the loss has been reduced.
- the ratio of one laser beam reflected by the transparent plate 24 is set to a surface reflectance of 5% determined by the refractive index of the material of the transparent plate 24, the reflection surface required for realizing with a normal half mirror can be obtained.
- the process of forming the metal deposition film can be omitted.
- the reflection mirror 26 is composed of a member such as a mirror that totally reflects the laser beam. Reflection mira
- the reference numeral 26 is disposed downstream of the laser beam transmitted through the transparent flat plate 24 at an angle of 45 degrees in front of the main body case 10, and totally reflects the laser beam to change the direction of the optical path by 90 degrees.
- the direction in which the laser beam is reflected by the reflecting mirror 26 (the direction of 0 ° with respect to the horizontal plane in front of the imaging device 1B) is referred to as a first direction.
- the rod lens 27 is formed of a cylindrical lens having a short positive focal length, and is disposed downstream of the laser beam reflected by the reflection mirror 26 so that the axial direction of the cylindrical shape is vertical. I have. Since the focal length of the rod lens 27 is short, one laser beam transmitted through the rod lens 27 spreads beyond the focal point near the red lens 27 and forms a slit light having a predetermined spread angle ⁇ (for example, 48 degrees). Go in the direction of 1.
- the slit light emitted from the rod lens 27 is referred to as first slit light 71.
- the cylindrical lens 25 is a lens having a concave shape in one direction so as to have a negative focal length, and is provided downstream of the laser beam reflected by the transparent flat plate 24 with respect to the second direction. They are arranged so that the planes are orthogonal.
- the cylindrical lens 25 emits the laser beam incident from the transparent flat plate 24 as slit light that spreads at a spread angle ⁇ (see FIG. 22 (b)).
- the slit light emitted from the cylindrical lens 25 is referred to as a second slit light 72.
- the spread angle ⁇ of the second slit light 72 is given by the spread angle ⁇ of the first slit light 71 and the spread angle ⁇ of the second slit light 72 with respect to the power ratio when the laser beam is split by the transparent plate 24.
- the angle is such that the ratio with the spread angle ⁇ becomes equal. That is, the spread angle ⁇ of the second slit light 72 is 5% (2.4 degrees) of the spread angle ⁇ .
- the slit light projecting unit 20 emits a laser beam from the laser diode 21 in response to a command from the processor 40, and emits the first slit light 71 in the first direction and Then, the second slit light 72 is emitted from the window 29 provided below the imaging lens 31 of the main body case 10 in the second direction.
- the power of the first slit light 71 divided by the transparent flat plate 24 is 95% of the power that is also output by the laser diode 21.
- the power of the second slit light 72 is as small as about 5%.
- the power of the first slit light 71 with a spread angle force of 8 degrees is about 20 WZ per unit angle
- the second slit light with a spread angle of 2.4 degrees Light per unit angle of light 72
- the power is also about 21 / z WZ, and the power of the two slit lights is almost the same.
- the illuminance by the first slit light 71 and the second slit light 72 is about 1260 lux. Therefore, even in a place of 500-1000 lux, which is the general indoor brightness, there is a sufficient difference in luminance between the trajectory of the slit light and the document P. Can be extracted.
- FIG. 23 is a block diagram showing an electrical configuration of the imaging device 1B.
- the processor 40 mounted on the imaging device 1B includes a CPU 41, a ROM 42, and a RAM 43.
- the CPU 41 executes various processes according to the program stored in the ROM 42 while using the RAM 43.
- the processing executed by the CPU 41 includes detection of the pressing operation of the release button 52, capture of the image data from the CCD image sensor 32, writing of the image data to the memory card 55, detection of the state of the mode switch 59, detection of the slit light, and the like. This includes switching of the emission of slit light by the light emitting unit 20, detection of a pressing operation of the operation button 57 and the confirmation button 58, and the like.
- the ROM 42 includes a camera control program 421, a difference extraction program 422, a triangulation calculation program 423, an original attitude calculation program 424, a plane conversion program 425, an in-plane displacement rotation recognition program 426, a rectangular area cutting program 427, and an editing program. 428 are stored.
- the camera control program 421 processes the flowchart shown in FIG. 24 (details will be described later).
- the difference extraction program 422 is a program for generating image data obtained by extracting the locus of the slit light from the image of the document P on which the slit light is projected.
- the triangulation calculation program 423 is a program for calculating the three-dimensional spatial position of each locus of the slit light locus extracted by the difference extraction program 422 with respect to each pixel.
- the document attitude calculation program 424 is a program for estimating and obtaining the three-dimensional shape of the document P from the three-dimensional spatial positions of the first slit light trajectory 71a and the second slit light trajectory 72a.
- the plane conversion program 425 converts the image data stored in the slit light non-image storage section 432 into an image that also captures the frontal force of the document P based on the position and orientation of the document P. It is a program to do.
- the in-plane displacement rotation recognition program 426 is a program for detecting the displacement and the rotation angle of the document P in the image data.
- the rectangular area cutout program 427 is a program for cutting out image data of an area where the document P is captured in the image data.
- the editing program 428 is a program that generates an edited image in response to the operation of the operation button 57.
- a slit light image storage unit 431 a slit light non-image storage unit 432, a difference image storage unit 433, a triangulation calculation result storage unit 434, a plane conversion result storage unit 435, an in-plane displacement rotation
- the recognition result storage section 436, the rectangular area cutout result image storage section 437, the working area 438, and the editing parameter storage section 439 are allocated as storage areas.
- the image data of the image with slit light and the image data of the image without slit light from the CCD image sensor 32 are stored in the image storage unit with slit light 431 and the image storage unit with slit light 432, respectively.
- the triangulation calculation result storage unit 434 stores the result of calculating the position of each point of the image with slit light.
- the document orientation calculation result storage unit 435 stores the image data converted by the plane conversion program 425.
- the in-plane displacement rotation recognition result storage unit 436 stores a result of an in-plane displacement rotation recognition process described later.
- the result of the rectangular area cutting process described later is stored in the rectangular area cutting result image storage unit 437.
- Working area 438 stores data temporarily used for calculation in CPU 41.
- the editing parameter storage unit 439 regarding the edited image edited by the editing program 428, the operation status of each button 57a-57h operated by the user at the time of editing is stored as an editing parameter.
- FIG. 24 is a flowchart showing a processing procedure in the processor 40 of the imaging device 1B.
- FIG. 25 (a) shows a slit light image obtained by illuminating the original P with slit light and capturing the state of the slit light.
- step S3011 the light emission command to the laser diode 21 of the slit light projecting unit 20 is stopped, and the image data is output from the CCD image sensor 32 in a state where the first slit light 71 and the second slit light 72 are not emitted. Is obtained, and the image data is stored in the card memory 55. After that, the process ends.
- a camera control process for acquiring an image with slit light and an image without slit light is performed (S2102). Specifically, the light emission of the laser diode 21 is commanded to the slit light projecting unit 20, and the first slit light 71 and the second slit light 72 are emitted. As shown in a), a slit image including the document P on which the first and second slit lights 71 and 72 are projected is obtained. The acquired image data is stored in the slit light image storage unit 431 of the RAM 43.
- the edited image edited by the editing process (S2107) is displayed on LCD 51 (S2108). If the user looks at the edited image and determines that the edited image is to be further edited, the confirm button 58 is not pressed down (S2105: No), and the processing from S2106 is repeated.
- the image displayed on the user's LCD 51 is stored in the card memory 55. Is performed (S2109).
- the resolution of the image is stored at a higher resolution than the resolution set before storage. Therefore, when outputting the stored image, a clear image can be output.
- the plane image generated in the three-dimensional measurement and plane conversion processing (S2103) may be deleted. This is because by erasing the planar image, the memory can be used effectively.
- the operation status of each button 57a-57d temporarily stored in the working area is related to the edited image stored in the memory card 55. It is stored in the edit parameter storage unit 439 as an edit parameter (S3010).
- the editing parameters for example, when an image captured in the same state as the edited image edited this time is desired to be edited in the same manner as this time, the editing parameters stored in the editing parameter storage unit 439 are stored in the editing parameters. If editing based on this, it is possible to perform the same editing as this time without newly operating each of the buttons 57a-57h. Therefore, operability can be improved.
- S3010 is completed, this process ends.
- FIG. 26 is a flowchart showing three-dimensional measurement and plane conversion processing.
- FIG. 27 (a) is a diagram schematically illustrating data stored in the difference image storage unit
- FIG. 27 (b) is a diagram schematically illustrating data stored in the triangulation calculation result storage unit. It is.
- FIG. 28 (a) -FIG. 28 (c) are diagrams showing image data subjected to image processing.
- a difference extraction process is executed (S2801).
- Difference extraction processing 23 the position information of the trajectory 71a of the first slit light and the trajectory 72a of the second slit light appearing in the image with slit light is obtained by the difference extraction program 422 shown in FIG.
- the image data of the difference between the image with slit light and the image without slit light ie, the locus 71a of the first slit light and the locus 72a of the second slit light projected on the document P
- the image data of the difference between the image with slit light and the image without slit light is Is generated.
- the lower half of the generated image data (height position ccdy between “800” and “1599”), every 20 pixels between width direction ccdx between “0” and “1199”, The position where the luminance level is maximum for one row of pixels in the height direction is determined. As shown in Fig.
- the slit light image position data 433a is stored in the difference image storage unit 433 shown in FIG.
- the maximum position of the luminance level is the position of the maximum value of the approximate curve connecting the luminance levels of the pixels in the height direction. If the luminance level at the maximum position is not higher than the predetermined level, it is determined that there is no locus 71a of the first slit light, and is not stored as the slit light image position data 433a. Next, the maximum value of the index idx is set to the number N of extracted pixels, which is the total number of pixels extracted as the position information of the locus 71a of the first slit light.
- the obtained position is stored in the difference image storage unit 433 in the same format as the slit light image position data 433a as position information of the second slit light locus 72a.
- the position information of the locus 71a of the first slit light if the luminance level at the maximum position is not higher than a predetermined level, and the locus 72a of the second slit light is absent, it is determined that there is no locus 72a. Not stored in 433.
- triangulation calculation processing is executed (S2802).
- the triangulation calculation processing for each point determined by the slit light image position data 433a stored in the difference image storage unit 433, the three-dimensional space position for each index idx is determined by the triangulation calculation program 423 shown in FIG. Desired.
- the calculation result is obtained by plotting the data of the X, Y, and Z values, which are the three-dimensional coordinate values for each index idx, as slit light three-dimensional position data 434a.
- the result is stored in the triangulation calculation result storage unit 434 shown in FIG.
- a document posture calculation process is executed (S2803).
- the original posture calculation program 424 shown in FIG. 23 uses the slit light three-dimensional position data 434a stored in the triangulation calculation result storage unit 434, and the three-dimensional spatial position (0, 0, L), slope ⁇ and curvature ⁇ are required.
- a plane conversion process is executed (S2804).
- the plane light conversion program 425 shown in FIG. 23 converts the slit light non-image stored in the slit light non-image storage section 432 into a plane image in which the document P is observed from a substantially orthogonal direction (for example, FIG. 28).
- the document P is converted into an image having a rectangular shape.
- the obtained plane image is stored in the plane conversion result image storage unit 435 shown in FIG.
- in-plane displacement rotation recognition processing is executed (S2805).
- the in-plane displacement rotation recognition program 426 shown in FIG. 23 identifies the image area of the document P in the plane image, and as shown in FIG.
- the width direction displacement DH and the height direction displacement DV of the center position of the image area of the document P with respect to the center position, and the rotation angle DROT of a predetermined side of the image area of the document P in the planar image are obtained.
- the obtained result is stored in the in-plane displacement rotation recognition result storage unit 436 shown in FIG.
- a rectangular area cutting process is executed (S2806).
- the three-dimensional measurement and the plane conversion processing ends.
- FIG. 29 is a flowchart showing the editing processing.
- the operation button 57 each button It is determined whether or not the buttons 57a-57h) have been pressed (S3061). That is, in the process of S2106 in FIG. 24, it is determined whether or not the operation state of the operation button 57 is stored in the working area 438. As a result, when the operation button 57 is operated (S3061: Yes), the displacement amount, the rotation angle, and the magnification are measured based on the output of the operation button 57 (S3062).
- the plane image displayed on the LCD 51 (the image stored in the rectangular area cutout image storage unit 437) is displaced by the measured displacement amount, rotated by the measured rotation angle, or measured by the magnification.
- An edited image is generated (S3063). Thereafter, the processing in FIG. 29 ends.
- the operation button 57 when the operation button 57 is not operated (S3061: No), it is determined whether or not the editing parameter selection button 65 is pressed (S3064). If the edit parameter selection button 65 is pressed (S3064: Yes), the plane image (rectangular) displayed on the LCD 51 is stored in the edit parameter storage unit 439 based on the latest edit parameter among the edit parameters. The edited image is generated by displacing, rotating, or setting the magnification to a predetermined magnification (the image stored in the area cutout result image storage unit 437) (S3065). As a result, the user can generate the same edited image as the previous one by a simple operation of pressing down the edit parameter selection button 65 without operating each of the buttons 57a-57h.
- the latest editing parameter is changed to the immediately preceding editing parameter (S3066). That is, every time the edit parameter selection button 65 is depressed, a plane image displayed on the LCD 51 and edited based on the immediately preceding edit parameter is edited. Thereafter, the processing in FIG. 29 ends.
- FIG. 30 (a) shows the force and FIG. 30 (h) is a diagram for explaining the operation method of the operation button 57.
- the state of the original P before editing is indicated by a two-dot chain line, and the state after editing is indicated by a solid line.
- the user presses the rightward movement button 57e until the original P indicated by the two-dot chain line moves to the position indicated by the solid line.
- the user presses the reduction button 57f until the original P indicated by the two-dot chain line is reduced to the size indicated by the solid line.
- the user presses down button 57g until document P indicated by the two-dot chain line moves to the position indicated by the solid line.
- the user presses the enlargement button 57h until the original P indicated by the two-dot chain line is enlarged to the size indicated by the solid line.
- FIGS. 31 (a) to 31 (d) are diagrams for explaining a method of operating the operation button 57 when only a specific portion (graph portion) of the document P is edited into an enlarged image. .
- FIG. 31 (a) it is assumed that the document P is displayed on the LCD 51 in a state where the document P is rotating rightward.
- the user presses the left rotation button 57a while checking the document P displayed on the LCD 51.
- the displayed document P is rotated to the left according to the number of times the left rotation button 57a is pressed.
- the user stops the operation of the left rotation button 57a when judging that the document P has turned to the front as shown in FIG. 31 (b).
- the planar image displayed on the LCD 51 can be edited by operating the operation button 57.
- an image intended by the user can be obtained.
- the imaging apparatus 1B of the present embodiment even if the document P is placed at a position where the central force of the image is shifted due to erroneous recognition in the in-plane displacement rotation recognition processing (S2805 in FIG. 26). , The user It can be corrected.
- the user slides the mode switching switch 59 to the "corrected imaging mode", sets the size input unit 56 according to the size of the original P to be imaged, and images the original P.
- the curved document P is imaged obliquely, it is possible to obtain an image in which the flat document P can be viewed from the front in the erect direction and only the portion where the document P is photographed is cut out.
- the posture of the document P is obtained by analyzing only the positions of the first and second slit lights 71 and 72 in the image. Therefore, the posture of the document P with high detection accuracy can be easily obtained without performing complicated image analysis. Therefore, the time required for the image analysis processing by the processor 40 of the imaging device 1B can be reduced, or the processor 40 can be simplified.
- the imaging device 100 according to the third embodiment has the same configuration as the imaging device 1B according to the second embodiment except for the features described below.
- characteristic portions of the imaging device 100 according to the third embodiment with respect to the imaging device 1B of the second embodiment will be sequentially described.
- the imaging device 1B of the second embodiment described above is configured to edit the planar image displayed on the LCD 51 in response to the operation of the operation button 57 in the editing process (S2107) in FIG. .
- the imaging device 100 of the third embodiment has a built-in detection means for detecting the amount of movement of the main body case 10 in a three-dimensional space, instead of the operation button 57.
- the planar image displayed on the LCD 51 is edited according to the movement amount detected by the detection means.
- the same components as those of the imaging device 1B of the second embodiment are denoted by the same reference numerals, and description thereof will be omitted.
- FIG. 32 is a diagram showing three-dimensional coordinates set in the imaging device 100 according to the third embodiment.
- the origin O is set at the center position inside the main body case 10 and the lateral direction of the imaging device 100 is Three-dimensional coordinates are set, with the X axis (positive in the right direction), the Y axis in the height direction (positive in the upward direction), and the Z axis in the thickness direction (positive in the front direction).
- FIG. 33 is a block diagram showing an electrical configuration of an imaging device 100 according to the third embodiment.
- the imaging device 100 of the third embodiment has an origin sensor 61, an X-axis sensor 62, a Y-axis sensor 63, and a Z-axis sensor 64, instead of the operation buttons 57 provided in the imaging device 1B of the second embodiment. V These sensors are connected to the CPU 41 via signal lines.
- Each of the sensors 61-64 is an acceleration sensor that outputs a signal representing the magnitude of acceleration by changing a duty ratio in one cycle of a waveform.
- the origin sensor 61 is a three-axis sensor that is installed at the coordinate origin O and is capable of detecting accelerations in the X-axis direction, the Y-axis direction, and the Z-axis direction.
- the X-axis sensor 62 is a sensor for one-axis direction that is installed on the X-axis and can detect acceleration only in the X-axis direction.
- the Y-axis sensor 63 is a one-axis sensor installed on the Y-axis and capable of detecting acceleration only in the Y-axis direction.
- the Z-axis sensor 64 is a single-axis sensor that is installed on the Z-axis and can detect acceleration only in the Z-axis direction.
- the displacement amount and the rotation angle with respect to each coordinate of the main body case are represented by a signal representing the magnitude of the acceleration output from each of the sensors 61-64 and, although illustration and detailed description are omitted, the signal is output. It is calculated on the basis of the count value of the clock signal counted at the timing of the execution. That is, the displacement and the rotation angle are even calculated by integrating the acceleration.
- FIGS. 34 (a) to 34 (f) show a case where a document P indicated by a two-dot chain line is edited like a document P indicated by a solid line.
- the user When the user wants to edit as shown in FIG. 34 (a), the user operates as shown by the arrow (a) in FIG. 35 until the original P indicated by the two-dot chain line is rotated to the position indicated by the solid line. Then, the imaging device 100 is rotated clockwise around the Y axis. Conversely, when the image capturing apparatus 100 is rotated counterclockwise about the Y axis by a predetermined angle, the document P indicated by the two-dot chain line is rotated counterclockwise by a predetermined angle.
- FIG. 36 is a flowchart showing an editing process corresponding to the editing process (S2107) in FIG.
- this process first, it is determined whether or not there is an output from the origin sensor 61 (S2291). That is, in the second embodiment, the operation status of each button 57a-57h in S2106 of FIG. 24 is temporarily stored in the working area 438. In the third embodiment, the operation status of each button 57a-57h is Instead, the moving state of the main body case is temporarily stored in the parking area 438, and the moving state of the main body case is stored in the working area 438, and it is determined whether or not the moving state of the main body case is stored.
- the displacement amount and magnification of the imaging device 100 are measured based on the output of the origin sensor force (S2292). That is, the output from the origin sensor 61
- the applied force means that the imaging device 100 has moved in at least one direction in the XYZ axis directions instead of the rotational movement. In this case, since the origin sensor 61 can detect the acceleration in each of the XYZ axis directions, if there is an output from the origin sensor 61, the output from the other sensors 62-64 is reserved. .
- planar image stored in the rectangular area clipping result image storage section 437 is moved by the measured displacement amount or set to the measured magnification, thereby generating an edited image. (S2293). After that, the process ends.
- S2296 it is determined whether or not the output from the Y-axis sensor 63 has been received. When there is an output from the Y-axis sensor 63 (S2296: Yes), the displacement amount in the Y direction is measured based on the output from the Y-axis sensor 63 (S2297). Next, the process proceeds to S2298. If there is no output from the Y-axis sensor 63 (S2296: No), the process proceeds by S2298.
- the imaging device 100 when the imaging device 100 is rotated as shown by the arrow (a) in FIG. 35, the outputs from the origin sensor 61 and the Y-axis sensor 63 are different.
- the acceleration force in the Z-axis direction is detected by the 3 ⁇ 4-axis sensor 64.
- the displacement in the X-axis direction is measured by the output of the X-axis sensor 62
- the displacement in the Z-axis direction is measured by the output of the Z-axis sensor 64.
- the rotation angle about the Y axis is measured based on the displacement amount in the X axis direction and the displacement amount in the Z axis direction.
- the imaging device 100 is set to the X axis, Z The same processing is performed when the object is rotated around the axis, and the rotation angle about the X axis and the rotation angle about the Y axis are measured.
- the imaging device 100 of the second embodiment by moving and rotating the imaging device 100, the image displayed on the LCD 51 can be edited according to the operation. Even a user who is unfamiliar with button operations can generate an edited image intuitively. Further, unlike the imaging device 1B of the second embodiment, there is no need to provide an operation button 57 on the outer surface of the main body case 10, so that there is no need to secure a space for disposing the operation button 57. Restrictions can be reduced.
- the original posture calculation process (S2803) in the flowchart of FIG. 26 is regarded as a three-dimensional shape calculation unit.
- the plane conversion processing (S2804) of the flowchart in FIG. 26 is regarded as plane conversion means.
- the editing process (S2107) in FIG. 24 is positioned as a first change unit.
- the process of S1065 in FIG. 29 is positioned as the selection means.
- the processing in S2805 in FIG. 26 is regarded as an in-plane position detection means.
- the process of S2806 in FIG. 26 is regarded as a second changing unit.
- the processing of S2805 in FIG. 26 is also positioned as in-plane angle detection means.
- the editing process (S2107 in FIG. 24), as the image to be edited, an image in which only the portion of the document P is cut out by the rectangular region cutting process (S2806 in FIG. 26) is edited.
- the image edited by the editing processing may be an image plane-converted by the plane conversion processing (S2804 in FIG. 26).
- the in-plane displacement rotation recognition processing (S2805 in FIG. 26) and the rectangular area cutting processing (S2806 in FIG. 26) can be omitted, and the processing load for performing such processing can be reduced.
- the operation button 57 is used to input the rotation direction of the document P, and the two buttons are the left rotation button 57a and the right rotation button 57c.
- the case where the button is provided has been described.
- the buttons for inputting the rotation direction of the document P while applying force are not limited to the two powerful buttons.
- a button that rotates around the Y axis and a button that rotates around the X axis are provided. Is also good. In the case where the image is dark, the number of editable patterns increases, and an edited image intended by the user can be generated.
- the original P is moved in the same direction as the moved or rotated direction. It was configured to move and rotate.
- the imaging device may be configured to move and rotate the document P in a direction opposite to the moving direction and the rotating direction of the main body case 10 while applying force.
- the slit light projecting unit 20 is configured to emit two rows of slit light, the first slit light 71 and the second slit light 72.
- the slit light to be emitted is not limited to the two rows of the slit light emitted by the imaging device, and the imaging device may be configured so that three or more rows are emitted.
- the third slit light similar to the second slit light 72
- the slit light projecting unit 20 may be configured to project the slit light above the second slit light 72 on the document P.
- the position of each point on the locus of the slit light It is also possible to estimate the vertical curved shape, whereby the image without slits can be corrected and an image that is easier to see can be obtained.
- a laser diode 21 that emits a red laser beam is used as a light source.
- a light source various types of light sources that output a light beam, including a surface emitting laser, an LED, and an EL element, can be used.
- a transparent flat plate formed on one surface with a diffraction grating for diffracting a predetermined ratio of the power of the incident laser beam in a predetermined direction may be used.
- the first-order laser beam diffracted by the diffraction grating can be used as the second slit light 72, and the zero-order laser beam transmitted as it is can be used as the first slit light 71.
- the pattern light emitted from the slit light projecting unit 20 can be of various types, such as a striped light pattern with a certain width, in addition to a fine line sharply narrowed in a direction perpendicular to the longitudinal direction. It can be light.
- the positional relationship between the first slit light 71 and the second slit light 72 may be reversed in the first direction, that is, the second slit light 72 is formed on the lower side when viewed from the imaging device 1B.
- Each optical element in the slit light projecting unit may be provided so that the first slit is formed in the direction of the arrow.
- the imaging device is configured to capture an image with slit light and an image without slit light using the imaging lens 31 and the CCD image sensor 32.
- an imaging lens for capturing an image with slit light and a CCD image sensor may be separately added to the imaging device.
- the input means is constituted by an operation element capable of inputting at least one of the moving amount and the magnification of the plane image displayed on the display means as a display mode change parameter. Good.
- the input unit displays at least one of the moving amount and the magnification of the planar image. Since it is composed of operators that can be input as the mode change parameters, the user can easily input the display mode change parameters by operating the operators as the operation of inputting the display mode parameters. be able to.
- the input means may be constituted by a detecting means for detecting the amount of movement of the imaging device body in the three-dimensional space as a display mode change parameter.
- the first changing unit associates the moving amount of the imaging device main body in the three-dimensional space detected by the detecting unit with at least one of the moving amount of the planar image displayed on the display unit and the magnification, Change the display mode of the plane image.
- the input unit is configured by the detection unit that detects the amount of movement of the imaging device body in the three-dimensional space as the display mode change parameter, the user can set the display mode parameter.
- the imaging device main body may be moved in a three-dimensional space. Therefore, the input of the display mode change parameter can be performed by a simple operation.
- the input means is composed of an operation element
- a space for providing the operation element on the outer surface of the imaging apparatus main body must be secured, which is a design restriction factor.
- the plane conversion means may be configured to convert the plane image into an image having a lower resolution than the captured image.
- the plane conversion unit converts the plane image into a lower resolution image than the captured image, the number of pixels of the plane image is reduced, and the conversion by the plane conversion unit and the use of the plane image are used. The amount of calculation in the processing can be reduced.
- the imaging means may further include a storage means for storing the image changed by the first changing means.
- the image stored in the storage means is stored at a higher resolution than the plane image.
- the imaging device includes a display mode change parameter storage unit that stores a display mode change parameter used to change the display mode of the planar image displayed on the display unit.
- the first changing unit changes the display mode of another planar image displayed on the display unit based on the display mode changing parameter selected by the selecting unit.
- the display mode of the plane image displayed on the display unit is to be changed in the same manner as the previously changed plane image
- the user is stored in the display mode storage unit.
- the display mode parameter used to change the previously changed plane image is changed based on the selected display mode parameter. Can be changed.
- the imaging apparatus before displaying the planar image on the display means, identifies a target object area where the target object is captured from the planar image, and sets the target in the planar image. Based on the in-plane position detecting means for obtaining the position of the object, the position obtained by the in-plane position detecting means, and the shape information, the plane image is arranged at the approximate center position of the target object viewed from a predetermined direction. And a second changing means for changing the image to the changed image. In this case, the plane image displayed on the display means is an image changed by the second changing means.
- the planar image displayed on the display means is an image in which the target object viewed from a predetermined direction is arranged at a substantially central position of the planar image. It is possible to omit the process of editing so as to be arranged substantially at the center. In general, many users intend that the target object is located substantially at the center of the planar image, and by automatically generating the image intended by such many users, The versatility of the device can be improved. [0240] In one embodiment of the present invention, before displaying the plane image on the display means, the imaging unit identifies a target object region where the target object is captured from the plane image, and determines a predetermined object area within the plane image plane.
- the second changing unit is configured to set the plane image based on the angle obtained by the in-plane angle detecting unit, the position obtained by the in-plane position detecting unit, and the shape information so that the target object viewed from the predetermined direction is substantially at the center position. And an image in which a predetermined side of the target object is oriented in a predetermined direction in the planar image.
- the planar image displayed on the display means is such that the target object viewed from a predetermined direction is arranged at a substantially central position, and furthermore, a predetermined side of the target object is positioned at a predetermined position in the planar image.
- the image is oriented in the direction of. Therefore, the above-mentioned effect that the versatility of the device can be improved can be further enhanced.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
- Image Input (AREA)
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/374,252 US7627196B2 (en) | 2003-09-18 | 2006-03-14 | Image processing device and image capturing device |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2003-326614 | 2003-09-18 | ||
JP2003326614A JP2005092629A (ja) | 2003-09-18 | 2003-09-18 | 画像処理装置、及び、撮像装置 |
JP2003-416343 | 2003-12-15 | ||
JP2003416343A JP2005174184A (ja) | 2003-12-15 | 2003-12-15 | 撮像装置 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/374,252 Continuation-In-Part US7627196B2 (en) | 2003-09-18 | 2006-03-14 | Image processing device and image capturing device |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2005029408A1 true WO2005029408A1 (ja) | 2005-03-31 |
Family
ID=34380326
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2004/013516 WO2005029408A1 (ja) | 2003-09-18 | 2004-09-16 | 画像処理装置、及び、撮像装置 |
Country Status (2)
Country | Link |
---|---|
US (1) | US7627196B2 (ja) |
WO (1) | WO2005029408A1 (ja) |
Families Citing this family (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8010898B2 (en) * | 2006-11-30 | 2011-08-30 | Oce-Technologies B.V. | Process of specifying print time fold method, location, and orientation |
JP5109803B2 (ja) * | 2007-06-06 | 2012-12-26 | ソニー株式会社 | 画像処理装置、画像処理方法及び画像処理プログラム |
US8608321B2 (en) | 2008-06-17 | 2013-12-17 | The Invention Science Fund I, Llc | Systems and methods for projecting in response to conformation |
US8936367B2 (en) | 2008-06-17 | 2015-01-20 | The Invention Science Fund I, Llc | Systems and methods associated with projecting in response to conformation |
US8376558B2 (en) | 2008-06-17 | 2013-02-19 | The Invention Science Fund I, Llc | Systems and methods for projecting in response to position change of a projection surface |
US8384005B2 (en) | 2008-06-17 | 2013-02-26 | The Invention Science Fund I, Llc | Systems and methods for selectively projecting information in response to at least one specified motion associated with pressure applied to at least one projection surface |
US8267526B2 (en) | 2008-06-17 | 2012-09-18 | The Invention Science Fund I, Llc | Methods associated with receiving and transmitting information related to projection |
US8641203B2 (en) | 2008-06-17 | 2014-02-04 | The Invention Science Fund I, Llc | Methods and systems for receiving and transmitting signals between server and projector apparatuses |
US20090310038A1 (en) | 2008-06-17 | 2009-12-17 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Projection in response to position |
US8733952B2 (en) | 2008-06-17 | 2014-05-27 | The Invention Science Fund I, Llc | Methods and systems for coordinated use of two or more user responsive projectors |
US8262236B2 (en) | 2008-06-17 | 2012-09-11 | The Invention Science Fund I, Llc | Systems and methods for transmitting information associated with change of a projection surface |
US8955984B2 (en) | 2008-06-17 | 2015-02-17 | The Invention Science Fund I, Llc | Projection associated methods and systems |
US20090309826A1 (en) | 2008-06-17 | 2009-12-17 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Systems and devices |
US8308304B2 (en) * | 2008-06-17 | 2012-11-13 | The Invention Science Fund I, Llc | Systems associated with receiving and transmitting information related to projection |
US8944608B2 (en) | 2008-06-17 | 2015-02-03 | The Invention Science Fund I, Llc | Systems and methods associated with projecting in response to conformation |
US8723787B2 (en) | 2008-06-17 | 2014-05-13 | The Invention Science Fund I, Llc | Methods and systems related to an image capture projection surface |
US8400564B2 (en) * | 2010-05-03 | 2013-03-19 | Microsoft Corporation | Image capture |
JP5159986B2 (ja) * | 2010-08-06 | 2013-03-13 | パナソニック株式会社 | 撮像装置および撮像方法 |
JP5644461B2 (ja) * | 2010-12-14 | 2014-12-24 | 富士ゼロックス株式会社 | 画像処理装置及びプログラム |
US20150254043A1 (en) * | 2014-03-04 | 2015-09-10 | Sung Jae Hwang | Display property determination |
CN105791660A (zh) * | 2014-12-22 | 2016-07-20 | 中兴通讯股份有限公司 | 一种纠正被摄物体拍摄倾斜的方法、装置及移动终端 |
KR101828376B1 (ko) * | 2016-05-13 | 2018-02-13 | (주)칼리온 | 휴대용 3차원 문서 스캐닝 장치 및 방법 |
US10325166B2 (en) * | 2017-04-13 | 2019-06-18 | Here Global B.V. | Method, apparatus, and system for a parametric representation of signs |
US10650631B2 (en) | 2017-07-28 | 2020-05-12 | Hand Held Products, Inc. | Systems and methods for processing a distorted image |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH07245700A (ja) * | 1994-03-04 | 1995-09-19 | Minolta Co Ltd | デジタル複写機 |
JPH09138869A (ja) * | 1995-11-15 | 1997-05-27 | Meidensha Corp | 画像整形方法 |
JPH09289610A (ja) * | 1996-04-23 | 1997-11-04 | Minolta Co Ltd | デジタルカメラ |
JP2002165083A (ja) * | 2000-11-24 | 2002-06-07 | Hitachi Ltd | 画像処理方法及び、それを用いた非接触画像入力装置 |
JP2003008841A (ja) * | 2001-06-20 | 2003-01-10 | Ricoh Co Ltd | 画像入力装置 |
JP2003234890A (ja) * | 2002-02-08 | 2003-08-22 | Ricoh Co Ltd | 画像補正装置、プログラム、記憶媒体及び画像補正方法 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6640004B2 (en) * | 1995-07-28 | 2003-10-28 | Canon Kabushiki Kaisha | Image sensing and image processing apparatuses |
US6449004B1 (en) * | 1996-04-23 | 2002-09-10 | Minolta Co., Ltd. | Electronic camera with oblique view correction |
GB0104664D0 (en) | 2001-02-23 | 2001-04-11 | Hewlett Packard Co | Improvements relating to document capture |
JP4480488B2 (ja) * | 2003-08-28 | 2010-06-16 | 富士通株式会社 | 計測装置、コンピュータ数値制御装置及びプログラム |
JP2005148813A (ja) * | 2003-11-11 | 2005-06-09 | Brother Ind Ltd | 3次元形状検出装置、撮像装置、及び、3次元形状検出プログラム |
-
2004
- 2004-09-16 WO PCT/JP2004/013516 patent/WO2005029408A1/ja active Application Filing
-
2006
- 2006-03-14 US US11/374,252 patent/US7627196B2/en not_active Expired - Fee Related
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH07245700A (ja) * | 1994-03-04 | 1995-09-19 | Minolta Co Ltd | デジタル複写機 |
JPH09138869A (ja) * | 1995-11-15 | 1997-05-27 | Meidensha Corp | 画像整形方法 |
JPH09289610A (ja) * | 1996-04-23 | 1997-11-04 | Minolta Co Ltd | デジタルカメラ |
JP2002165083A (ja) * | 2000-11-24 | 2002-06-07 | Hitachi Ltd | 画像処理方法及び、それを用いた非接触画像入力装置 |
JP2003008841A (ja) * | 2001-06-20 | 2003-01-10 | Ricoh Co Ltd | 画像入力装置 |
JP2003234890A (ja) * | 2002-02-08 | 2003-08-22 | Ricoh Co Ltd | 画像補正装置、プログラム、記憶媒体及び画像補正方法 |
Also Published As
Publication number | Publication date |
---|---|
US20060164526A1 (en) | 2006-07-27 |
US7627196B2 (en) | 2009-12-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2005029408A1 (ja) | 画像処理装置、及び、撮像装置 | |
US8123361B2 (en) | Dual-projection projector and method for projecting images on a plurality of planes | |
CN102401646B (zh) | 基于结构光测量的方法 | |
JP6506507B2 (ja) | 測定装置およびその制御方法 | |
EP3468163B1 (en) | Intelligent internet high-definition scanner with laser correction | |
JP2007271530A (ja) | 3次元形状検出装置及び3次元形状検出方法 | |
JPH1183459A (ja) | 凹凸面情報検出装置 | |
JP2006279546A (ja) | 電子カメラ、画像処理プログラム、および画像処理方法 | |
WO2006112297A1 (ja) | 3次元形状測定装置 | |
US20060192082A1 (en) | Three-dimensional shape detecting device, image capturing device, and three-dimensional shape detecting program | |
US11016366B2 (en) | Printer-equipped imaging apparatus, operation method of printer-equipped imaging apparatus, program, and recording medium | |
JP2008292570A (ja) | 投射型映像表示装置 | |
JPH09196632A (ja) | 3次元計測のための分光装置 | |
WO2007119396A1 (ja) | 三次元情報測定装置 | |
WO2005008174A1 (ja) | 3次元形状検出装置、撮像装置、及び、3次元形状検出方法 | |
JP2000004383A (ja) | 多方向画像取込装置 | |
JP4221808B2 (ja) | 3次元データ入力方法及び装置 | |
WO2021192706A1 (ja) | 照明制御装置、撮像装置、照明制御方法及びプログラム | |
US7372580B2 (en) | Three-dimensional shape detecting device, three-dimensional shape detecting system, and three-dimensional shape detecting program | |
JP2005092629A (ja) | 画像処理装置、及び、撮像装置 | |
JP2005092629A5 (ja) | ||
JP4360145B2 (ja) | 3次元形状検出装置、撮像装置、及び、3次元形状検出方法 | |
WO2005080915A1 (ja) | 3次元形状検出装置および撮像装置 | |
JP2005148813A5 (ja) | ||
JP2004132703A (ja) | 測定システム、演算方法および演算プログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BW BY BZ CA CH CN CO CR CU CZ DK DM DZ EC EE EG ES FI GB GD GE GM HR HU ID IL IN IS KE KG KP KR LC LK LR LS LT LU LV MA MD MG MN MW MX MZ NA NI NO NZ OM PG PL PT RO RU SC SD SE SG SK SL SY TM TN TR TT TZ UA UG US UZ VC YU ZA ZM |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): BW GH GM KE LS MW MZ NA SD SZ TZ UG ZM ZW AM AZ BY KG MD RU TJ TM AT BE BG CH CY DE DK EE ES FI FR GB GR HU IE IT MC NL PL PT RO SE SI SK TR BF CF CG CI CM GA GN GQ GW ML MR SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 11374252 Country of ref document: US |
|
WWP | Wipo information: published in national office |
Ref document number: 11374252 Country of ref document: US |
|
122 | Ep: pct application non-entry in european phase |