JP2005293290A5 - - Google Patents

Description

Image input / output device

  The present invention relates to an image input / output device capable of imaging and projecting, and in particular, can project a comparison result obtained by comparing imaging data of a subject obtained by imaging and reference information about the subject onto the subject. The present invention relates to a possible image input / output device.

2. Description of the Related Art Conventionally, a three-dimensional measurement apparatus that projects a light pattern onto a subject, images the light pattern projected onto the subject, and performs predetermined computation on the image signal to obtain three-dimensional information about the subject. Are known. For example, in Patent Document 1, a coded multi-slit light pattern is projected onto a measurement object, and the coded multi-slit light pattern imaging device projected onto the measurement object is imaged, and an image signal obtained by the imaging is obtained. A three-dimensional measuring device that calculates the three-dimensional position of the contour of the object to be measured is described.
Japanese Patent Publication No. 8-20232

  However, a general image input / output device including a three-dimensional measuring device as described in Patent Document 1 and a projection device and an imaging device only provides information on the appearance of the object to be measured. There was a problem that its use was limited.

  The present invention has been made to solve the above-described problems, and compares a subject such as an object to be measured obtained from imaging data with reference information about the subject, and provides information related to the comparison result. An object of the present invention is to provide an image input / output device capable of projecting onto a subject.

  In order to achieve this object, the image input / output device according to claim 1 includes: a light source unit that emits light; and a spatial modulator unit that performs spatial modulation on the light emitted from the light source unit and outputs image signal light. A projection optical system that projects the image signal light output by the spatial modulation unit in a projection direction; an imaging unit that captures an image of a subject existing in a space including the projection direction; and Reference information acquisition means for acquiring reference information on the subject to be imaged by the imaging means, and image signal light based on the reference information acquired by the reference information acquisition means is output by the spatial modulation means and is output. And a reference information projecting unit that projects the image signal light onto the subject or a surface capable of projecting the image in the projection direction by the projection optical system.

  According to the image input / output device of claim 1, when the light emitted from the light source means is spatially modulated by the spatial modulation means and output as image signal light, the image signal light is Projection is performed in the projection direction by the projection optical system. On the other hand, the subject existing in the space including the projection direction is imaged by the imaging means, and the imaging data is acquired. Here, when the reference information regarding the subject to be imaged by the imaging device is acquired by the reference information acquisition unit, the image signal light based on the reference information is output from the spatial modulation unit by the reference image projection unit, The image signal light is projected from the projection optical system onto the subject or a surface capable of projecting an image in the projection direction. As a result, an image corresponding to the reference information is projected onto the subject or the surface on which the image can be projected in the projection direction.

  The image input / output device according to claim 2 is the image input / output device according to claim 1, wherein the reference information acquired by the reference information acquisition unit and the imaging data of the subject imaged by the imaging unit are compared. The comparison result information acquisition means for acquiring the comparison result as comparison result information, and the image signal light based on the comparison result information acquired by the comparison result information acquisition means are output from the spatial modulation means and output. And comparison result information projection means for projecting the image signal light onto the subject or a surface capable of projecting the image in the projection direction by the projection optical system.

  According to the image input / output device described in claim 2, the same function as the image input / output device described in claim 1 is performed, and the reference image and the subject imaged by the imaging unit are obtained by the comparison result information acquisition unit. When the comparison result information is acquired by comparison with the imaging data, the comparison result information projection unit outputs the image signal light based on the comparison result information from the spatial modulation unit, and the image signal light is output from the projection optical system. The image is projected onto the subject or the surface on which the image can be projected in the projection direction. As a result, an image corresponding to the comparison result information is projected together with the image corresponding to the reference information onto the subject or the surface on which the image in the projection direction can be projected.

  The image input / output device according to claim 3 is a light source unit that emits light, a spatial modulation unit that performs spatial modulation on the light emitted from the light source unit and outputs image signal light, and an output from the spatial modulation unit. A projection optical system that projects the image signal light to be projected onto the projection plane, an imaging unit that captures an image of a subject existing in a space including the projection range, and acquires the imaging data; Reference information acquisition means for acquiring reference information about a subject, reference information acquired by the reference information acquisition means, and imaging data of the subject imaged by the imaging means, and the comparison result is compared with comparison result information The comparison result information acquisition means acquired as the above, and the image signal light based on the comparison result information acquired by the comparison result information acquisition means is output from the spatial modulation means, The force image signal light, and a comparison result information projection means for projecting the image projectable surface on the object or the projection direction by the projection optical system.

  According to the image input / output device according to claim 3, when the light emitted from the light source means is spatially modulated by the spatial modulation means and output as image signal light, the image signal light is Projection is performed in the projection direction by the projection optical system. On the other hand, the subject existing in the space including the projection direction is imaged by the imaging means, and the imaging data is acquired. Here, when reference information regarding the subject to be imaged by the imaging device is acquired by the reference information acquisition means, comparison result information is obtained by comparing the reference information with imaging data of the subject imaged by the imaging means. Is acquired. Then, the comparison result information projecting means outputs image signal light based on the comparison result information from the spatial modulation means, and the image signal light is projected from the projection optical system onto the subject or a surface capable of projecting an image in the projection direction. Projected. As a result, an image corresponding to the comparison result information is projected onto the subject or the surface on which the image can be projected in the projection direction.

  The image input / output device according to claim 4 is the image input / output device according to any one of claims 1 to 3, wherein the image signal light has a predetermined pattern shape, and the image signal light is the projection optical system. Based on the image data of the image signal light imaged by the imaging means, the image projection in the object or in the projection direction is performed on the subject or on a surface on which the image can be projected in the projection direction. 3D information generating means for generating 3D information of a possible surface, and the imaging based on the 3D information of the object or the surface capable of projecting an image in the projection direction generated by the 3D information generating means Imaging data of the subject acquired by the imaging means when the imaging means images the subject existing in the space including the projection direction from a direction substantially perpendicular to the surface thereof And the comparison result information acquisition means uses the imaging data corrected by the planar image correction means as the imaging data of the object, and the comparison result information Is something to get.

  According to the image input / output device according to claim 4, the image signal light having the predetermined pattern shape is operated in the same manner as the image input / output device according to any one of claims 1 to 3, and the projection optical system. When the image is projected on the subject or on a plane where the image can be projected in the projection direction, the three-dimensional information generating unit uses the image data of the image signal light imaged by the imaging unit. Three-dimensional information of a surface on which the image can be projected in the projection direction is generated. Based on the three-dimensional information generated as described above, the imaging data of the subject acquired by the imaging unit by the plane image correction unit is the subject in which the imaging unit exists in the space including the projection direction. Correction is performed so as to be equivalent to imaging data when imaging is performed from a direction substantially perpendicular to the surface. Then, the comparison result information acquisition unit acquires the comparison result information using the imaging data corrected by the planar image correction unit as the imaging data of the object.

  The image input / output device according to claim 5 is the image input / output device according to claim 4, wherein the reference information projection unit or the comparison result information projection unit is configured to project the image on the subject or in the projection direction. Spatial modulation control means is provided for controlling the spatial modulation means based on the three-dimensional information generated by the three-dimensional information generation means so that undistorted image signal light is projected.

  The image input / output device according to a sixth aspect is the image input / output device according to any one of the first to fifth aspects, wherein the reference information is model information for the subject.

  The image input / output device according to claim 7 is the image input / output device according to any one of claims 2 to 5, wherein the reference information is model information of a work process including one or more steps, The comparison result information acquisition unit compares the imaging data with the sample information corresponding to the process after the imaging unit acquires the imaging data of the subject for at least one process in the work. Result information is acquired, and the comparison result information projection unit projects the comparison result information onto the subject or a surface capable of projecting an image in the projection direction by the comparison result information acquisition unit.

  According to the image input / output device described in claim 7, the image input / output device operates in the same manner as the image input / output device according to any one of claims 2 to 5, and the operation of the work including reference information including one or more steps. If it is this information, after the imaging data of the subject for at least one process in the work is acquired by the imaging unit, the imaging data and the sample information corresponding to the process are acquired by the comparison result information acquisition unit. Are compared with each other to obtain comparison result information. Then, the comparison result information acquired in this way is projected by the comparison result projecting unit onto the subject or a surface capable of projecting an image in the projection direction. Therefore, an image corresponding to the comparison result information can be projected for each process unit constituting one operation.

  The image input / output device according to claim 8 is the image input / output device according to claim 7, wherein the comparison result information is projected onto the subject by the comparison result information projecting means together with the comparison result information for the first step. Reference information projecting means for projecting this information onto the subject or a surface capable of projecting an image in the projection direction is provided.

  The image input / output device according to claim 9 is the image input / output device according to any one of claims 2 to 8, wherein the comparison result information is correction information of the subject with respect to the reference information.

  The image input / output device according to claim 10 is the image input / output device according to any one of claims 2 to 9, wherein the comparison result information is evaluation information of the subject with respect to the reference information.

  The image input / output device according to claim 11 is the image input / output device according to any one of claims 1 to 10, wherein the projection unit includes the light source unit, the spatial modulation unit, and the projection optical system, and the imaging. And a movable means capable of moving an imaging unit provided integrally with the projection unit according to the subject.

  The image input / output device according to claim 12 is the image input / output device according to claim 11, wherein the movable means projects the image signal light onto the subject by the projection means and images the subject by the imaging means. However, the projection unit and the imaging unit can be moved so that they can be performed from the side facing the subject in the top view with respect to the direction in which the work is performed.

  13. The image input / output device according to claim 13, wherein the movable means projects the image signal light onto the subject by the projection means and images the subject by the imaging means. However, the projection unit and the imaging unit can be moved so that the operator who works on the subject can be performed from the left front.

  According to the image input / output device of the first aspect, the image corresponding to the reference information related to the subject to be imaged by the imaging device is projected onto the subject or the surface on which the image can be projected in the projection direction. After the user of the input / output device performs some work on the subject, an image corresponding to the reference information is projected on the subject, so that the user can identify the difference between the work and the reference information. It can be clearly recognized. Therefore, when the work has a problem with the reference information, there is an effect that the solution point can be surely recognized.

  According to the image input / output device described in claim 2, in addition to the effect produced by the image input / output device according to claim 1, it corresponds to the comparison result information between the subject to be imaged by the imaging device and the reference information related to the subject. And the image corresponding to the reference information are projected onto the subject or a surface where the image can be projected in the projection direction. For example, the user of the image input / output device has performed some work on the subject. Thereafter, an image corresponding to the comparison result between the work result and the reference information is projected on the subject together with the image corresponding to the reference information, so that the user can make a difference between the work and the reference information. Can be recognized more clearly. Therefore, when the work has a problem with the reference information, there is an effect that the solution point can be surely recognized.

  According to the image input / output device according to claim 3, an image corresponding to comparison result information between a subject to be imaged by the imaging device and reference information related to the subject is displayed together with the subject or the image corresponding to the reference information. Since the image is projected onto the projectable surface in the projection direction, for example, after the user of the image input / output device performs some work on the subject, the image corresponding to the comparison result between the work result and the reference information Is projected onto the subject, the user can clearly recognize the difference between the work and the reference information. Therefore, when the work has a problem with the reference information, there is an effect that the solution point can be surely recognized.

  According to the image input / output device according to claim 4, in addition to the effect exhibited by the image input / output device according to any one of claims 1 to 3, the three-dimensional shape of the subject and the surface capable of projecting the image in the projection direction Accordingly, the imaging data of the subject is corrected so as to be equivalent to the imaging data in the case of imaging from a direction substantially perpendicular to the surface, and the corrected image data is used to Make a comparison. As a result, there is an effect that it is possible to suppress a decrease in detection accuracy of the comparison result due to the distortion of the imaging data.

  According to the image input / output device described in claim 5, in addition to the effect produced by the image input / output device described in claim 4, the image corresponding to the reference information and the image corresponding to the comparison result information Based on the three-dimensional shape of the image projectable surface, the image is projected as a non-distorted image on the subject or the image projectable surface in the projection direction. There is an effect that it can be suppressed that it becomes difficult to identify.

  According to the image input / output device of the sixth aspect, in addition to the effect produced by the image input / output device according to any one of the first to fifth aspects, the reference information is model information for the subject. There is an effect that a comparison result based on information of a certain model can be provided.

  According to the image input / output device described in claim 7, in addition to the effect produced by the image input / output device according to any one of claims 2 to 5, for each process unit constituting one work, Since the image corresponding to the comparison result information as the comparison result is projected, it is possible to provide the comparison result based on the information as the exemplary model information for each process unit.

  According to the image input / output device of the eighth aspect, in addition to the effect of the image input / output device according to the seventh aspect, a comparison that is a comparison result with the model information in each process unit constituting one work. Since the image corresponding to the result information is projected together with the model information of the corresponding process, there is an effect that the comparison result based on the information of the exemplary model can be provided with more specificity in each process unit. is there.

  According to the image input / output device of the ninth aspect, in addition to the effect produced by the image input / output device according to any one of the second to eighth aspects, the comparison result information is subject correction information with respect to the reference information. There is an effect that it is possible to clearly recognize the solution to the problem with respect to the difference from the reference information.

  According to the image input / output device of the tenth aspect, in addition to the effect produced by the image input / output device according to any one of the second to ninth aspects, the comparison result information is the evaluation information of the subject with respect to the reference information. There is an effect that the evaluation based on the reference information can be specifically recognized.

  According to the image input / output device according to claim 11, in addition to the effect produced by the image input / output device according to any one of claims 1 to 10, a projection unit including a light source unit, a spatial modulation unit, and a projection optical system. And the imaging unit that includes the imaging unit and is provided integrally with the projection unit can be moved according to the subject by the movable unit, so that the imaging direction and the projection direction can be changed as necessary. For example, in the case where a work is performed on a subject, there is an effect that the imaging direction and the projection direction can be reduced from being obstructed by the work.

  According to the image input / output device described in claim 12, in addition to the effect achieved by the image input / output device according to claim 11, the projection unit and the imaging unit are moved by the movable means, and the projection direction and the imaging direction are determined by the subject. Therefore, the imaging direction and the projection direction can be reduced from being obstructed by the work.

  According to the image input / output device of the thirteenth aspect, in addition to the effect achieved by the image input / output device according to the eleventh aspect, the projection unit and the imaging unit are moved by the movable means, and the projection direction and the imaging direction are determined by the subject. In particular, when a right-handed worker is working, the imaging direction and the projection direction can be reduced from being obstructed by the work. .

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is an external perspective view of an image input / output device 1 of the present invention.

  The image input / output device 1 includes a digital camera mode that functions as a digital camera, a webcam mode that functions as a web camera, a stereoscopic image mode for detecting a three-dimensional shape and acquiring a stereoscopic image, and a planarized flat surface of a curved document. The apparatus includes various modes such as a planarized image mode for acquiring a normalized image, a sampled trace mode in which captured image data of a subject is compared with sample data, and the result is projected and displayed.

  In FIG. 1, in order to detect the three-dimensional shape of the document P as a subject, particularly in the stereoscopic image mode and the planarized image mode, striped pattern light obtained by alternately arranging light and dark from the image projection unit 13 described later is used. A state of projection is illustrated.

  The image input / output device 1 is connected to an imaging head 2 formed in a substantially box shape, a pipe-shaped arm member 3 having one end connected to the imaging head 2, and the other end of the arm member 3. And a base 4 formed in a substantially L shape.

  The imaging head 2 is a case in which an image projection unit 13 and an image imaging unit 14 described later are included. On the front surface of the imaging head 2, a cylindrical barrel 5 is disposed at the center thereof, a finder 6 is disposed obliquely above the barrel 5, and a flash 7 is disposed on the opposite side of the finder 6. In addition, a part of the lens of the imaging optical system 21 which is a part of the image capturing unit 14 described later is exposed on the outer surface between the finder 6 and the flash 7, and an image of the subject is input from the exposed part. The

  The lens barrel 5 is a cover that protrudes from the front of the imaging head 2 and encloses the projection optical system 20 that is a part of the image projection unit 13 therein. The lens barrel 5 holds the projection optical system 20 so that the projection optical system 20 can be moved as a whole for focus adjustment and is prevented from being damaged. Further, from the end surface of the lens barrel 5, a part of the lens of the projection optical system 20 that is a part of the image projection unit 13 is exposed on the outer surface, and image signal light is projected from the exposed part toward the projection surface. The

  The viewfinder 6 is composed of an optical lens that is disposed from the back of the imaging head 2 through the front. When the user looks in from the back of the image pickup apparatus 1, a range that substantially matches the range in which the image pickup optical system 21 forms an image on the CCD 22 can be seen.

  The flash 7 is a light source for supplementing a necessary light amount in, for example, a digital camera mode or a sample trace mode, and is composed of a discharge tube filled with xenon gas. Therefore, it can be repeatedly used by discharging from a capacitor (not shown) built in the imaging head 2.

  On the upper surface of the imaging head 2, a release button 8 is disposed on the front side, a mode switch 9 is disposed behind the release button 8, and a monitor LCD 10 is disposed on the opposite side of the mode switch 9.

  The release button 8 is composed of a two-stage push button switch that can be set in two types of states, “half-pressed state” and “full-pressed state”. The state of the release button 8 is managed by a processor 15 which will be described later, and in the “half-pressed state”, the well-known autofocus (AF) and automatic exposure (AE) functions are activated, and the focus, aperture, and shutter speed are adjusted. Imaging or the like is performed in the “fully pressed state”.

  The mode switch 9 is a switch that can be set to various modes such as a digital camera mode, a webcam mode, a stereoscopic image mode, a planarized image mode, a sample trace mode, and an off mode. The state of the mode changeover switch 9 is managed by the processor 15, and the processing of each mode is executed when the state of the mode changeover switch 9 is detected by the processor 15.

  The monitor LCD 10 is composed of a liquid crystal display, and receives an image signal from the processor 15 and displays an image to the user. For example, the monitor LCD 10 displays a captured image in a digital camera mode, a webcam mode, a model trace mode, a three-dimensional shape detection result image in a stereoscopic image mode, a planarized image in a planarized image mode, and the like.

  Furthermore, an antenna 11 as an RF (wireless) interface and a connecting member 12 that connects the imaging head 2 and the arm member 3 are disposed on the side surface of the imaging head 2.

  The antenna 11 wirelessly communicates captured image data acquired in the digital camera mode, stereoscopic image data acquired in the stereoscopic image mode, comparison result information data acquired in the sample trace mode, and the like to the external interface via the RF driver 24 described later. On the other hand, sample data used in the sample trace mode, a work result determination program, and the like are received from the external interface by wireless communication and output to the RF driver 24 described later.

  The connecting member 12 is formed in a ring shape in which an internal thread is formed on the inner peripheral surface, and is fixed to the side surface of the imaging head 2 so as to be rotatable. A male screw is formed on one end side of the arm member 3. By fitting the female screw and the male screw, the imaging head 2 and the arm member 3 can be detachably connected, and the imaging head 2 can be fixed at an arbitrary angle. Therefore, the imaging head 2 can be removed and used as a normal digital camera (digital camera).

  The arm member 3 is for holding the imaging head 2 in a predetermined imaging position so as to be changeable, and is composed of a bellows-like pipe that can be bent into an arbitrary shape. Therefore, the imaging head 2 can be directed to an arbitrary position.

  The base 4 is mounted on a mounting table such as a desk and supports the imaging head 2 and the arm member 3. Since it is formed in a substantially L shape in plan view, the imaging head 2 and the like can be stably supported. In addition, since the base 4 and the arm member 3 are detachably connected, it is convenient to carry and can be stored in a small space.

  FIG. 2 is a diagram schematically illustrating the internal configuration of the imaging head 2. The imaging head 2 mainly includes an image projection unit 13, an image imaging unit 14, and a processor 15.

  The image projection unit 13 is a unit for projecting an arbitrary projection image on a projection plane, and along the projection direction, a substrate 16 and a plurality of LEDs 17 (hereinafter collectively referred to as “LED array 17A”), A light source lens 18, a projection LCD 19, and a projection optical system 20 are provided. The image projection unit 13 will be described in detail with reference to FIG.

  The image capturing unit 14 is a unit for capturing an image of a document P as a subject, and includes an image capturing optical system 21 and a CCD 22 along the light input direction.

  The imaging optical system 21 is composed of a plurality of lenses and has a known autofocus function, and automatically adjusts the focal length and aperture to form an image of light from the outside on the CCD 22.

  The CCD 22 is configured by arranging photoelectric conversion elements such as CCD (Charge Coupled Device) elements in a matrix, and a signal corresponding to the color and intensity of light of an image formed on the surface via the imaging optical system 21. Is converted into digital data and output to the processor 15.

  The processor 15 includes a flash 7, a release button 8, a mode switch 9, a monitor LCD driver 23 through a monitor LCD 10, an RF driver 24 through an antenna 11, a power supply interface 25 through a battery 26, an external memory 27, a cache The LED array 17A via the memory 28, the light source driver 29, the projection LCD 19 via the projection LCD driver 30, and the CCD 22 via the CCD interface 31 are electrically connected and managed by the processor 15.

  The external memory 27 is a detachable flash ROM, and displays a captured image captured in the digital camera mode, the webcam mode, the stereoscopic image mode, three-dimensional information, a captured image when the comparison result information is captured in the sample trace mode, and the like. Remember. Specifically, an SD card, a compact flash (registered trademark) card, or the like can be used.

  The cache memory 28 is a high-speed storage device. For example, the captured image captured in the digital camera mode or the sample trace mode is transferred to the cache memory 28 at high speed, and is processed in the processor 15 before being stored in the external memory 27. Specifically, SDRAM, DDRRAM, or the like can be used.

  The power interface 25 includes a battery 26, the light source driver 29 includes an LED array 17A, the projection LCD driver 30 includes a projection LED 19, and the CCD interface 31 includes various ICs (integrated circuits) that control the CCD 22. ing.

  3A is an enlarged view of the image projection unit 13, FIG. 3B is a plan view of the light source lens 18, and FIG. 3C is a diagram showing an arrangement relationship between the projection LCD 19 and the CCD 22.

  As described above, the image projection unit 13 includes the substrate 16, the LED array 17A, the light source lens 18, the projection LCD 19, and the projection optical system 20 along the projection direction.

  The substrate 16 is used to mount the LED array 17A and to perform electrical wiring with the LED array 17A. Specifically, it is possible to use a substrate having a single layer or a multilayer structure in which an insulating resin is applied to an aluminum substrate and then a pattern is formed by electroless plating, or a core of a color epoxy substrate.

  The LED array 17A is a light source that emits radial light toward the projection LCD 19, and a plurality of LEDs 17 (light emitting diodes) are arranged in a staggered pattern on the substrate 16 and bonded via a silver paste. Further, they are electrically connected via bonding wires.

  By using a plurality of LEDs 17 as a light source in this way, the efficiency of converting electricity into light (electro-optical conversion efficiency) is improved compared with the case where an incandescent bulb, a halogen lamp, or the like is used as a light source. Generation of ultraviolet rays can be suppressed. Therefore, it is possible to drive with power saving, and to save power and extend the life. Moreover, the temperature rise of an apparatus can be reduced.

  As described above, the LED 17 generates extremely low heat rays as compared with a halogen lamp or the like. Therefore, a resin lens can be used as the light source lens 18 and the projection optical system 20 described later. Therefore, compared with the case where a glass lens is employed, each of the lenses 18 and 20 can be configured at low cost and light weight.

  Each LED 17 constituting the LED array 17A emits the same emission color, and is configured to emit amber color using four elements of Al, In, Ga, and P as materials. Therefore, it is not necessary to consider correction of chromatic aberration that occurs when emitting a plurality of emission colors, and it is not necessary to employ an achromatic lens as the projection optical system 20 in order to correct chromatic aberration. The degree of freedom in design can be improved.

  In addition, by using an amber LED of a four-element material having an electro-optical conversion rate as high as about 80 lumen / W compared to other luminescent colors, it is possible to achieve higher brightness, power saving, and longer life. The effect related to arranging the LEDs 17 in a staggered manner will be described with reference to FIG.

  Specifically, the LED array 17A is composed of 59 LEDs 17, and each LED 17 is driven at 50 mW (20 mA, 2.5 V). Eventually, all 59 LEDs 17 are driven with power consumption of approximately 3 W. Further, the brightness as the luminous flux value when the light emitted from each LED 17 passes through the light source lens 18 and the projection LCD 19 and is irradiated from the projection optical system 20 is about 25 ANSI lumens even in the case of the entire surface irradiation. Is set to

  By adopting this brightness, for example, in the stereoscopic image mode, when detecting the three-dimensional shape of a subject such as the face of a person or animal, the person or animal does not give glare to the person or animal. It is possible to detect a three-dimensional shape that is not collapsed.

  The light source lens 18 is a lens that collects light emitted radially from the LED array 17A, and the material thereof is made of an optical resin typified by acrylic.

  Specifically, the light source lens 18 includes a convex lens portion 18a that protrudes toward the projection LED 19 at a position facing each LED 17 of the LED array 17A, and a base portion 18b that supports the lens portion 18a. An epoxy encapsulant 18c for sealing the LED 17 and filling the substrate 16 and the light source lens 18 in an internal space of the base portion 18b and filling an opening containing the LED array 17A, and the base portion 18b And a positioning pin 18d for connecting the light source lens 18 and the substrate 16 to each other.

  The light source lens 18 is fixed on the substrate 16 by inserting positioning pins 18 d into the long holes 16 formed in the substrate 16 while enclosing the LED array 17 </ b> A inside the opening.

  Therefore, the light source lens 18 can be disposed in a space-saving manner. In addition to the function of mounting the LED array 17A on the substrate 16, the function of supporting the light source lens 18 is also used, so that a part for supporting the light source lens 18 is not required and the number of parts is reduced. be able to.

  Each lens portion 18a is disposed at a position facing each LED 17 of the LED array 17A in a one-to-one relationship.

Therefore, the radial light emitted from each LED 17 is efficiently condensed by each lens unit 18 facing each LED 17 and irradiated to the projection LED 19 as radiation having high directivity as shown in the figure. The reason why the directivity is increased in this way is that the uneven transmittance in the surface can be suppressed by making light enter the projection LCD 19 substantially perpendicularly. At the same time, since the projection optical system 20 has telecentric characteristics and its incident NA is about 0.1, it is regulated so that only light within ± 5 ° vertical can pass through the internal diaphragm. . Thus, vertically aligned the emission angle of the light from the LED 17, and, placing a light flux Ho and command to ± 5 ° is gist of image quality.

  The projection LCD 19 is a spatial modulation element that spatially modulates the light collected through the light source lens 18 and outputs image signal light to the projection optical system 20. It is composed of plate-like liquid crystal displays having different ratios.

  Further, as shown in (C), each pixel constituting the projection LCD 19 includes one pixel column arranged in a straight line along the longitudinal direction of the liquid crystal display, and the one pixel column is a liquid crystal display. Other pixel columns that are shifted by a predetermined distance in the longitudinal direction are alternately arranged in parallel.

  Note that (C) is a state in which the front surface of the imaging head 2 is directed to the front side of the paper, light is irradiated toward the projection LCD 19 from the back side of the paper, and a subject image is formed on the CCD 22 from the side of the paper.

  In this way, by arranging the pixels constituting the projection LCD 19 in a staggered pattern in the longitudinal direction, the light subjected to spatial modulation by the projection LCD 19 in the direction orthogonal to the longitudinal direction (short direction) is ½ pitch. Can be controlled. Therefore, the projection pattern can be controlled with a fine pitch, and the three-dimensional shape can be detected with high accuracy by increasing the resolution.

  In particular, in the stereoscopic image mode and planarized image mode, which will be described later, in order to detect a three-dimensional shape of a subject, when projecting a striped pattern light in which light and dark are alternately arranged toward the subject, the direction of the stripe is changed. By matching with the short direction of the projection LCD 19, the boundary between light and dark can be controlled at ½ pitch, and thus a three-dimensional shape can be detected with high accuracy.

  In the imaging head 2, the projection LCD 19 and the CCD 22 are arranged in a relationship as shown in (C). Specifically, since the wide surface of the projection LCD 19 and the wide surface of the CCD 22 are arranged in substantially the same direction, when an image projected on the projection surface from the projection LCD 19 is formed on the CCD 22, The projected image can be formed as it is without bending the projected image with a half mirror or the like.

  The CCD 22 is disposed on the longitudinal direction side (the direction side in which the pixel column extends) of the projection LCD 19. Therefore, in particular, when detecting the three-dimensional shape of the subject using the principle of triangulation in the stereoscopic image mode or the planarized image mode, the inclination formed by the CCD 22 and the subject is controlled at 1/2 pitch. Therefore, a three-dimensional shape can be similarly detected with high accuracy.

  The projection optical system 20 is a plurality of lenses that project the image signal light that has passed through the projection LED 19 toward the projection surface, and is composed of a telecentric lens made of a combination of glass and resin. Telecentric means a configuration in which the principal ray passing through the projection optical system 20 is parallel to the optical axis in the incident side space and the position of the exit pupil is infinite. By using telecentricity in this way, only light passing through the projection LCD 19 at a vertical angle of ± 5 ° can be projected as described above, so that the image quality can be improved.

  FIG. 4 is a diagram for explaining the arrangement of the LED array 17A. (A) is a figure which shows the illumination intensity distribution of the light which passed the light source lens 18, (b) is a top view which shows the arrangement state of LED array 17A, (c) shows the synthetic | combination illumination intensity distribution in projection LCD19 surface. FIG.

  As shown in (a), the light that has passed through the light source lens 18 reaches the surface of the projection LCD 19 as light having an illuminance distribution as shown on the left side at a half-value spread half angle θ (= approximately 5 °). Designed to be.

  Further, as shown in (b), the plurality of LEDs 17 are arranged in a staggered pattern on the substrate 16. Specifically, an LED row in which a plurality of LEDs 17 are arranged in series at a d pitch is arranged in parallel at a √3 / 2d pitch, and the LED rows are moved 1 / 2d in the same direction every other row. It is arranged to be in a state.

  In other words, the distance between one LED 17 and the LCD 17 around the one LED 17 is set to be d (triangular lattice arrangement).

  The length d is determined to be equal to or less than the full width at half maximum (FWHM (Full Width Half Maximun)) of the illuminance distribution formed in the projection LCD 19 by the light emitted from one of the LEDs 17.

  Accordingly, the combined illuminance distribution of the light passing through the light source lens 18 and reaching the surface of the projection LCD 19 is substantially linear including small ripples as shown in FIG. Can be irradiated. Therefore, the illuminance unevenness in the projection LCD 19 can be suppressed, and as a result, a high-quality image can be projected.

  FIG. 5 is an electrical block diagram of the image input / output device 1. Note that the description of the above-described configuration is omitted. The processor 15 includes a CPU 35, a ROM 36, and a RAM 37.

  The CPU 35 uses the RAM 37 in accordance with the processing by the program stored in the ROM 36 to detect the pressing operation of the release button 8, fetch the image data from the CCD 22, transfer and store the image data, and the mode switch 9. Various processes, such as detection of the state of, are performed.

  The ROM 36 includes a camera control program 36a, a pattern light photographing program 36b, a luminance image generation program 36c, a code image generation program 36d, a code boundary extraction program 36e, a lens aberration correction program 36f, and a triangulation calculation program 36g. A document orientation calculation program 36h and a plane conversion program 36i are stored.

  The camera control program 36a is a program related to the control of the entire imaging apparatus 1 including the main process shown in FIG.

  The pattern light imaging program 36b is a program for imaging a state in which pattern light is projected onto a subject and a state in which the original is not projected in order to detect the three-dimensional shape of the document P.

  The luminance image generation program 36c takes the difference between the pattern light presence image obtained by imaging the pattern light projected by the pattern light imaging program 36b and the pattern light no image obtained by imaging the state where the pattern light is not projected. This is a program for generating a luminance image of the patterned light.

  In addition, a plurality of types of pattern light are projected in time series and imaged for each pattern light, and the difference between each of the plurality of captured pattern light images and the pattern light no image is obtained. A luminance image is generated.

  The code image generation program 36d is a program that generates a code image in which a plurality of luminance images generated by the luminance image generation program 36c are superimposed and a predetermined code is assigned to each pixel.

  The code boundary extraction program 36e obtains code boundary coordinates with sub-pixel accuracy using the code image generated by the code image generation program 36d and the luminance image generated by the luminance image generation program 36c. It is a program.

  The lens aberration correction program 36f is a program that corrects the aberration of the imaging optical system 20 with respect to the code boundary coordinates obtained with subpixel accuracy by the code boundary extraction program 36e.

  The triangulation calculation program 36g is a program for calculating the three-dimensional coordinates of the real space related to the boundary coordinates from the boundary coordinates of the code subjected to the aberration correction by the lens aberration correction program 36f.

  The document orientation calculation program 36h is a program for estimating the three-dimensional shape of the document P from the three-dimensional coordinates calculated by the triangulation calculation program 36g.

  The plane conversion program 36i is a program that generates a planarized image that is captured from the front of the document P, based on the three-dimensional shape of the document P calculated by the document orientation calculation program 36h.

  The RAM 37 includes a pattern light existence image storage unit 37a, a pattern light no image storage unit 37b, a luminance image storage unit 37c, a code image storage unit 37d, a code boundary coordinate storage unit 37e, an ID storage unit 37f, Aberration correction coordinate storage unit 37g, three-dimensional coordinate storage unit 37h, document orientation calculation result storage unit 37i, plane conversion result storage unit 37j, projection image storage unit 37k, work result determination program storage unit 37l, working Area 37m is allocated as a storage area.

  The pattern light present image storage unit 37a stores a pattern light present image obtained by imaging the state in which the pattern light is projected onto the document P by the pattern light photographing program 36b. The pattern light no image storage unit 37b stores a pattern light no image obtained by imaging the state in which the pattern light is not projected onto the document P by the pattern light photographing program 36b.

  The luminance image storage unit 37c stores a luminance image generated by the luminance image generation program 36c. The code image storage unit 37d stores a code image generated by the code image generation program 36d. The code boundary coordinate storage unit 37e stores the boundary coordinates of each code obtained by the code boundary extraction program 36e with subpixel accuracy extracted. The ID storage unit 37f stores an ID assigned to a luminance image having a change in brightness at a pixel position having a boundary. The aberration correction coordinate storage unit 37g stores the boundary coordinates of the code subjected to the aberration correction by the lens aberration correction program 36f. The three-dimensional shape coordinate storage unit 37h stores the three-dimensional coordinates of the real space calculated by the triangulation calculation program 36g.

  The document orientation calculation result storage unit 37i stores parameters related to the three-dimensional shape of the document P calculated by the document orientation calculation program 36h. The plane conversion result storage unit 37j stores the plane conversion result generated by the plane conversion program 36i. The projection image storage unit 37k stores image information to be projected from the image projection unit 13. The working area 37m stores data temporarily used for calculation by the CPU 15.

  The work result determination program storage unit 371 stores a work result determination program received via the antenna 11. The “work result determination program” is a program for generating, as comparison result information, a comparison result between captured image data of a subject and reference information related to the subject in processing executed in the example trace mode. .

  FIG. 6 is a flowchart of the main process. Details of the digital camera processing (S605), webcam processing (S607), stereoscopic image processing (S609), planarized image processing (S611), and example tracing processing (S613) will be described later.

  In the main process, first, when the power supply is activated (S601), the processor 15 and other interfaces are initialized (S602).

  Then, a key scan for determining the state of the mode switch 9 is performed (S603), it is determined whether or not the setting of the mode switch 9 is the digital camera mode (S604), and if it is the digital camera mode (S604: Yes), The process proceeds to a digital camera process to be described later (S605).

  On the other hand, if it is not the digital camera mode (S604: No), it is determined whether or not the setting of the mode switch 9 is the webcam mode (S606). If it is the webcam mode (S606: Yes), the process proceeds to the webcam process described later. (S607).

  On the other hand, if it is not the webcam mode (S605: No), it is determined whether the setting of the mode switch 9 is the stereoscopic image mode (S608). If it is the stereoscopic image mode (S608: Yes), the stereoscopic image processing described later is performed. (S609).

  On the other hand, if it is not the stereoscopic image mode (S608: No), it is determined whether or not the setting of the mode switch 9 is the planar image mode (S610). If it is the planar image mode (S610: Yes), it will be described later. The process proceeds to planarized image processing (S611).

  On the other hand, if it is not the planar image mode (S610: No), it is determined whether or not the setting of the mode switch 9 is the sample trace mode (S612). If it is the sample trace mode (S612: Yes), it will be described later. The process proceeds to a model trace process (S613).

  On the other hand, if it is not the sample trace mode (S612: No), it is determined whether or not the mode switch 9 is in the off mode (S612). If it is not the off mode (S612: No), the processing from S603 is repeated. If it is an off mode (S612: Yes), the process is terminated.

  FIG. 7 is a flowchart of the digital camera process (S605 in FIG. 6). The digital camera process is a process for acquiring an image captured by the image capturing unit 14.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S701). Thereby, a high quality captured image can be provided to the user.

  Next, a finder image (an image in a range visible through the finder 6) is displayed on the monitor LCD 10 (S702). Therefore, the user can confirm the captured image (imaging range) before actual imaging with the image displayed on the monitor LCD 10 without looking into the finder 6.

  Next, the release button 8 is scanned (S703a), and it is determined whether the release button 8 is half-pressed (S703b). If half-pressed (S703b: Yes), the auto focus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S703c). If it is not half-pressed (S703b: No), the processing from S703a is repeated.

  Next, the release button 8 is scanned again (S703d), and it is determined whether or not the release button 8 has been fully pressed (S703e). If it is fully pressed (S703e: Yes), it is determined whether or not the flash mode is set (S704).

  As a result, if the flash mode is selected (S704: Yes), the flash 7 is projected (S705) and photographed (S706). If the flash mode is not selected (S704: No), the flash 7 is not projected. A picture is taken (S706). If it is determined in S703e that the button has not been fully pressed (S703e: No), the processing from S703a is repeated.

  Next, the captured image is transferred from the CCD 22 to the cache memory 28 (S707), and the captured image stored in the cache memory 28 is displayed on the monitor LCD 10 (S708). In this way, by transferring the captured image to the cache memory 28, the captured image can be displayed on the monitor LCD 10 at a higher speed than when transferring to the main memory. Then, the captured image is stored in the external memory 27 (S709).

  Finally, it is determined whether or not the mode change switch 9 has changed (S710). If there is no change (S710: Yes), the processing from S702 is repeated, and if there is a change (S710: No), The process ends.

  FIG. 8 is a flowchart of the webcam process (S607 in FIG. 6). The webcam process is a process of transmitting a captured image (including a still image and a moving image) captured by the image capturing unit 14 to an external network. In this embodiment, it is assumed that a moving image is transmitted as an imaged image to an external network.

  In this process, a low resolution setting signal is first transmitted to the CCD 22 (S801), the well-known autofocus (AF) and automatic exposure (AE) functions are activated, and the focus, aperture, and shutter speed are adjusted (S802). ), Shooting is started (S803).

  Then, the captured image is displayed on the monitor LCD 10 (S804), the finder image is stored in the projection image storage unit 37k (S805), the projection process described later is performed (S806), and stored in the projection image storage unit 37k. Project the projected image onto the projection plane.

  Further, the captured image is transferred from the CCD 22 to the cache memory 28 (S807), and the captured image transferred to the cache memory 28 is transmitted to the external network via the RF driver 24 and the antenna 11 which are RF interfaces (S808).

  Finally, it is determined whether or not the mode change switch 9 has changed (S809). If there is no change (S809: Yes), the processing from S802 is repeated. If there is a change (S809: No), The process ends.

  FIG. 9 is a flowchart of the projection process (S806 in FIG. 8). This process is a process of projecting the image stored in the shadow image storage unit 37k from the projected image projection unit 13 onto the projection plane. In this process, first, it is confirmed whether or not an image is stored in the projection image storage unit 37k (S901). If stored (S901: Yes), the image stored in the projection image storage unit 37k is transferred to the projection LCD driver 30 (S902), and the projection LCD driver 30 outputs an image signal corresponding to the image to the projection LCD 19. The image is displayed on the projection LCD 19 (S903).

  Next, the light source driver 29 is driven (S904), the LED array 17A is turned on by an electrical signal from the light source driver 29 (S905), and the process is terminated.

  Thus, when the LED array 17A is turned on, the light emitted from the LED array 17A reaches the projection LCD 19 via the light source lens 18, and the projection LCD 19 performs spatial modulation according to the image signal transmitted from the projection LCD driver 30. And output as image signal light. The image signal light output from the projection LCD 19 is projected as a projection image on the projection surface via the projection optical system 20.

  FIG. 10 is a flowchart of the stereoscopic image processing (S609 in FIG. 6). The stereoscopic image processing is processing for detecting a three-dimensional shape of a subject and acquiring, displaying, and projecting a three-dimensional shape detection result image as the stereoscopic image.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S1001), and a finder image is displayed on the monitor LCD 10 (S1002).

  Next, the release button 8 is scanned (S1003a), and it is determined whether the release button 8 is half-pressed (S1003b). If half-pressed (S1003b: Yes), the auto focus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S1003c). If not half-pressed (S1003b: No), the processing from S1003a is repeated.

  Next, the release button 8 is scanned again (S1003d), and it is determined whether or not the release button 8 is fully pressed (S1003e). If it is fully pressed (S1003e: Yes), it is determined whether or not the flash mode is set (S1003f).

  As a result, if the flash mode is selected (S1003f: Yes), the flash 7 is projected (S1003g), photographed (S1003h), and if the flash mode is not selected (S1003f: No), the flash 7 is not projected. A picture is taken (S1003h). If it is determined in S1003e that the button has not been fully pressed (S1003e: No), the processing from S1003a is repeated.

  Next, a three-dimensional shape detection process described later is performed to detect the three-dimensional shape of the subject (S1006).

  Next, the three-dimensional shape detection result in the three-dimensional shape detection process (S1006) is stored in the external memory 27 (S1007), and the three-dimensional shape detection result is displayed on the monitor LCD 10 (S1008). The three-dimensional shape detection result is displayed as an aggregate of three-dimensional coordinates (X, Y, Z) in the real space of each measurement vertex.

  Next, a three-dimensional shape detection result image as a three-dimensional image (3D CG image) displaying the surface by connecting the measurement vertices as a three-dimensional shape detection result with a polygon is stored in the projection image storage unit 37k (S1009). A projection process similar to the projection process of S806 in FIG. 8 is performed (S1010). In this case, the coordinates on the projection LCD 19 with respect to the obtained three-dimensional coordinates are obtained by using the inverse function of the formula for converting the coordinates on the projection LCD 19 described in FIG. 18 into the three-dimensional space coordinates. The three-dimensional shape result coordinates can be projected on the projection plane.

  Then, it is determined whether or not the mode change switch 9 has changed (S1011). If there is no change (S1011: Yes), the processing from S702 is repeated. If there is a change (S1011: No), the processing is performed. finish.

  FIG. 11A is a diagram for explaining the principle of the spatial code method used to detect a three-dimensional shape in the above-described three-dimensional shape detection process (S1006 in FIG. 10). It is a figure which shows the pattern light different from (a). Either (a) or (b) may be used for the pattern light, and further, a gray level code which is a multi-tone code may be used.

  Details of this spatial coding method are described by Kosuke Sato and one other, “Distance Image Input by Spatial Coding”, IEICE Transactions, 85/3 Vol. J 68-D No3 p369-375.

  The spatial code method is one type of method for detecting the three-dimensional shape of a subject based on triangulation between projected light and an observed image. As shown in (a), the distance between the projection light source L and the observation device O is determined. It is characterized in that it is set apart by D, and the space is divided into long and narrow fan-shaped regions and coded.

  When the three mask patterns A, B, and C in the figure are projected in order from the MSB, each fan-shaped area is coded into bright “1” and dark “0” by the mask. For example, since the area including the point P is not exposed to light in the masks A and B and bright in the mask C, it is encoded as 001 (A = 0, B = 0, C = 1).

  Each fan-shaped region is assigned a code corresponding to the direction φ, and each can be regarded as one slit beam. Therefore, the scene is photographed for each mask with a camera as an observation device, and the bit plane of the memory is constructed by binarizing the light / dark pattern.

  Thus, the horizontal position (address) of the obtained multiple bit-plane image corresponds to the observation direction θ, and the contents of the memory at this address give the projected light code, that is, φ. The coordinates of the point of interest are determined from θ and φ.

  In addition, as a mask pattern used in this method, a case where a pure binary code such as mask patterns A, B, and C is used is illustrated in FIG. There is a risk that a large error will occur.

  For example, the point Q in (a) indicates the boundary between the region 3 (011) and the region 4 (100), but if 1 of the mask A is shifted, the code in the region 7 (111) may be generated. In other words, a large error may occur when the Hamming distance is 2 or more between adjacent regions.

  Therefore, as the mask pattern used in this method, as shown in (b), by using a code whose hamming distance is always 1 between adjacent regions, the coding error as described above can be avoided. It is said that.

  FIG. 12A is a flowchart of the three-dimensional shape detection process (S1006 in FIG. 10). In this process, first, an imaging process is performed (S1210). This imaging process uses a plurality of pure binary code mask patterns shown in FIG. 11 (a) to emit striped pattern light (see FIG. 1) in which light and dark are alternately arranged from the image projection unit 13. This is a process of acquiring a pattern light existence image obtained by sequentially projecting on a subject and imaging a state in which each pattern light is projected, and a pattern light no image obtained by imaging a state in which the pattern light is not projected.

  When the imaging process is completed (S1210), a three-dimensional measurement process is performed (S1220). The three-dimensional measurement process is a process of actually measuring the three-dimensional shape of the subject using the pattern light existence image and the pattern light no image acquired by the imaging process. Thus, when the three-dimensional measurement process ends (S1220), the process ends.

  FIG. 12B is a flowchart of the imaging process (S1210 in FIG. 12A). This process is executed based on the pattern light imaging program 36a. First, the image imaging unit 14 images a subject without projecting pattern light from the image projection unit 13, thereby acquiring a pattern light no-image (S1211). ). The acquired pattern light no-image is stored in the pattern light no-image storage unit 37b.

  Next, the counter i is initialized (S1212), and it is determined whether or not the value of the counter i is the maximum value imax (S1213). The maximum value imax is determined by the number of mask patterns to be used. For example, when 8 types of mask patterns are used, the maximum is imax (= 8).

  If it is determined that the value of the counter i is smaller than the maximum value imax (S1213: Yes), the i-th mask pattern among the mask patterns to be used is displayed on the projection LCD 19, and the i-th mask pattern is displayed. The i-th pattern light projected by the above is projected onto the projection surface (S1214), and the state where the pattern light is projected is photographed by the image capturing unit 14 (S1215).

  In this way, a pattern light existence image obtained by imaging the state in which the i-th pattern light is projected onto the subject is acquired. The acquired pattern light existence image is stored in the pattern light existence image storage unit 37a.

  When the photographing is finished, the projection of the i-th pattern light is finished (S1216), "1" is added to the counter i to project the next pattern light (S1217), and the processing from S1213 is repeated.

  If it is determined that the value of the counter i is greater than the maximum value imax (S1213: No), the process ends. That is, in this imaging process, one pattern light no image and the maximum value imax pattern light existence images are acquired.

  FIG. 12C is a flowchart of the three-dimensional measurement process (S1220 in FIG. 12A). This process is executed based on the luminance image generation program 36c, and first generates a luminance image (S1221). Here, the luminance is a Y value in the YCbCr space, and is a value calculated from Y = 0.22989 · R + 0.5866 · G + 0.1145 · B from the RGB value of each pixel. By obtaining the Y value for each pixel, a luminance image relating to the image with and without pattern light is generated. The generated luminance image is stored in the luminance image storage unit 37c. A number corresponding to the pattern light number is assigned to each luminance image.

  Next, the code image generation program 36d generates a code image coded for each pixel by combining the generated luminance images using the spatial code method described above (S1222).

  The code image can be generated by comparing each pixel of the luminance image related to the image with pattern light stored in the luminance image storage unit 37c with a predetermined luminance threshold and combining the results. The generated code image is stored in the code image storage unit 37d.

  Next, a code boundary coordinate detection process described later is performed by the code boundary extraction program 36e (S1223), and the boundary coordinates of the code assigned to each pixel are detected with subpixel accuracy.

  Next, lens aberration correction processing is performed by the lens aberration correction program 36f (S1224). By this processing, it is possible to correct the error in the code boundary coordinates detected in S1223 that includes an error due to the influence of distortion or the like of the imaging optical system 21.

  Next, real space conversion processing based on the triangulation principle is performed by the triangulation calculation program 36g (S1225). The code boundary coordinates on the CCD space after the aberration correction is performed by this processing are converted into the three-dimensional coordinates in the real space, and the three-dimensional coordinates as the three-dimensional shape detection result are obtained.

  FIG. 13 is a diagram for explaining the outline of the code boundary coordinate detection process (S1223 in FIG. 12). In the upper diagram, the bright and dark boundaries of the actual pattern light in the CCD space are indicated by the boundary line K. The pattern light is encoded by the spatial code method described above, and the boundary between one code and another code is indicated by a bold line in the figure. It is the figure shown by.

  That is, since the coding in the spatial coding method described above is performed on a pixel-by-pixel basis, an error in sub-pixel accuracy occurs between the boundary line K of the actual pattern light and the coded boundary (thick line in the figure). Therefore, the purpose of this code boundary coordinate detection process is to detect code boundary coordinates with sub-pixel accuracy.

  In this process, first, at a certain detection position (hereinafter referred to as “curCDX”), a first pixel G that changes from a certain target code (hereinafter referred to as “curCode”) to another code is detected (first pixel detection step). .

  For example, in curCDXX, when each pixel is detected in order from the top, it is a pixel having curCode up to the boundary (thick line), but curCode changes in the pixel next to the boundary, that is, the first pixel G. This is detected as the first pixel G.

  Next, at the pixel position of the first pixel G, all of the luminance images having a change in brightness are extracted from the luminance images stored in the luminance image storage unit 37c in S1221 of FIG. 12 (luminance image extraction step). ).

  Next, in order to specify a pixel area to be used for approximation, the detection position is moved to the left by “2”, and the code image is referred to at the position of the detection position curCDXX-2 to change from the target code (curCode). Pixel (boundary pixel (pixel H at the detection position of cur CCDX-2)) is searched for, and a predetermined range centered on that pixel (in the present embodiment, -3 pixels and +2 pixels in the Y-axis direction) Range) is specified (part of the pixel area specifying means).

  Next, within the predetermined range, as shown in the lower left graph in the figure, an approximate expression (shown by a solid line) in the Y direction is obtained, and the approximate expression The Y coordinate Y1 at the intersection with the luminance threshold value bTh is obtained (part of the boundary coordinate detection step).

  Note that the luminance threshold value bTh may be calculated from a predetermined range (for example, one half of the average luminance of each pixel) or may be a fixed value given in advance. As a result, the boundary between light and dark can be detected with sub-pixel accuracy.

  Next, the detection position is moved to the right side of “1” from curCCDX-2, and the same processing as described above is performed in curCCDX-1 to obtain a representative value in curCCDX-1 (part of the boundary coordinate detection step).

  In this manner, each detection is performed in a pixel area (see the lower right hatched portion in the figure) composed of a predetermined range in the Y-axis direction centering on the boundary pixel and a range of curCCDX-2 to curCCDX + 2 in the X-axis direction. The representative value at the position is obtained.

  The above processing is performed on all luminance images having pixels that change from curCode to another code, and a weighted average value of representative values for each luminance image is finally adopted as boundary coordinates in curCode (boundary coordinate detection step) Part of).

  As a result, the boundary coordinates of the code can be detected with high accuracy with sub-pixel precision, and by performing the real space conversion process (S1225 in FIG. 12) based on the above-described triangulation principle using the boundary coordinates, The three-dimensional shape of the subject can be detected with high accuracy.

  In addition, since the boundary coordinates can be detected with sub-pixel accuracy by using the approximate expression calculated based on the luminance image in this way, the number of images to be captured is not increased as in the prior art, and pure binary. Pattern light brightened and darkened by a code may be used, and it is not necessary to use a gray code which is a special pattern light.

  In this embodiment, each detection position is composed of a range from “−3” to “+2” in the Y-axis direction around the boundary pixel, and a range from curCCDX-2 to curCCDX + 2 as the detection position in the X-axis direction. Although the region to be processed has been described as a pixel region for obtaining approximation, the ranges of the Y axis and the X axis of the pixel region are not limited to these. For example, only a predetermined range in the Y-axis direction centering on the boundary pixel at the curCDX detection position may be set as the pixel region.

  FIG. 14 is a flowchart of the code boundary coordinate detection process (S1223 in FIG. 12). This process is executed based on the code boundary extraction program 36e. First, each element of the code boundary coordinate sequence in the CCD space is initialized (S1401), and curCCDX is set as the start coordinate (S1402).

  Next, it is determined whether curCCDX is equal to or less than the end coordinate (S1403). If it is equal to or less than the end coordinate (S1403: Yes), curCode is set to “0” (S1404). That is, curCode is initially set to a minimum value.

  Next, it is determined whether curCode is smaller than the maximum code (S1405). If the curCode is smaller than the maximum code (S1405: Yes), the curCCDX refers to the code image to search for the curCode pixel (S1406), and determines whether the curCode pixel exists (S1407).

  As a result, if a curCode pixel exists (S1407: Yes), the curCDXX searches for a code pixel larger than the curCode by referring to the code image (S1408), and a curCode pixel larger than the curCode is found. It is determined whether or not it exists (S1409).

  As a result, if there is a pixel with a code larger than curCode (S1409: Yes), a process for obtaining a later-described boundary with subpixel accuracy is performed (S1410). Then, in order to obtain boundary coordinates for the next curCode, “1” is added to curCode (S1411), and the processing from S1405 is repeated.

  That is, since the boundary exists at the pixel position of the pixel having the curCode or the pixel position of the pixel of the Code that is larger than the curCode, in the present embodiment, the boundary is temporarily the pixel position of the pixel of the CurCode that is larger than the curCode. It is assumed that the process is in progress.

  In addition, when curCode does not exist (S1407: No), or when a pixel with a code larger than curCode does not exist (S1409: No), in order to obtain boundary coordinates for the next curCode, “ 1 "is added (S1411), and the processing from S1405 is repeated.

  Thus, for curCode from 0 to the maximum code, the processing from S1405 to S1411 is repeated, and when curCode becomes larger than the maximum code (S1405: No), “dCCDX” is added to curCCDX to change the detection position (S1412). ) The process from S1403 is repeated at the new detection position in the same manner as described above.

  Then, curCDXX is changed, and finally when curCDX becomes larger than the end coordinate (S1403), that is, when the detection from the start coordinate to the end coordinate is completed, the process ends.

  FIG. 15 is a flowchart of processing (S1410 in FIG. 14) for obtaining code boundary coordinates with subpixel accuracy.

  In this process, first, a change in brightness at a pixel position of a pixel having a code larger than the curCode detected in S1409 of FIG. 14 from the luminance image stored in the luminance image storage unit 37c in S1221 of FIG. All the luminance images having are extracted (S1501).

  Then, the mask pattern number of the extracted luminance image is stored in the array PatID [], and the number of extracted luminance images is stored in noPatID (S1502). The arrays PatID [] and noPatID are stored in the ID storage unit 37f.

  Next, the counter i is initialized (S1503), and it is determined whether or not the value of the counter i is smaller than noPatID (S1504). As a result, if it is determined to be small (S1504: Yes), the CCDY value at the boundary is obtained for the luminance image having the mask pattern number of PatID [i] corresponding to the counter i, and the value is stored in fCDY [i]. (S1505).

  When the processing of S1505 is completed, “1” is added to the counter i (S1506), and the processing from S1504 is repeated. If it is determined in S1504 that the value of the counter i is greater than noPatID (S1504: No), that is, when the processing of S1505 is completed for all the luminance images extracted in S1501, the fCCDY obtained in the processing of S1505 The weighted average value of [i] is calculated, and the result is set as a boundary value (S1507).

  In place of the weighted average value, the median value of fCCDY [i] obtained in the processing of S1505 can be calculated, and the result can be used as a boundary value, or the boundary value can be calculated by statistical calculation.

  In other words, the boundary coordinates are expressed by the curCDXX coordinates and the weighted average value obtained in S1507. The boundary coordinates are stored in the code boundary coordinate storage unit 37e, and the process ends.

  FIG. 16 is a flowchart of the process (S1505 in FIG. 15) for obtaining the CCDY value of the boundary for the luminance image having the mask pattern number of PatID [i].

  In this process, first, a process represented by “ccdx = MAX (curCCDX−dx, 0)” for setting a larger value of “curCDX−dx” and “0” as ccdx is performed, and a counter j is set. Initialization is performed (S1601).

  Specifically, “0” in S1601 means the minimum value of the CCDX value. For example, the curCDX value as a detection position is “1” and the preset dx value is “2”. Assuming that “curCDXX-dx” is “−1”, which is smaller than “0” which is the minimum value of the CCDX value, the subsequent processing in “−1” is set as “ccdx = 0”. I do.

  That is, for a position smaller than the minimum value of the CCDX value, processing for excluding the subsequent processing is performed.

  The value of “dx” can be set in advance to an appropriate integer including “0”. In the example described with reference to FIG. 13, this “dx” is set to “2”. According to the example, the ccdx is set to “curCDX-2”.

  Next, it is determined whether or not ccdx <= MIN (curCCDX + dx, ccdW−1) (S1602). That is, “MIN (curCCDX + dx, ccdW−1)” on the left side is a smaller value among “curCDXX + dx” and “ccdW−1” obtained by subtracting “1” from the maximum CCDX value “ccdW”. Therefore, the value is compared with the “ccdx” value.

  That is, for a position larger than the maximum value of the CCDX value, processing for excluding the subsequent processing is performed.

  If the result of determination is that ccdx is smaller than MIN (curCCDX + dx, ccdW−1) (S1602: Yes), the code image and the luminance image to which PatID [i] is assigned are referred to and the pixel having the boundary exists. The eCDY value of the pixel position is obtained (S1603).

  For example, if the detection position is curCCDX-1 shown in FIG. 13, the pixel I is detected as a pixel candidate having a boundary, and the eCDY value is obtained at the position of the pixel I.

  Next, from the luminance image having the mask pattern number of PatID [i], an approximation regarding the luminance in the ccdy direction within the range of MAX (eCDY-dy, 0) <= ccdy <= MIN (eCCY + dy-1, ccdH-1). A polynomial Bt = fb (ccdy) is obtained (S1604).

  Next, the ccdy value at which the approximate polynomial Bt and the luminance threshold value bTh intersect is obtained, and the value is stored in efCCDY [j] (S1605). By these S1604 and S1605, the boundary coordinates with subpixel accuracy can be detected.

  Next, “1” is added to each of ccdx and counter j (S1605), and the processing from S1602 is repeated. That is, the boundary of subpixel accuracy is detected at each detection position within a predetermined range on the left and right with the cur CCDX as the center.

  If it is determined in S1602 that “ccdx” is larger than “MIN (curCCDX + dx, ccdW−1)” (S1602: No), ccCDY [j] calculated in the range of curCCDX−dx to curCCDX + dx is ccdy. = Approximate polynomial of fy (ccdx) is obtained (S1606). Since each value detected in S1605 by this process is used, the detection accuracy of the boundary coordinates can be improved as compared with the case of detecting the boundary coordinates at one detection position.

  The intersection of the approximate polynomial obtained in this way and curCCDX is set as the CCDY value of the boundary for the luminance image having the mask pattern number of PatID [i] (S1607), and the process ends. As shown in the flowchart of FIG. 15, the processing up to this point is executed for each of the extracted luminance images, and a weighted average value is calculated for the obtained boundary coordinates. Since the boundary coordinates are used (S1507), the detection accuracy of the boundary coordinates can be further improved.

  FIG. 17 is a diagram for explaining the lens aberration correction process (S1224 in FIG. 12). In the lens aberration correction processing, as shown in FIG. 17A, the incident light beam is deviated from the position to be imaged by the ideal lens due to the aberration of the imaging optical system 21. This is a process of correcting the position to the position where the image should be originally formed.

  For example, as shown in FIG. 17B, this aberration correction is data obtained by calculating the aberration of the optical system in the imaging range of the imaging optical system 21 using the half field angle hfa that is the angle of incident light as a parameter. Correct based on.

  This aberration correction processing is executed based on the lens aberration correction program 36f and performed on the code boundary coordinates stored in the code boundary coordinate storage unit 37e, and the data subjected to the aberration correction processing is stored in the aberration correction coordinate storage unit 37g. Stored.

  More specifically, the following camera calibration (approximate expression) (1) to (3) is used to convert arbitrary point coordinates (ccdx, ccdy) in the real image to coordinates (ccdcx, ccdcy) in the ideal camera image. To correct.

  In this embodiment, the aberration amount dist (%) is described as dist = f (hfa) using a half angle of view hfa (deg). In addition, the focal length of the imaging optical system 21 is focal length (mm), the ccd pixel length pixel length (mm), and the center coordinates of the lens in the CCD 22 are (Centx, Centy).

(1) ccdcx = (ccdx−Centx) / (1 + dist / 100) + Centx
(2) ccdcy = (ccdy−Centy) / (1 + dist / 100) + Centy
(3) hfa = arctan [(((ccdx-Centx) ² + (ccdy-Centy) ² ) ^0.5 ) × pixellength / focallength]
FIG. 18 is a diagram for explaining a method of calculating the three-dimensional coordinates in the three-dimensional space from the coordinates in the CCD space in the real space conversion process (S1225 in FIG. 12) based on the triangulation principle.

  In the real space conversion process based on the triangulation principle, the triangulation calculation program 36g calculates the three-dimensional coordinates in the three-dimensional space for the code boundary coordinates subjected to the aberration correction stored in the aberration correction coordinate storage unit 37g. The three-dimensional coordinates calculated in this way are stored in the three-dimensional coordinate storage unit 37h.

  In this embodiment, as the coordinate system of the image input / output apparatus 1 for the document P that is curved in the lateral direction to be imaged, the optical axis direction of the imaging optical system 21 is the Z axis, and the imaging optical system 21 is along the Z axis. A point that is VPZ away from the principal point position is an origin, a horizontal direction with respect to the image input / output device 1 is an X axis, and a vertical direction is a Y axis.

  Further, the projection angle θp from the image projection unit 13 to the three-dimensional space (X, Y, Z), D is the distance between the optical axis of the imaging lens optical system 20 and the optical axis of the image projection unit 13, and the imaging optical system 21. The field of view in the Y direction is Yftop to Yfbottom, the field of view in the X direction is from Xfstart to Xfend, the length (height) in the Y-axis direction of the CCD 22 is Hc, and the length (width) in the X-axis direction is Wc. The projection angle θp is given based on a code assigned to each pixel.

In this case, the three-dimensional space position (X, Y, Z) corresponding to the arbitrary coordinates (ccdx, ccdy) of the CCD 22 is on the point on the image plane of the CCD 22, the projection point of the pattern light, and the XY plane. It can be obtained by solving five equations for the triangle formed by the intersecting points.
(1) Y = − (tan θp) Z + PPZ + tan θp−D + cmp (Xtarget)
(2) Y = − (Ytarget / VPZ) Z + Ytarget
(3) X = − (Xtarget / VP) Z + Xtarget
(4) Ytarget = Yftop− (ccdcy / Hc) × (Yftop−Yfbottom)
(5) Xtarget = Xfstart + (ccdcx / Wc) × (Xfend−Xfstart)
Note that cmp (Xtarget) in (1) is a function for correcting the deviation between the imaging optical system 20 and the image projection unit 13, and in an ideal case where there is no deviation, it can be considered that cmp (Xtarget) = 0. it can.

  On the other hand, as described above, the relationship between the arbitrary coordinates (lcdcx, lcdccy) on the projection LCD 19 included in the image projection unit 13 and the three-dimensional coordinates (X, Y, Z) in the three-dimensional space is as follows: It can be expressed by the formulas (1) to (4).

In this embodiment, the principal point position (0, 0, PPZ) of the image projection unit 13, the Y-direction field of the image projection unit 13 from Ypftop to Ypfbottom, the X-direction field of view from Xpfstart to Xpfend, and the Y of the projection LED 19 The length (height) in the axial direction is Hp, and the length (width) Wp in the X-axis direction.
(1) Y = − (Yptarget / PPZ) Z + Yptarget
(2) X = − (Xptarget / PPZ) Z + Xptarget
(3) Yptarget = Ypftop− (lcdcy / Hp) × (Xpftop−Xpfbottom)
(4) Xptarget = Xpfstart + (lcdcx / Wp) × (Xpfend−Xpfstart)
By using this relational expression, the LCD space coordinates (lcdcx, lcdcy) can be calculated by giving the three-dimensional space coordinates (X, Y, Z) to the above expressions (1) to (4). it can. Therefore, for example, an LCD element pattern for projecting an arbitrary shape and character in a three-dimensional space can be calculated.

  FIG. 19 is a flowchart of planarized image processing (S611 in FIG. 6). Planarized image processing is, for example, when a document P in a curved state as shown in FIG. 1 is imaged or when a rectangular document is imaged from an oblique direction (the captured image has a trapezoidal shape). This is a process of acquiring and displaying a flattened image corrected to a state in which the document is not curved or a state in which an image is taken from a direction perpendicular to the surface.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S1901), and a finder image is displayed on the monitor LCD 10 (S1902).

  Next, the release button 8 is scanned (S1903a), and it is determined whether the release button 8 is half-pressed (S1903b). If half-pressed (S1903b: Yes), the auto focus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S1903c). If not half-pressed (S1903b: No), the processing from S1903a is repeated.

  Next, the release button 8 is scanned again (S1903d), and it is determined whether the release button 8 has been fully pressed (S1903e). If it is fully pressed (S1903e: Yes), it is determined whether or not the flash mode is set (S1903f).

  As a result, if the flash mode is selected (S1903f: Yes), the flash 7 is projected (S1903g), photographed (S1903h), and if the flash mode is not selected (S1903f: No), the flash 7 is not projected. A picture is taken (S1903h). If it is determined in S1903e that the button has not been fully pressed (S1903e: No), the processing from S1903a is repeated.

  Next, a three-dimensional shape detection process that is the same as the above-described three-dimensional shape detection process (S1006 in FIG. 10) is performed to detect the three-dimensional shape of the subject (S1906).

  Next, based on the three-dimensional shape detection result obtained by the three-dimensional shape detection process (S1906), a document attitude calculation process for calculating the attitude of the document P is performed (S1907). With this process, the position L, the angle θ, and the curvature φ (x) of the document P with respect to the image input apparatus 1 are calculated as the posture parameters of the document P.

  Next, based on the calculation result, a plane conversion process to be described later is performed (S1908), and a flattened image flattened in a state where the original P is not curved even if it is curved is generated.

  Next, the planarized image obtained by the planar change process (S1908) is stored in the external memory 27 (S1909), and the planarized image is displayed on the monitor LCD 10 (S1910).

  Then, it is determined whether or not the mode change switch 9 has changed (S1911). If there is no change (S1911: Yes), the processing from S702 is repeated again, and if there is a change (S1911: No). ), The process ends.

  FIG. 20 is a diagram for explaining the document orientation calculation process (S1907 in FIG. 19). Note that, as an assumption condition for a document such as a book, it is assumed that the curvature of the document P is uniform in the y direction. In this document orientation calculation process, first, as shown in FIG. 20A, regression line approximation is performed on points arranged in two columns at a three-dimensional space position from the coordinate data related to the code boundary stored in the three-dimensional coordinate storage unit 37h. Obtain the two curves.

  For example, it can be obtained from position information (a boundary relating to the boundary between the code 63 and the code 64 and the boundary between the code 191 and the code 192) of the upper and lower quarters of the projected range of the pattern light.

  Assuming a straight line connecting points where the positions of the two curves in the X-axis direction are “0”, the point where the straight line intersects the Z-axis, that is, the point where the optical axis intersects the original P, is 3 A dimension space position (0, 0, L) is defined, and an angle formed by the straight line with the XY plane is defined as an inclination θ around the X axis of the document P.

  Next, as shown in FIG. 20B, the document P is rotationally converted in the reverse direction by the inclination θ around the X axis obtained earlier, that is, the document P is parallel to the XY plane. Assuming that

  Then, as shown in FIG. 20C, the displacement in the Z-axis direction can be expressed as a curve φ (X) as a function of X for the cross section of the document P in the XZ plane. Thus, the position L, the angle θ, and the curvature φ (x) of the document P are calculated as the document orientation parameters, and the process ends.

  FIG. 21 is a flowchart of the plane conversion process (S1908 in FIG. 19). In this process, first, a processing area of the process is assigned to the working area 37m of the RAM 37, and a variable of the counter b used for the process is set to an initial value (b = 0) (S2101).

  Next, based on the position L of the original P, the inclination θ, and the curvature φ (x) based on the calculation result in the original attitude calculation program 36h, the pattern light no-image 4 stored in the pattern light no-image storage unit 37b. Each corner point is obtained by inverse transformation of the curve (a process equivalent to a “curving process” described later) that moves −L in the Z direction, rotates −θ in the X-axis direction, and further changes to φ (x). A rectangular area formed by dots (that is, a rectangular area that is an image in which the surface of the original P on which characters or the like are written is observed from a substantially orthogonal direction) is set, and pixels included in the rectangular area The number a is obtained (S2102).

  Next, coordinates on the non-patterned light image corresponding to each pixel constituting the set rectangular area are obtained, and pixel information of each pixel of the planarized image is set from pixel information around this coordinate.

  That is, first, it is determined whether or not the counter b has reached the pixel number a (S2103). If the counter b has not reached the pixel number a (S2103: No), a curve calculation process is performed in which one pixel constituting the rectangular area is rotated and moved by a curve φ (x) around the Y axis (S2104). A tilt θ is rotated about the X axis (S2105), and is shifted by a distance L in the Z axis direction (S2106).

  Next, the coordinates (ccdcx, ccdcy) on the CCD image obtained by imaging the ideal camera with the inverse function of the previous triangulation are obtained from the obtained three-dimensional spatial position (S2107), and the imaging optical system used According to the aberration characteristics of 20, the coordinates (ccdx, ccdy) on the CCD image captured by the actual camera are obtained by the inverse function of the previous camera calibration (S2108), and the pixel of the pattern light no image corresponding to this position Is stored in the working area 37m of the RAM 37 (S2109).

  Then, “1” is added to the counter b in order to execute the above-described processing from S2103 to S2109 for the next pixel (S2110).

  Thus, when the processing from S2104 to S2110 is repeated until the counter b reaches the number of pixels a (S2103: Yes), the processing area allocated to the working area 37m is released in S2101 to execute the processing (S2111). ), The process ends.

  FIG. 22A is a diagram for explaining the outline of the bending process (S2104 in FIG. 21), and FIG. 22B shows the document P flattened by the plane conversion process (S1908 in FIG. 19). Yes. The details of the curving process are disclosed in detail in the IEICE Transactions DIIVol. J86-D2 No. 3 p409 “Curved Document Shooting with an Eye Scanner”.

  The curve Z = φ (x) is obtained by converting a three-dimensional shape constituted by the obtained code boundary coordinate sequence (real space), a cross-sectional shape parallel to the XZ plane at an arbitrary Y value, and a least square method. Is expressed by an equation approximated by a polynomial.

  When flattening a curved curved surface, as shown in (a), the flattened points corresponding to the points on Z = φ (x) are from Z = φ (0) to Z = φ (x). Are associated with each other according to the length of the curve.

  For example, even when the document P in a curved state as shown in FIG. 1 is imaged by such plane conversion processing including the bending processing, as shown in FIG. 22B, a planarized planar image is obtained. Since the accuracy of the OCR process can be improved by using the flattened image in this way, it is possible to clearly recognize characters, figures, and the like described in the original by the image. .

  Next, with reference to FIG. 23 to FIG. 28, a sample trace mode that is one of operation modes in the image input / output device 1 will be described. In this example trace mode, when a user performs work such as calligraphy according to reference information indicating the standard of work, the work result is compared with the reference information, and the comparison result is projected as an image. It is a mode to do.

  FIG. 23 is a diagram for explaining a first example in the example trace mode. In FIG. 23, as calligraphy reference information that is a work performed by the user, a reference information image R1 that is a model of calligraphy is sent from the image projection unit 14 of the image input / output device 1 to the image input / output device 1. A state in which the image is projected onto the half paper OB1 that is a subject arranged in the imaging area 100 that is an area on the projection plane that can be imaged by the image imaging unit 13 is illustrated. After the user performs a work following the reference information image R1, the user captures the work result and compares the captured image with the reference information image R1. The imaging region 100 is also a surface on which an image can be projected in the projection direction by the image projection unit 14 in the image input / output device 1, that is, an image projection surface. Here, a frame indicating the boundary of the imaging region 100 may be projected from the projection LCD 19 as a projected image. By indicating the boundary of the imaging region 100, the user can clearly grasp the region that can be imaged.

  FIG. 24 is a flowchart of the example trace process (S613 in FIG. 6) executed in this example trace mode. The sample tracing process is a process of comparing captured image data of a subject with reference information such as a model and projecting the result as an image.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S2401). Thereby, a high quality captured image can be provided to the user.

  Next, the reference information and the work result determination program are received via the antenna 11 (S2402), and the received work result determination program is stored in the work result determination program storage unit 37l (S2403), corresponding to the received reference information. The reference information image to be stored is stored in the projection image storage unit 37k (S2404).

  Next, a finder image (an image in a range visible through the finder 6) is displayed on the monitor LCD 10 (S2405). Therefore, the user can confirm the captured image (imaging range) before actual imaging with the image displayed on the monitor LCD 10 without looking into the finder 6.

  Next, a projection process similar to the projection process of S806 in FIG. 8 is performed (S2406). In the projection processing of S2406, since the reference information image is stored in the projection image storage unit 37k, the reference information image is projected into the imaging region 100. Then, the user performs work according to the reference information image.

  For example, according to the first example shown in FIG. 23 described above, when a calligraphic model image is projected onto the half paper OB1 arranged in the imaging region 100 as the reference information image R1, the user Using a brush with ink, a book is written using the reference information image R1 projected on the half paper OB1 as a model.

  Next, the release button 8 is scanned (S2407a), and it is determined whether or not the release button 8 is half-pressed by the user who performed the work following the reference information image (S2407b). If it is half-pressed (S2407b: Yes), the light source driver 29 is turned off (S2407c), and the LED array 17A is turned off by an electrical signal from the light source driver 29 (S2407d). Therefore, for example, in the case of the first example shown in FIG. 23 described above, the reference image R1 projected on the half paper OB1 is not projected.

  Next, the auto focus (AF) and auto exposure (AE) functions are activated, and the focus, aperture, and shutter speed are adjusted (S2407e). If not half-pressed (S2407b: No), the processing from S2407a is repeated.

  Next, the release button 8 is scanned again (S2407f), and it is determined whether or not the release button 8 has been fully pressed (S2407g). If it is fully pressed (S2407g: Yes), it is determined whether or not the flash mode is set (S2412).

  As a result, if the flash mode is set (S2412: Yes), the flash 7 is turned on (S2413), and the subject in the imaging area 100 is shot (S2414). On the other hand, if the flash mode is not set (S2412: No), the image is shot without hitting the flash 7 (S2414). If it is not fully depressed in the determination of S2407g (S2407g: No), the processing from S2407a is repeated.

  Next, the captured image is transferred from the CCD 22 to the cache memory 28 (S2415), and the captured image stored in the cache memory 28 is displayed on the monitor LCD 10 (S2416). In this way, by transferring the captured image to the cache memory 28, the captured image can be displayed on the monitor LCD 10 at a higher speed than when transferring to the main memory.

  Next, the captured image data of the subject imaged in the process of S2414 is determined by the work result determination program stored in the work result determination program storage unit 37l (S2417). In the processing of S2417, the reference information is compared with the captured image data of the subject according to the work result determination program, the comparison result is acquired, and comparison result information such as correction information and evaluation information is generated according to the comparison result. To do.

  The “correction information” is information regarding a point where the subject should be corrected with respect to a portion related to a difference from the reference information as a result of comparing the reference information with the captured image data of the subject. The “evaluation information” indicates the evaluation of the subject when the reference information is used as a reference as a result of comparing the reference information with the captured image data of the subject.

  Next, the comparison result information image corresponding to the generated comparison result information is stored in the projection image storage unit 37k (S2418). In the process of 2418, the comparison result information image may be stored so as to overwrite the reference information image stored in the projection image storage unit 37k by the process of S2404, or the projection image storage unit 37k by the process of S2404. May be stored in association with the reference information image stored in.

  Next, a projection process similar to the projection process of S806 in FIG. 8 is performed (S2419). In the projection processing of S2419, at least the comparison result information image stored in the projection image storage unit 37k is projected into the imaging region 100. For example, if the comparison result information image is a correction information image corresponding to the correction information, the correction information image is projected, and if the comparison result information image is an evaluation information image corresponding to the evaluation information, the evaluation information image Is projected. Therefore, the user can clearly recognize the defects, completeness, proficiency, and the like of his / her work by visually checking the comparison result information image such as the projected correction information image and evaluation information image.

In the process of S2419, when the comparison result information image is stored together with the reference information image in the projection image storage unit 37k, the reference information image is projected together with the comparison result information image. By projecting the reference information image together with the comparison result information image, it is possible to clearly compare the difference between the reference information that is the standard information for obtaining the comparison result information and the work performed by the operator. it can.

  Next, the release button 8 is scanned again (S2420a), and it is determined whether or not the release button 8 is fully pressed by the user (S2420b). If the release button 8 is not fully pressed (S2420b: No), the processing of S2420a is performed. repeat. On the other hand, if it is fully pressed (S2420b: Yes), it is determined whether or not it is in the flash mode (S2421). If it is in the flash mode (S2421: Yes), the flash 7 is applied (S2422). The subject at is photographed (S2423). On the other hand, if the flash mode is not set (S2421: No), the image is shot without hitting the flash 7 (S2423). In the process of S2420b, before it is determined that the release button 8 is fully pressed, the half-pressed state of the release button is detected, and the auto focus (AF) and automatic exposure (AE) functions are activated, It is assumed that the focus, aperture, and shutter speed are adjusted.

  Next, the captured image captured by the processing of S2423 is transferred from the CCD 22 to the cache memory 28 (S2424), and the captured image transferred to the cache memory 28 is stored in the external memory 27 (S2425). Therefore, the comparison result information can be saved as an image.

  Next, the light source driver 29 is turned off (S2426), and the LED array 17A is turned off by an electric signal from the light source driver 29 (S2427). Therefore, the comparison result information image projected in the imaging region 100 is not projected.

  Next, it is determined whether or not the mode selector switch 9 has changed (S2428). If there is no change (S2428: Yes), the processing from S2402 is repeated. For example, when the reference information is reference information of each process in an operation including one or more processes such as an operation manual, when the processes of S2402 to S2428 are repeatedly executed for each process, the user It is possible to know the comparison result information between the own work and the reference information in each process. Alternatively, the work result corrected in accordance with the correction information can be determined again by the work result determination program, and the processes of S2402 to S2428 can be repeatedly executed until the corrected portion is completely corrected.

  On the other hand, if the result of the determination in S2428 is that there is a change in the mode selector switch 9 (S2428: No), the process ends.

  FIG. 25 is a diagram for explaining a state in which a comparison result information image is projected as a result of the processing of S2419 in the above-described example trace processing. FIG. 25 shows the result of executing the above-described example tracing process for the two-character book M1 written by the user according to the calligraphy example projected as the reference information image R1 shown in FIG. Show. In FIG. 25, the book M1 written on the half paper OB1 is indicated by white letters for easy understanding of the drawing.

  As a result of judging the picked-up image data when the half paper OB1 on which the book M1 is written as a subject is picked up by the work result judging program, there are six places to be corrected compared with the reference information as shown in FIG. The correction information images C1a to C1f are projected. By projecting such correction information images C1a to C1f, the user can obtain the defect of his / her own book and information necessary for correcting the defect.

  In addition to the correction information images C1a to C1f, in FIG. 25, an evaluation information image C2 indicating an evaluation result for the reference information is projected as a comparison result information image obtained by the work result determination program. In the case of FIG. 25, a letter “pass” is projected, which is an evaluation that the book M1 has reached a satisfactory level with respect to the reference information as a model. By projecting such an evaluation information image C2, the user can obtain information on the degree of perfection and proficiency of his / her book.

  FIG. 26 is a diagram for explaining a second example in the above-described example trace mode, and shows a state in which the example trace mode is executed for the creation of a folded paper crane using origami. The reference information in the second example is composed of reference information of a plurality of steps (a plurality of steps) according to the folding order of the folded paper crane.

  In FIG. 26, the origami folded in a diagonal line as the first step arranged in the imaging region 100 is set as the subject OB2, and the reference information image R2a showing the valley fold line as reference information for the next step is set on the subject OB2. Is projected. In addition, the reference information image R2b and the reference information image R2a indicating that the projected reference information image R2a portion is folded in the imaging region 100 as character information correspond to the second step in the folding order of the paper cranes. A reference information image R2c indicating this as character information is projected.

  By visually recognizing these reference information images R2a to R2c, the user works on a manual book to determine what should be done in the next step in a series of “fold paper crane” operations. Can be recognized without confirmation during. Therefore, work labor can be reduced.

  Although not shown, after the user performs work according to the reference information images R2a to R2c, the subject OB2 after the work is imaged as the next subject, and the captured image data is the work result determination program. The comparison result information image is displayed at a predetermined position in the imaging region 100. In that case, only when the folding method is correct, the reference information image corresponding to the next step is projected. On the other hand, when the folding method is incorrect, the correction information image is projected and the correction information is projected. Based on the correction information indicated by the image, the operation may be performed again. In this way, by projecting the comparison result information image for each process in each process unit, the user can recognize the completeness of the work, correction points, etc. in each process unit, and as a result, a plurality of A series of operations consisting of processes can be performed accurately. Further, when the comparison result information image is projected for each process, the reference information image of the corresponding process may be projected together if necessary. As a result, the user can more specifically recognize the completeness and correction points of the work in each process unit.

  FIG. 27 is a diagram for explaining the third example in the above-described example trace mode, in which the example trace mode is executed for the operation of covering the box-like body and tightening the cover with screws. It is a figure which shows a state. FIG. 27A is a diagram in which a reference information image is projected before work, and FIG. 27B is a diagram in which a comparison result information image is projected after work.

  As shown in FIG. 27A, a box-shaped body before being screwed is arranged as a subject OB3 in the imaging region 100, and the next work is to tighten four screws. A reference information image R3a shown as character information is projected into the imaging region 100, and an arrow indicating a position where a screw is to be tightened is projected as a reference information image R3b on the subject OB3.

  The user performs a work following these reference information images R3a to R3b, and after the work is performed, the subject OB3 on which the work has been performed is imaged by the image input / output device 1. The captured image data is determined by the work result determination program in accordance with the above-described example trace processing, and the comparison result information image is displayed at a predetermined position in the imaging region 100 as shown in FIG. The

  FIG. 27B shows a comparison result information image projected when one place where a screw is to be tightened is dropped. As shown in FIG. 27 (b), a correction information image C3a indicating that there is a forgotten tightening of the screw as character information is projected in the imaging region 100, and a place where the forgetting to tighten the screw is on the subject OB3. Is projected as a correction information image C3b.

  The user can appropriately recognize the shortage in the work from the correction information images C3a to C3b as shown in FIG. 27B, and correct work can be performed by correcting the points. It becomes.

  In FIG. 28, in the first example of the above-described example trace mode, the arm member 3 is bent so that the image capturing unit 14 in the image input / output device 1 captures the half paper OB1 that is a subject from the left front. FIG. When the image input / output device 1 is arranged as shown in FIG. 26, the image projection unit 13 also projects a projection image such as a reference image or a comparison result information image on the half paper OB1 from the left front.

  Since the image input / output device 1 can change the imaging direction and the projection direction of the subject by bending the arm member 3, it can be arranged at a position according to the work, thereby improving the work efficiency. . In general, since the user's dominant hand is often right-handed, as shown in FIG. 28, the imaging direction and the projection direction of the image input / output device 1 are set from the left front direction by bending the arm member 3. As a result, it is possible to suppress interference in the execution of work due to the presence of the image input / output device 1 itself.

  Note that the imaging direction and the projection direction of the image input / output device 1 are not limited to the left front as shown in FIG. 28, and are the opposite directions in the top view with respect to the direction in which the work is performed on the subject. The image input / output device 1 can have an effect of suppressing work interference. For example, in the above-described first example, when a left-handed user writes a writing on the half paper OB1, the imaging direction by the image imaging unit 14 and the projection direction by the image projection unit 13 may be set from the right front. .

  As shown in FIG. 28, when the image capturing direction by the image capturing unit 14 is set to a direction other than a direction substantially perpendicular to the surface of the subject, a trapezoidal captured image is obtained. In order to improve, this trapezoidal captured image is corrected to obtain a planarized image that is equivalent to when captured from a substantially vertical direction or planarized to a state corresponding to when captured from a substantially vertical direction It is preferable to do.

  In the example tracing process (S613) shown in FIG. 24, when comparison result information is obtained using a flattened image, after photographing the subject by the process of S2414, before proceeding to the process of S2415. The processes of S2431 to S2434 shown below may be executed.

Specifically, after the process of S2 4 14, the same 3D shape detection process as the 3D shape detection process of FIG. 1 2 (a) is executed (S2431), and then obtained by the 3D shape detection process. three-dimensional shape detection result in a 3-dimensional coordinates (X, Y, Z) and stored in the external memory 27 (S2432), then performs the same original orientation processing and document orientation calculation process S1907 of FIG. 19 (S2433) Next, a planar conversion process similar to the process of S1908 in FIG. 19 is executed to obtain a planarized image (S2434).

  Then, the planarized image data of the subject acquired in the process of S2434 may be determined by the work result determination program stored in the work result determination program storage unit 37l in the process of S2417.

  In addition, after the subject on which the comparison result information image is projected is imaged by the process of S2423 in the example trace process (S613) shown in FIG. 24, the process similar to the above-described S2431 to S2434 is performed before the process of S2424. The captured image may be converted into a flattened image by performing the above. Thereby, the comparison result information can be stored as a non-distorted captured image.

  As described above, in the model tracing process, the captured image is converted into a planarized image, so that the subject has a three-dimensional shape as well as a case where the imaging direction is oblique, and the surface thereof is curved. Even in such a case, it is possible to reduce the error of the comparison result information due to the distortion of the imaging data, thereby obtaining highly accurate comparison result information.

  On the other hand, even when the projection direction by the image projection unit 13 is a direction other than a direction substantially perpendicular to the surface of the subject, the image projected on the subject is distorted. Projection image that cannot be recognized by a person. Therefore, it is preferable to perform correction so that the projected image projected with distortion is projected as an undistorted projected image.

  In the example tracing process (S613) shown in FIG. 24, when an undistorted projection image is projected without depending on the projection direction, the projection process executed in the same manner as the projection process in S806 of FIG. Instead of the process of S902 in (S2406, S2419), a projection image conversion process (S2900) as described later may be executed.

  FIG. 29 is a flowchart of the distortion-free image conversion process (S2900). The distortion-free image conversion process (S2900) is a process for converting an image displayed on the projection LCD 19 in accordance with image information stored in the projection image storage unit 37k into an image that can be projected onto the subject in an undistorted state. is there.

  In this process, first, the processing area of the process is assigned to the working area 37m of the RAM 37, and the variable of the counter q used for the process is set to an initial value (q = 0) (S2901).

  Next, a space of LCD space coordinates (lcdcx, lcdcy) is set as a rectangular area that becomes an image after being transformed into an image for distortion-free projection (that is, an image that is undistorted on a curved subject) The number of pixels Qa included in this rectangular area is obtained (S2902).

  Next, image information such as a comparison result information image and a reference information image stored in the projection image storage unit 37k is arranged at coordinates (ccdcx, ccdcy) on the ideal camera image (S2903).

Next, for each pixel on the LCD space coordinates (lcdcx, lcdccy) constituting the set rectangular area, three- dimensional coordinates (points on the surface of the subject stored in the external memory 27 by the processing of S2432 described above) ( By using (X, Y, Z), the pixel information of each pixel of the image for distortion-free projection is set.

  That is, first, it is determined whether or not the counter q has reached the number of pixels Qa (S2904). If the counter q has not reached the number of pixels Qa (S2904: No), the LCD space coordinates (lcdcx, lcdcy) of the pixel corresponding to the value of the counter q are coordinates on the subject stored in the external memory 27 (X , Y, Z) (S2905).

  Next, the coordinates (X, Y, Z) on the subject obtained by the conversion in the process of S2905 are converted into coordinates (ccdcx, ccdcy) on the ideal camera image (S2906).

  Next, the pixel information arranged at the coordinates (ccdcx, ccdcy) obtained by the conversion in the process of S2906 is acquired, and the pixel information is converted into the LCD space coordinates (lcdcx, lcdcy) corresponding to the value of the counter q. (S2907).

  Then, “1” is added to the counter q in order to execute the above-described processing from S2904 to S2907 for the next pixel (S2908).

  Thus, when the processing from S2904 to S2908 is repeated until the counter q reaches the number of pixels Qa (S2904: Yes), the pixel information associated with the LCD space coordinates (lcdcx, lcdcy) that constitute the set rectangular area Is transferred to the projection LCD driver 30 (S2909).

  Finally, in S2901, the processing area allocated to the working area 37m for executing the process is released (S2910), and the process is terminated.

  By the processing of S2909, the pixel information on the LCD space coordinates (lcdcx, lcdcy) is transferred to the projection LCD driver 30, whereby the projection LCD 19 displays a projected image projected without distortion on the distorted curved surface. Therefore, an undistorted image is projected on the subject.

  Therefore, by executing the distortion-free image conversion process (S2900), not only when the projection direction is oblique, but also when the subject has a three-dimensional shape and the surface thereof is curved. However, an undistorted projection image can be projected. As a result, particularly when a correction information image or an evaluation information image is projected as a comparison result information image, the user can be made to recognize the information accurately.

  FIG. 30 is a view for explaining the light source lens 60 of the second embodiment related to the light source lens 18 of the first embodiment described above, and (a) is a side view showing the light source lens 60 of the second embodiment. (B) is a top view which shows the light source lens 60 of 2nd Example. In addition, the same code | symbol is attached | subjected to the same member as mentioned above, and the description is abbreviate | omitted.

  The light source lens 18 in the first embodiment is configured by integrally arranging the lens portions 18a on the base 18b from a convex aspherical shape corresponding to each LED 17, whereas in the second embodiment The light source lens 50 is configured by separately forming a resin lens formed in a bullet shape including each of the LEDs 17.

  In this way, by configuring the light source lenses 50 including the respective LEDs 17 separately, the positions of the respective LEDs 17 and the corresponding optical lenses 50 are determined on a one-to-one basis. The accuracy can be improved and the light emission direction is aligned.

  On the other hand, when the lens array is collectively aligned on the substrate 16, the light emission direction varies due to the positioning error when each LED 17 is die-bonded and the difference between the linear expansion coefficients of the lens array and the substrate. There is a risk of becoming.

  Therefore, the surface of the projection LCD 19 is irradiated with light whose light incident directions from the LEDs 17 are aligned perpendicularly to the surface of the projection LCD 19 so that it can pass through the aperture of the projection optical system 20 uniformly. Irradiance unevenness can be suppressed, and as a result, a high-quality image can be projected. The LED 17 included in the light source lens 50 is mounted on the substrate 16 via an electrode 51 composed of a lead and a reflector.

  Further, on the outer peripheral surface of the group of light source lenses 50 in the second embodiment, a fixing member 52 having a frame-like elasticity that binds the light source lenses 50 and restricts them in a predetermined direction is disposed. The fixing member 52 is made of a resin material such as rubber or plastic.

  Since the light source lens 50 of the second embodiment is configured separately from each LED 17, the angle of the optical axis formed by the convex tip of each light source lens 50 is correctly aligned and faces the projection LCD 19. It is difficult to install.

  Therefore, the group of light source lenses 50 is surrounded by the fixing member 52, the outer peripheral surfaces of the light source lenses 50 are brought into contact with each other, and the light source lenses 50 are arranged so that the optical axis of each light source lens 50 faces the projection LCD 19 at a correct angle. By restricting the position, the light can be irradiated from each light source lens 50 toward the projection LCD 19 substantially vertically. Therefore, the light aligned in the direction perpendicular to the surface of the projection LCD 19 is irradiated so that it can pass through the aperture of the projection lens uniformly, so that unevenness in the illuminance of the projection image can be suppressed. Therefore, a higher quality image can be projected.

  The fixing member 52 may have a rigidity defined in advance to a predetermined size. The fixing member 52 is made of a material having an elastic force, and the position of each light source lens 50 is set to a predetermined position by the elastic force. You may make it regulate to.

  FIG. 31 is a view for explaining a second embodiment relating to the fixing member 52 for restricting the light source lens 50 described in FIG. 30 to a predetermined position, and (a) is a perspective view showing a state in which the light source lens 50 is fixed. (B) is a partial sectional view thereof. In addition, the same code | symbol is attached | subjected to the same member as mentioned above, and the description is abbreviate | omitted.

  The fixing member 60 of the second embodiment is formed in a plate shape in which a through-hole 60a having a conical cross section having a cross section along the outer peripheral surface of each light source lens 50 is formed. Each light source lens 50 is inserted and fixed in each through hole 60a.

  Further, an urging plate 61 having elasticity is interposed between the fixing member 60 and the substrate 16, and an electrode 51 is provided between the urging plate 61 and the lower surface of each light source lens 50. An annular O-ring 62 having elasticity is arranged so as to surround the outer periphery.

  The LED 17 included in the light source lens 50 is mounted on the substrate 16 via an urging plate 61 and an electrode 51 that penetrates a through hole formed in the substrate 16.

  According to the fixing member 60 described above, each light source lens 50 is fixed by being passed through each through hole 60a having a cross section along the outer peripheral surface of the light source lens. The optical axis of the light source lens 50 can be fixed so as to face the projection LCD 19 at a correct angle.

  Further, at the time of assembly, the LED 17 can be urged and fixed to the correct position by the urging force of the O-ring 62.

  Further, an impact force that may occur when the apparatus 1 is transported can be absorbed by the elastic force of the O-ring 62, and the position of the light source lens 50 is shifted due to the impact, and the light source lens. The inconvenience that light cannot be irradiated from 50 to the projection LCD 19 vertically can be prevented.

  In the above-described embodiment, the reference information acquisition means described in claim 1 corresponds to the process of S2402 in the example trace process of FIG. 24, and the reference information projection means includes S2406 and S2419 in the example trace process of FIG. This projection processing is applicable.

  Further, the comparison result information acquisition means according to claims 2 and 3 corresponds to the processing of S2417 in the example trace process of FIG. 24, and the comparison result projection means includes the projection of S2419 in the example trace process of FIG. Processing is applicable.

  The three-dimensional information generating means described in claim 4 corresponds to the above-described processes of S2431 to S2432, and the planar image correcting means corresponds to the above-described processes of S2433 to 2434.

  The spatial modulation control means according to claim 5 corresponds to the distortion-free image conversion processing of FIG.

  Further, the reference information projecting means according to claim 8 corresponds to the projecting process of S2419 in the model trace process of FIG.

  Although the present invention has been described based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be easily made without departing from the gist of the present invention. It can be done.

  For example, in the above-described embodiment, the process of acquiring and displaying a planarized image has been described as the planarized image mode. However, a known OCR function is installed, and the planarized planar image is read by this OCR function. You may comprise as follows. In such a case, it is possible to read the text written on the document with higher accuracy than when reading a curved document by the OCR function.

  Further, in S1501 of FIG. 15 in the above-described embodiment, a case has been described in which all luminance images having light and dark changes are extracted and provisional CCDY values are obtained for all of them. It is not necessary that the number of sheets is limited to one or more. The boundary coordinates can be obtained at high speed by reducing the number of extractions.

  Further, in S1507 of FIG. 15 in the above embodiment, fCCDY [i] is weighted and averaged, and in S1607 of FIG. 16 efCCDY [j] is an approximate polynomial, each value is averaged. The averaging method is not limited to these. For example, a method of taking a simple average value of each value, a method of employing a median value of each value, calculating an approximate expression of each value, and calculating the approximate expression For example, a method of using the detection position in the boundary coordinates as a boundary coordinate, a method of obtaining by statistical calculation, or the like may be used.

  Further, for example, in the three-dimensional shape detection process in the planar image mode in the above embodiment, in order to detect the three-dimensional shape of the document P, a striped pattern light in which a plurality of types of light and dark are alternately arranged is projected. Although the case has been described, the light for detecting the three-dimensional shape is not limited to such pattern light.

  For example, as shown in FIG. 32, when the three-dimensional shape of a curved document is simply detected, two strip-shaped slit lights 70 and 71 may be projected from the image projection unit 13. In this case, a three-dimensional shape can be detected at high speed from only two captured images, as compared with the case of projecting eight pattern lights.

Further, in the process of S2419 of the sample trace process described with reference to FIG. 24, if the reference information of one work is composed of reference information of a plurality of processes, the comparison result obtained for each process You may comprise so that an information image and the reference information image corresponding to the process may be successively projected by a moving image. For example, in the case of creating a calligraphy calligraphy as one work, if the reference information is model information for each order in the book, in the first example of the model trace mode described above, The comparison result information for M1 may be divided according to the document order of the book M1, and sequentially projected as a moving image together with the model information image of the corresponding book order. As a result, similarities and differences between the user's work and the reference information can be recognized more clearly.

1 is an external perspective view of an image input / output device. It is a figure which shows the internal structure of an imaging head. (A) is an enlarged view of an image projection part, (b) is a top view of a light source lens, (c) is a front view of projection LCD19. It is a figure for demonstrating regarding the arrangement | sequence of a LED array. It is an electrical block diagram of an image input / output device. It is a flowchart of a main process. It is a flowchart of a digital camera process. It is a flowchart of a webcam process. It is a flowchart of a projection process. It is a flowchart of a three-dimensional image process. (A) is a figure for demonstrating the principle of the space code method, (b) is a figure which shows the mask pattern (gray code) different from (a). (A) is a flowchart of a three-dimensional shape detection process. (B) is a flowchart of an imaging process. (C) is a flowchart of a three-dimensional measurement process. It is a figure for demonstrating the outline of a code boundary coordinate detection process. It is a flowchart of a code boundary coordinate detection process. It is a flowchart of the process which calculates | requires a code boundary coordinate with subpixel precision. It is a flowchart of the process which calculates | requires the boundary CCDY value about the brightness | luminance image with the mask pattern number of PatID [i]. It is a figure for demonstrating a lens aberration correction process. It is a figure for demonstrating the method of calculating the three-dimensional coordinate in a three-dimensional space from the coordinate in CCD space. It is a flowchart of planarization image processing. FIG. 6 is a diagram for explaining document orientation calculation processing. It is a flowchart of a plane conversion process. (A) is a figure for demonstrating the outline about a curvature calculation process, (b) is a figure which shows the planarization image planarized by the plane conversion process. It is a figure for demonstrating the 1st example in a sample trace mode. It is a flowchart of a sample trace process. It is a figure explaining the state by which the comparison result information image was projected in the example trace process. It is a figure explaining the 2nd example in a sample trace mode. It is a figure explaining the 3rd example in a model trace mode, and is a figure showing the state where the model trace mode is performed for the operation which covers a box-like body and screws the lid, (A) is a figure in which a reference information image is projected before work, and (b) is a figure in which a comparison result information image is projected after work. FIG. 6 is a diagram illustrating a state in which the image input / output device captures an image of a subject from the left front by bending an arm member in the first example of the sample trace mode. It is a flowchart of the image conversion process for distortion-free projection. (A) is a side view which shows the light source lens 60 of 2nd Example, (b) is a top view which shows the light source lens 60 of 2nd Example. (A) is a perspective view which shows the state which fixed the light source lens 50, (b) is the fragmentary sectional drawing. It is a figure which shows the other example as pattern light projected on a to-be-photographed object.

Explanation of symbols

3 Arm member (movable means)
13 Image projection unit (projection unit)
14 Image capturing unit (imaging unit)
17 LED (light source means)
19 Projection LCD (spatial modulation means)
20 Projection optical system 21 Imaging optical system (part of imaging means)
22 CCD (part of imaging means)