JP2005354306A - Image input output apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image input output apparatus capable of imaging an object existing in a direction wherein a projection means projects image signal light, an object existing in a direction different from the direction wherein the projection means projects the image signal light, and providing a display output of optional information in the projection direction. <P>SOLUTION: Revolving an imaging case 11 around an axis center of an imaging head 2 relatively revises an imaging direction by an image imaging section 14 fixed to the imaging case 11 with respect to a projection direction by an image projection section 13 fixed to the imaging head 2. Since the image imaging section 14 can be opposed to the object existing in the direction different from the projection direction, the image input output apparatus can image even the object existing in the direction different from the direction wherein the image projection section 13 projects the image signal light as well as the object existing in the direction wherein the image projection section 13 projects the image signal light. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention can capture a subject existing in a direction in which the image signal light is projected by the projection unit and a subject present in a direction different from the direction in which the image signal light is projected by the projection unit. The present invention relates to an image input / output device that can display and output the above information.

Conventionally, in an image input / output device capable of capturing an image of a subject placed on a surface and acquiring the captured image data, light is applied to the imageable region on the surface in order to visually display the imageable region. There is known one provided with a projection means for projecting. The projection means provided in such an image input / output device includes a liquid crystal panel that performs spatial modulation on the light emitted from the light source, and controls the transmission of the light emitted from the light source for each pixel, thereby capturing the image. It is comprised so that the spot light which shows an area | region can be projected.
JP-A-8-32848

  However, in the image input / output device described in Patent Document 1, the relative positional relationship between the projection unit and the imaging unit is fixed, and the subject exists in the direction in which the image signal light is projected by the projection unit. However, there is a problem that the use of the image input / output device is limited.

  For example, the user may point the imaging means toward himself and try to image the user himself. In that case, the projection means may be used, for example, in a direction that does not include the imaging direction of the subject, for example, on a desk Information cannot be displayed and output with the projection direction of the projection means directed.

  The present invention has been made in order to solve the above-described problems. A subject existing in a direction in which image signal light is projected by the projection unit and a direction different from the direction in which image signal light is projected by the projection unit. It is an object of the present invention to provide an image input / output device capable of capturing an image of a subject existing in the camera and displaying and outputting arbitrary information in the projection direction.

  In order to achieve this object, an image input / output apparatus according to claim 1 is a light source means for emitting light, and spatial modulation for spatially modulating light emitted from the light source means to output arbitrary image signal light. Means for projecting image signal light output by the spatial modulation means in the projection direction, and imaging means capable of capturing an image of a subject existing at least in the projection direction and acquiring the image data The imaging direction of the imaging unit is relatively relative to the projection direction of the projection unit so that the subject existing in a direction different from the projection direction can be captured by the imaging unit. It can be changed.

  According to the image input / output apparatus of the first aspect, the imaging direction of the imaging unit and the projection direction of the projection unit can be arbitrarily changed relatively, and arbitrary image signal light is output by the spatial modulation unit. Since the image signal light is projected in the projection direction by the projection means, arbitrary information can be displayed and output by the projection means in a direction different from the imaging direction.

  The image input / output device according to claim 2 is the image input / output device according to claim 1, wherein the spatial modulation unit includes a liquid crystal panel, and the projection unit outputs image signal light output from the liquid crystal panel to a predetermined level. A projection optical system that forms an image on a projection surface, and the imaging unit includes a light receiving element that converts light into an electrical signal and an imaging optical system that forms an image of incident light on the light receiving element; And a projection housing for holding the projection means, and the imaging means has an imaging housing for holding the imaging means disposed so as to be movable relative to the projection housing.

  According to the image input / output device described in claim 2, the image input / output device operates in the same manner as the image input / output device described in claim 1, and the projection means forms image signal light output from the liquid crystal panel on a predetermined projection plane. The imaging means has an imaging optical system that forms an image of incident light on the light receiving element. In general, the light receiving elements provided in the imaging means are often smaller than the liquid crystal panel provided in the projection means. In such a case, the imaging optical system included in the imaging unit is configured to be smaller than the projection optical system included in the projection unit. Furthermore, the projection means is provided with a light source means. Therefore, the imaging housing that holds the imaging means can be easily configured smaller than the projection housing that holds the projection means, and the imaging housing is disposed so as to be movable relative to the projection housing. .

  The image input / output device according to claim 3 is the image input / output device according to claim 2, wherein the projection means includes a projection direction determining means for projecting light emitted from the light source means in a predetermined projection direction. Have.

  The image input / output device according to claim 4 is the image input / output device according to claim 1, wherein the projection unit and the imaging unit are supported by at least one of the projection unit and the imaging unit with respect to the substrate. And a support member that movably supports the position.

  The image input / output device according to claim 5 is the image input / output device according to claim 4, wherein one end side of the support member is connected to at least one of the projection unit and the imaging unit, and the other end side. And an attachment portion for attaching the support member to the base, and at least one of the base and the support member is provided with a power supply for supplying power to at least one of the projection means and the imaging means. It has been.

  6. The image input / output device according to claim 6, wherein the image input / output device according to any one of claims 1 to 5 is configured to project an image signal light by the projection unit, and execute imaging by the imaging unit. Control means are provided.

  The image input / output device according to claim 7 is the image input / output device according to claim 6, wherein the projection imaging control unit controls the projection unit, thereby relating information relating to a captured image acquired by the imaging unit. Is projected by the projection means.

  Here, the related information related to the imaging unit corresponds to, for example, a character string or a figure indicating the operation method, operation procedure, and imaging mode of the imaging unit.

  The image input / output device according to claim 8 is the image input / output device according to claim 6, wherein the projection imaging control means controls the projection means to obtain a captured image of the subject imaged by the imaging means. Projecting is performed by the projecting means.

  An image input / output device according to a ninth aspect of the present invention is the image input / output device according to any one of the sixth to eighth aspects, further comprising a projected image update means for updating image signal light projected by the projection means.

  The image input / output device according to claim 10 is the image input / output device according to any one of claims 1 to 9, wherein image signal light having a predetermined pattern is projected onto the subject by the projection means. In addition, the apparatus includes a three-dimensional information detection unit that detects three-dimensional information of the subject based on a captured image captured by the imaging unit.

  The image input / output device according to claim 11 is the image input / output device according to any one of claims 1 to 10, wherein a relative positional relationship between the imaging unit and the projection unit is within a predetermined range. Position determining means for determining whether or not.

  The image input / output device according to claim 12 is the image input / output device according to claim 11, wherein the image pickup means and the projection means are in a state in which a relative positional relationship between the image pickup means and the projection means is in the predetermined position relationship. And a fixing means for fixing the positional relationship between the projection means.

  An image input / output device according to a thirteenth aspect of the present invention is the image input / output device according to the tenth aspect, further comprising a position determination unit that determines a relative positional relationship between the imaging unit and the projection unit. Therefore, when it is determined that the relative positional relationship between the imaging unit and the projecting unit is not within a predetermined range, the three-dimensional information that prohibits the detection of the three-dimensional information by the three-dimensional information detection unit Detection prohibiting means.

  The image input / output device according to claim 14 is the image input / output device according to any one of claims 1 to 13, wherein the resolution is reduced and transmitted to the outside by reducing the resolution of the captured image acquired by the imaging means. Have

  According to the image input / output device of claim 1, when the light emitted from the light source means is spatially modulated by the spatial modulation means and output as arbitrary image signal light, the arbitrary image is displayed. The signal light is projected in the projection direction by the projection means. Here, the imaging direction of the imaging means and the projection direction of the projection means can be relatively changed, and any image signal light is projected in the projection direction by the projection means, so that the image signal light is projected by the projection means. A subject existing in the projected direction and a subject present in a direction different from the direction in which the image signal light is projected by the projection unit can be imaged, and arbitrary information can be displayed and output in the projection direction. effective.

  According to the image input / output device of the second aspect, in addition to the effect produced by the image input / output device according to the first aspect, the imaging housing can be easily configured smaller than the projection housing, and the imaging housing Is disposed so as to be relatively movable with respect to the projection housing, so that the imaging means held by the imaging housing and the projection means held by the projection housing can be easily moved relatively. There is.

  According to the image input / output device of the third aspect, in addition to the effect produced by the image input / output device according to the second aspect, the projection unit projects the light emitted from the light source unit in a predetermined projection direction. Therefore, it is not necessary to provide the light source means, the spatial modulation means, and the projection optical system on a straight line, and the projection means can be provided in a compact manner according to the shape of the projection casing, thereby reducing the size of the projection casing. There is an effect that can be done.

  According to the image input / output device according to claim 4, in addition to the effect of the image input / output device according to claim 1, it is possible to move the position of at least one of the projection unit and the imaging unit with respect to the base body. Even when the projection unit and the imaging unit are supported by the base, there is an effect that flexibility can be provided in the direction in which the image signal light can be projected by the projection unit and the direction in which the imaging unit can capture the image. .

  According to the image input / output device of the fifth aspect, in addition to the effect of the image input / output device according to the fourth aspect, the power supply unit for supplying electric power to at least one of the projection unit and the imaging unit is the base and the support. Since it is provided on at least one of the members, the projection unit and the imaging unit can be reduced in weight, and there is an effect that the operation of moving the position of the projection unit and the imaging unit with respect to the base becomes easy. In addition, since the projection unit and the imaging unit are reduced in weight, there is an effect that the strength of the support member is not so required as compared with the case where the power supply unit is provided in the projection unit and the imaging unit.

  According to the image input / output device of the sixth aspect, in addition to the effect produced by the image input / output device according to any one of the first to fifth aspects, the image signal light is projected by the projecting unit and the image is captured by the image capturing unit. Executed. Here, since the imaging means can capture a subject that exists in a direction different from the projection direction, for example, the image signal light is projected in a direction different from the direction in which the user is present, and used by the imaging means. By imaging the person himself / herself, there is an effect that the user can image himself / herself while visually recognizing the image projected by the projection means. Further, for example, when an image of a subject that exists in a direction different from the projection direction is picked up by the image pickup means, the image signal light is not projected onto the subject, so that the image of the subject is not included without including the image projected by the projection means. There is an effect that imaging can be performed.

  According to the image input / output device of the seventh aspect, in addition to the effect produced by the image input / output device according to the sixth aspect, the related information regarding the captured image acquired by the imaging unit is projected by the projection unit. Has an effect that imaging can be executed while visually recognizing related information regarding the captured image acquired by the imaging means.

  According to the image input / output device of the eighth aspect, in addition to the effect produced by the image input / output device according to the sixth aspect, the captured image of the subject imaged by the imaging means is projected by the projection means. There is an effect that imaging can be executed while visually recognizing the captured image of the subject imaged by the imaging means.

  According to the image input / output device of the ninth aspect, in addition to the effect produced by the image input / output device according to any one of the sixth to eighth aspects, the image signal light projected by the projecting means is updated. There is an effect that the person can execute imaging while visually checking the update of the image projected by the projection unit.

  According to the image input / output device of the tenth aspect, in addition to the effect produced by the image input / output device according to any one of the first to ninth aspects, the image signal light having a predetermined pattern is projected onto the subject by the projection means. In the image input / output device in which the three-dimensional information of the subject is detected based on the captured image captured by the image capturing unit when projected, the image capturing unit and the projecting unit are moved relative to each other to perform projection. Since it is configured so that a subject existing in a direction different from the direction can be imaged, the application of the projection unit and the imaging unit provided for detecting three-dimensional information in the image input / output device is expanded. There is.

  According to the image input / output device described in claim 11, in addition to the effect produced by the image input / output device according to claim 10, the position determination means causes the relative positional relationship between the imaging means and the projection means to be within a predetermined range. There is an effect that it can be determined that there is no relationship.

  According to the image input / output device according to claim 12, in addition to the effect achieved by the image input / output device according to claim 11, the relative positional relationship between the imaging unit and the projection unit is in a predetermined positional relationship, There is an effect that the positional relationship between the imaging unit and the projection unit can be fixed by the fixing unit.

  According to the image input / output device described in claim 13, in addition to the effect produced by the image input / output device according to claim 10, the position determination means causes the relative positional relationship between the imaging means and the projection means to be within a predetermined range. If it is determined that the relationship is not relevant, the detection of the three-dimensional information by the three-dimensional information detecting means is prohibited, so that it is possible to reliably detect highly accurate three-dimensional information. That is, the three-dimensional information detection unit detects the three-dimensional information based on the captured image captured by the imaging unit when image signal light having a predetermined pattern is projected onto the subject by the projection unit. Therefore, the relative positional relationship between the imaging unit and the projecting unit is not within a predetermined range, and the image signal light having a predetermined pattern cannot be projected onto the subject by the projecting unit, or the tertiary is highly accurate. When the original information cannot be obtained, the detection of the three-dimensional information is prohibited, so that the detection of the three-dimensional information with low accuracy can be suppressed.

  According to the image input / output device according to claim 14, in addition to the effect produced by the image input / output device according to any one of claims 1 to 13, the resolution of the captured image acquired by the imaging unit is lowered and externally provided. Since it is transmitted, the processing speed can be improved.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is an external perspective view of an image input / output device 1 according to the present invention, where (a) is a view as seen from the upper surface side, and (b) is an enlarged view of an imaging head 2.

  The image input / output device 1 includes a digital camera mode that functions as a digital camera, a webcam mode that functions as a web camera, a stereoscopic image mode for detecting a three-dimensional shape and acquiring a stereoscopic image, and a planarized flat surface of a curved document. In addition to a device equipped with various modes such as a planarized image mode for acquiring a digitized image, it is a device capable of projecting an arbitrary image. Furthermore, the image input / output device 1 is configured to capture not only a subject positioned in the direction in which the image is projected, but also a subject positioned in a direction different from the direction in which the image is projected.

  In FIG. 1, in particular, in the stereoscopic image mode and the planarized image mode, in order to detect the three-dimensional shape of the subject P, striped pattern light (predetermined pattern) formed by alternately arranging light and dark from the image projection unit 13 described later. A state of projecting image signal light having a light beam is illustrated.

  The image input / output device 1 includes an imaging head 2 and an arm member 3 having one end detachably connected to the imaging head 2, and a laptop personal computer 49 is provided at the other end of the arm member 3. (Hereinafter simply referred to as “PC49”) includes an attachment portion 4 to which the imaging head 2 and the arm member 3 can be attached.

  The imaging head 2 is a case that includes an image projection unit 13 (described later) inside and holds a cylindrical imaging case 11 that is rotatable about an axial center on the outside thereof. On one surface of the imaging head 2, a cylindrical lens barrel 5 is disposed at the center. In the following description, the surface on which the lens barrel 5 is provided is the front surface of the imaging head 2.

  The lens barrel 5 is a member that protrudes from the front surface of the imaging head 2 and encloses the projection optical system 20 that is a part of the image projection unit 13 therein. The projection optical system 20 is held by the lens barrel 5 and can be moved as a whole for focus adjustment. Further, from the end surface of the lens barrel 5, a part of the lens of the projection optical system 20 that is a part of the image projection unit 13 is exposed on the outer surface, and image signal light is projected from the exposed part toward the projection surface. The

  The imaging case 11 is provided with a white balance sensor 6 and a flash 7. Further, between the white balance sensor 6 and the flash 7, a part of the lens of the imaging optical system 21 which is a part of the image imaging unit 14 described later is exposed on the outer surface and faces the imaging optical system 21. A subject is imaged.

  The flash 7 is a light source for supplementing necessary illuminance in the digital camera mode, for example, and includes a discharge tube filled with xenon gas. Further, it can be repeatedly used by discharging from a capacitor (not shown) built in the imaging head 2.

  A monitor LCD 10 is disposed on the back surface of the imaging head 2. The monitor LCD 10 is composed of a liquid crystal display and receives an image signal from the processor 15 described later and displays an image to the user. For example, the monitor LCD 10 displays a captured image in the digital camera mode or the webcam mode, a three-dimensional shape detection result image in the stereoscopic image mode, a planarized image in the planarized image mode, or the like.

  Furthermore, a connecting member 12 that detachably connects the imaging head 2 and the arm member 3 is disposed on the side surface of the imaging head 2. In the image input / output device 1, the imaging head 2 portion can be used as a simple mobile digital camera.

  The connecting member 12 is formed in a ring shape and is fixed to the side surface of the imaging head 2. Further, a portion that fits the connecting member 12 is formed on one end side of the arm member 3. With this fitting, the imaging head 2 and the arm member 3 can be connected, and the imaging head 2 can be fixed to the PC 49 at an arbitrary angle. Further, the arm member 3 and the imaging head 2 can be separated by releasing the fitting.

  The arm member 3 is for holding the imaging head 2 so that the orientation of the imaging head 2 can be changed with respect to the PC 49, and is constituted by a bellows-like pipe that can be bent into an arbitrary shape. Accordingly, since the imaging head 2 can be directed to an arbitrary position, the direction in which an image can be projected and the direction in which an image can be captured by the image input / output device 1 can be arbitrarily set even when attached to the PC 49.

  Further, as described above, the arm member 3 has one end connected to the imaging head 2 via the connecting member 12 and the other end attached to the attachment portion 4 for detachably attaching the arm member 3 to the PC 49. Is provided.

  The mounting portion 4 includes a battery 26, an electronic circuit board, and the like which will be described later. Further, the mounting portion 4 is provided with a release button 8 on the front side, an on / off switch 9a above the release button 8, a mode changeover switch 9b below, and a connector for the personal computer interface 24. . The image input / output device 1 is connected to a PC 49 via a cable (not shown) connected to the personal computer interface 24.

  The release button 8 is composed of a two-stage push button switch that can be set in two types of states, “half-pressed state” and “full-pressed state”. The state of the release button 8 is managed by a processor 15 which will be described later, and in the “half-pressed state”, the well-known autofocus (AF) and automatic exposure (AE) functions are activated, and the focus, aperture, and shutter speed are adjusted. Imaging or the like is performed in the “fully pressed state”.

  The on / off switch 9a switches the main power source of the image input / output device 1 on and off by pressing this switch.

  The mode switch 9b is a switch that can be set to various modes such as a digital camera mode, a webcam mode, a stereoscopic image mode, a planarized image mode, and an off mode. The state of the mode changeover switch 9b is managed by the processor 15, and processing of each mode is executed when the state of the mode changeover switch 9b is detected by the processor 15.

  FIG. 2 is a diagram illustrating an internal configuration of the imaging head 2. As shown in FIG. 2A, the imaging head 2 includes a lid portion 2a and a bottom portion 2b. The imaging case 11 is a cylindrical case containing an image imaging unit 14 described later, and a part of the lens of the imaging optical system 21 that is a part of the image imaging unit 14 is exposed on the outer peripheral surface thereof. In addition, a plurality of ribs 11a are provided upright.

  FIG. 2B is a diagram illustrating a state in which the lid 2 a of the imaging head 2 is removed and the image projection unit 13 included in the imaging head 2 is removed. As shown in FIG. 2B, the bottom 2b is provided with an imaging head position detection sensor S1, an imaging unit position detection sensor S2, and a hole 2c.

  The imaging head position detection sensor S1 is a biaxial acceleration sensor for detecting the tilt of the imaging head 2, and the projection direction of the image projection unit 13 fixed to the imaging head 2 (the optical axis of the projection optical system 20 described later). It is detected by two axes how many times the direction is inclined with respect to the vertical direction, and is output as an electric signal.

  The imaging unit position detection sensor S <b> 2 is a sensor for detecting the position of the image imaging unit 14 with respect to the imaging head 2.

  The hole 2c is a substantially circular hole formed on the surface constituting the front surface of the imaging head 2, and is threaded. The lens barrel 5 (see FIG. 1) is screwed into the hole 2c. As will be described later, the image projection unit 13 included in the imaging head 2 projects an image from a lens barrel 5 that is movably screwed into the hole 2c.

  The imaging case 11 has a small-diameter portion 11b that is a cylindrical member having a smaller diameter than the imaging case 11 on both end faces, and an imaging-unit position detection rib 11c is provided on one of the small-diameter portions 11b.

  The small-diameter portion 11b is supported by a notch provided in the bottom portion 2b so as to be rotatable around the axis. Thereby, the imaging case 11 is held so as to be rotatable relative to the imaging head 2.

  The imaging portion position detection rib 11 c is fixed to one of the small diameter portions 11 b and rotates around the axis of the imaging case 11 as the imaging case 11 rotates. In the vicinity of the imaging unit position detection rib 11c, the above-described imaging unit position detection sensor S2 fixed to the bottom 2b is located. By detecting the position of the imaging unit position detection rib 11c by the imaging unit position detection sensor S2, the direction of the imaging direction of the image imaging unit 14 with respect to the imaging head 2 is detected, and the projection direction and the imaging direction are predetermined. An electric signal for determining whether or not the positional relationship is within the range is output.

  2C is a diagram illustrating a state where the imaging case 11 is removed from the bottom 2b, and FIG. 2D is a diagram illustrating the imaging case 11. As illustrated in FIG. As shown in FIG. 2C, the bottom 2b is provided with a predetermined position fixing rib 2d and a rotation stopper 2e.

  The predetermined position fixing rib 2d is urged and engaged with the small-diameter portion 11b of the imaging case 11, so that the relative positional relationship between the image projection unit 13 and the image capturing unit 14 is in a predetermined positional relationship. The case 11 is fixed at a predetermined position.

  The rotation stopper 2 e is provided so as to engage with a plurality of ribs 11 a erected on the outer peripheral surface of the imaging case 11. Therefore, the imaging case 11 can be intermittently rotated and fixed by urging the imaging case 11 with the plurality of ribs 11a.

  FIG. 2E is a diagram illustrating an internal configuration of the imaging case 11. As shown in FIG. 2E, the imaging case 11 is fixed with an imaging optical system 21 and a CCD 22 that converts light incident through the imaging optical system 21 into an electrical signal. The image capturing unit 14 including the image capturing optical system 21 and the CCD 22 can capture an image of a subject that is present facing the image capturing unit 14 and acquire the captured image data. Further, by rotating the imaging case 11 about the axis about the imaging head 2, the imaging direction by the image imaging unit 14 fixed to the imaging case 11 is projected by the image projection unit 13 fixed to the imaging head 2. It is changed relative to the direction. Therefore, since the image capturing unit 14 can be opposed to a subject existing in a direction different from the projection direction, not only the subject existing in the direction in which the image signal light is projected by the image projecting unit 13 but also the image projecting unit 13. Thus, it is possible to image a subject that exists in a direction different from the direction in which the image signal light is projected.

  Moreover, as shown in FIG.2 (e), one side of the small diameter part 11b is comprised by the tubular shape, and a signal cable is penetrated. With this signal cable, the electrical components in the imaging case 11 and the electrical components in the imaging head 2 are electrically connected.

  FIG. 3 is a block diagram showing an internal electrical configuration of the image input / output device 1. The imaging head 2 is mainly provided with an image projection unit 13 and an image imaging unit 14, and the attachment unit 4 mainly includes a processor 15 and a battery 26.

  A signal cable is inserted into the arm member 3. The signal cable extends to the inside of the attachment portion 4 and connects the electrical components in the attachment portion 4 and the electrical components in the imaging head 2.

  The image projection unit 13 is a unit for projecting an arbitrary projection image on a projection surface, and includes a substrate 16, a plurality of LEDs 17 (hereinafter collectively referred to as “LED array 17 </ b> A”), a light source lens 18, and a projection. The LCD 19 and the projection optical system 20 are provided on a straight line along the projection direction (the optical axis of the projection optical system 20) 13a. The image projection unit 13 will be described in detail with reference to FIG.

  The image capturing unit 14 is a unit for capturing an image of the subject P, and includes an image capturing optical system 21 and a CCD 22 along the light input direction (the optical axis of the image capturing optical system 21) 14a.

  The imaging optical system 21 is composed of a plurality of lenses and has a known autofocus function, and automatically adjusts the focal length and aperture to form an image of light from the outside on the CCD 22.

  The CCD 22 is configured by arranging photoelectric conversion elements such as CCD (Charge Coupled Device) elements in a matrix, and a signal corresponding to the color and intensity of light of an image formed on the surface via the imaging optical system 21. Is converted into digital data and output to the processor 15.

  The processor 15 includes a flash 7, a release button 8, an on / off switch 9 a, a mode switch 9 b, a personal computer interface 24, a power interface 25, a battery 26, an external memory 27, a cache memory 28, and a light source driver 29. The LED array 17A, the projection LCD driver 30 via the projection LCD 19, and the CCD 22 via the CCD interface 31 are electrically connected and managed by the processor 15.

  The external memory 27 is a detachable flash ROM, and stores captured images and three-dimensional information captured in the digital camera mode, the webcam mode, and the stereoscopic image mode. Specifically, an SD card, an XD card (both are registered trademarks), or the like can be used.

  The cache memory 28 is a high-speed storage device. For example, the captured image captured in the digital camera mode is transferred to the cache memory 28 at a high speed, and is processed in the processor 15 before being stored in the external memory 27. Specifically, SDRAM, DDRRAM, or the like can be used.

  The power interface 25 includes a battery 26, the light source driver 29 includes an LED array 17A, the projection LCD driver 30 includes a projection LCD 19, and the CCD interface 31 includes various ICs (integrated circuits) that control the CCD 22. ing.

  Here, as shown in FIG. 3, the main battery 26 for supplying driving power to the imaging head 2 is provided in the mounting portion 4, so that the imaging head 2 can be reduced in weight. Therefore, an operation for changing the orientation of the imaging head 2 with respect to the PC 49 is facilitated. Further, since the imaging head 2 is reduced in weight, the strength of the arm member 3 is not so required as compared with the case where the main battery 26 is provided in the imaging head 2.

  4A is an enlarged view of the image projection unit 13, FIG. 4B is a plan view of the light source lens 18, and FIG. 4C is a diagram showing an arrangement relationship between the projection LCD 19 and the CCD 22.

  As described above, the image projection unit 13 includes the substrate 16, the LED array 17A, the light source lens 18, the projection LCD 19, and the projection optical system 20 along the projection direction.

  The substrate 16 is used to mount the LED array 17A and to perform electrical wiring with the LED array 17A. Specifically, an insulating resin is applied to an aluminum substrate with through holes and then a copper pattern is formed by electroless plating, or a single-layer or multi-layer substrate having a glass epoxy substrate as a core is used. be able to.

  The LED array 17A is a light source that emits radial light. A plurality of LEDs 17 (light emitting diodes) are arranged in a staggered pattern on the substrate 16, and are bonded and electrically connected via a silver paste. Yes. Moreover, it is electrically connected also via the bonding wire.

  By using a plurality of LEDs 17 as a light source in this way, the efficiency of converting electricity into light (electro-light conversion efficiency) is increased compared with the case where an incandescent bulb, a halogen lamp, or the like is used as a light source, and at the same time, infrared or Generation of ultraviolet rays can be suppressed. Therefore, it is possible to drive with power saving, and to save power and extend the life. Moreover, the temperature rise of an apparatus can be reduced.

  As described above, the LED 17 generates extremely less heat rays than a halogen lamp or the like, and the total heat generation amount of the light source itself is low. Therefore, a resin lens is used as the light source lens 18 and the projection optical system 20 described later. Can do. Therefore, compared with the case where a glass lens is employed, each of the lenses 18 and 20 can be configured at low cost and light weight.

  Each LED 17 constituting the LED array 17A emits the same emission color, and is configured to emit amber color using four elements of Al, In, Ga, and P as materials. Therefore, it is not necessary to consider correction of chromatic aberration that occurs when emitting a plurality of emission colors, and it is not necessary to employ an achromatic lens as the projection optical system 20 to correct chromatic aberration. There is an effect that an inexpensive material projection means can be provided.

  In addition, by using an amber LED of a four-element material having an electro-optical conversion rate as high as about 80 lumen / W compared to other luminescent colors, it is possible to achieve higher brightness, power saving, and longer life. The effect related to arranging the LEDs 17 in a staggered manner will be described with reference to FIG.

  Specifically, the LED array 17A is composed of 59 LEDs 17, and each LED 17 is driven at 50 mW (20 mA, 2.5 V). Eventually, all 59 LEDs 17 are driven with power consumption of approximately 3 W. Further, the brightness as the luminous flux value when the light emitted from each LED 17 passes through the light source lens 18 and the projection LCD 19 and is irradiated from the projection optical system 20 is about 25 ANSI lumens even in the case of the entire surface irradiation. Is set to

  By adopting this brightness, for example, in the stereoscopic image mode, when detecting the three-dimensional shape of a subject such as the face of a person or animal, the person or animal does not give glare to the person or animal. A three-dimensional shape that is not collapsed can be detected.

  The light source lens 18 is a lens that collects light emitted radially from the LED array 17A, and the material thereof is made of an optical resin typified by acrylic.

  Specifically, the light source lens 18 includes a convex lens portion 18a that protrudes toward the projection LED 19 at a position facing each LED 17 of the LED array 17A, and a base portion 18b that supports the lens portion 18a. An epoxy or silicon resin sealing material 18c for sealing the LED 17 that fills an opening inside the base portion 18b and enclosing the LED array 17A and bonding the substrate 16 to the light source lens 18; , And a positioning pin 18 d that protrudes from the base portion 18 b toward the substrate 16 and connects the light source lens 18 and the substrate 16.

  The light source lens 18 is fixed on the substrate 16 by inserting positioning pins 18 d into the long holes 16 formed in the substrate 16 while enclosing the LED array 17 </ b> A inside the opening.

  Therefore, the light source lens 18 can be disposed in a space-saving manner. In addition to the function of mounting the LED array 17A on the substrate 16, the function of supporting the light source lens 18 is also used, so that a part for supporting the light source lens 18 is not required and the number of parts is reduced. be able to.

  Each lens portion 18a is disposed at a position facing each LED 17 of the LED array 17A in a one-to-one relationship.

  Therefore, the radial light emitted from each LED 17 is efficiently condensed by each lens unit 18 facing each LED 17 and irradiated to the projection LED 19 as radiation having high directivity as shown in the figure. The reason why the directivity is improved in this way is that light non-uniformity in the plane caused by the optical rotation of the liquid crystal can be suppressed by allowing light to enter the projection LCD 19 substantially perpendicularly. At the same time, since the projection optical system 20 has telecentric characteristics and its incident NA is about 0.1, it is regulated so that only light within ± 5 ° vertical can pass through the internal diaphragm. . Therefore, it is important to improve the image quality to align the light emitted from the LED 17 with the vertical emission angle and to enter a light flux of about ± 5 °. This is because when light deviating from the vertical direction is incident on the projection LCD 19, the transmittance changes depending on the incident angle due to the optical rotation of the liquid crystal, resulting in uneven transmittance.

  The projection LCD 19 is a spatial modulation element that performs spatial modulation on the light collected through the light source lens 18 and outputs an arbitrary image signal light to the projection optical system 20. It is composed of a plate-like liquid crystal display (Liquid Crystal Display) having a different aspect ratio.

  As shown in (C), each pixel constituting the projection LCD 19 includes one pixel column arranged in a straight line along the longitudinal direction of the liquid crystal display, and the one pixel column is a liquid crystal display. Other pixel columns that are shifted by a predetermined distance in the longitudinal direction are alternately arranged in parallel.

  Note that (C) is a state in which the front surface of the imaging head 2 is directed to the front side of the paper, light is emitted toward the projection LCD 19 from the back side of the paper, and a subject image is formed on the CCD 22 from the front side of the paper.

  In this way, by arranging the pixels constituting the projection LCD 19 in a staggered pattern in the longitudinal direction, the light subjected to spatial modulation by the projection LCD 19 in the direction orthogonal to the longitudinal direction (short direction) is ½ pitch. Can be controlled. Therefore, the projection pattern can be controlled with a fine pitch, and the three-dimensional shape can be detected with high accuracy by increasing the resolution.

  In particular, in the stereoscopic image mode, which will be described later, in the case of projecting striped pattern light in which light and dark are alternately arranged toward the subject in order to detect the three-dimensional shape of the subject, the direction of the stripe is determined by the short side of the projection LCD 19. By matching the directions, the boundary between light and dark can be controlled at ½ pitch, and thus a three-dimensional shape can be detected with high resolution points and high accuracy.

  In the imaging head 2, the projection LCD 19 and the CCD 22 are arranged in a relationship as shown in (C). Specifically, since the wide surface of the projection LCD 19 and the wide surface of the CCD 22 are arranged in substantially the same direction, when an image projected on the projection surface from the projection LCD 19 is formed on the CCD 22, The projected image can be formed as it is without bending the projected image with a half mirror or the like.

  The CCD 22 is disposed on the longitudinal direction side (the direction side in which the pixel column extends) of the projection LCD 19. Therefore, in particular, in the stereoscopic image mode, when detecting the three-dimensional shape of the subject using the principle of triangulation, the inclination formed by the CCD 22 and the subject can be controlled at ½ pitch. In addition, a three-dimensional shape can be detected with high accuracy.

  The projection optical system 20 is a plurality of lenses that project the image signal light that has passed through the projection LED 19 toward the projection surface, and is composed of a telecentric lens made of a combination of glass and resin. Telecentric means a configuration in which the principal ray passing through the projection optical system 20 is parallel to the optical axis in the incident-side space and the position of the exit pupil is infinite. By using telecentricity in this way, only light passing through the projection LCD 19 at a vertical angle of ± 5 ° can be projected as described above, so that the image quality can be improved.

  FIG. 5 is a diagram for explaining the arrangement of the LED array 17A. (A) is a figure which shows the illumination intensity distribution of the light which passed the light source lens 18, (b) is a top view which shows the arrangement state of LED array 17A, (c) shows the synthetic | combination illumination intensity distribution in projection LCD19 surface. FIG.

  As shown in (a), the light that has passed through the light source lens 18 reaches the surface of the projection LCD 19 as light having an illuminance distribution as shown on the left side at a half-value spread half angle θ (= approximately 5 °). Designed to be.

  Further, as shown in (b), the plurality of LEDs 17 are arranged in a staggered pattern on the substrate 16. Specifically, an LED row in which a plurality of LEDs 17 are arranged in series at a d pitch is arranged in parallel at a √3 / 2d pitch, and the LED rows are moved 1 / 2d in the same direction every other row. It is arranged to be in a state.

  In other words, the distance between one LED 17 and the LCD 17 around the one LED 17 is set to be d (triangular lattice arrangement).

  The length d is determined to be equal to or less than the full width at half maximum (FWHM (Full Width Half Maximun)) of the illuminance distribution formed in the projection LCD 19 by the light emitted from one of the LEDs 17.

  Accordingly, the combined illuminance distribution of the light passing through the light source lens 18 and reaching the surface of the projection LCD 19 is substantially linear including small ripples as shown in FIG. Can be irradiated. Therefore, the illuminance unevenness in the projection LCD 19 can be suppressed, and as a result, a high-quality image can be projected.

  FIG. 6 is an electrical block diagram of the image input / output device 1. Note that the description of the above-described configuration is omitted. The processor 15 includes a CPU 35, a ROM 36, and a RAM 37.

  The CPU 35 uses the RAM 37 in accordance with the processing by the program stored in the ROM 36, detects the pressing operation of the release button 8, fetches the image data from the CCD 22, transfers and stores the image data, and the mode switch 9b. Various processes such as the detection of the state of the

  The ROM 36 includes a camera control program 36a, a pattern light photographing program 36b, a luminance image generation program 36c, a code image generation program 36d, a code boundary extraction program 36e, a lens aberration correction program 36f, and a triangulation calculation program 36g. A document orientation calculation program 36h and a plane conversion program 36i are stored.

  The camera control program 36a is a program related to the control of the entire image input / output device 1 including the main process shown in FIG.

  The pattern light imaging program 36b is a program for imaging a state in which pattern light is projected onto a subject and a state in which the subject is not projected in order to detect the three-dimensional shape of the subject P.

  The luminance image generation program 36c uses a pattern light existence image obtained by imaging the pattern light projected state by the pattern light photographing program 36b and a pattern light no image obtained by imaging the state in which the pattern light is not projected in the YCbCr space. This is a program for calculating a Y value from an RGB value of an image.

  Also, a plurality of types of pattern light are projected in time series and imaged for each pattern light, and a plurality of types of luminance images are generated.

  The code image generation program 36d binarizes a plurality of luminance images generated by the luminance image generation program 36c with reference to a predetermined luminance threshold value or a luminance image of no pattern light image, and from the result, for each pixel. This is a program for generating a code image in which a predetermined code is assigned.

  The code boundary extraction program 36e obtains code boundary coordinates with sub-pixel accuracy using the code image generated by the code image generation program 36d and the luminance image generated by the luminance image generation program 36c. It is a program.

  The lens aberration correction program 36f is a program that corrects the aberration of the imaging optical system 20 with respect to the code boundary coordinates obtained with subpixel accuracy by the code boundary extraction program 36e.

  The triangulation calculation program 36g is a program for calculating the three-dimensional coordinates of the real space related to the boundary coordinates from the boundary coordinates of the code subjected to the aberration correction by the lens aberration correction program 36f.

  The document orientation calculation program 36h is a program that estimates and determines the three-dimensional shape of the subject P such as a book from the three-dimensional coordinates calculated by the triangulation calculation program 36g.

  The plane conversion program 36i is a program for generating a flattened image such as an image of the subject P such as a book from the front based on the three-dimensional shape of the subject P such as a document calculated by the document orientation calculation program 36h.

  The RAM 37 includes a pattern light existence image storage unit 37a, a pattern light no image storage unit 37b, a luminance image storage unit 37c, a code image storage unit 37d, a code boundary coordinate storage unit 37e, an ID storage unit 37f, An aberration correction coordinate storage unit 37g, a three-dimensional coordinate storage unit 37h, a document orientation calculation result storage unit 37i, a plane conversion result storage unit 37j, a projection image storage unit 37k, and a working area 37l are allocated as storage areas. ing.

  The pattern light present image storage unit 37a stores a pattern light present image obtained by imaging a state in which the pattern light is projected onto the subject P by the pattern light photographing program 36b. The pattern light no-image storage unit 37b stores a pattern light no-image obtained by imaging the state where the pattern light is not projected onto the subject P by the pattern light photographing program 36b.

  The luminance image storage unit 37c stores a luminance image generated by the luminance image generation program 36c. The code image storage unit 37d stores a code image generated by the code image generation program 36d. The code boundary coordinate storage unit 37e stores the boundary coordinates of each code obtained by the code boundary extraction program 36e with subpixel accuracy extracted. The ID storage unit 37f stores an ID assigned to a luminance image having a change in brightness at a pixel position having a boundary. The aberration correction coordinate storage unit 37g stores the boundary coordinates of the code subjected to the aberration correction by the lens aberration correction program 36f. The three-dimensional shape coordinate storage unit 37h stores the three-dimensional coordinates of the real space calculated by the triangulation calculation program 36g.

  The document orientation calculation result storage unit 37i stores parameters related to the three-dimensional shape of the subject P such as a document calculated by the document orientation calculation program 36h. The plane conversion result storage unit 37j stores the plane conversion result generated by the plane conversion program 36i. The projection image storage unit 37k stores image information to be projected from the image projection unit 13. The working area 371 stores data temporarily used for calculation by the CPU 15.

  In addition to the configuration described above, an amplifier 32 is connected to the processor 15 via a bus line. The amplifier 32 sounds a speaker 33 connected to the amplifier 32 to output a warning sound or the like. Is.

  The PC 49 includes a CPU 50, a ROM 51 and a RAM 52 connected to the CPU 50 via an internal bus, an interface 55 connectable to the personal computer interface 24, a CRT display 58, and a keyboard 59. It is electrically connected via.

  FIG. 7 is a flowchart of the main process. Details of the digital camera processing (S605), webcam processing (S607), stereoscopic image processing (S609), and planarized image processing (S611) in the main processing will be described later.

  In the main process, first, when the power supply is activated (S601), the processor 15 and other interfaces are initialized (S602).

  Then, a key scan for determining the state of the mode switch 9 is performed (S603), it is determined whether or not the setting of the mode switch 9 is the digital camera mode (S604), and if it is the digital camera mode (S604: Yes), The process proceeds to a digital camera process to be described later (S605).

  On the other hand, if it is not the digital camera mode (S604: No), it is determined whether or not the setting of the mode change switch 9 is the webcam mode (S606). (S607).

  On the other hand, if it is not the webcam mode (S606: No), it is determined whether the setting of the mode switch 9 is the stereoscopic image mode (S608). If it is the stereoscopic image mode (S608: Yes), the stereoscopic image processing described later is performed. (S609).

  On the other hand, if it is not the stereoscopic image mode (S608: No), it is determined whether or not the setting of the mode switch 9 is the planar image mode (S610). If it is the planar image mode (S610: Yes), it will be described later. The process proceeds to planarized image processing (S611).

  On the other hand, if it is not the planar image mode (S610: No), it is determined whether or not the setting of the mode switch 9 is in the off mode (S612). If it is not the off mode (S612: No), the processing from S603 is performed. If it is repeatedly in the off mode (S612: Yes), the process ends.

  Hereinafter, the digital camera processing, webcam processing, and stereoscopic image processing will be described in detail. In these processes, since the condition determination is performed based on the relative positional relationship between the image projecting unit 13 and the image capturing unit 14, first, the relative positional relationship between the image projecting unit 13 and the image capturing unit 14 is used. Will be described.

  FIG. 8 is a diagram illustrating a positional relationship between the image projection unit 13 and the image capturing unit 14. First, based on signals output from the imaging head position detection sensor S1 and the imaging unit position detection sensor S2, the projection direction of the image projection unit 13 (the optical axis direction of the projection optical system 20) is 0 ° ± 30 with respect to the vertical direction. ° (approximately -30 ° to approximately 30 °), and the imaging direction of the image imaging unit 14 (the optical axis direction of the imaging optical system 21) is relative to the front of the imaging head 2 (the surface on which the barrel 5 is provided) The case where it is detected that the angle is other than 0 ° is hereinafter referred to as A mode (the state shown in FIG. 8 (b) A).

  Further, based on signals output from the imaging head position detection sensor S1 and the imaging unit position detection sensor S2, the projection direction of the image projection unit 13 is 0 ° ± 30 ° with respect to the vertical direction, and the imaging of the image imaging unit 14 is performed. A case where the direction is detected to be approximately 0 ° with respect to the front of the imaging head 2 is hereinafter referred to as a B mode (the state shown in FIG. 8B).

  In this B mode, the direction in which the subject imaged by the image capturing unit 14 is present and the direction in which the image signal light is projected by the image projecting unit 13 are at a large angle close to each other. Even if the image light is not projected and the image pickup unit 14 projects the image signal light while projecting the image signal light, the projected image can be prevented from being captured. Therefore, when the positional relationship between the image projecting unit 13 and the image capturing unit 14 is in the B mode, for example, a character string indicating an operation method or an operation procedure (related information regarding the captured image acquired by the image capturing unit) is projected. In this state, the user can appropriately perform imaging while confirming the projected character strings and the like. Further, for example, the image projection unit 13 projects the image signal light in a direction different from the direction in which the user is present, and the image is projected with the image imaging unit 14 facing the user himself. The user can take an image while viewing the recorded image.

  Further, based on the signals output by the imaging head position detection sensor S1 and the imaging unit position detection sensor S2, the projection direction of the image projection unit 13 is other than 0 ° ± 30 ° with respect to the vertical direction, and the image imaging unit 14 A case where the imaging direction is detected to be other than approximately 0 ° with respect to the front surface of the imaging head 2 is hereinafter referred to as a C mode (state shown in FIG. 8 (b) C).

  Further, based on the signals output by the imaging head position detection sensor S1 and the imaging unit position detection sensor S2, the projection direction of the image projection unit 13 is other than 0 ° ± 30 ° with respect to the vertical direction, and the image imaging unit 14 A case where the imaging direction is detected to be approximately 0 ° with respect to the front of the imaging head 2 is hereinafter referred to as a D mode (the state shown in FIG. 8 (b) D).

  In this D mode, the direction in which the subject imaged by the image capturing unit 14 is present and the direction in which the image signal light is projected by the image projecting unit 13 are at a large angle close to each other. Even if the image light is not projected and the image pickup unit 14 projects the image signal light while projecting the image signal light, the projected image can be prevented from being captured.

  FIG. 9 is a flowchart of the digital camera process (S605 in FIG. 6). The digital camera process is a process for acquiring an image captured by the image capturing unit 14.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S701). Thereby, a high quality captured image can be provided to the user.

  Next, the position of the image projection unit 13 is acquired based on the signal output from the imaging head position detection sensor S1, and the position of the image imaging unit 14 is determined based on the signal output from the imaging unit position detection sensor S2. Obtain (S702b).

  Subsequently, it is checked whether or not the positional relationship between the image projecting unit 13 and the image capturing unit 14 is the above-described A mode (see FIG. 8) (S702c). If the mode is the A mode (S702c: Yes), the image projecting unit 13 is checked. The rectangular image light, which is an image indicating the imageable range, is projected onto the imageable area by (S702d). FIG. 10 is a diagram illustrating a state in which rectangular image light indicating an imageable area is projected on the surface. As described above, the user can visually recognize the area that can be imaged by the rectangular image light before actually imaging by pressing the release button 8.

  On the other hand, when the positional relationship between the image projecting unit 13 and the image capturing unit 14 is not the above-described A mode (S702c: No), it is checked whether the mode is the B mode (see FIG. 8) (S702e). (S702e: Yes), an image such as a character for notifying that imaging is possible is projected by the image projection unit 13 (S702f).

  On the other hand, when the positional relationship between the image projecting unit 13 and the image capturing unit 14 is not the above-described B mode (S702e: No), the speaker 33 (see FIG. 6) is sounded to notify that the image can be captured (S702g). ). That is, when the projection direction of the image signal light by the image projection unit 13 is other than 0 ° ± 30 ° with respect to the vertical direction (C mode or D mode), there is a surface that can be projected in the projection direction of the image projection unit 13. If the positional relationship between the image projecting unit 13 and the image capturing unit 14 is neither the A mode nor the B mode described above, the fact that the image can be captured is not based on the projected image but the sound. Is notified. In place of the notification by voice, a notification that the image can be taken may be provided by turning on and blinking a pilot lamp (not shown).

  Next, the release button 8 is scanned (S703a), and it is determined whether the release button 8 is half-pressed (S703b). If half-pressed (S703b: Yes), the auto focus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S703c). If it is not half-pressed (S703b: No), the processing from S703a is repeated.

  Next, the release button 8 is scanned again (S703d), and it is determined whether or not the release button 8 has been fully pressed (S703e). If it is fully pressed (S703e: Yes), it is determined whether or not the flash mode is set (S704).

  As a result, if the flash mode is selected (S704: Yes), the flash 7 is projected (S705) and photographed (S706). If the flash mode is not selected (S704: No), the flash 7 is not projected. A picture is taken (S706). If it is determined in S703e that the button has not been fully pressed (S703e: No), the processing from S703a is repeated.

  If it is determined in step S702e that the mode is the B mode and an image notifying that imaging is possible is being projected in step S702f, the projection of the image is continued in step S706 while the subject P is being projected. Imaging is performed. That is, in the B mode, the imaging direction is substantially 0 ° with respect to the front of the imaging head 2, and the direction in which the image imaging unit 14 images and the direction in which the image signal light is projected by the image projection unit 13 are substantially perpendicular to each other. Therefore, the image signal light is not projected on the subject to be imaged by the image imaging unit 14. Therefore, even if the image projection unit 13 captures an image while projecting the image, the subject P can be imaged without including the projected image.

  Next, the captured image is transferred from the CCD 22 to the cache memory 28 (S707), and the captured image stored in the cache memory 28 is displayed on the monitor LCD 10 (S708). In this way, by transferring the captured image to the cache memory 28, the captured image can be displayed on the monitor LCD 10 at a higher speed than when transferring to the main memory. Then, the captured image is stored in the external memory 27 (S709).

  Finally, it is determined whether or not the mode change switch 9 has changed (S710). If there is no change (S710: Yes), the processing from S702b is repeated, and if there is a change (S710: No), The process ends.

  FIG. 11 is a flowchart of the webcam process (S607 in FIG. 7). The webcam process is a process of transmitting a captured image (including a still image and a moving image) captured by the image capturing unit 14 to an external network. In this embodiment, it is assumed that a moving image is transmitted as an imaged image to an external network.

  In this process, first, a low resolution setting signal is transmitted to the CCD 22 (S801), and the known autofocus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S802a). .

  Next, the position of the image projection unit 13 is acquired based on the signal output from the imaging head position detection sensor S1, and the position of the image imaging unit 14 is determined based on the signal output from the imaging unit position detection sensor S2. Obtain (S802b).

  Subsequently, it is checked whether or not the positional relationship between the image projecting unit 13 and the image capturing unit 14 is the above-described A mode (see FIG. 8) (S802c). If the mode is the A mode (S802c: Yes), digital camera processing (see FIG. 9), the image projection unit 13 projects rectangular image light that is an image showing the imageable range (S802d).

  On the other hand, when the positional relationship between the image projecting unit 13 and the image capturing unit 14 is not the A mode described above (S802c: No), it is checked whether or not the mode is the B mode (see FIG. 8) (S802e). When the mode is the B mode (S802e: Yes), a projection process described later is performed (S802f), and then the image capturing unit 14 captures an image. When the positional relationship between the image projecting unit 13 and the image capturing unit 14 is the B mode, the image capturing direction intersects at a large angle that is nearly perpendicular to the projection direction. The image signal light is not projected by the image projection unit 13. Therefore, similarly to the processing of S702f in the digital camera process (see FIG. 9), even if the image projection unit 13 projects an image while projecting an image, the subject P can be captured without including the projected image. .

  FIG. 12 is a diagram illustrating a state in which image output light is projected by the image projection unit 13 in the projection processing (S802f) of the webcam processing. As shown in FIG. 12, in the webcam process, for example, a message image M composed of a character string indicating an imaging mode such as “WEBCAM mode” and the captured image f are combined and projected. At the start of the webcam process, since the captured image f has not been obtained yet, only the message M is projected.

  On the other hand, when the positional relationship between the image projecting unit 13 and the image capturing unit 14 is not the above-described B mode (S802e: No), the speaker 33 (see FIG. 6) is sounded to notify that the image can be captured (S802g). ), Shooting is started (S803). That is, for the same reason as the digital camera processing (see FIG. 9), when the positional relationship between the image projecting unit 13 and the image capturing unit 14 is neither the A mode nor the B mode, the image projecting unit 13 projects an image. In other words, the user is notified only by voice.

  Then, the captured image is displayed on the monitor LCD 10 (S804). Next, for example, the message image M composed of a character string such as “being imaged” and the captured image f are combined and stored in the projection image storage unit 37k (S805). That is, since the projection image stored in the projection image storage unit 37k is updated in the process of S804, the captured image f and the message image M are projected and used by the updated image output light in the projection process of S802f. The person can visually recognize the change in the imaging state by the image imaging unit 14 from the projected image.

  Further, the captured image is transferred from the CCD 22 to the cache memory 28 (S807), and the captured image transferred to the cache memory 28 is transmitted to the external network via the RF driver 24 and the antenna 11 which are RF interfaces (S808).

  Finally, it is determined whether there is any change in the mode changeover switch 9 (S809). If there is no change (S809: Yes), the processing from S802 is repeated, and if there is a change (S809: No), the processing Exit.

  In the webcam process described above, the message image M and the captured image f are projected as shown in FIG. 12, but only the captured image f is projected by the image projection unit 13 as shown in FIG. You may do it.

  As shown in FIG. 13, when the relative positional relationship between the image projecting unit 13 and the image capturing unit 14 is in the B mode, for example, the user acquired the image while imaging himself / herself. The captured image f can be confirmed at any time.

  FIG. 14 is a flowchart of the projection process (S802f in FIG. 11). This process is a process of projecting the image stored in the projection image storage unit 37k from the image projection unit 13 onto the projection plane. In this process, first, it is confirmed whether or not an image is stored in the projection image storage unit 37k (S901). If stored (S901: Yes), the image stored in the projection image storage unit 37k is transferred to the projection LCD driver 30 (S902), and the projection LCD driver 30 outputs an image signal corresponding to the image to the projection LCD 19. The image is displayed on the projection LCD 19 (S903).

  Next, the light source driver 29 is driven (S904), the LED array 17A is turned on by an electrical signal from the light source driver 29 (S905), and the process is terminated.

  Thus, when the LED array 17A is turned on, the light emitted from the LED array 17A reaches the projection LCD 19 via the light source lens 18, and the projection LCD 19 performs spatial modulation according to the image signal transmitted from the projection LCD driver 30. And output as image signal light. The image signal light output from the projection LCD 19 is projected as a projection image on the projection surface via the projection optical system 20.

  FIG. 15 is a flowchart of the stereoscopic image processing (S609 in FIG. 7). The stereoscopic image processing is processing for detecting a three-dimensional shape of a subject and acquiring and displaying a three-dimensional shape detection result image as the stereoscopic image.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S1001), the position of the image projection unit 13 is acquired based on the signal output from the imaging head position detection sensor S1, and the imaging unit position detection sensor S2 is acquired. The position of the image capturing unit 14 is acquired based on the signal output from (S1002b).

  Subsequently, it is checked whether or not the positional relationship between the image projecting unit 13 and the image capturing unit 14 is the above-described C mode (see FIG. 8) (S1002c). If the mode is the C mode (S1002c: Yes), the image can be captured. The speaker 33 is sounded to notify the user by voice (S1002d).

  On the other hand, if the relative positional relationship between the image projection unit 13 and the image capturing unit 14 is not the C mode described above (S1002c: No), it is checked whether or not the mode is the A mode (see FIG. 8) (1002e). When the mode is the A mode (S1002e: Yes), the speaker 33 is sounded to notify that it can be imaged (S1002d). In the case of the A mode, instead of this sound notification, a message may be projected, or rectangular image light indicating an imageable range may be projected.

  On the other hand, when the relative positional relationship between the image projecting unit 13 and the image capturing unit 14 is not the C mode and the A mode (S1002e: No), the speaker 33 is set to be in the A mode or the C mode. It rings and warns by voice (S1002g), and returns to the processing of S1002b. That is, detection of a three-dimensional shape by a three-dimensional shape detection process (S1006) to be described later is prohibited until the user adjusts the imaging head 2 and the imaging case 11 to enter the A mode or the C mode.

  As will be described in detail later, in the three-dimensional shape detection process (S1006), a subject is imaged by the image imaging unit 14 in a state where a predetermined pattern light is projected by the image projecting unit 13, and imaging obtained by the imaging is performed. Based on the data, the three-dimensional shape of the subject P is detected. Therefore, when the relative positional relationship between the image projection unit 13 and the image capturing unit 14 is not in a predetermined relationship, the accuracy of measurement of the three-dimensional shape is significantly reduced. The execution of the three-dimensional shape detection process (S1006) is permitted only when the image capturing unit 14 has a predetermined relationship.

  Next, the release button 8 is scanned (S1003a), and it is determined whether or not the release button 8 is half-pressed (S1003b). If half-pressed (S1003b: Yes), the auto focus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S1003c). If not half-pressed (S1003b: No), the processing from S1003a is repeated.

  Next, the release button 8 is scanned again (S1003d), and it is determined whether or not the release button 8 is fully pressed (S1003e). If it is fully pressed (S1003e: Yes), it is determined whether or not the flash mode is set (S1003f).

  As a result, if the flash mode is selected (S1003f: Yes), the flash 7 is projected (S1003g), photographed (S1003h), and if the flash mode is not selected (S1003f: No), the flash 7 is not projected. A picture is taken (S1003h). If it is determined in S1003e that the button has not been fully pressed (S1003e: No), the processing from S1003a is repeated.

  Next, a three-dimensional shape detection process described later is performed to detect the three-dimensional shape of the subject (S1006).

  Next, the three-dimensional shape detection result in the three-dimensional shape detection process (S1006) is stored in the external memory 27 (S1007), and the three-dimensional shape detection result is displayed on the monitor LCD 10 (S1008a). The three-dimensional shape detection result is displayed as an aggregate of three-dimensional coordinates (X, Y, Z) in the real space of each measurement vertex. That is, a three-dimensional shape detection result image is displayed on the monitor LCD 10 as a stereoscopic image (3D CG image) obtained by connecting the measurement vertices as a three-dimensional shape detection result with polygons and displaying the surface.

  Then, it is determined whether or not the mode change switch 9 has changed (S1011). If there is no change (S1011: Yes), the processing from S702 is repeated. If there is a change (S1011: No), the processing is performed. finish.

  FIG. 16A is a diagram for explaining the principle of the spatial code method used to detect a three-dimensional shape in the above-described three-dimensional shape detection process (S1006 in FIG. 15). It is a figure which shows the pattern light different from (a). Either (a) or (b) may be used for the pattern light, and further, a gray level code which is a multi-tone code may be used.

  Details of this spatial coding method are described by Kosuke Sato and one other, “Distance Image Input by Spatial Coding”, IEICE Transactions, 85/3 Vol. J 68-D No3 p369-375.

  The spatial code method is one type of method for detecting the three-dimensional shape of a subject based on triangulation between projected light and an observed image. As shown in (a), the distance between the projection light source L and the observation device O is determined. It is characterized in that it is set apart by D, and the space is divided into long and narrow fan-shaped regions and coded.

  When the three mask patterns A, B, and C in the figure are projected in order from the MSB, each fan-shaped area is coded into bright “1” and dark “0” by the mask. For example, since the area including the point P is not exposed to light in the masks A and B and bright in the mask C, it is encoded as 001 (A = 0, B = 0, C = 1).

  Each fan-shaped region is assigned a code corresponding to the direction φ, and each can be regarded as one slit beam. Therefore, the scene is photographed for each mask with a camera as an observation device, and the bit plane of the memory is constructed by binarizing the light / dark pattern.

  Thus, the horizontal position (address) of the obtained multiple bit-plane image corresponds to the observation direction θ, and the contents of the memory at this address give the projected light code, that is, φ. The coordinates of the point of interest are determined from θ and φ.

  In addition, as a mask pattern used in this method, a case where a pure binary code such as mask patterns A, B, and C is used is illustrated in FIG. There is a risk that a large error will occur.

  For example, the point Q in (a) indicates the boundary between the region 3 (011) and the region 4 (100), but if 1 of the mask A is shifted, the code in the region 7 (111) may be generated. In other words, a large error may occur when the Hamming distance is 2 or more between adjacent regions.

  Therefore, as the mask pattern used in this method, as shown in (b), by using a code whose hamming distance is always 1 between adjacent regions, the coding error as described above can be avoided. It is said that.

  FIG. 17A is a flowchart of the three-dimensional shape detection process (S1006 in FIG. 15). In this process, first, an imaging process is performed (S1210). This imaging process is performed by using a plurality of pure binary code mask patterns shown in FIG. 16 (a) from the image projection unit 13 in a striped pattern light in which light and dark are alternately arranged (see FIGS. 1 and 18). ) Are projected onto the subject in time series, and a pattern light existence image obtained by imaging the state where each pattern light is projected and a pattern light no image obtained by imaging the state where the pattern light is not projected are obtained. is there. FIG. 1 is a diagram illustrating a state in which the pattern light is projected when the relative positional relationship between the image projecting unit 13 and the image capturing unit 14 is the A mode. FIG. It is a figure which shows the state in which the said pattern light is projected when the relative positional relationship of the part 13 and the image pick-up part 14 is C mode.

  When the imaging process is completed (S1210), a three-dimensional measurement process is performed (S1220). The three-dimensional measurement process is a process of actually measuring the three-dimensional shape of the subject using the pattern light existence image and the pattern light no image acquired by the imaging process. Thus, when the three-dimensional measurement process ends (S1220), the process ends.

  FIG. 17B is a flowchart of the imaging process (S1210 in FIG. 12A). This process is executed based on the pattern light imaging program 36a. First, the image imaging unit 14 images a subject without projecting pattern light from the image projection unit 13, thereby acquiring a pattern light no-image (S1211). ). The acquired pattern light no-image is stored in the pattern light no-image storage unit 37b.

  Next, the counter i is initialized (S1212), and it is determined whether or not the value of the counter i is the maximum value imax (S1213). The maximum value imax is determined by the number of mask patterns to be used. For example, when 8 types of mask patterns are used, the maximum is imax (= 8).

  If it is determined that the value of the counter i is smaller than the maximum value imax (S1213: Yes), the i-th mask pattern among the mask patterns to be used is displayed on the projection LCD 19, and the i-th mask pattern is displayed. The i-th pattern light projected by the above is projected onto the projection surface (S1214), and the state where the pattern light is projected is photographed by the image capturing unit 14 (S1215).

  In this way, a pattern light existence image obtained by imaging the state in which the i-th pattern light is projected onto the subject is acquired. The acquired pattern light existence image is stored in the pattern light existence image storage unit 37a.

  When the photographing is finished, the projection of the i-th pattern light is finished (S1216), "1" is added to the counter i to project the next pattern light (S1217), and the processing from S1213 is repeated.

  If it is determined that the value of the counter i is greater than the maximum value imax (S1213: No), the process ends. That is, in this imaging process, one pattern light no image and the maximum value imax pattern light existence images are acquired.

  FIG. 17C is a flowchart of the three-dimensional measurement process (S1220 in FIG. 17A). This process is executed based on the luminance image generation program 36c, and first generates a luminance image (S1221). Here, the luminance is a Y value in the YCbCr space, and is a value calculated from Y = 0.22989 · R + 0.5866 · G + 0.1145 · B from the RGB value of each pixel. By obtaining the Y value for each pixel, a luminance image relating to the image with and without pattern light is generated. The generated luminance image is stored in the luminance image storage unit 37c. A number corresponding to the pattern light number is assigned to each luminance image.

  Next, the code image generation program 36d generates a code image coded for each pixel by combining the generated luminance images using the spatial code method described above (S1222).

  This code image is binarized by comparing each pixel of the luminance image related to the image with pattern light stored in the luminance image storage unit 37c with a preset luminance threshold value or no pattern light image, and the result is shown in FIG. As described in a) and (b), it can be generated by assigning to LSB to MSB. The generated code image is stored in the code image storage unit 37d.

  Next, a code boundary coordinate detection process described later is performed by the code boundary extraction program 36e (S1223), and the boundary coordinates of the code assigned to each pixel are detected with subpixel accuracy.

  Next, lens aberration correction processing is performed by the lens aberration correction program 36f (S1224). By this processing, it is possible to correct the error in the code boundary coordinates detected in S1223 that includes an error due to the influence of distortion or the like of the imaging optical system 21.

  Next, real space conversion processing based on the triangulation principle is performed by the triangulation calculation program 36g (S1225). The code boundary coordinates on the CCD space after the aberration correction is performed by this processing are converted into the three-dimensional coordinates in the real space, and the three-dimensional coordinates as the three-dimensional shape detection result are obtained.

  FIG. 19 is a diagram for explaining the outline of the code boundary coordinate detection process (S1223 in FIG. 17). In the upper diagram, the bright and dark boundaries of the actual pattern light in the CCD space are indicated by the boundary line K. The pattern light is encoded by the spatial code method described above, and the boundary between one code and another code is indicated by a bold line in the figure. It is the figure shown by.

  That is, since the coding in the spatial coding method described above is performed on a pixel-by-pixel basis, an error in sub-pixel accuracy occurs between the boundary line K of the actual pattern light and the coded boundary (thick line in the figure). Therefore, the purpose of this code boundary coordinate detection process is to detect code boundary coordinates with sub-pixel accuracy.

  In this process, first, at a certain detection position (hereinafter referred to as “curCDX”), a first pixel G that changes from a certain target code (hereinafter referred to as “curCode”) to another code is detected (first pixel detection step). .

  For example, in curCDXX, when each pixel is detected in order from the top, it is a pixel having curCode up to the boundary (thick line), but curCode changes in the pixel next to the boundary, that is, the first pixel G. This is detected as the first pixel G.

  Next, at the pixel position of the first pixel G, all of the luminance images having a change in brightness are extracted from the luminance images stored in the luminance image storage unit 37c in S1221 of FIG. 17 (luminance image extraction step). ).

  Next, in order to specify a pixel area to be used for approximation, the detection position is moved to the left by “2”, and the code image is referred to at the position of the detection position curCDXX-2 to change from the target code (curCode). Pixel (boundary pixel (pixel H at the detection position of cur CCDX-2)) is searched for, and a predetermined range centered on that pixel (in the present embodiment, -3 pixels and +2 pixels in the Y-axis direction) Range) is specified (part of the pixel region specifying step).

  Next, within the predetermined range, as shown in the lower left graph in the figure, an approximate expression (shown by a solid line) in the Y direction is obtained, and the approximate expression The Y coordinate Y1 at the intersection with the brightness threshold value bTh is obtained (part of the boundary coordinate detection step).

  Note that the luminance threshold value bTh may be calculated from a predetermined range (for example, one half of the average luminance of each pixel) or may be a fixed value given in advance. As a result, the boundary between light and dark can be detected with sub-pixel accuracy.

  Next, the detection position is moved to the right side of “1” from curCCDX-2, and the same processing as described above is performed in curCCDX-1 to obtain a representative value in curCCDX-1 (part of the boundary coordinate detection step).

  In this manner, each detection is performed in a pixel area (see the lower right hatched portion in the figure) composed of a predetermined range in the Y-axis direction centering on the boundary pixel and a range of curCCDX-2 to curCCDX + 2 in the X-axis direction. The representative value at the position is obtained.

  The above processing is performed on all luminance images having pixels that change from curCode to another code, and a weighted average value of representative values for each luminance image is finally adopted as boundary coordinates in curCode (boundary coordinate detection step) Part of).

  As a result, the boundary coordinates of the code can be detected with high accuracy with sub-pixel precision, and by performing the real space conversion process (S1225 in FIG. 17) based on the above-described triangulation principle using the boundary coordinates, high The three-dimensional shape of the subject can be detected with high accuracy.

  In addition, since the boundary coordinates can be detected with sub-pixel accuracy by using the approximate expression calculated based on the luminance image in this way, the number of images to be captured is not increased as in the prior art, and pure binary. Pattern light brightened and darkened by a code may be used, and it is not necessary to use a gray code which is a special pattern light.

  In this embodiment, each detection position is composed of a range from “−3” to “+2” in the Y-axis direction around the boundary pixel, and a range from curCCDX-2 to curCCDX + 2 as the detection position in the X-axis direction. Although the region to be processed has been described as a pixel region for obtaining approximation, the ranges of the Y axis and the X axis of the pixel region are not limited to these. For example, only a predetermined range in the Y-axis direction centering on the boundary pixel at the curCDX detection position may be set as the pixel region.

  FIG. 20 is a flowchart of the code boundary coordinate detection process (S1223 in FIG. 17). This process is executed based on the code boundary extraction program 36e. First, each element of the code boundary coordinate sequence in the CCD space is initialized (S1401), and curCCDX is set as the start coordinate (S1402).

  Next, it is determined whether curCCDX is equal to or less than the end coordinate (S1403). If it is equal to or less than the end coordinate (S1403: Yes), curCode is set to “0” (S1404). That is, curCode is initially set to a minimum value.

  Next, it is determined whether curCode is smaller than the maximum code (S1405). If the curCode is smaller than the maximum code (S1405: Yes), the curCCDX refers to the code image to search for the curCode pixel (S1406), and determines whether the curCode pixel exists (S1407).

  As a result, if there is a curCode pixel (S1407: Yes), the curCDXX searches for a code pixel larger than the curCode with reference to the code image (S1408), and a curCode pixel larger than the curCode is found. It is determined whether or not it exists (S1409).

  As a result, if there is a pixel with a code larger than curCode (S1409: Yes), a process for obtaining a later-described boundary with subpixel accuracy is performed (S1410). Then, in order to obtain boundary coordinates for the next curCode, “1” is added to curCode (S1411), and the processing from S1405 is repeated.

  That is, since the boundary exists at the pixel position of the pixel having the curCode or the pixel position of the pixel of the Code that is larger than the curCode, in the present embodiment, the boundary is temporarily the pixel position of the pixel of the CurCode that is larger than the curCode. It is assumed that the process is in progress.

  In addition, when curCode does not exist (S1407: No), or when a pixel with a code larger than curCode does not exist (S1409: No), in order to obtain boundary coordinates for the next curCode, “ 1 "is added (S1411), and the processing from S1405 is repeated.

  Thus, for curCode from 0 to the maximum code, the processing from S1405 to S1411 is repeated, and when curCode becomes larger than the maximum code (S1405: No), “dCCDX” is added to curCCDX to change the detection position (S1412). ) The process from S1403 is repeated at the new detection position in the same manner as described above.

  Then, curCDXX is changed, and finally when curCDX becomes larger than the end coordinate (S1403), that is, when the detection from the start coordinate to the end coordinate is completed, the process ends.

  FIG. 21 is a flowchart of the process (S1410 in FIG. 20) for obtaining the code boundary coordinates with subpixel accuracy.

  In this process, first, a change in brightness at the pixel position of a pixel having a code larger than the curCode detected in S1409 of FIG. 20 from the luminance image stored in the luminance image storage unit 37c in S1221 of FIG. All the luminance images having are extracted (S1501).

  Then, the mask pattern number of the extracted luminance image is stored in the array PatID [], and the number of extracted luminance images is stored in noPatID (S1502). The arrays PatID [] and noPatID are stored in the ID storage unit 37f.

  Next, the counter i is initialized (S1503), and it is determined whether or not the value of the counter i is smaller than noPatID (S1504). As a result, if it is determined to be small (S1504: Yes), the CCDY value at the boundary is obtained for the luminance image having the mask pattern number of PatID [i] corresponding to the counter i, and the value is stored in fCDY [i]. (S1505).

  When the processing of S1505 is completed, “1” is added to the counter i (S1506), and the processing from S1504 is repeated. If it is determined in S1504 that the value of the counter i is greater than noPatID (S1504: No), that is, when the processing of S1505 is completed for all the luminance images extracted in S1501, the fCCDY obtained in the processing of S1505 The weighted average value of [i] is calculated, and the result is set as a boundary value (S1507).

  In place of the weighted average value, the median value of fCCDY [i] obtained in the processing of S1505 can be calculated, and the result can be used as a boundary value, or the boundary value can be calculated by statistical calculation.

  In other words, the boundary coordinates are expressed by the curCDXX coordinates and the weighted average value obtained in S1507. The boundary coordinates are stored in the code boundary coordinate storage unit 37e, and the process ends.

  FIG. 22 is a flowchart of processing (S1505 in FIG. 21) for obtaining the CCDY value of the boundary for the luminance image having the mask pattern number of PatID [i].

  In this process, first, a process represented by “ccdx = MAX (curCCDX−dx, 0)” for setting a larger value of “curCDX−dx” and “0” as ccdx is performed, and a counter j is set. Initialization is performed (S1601).

  Specifically, “0” in S1601 means the minimum value of the CCDX value. For example, the curCDX value as a detection position is “1” and the preset dx value is “2”. Assuming that “curCDXX-dx” is “−1”, which is smaller than “0” which is the minimum value of the CCDX value, the subsequent processing in “−1” is set as “ccdx = 0”. I do.

  That is, for a position smaller than the minimum value of the CCDX value, processing for excluding the subsequent processing is performed.

  The value of “dx” can be set in advance to an appropriate integer including “0”. In the example described with reference to FIG. 19, this “dx” is set to “2”. According to the example, the ccdx is set to “curCDX-2”.

  Next, it is determined whether or not ccdx <= MIN (curCCDX + dx, ccdW−1) (S1602). That is, “MIN (curCCDX + dx, ccdW−1)” on the left side is a smaller value among “curCDXX + dx” and “ccdW−1” obtained by subtracting “1” from the maximum CCDX value “ccdW”. Therefore, the value is compared with the “ccdx” value.

  That is, for a position larger than the maximum value of the CCDX value, processing for excluding the subsequent processing is performed.

  If the result of determination is that ccdx is smaller than MIN (curCCDX + dx, ccdW−1) (S1602: Yes), the code image and the luminance image to which PatID [i] is assigned are referred to and the pixel having the boundary exists. The eCDY value of the pixel position is obtained (S1603).

  For example, assuming that the detection position is curCCDX-1 shown in FIG. 19, the pixel I is detected as a pixel candidate having a boundary, and the eCDY value is obtained at the position of the pixel I.

  Next, from the luminance image having the mask pattern number of PatID [i], an approximation regarding the luminance in the ccdy direction within the range of MAX (eCDY-dy, 0) <= ccdy <= MIN (eCCY + dy-1, ccdH-1). A polynomial Bt = fb (ccdy) is obtained (S1604).

  Next, the ccdy value at which the approximate polynomial Bt and the luminance threshold value bTh intersect is obtained, and the value is stored in efCCDY [j] (S1605). By S1604 and S1605, it is possible to detect the boundary coordinates with subpixel accuracy.

  Next, “1” is added to each of ccdx and counter j (S1605), and the processing from S1602 is repeated. That is, the boundary of subpixel accuracy is detected at each detection position within a predetermined range on the left and right with the cur CCDX as the center.

  If it is determined in S1602 that “ccdx” is larger than “MIN (curCCDX + dx, ccdW−1)” (S1602: No), ccCDY [j] calculated in the range of curCCDX−dx to curCCDX + dx is ccdy. = Approximate polynomial of fy (ccdx) is obtained (S1606). Since each value detected in S1605 by this process is used, the detection accuracy of the boundary coordinates can be improved as compared with the case of detecting the boundary coordinates at one detection position.

  The intersection of the approximate polynomial and curCDXX obtained in this way is set as the CCDY value at the boundary for the luminance image having the mask pattern number of PatID [i] (S1607), and the process ends. As shown in the flowchart of FIG. 15, the processing up to this point is executed for each of the extracted luminance images, and a weighted average value is calculated for the obtained boundary coordinates. Since the boundary coordinates are used (S1507), the detection accuracy of the boundary coordinates can be further improved.

  FIG. 23 is a diagram for explaining lens aberration correction processing (S1224 in FIG. 17). In the lens aberration correction processing, as shown in FIG. 23A, the incident light beam is deviated from the position to be imaged by the ideal lens due to the aberration of the imaging optical system 21. This is a process of correcting the position to the position where the image should be originally formed.

  For example, as shown in FIG. 23B, this aberration correction is data obtained by calculating the aberration of the optical system in the imaging range of the imaging optical system 21 using the half field angle hfa that is the angle of incident light as a parameter. Correct based on.

  This aberration correction processing is executed based on the lens aberration correction program 36f and performed on the code boundary coordinates stored in the code boundary coordinate storage unit 37e, and the data subjected to the aberration correction processing is stored in the aberration correction coordinate storage unit 37g. Stored.

  Specifically, the following camera calibration (approximate expression) from (a) to (c) for converting arbitrary point coordinates (ccdx, ccdy) in the real image into coordinates (ccdcx, ccdcy) in the ideal camera image is used. To correct.

  In this embodiment, the aberration amount dist (%) is described as dist = f (hfa) using a half angle of view hfa (deg). In addition, the focal length of the imaging optical system 21 is focal length (mm), the ccd pixel length pixel length (mm), and the center coordinates of the lens in the CCD 22 are (Centx, Centy).

(A) ccdcx = (ccdx−Centx) / (1 + dist / 100) + Centx
(B) ccdcy = (ccdy−Centy) / (1 + dist / 100) + Centy
(C) hfa = arctan [(((ccdx−Centx) ² + (ccdy−Centy) ² ) ^0.5 ) × pixellength / focallength]
FIG. 24 is a diagram for explaining a method of calculating the three-dimensional coordinates in the three-dimensional space from the coordinates in the CCD space in the real space conversion process (S1225 in FIG. 17) based on the triangulation principle.

  In the real space conversion process based on the triangulation principle, the triangulation calculation program 36g calculates the three-dimensional coordinates in the three-dimensional space for the code boundary coordinates subjected to the aberration correction stored in the aberration correction coordinate storage unit 37g. The three-dimensional coordinates calculated in this way are stored in the three-dimensional coordinate storage unit 37h.

  In this embodiment, as the coordinate system of the image input / output device 1 for the subject P curved in the lateral direction to be imaged, the optical axis direction of the imaging optical system 21 is the Z axis, and the main of the imaging optical system 21 is along the Z axis. A point that is VPZ away from the point position is an origin, a horizontal direction with respect to the image input / output device 1 is an X axis, and a vertical direction is a Y axis.

  Further, the projection angle θp from the image projection unit 13 to the three-dimensional space (X, Y, Z), D is the distance between the optical axis of the imaging optical system 21 and the optical axis of the image projection unit 13, and The field of view in the Y direction is Yftop to Yfbottom, the field of view in the X direction is from Xfstart to Xfend, the length (height) in the Y-axis direction of the CCD 22 is Hc, and the length (width) in the X-axis direction is Wc. The projection angle θp is given based on a code assigned to each pixel.

In this case, the three-dimensional space position (X, Y, Z) corresponding to the arbitrary coordinates (ccdx, ccdy) of the CCD 22 is on the point on the image plane of the CCD 22, the projection point of the pattern light, and the XY plane. It can be obtained by solving five equations for the triangle formed by the intersecting points.
(1) Y = − (tan θp) Z + PPZ + tan θp−D + cmp (Xtarget)
(2) Y = − (Ytarget / VPZ) Z + Ytarget
(3) X = − (Xtarget / VP) Z + Xtarget
(4) Ytarget = Yftop− (ccdcy / Hc) × (Yftop−Yfbottom)
(5) Xtarget = Xfstart + (ccdcx / Wc) × (Xfend−Xfstart)
Note that cmp (Xtarget) in (1) is a function for correcting the deviation between the imaging optical system 21 and the image projection unit 13 and can be regarded as cmp (Xtarget) = 0 in an ideal case where there is no deviation. it can. This equation is not limited to the curved subject P, but can be used for an object having an arbitrary three-dimensional shape.

  On the other hand, as described above, the relationship between the arbitrary coordinates (lcdcx, lcdccy) on the projection LCD 19 included in the image projection unit 13 and the three-dimensional coordinates (X, Y, Z) in the three-dimensional space is as follows: 6) to (9).

In this embodiment, the principal point position (0, 0, PPZ) of the image projection unit 13, the Y-direction field of the image projection unit 13 from Ypftop to Ypfbottom, the X-direction field of view from Xpfstart to Xpfend, and the Y of the projection LED 19 The length (height) in the axial direction is Hp, and the length (width) Wp in the X-axis direction.
(6) Y = − (Yptarget / PPZ) Z + Yptarget
(7) X = − (Xptarget / PPZ) Z + Xptarget
(8) Yptarget = Ypftop− (lcdcy / Hp) × (Ypftop−Ypfbottom)
(9) Xptarget = Xpfstart + (lcdcx / Wp) × (Xpfend−Xpfstart)
By using this relational expression, the LCD spatial coordinates (lcdcx, lcdcy) can be calculated by giving the three-dimensional spatial coordinates (X, Y, Z) to the above formulas (6) to (9). it can. Therefore, for example, an LCD element pattern for projecting an arbitrary shape and character in a three-dimensional space can be calculated.

  FIG. 25 is a flowchart of planarized image processing (S611 in FIG. 7). The planarized image processing is performed when, for example, an image of a curved document (subject) P such as a book is captured or when a rectangular document P is imaged from an oblique direction (the captured image has a trapezoidal shape). Even if it exists, it is the process which acquires and displays the planarized image corrected in the state which the original is not curving, or the state imaged from the orthogonal | vertical direction with respect to the surface.

  In this process, first, a high resolution setting signal is transmitted to the CCD 22 (S1901).

  Next, the release button 8 is scanned (S1903a), and it is determined whether the release button 8 is half-pressed (S1903b). If half-pressed (S1903b: Yes), the auto focus (AF) and automatic exposure (AE) functions are activated to adjust the focus, aperture, and shutter speed (S1903c). If not half-pressed (S1903b: No), the processing from S1903a is repeated.

  Next, the release button 8 is scanned again (S1903d), and it is determined whether or not the release button 8 is fully pressed (S1903e). If it is fully pressed (S1903e: Yes), it is determined whether or not the flash mode is set (S1903f).

  As a result, if the flash mode is selected (S1903f: Yes), the flash 7 is projected (S1903g), photographed (S1903h), and if the flash mode is not selected (S1903f: No), the flash 7 is not projected. A picture is taken (S1903h). If it is determined in S1903e that the button has not been fully pressed (S1903e: No), the processing from S1903a is repeated.

  Next, a three-dimensional shape detection process that is the same as the above-described three-dimensional shape detection process (S1006 in FIG. 15) is performed to detect the three-dimensional shape of the subject (S1906).

  Next, based on the three-dimensional shape detection result obtained by the three-dimensional shape detection process (S1906), a document attitude calculation process for calculating the attitude of the document P is performed (S1907). With this process, the position L, the angle θ, and the curvature φ (x) of the document P with respect to the image input apparatus 1 are calculated as the posture parameters of the document P.

  Next, based on the calculation result, a plane conversion process to be described later is performed (S1908), and a flattened image flattened in a state where the original P is not curved even if it is curved is generated.

  Next, the planarized image obtained by the planar change process (S1908) is stored in the external memory 27 (S1909), and the planarized image is displayed on the monitor LCD 10 (S1910).

  Then, it is determined whether or not the mode change switch 9 has changed (S1911). If there is no change (S1911: Yes), the processing from S702 is repeated again, and if there is a change (S1911: No). ), The process ends.

  FIG. 26 is a diagram for explaining the document orientation calculation process (S1907 in FIG. 25). Note that, as an assumption condition for a document such as a book, it is assumed that the curvature of the document P is uniform in the y direction. In this manuscript attitude calculation processing, first, as shown in FIG. 26A, regression line approximation is performed on points arranged in two columns at a three-dimensional space position from the coordinate data regarding the code boundary stored in the three-dimensional coordinate storage unit 37h. Obtain the two curves.

  For example, it can be obtained from position information (a boundary relating to the boundary between the code 63 and the code 64 and the boundary between the code 191 and the code 192) of the upper and lower quarters of the projected range of the pattern light.

  Assuming a straight line connecting points where the positions of the two curves in the X-axis direction are “0”, the point where the straight line intersects the Z-axis, that is, the point where the optical axis intersects the original P, is 3 A dimension space position (0, 0, L) is defined, and an angle formed by the straight line with the XY plane is defined as an inclination θ around the X axis of the document P.

  Next, as shown in FIG. 26B, the document P is rotationally converted in the opposite direction by the inclination θ around the X axis obtained earlier, that is, the document P is parallel to the XY plane. Assuming that

  Then, as shown in FIG. 26C, with respect to the cross section of the document P in the XZ plane, the displacement in the Z-axis direction can be expressed as a curve φ (X) as a function of X. Thus, the position L, the angle θ, and the curvature φ (x) of the document P are calculated as the document orientation parameters, and the process ends.

  FIG. 27 is a flowchart of the plane conversion process (S1908 in FIG. 25). In this process, first, the processing area of the process is assigned to the working area 37l of the RAM 37, and the variable of the counter b used for the process is set to an initial value (b = 0) (S2101).

  Next, based on the position L of the original P, the inclination θ, and the curvature φ (x) based on the calculation result in the original attitude calculation program 36h, the pattern light no-image 4 stored in the pattern light no-image storage unit 37b. Each corner point is obtained by inverse transformation of the curve (a process equivalent to a “curving process” described later) that moves −L in the Z direction, rotates −θ in the X-axis direction, and further changes to φ (x). A rectangular area formed by dots (that is, a rectangular area that forms an image in which the surface of the original P on which characters or the like are written is observed from a substantially orthogonal direction) is set, and pixels included in the rectangular area The number a is obtained (S2102).

  Next, coordinates on the non-patterned light image corresponding to each pixel constituting the set rectangular area are obtained, and pixel information of each pixel of the planarized image is set from pixel information around this coordinate.

  That is, first, it is determined whether or not the counter b has reached the pixel number a (S2103). If the counter b has not reached the pixel number a (S2103: No), a curve calculation process is performed in which one pixel constituting the rectangular area is rotated and moved by a curve φ (x) around the Y axis (S2104). A tilt θ is rotated about the X axis (S2105), and is shifted by a distance L in the Z axis direction (S2106).

  Next, the coordinates (ccdcx, ccdcy) on the CCD image obtained by imaging the ideal camera with the inverse function of the previous triangulation are obtained from the obtained three-dimensional spatial position (S2107), and the imaging optical system used According to the aberration characteristics of 20, the coordinates (ccdx, ccdy) on the CCD image captured by the actual camera are obtained by the inverse function of the previous camera calibration (S2108), and the pixel of the pattern light no image corresponding to this position Is stored in the working area 37l of the RAM 37 (S2109).

  Then, “1” is added to the counter b in order to execute the above-described processing from S2103 to S2109 for the next pixel (S2110).

  In this way, when the processing from S2104 to S2110 is repeated until the counter b reaches the pixel number a (S2103: Yes), in S2101, the processing area allocated to the working area 37l is released to execute the processing (S2111). ), The process ends.

  FIG. 28A is a diagram for explaining the outline of the bending process (S2104 in FIG. 27), and FIG. 28B shows the document P flattened by the plane conversion process (S1908 in FIG. 25). Yes. The details of the curving process are disclosed in detail in the IEICE Transactions DIIVol. J86-D2 No. 3 p409 “Curved Document Shooting with an Eye Scanner”.

  The curve Z = φ (x) is obtained by converting a three-dimensional shape constituted by the obtained code boundary coordinate sequence (real space), a cross-sectional shape parallel to the XZ plane at an arbitrary Y value, and a least square method. Is expressed by an equation approximated by a polynomial.

  When flattening a curved curved surface, as shown in (a), the flattened points corresponding to the points on Z = φ (x) are from Z = φ (0) to Z = φ (x). Are associated with each other according to the length of the curve.

  By plane conversion processing including such bending processing, for example, even when a document P in a curved state is imaged, a planarized planar image can be acquired as shown in FIG. If the flattened image is used, the accuracy of the OCR process can be increased. Therefore, characters, figures, and the like written on the original can be clearly recognized from the image.

  In the digital camera process shown in FIG. 9, the webcam process shown in FIG. 11, and the stereoscopic image process shown in FIG. 15, in the case of projecting an undistorted projection image without depending on the projection direction, the process of S806 in FIG. Instead of the projection processing (S702f, S802f, S1010) executed in the same manner as the projection processing, a projection image conversion processing (S2900) as will be described later is executed.

  FIG. 29 is a flowchart of the distortion-free image conversion processing (S2900). The distortion-free image conversion process (S2900) is a process for converting an image displayed on the projection LCD 19 in accordance with image information stored in the projection image storage unit 37k into an image that can be projected onto the subject in an undistorted state. is there.

  In this process, first, the processing area of the process is allocated to the working area 37l of the RAM 37, and the variable of the counter q used for the process is set to an initial value (q = 0) (S2901).

  Next, a memory corresponding to the space of LCD space coordinates (lcdcx, lcdcy) is used as a rectangular area that becomes an image after being converted into an image for distortion-free projection (that is, an image that is undistorted on a curved subject). The area is secured and set in the working area 37l of the RAM 37, and the number of pixels Qa included in the rectangular area is obtained (S2902).

  Next, each pixel value of image information such as the message image M and the captured image f stored in the projection image storage unit 37k is arranged in each pixel of the ideal camera image coordinate system (ccdcx, ccdcy) (S2903). .

  Next, for each pixel on the LCD space coordinates (lcdcx, lcdccy) constituting the set rectangular area, the surface of the subject stored in the three-dimensional coordinate storage unit 37h according to the above-described equations (6) to (9). The three-dimensional coordinates (X, Y, Z) which are the corresponding points above are obtained, and further, by solving for (ccdcx, ccdcy) in equations (1) to (5), pixels of each pixel of the image for distortion-free projection Calculate and set information.

  That is, first, it is determined whether or not the counter q has reached the number of pixels Qa (S2904). If the counter q has not reached the number of pixels Qa (S2904: No), the LCD space coordinates (lcdcx, lcdcy) of the pixel corresponding to the value of the counter q are obtained from the expressions (6) to (9) according to the working area 37l. Is converted into the coordinates (X, Y, Z) on the subject stored in (S2905).

  Next, the coordinates (X, Y, Z) on the subject obtained by the conversion by the processing of S2905 are ideally obtained using the equations solved for (ccdcx, cdccy) in equations (1) to (5). The coordinates (ccdcx, cdccy) on the camera image are converted (S2906).

  Next, the pixel information arranged at the coordinates (ccdcx, ccdcy) obtained by the conversion in the process of S2906 is acquired, and the pixel information is converted into the LCD space coordinates (lcdcx, lcdcy) corresponding to the value of the counter q. (S2907).

  Then, “1” is added to the counter q in order to execute the above-described processing from S2904 to S2907 for the next pixel (S2908).

  Thus, when the processing from S2904 to S2908 is repeated until the counter q reaches the number of pixels Qa (S2904: Yes), the pixel information associated with the LCD space coordinates (lcdcx, lcdcy) that constitute the set rectangular area Is transferred to the projection LCD driver 30 (S2909).

  Finally, in S2901, the processing area allocated to the working area 371 for executing the process is released (S2910), and the process is terminated.

  By the processing of S2909, the pixel information on the LCD space coordinates (lcdcx, lcdcy) is transferred to the projection LCD driver 30, whereby the projection LCD 19 displays a projected image projected without distortion on the distorted curved surface. Therefore, an undistorted image is projected on the subject.

  Accordingly, by executing the distortion-free image conversion process (S2900), a distortion-free projection image is projected not only when the projection direction is oblique but also when the projection surface is curved. be able to. As a result, particularly when a character string such as the message image M is projected, the user can be made to accurately recognize the information.

  FIG. 30 is a view for explaining the light source lens 50a of the second embodiment related to the light source lens 18 of the first embodiment described above, and (a) is a side view showing the light source lens 50a of the second embodiment. (B) is a top view which shows the light source lens 50a of 2nd Example. In addition, the same code | symbol is attached | subjected to the same member as mentioned above, and the description is abbreviate | omitted.

  The light source lens 18 in the first embodiment is configured by integrally arranging the lens portions 18a on the base 18b from a convex aspherical shape corresponding to each LED 17, whereas in the second embodiment The light source lens 50a is formed by separately forming a resin lens formed in a bullet shape containing each of the LEDs 17.

  In this manner, by configuring the light source lenses 50a including the respective LEDs 17 separately, the positions of the respective LEDs 17 and the corresponding optical lenses 50a are determined on a one-to-one basis. The accuracy can be increased and the light emission direction is aligned.

  On the other hand, when the lens array is collectively aligned on the substrate 16, the light emission direction varies due to the positioning error when each LED 17 is die-bonded and the difference between the linear expansion coefficients of the lens array and the substrate. There is a risk of becoming.

  Therefore, the surface of the projection LCD 19 is irradiated with light whose light incident directions from the LEDs 17 are aligned perpendicularly to the surface of the projection LCD 19 so that it can pass through the aperture of the projection optical system 20 uniformly. Irradiance unevenness can be suppressed, and as a result, a high-quality image can be projected. The LED 17 included in the light source lens 50a is mounted on the substrate 16 via an electrode 51a composed of a lead and a reflector.

  Further, on the outer peripheral surface of the group of light source lenses 50a in the second embodiment, a fixing member 52a having a frame-like elasticity for bundling the light source lenses 50a and restricting them in a predetermined direction is disposed. The fixing member 52a is made of a resin material such as rubber or plastic.

  Since the light source lens 50a of the second embodiment is configured separately from each LED 17, it faces the projection LCD 19 with the angle of the optical axis formed by the convex tip of each light source lens 50a properly aligned. It is difficult to install.

  Therefore, the light source lens 50a is surrounded by the fixing member 52a, the outer peripheral surfaces of the light source lenses 50a are brought into contact with each other, and the light source lenses 50a are arranged so that the optical axis of each light source lens 50a faces the projection LCD 19 at a correct angle. By restricting the position of the light, it is possible to irradiate light from each light source lens 50a toward the projection LCD 19 substantially vertically. Therefore, the light aligned in the direction perpendicular to the surface of the projection LCD 19 is irradiated and can pass uniformly through the aperture of the projection lens, so that unevenness in illuminance of the projection image can be suppressed. Therefore, a higher quality image can be projected.

  The fixing member 52a may have a predetermined rigidity, and may be made of a material having an elastic force, and the position of each light source lens 50 may be set to a predetermined position by the elastic force. You may make it regulate to.

  FIG. 31 is a view for explaining a second embodiment relating to the fixing member 52a for restricting the light source lens 50a described in FIG. 30 to a predetermined position, and (a) is a perspective view showing a state in which the light source lens 50a is fixed. (B) is a partial cross-sectional view thereof. In addition, the same code | symbol is attached | subjected to the same member as mentioned above, and the description is abbreviate | omitted.

  The fixing member 60 of the second embodiment is formed in a plate shape in which a through-hole 60a having a conical shape in cross section having a cross section along the outer peripheral surface of each light source lens 50a is formed. Each light source lens 50a is inserted into and fixed to each through hole 60a.

  Further, an urging plate 61 having elasticity is interposed between the fixing member 60 and the substrate 16, and an electrode 51a is provided between the urging plate 61 and the lower surface of each light source lens 50a. An annular O-ring 62 having elasticity is arranged so as to surround the outer periphery.

  The LED 17 included in the light source lens 50 a is mounted on the substrate 16 via the biasing plate 61 and an electrode 51 a that penetrates a through hole formed in the substrate 16.

  According to the fixing member 60 described above, each light source lens 50a is fixed by being passed through each through-hole 60a having a cross section along the outer peripheral surface of the light source lens, so that it is more reliable than the above-described fixing member 50a. The optical axis of the light source lens 50 can be fixed so as to face the projection LCD 19 at a correct angle.

  Further, at the time of assembly, the LED 17 can be urged and fixed to the correct position by the urging force of the O-ring 62.

  Further, an impact force that may be generated when the apparatus 1 is transported can be absorbed by the elastic force of the O-ring 62, and the position of the light source lens 50a is shifted due to the impact, and the light source lens. It is possible to prevent the inconvenience that light cannot be irradiated from 50a toward the projection LCD 19 vertically.

  Note that the imaging control unit during projection described in claim 6 corresponds to the processing of S706 in the digital camera processing of FIG. 9 and the processing of S803 in the webcam processing of FIG.

  Further, the projection image updating means according to claim 9 corresponds to the processing of S805 in the webcam processing of FIG. The three-dimensional information detection means according to claim 10 corresponds to the processing of S1006 in the stereoscopic image processing of FIG.

  Further, the three-dimensional information detection prohibiting means according to claim 13 corresponds to the processing of S1002g in the stereoscopic image processing of FIG. The resolution reduction transmission means according to claim 14 corresponds to the processing of S801 in the webcam processing of FIG.

  Although the present invention has been described based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be easily made without departing from the gist of the present invention. It can be done.

  For example, in the above-described embodiment, the process of acquiring and displaying a planarized image has been described as the planarized image mode. However, a known OCR function is installed, and the planarized planar image is read by this OCR function. You may comprise as follows. In such a case, there is a great effect that the text described on the original can be read with higher accuracy than the case of reading the original in a curved state by the OCR function.

  In the web cam process of FIG. 11 in the above embodiment, instead of the process of transmitting the low resolution setting signal to the CCD 22 (S801), the process of transferring from the CCD 22 to the cache memory 28 (S807) or the captured image is transmitted to the external network. In the processing (S808), the resolution of the captured image may be reduced and transmitted. Even in this case, the subsequent processing speed can be improved by reducing the resolution of the captured image.

  Further, in S1501 of FIG. 21 in the above-described embodiment, a case has been described in which all luminance images having a change in brightness are extracted, and provisional CCDY values are obtained for all of them. It is not necessary that the number of sheets is limited to one or more. The boundary coordinates can be obtained at high speed by reducing the number of extractions.

  Further, in S1507 of FIG. 21 in the above embodiment, fCCDY [i] is weighted average, and in S1607 of FIG. 22 efCCDY [j] is an approximate polynomial, each value is averaged. The averaging method is not limited to these. For example, a method of taking a simple average value of each value, a method of employing a median value of each value, calculating an approximate expression of each value, and calculating the approximate expression For example, a method of using the detection position in the boundary coordinates as a boundary coordinate, a method of obtaining by statistical calculation, or the like may be used.

  Further, for example, in the three-dimensional shape detection process in the planar image mode in the above-described embodiment, in order to detect the three-dimensional shape of the document P, a striped pattern light formed by alternately arranging a plurality of types of light and dark is projected. Although the case has been described, the light for detecting the three-dimensional shape is not limited to such pattern light.

  For example, as shown in FIG. 32, when the three-dimensional shape of a curved document is simply detected, two strip-shaped slit lights 70 and 71 may be projected from the image projection unit 13. In this case, a three-dimensional shape can be detected at high speed from only two captured images, as compared with the case of projecting eight pattern lights.

  Further, in the image projection unit 13 in the above embodiment, the light source lens 18, the projection LCD 19, and the projection optical system 20 are provided on a straight line along the projection direction. However, the configuration of the image projection unit 13 is not necessarily limited. It is not limited to this.

  FIG. 33 is a diagram illustrating a modification of the image projecting unit 13, where (a) is a plan view of the imaging head 2, and (b) and (c) are cross-sectional views taken along line AA of (a). is there. Note that (a) omits the description of the lid 2a to show the inside of the imaging head 2. As shown in FIGS. 33B and 33C, a projection optical system 20 composed of a plurality of lenses is provided with a reflection mirror 200, and the direction of light emitted from the light source lens 18 is changed to a predetermined projection direction. You may project.

  In this way, since the optical path can be changed by the reflection mirror 200, the light source lens 18, the LCD 19, and the projection optical system 20 do not necessarily need to be provided on a straight line along the projection direction, and the image projection unit 13 of the imaging head 2 It can be stored compactly according to the shape.

  Moreover, although the said Example was equipped with LED array 17A and the light source lens 18 as a light source means, it replaces with these light source means, and as shown in FIG.33 (c), a light source means light-emits white light. A light source 170 such as a halogen lamp or a white LED (Light Emitting Diode) and a reflection plate 180 for efficiently making white light emitted from the light source 170 incident on the LCD 19 may be used.

  In addition, in the digital camera processing (see FIG. 9), webcam processing (see FIG. 11), and stereoscopic image processing (see FIG. 15) of the above embodiment, the position of the imaging unit fixed to the small-diameter portion 11b of the imaging case 11 is detected. The position of the image pickup unit 14 with respect to the image pickup head 2 is detected by detecting the position of the rib 11c for use with the image pickup unit position detection sensor S2, but the image projection unit 13 and the image pickup unit 14 are relatively The means for detecting the positional relationship is not limited to this. For example, the predetermined position fixing rib 2d is engaged with a stopper (not shown) provided on one of the small-diameter portions 11b of the imaging case 11, and the further rotation of the imaging case 11 is restricted. It may be detected that the imaging unit 14 is in a predetermined positional relationship with the image projection unit 13. In this way, detection of the three-dimensional shape is permitted only when the positional relationship between the image projection unit 13 and the image capturing unit 14 is surely a predetermined positional relationship, and detection of the three-dimensional shape is performed with higher accuracy. Can be done.

FIG. 2 is an external perspective view of the image input / output device, where (a) is a view as seen from the upper surface side, and (b) is an enlarged view of the imaging head. It is a figure which shows the internal structure of an imaging head. It is a figure which shows typically the electric structure inside an image input / output device. (A) is an enlarged view of an image projection part, (b) is a top view of a light source lens, (c) is a front view of projection LCD. It is a figure for demonstrating regarding the arrangement | sequence of a LED array. It is an electrical block diagram of an image input / output device. It is a flowchart of a main process. It is a figure which shows the relative positional relationship of an image projection part and an image imaging part. It is a flowchart of a digital camera process. It is a figure which shows the state in which the rectangular image light which shows the area | region which can be imaged is projected on the surface. It is a flowchart of a webcam process. It is a figure which shows the state in which the image output light is projected by the image projection part in the projection process of a webcam process. It is a figure which shows the state in which the image output light is projected by the image projection part in the projection process of a webcam process. It is a flowchart of a projection process. It is a flowchart of a three-dimensional image process. (A) is a figure for demonstrating the principle of the space code method, (b) is a figure which shows the mask pattern (gray code) different from (a). (A) is a flowchart of a three-dimensional shape detection process. (B) is a flowchart of an imaging process. (C) is a flowchart of a three-dimensional measurement process. It is a figure which shows the state in which the pattern light is projected when the relative positional relationship of an image projection part and an image imaging part is C mode. It is a figure for demonstrating the outline of a code boundary coordinate detection process. It is a flowchart of a code boundary coordinate detection process. It is a flowchart of the process which calculates | requires a code boundary coordinate with subpixel precision. It is a flowchart of the process which calculates | requires the boundary CCDY value about the brightness | luminance image with the mask pattern number of PatID [i]. It is a figure for demonstrating a lens aberration correction process. It is a figure for demonstrating the method of calculating the three-dimensional coordinate in a three-dimensional space from the coordinate in CCD space. It is a flowchart of planarization image processing. It is a figure for demonstrating document orientation calculation processing. It is a flowchart of a plane conversion process. (A) is a figure for demonstrating the outline about a curvature calculation process, (b) is a figure which shows the planarization image planarized by the plane conversion process. It is a flowchart of the image conversion process for distortion-free projection. (A) is a side view which shows the light source lens 50 of 2nd Example, (b) is a top view which shows the light source lens 60 of 2nd Example. (A) is a perspective view which shows the state which fixed the light source lens 50, (b) is the fragmentary sectional drawing. It is a figure which shows the other example as pattern light projected on a to-be-photographed object. It is a figure which shows the modification of an image projection part, Comprising: (a) is a top view of an imaging head, (b), (c) is AA sectional view taken on the line of (a).

Explanation of symbols

2 Imaging head (projection housing, base)
2d rib for fixing a predetermined position (fixing means)
3 Arm member (support member)
4 Mounting part 11 Imaging case (imaging housing)
13 Image projection unit (projection means)
14 Image pickup unit (image pickup means)
17 LED (light source means)
19 Projection LCD (spatial modulation means, liquid crystal panel)
20 Projection optical system (part of projection means)
21. Imaging optical system (part of imaging means)
22 CCD (part of imaging means, light receiving element)
26 Main battery (power supply)
49 PC (base)
200 Reflection mirror (projection direction determination means)
S2 Imaging unit position detection sensor (position determination means)

Claims (14)

Light source means for emitting light, and spatial modulation means for spatially modulating the light emitted from the light source means to output an arbitrary image signal light, and the image signal light output by the spatial modulation means Projection means for projecting in the projection direction;
An image input / output device comprising at least an imaging means capable of capturing an image of a subject existing in the projection direction and acquiring the captured image data;
The imaging direction of the imaging unit is provided so as to be relatively changeable with respect to the projection direction of the projection unit so that a subject existing in a direction different from the projection direction can be captured by the imaging unit. A characteristic image input / output device. The spatial modulation means is composed of a liquid crystal panel, and the projection means has a projection optical system that forms an image of the image signal light output from the liquid crystal panel on a predetermined projection surface,
The imaging means includes a light receiving element that converts light into an electrical signal, and an imaging optical system that forms an image of incident light on the light receiving element,
2. The image input / output apparatus according to claim 1, further comprising: a projection housing that holds the projection means; and an imaging housing that holds the imaging means arranged to be movable relative to the projection housing.   3. The image input / output apparatus according to claim 2, wherein the projection unit includes a projection direction determination unit for projecting light emitted from the light source unit in a predetermined projection direction. A base that supports the projection means and the imaging means;
2. The image input / output device according to claim 1, further comprising a support member that movably supports at least one position of the projection unit and the imaging unit with respect to the base. The support member has one end side connected to at least one of the projection unit and the imaging unit, and has an attachment portion for attaching the support member to the base on the other end side,
5. The image input / output device according to claim 4, wherein at least one of the base and the support member is provided with a power supply unit for supplying power to at least one of the projection unit and the imaging unit. .   6. The image input / output device according to claim 1, further comprising a projection imaging control unit that performs imaging by the imaging unit while projecting image signal light by the projection unit.   The image according to claim 6, wherein the projection imaging control unit is configured to project the related information regarding the captured image acquired by the imaging unit by the projection unit by controlling the projection unit. I / O device.   7. The image input / output according to claim 6, wherein the projection imaging control unit projects the captured image of the subject imaged by the imaging unit by the projection unit by controlling the projection unit. apparatus.   The image input / output device according to claim 6, further comprising a projection image update unit that updates image signal light projected by the projection unit.   3D information detection for detecting 3D information of the subject based on a captured image captured by the imaging means when image signal light having a predetermined pattern is projected onto the subject by the projection means 10. The image input / output device according to claim 1, further comprising means.   11. The image input / output according to claim 1, further comprising a position determination unit that determines whether a relative positional relationship between the imaging unit and the projection unit is within a predetermined range. apparatus.   12. A fixing unit that fixes the positional relationship between the imaging unit and the projection unit in a state where the relative positional relationship between the imaging unit and the projection unit is in the predetermined positional relationship. The image input / output device described.   A position determination unit that determines a relative positional relationship between the imaging unit and the projection unit, and the positional determination unit determines that the relative positional relationship between the imaging unit and the projection unit is within a predetermined range. The image input / output apparatus according to claim 10, further comprising: a three-dimensional information detection prohibiting unit that prohibits the detection of three-dimensional information by the three-dimensional information detecting unit when it is determined that the three-dimensional information is not present.   The image input / output apparatus according to claim 1, further comprising a resolution reduction transmission unit that lowers the resolution of a captured image acquired by the imaging unit and transmits the image to the outside.

Images