JP2007036843A - Imaging apparatus with projection display function, and portable electronic device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To correct shaking of an object image formed in an image sensor or a projection image, when there is camera shake. <P>SOLUTION: A projection/display function integrated camera module 15, comprising a combination lens 31 for combination of photographing and projection, an image sensor 33, a light source 34 and a liquid crystal light projector 35, is integrated in a mobile phone 10. The camera module 15 comprises a shake correction mechanism 37 for displacing a second lens group 43 of the combination lens 31 and a shake detection sensor 38 for detecting shake added to the mobile phone 10. Based on a shake detecting signal outputted from the shake detection sensor 38, the shake correction mechanism 37 is driven to move the second lens group 43 of the combination lens 31 for performing optical camera shake correction. If camera shake occurs, at the time of photographing, shaking of an object image to be formed in the image sensor 33 can be corrected; and at the time of projection display, shake of a projection image can be corrected. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus with a projection display function capable of projecting and displaying a captured image on a screen or the like, and a portable electronic device including the imaging apparatus.

  In recent years, a camera module including an imaging lens and an image sensor that captures a subject image formed by the photographing lens has been reduced in size and cost. As a result, it has become possible to add a photographing function by incorporating a camera module into a portable electronic device such as a personal computer, an electronic notebook, or a cellular phone. As a result, mobile phones with a camera in which a camera module is incorporated into a mobile phone are rapidly spreading.

  Image data of a subject image taken with a camera-equipped mobile phone is recorded in a flash ram or the like provided in the phone body. Then, the captured image can be displayed on a display monitor such as a liquid crystal monitor by switching the operation mode of the mobile phone to a reproduction mode in which the captured image is displayed. Thereby, the user can confirm the photographed image on the spot.

  However, since the liquid crystal monitor provided in the camera-equipped mobile phone is small, there is a problem that it is difficult to confirm an image. Also, even when showing images taken by a friend or family member in the vicinity, it is necessary to point the display surface of the liquid crystal monitor to the other party or to pass the mobile phone to the other party, which is very troublesome.

  In order to solve this problem, as described in Patent Document 1, a projection display function-integrated camera module capable of projecting and displaying a captured image on a screen or the like is well known. The projection display function integrated camera module is provided with a light source for illuminating a conventional camera module and a liquid crystal light valve for modulating the illumination light into image light in accordance with the read image data. In addition, a mirror is added to the camera module so that the photographing lens can be used as a projection lens. The mirror can be displaced to a photographing position where a subject image is formed on the image sensor by the photographing lens, and a projection display position where the image light modulated by the liquid crystal light valve is incident on the photographing lens to perform projection display.

By incorporating such a projection display function-integrated camera module into a mobile phone, the mobile phone can be used as a hand-held projector that can be held by hand. Thereby, the captured image can be projected and displayed on a screen or the like, so that the user can easily check the image. You can also show your friends and family at the same time.
Japanese Patent Application Laid-Open No. 3-65879 (see pages 2 to 3 and FIG. 1)

  By the way, projection display by a camera-equipped cellular phone having a projection display function is performed with the user holding the cellular phone in hand. At this time, if vibration due to camera shake or the like is generated, the projected image projected and displayed on the screen or the like is also shaken, which makes it difficult to see. Although many methods have been disclosed for performing camera shake correction to obtain a clear image without shaking when shooting with a camera-equipped mobile phone, projection display such as a camera-equipped mobile phone with a projection display function has been disclosed. However, there is no disclosure of a method for performing both camera shake correction during shooting and camera shake correction during projection display.

  The present invention is for solving the above-described problems, and has a photographing function and a projection display function, and corrects a shake of a subject image formed on an image sensor and a shake of a projection image even if a hand shake occurs. An object of the present invention is to provide an imaging device with a projection display function and a portable electronic device including the imaging device.

  An image pickup apparatus with a projection display function according to the present invention includes an image sensor that picks up a subject image formed by an optical system and outputs an image pickup signal, and a storage unit that stores image data obtained by performing signal processing on the output image pickup signal. And a light modulation unit that irradiates illumination light from a light source and modulates the illumination light into image light according to the image data read from the storage means, and the subject image is applied to the image sensor by the optical system. A switching unit capable of switching to an imaging state in which an image is formed, and a projection state in which the modulated image light is incident on the optical system and a projection image is projected and displayed on a projection target; at least the image sensor and the Based on the detection result of the shake detection means and the shake detection means for detecting the direction and amount of shake given from the outside with respect to the housing that houses the light modulation unit, Sometimes with correcting the shake of the object image, wherein at the time of projection state, characterized in that it comprises a stabilization means for correcting a shake of the projected image.

  Further, it is preferable that a holding member that has a plane perpendicular to the optical axis of the optical system and holds the image sensor and the light modulation unit on the plane is provided. In addition, the switching unit includes either the optical system or the holding member, a shooting position where the subject image is formed on the image sensor by the optical system, and the image modulated by the light modulation unit. It is preferable that the displacement mechanism be capable of being displaced to a projection position where light is incident on the optical system and projected.

  Further, it is preferable that the displacement mechanism slides either one of the optical system and the holding member in a direction perpendicular to the optical axis. The displacement mechanism preferably rotates at least one of the optical system and the holding member around a rotation axis extending in a direction parallel to the optical axis.

  In addition, an auxiliary optical system for forming a subject image on the image sensor when any one of the optical system and the holding member is displaced to the projection position is provided, and the shake detection unit is configured to be in the shooting state. The direction and amount of the shake are detected from the shake of the subject image formed on the image sensor by the optical system, and from the shake of the subject image formed on the image sensor by the auxiliary optical system in the projection state. It is preferable to detect the direction and amount of the shake.

  The shake correction unit includes a correction optical system that forms at least a part of the optical system, a correction optical system displacement mechanism that deflects the optical axis of the optical system by displacing the correction optical system, and the shake detection unit. Based on the detection result of the correction, the correction calculation means for calculating the direction and amount of displacement of the correction optical system so that the shake of the subject image or the projection image is corrected, and the calculation result of the shake correction calculation means Preferably, the correction optical system displacement control means drives the correction optical system displacement mechanism so that the correction optical system is displaced.

  Further, the shake correction means is based on a detection holding member displacement mechanism for displacing the holding member holding the image sensor and the light modulation unit, and a detection result of the shake detection means. Correction for calculating the direction and amount for displacing the image sensor so that shake is corrected, and for calculating the direction and amount for displacing the light modulation unit so that shake of the projected image is corrected in the projection state It is preferable to include a calculation unit and a holding member displacement control mechanism that controls driving of the correction holding member displacement mechanism so that the holding member is moved based on a calculation result of the correction calculation unit.

  The image sensor includes an effective pixel area larger than an actual effective pixel area that outputs the imaging signal, and the light modulation unit is larger than an actual effective modulation area that modulates the illumination light into the image light. An effective modulation area is provided, and the shake correction unit moves the actual effective pixel area within the effective pixel area so that shake of the subject image is corrected in the shooting state based on a detection result of the shake detection unit. Preferably, the image forming apparatus includes: a pixel area moving unit that moves the image area; and a modulation area moving unit that moves the actual effective modulation area within the effective modulation area so that a shake of the projected image is corrected in the projection state.

  According to another aspect of the present invention, there is provided a portable electronic device comprising the imaging device with a projection display function according to any one of claims 1 to 9.

  An imaging apparatus with a projection display function of the present invention includes an image sensor that captures a subject image formed by an optical system, storage means that stores image data of the captured subject image, and illumination light in accordance with the image data. A light modulation unit that modulates image light; a shooting state in which the subject image is formed on the image sensor by the optical system; and a projection state in which the image light is incident on the optical system and a projection image is projected and displayed. Based on the detection result of the shake detecting means, the shake detecting means for detecting shake given from the outside, and correcting the shake of the subject image in the photographing state and in the projecting state. And a shake correction unit that corrects the shake of the projected image, so that even if a shake occurs, a good subject image without shake can be obtained at the time of shooting. Possibility that image projected displayed on the projection target object such as a screen is difficult to see swings is eliminated.

  In addition, since the portable electronic device of the present invention is provided with the imaging device with a projection display function according to any one of claims 1 to 9, a good subject image with no shake can be obtained in the same manner, At the time of projection display, a projection image without shake is projected and displayed.

  FIG. 1 is a schematic diagram of a configuration of a mobile phone (hereinafter simply referred to as a mobile phone) 10 with photographing and projection display functions according to the present invention. The cellular phone 10 includes a telephone unit 14 composed of a receiving unit 12 and a transmitting unit 13 that are foldably connected by a hinge unit 11, and a projection display function integrated camera module (hereinafter referred to as a camera module) incorporated in the receiving unit 12. 15).

  A liquid crystal display monitor (hereinafter simply referred to as a display monitor) 20 and a reception speaker 21 are provided on the inner surface side of the reception unit 12. A transmitter microphone 23 and an operation unit 24 are provided on the inner surface side of the transmitter unit 13.

  The operation unit 24 includes a power button for turning on / off the power, a dial button for performing dial operation during transmission, a shutter button and a zoom button operated during photographing, and a mode switching button for switching the operation mode of the mobile phone 10. Etc. The mode switch button changes the operation mode of the mobile phone to a call mode for making a call, a shooting mode for shooting, a playback mode for displaying a shot image on the display monitor 20, and a projection display mode for projecting and displaying a shot image. Switch.

  FIG. 2 is a perspective view of the camera module 15 embodying the present invention. The camera module 15 has a photographing function and a projection display function, and includes a zoom lens barrel 30, a photographing / projecting combined lens (hereinafter referred to as a combined lens) 31, a holding plate 32, a CCD image sensor (hereinafter simply referred to as a lens). Image sensor) 33, light source 34, liquid crystal light valve 35, shift mechanism 36, shake correction mechanism 37, shake detection sensor 38, and the like.

  In the zoom lens barrel 30, an opening covered with a transparent protective plate 40 is formed on the front surface (upper surface in the figure) facing the subject, and a dual-purpose lens 31 is incorporated therein. The dual-purpose lens 31 is used for both photographing and projection as will be described in detail later. The combined lens 31 includes a first lens group 42 and a second lens group 43 corresponding to the correction optical system of the present invention. The first lens group 42 is slid in a direction parallel to the optical axis OA by a lens driving mechanism (not shown). Thereby, the zoom magnification can be changed from the wide angle side to the telephoto side. The second lens group 43 is held by a shake correction mechanism 37 described later, and its position is displaced in a direction perpendicular to the optical axis OA.

  The holding plate 32 is disposed on the back side of the zoom lens barrel 30. The holding plate 32 has a plane perpendicular to the optical axis OA, and an image sensor 33 and a liquid crystal light valve 35 are arranged on the front surface thereof with a predetermined interval. A through hole 45 (see FIG. 3) is formed at a position of the holding plate 32 that faces the bottom surface of the liquid crystal light valve 35. A light source 34 is provided on the back surface of the holding plate 32 so as to close the other opening of the through hole 45.

  The image sensor 33 includes a light receiving surface 33a in which a large number of photodiodes are arranged in a matrix, and captures a subject image formed on the light receiving surface 33a by the first and second lens groups 42 and 43. The image sensor 33 is driven by an image sensor driver (not shown), and outputs the electric charge accumulated for each pixel as an image pickup signal line by line in synchronization with the vertical transfer clock and the horizontal transfer clock output from the driver.

  The imaging signal output from the image sensor 33 is generated into original image data before image processing by an amplifier (not shown) and an A / D conversion processing circuit. This original image data is input to an image processing circuit 47 (see FIG. 3) and subjected to various image processing such as matrix calculation processing, white balance adjustment, and gamma correction. Image processed image data is temporarily stored in a frame memory (not shown). On the display monitor 20, the image data stored in the frame memory is displayed as a through image. When a shutter button (not shown) of the operation unit 24 is fully pressed, the stored image data is stored in a recording memory 48 (FIG. 3) such as a flash ram provided in the mobile phone 10 after being subjected to compression processing. Record).

  The light source 34 irradiates the liquid crystal light valve 35 with illumination light composed of white light through the through hole 45 (see FIG. 3). As the light source 34, for example, a white LED (light emitting diode) arranged in a matrix, an ultrahigh pressure mercury lamp, a metal halide lamp, a xenon lamp, or the like is used.

  In the present embodiment, the liquid crystal light valve 35 is composed of a transmissive color liquid crystal panel 50 (active matrix liquid crystal panel). In the projection display mode, the color liquid crystal panel 50 is driven according to the image data of the image selected by the user. As a result, the illumination light emitted from the light source 34 is transmitted through the color liquid crystal panel 50 and modulated into image light. The modulated image light is projected and displayed by the first and second lens groups 42 and 43.

  The formation of the subject image on the image sensor 33 by the dual-purpose lens 31 (first and second lens groups 42 and 43) and the projection of the image light modulated by the liquid crystal light valve 35 correspond to the displacement mechanism of the present invention. The shift mechanism 36 is switched. The shift mechanism 36 includes a cam, an actuator, and the like. The shift mechanism 36 is perpendicular to the optical axis OA and is parallel to a straight line connecting the image sensor 33 and the liquid crystal light valve 35 (hereinafter referred to as X direction). ) To slide. The shift mechanism 36 has a photographing position (see FIG. 3) where a subject image is formed on the image sensor 33 by the dual-purpose lens 31 and a projection display in which the image light modulated by the liquid crystal light valve 35 enters the dual-purpose lens 31. The holding plate 32 is slid to the position (see FIG. 4).

  When the operation mode of the mobile phone 10 is a call mode, a shooting mode, or a playback mode, the holding plate 32 is in the shooting position. When the user operates the operation unit 24 to switch the operation mode to the projection display mode, the holding plate 32 is moved to the projection display position by the shift mechanism 36. As a result, the image light is incident on the dual-purpose lens 31 and a projection image is projected and displayed by the dual-purpose lens 31.

  The photographing by the mobile phone 10 and the projected display of the photographed image are performed with the user holding the mobile phone 10 by hand. Therefore, the shake correction mechanism 37 prevents blurring of an image captured due to vibration caused by camera shake or shake of a projected image. The shake correction mechanism 37 performs so-called optical shake correction that deflects the optical axis OA by moving the second lens group 43 described above in a plane perpendicular to the optical axis OA.

  The shake correction mechanism 37 includes a first lens support plate 52 that supports the second lens group 43, a second lens support plate 53 that supports the first lens support plate 52, and a third lens that supports the second lens support plate 53. And a lens support plate 54. Actuators 56 and 57 made of piezoelectric elements are provided on the front surfaces of the second lens support plate 53 and the third lens support plate 54, respectively. A drive rod 56 a of an actuator 56 is attached to the first lens support plate 52. In addition, a drive rod 57 a of an actuator 57 is attached to the second lens support plate 53.

  The actuators 56 and 57 are controlled by an actuator control driver (see FIG. 3) 59 to expand or compress and deform the drive rods 56a and 57a linearly. The actuator 56 moves the first lens support plate 52 in the Y direction orthogonal to the X direction in the drawing. The actuator 57 moves the second lens support plate 53 and the first lens support plate 52 together in the X direction. Accordingly, the second lens group 43 can be moved in a direction perpendicular to the optical axis OA so that the optical axis OA is deflected according to the direction and amount of camera shake.

  The direction and amount of camera shake applied to the mobile phone 10 is detected by a shake detection sensor 38 provided on the back surface of the third lens support plate 54. The shake detection sensor 38 includes an X-direction acceleration sensor 38 a that detects a shake in the X direction of the mobile phone 10 and a Y-direction acceleration sensor 38 b that detects a shake in the Y direction. The position at which the camera shake detection sensor 38 is provided is not limited to the back surface of the third lens support plate 54, and may be provided at an arbitrary location in the mobile phone 10. Further, as the shake detection sensor 38, for example, an angular velocity sensor or the like may be used instead of using the acceleration sensors 38a and 38b. The shake detection sensor 38 detects a shake applied to the mobile phone 10 and inputs a shake detection signal indicating the direction and magnitude of the detected shake to the control unit 60 as shown in FIG.

  The control unit 60 controls the operation of each unit of the mobile phone 10 in response to input operations from various buttons of the operation unit 24. The control unit 60 is not shown in addition to the display monitor 20, the operation unit 24, the shift mechanism 36, the shake detection sensor 38, the recording memory 48, the image processing circuit 47, the actuator control driver 59, and the shake correction calculation circuit 62. However, an image sensor driver for controlling driving of the image sensor 33, a light valve driver for controlling driving of the liquid crystal light valve 35 (color liquid crystal panel 50), and the like are connected.

  When the camera shake occurs in the shooting mode, the camera shake correction calculation circuit 62 corrects the shake of the subject image formed on the image sensor 33 based on the camera shake detection signal from the camera shake detection sensor 38. The moving direction and amount for moving the group 43 are calculated. Specifically, the first LUT 62a in which the shake direction and amount due to camera shake and the movement direction and amount of the second lens group 43 necessary for correcting the shake of the subject image is obtained in advance. Then, when the shake detection signal is input to the control unit 60, the shake correction calculation circuit 62 refers to the first LUT 62a and calculates the movement direction and amount of the second lens group 43.

  The actuator control driver 59 drives the actuators 56 and 57 to move the second lens group 43 based on the calculation result by the shake correction calculation circuit 62. As a result, even if camera shake occurs when shooting the subject 64, a good subject image without shake can be obtained. The image data of the subject image is recorded in the recording memory 48 after being subjected to image processing and compression processing as described above.

  When the operation mode of the mobile phone 10 is switched to the projection display mode by the input operation of the operation unit 24, the holding plate 32 is moved from the photographing position to the projection display position by the shift mechanism 36 as shown in FIG. At the same time, an image list (not shown) of captured images recorded in the recording memory 48 is displayed on the display monitor 20. When the user selects an image to be projected and displayed from the image list, the image data of the selected image is read from the recording memory 48, and the color liquid crystal panel 50 of the liquid crystal light valve 35 is driven according to the image data. The Next, illumination light is emitted from the light source 34 to the liquid crystal light valve 35, and the illuminated illumination light is modulated into image light by the liquid crystal light valve 35. The modulated image light is projected onto the screen 66 by the dual-purpose lens 31, and a projected image 67 is projected and displayed on the screen 66.

  Based on the shake detection signal from the shake detection sensor 38, the shake correction calculation circuit 62 calculates a moving direction and an amount for moving the second lens group 43 so that the shake of the projected image 67 due to the shake is corrected. Specifically, as in the shooting mode, a second LUT 62b is obtained in advance that associates the shake direction and amount due to camera shake with the movement direction and amount of the second lens group 43 necessary to correct the shake of the projected image 67. Keep it. Then, the shake correction calculation circuit 62 calculates the moving direction and amount of the second lens group 43 with reference to the second LUT 62b based on the shake detection signal input to the control unit 60. The actuator control driver 59 drives the actuators 56 and 57 to move the second lens group 43 based on the calculation result of the moving direction and amount. Thereby, even if camera shake occurs during projection display, there is no possibility that the projection image 67 projected and displayed on the screen 66 will shake.

  Next, the operation of the mobile phone 10 of the present invention will be described. When shooting using the mobile phone 10, the user unfolds the reception unit 12 and the transmission unit 13 that are folded, and then operates the mode switching button of the operation unit 24 to set the operation mode of the mobile phone 10 to the shooting mode. Switch to. When the holding plate 32 is at the projection display position, the control unit 60 drives the shift mechanism 36 to slide the holding plate 32 to the photographing position. Next, the control unit 60 drives the image sensor 33 via an image sensor driver (not shown), and starts capturing the subject image formed on the image sensor 33 by the dual-purpose lens 31. Thereby, an imaging signal is output from the image sensor 33.

  The imaging signal output from the image sensor 33 is generated as original image data, and then subjected to various image processing by the image processing circuit 47. Image processed image data is temporarily stored in a frame memory (not shown). A through image is displayed on the display monitor 20 based on the image data stored in the frame memory. The user performs framing of the image to be taken while viewing the through image. When a shutter button (not shown) of the operation unit 24 is fully pressed, the image data stored in the frame memory is recorded in the recording memory 48 after being subjected to compression processing.

  When camera shake occurs when the shutter button of the operation unit 24 is fully pressed, the shake detection sensor 38 detects the shake applied to the mobile phone 10 and sends a shake detection signal indicating the direction and magnitude of the shake to the control unit 60. input. The shake correction calculating circuit 62 calculates the moving direction and amount of the second lens group 43 with reference to the first LUT 62a based on the shake detection signal input to the control unit 60. Based on this calculation result, the actuator control driver 59 drives the actuators 56 and 57 to move the second lens group 43. Thereby, even if hand shake occurs when photographing the subject 64, a good subject image without shake is obtained (see FIGS. 2 and 3).

  Next, when the captured image is projected and displayed, the user operates the mode switching button of the operation unit 24 to switch the operation mode of the mobile phone 10 to the projection display mode. When the operation mode is switched to the projection display mode, the control unit 60 drives the shift mechanism 36 to slide the holding plate 32 to the projection display position. At the same time, the control unit 60 causes the display monitor 20 to display an image list (not shown) of captured images recorded in the recording memory 48.

  When the user selects an image to be projected and displayed from the image list, a light valve driver (not shown) reads out the image data of the selected image from the recording memory 48, and the color liquid crystal panel of the liquid crystal light valve 35 according to the image data. 50 is driven. Next, the control unit 60 controls the light source 34 to irradiate the liquid crystal light valve 35 with illumination light. The irradiated illumination light is modulated into image light by the liquid crystal light valve 35. The modulated image light is projected onto the screen 66 by the dual-purpose lens 31, and a projected image 67 is projected and displayed on the screen 66 (see FIG. 4).

  When a hand shake occurs during the projection display, the hand shake detection sensor 38 detects a shake applied to the mobile phone 10 and inputs a shake detection signal indicating a shake direction and magnitude to the control unit 60. The shake correction calculating circuit 62 calculates the moving direction and amount of the second lens group 43 with reference to the second LUT 62b based on the shake detection signal input to the control unit 60. Based on this calculation result, the actuator control driver 59 drives the actuators 56 and 57 to move the second lens group 43. As a result, there is no possibility that the projected image 67 projected and displayed on the screen 66 will be shaken even if camera shake occurs.

  As described above, in this embodiment, even if camera shake occurs during shooting and projection display, the camera shake detection sensor 38 detects the direction and magnitude of the shake applied to the mobile phone 10, and the detection result is Since the shake correction mechanism 37 is driven on the basis of this, a good subject image without shake can be obtained during shooting. Further, during projection display, there is no possibility that the projected image 67 projected and displayed on the screen 66 is shaken and difficult to see.

  In the above embodiment, the shake detection sensor 38 is used to detect shake applied to the mobile phone 10, and the shake correction mechanism 37 is driven based on the detection result to move the second lens group 43. Although the type image stabilization is performed, the present invention is not limited to this. For example, the direction and amount of shake applied to the mobile phone 10 is detected from the shake of the subject image formed on the image sensor 33, and the image sensor 33 and the color liquid crystal panel 50 are controlled based on the detection result to control the electronic type. The shake correction may be performed. Hereinafter, the projection display function integrated camera module (camera module) 70 according to the second embodiment of the present invention will be described with reference to FIGS. 5 and 6. Here, the same members as those of the camera module 15 are denoted by the same reference numerals, and the description thereof is omitted.

  The cellular phone 70 is provided with a shake detection circuit 72 and an auxiliary optical system 73 instead of the shake detection sensor 38 described above, and shake correction control instead of the shake correction mechanism 37 and the shake correction calculation circuit 62 described above. A circuit 74 is provided.

  The shake detection circuit 72 detects the direction and amount of shake applied to the mobile phone 70 by detecting the shake of the subject image formed on the image sensor 33 using, for example, a known representative point matching method. Specifically, the shake detection circuit 72 divides an image of the current frame into a plurality of detection areas based on image processed image data temporarily recorded in a frame memory (not shown), and for each detection area. Decide on a representative point. Then, when the image data of the next frame is recorded in the frame memory, the shake detection circuit 72 compares the representative point of the current frame with respect to each detection area, and compares it with the image sensor 33. A shake of a subject image to be formed is detected. Note that various detection methods such as a block matching method may be used instead of the representative point matching method as long as the shake of the subject image can be detected.

  The shake detection circuit 72 detects the direction and amount of shake applied to the mobile phone 10 based on the shake of the subject image formed on the image sensor 33 by the dual-purpose lens 31 in the shooting mode (see FIG. 5). Then, the shake detection circuit 72 detects the direction and amount of shake applied to the mobile phone 10 based on the shake of the subject image formed on the image sensor 33 by the auxiliary optical system 73 in the projection display mode (FIG. 6). reference).

  The auxiliary optical system 73 is for forming a subject image on the image sensor 33 when the holding plate 32 is at the projection display position. The auxiliary lens group 76 and a lens barrel 77 in which the lens group 76 is incorporated. It consists of. The lens barrel 77 is attached to the zoom lens barrel 30 by a holding member 78. The holding member 78 holds the lens barrel 77 at a position where a subject image can be formed on the image sensor 33 by the auxiliary lens group 76 when the holding plate 32 is at the projection display position. By providing the auxiliary optical system 73, a subject image can be formed on the image sensor 33 even in the projection display mode, and the shake direction and amount of the shake applied to the mobile phone 70 can be detected by the shake detection circuit 72.

  The shake correction control circuit 74 includes a pixel area moving circuit 80 and a modulation area moving circuit 81. The pixel area moving circuit 80 moves the actual effective pixel area of the light receiving surface 33a of the image sensor 33 to which the imaging signal is output. Here, the light receiving surface 33a of the image sensor 33 has an effective pixel area larger than the actual effective pixel area for outputting the imaging signal. Accordingly, by moving the actual effective pixel area within the effective image area, the shake of the subject image formed on the image sensor 33 can be corrected.

  The pixel area moving circuit 80 moves the actual effective pixel area and the amount so that the shake of the subject image formed on the image sensor 33 is corrected based on the detection result of the shake detection circuit 72 in the shooting mode. Is calculated. In this case as well, it is preferable to obtain in advance an LUT that associates the shake direction and amount due to camera shake with the movement direction and amount of the actual effective pixel region. Then, the pixel area moving circuit 80 drives an image sensor driver (not shown) based on the calculated result to move the actual effective pixel area. Thereby, even if hand shake occurs, a good subject image without shake is obtained (see FIG. 5).

  The modulation area moving circuit 81 moves the actual effective modulation area of the color liquid crystal panel 50 of the liquid crystal light valve 35. The color liquid crystal panel 50 also has an effective modulation area larger than the actual effective modulation area for projection. Therefore, in the projection display mode, the modulation area moving circuit 81 moves the actual effective modulation area so that the shake of the projection image 67 projected and displayed on the screen 66 is corrected based on the detection result by the shake detection circuit 72. And calculate the quantity. In this case as well, it is preferable to obtain in advance an LUT that associates the shake direction and amount due to camera shake with the movement direction and amount of the actual effective modulation region. Then, the modulation area moving circuit 81 drives a light valve driver (not shown) based on the calculated result to move the actual effective modulation area. As a result, even if hand shake occurs, there is no possibility that the projected image 67 projected and displayed on the screen 66 similarly shakes (see FIG. 6).

  As described above, also in the second embodiment, when camera shake occurs, the shake detection circuit 72 detects shake of the subject image formed on the image sensor 33, and based on the detection result, the pixel in the shooting mode is detected. Since the actual effective pixel area of the image sensor 33 is moved by the area moving circuit 80, a good subject image without shake can be obtained. Also, since the actual effective modulation area of the liquid crystal light valve 35 is moved by the modulation area moving circuit 81 in the projection display mode based on the detection result of the shake detection circuit 72, the projection image 67 projected and displayed on the screen 66 is displayed. There is no risk of shaking and difficult to see.

  In the above-described embodiment, the shift position 36 is used to modulate the holding plate 32 and the imaging position where the subject image is formed on the image sensor 33 by the dual lens 31 (see FIGS. 3 and 5) and the liquid crystal light valve 35. Although the image light thus made is slid to the projection display position (see FIGS. 4 and 6) where it enters the dual-purpose lens 31, the present invention is not limited to this. For example, as shown in FIG. 7, a rotating shaft 85 extending in a direction parallel to the optical axis OA (see FIG. 2) is attached to the holding plate 32. The holding plate 32 may be rotated about the rotation shaft 85 by a motor 86 connected to the rotation shaft 85 via a drive transmission mechanism (not shown).

  In the above embodiment, the holding plate 32 is slid by the shift mechanism 36. However, the present invention is not limited to this, and instead of sliding the holding plate 32, a zoom lens barrel is used. 30 may be slid. Further, as described with reference to FIG. 7 described above, the zoom lens barrel 30 and the holding member 78 between the zoom lens barrel 30 and the auxiliary optical system 73 extend in a direction parallel to the optical axis OA. You may make it rotate centering around the rotating shaft.

  In the above embodiment (camera module 15), so-called optical camera shake correction is performed in which the shake correction mechanism 37 is driven to move the second lens group 43 based on the shake detection signal from the shake detection sensor 38. However, the present invention is not limited to this. Instead of moving the second lens group 43, the holding plate 32 may be moved using a shake correction mechanism 90 as shown in FIG. 8, for example.

  The shake correction mechanism 90 moves the holding plate 32 in a direction perpendicular to the optical axis OA. The shake correction mechanism 90 includes a holding member 92 that holds the holding plate 32, a first support plate 93 that is formed integrally with the holding member 92, a second support plate 94 that supports the first support plate 93, and a second support. A third support plate 95 that supports the plate 94, an actuator 96 that moves the first support plate 93 in the Y direction in the figure, and a second support plate 94 that moves together with the first support plate 93 in the X direction in the figure. And an actuator 97. Both actuators 96 and 97 are controlled by an actuator control driver 98.

  In this case as well, based on the shake detection signal from the shake detection sensor 38, the shake of the subject image formed on the image sensor 33 and the shake of the projection image projected and displayed on the screen or the like are corrected. The holding plates 32 may be moved by driving the actuators 96 and 97.

  In each of the above embodiments, a mobile phone with a projection display function in which a camera module integrated with a projection display function is incorporated has been described as an example. However, the present invention is not limited to this, and such a projection is possible. The present invention can be applied to various portable electronic devices such as a digital camera, a PDA, and an electronic notebook in which a display function integrated camera module is incorporated.

It is the schematic of the mobile telephone with an imaging | photography and projection display function which implemented this invention. It is a perspective view of a camera module integrated with a projection display function incorporated in the mobile phone. It is sectional drawing of the camera module at the time of imaging | photography mode. It is sectional drawing of the camera module at the time of projection display mode. It is sectional drawing of the camera module of 2nd Embodiment which performs electronic camera shake correction based on the shake of the to-be-photographed image imaged by an image sensor, and shows the state at the time of imaging | photography mode. It is sectional drawing of the camera module of 2nd Embodiment, and shows the state at the time of projection display mode. It is sectional drawing of the camera module of other embodiment using the rotating shaft instead of the shift mechanism. It is a perspective view of the camera module of other embodiments which performs camera shake correction by moving a holding plate instead of moving the 2nd lens group.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Mobile phone with imaging | photography and projection display function 15 Projection display function integrated camera module 31 Combined lens 33 Image sensor 35 Liquid crystal light valve 36 Shift mechanism 37 Shake correction mechanism 38 Shake detection sensor 43 2nd lens group 56,57 Actuator 59 Actuator control Driver 62 shake correction calculation circuit

Claims (10)

An image sensor that captures a subject image formed by an optical system and outputs an imaging signal;
Storage means for storing image data obtained by performing signal processing on the output imaging signal;
A light modulation unit that irradiates illumination light from a light source and modulates the illumination light into image light according to the image data read from the storage means;
Switching between a photographing state in which the subject image is formed on the image sensor by the optical system and a projection state in which the modulated image light is incident on the optical system and a projected image is projected and displayed on the projection target. Possible switching means;
A shake detection means for detecting the direction and amount of shake given from the outside with respect to a housing housing at least the image sensor and the light modulation unit;
A projection display function comprising: a shake correction unit that corrects shake of the subject image in the shooting state and corrects shake of the projection image in the projection state based on a detection result of the shake detection unit. An image pickup device.   The imaging with a projection display function according to claim 1, further comprising a holding member that has a plane perpendicular to the optical axis of the optical system and holds the image sensor and the light modulation unit on the plane. apparatus. The switching means is
Either one of the optical system and the holding member is incident on the optical system by an imaging position where the subject image is formed on the image sensor by the optical system, and the image light modulated by the light modulation unit. The imaging device with a projection display function according to claim 2, wherein the imaging device is a displacement mechanism that can be displaced to a projection position projected.   4. The imaging apparatus with a projection display function according to claim 3, wherein the displacement mechanism slides either one of the optical system and the holding member in a direction perpendicular to the optical axis.   4. The projection display according to claim 3, wherein the displacement mechanism rotates at least one of the optical system and the holding member about a rotation axis extending in a direction parallel to the optical axis. Imaging device with function. An auxiliary optical system for forming a subject image on the image sensor when either one of the optical system and the holding member is displaced to the projection position;
The shake detection means detects the direction and amount of the shake from the shake of the subject image formed on the image sensor by the optical system in the photographing state, and the image sensor by the auxiliary optical system in the projection state. 6. The imaging apparatus with a projection display function according to claim 3, wherein the direction and amount of the shake are detected from the shake of the subject image formed on the image. The shake correction means includes
A correction optical system forming at least a part of the optical system;
A correction optical system displacement mechanism that deflects the optical axis of the optical system by displacing the correction optical system;
Correction calculation means for calculating a direction and an amount for displacing the correction optical system so that shake of the subject image or the projection image is corrected based on a detection result of the shake detection means;
7. The correction optical system displacement control means for driving the correction optical system displacement mechanism so that the correction optical system is displaced based on the calculation result of the shake correction calculation means. An imaging apparatus with a projection display function according to claim 1. The shake correction means includes
A correction holding member displacement mechanism for displacing the holding member holding the image sensor and the light modulation unit;
Based on the detection result of the shake detection means, the direction and amount of displacement of the image sensor are calculated so that the shake of the subject image is corrected in the shooting state, and the shake of the projection image is corrected in the projection state. Correction calculating means for calculating the direction and amount for displacing the light modulation unit,
7. A holding member displacement control mechanism that controls driving of the correction holding member displacement mechanism so that the holding member is moved based on a calculation result of the correction calculating means. An imaging apparatus with a projection display function according to claim 1. The image sensor includes an effective pixel area larger than an actual effective pixel area that outputs the imaging signal, and the light modulation unit has an effective modulation larger than an actual effective modulation area that modulates the illumination light into the image light. With areas,
The shake correction means includes
Based on the detection result of the shake detection means, a pixel area moving means for moving the actual effective pixel area within the effective pixel area so that shake of the subject image is corrected in the shooting state, and in the projection state, 7. The projection display function according to claim 1, further comprising modulation area moving means for moving the actual effective modulation area within the effective modulation area so that a shake of the projected image is corrected. An image pickup device. A portable electronic device comprising the imaging device with a projection display function according to claim 1.

Images