JP2008067081A - Portable imaging display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a portable imaging display device capable of restraining the deviation of a projection display image following the vibration of equipment itself corresponding to the widening of a usage situation generated following the portability of the equipment, and projecting the captured image at the same position even if a moving object is followed for photographing in a fixed range. <P>SOLUTION: The portable imaging display device comprises an imaging section for obtaining a captured image by photographing the object, and a projection display section for projecting a display image on a screen, thus detecting a change in the imaging light axis of the imaging section and correcting a change in the projection light axis of the projection display section, based on the detected change in the imaging light axis. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging display device that is portable and has both image capturing and projection functions.

  With the progress of digital technology in recent years, portable devices equipped with an image photographing function have become more widespread, and the improvement in image quality and the miniaturization of the entire device are progressing. In such devices that are commercialized as digital still cameras, digital video cameras, mobile phones, etc., there is a demand for downsizing the device size, but a display that displays captured images as the size is reduced Because of the downsizing, there was a problem that many people could not watch it at the same time. In order to solve the conflicting problems, there has been proposed a portable device having an imaging function that has an image projection function.

  For example, in the video camera 31 whose block diagram of the internal structure is shown in FIG. 5, a captured image with sound obtained by the imaging unit 37 and the microphone 34 is projected onto a nearby screen by the projector unit 38 and the speaker 35. It is possible to do. The control unit 36 that controls these series of operations can also be used as a TV conference system by controlling the communication unit 32 connected to the antenna 33. Such a portable image pickup display device has an advantage that a picked-up image can be easily viewed by a large number of people by projecting and displaying an image picked up by an image pickup unit with one device (patent) Reference 1).

Further, although not a portable type, an imaging display device as shown in FIG. 6 has been proposed as a method for projecting and displaying a captured image on a subject at the same time as imaging. In this apparatus, the subject 40 is imaged on the imaging surface 46 of the camera 45 through the lens 43 and the filter 41, and this captured image is image-processed while being confirmed by the image processor 47 on the monitor 48 or the like. The processed image is sent to the liquid crystal display device 49 for projection via the liquid crystal controller 50 and is projected via the lens 44 by the light from the light source 51 for projection. At this time, since the projected image follows the same optical path as the captured image by the half mirror (or dichroic mirror) 42, the image processed image can be directly superimposed on the imaged object. Because of such advantages, this imaging display device can be used for projection display of the internal structure superimposed on the wall surface of a house, for example (Patent Document 2).
JP 2001-24926 A JP 2005-311837 A

  In the imaging display device as described above, it is possible to immediately confirm the captured image by another person, and it has the advantage that the image captured by the imaging unit can be projected and confirmed simultaneously. These advantages become more effective by making this imaging display device portable. Here, along with the progress of miniaturization technology for making a device portable, it has become possible to realize a portable imaging display device having these functions, but the device itself can be made smaller and portable. The problem that the projected image is blurred at the time of projection has been regarded as a problem the more it is made. In particular, in order to take advantage of the portable imaging display device that can easily follow and image a moving subject, the captured image can be used as it is when the subject that moves within a certain range is imaged. Projected display is assumed, and not only camera shake correction as a simple portable projection display device, but also cancels the movement of equipment for imaging, and projects the display image at a fixed position. Expected to continue to do so.

  The present invention solves the above-described problems in the prior art, copes with a wide range of usage situations that occur as a device becomes portable, suppresses blurring of a projected display image due to vibration of the device itself, and within a certain range. An object of the present invention is to provide a portable imaging display device capable of projecting and displaying a captured image at the same position even when shooting a subject that moves in the following manner.

  In order to solve the above-described problems, a portable imaging display device of the present invention includes an imaging unit that captures an image of a subject and obtains a captured image, and a projection display unit that projects a display image on a screen. A change in the axis is detected, and a change in the projection optical axis of the projection display unit is corrected based on the detected change in the imaging optical axis.

  In this way, the portable image display device of the present invention suppresses the shake caused by the vibration of the device itself during the projection of the display image, and at the time of shooting a subject that moves within a certain range. Also, the position of the projected image can be kept constant, and good visibility of the projected image can be ensured.

  In the above-described portable imaging display device of the present invention, it is preferable to detect the change in the imaging optical axis from the temporal change in the captured image, and display area marker data for detecting the change in the imaging optical axis Is preferably superimposed on the display image and projected by the projection display unit. In this way, the movement of the device itself can be accurately detected and reflected in the correction of the projected display image.

  Further, by sensing the change in the imaging optical axis with a sensor, the projection display image can be corrected more accurately in addition to the information based on the temporal change in the captured image. Even when information based on the change cannot be obtained sufficiently, it is possible to correct the projection display image by detecting the movement of the device.

  Further, it is desirable that the subject being imaged by the imaging unit is projected by the projection display unit simultaneously with imaging. By doing in this way, the further use expansion which utilized the portability of the apparatus can be aimed at.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.
(Embodiment 1)
FIG. 1 is a block diagram illustrating a configuration example describing a first embodiment of a portable imaging display device of the present invention. The portable imaging display device of the present invention has an imaging unit that images the subject 1 with an imaging element 4 such as a CCD sensor through an imaging lens 3, and a captured image is obtained from the signal from the imaging element by an imaging signal processing unit 9. In addition, a projection display unit that projects a display image displayed on the display device 6 such as an LCD or DMD onto the screen 2 through the projection lens 5 by light from the light source 7 is provided. Here, what is shown as the screen 2 is a concept indicating an area in which a display image is projected and displayed, and is not limited to a regular screen for a projector, for example, a white board, a wall surface, or other media on which other images are projected. It is a concept that points to everything. Note that a display signal processing unit 10 that adjusts the display image on the display device 6 and a light source luminance control unit 8 that adjusts the luminance of the light source 7 are provided to make the display image to be projected appropriate.

  A feature of the portable imaging display device of the present embodiment is that a change in the imaging optical axis is detected by processing the temporal change of the captured image captured by the imaging unit by the optical axis correction calculation unit 12, and the detected imaging The projection optical axis correction signal 14 is generated based on the change of the optical axis, and the projection optical axis is changed by moving the projection lens 5 of the projection display unit by the lens driving unit 11. The imaging optical axis indicates the direction of the center line of the imaging optical system, and the change in the imaging optical axis is intentional for the user when imaging a subject moving within a certain range. The movement of the portable imaging display device caused by the movement of the portable imaging display device unintentionally caused by the user, such as a change in the imaging direction of the portable imaging display device, a so-called camera shake or a slight vibration of the device case 13 itself. Of both. Similarly, the projection optical axis indicates a center line directed in the projection direction of the optical system of the projection display unit, and the change is the direction in which the image is projected by adjusting the projection lens system or the like. It means to move. As shown in FIG. 1, in this embodiment, since the imaging optical axis and the projection optical axis are orthogonal, the direction of change differs by 90 degrees, but the change in the same angle as the change in the imaging optical axis This also occurs on the projection optical axis.

  Next, detection of an imaging optical axis change and generation of the correction signal 14 in the optical axis correction calculation unit 12 will be specifically described.

  First, a high-pass filter process is performed on the imaging signal, an edge of the captured image is detected, and the appearance image S of the subject is temporarily stored. Next, after a certain time Δt1 has elapsed, high-pass filter processing is again performed on the image signal, and the appearance image St1 of the subject at time t1 is temporarily stored.

  Here, considering the case where the S image is moved by + k pixels in the + x-axis direction and the case where the S image is moved by -k pixels in the -x-axis direction, an S + k image and an Sk image are formed. And the difference amount of all pixels of the S image and the Sk image are compared, and it is determined that the optical axis has changed to a smaller numerical value. That is, when (St1 image) − (S + k image) <(St1 image) − (S−k image), it is determined that the imaging optical axis is shaken in the + x-axis direction.

  Next, after the time Δt <b> 2 elapses, the high-pass filter process is performed again on the imaging signal, thereby temporarily storing the appearance image St <b> 2 of the subject. As before, when (St2 image) − (St1 + k image) <(St2 image) − (St1-k image), it is determined that the imaging optical axis is changing in the + x-axis direction. At this time, if (St1 image) − (S + k image) <(St2 image) − (St1 + k image), the change in the imaging optical axis is larger. Conversely, when (St1 image) − (S + k image)> (St2 image) − (St1 + k image), the change in the imaging optical axis is converged.

  If (Stα image) − (Stα−1 + k image) <(Stα + 1 image) − (Stα + k image) and the amount of change increases in the same direction, the k value is set to 2k values. Alternatively, it may be replaced with a 3k value to increase the ability to follow changes in the optical axis. As described above, the high-pass filter image St of the captured image is determined every fixed time Δt with respect to the imaging signal, and the optical axis change direction + − and the change amount k are determined. Actually, the detection of the change in the optical axis is simultaneously performed in the three axis directions of XYZ.

  Further, the optical axis correction calculation unit 12 generates and displays a projection optical axis correction signal 14 for correcting the projection optical axis based on the change direction and change amount data of the imaging optical axis detected as described above. The projection lens 5 is driven by the lens driving unit 11 so as not to move the projected image. In this way, is the user intended, such as camera shake when the user is shooting following a subject moving within a certain range or holding the portable imaging display device by hand? It is possible to detect changes in the imaging optical axis that occur regardless of whether or not, and to correct the changes in the projection optical axis, thereby preventing the visibility of the image projected and displayed by the projection display unit from moving. can do.

  In the specific example of the detection of the change in the optical axis of the image pick-up by the optical axis correction calculation unit 12, an example of obtaining the difference amount between all the pixels of the S image and the S + k image when calculating the change amount of the optical axis is shown. However, the present invention is not limited to this, and a rough determination may be made based on the difference amount of the representative pixel. Furthermore, after making a rough determination, the number of pixels for which the difference amount is obtained as necessary. You may make it judge more finely by increasing. In actual use, the change in the imaging optical axis is often caused by the habit of the user, and the change amount k value is often a value unique to the photographer. For this reason, for example, by detecting and storing the average value of the amount of change in the past several times, by using this as a correction amount in the next change amount correction, it is possible to perform more rapid correction. There is a possibility that can be done.

(Embodiment 2)
Next, a second embodiment of the portable imaging display device of the present invention will be specifically described with reference to FIG. FIG. 2 is a block diagram illustrating a specific configuration example of the portable image display device according to the present embodiment.

  As shown in FIG. 2, in the portable imaging display device of the present embodiment, an imaging unit including an imaging lens 3 and an imaging element 4, a projection display unit including a display device 6, a projection lens 5, and a light source 7, and imaging A signal processing unit 9, a display signal processing unit 10, and a light source luminance control unit 8 are included. This embodiment is different from the portable imaging display device according to the first embodiment shown in FIG. 1 in that the acceleration applied to the device case 13 is not detected from the temporal change of the captured image, but from the temporal change of the captured image. And a sensor that detects physical quantities such as movement of the apparatus itself. In the specific example shown in FIG. 2, three sensors, an X-axis gyro sensor unit A (15), a Y-axis gyro sensor unit A (16), and a Z-axis gyro sensor unit A (17) respectively corresponding to the three-dimensional direction, are provided. It has been. In these three gyro sensor units, since the gyro sensor detects the movement amount and the movement time in the corresponding axial direction and detects the movement of the apparatus itself, the change in the imaging optical axis is detected as a result. Become. In addition, the three gyro sensor units of the present embodiment detect changes in the imaging optical axis and correct the correction so as not to move the position of the projected image in each axial direction. A signal for moving in the opposite direction is sent to the lens driving unit 11 for the same movement time with the same movement amount as detected. That is, the X-axis gyro sensor A (15) moves the lens in the X-axis direction reverse direction, and the Y-axis gyro sensor A (16) moves the lens in the Y-axis direction reverse. The Y-axis direction correction signal 19 and the Z-axis gyro sensor part A (17) generate a Z-axis direction correction signal 20 for moving the lens in the opposite direction to the Z-axis direction, respectively.

  For example, when a user is projecting and displaying an image on the screen 2 and tracks a moving subject, the movement distance Lx in the unit time T in the X-axis direction of the present apparatus is set as the X-axis gyro. It is detected by the sensor part A (15). At the same time, the X-axis gyro sensor unit A (15) outputs a drive signal 18 for driving the lens by lx in the X-axis direction per unit time T so as to be similar to the detected movement amount and unit time as the X-axis lens drive. This is sent to the lens driving unit 11. By performing the same for the Y-axis direction and the Z-axis direction, it is possible to give a change in the projection optical axis for correcting the change in the imaging optical axis to the three-dimensional direction.

  Here, in the case of imaging following a moving subject, detection of a change in the imaging optical axis when the imaging optical axis changes in the xy plane or when the imaging optical axis changes by rotation, and a correction method thereof Will be described in detail. A change in the imaging optical axis in such a case can be detected by two acceleration sensors in the x-axis direction and the y-axis direction. For example, when the acceleration in the x-axis direction at time t is xa and the acceleration in the y-axis direction is yb, and this is detected, the magnitude of the acceleration obtained by combining √ (square of xa + square of yb) Now, tan (yb / xa) represents the rotation angle of the synthesized acceleration.

  When the magnitude of the acceleration and the rotation angle are detected, only tan (yb / xa) is detected in the direction opposite to the direction in which the lens rotation actuator of the display device is detected as the rotation direction of the imaging optical axis at time t. By rotating it, the projection optical axis can be corrected. In this case, the lens rotation actuator may rotate both the projection lens 5 and the display device 6, or the moving coil may be rotated by a small angle within a range of angles limited to only the lens 5.

  In the above description, the change in the imaging optical axis has been described in the xy plane that is two-dimensional with the x-axis and the y-axis. The same applies to the case of the two-dimensional yz plane.

  Next, a development example of this embodiment is shown in FIG. The portable image pickup display device of FIG. 3 has, in addition to that of FIG. 2, another system and three gyro sensor portions B corresponding to the three-dimensional direction. Specifically, there are three sensors: an X-axis gyro sensor unit B (21), a Y-axis gyro sensor unit B (22), and a Z-axis gyro sensor unit B (23).

  Thus, by having two gyro sensors for each three-dimensional axial direction, it becomes possible to install the sensors at two different locations of the device case 13 of the portable imaging display device. Thus, it is possible to easily detect whether the change in the imaging optical axis is parallel motion or rotational motion. As a result, when a subject moving within a certain range is tracked and imaged by the imaging unit, whether the movement of the imaging optical axis is a parallel motion or a rotational motion, it is detected and a projected display image is detected. To be fixed on the screen 2, the projection optical axis can be changed.

  In the present embodiment, the change in the imaging optical axis of the portable imaging display device is detected by a gyro inside the device. However, as a sensor in the portable imaging display device of the present invention, an optical gyroscope or a vibrating gyroscope is used. Other gyros such as ring laser gyros, other angular velocity sensors, vibration sensors, piezoelectric sensors, and the like can be used.

  Further, as in the present embodiment, when a change in the imaging optical axis is detected by a sensor attached to the apparatus, the imaging lens 3 is also provided with a lens drive system similar to that of the projection lens 5, and image capturing is performed. It can be used as a camera shake correction means. Specifically, camera shake or the like can be detected by a sensor, and camera shake correction can be performed simultaneously on the imaging optical axis and the projection optical axis of the device.

(Embodiment 3)
Next, a portable imaging display device according to a third embodiment of the present invention will be described with reference to FIG. FIG. 4 is a block diagram showing a specific configuration example of the portable imaging display device of the present embodiment. The portable imaging display device of this embodiment is characterized in that a change in the imaging optical axis is detected by both a temporal change in the captured image and a sensor that detects a physical movement of the device.

  At the same time, in the present embodiment, the display area marker data is projected onto the imaging area from the projection unit so that the temporal change of the captured image can be easily detected from the captured image captured by the imaging element 4 through the imaging lens 3. I am doing so. Specifically, in addition to the projection image projected on the screen 2 by the imaging display unit, display area marker data is represented on the display device, and this display area marker data is displayed on the subject 1 captured by the imaging unit with the half mirror 27. Project to. The display area marker data is specifically a “+” display that is displayed at four locations in the center or the periphery, and when this is imaged by the imaging unit, the imaging optical axis is the X axis, Y axis, or Z axis. It is possible to easily detect how much the change has occurred with respect to the three directions.

  Then, based on the captured image processed by the imaging signal processing unit 9, the optical axis correction calculation unit 12 detects a change in the imaging optical axis from the temporal change. The imaging optical axis change information in the X-axis direction is combined with the imaging optical axis change information in the X-axis gyro sensor unit A (15) in which the X-axis correction calculation unit 24 detects the movement of the physical device in the X-axis direction. The X-axis direction correction signal 18 is applied to the lens driving unit 11 of the projection display unit. Similarly, the imaging optical axis change amount in the Y-axis direction is the imaging optical axis change in the Y-axis gyro sensor unit A (16) in which the Y-axis correction calculation unit 25 detects the movement of the physical device in the Y-axis direction. It is combined with the information and applied to the projection lens driving unit 5 as the Y-axis direction correction signal 19. Further, the amount of change in the imaging optical axis in the Z-axis direction is the information on the change in the imaging optical axis in the Z-axis gyro sensor unit A (17) that detects the movement of the physical device in the Z-axis direction by the Z-axis correction calculation unit 26. The synthesized signal is applied to the projection lens driving unit 5 as the Z-axis direction correction signal 20.

  Thus, by detecting the change of the imaging optical axis by both the method of detecting from the temporal change of the captured image and the method of physically detecting the movement of the apparatus by a sensor such as a gyro, As the width increases, the detection accuracy improves, and the projection optical axis can be corrected accurately and quickly.

  A specific adjustment method in this case will be described below. As marker data display, for example, “+” is displayed at four locations around the projection screen. If shake or rotation is reduced, “+” is not displayed. When displaying the marker, this “+” is imaged, and four pixels (x0, y0) (x1, y1) (x2, y2) (x3, y3) where the center of the “+” mark is imaged are recorded. After a certain time ΔT, “+” is imaged again, and recording is performed for the four pixels (x0ΔT, y0ΔT) (x1ΔT, y1ΔT) (x2ΔT, y2ΔT) (x3ΔT, y3ΔT) captured at the center of “+”. Difference values (x0ΔT-x0, y0ΔT-y0), (x1ΔT-x1, y1ΔT-y1), (x2ΔT-x2, y2ΔT-y2), (x3ΔT-x3, y3ΔT-y3) before and after the passage of time. And the change vector and change amount of each point are calculated. When all the points are moving in the X direction or the Y direction by the same amount, it is a parallel movement. On the other hand, when the vector values are different, rotation is performed. In this way, it is possible to easily determine whether the movement is a parallel movement or a rotational movement.

  Further, the number of pixels on the imaging screen imaged by the imaging unit may be different from the number of pixels to be projected. For example, when the horizontal is M pixels and the vertical is N pixels, and the number of imaging pixels (M1, N1) is different from the number of projection pixels (M2, N2), the imaging pixel S (x, y) is used as the projection pixel D. By converting (x, y) = (M2 × x / M1, N2 × y / N1), data processing can proceed.

  In the present embodiment, the use of the display area marker data for detecting the change in the imaging optical axis due to the change in the captured image over time, the change in the imaging optical axis, the temporal change in the captured image, the gyro, etc. The display area marker data usage is described as the first embodiment for detecting the change of the imaging optical axis from the temporal change of the captured image. Needless to say, the effect is also exhibited when combined with.

  As described above, various embodiments of the portable imaging display device according to the present invention have been shown. However, an image captured and projected by the portable imaging display device of the present invention may be a still image or a moving image. good. Further, as an example of changing the projection optical axis, the example in which the projection lens 5 is moved by the lens driving unit 11 has been described. Specifically, a moving coil method or a lens actuator method can be used, and a method for moving the imaging lens 5 is used. In addition, the optical axis of projection can be changed using an optical element such as a reflecting mirror or a vari-angle prism. Furthermore, a known method of changing the projection optical axis such as a method of moving the position of the display device itself of the projection display unit or a method of moving the image display position on the display device can be used.

  The portable image-pickup display device according to the present invention suppresses the shake caused by the vibration of the device itself when projecting the display image, and the position of the projection image also when shooting a subject that moves within a certain range. Can be kept constant. For this reason, the visibility of a projection image can be improved and the use of a portable imaging display device can be expanded.

The block diagram which shows the structure of the portable imaging display apparatus which concerns on the 1st Embodiment of this invention. The block diagram which shows the structure of the portable imaging display apparatus which concerns on the 2nd Embodiment of this invention. The block diagram which shows the structure of the example of expansion | deployment of the portable imaging display apparatus which concerns on the 2nd Embodiment of this invention. The block diagram which shows the structure of the portable imaging display apparatus which concerns on the 3rd Embodiment of this invention. The block diagram which shows the structure of the conventional portable imaging display apparatus The block diagram which shows the structure of the conventional imaging display apparatus

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Subject 2 Screen 3 Imaging lens 4 Imaging element 5 Projection lens 6 Display device 7 Light source 8 Light source luminance control part 9 Imaging signal processing part 10 Display signal processing part 11 Lens drive part 12 Optical axis correction calculation part 13 Device case 14 Projection optical axis Correction signal 15 X-axis gyro sensor A
16 Y-axis gyro sensor A
17 Z-axis gyro sensor part A
18 X-axis direction correction signal 19 Y-axis direction correction signal 20 Z-axis direction correction signal 21 X-axis gyro sensor B
22 Y-axis gyro sensor B
23 Z-axis gyro sensor B
24 X-axis correction calculation unit 25 Y-axis correction calculation unit 26 Z-axis correction calculation unit 27 Half mirror 31 Video camera 32 Communication unit 33 Antenna 34 Microphone 35 Speaker 36 Control unit 37 Imaging unit 38 Projector unit 40 Subject 41 Filter 42 Half mirror 43 Lens 44 Lens 45 Camera 46 Imaging surface 47 Image processor 48 Monitor screen 49 Liquid crystal display device 50 Liquid crystal controller 51 Projection light source

Claims (5)

  An imaging unit that captures an image of a subject and obtains a captured image, and a projection display unit that projects a display image onto a screen, detects a change in the imaging optical axis of the imaging unit, and detects the detected change in the imaging optical axis. A portable imaging display device that corrects a change in the projection optical axis of the projection display unit based on the projection optical axis.   The portable imaging display device according to claim 1, wherein a change in the imaging optical axis is detected from a temporal change in the captured image.   The portable imaging display device according to claim 2, wherein display area marker data for detecting a change in the imaging optical axis is projected on the projection display unit by superimposing the display area marker data on the display image.   The portable imaging display device according to any one of claims 1 to 3, wherein a change in the imaging optical axis is detected by a sensor.   The portable imaging display device according to any one of claims 1 to 4, wherein a subject being imaged by the imaging unit is projected by the projection display unit simultaneously with imaging.

Images