JP5742301B2 - Image projection apparatus, digital camera and optical apparatus - Google Patents

Description

The present invention relates to an image projection apparatus , a digital camera, and an optical apparatus .

  2. Description of the Related Art An image display system that searches for a stored photographed image and displays it on the display screen according to the position and direction (tilt) of a mobile terminal having a display screen (display panel) such as a mobile phone is known. (Patent Document 1).

JP 2010-134881 A

  By the way, recently, portable projectors and digital cameras with projectors have been put into practical use. By applying the above-described conventional technology to such an image projection apparatus, it is considered to search for a stored captured image according to the position and direction of the image projection apparatus and to project and display a flat plane as a projection plane. In this case, from the state in which the image is displayed with the projection direction of the image projection apparatus (the direction of the optical axis of the projection optical system) set substantially perpendicular to the projection plane, the shooting direction is different from the currently displayed image. If the image projection apparatus is tilted (changes direction) in order to view another image (adjacent image), the projected image is distorted because the optical axis is not perpendicular to the projection plane. There is a problem in that the display quality of an image is deteriorated by causing a phenomenon such as a change in size or brightness or out of focus.

An object of the present invention is to provide an image projection apparatus , a digital camera, and an optical apparatus that can realize good display quality when the posture of the apparatus is changed in order to change an image to be projected and displayed.

An image projection apparatus according to the present invention is acquired by an image recording unit that records a plurality of images, an image projection unit that projects an image, a direction acquisition unit that acquires the direction of the image projection unit, and the direction acquisition unit. An image selection unit that selects an image to be projected from images recorded in the image recording unit based on the determined direction, and an image selected by the image selection unit is corrected based on the direction of the image projection unit An image correcting unit configured to detect the plurality of images within a predetermined range with respect to the image selected by the image selecting unit when projecting the image. Based on the predicted direction, the image selection unit selects an image to be projected from the images corrected by the image correction unit based on the direction acquired by the direction acquisition unit. that And butterflies.
According to another aspect of the present invention, there is provided a digital camera including a photographing unit that captures an image of a subject and records the image on the image recording unit, and the image projection apparatus according to the present invention.
Further, an optical apparatus according to the present invention is photographed by a photographing unit that photographs a subject, a projection unit that projects a selected selected image, a detection unit that detects a projection direction by the projection unit, and the photographing unit. A correction unit that corrects the image captured at an angle within a predetermined range from a direction detected by the detection unit, using the direction detected by the detection unit, and the image corrected by the correction unit. And a selection unit that selects the selected image.

  According to the present invention, it is possible to realize good display quality when the posture of the apparatus is changed in order to change an image to be projected and displayed.

It is a block diagram which shows the structure of the image projector of embodiment of this invention. It is a flowchart which shows the process of the image projector of embodiment of this invention. It is a figure for demonstrating an example of the image correction in embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the image projection apparatus of this embodiment. The image projector 1 is a digital camera with a projector, for example. The image projection apparatus 1 includes a control unit 11, an imaging unit 12, an image recording unit 13, an operation unit 14, a position information acquisition unit 15, an orientation information acquisition unit 16, a distance information acquisition unit 17, a tilt information acquisition unit 18, and movement information acquisition. Unit 19, time information acquisition unit 20, server image acquisition unit 21, and image projection unit 22 are roughly provided.

  The control unit 11 is configured by a CPU, a memory, and other peripheral circuits, and comprehensively controls the entire image projection apparatus 1. In addition, the memory which comprises the control part 11 is volatile memories, such as SDRAM, for example. This memory includes a work memory for developing a program when the CPU executes the program, and a buffer memory for temporarily recording data (such as data relating to a corrected image described later).

  The image capturing unit 12 captures an image of a subject formed through the photographing lens 23, converts it into an electrical signal, and outputs an image signal such as a CMOS, and converts an analog signal output from the image sensor into a digital signal. An A / D conversion circuit that performs various image processing on the image data that has been A / D converted by the A / D conversion circuit, and the image data processed by the image processing circuit has a predetermined value. The additional information is added and recorded in the image recording unit 13 as a captured image.

  The image recording unit 13 includes an information recording medium such as a memory card that is detachably mounted in a slot (not shown). The image recording unit 13 records a photographed image photographed by the photographing unit 12 together with additional information. The The captured image includes a still image and a moving image. Additional information added to the photographed image includes position information indicating the position where the photographed image was photographed, direction information (azimuth and elevation angle or depression angle) where the photographed image was photographed, and time information (year / month) regarding the photographing time of the photographed image. Date and time). The image recording unit 13 may record not only a captured image captured by the imaging unit 12 but also an image transferred via a server image acquisition unit 21 described later.

  The operation unit 14 includes buttons, keys, dials, and the like that are operated by the user. The operation unit 14 performs shooting instructions, image projection start / end instructions, mode selection (for example, planetarium mode selection described later), various mode settings, and the like. Is what you do.

  The position information acquisition unit 15 has a GPS (Global Positioning System: Global Positioning System) function, and acquires position information regarding the current position of the image projection apparatus 1. The position information includes, for example, information on longitude, latitude, and altitude.

  The azimuth information acquisition unit 16 includes an electronic compass (geomagnetic sensor) that detects the north-south geomagnetism in which the earth occurs and obtains the direction (azimuth), and the current azimuth (direction around the vertical axis) of the image projector 1. ) Direction information. The azimuth information is represented by, for example, a clockwise angle with respect to true north.

  The distance information acquisition unit 17 includes a distance sensor such as an infrared distance sensor or a laser distance sensor, and acquires distance information related to the distance between the image projection apparatus 1 and the projection surface (screen, ceiling, wall surface, etc.) 2. . When the imaging unit 12 includes a distance measuring sensor for AF (autofocus), for example, the imaging unit 12 can also be used as the distance measuring sensor of the imaging unit 12.

  The inclination information acquisition unit 18 includes an inclination sensor that detects rotation about three axes (two axes in the horizontal plane and a vertical axis), and acquires inclination information (elevation angle or depression angle) with respect to the horizontal plane of the image projector 1. In the present embodiment, the inclination information acquisition unit 18 can also acquire an angle around the vertical axis.

  The movement information acquisition unit 19 includes an acceleration sensor, and acquires the movement amount of the image projection apparatus 1 in the horizontal direction. The time information acquisition unit 20 includes a timer and acquires the current date and time (year / month / day / hour / minute / second).

  The server image acquisition unit 21 includes a communication unit that receives an image recorded and held by a server (not shown). Examples of the communication unit include a wireless LAN module and a Bluetooth (Bluetooth) module. it can. As the communication unit, in addition to or in addition to such a wireless communication module, a module that performs wired connection using a wired LAN module or a USB interface may be provided.

  For example, the image projection unit 22 includes a light source that emits illumination light, a liquid crystal element (reflection type or transmission type) that forms an image, and a projection optical system (lens system) 24. In the present embodiment, the projection optical system 24 has a function of optically performing zoom correction and AF (autofocus) correction. The image projection unit 22 including the projection optical system 24 is provided integrally with the image projection apparatus 1, and changing the projection direction (the direction of the optical axis of the projection optical system 24) changes the inclination of the image projection apparatus 1 itself. This is done by changing the (posture). However, the inclination (posture) of only the image projection unit 22 may be changed by supporting the image projection unit 22 so as to be rotatable about two axes with respect to the main body of the image projection apparatus 1.

  The control unit 11 includes an image selection unit 31, an image correction unit 32, and the like as functional configurations realized by the CPU executing a predetermined program. The image selection unit 31 is acquired by the position information acquired by the position information acquisition unit 15, the direction information acquired by the direction information acquisition unit 16, the inclination information acquired by the inclination information acquisition unit 18, and the movement information acquisition unit 19. The image recorded in the image recording unit 13 or the image corrected by the image correcting unit 32 described later based on one or two or more combinations of the movement information and the time information acquired by the time information acquiring unit 20 An image to be projected is selected from the list.

  The image correction unit 32 corrects the image selected by the image selection unit 31 based on the direction (posture) of the image projection apparatus 1 (image projection unit 22), and corrects the shape of the image (in the present embodiment). The image processing unit 33 that performs trapezoidal correction) and the drive unit 34 that sends a control signal to the projection optical system 24 in order to correct the projection conditions (zoom and focus) of the image. In the present embodiment, the image correcting unit 32 is configured to project the image projecting apparatus 1 (image projecting unit) when projecting a plurality of images within a predetermined range with respect to the image initially selected by the image selecting unit 31. 22) is corrected in advance based on the direction predicted to be suitable. The image correction unit 32 corrects the image selected by the image selection unit 31 based on one or both of the direction information acquired by the direction information acquisition unit 16 and the inclination information acquired by the inclination information acquisition unit 18. You may make it do.

  Next, the main operation (planetarium mode) of the image projection apparatus 1 will be described with reference to the flowchart shown in FIG. In the planetarium mode, for example, when an image is projected toward a ceiling (substantially horizontal flat surface) as the projection surface 2 in a camping tent at night, the projection direction (projection optical system 24) of the image projection device 1 is used. A starry sky photograph that will be visible in the direction of the optical axis) is projected, and when the image projection apparatus 1 is tilted (changed in posture), a starry sky photograph that appears in the tilted direction is projected.

  As a premise for executing this planetarium mode, a starry sky photographed image obtained by changing the direction (azimuth and elevation angle) from the same position, for example, at approximately the same date and time of the previous year, using the photographing unit 12 of the image projection device 1 Are recorded in the image recording unit 13, and the captured images are projected and displayed by the image projection apparatus 1. Each image recorded in the image recording unit 13 includes position information acquired by the position information acquisition unit 15, direction information acquired by the direction information acquisition unit 16, and inclination information acquired by the inclination information acquisition unit 18 (here Now, it is assumed that the clock information acquired by the time information acquisition unit 20 is added as additional information. Note that the projected image may be a plurality of captured images captured by the capturing unit 12 and stored in the image recording unit 13, or may be a seamless image (including a panoramic image). In addition, the image recorded in the image recording unit 13 is not limited to the image captured by the imaging unit 12 but may be an image captured by another digital camera or the like and recorded in the image recording unit 13. Furthermore, the image recorded in the image recording unit 13 may be an image that is transferred from the server as an external device via the server image acquisition unit 21 and recorded in the image recording unit 13.

  First, the user of the image projection apparatus 1 projects and displays the image projection apparatus 1 in the direction in which the image projection apparatus 1 is desired to be projected (here, the vertical upward direction, that is, the direction substantially orthogonal to the ceiling surface as the projection plane 2). Is started, the projection button of the operation unit 14 is turned on. The control unit 11 determines whether or not the projection button of the operation unit 14 is turned on (step S1). If the projection button is turned on (in the case of Yes), the position information and direction information acquisition unit by the position information acquisition unit 15 is determined. The direction information by 16, the inclination information by the inclination information acquisition unit 18, and the time information by the time information acquisition unit 20 are acquired (step S <b> 2).

  Next, the control unit 11 (image selection unit 31) searches for additional information of each image recorded in the image recording unit 13 based on these pieces of information (position information, orientation information, inclination information, and time information). Then, for example, the starry sky photographed image photographed in the same direction (azimuth and elevation angle) from the same position at the same date and time of the previous year (month date and time minute second) is searched and selected (step S3). Note that when there is no perfect match in date and time, position, and direction (azimuth and elevation or depression) at the time of image selection in step S3, a closer one is selected.

  Next, the control unit 11 acquires distance information between the image projection device 1 and the ceiling surface as the projection surface 2 from the distance information acquisition unit 17 (step S4), and automatically focuses the image to be projected based on the acquired distance information. (Step S5). Note that AF (autofocus) of the projected image may be adjusted so as to realize an optimum contrast by detecting a change in contrast of information from the photographing unit 12. In this case, the photographing unit 12 is preferably driven only during focus adjustment from the viewpoint of energy saving.

  Next, the control unit 11 projects the image selected in step S3 after adjusting the focus in step S5 (step S6). Thereafter, peripheral images that are a plurality of captured images (for example, captured images having relatively close directions and times) within a predetermined range with respect to the image selected in step S3 (that is, the image projected in step S6). For each searched image, trapezoidal correction is performed in advance in consideration of the inclination of the image projecting apparatus 1 when projecting the image, and the corrected images (corrected images) are separated. It is saved in a file (step S7).

  Here, the trapezoidal correction is an image on the projection surface 2 in order to prevent the projection direction (the direction of the optical axis of the projection optical system 24) from being orthogonal to the projection surface 2 so as to suppress display in a trapezoidal shape. Is a process of correcting the shape of the image so as to form a rectangle (rectangle). Information indicating that the image has been corrected is added to the corrected image as additional information, and in principle, the same additional information as before correction (position information, orientation information, inclination information, time information, etc.) is added. Shall be. In addition, as the predetermined range, those registered in advance as the number of sheets or the angle range can be used. Here, the work area (buffer memory) of the volatile memory provided in the control unit 11 is used as the recording location of the corrected image. However, the image recording unit 13 may be used.

  Next, the control unit 11 determines whether or not the user has tilted the image projection apparatus 1 in order to project another captured image, based on the tilt information acquired by the tilt information acquisition unit 18 (step S8). If it is determined that the vehicle is tilted (Yes), the tilt information is acquired (step S9). Next, the control unit 11 (image selection unit 31) searches the additional information (tilt information) of the corrected image recorded in step S7 based on the tilt information of the image projection device 1 acquired in step S9, and tilts the tilt. A corrected image having the closest angle related to the information is selected (step S10). Since the image selected here is already trapezoidally corrected, it can be projected more smoothly than the case of selecting and projecting after correcting the keystone. However, step S7 is omitted, and additional information of the captured image recorded in the image recording unit 13 is searched based on the tilt information acquired in step S9, and the captured image with the closest angle related to the tilt information is selected. Then, the image selected here may be trapezoidally corrected based on the tilt information acquired in step S9.

  Next, the control unit 11 (image correction unit 32) automatically corrects the zoom (magnification) and focus of the image to be projected from the tilt information of the image projection device 1 acquired in step S9 (step S11), and step S10. The image selected in (1) is projected (step S12). Thereafter, the process proceeds to step S15.

  If it is determined in step S8 that the image projection apparatus 1 is not tilted (in the case of No), the process proceeds to step S13, and the movement information acquisition unit 19 determines whether or not the image projection apparatus 1 has been translated. Judgment is made based on the acquired movement information, and if it is determined that the movement has been made in parallel (in the case of Yes), the movement amount information is acquired (step S14), and the process returns to step S3. In this case, in step S3, the captured image is re-searched and selected based on the position information obtained by correcting the acquired position information with the movement amount acquired in step S14. Since the captured image re-searched here has no distortion related to the trapezoid, it is selected from the original image (image recorded in the image recording unit 13) that has not been trapezoidally corrected in step S7, and the image is selected in step S6. Projected. If it is determined in step S13 that the translation is not performed (No), the process proceeds to step S15.

  In step S15, the control unit 11 determines whether or not the projection button of the operation unit 14 is turned off (OFF). When the projection button is not turned off (in the case of No), the control unit 11 returns to step S8 and performs steps S8 to S8. When the process of S13 is repeated and the projection button is turned off (in the case of Yes), each of the corrected images corrected in the trapezoid in step S7 is deleted (step S16), and the projection process related to the planetarium mode is ended.

  FIG. 3 is a diagram for explaining an example of image correction (trapezoid correction). Here, a projection possible area of 5 m is considered in each of a predetermined X direction and a Y direction orthogonal to the ceiling surface as the projection plane 2. Assume that the image P1 is selected as the first image to be projected by turning on the projection button of the operation unit 14, and at this time, the image projection device 1 is directed vertically upward (that is, the optical axis and the projection of the projection optical system 24). (Substantially orthogonal to the surface 2). In this case, if an image selected from the images recorded in the image recording unit 13 is projected as it is without correcting the shape, a rectangular projection image is displayed as shown as an image P1 in FIG.

  From such a vertically upward state, for example, the image projection device 1 (image projection unit 22) is tilted by + 15 ° around the X axis and −15 ° around the Y axis, and an image shown as an image P2 in FIG. If it is selected, as shown in the figure, the display is distorted into a trapezoidal shape by the tilted angle. From the distance between the image projection apparatus 1 and the projection plane 2 when the image P1 is projected, the angle of inclination, and information of the projection optical system 24, an image having an appropriate rectangular shape when the image projection apparatus 1 is inclined as described above. Since the correction amount for projecting and displaying P3 can be calculated geometrically, the corrected image is created and stored in advance.

  Accordingly, when the image projection apparatus 1 is tilted to change the image to be projected and displayed, the second and subsequent images can be projected and displayed as an appropriate rectangular image in the same manner as the first image. Quality can be realized. In this embodiment, when an image to be displayed first is selected, trapezoidal correction is performed in advance on a plurality of peripheral images (images within a predetermined range), and a corrected image is generated and stored. (See step S7 in FIG. 2). As a result, when the user actually tilts the image projection apparatus 1, a correction image corresponding to this is prepared in advance, so that the display speed of the image is increased by the time required for the correction calculation. And a smooth image display can be realized.

  Further, when the image projection apparatus 1 is tilted to change the image to be projected and displayed, the optimum value is shifted (shifted) from the zoom and focus at the time of projecting the first image by the tilt. In consideration of these, zoom correction and focus correction are performed (see step S11 in FIG. 2). Therefore, when the image projection apparatus 1 is tilted to change the image to be projected and displayed, the second and subsequent images can be projected and displayed with substantially the same image quality as the first image, and this also provides high display quality. Can be realized.

  When the image recorded in the image recording unit 13 is a seamless image (including a panoramic image), a plurality of projections related to adjacent portions around the first projection range in the seamless image. The range may be corrected in a trapezoid according to the position. When the image projection apparatus 1 is tilted, the projection range corrected accordingly is displayed.

  In the embodiment described above, a digital camera with a projector is exemplified as the image projecting device 1, but the present invention is also applicable to a projector that does not have a photographing function and other electronic devices with a projector (such as a mobile phone, a PDA, and a tablet computer). be able to.

  Further, in the above-described embodiment, the planetaryium mode is set such that the image of the starry sky that is likely to be seen in the direction with the image projection device 1 facing the ceiling surface is switched according to the orientation of the image projection device 1. The image of the starry sky that would appear vertically upward when the image projection device 1 is oriented substantially horizontally, with the wall surface (substantially vertical surface) as the projection surface, is projected upward, and vertically upward according to the angle tilted from the horizontal. An image of a starry sky that would be visible when tilted by this angle from the above may be projected.

  The embodiment described above is described for facilitating the understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment includes all design changes and equivalents belonging to the technical scope of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Image projection apparatus, 2 ... Projection surface, 11 ... Control part, 12 ... Imaging | photography part, 13 ... Image recording part, 14 ... Operation part, 15 ... Position information acquisition part, 16 ... Direction information acquisition part, 17 ... Distance information Acquisition unit, 18 ... inclination information acquisition unit, 19 ... movement information acquisition unit, 20 ... time information acquisition unit, 21 ... server image acquisition unit, 22 ... image projection unit, 23 ... photographing lens, 24 ... projection optical system, 31 ... Image selection unit, 32... Image correction unit, 33... Image processing unit, 34.

Claims (11)

An image recording unit for recording a plurality of images;
An image projection unit for projecting an image;
A direction acquisition unit for acquiring a direction of the image projection unit;
An image selection unit that selects an image to be projected from the images recorded in the image recording unit based on the direction acquired by the direction acquisition unit;
An image correction unit that corrects the image selected by the image selection unit based on the direction of the image projection unit ;
The image correction unit is configured to preliminarily determine a plurality of images within a predetermined range with respect to the image selected by the image selection unit based on a direction in which the image projection unit is expected to face when projecting the image. Correct it,
The image selection device, wherein the image selection unit selects an image to be projected from images corrected by the image correction unit based on the direction acquired by the direction acquisition unit . The image projection apparatus according to claim 1 , wherein the image correction unit corrects the shape of the image. The image correction section, an image projection apparatus according to claim 1 or claim 2, characterized in that to correct the projection condition of the image. A position acquisition unit for acquiring the position of the image projection unit;
The said image selection part selects the image which should be projected based on the direction acquired by the said direction acquisition part, and the position acquired by the said position acquisition part, The any one of Claims 1-3 characterized by the above-mentioned. The image projection device according to item. It further includes a time acquisition unit that acquires the current time,
The image selection unit selects an image to be projected based on the direction acquired by the direction acquisition unit, the position acquired by the position acquisition unit, and the time acquired by the time acquisition unit. The image projection apparatus according to claim 4 . The image projection apparatus according to any one of claims 1 to 5, further comprising a communication unit for recording the transferred image on the image recording unit by wireless communication from an external device. A photographing unit for photographing an image of a subject and recording the image on the image recording unit;
Digital camera, comprising an image projection apparatus according to any one of claims 1-6. 8. The digital camera according to claim 7 , wherein focus adjustment at the time of image projection by the image projection unit is performed based on a change in contrast based on an output of the photographing unit. A shooting section for shooting the subject;
A projection unit that projects the selected selected image;
A detection unit for detecting a projection direction by the projection unit;
A correction unit that corrects, using the direction detected by the detection unit, the image captured at an angle within a predetermined range from the direction detected by the detection unit among the images captured by the imaging unit;
An optical apparatus comprising: a selection unit that selects the corrected image as the selection image. The optical apparatus according to claim 9,
The correction unit performs trapezoidal correction on the image captured at an angle within the predetermined range from the direction detected by the detection unit, and is captured at an angle within the predetermined range from the direction detected by the detection unit. Do not perform keystone correction on the image that is not
The optical unit is characterized in that, out of the images corrected by the correction unit, the image captured at an angle closest to the direction detected by the detection unit is selected as the selection image. The optical apparatus according to claim 10,
An optical apparatus comprising: a focus adjustment unit that performs focus adjustment of the selected image projected onto the projection unit using an output of the photographing unit.

Images