JP2014003715A - Imaging apparatus, imaging method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a general user to easily photograph a subject in omni-directional three-dimensions without using a special device.SOLUTION: A control section (CPU) 20 acquires a pitch, a yaw and a roll of a sensor, which are measured by a triaxial magnetic sensor 34, on a real time basis by a shutter button which is in a half-depressed state during photographing of omni-directional three-dimensions. An optical axis coordinate of a digital camera, a rotation angle of the digital camera and a spherical coordinate of a photographed image are calculated from the pitch, the yaw and the roll, and a memory address in which the image photographed in directions of the optical axis coordinate, the rotation angle and the spherical coordinate is to be recorded is calculated in accordance with the optical axis coordinate, the rotation angle and the spherical coordinate. An image memory 31 is accessed in accordance with the memory address, and a photographed region is displayed on a display section 25 with a through image so that the photographed region can be identified in a monochromatic (translucent) state when a finished photograph's image data (referred to below as a finished photograph's image) has already been recorded.

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program.

  Conventionally, in a digital camera, images are taken in order from left (right) to right (left), and the captured images are combined to perform 360 ° panoramic shooting in the horizontal direction, as well as from the bottom (top). A technique is known in which images are taken in order upward (down), and the images taken are combined to perform continuous vertical shooting (see, for example, Patent Documents 1 and 2). In either case, each time a photograph is taken, a part of the previous photographed image is displayed semi-transparently on the display unit so that the image is not interrupted. The user is guided to determine the composition (photographing position, photographing direction) so as to match the transparent photographed image.

  In recent years, a technique has been known in which a desired image of a two-dimensional map can be specified on the Internet so that a photographed image of 360 ° in the horizontal direction at that position can be viewed (for example, see Non-Patent Document 1). ). In this technology, four digital cameras facing at least four directions (front / rear / left / right) are installed on the roof of an automobile, and the above four digital cameras are photographed at a predetermined interval while traveling on the road. Are combined to obtain a captured image of 360 ° in the horizontal direction.

JP 2007-174301 A JP 2005-184533 A

“Gallgle View, starting in major cities in Japan”, [Search February 9, 2009], Internet <URL: http://www.google.co.jp/intl/ja/press/pressrel/20080805. html>

  By the way, with the widespread use of digital cameras in recent years, it has become possible for anyone to enjoy various special shooting techniques easily (or automatically), but there are still techniques for shooting in all three-dimensional directions. Not proposed. However, a technique for photographing the zenith direction using a special lens (fisheye lens) is also known for a silver salt camera, but in this case, there is a problem that a photographed image is severely distorted.

  In addition, in the digital camera that performs 360 ° panoramic shooting in the horizontal direction or continuous shooting in the vertical direction according to the conventional technique, continuous images are shot only in a predetermined direction, that is, 360 ° in the horizontal direction and / or the vertical direction. There is a problem that can not be.

  In addition, as described above, a technique for capturing a horizontal 360 ° captured image at a time has been put into practical use, but it can also be captured only in the lateral 360 °, and is a special device to the last. There is a problem that it is not common and cannot be easily used by normal users.

  Thus, conventionally, there is no known technique that allows a general user to easily shoot in all three-dimensional directions without using a special device. However, it is clear that if there are captured images in all three-dimensional directions, it is possible to broaden the enjoyment of photography. When it becomes all three-dimensional orientation, it becomes difficult for the user to memorize and photograph which part has been photographed two-dimensionally. However, there is a problem that it is impossible to take an image efficiently without failure because there are unphotographed gaps or excessive overlapping portions.

  SUMMARY An advantage of some aspects of the invention is that it provides an imaging apparatus, an imaging method, and a program that allow a general user to easily perform imaging in all three-dimensional directions without using a special apparatus.

  In order to achieve the above object, the present invention provides an imaging means, a display means, a through image display means for sequentially displaying captured images sequentially captured by the imaging means as a through image on the display means, and photographing by the imaging means. Within a predetermined area that can be taken, the detection means for detecting the vertical and horizontal positions and the angle of the rotation direction of the partial area that is the current imaging target, and the partial image captured by the imaging means are captured. Based on the storage status of the storage means, the storage control means for additionally storing in the storage means in association with the vertical and horizontal positions and the rotational direction angle in the predetermined area detected by the detection means. Identification display means for identifying and displaying the photographed area and the non-photographed area in the predetermined area on the display means, the identification display means Position and angle corresponding to the positional relationship and angular relationship between the position and angle of the through image displayed by the through image display means in the predetermined region and the position and angle of the plurality of captured images in the predetermined region Then, identification display indicating the positions and angles of a plurality of captured images is performed on the through image, and identification is performed on the through image as the position and angle of the through image in the predetermined area sequentially change. The position and angle of a plurality of captured images to be displayed are sequentially changed.

  Further, as a preferred aspect, the apparatus further includes a control unit that performs identification display on the through image by the identification display unit when a predetermined operation is performed during display of the through image by the through image display unit. It is characterized by that.

  Further, as a preferred aspect, the control means further performs identification display on the through image by the identification display means when a half-pressing operation of the shutter is performed during display of the through image by the through image display means. In addition, a partial image captured by the imaging unit when the shutter is fully pressed is additionally stored in the storage unit in association with the position or angle detected by the detection unit.

  Further, as a preferred aspect, the control means further completes photographing of a plurality of partial images including all of the predetermined area by additionally storing a new partial image captured in response to the shutter full-press operation in the storage means. If it is determined that the shooting has been completed, the completion of shooting is notified.

  As a preferred aspect, the identification display means further changes the display state of the photographed area while maintaining the display state of the non-photographed area on the through image displayed by the through image display means. In this case, the identification display is performed.

  Further, as a preferred aspect, the detecting means further detects the vertical and horizontal positions and the rotational direction angle of the partial region currently being imaged by a triaxial sensor. .

  Further, as a preferred aspect, the detection unit further detects a three-dimensional direction by a three-axis sensor, and the storage control unit selects a partial image captured by the imaging unit as a position or an angle within the total three-dimensional direction. It is characterized in that it is additionally stored in the storage means in association with.

  Moreover, as a preferable aspect, the image processing device further includes a conversion unit that converts a shooting direction by the imaging unit detected by the detection unit into a storage area address of the storage unit, and the storage unit is converted into the conversion unit. In the storage area indicated by the storage area address, images taken in all directions taken by the imaging means are stored.

  In order to achieve the above object, an invention according to another aspect is an imaging method by an imaging apparatus comprising imaging means, display means for displaying an image taken by the imaging means, and storage means for storing the taken image. A through image display step of sequentially displaying captured images sequentially captured by the imaging unit as a through image on the display unit, and a portion that is currently captured within a predetermined area that can be captured by the imaging unit A detection step for detecting the vertical direction of the region, the position in the horizontal direction and the angle in the rotation direction, and the partial image captured by the imaging means, the vertical direction within the predetermined region detected by the detection step at the time of shooting, A storage control step of additionally storing in the storage means in association with the position in the left-right direction and the angle in the rotation direction, and the storage status of the storage means And an identification display step for identifying and displaying the photographed region and the non-photographed region in the predetermined region on the display means, and the identification display step is displayed by the through image display step. A plurality of images are taken on the through image at positions and angles corresponding to the positional relationship and the angle relationship between the position and angle of the through image in the predetermined region and the position and angle of the plurality of captured images in the predetermined region. The positions of a plurality of photographed images that are identified and displayed on the through image as the position and angle of the through image in the predetermined area are sequentially changed. And the angle is changed sequentially.

  In order to achieve the above object, another aspect of the invention provides an imaging apparatus comprising: imaging means; display means for displaying an image captured by the imaging means; and storage means for storing the captured image. A through image display means for sequentially displaying captured images picked up sequentially by the means as a through image on the display means, and a vertical direction of a partial area that is currently imaged within a predetermined area that can be photographed by the imaging means Detection means for detecting the position in the left-right direction and the angle in the rotation direction, and the partial image captured by the imaging means, the position in the up-down direction and the left-right direction in the predetermined area detected by the detection means at the time of shooting And a storage control means for additionally storing in the storage means in association with the angle of the rotation direction, and the predetermined area based on the storage status of the storage means A program for functioning as an identification display means for discriminating a photographed area and a non-photographed area and displaying them on the display means, wherein the identification display means is displayed by the through image display means A plurality of images are taken on the through image at positions and angles corresponding to the positional relationship and the angle relationship between the position and angle of the through image in the predetermined region and the position and angle of the plurality of captured images in the predetermined region. The positions of a plurality of photographed images that are identified and displayed on the through image as the position and angle of the through image in the predetermined area are sequentially changed. And the angle is changed sequentially.

  According to the present invention, there is an advantage that a general user can easily and freely photograph a wider range without using a special device.

It is a block diagram which shows the structure of the digital camera by embodiment of this invention. It is a conceptual diagram for demonstrating the optical axis coordinate of the camera and the spherical coordinate of a picked-up image in the all-solid-direction imaging | photography with the digital camera by this embodiment. It is a schematic diagram for demonstrating a mode when the digital camera by this embodiment is made into the shutter half-pressed state toward a certain direction. It is a schematic diagram which shows the example of a through image display when the digital camera by this embodiment is made into the shutter half-pressed state toward a certain direction. In the digital camera by this embodiment, it is a schematic diagram which shows the example of a display for notifying a user of the imaging condition of all the three-dimensional directions. It is a flowchart for demonstrating the operation | movement at the time of the all-solid-directional imaging | photography of the digital camera by this embodiment. In the digital camera by this embodiment, it is a schematic diagram which shows the example of a display in the case of guiding appropriately the overlap condition with the already image | photographed image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

A. Configuration of Embodiment FIG. 1 is a block diagram showing a configuration of a digital camera according to an embodiment of the present invention. In the figure, the image acquisition unit 10 includes a lens 11, a shutter 12, and an LPF 13. The lens 11 is a normal optical lens and includes a lens group in which aspherical lenses are stacked. The shutter 12 is a so-called mechanical shutter that is operated by a driver 14 driven by the control unit 20 when a shutter button is operated. Depending on the digital camera, a mechanical shutter may not be provided, and in the case of a model equipped with a retractable lens structure and mechanical zoom, these drivers are also controlled by the driver 14. The LPF 13 is a crystal low-pass filter and is mounted to prevent the occurrence of moire.

  Next, the analog signal processing unit 15 includes an image sensor (CCD, CMOS) 16, a sampling / signal amplification processing unit 17, and an A / D converter 18. The imaging sensor 16 forms a subject image (image), and converts the intensity of light of each color of RGB into a current value. The sampling / signal amplification processing unit 17 performs correlated double sampling processing and signal amplification processing for suppressing noise and color unevenness. The A / D converter 18 is also called an analog front end, and converts the sampled / amplified analog signal into a digital signal (converts each color of RGB and CMYG into 12-bit data and outputs it to the bus line).

  Next, the image memory 31 is a memory for recording captured image data in association with position information in all stereoscopic orientations, recording and managing image data of all captured portions in all stereoscopic orientations, It plays a role of managing a photographed area and a non-photographed area in all three-dimensional directions. Several methods are conceivable as specific methods for managing image data and photographed areas in consideration of all three-dimensional orientations, but one method is as follows: In this method, the information is recorded individually as a normal image file, and information on shooting positions (shooting areas) in all three-dimensional directions is recorded in association with each image file.

  The other method is a method in which one large bitmap memory capable of mutual address conversion with each position in all three-dimensional directions is provided, and image data obtained by photographing is overwritten and synthesized on this bitmap memory. Yes, unphotographed areas are filled with black (or white) in the initial state, and areas that are filled with black (or white) are detected by image processing etc. It is possible to determine the area.

  Any of the above methods may be used, but the latter method is used here. Moreover, since these methods are well-known techniques, detailed description is abbreviate | omitted.

  Next, the control unit (CPU) 20 executes a program stored in a program memory 30 described later to control the entire digital camera (imaging device). In particular, in the present embodiment, the control unit (CPU) 20 performs a sensor pitch (azimuth angle θ) measured by a triaxial magnetic sensor 34, which will be described later, in real time when the shutter is half-pressed during all-stereoscopic shooting. The yaw (elevation angle φ) and roll (rotation angle ψ) are acquired, and the pitch (azimuth angle θ), yaw (elevation angle φ), and roll (rotation angle ψ) from the optical axis coordinates of the digital camera, the rotation angle of the digital camera, And the spherical coordinates of the photographed image are calculated, a memory address at which an image photographed in that direction is to be recorded is calculated according to the optical axis coordinate, the rotation angle, and the spherical coordinate, and the image memory 31 is stored according to the memory address. When accessing and already recorded image data (hereinafter referred to as an already imaged image) is recorded, the area corresponding to the already imaged portion on the through image is monochrome (translucent) In controls to identification. Further, at this time, instead of identifying and displaying on the through image in monochrome (semi-transparent), a part of the already photographed image is combined with the through image in monochrome (semi-transparent) and displayed on the display unit 25. You may control so. If the image has not yet been photographed in that direction, the image is naturally displayed in monochrome (semi-transparent) and the already photographed image is not displayed.

  Therefore, if the user shakes the digital camera in an arbitrary direction while the shutter is half-pressed, if the image has been taken even once, the already taken area is identified and displayed in monochrome (semi-transparent) in real time ( (Alternatively, the already captured image is combined with the through image in monochrome (translucent) in real time and displayed on the display unit 25). The user can determine the shooting frame so as to be continuous with the already shot image while confirming the positional relationship between the through image and the already shot image in real time. That is, in order to capture an image in all three-dimensional directions without a gap, the user may determine a shooting frame so that the already-photographed area is identified and displayed on the display unit 25 in monochrome (translucent).

  When the shutter is fully pressed, the control unit (CPU) 20 is based on the pitch (azimuth angle θ), yaw (elevation angle φ), and roll (rotation angle ψ) measured by the triaxial magnetic sensor 34 at that time. The captured image data is stored in the image memory 31 in accordance with the calculated or converted memory address. Therefore, when there is a pre-photographed image portion, the pre-photographed image portion, that is, the overlapping portion is overwritten, and the non-photographed portion is newly recorded.

  In the present embodiment, the order of shooting is not determined as in the prior art, and it is possible to shoot at an arbitrary position in an arbitrary (random) order. In this embodiment, the conversion from the optical axis coordinates of the digital camera and the spherical coordinates of the captured image to the memory addresses of the image memory 31 includes the optical axis coordinates, the rotation angle, and the spherical coordinates and the memory address of the captured image. An associated conversion table may be prepared and converted via the conversion table, or may be calculated according to a predetermined arithmetic expression.

  The preview engine 22 is stored in the image buffer 26 immediately after detection of digital data input via the image acquisition unit 10 and the analog signal processing unit 15 in the recording mode (also referred to as recording mode or photographing mode) or shutter operation detection. Thinning processing is performed in order to display the digital data and the digital data stored in the image memory 31 on the display unit 25.

  The D / A converter 23 converts the digital data thinned out by the preview engine 22 and outputs it to the driver 24 at the subsequent stage. The driver 24 includes a buffer area for temporarily storing digital data displayed on the display unit 25 at the subsequent stage, and drives the display unit 25 based on a control signal input via the key operation unit 27 and the control unit 20. The display unit 25 includes a color TFT liquid crystal, an STN liquid crystal, or the like, and displays a preview image, image data after shooting, a setting menu, and the like.

  The image buffer 26 temporarily stores digital data immediately after photographing until it is input via the analog signal processing unit 15 or the digital signal processing unit 28 and passed to the digital signal processing unit 28. The key operation unit 27 includes a shutter button 7 and various operation keys such as a recording / reproducing mode switching key, a cross key, and a menu key. The digital signal processing unit 28 performs white balance processing, color processing, gradation processing, and outline emphasis on the digital data (uncompressed RAW image data) input via the analog signal processing unit 15. The image compression / decompression processing unit 29 compresses and encodes digital data (uncompressed RAW image data) input via the digital signal processing unit 28 into the JPEG format, or decompresses a JPEG format file in the playback mode. To do.

  The program memory 30 stores various programs loaded on the control unit 20. The image memory 31 stores digital data converted into various file formats. The card I / F 32 controls data exchange between the external recording medium 33 and the imaging apparatus main body. The external recording medium 33 includes a compact flash (registered trademark), a memory stick, an SD card, and the like.

  The triaxial magnetic sensor 34 is a sensor that measures the magnetic level of each of the three axes, and uniquely defines the sensor pitch (azimuth angle θ), yaw (elevation angle φ), and roll (rotation angle ψ) (or posture). Three angles). By acquiring the pitch (azimuth angle θ), yaw (elevation angle φ), and roll (rotation angle ψ) by the sensor, the optical axis direction of the camera when the shutter is half-pressed or when the shutter is fully pressed (shooting) It is possible to calculate (the azimuth and elevation angles of the east, west, north, and south of the optical axis) and the rotation angle of the camera (information indicating how much the optical axis is tilted about the rotation axis). Note that details of the optical axis coordinates of the camera and the spherical coordinates of the captured image in all stereoscopic shooting will be described later.

  The microphone 36 inputs sound when shooting a moving image. The speaker 37 outputs sound when playing back a moving image. The audio processing unit 38 performs encoding (compression) / decoding (decompression) or the like using a predetermined codec, thereby converting the audio data output to the speaker 37 into an analog signal and digitizing the audio signal from the microphone 36. .

FIG. 2 is a conceptual diagram for explaining the optical axis coordinates of the camera and the spherical coordinates of the photographed image in omnidirectional photographing with the digital camera according to the present embodiment. For example, when the digital camera is turned halfway toward a certain direction, as shown in FIG. 2, the pitch (azimuth angle θ), yaw (elevation angle φ), roll (rotation angle) of the three-axis magnetic sensor 34 at that time is shown. ψ) is obtained. Then, from the pitch (azimuth angle θ), yaw (elevation angle φ), and roll (rotation angle ψ), when the shutter is half-pressed or when the shutter is fully pressed (shooting), the optical axis direction (1, θ, φ), camera rotation angle (ψ), spherical coordinates (1, θ ₀ , φ ₀ ), (1, θ ₁ , φ ₁ ), (1, θ ₂ , φ ₂ ), (1 , Θ ₃ , φ ₃ ) are calculated.

When the shutter is half-pressed or fully pressed, the control unit (CPU) 20 determines which part of the spherical surface the shooting range is in the optical axis direction (1, θ, φ) of the camera and the rotation of the camera. Angle (ψ) and spherical coordinates (1, θ ₀ , φ ₀ ), (1, θ ₁ , φ ₁ ), (1, θ ₂ , φ ₂ ), (1, θ ₃ , φ ₃ ) of the captured image And converted to a corresponding memory address or calculated.

  FIG. 3 is a schematic diagram for explaining a state in which the digital camera according to the present embodiment is in a half-pressed state toward a certain direction. In a certain landscape, hatched portions A1 and A2 are portions that have already been shot, and the rectangular frame L1 corresponds to a frame (view angle) that is shot in the direction in which the digital camera is currently directed.

  FIG. 4 is a schematic diagram illustrating a display example of a through image when the digital camera according to the present embodiment is in a half-pressed state toward a certain direction. In the case shown in FIG. 3, the digital camera display unit 25 has a monochrome (semi-transparent) through image as shown in FIG. 4 where the already photographed part (corresponding to the hatched parts A1 and A2) A is monochrome. It will be identified and displayed above. By viewing the through image displayed on the display unit 25, the user can easily identify a portion that has not yet been photographed, and can easily photograph images in all three-dimensional directions without gaps.

  FIG. 5 is a schematic diagram showing a display example for informing the user of the shooting situation of all stereoscopic directions in the digital camera according to the present embodiment. As described above, at the time of shooting, the already shot portion is displayed in monochrome (translucent) on the display unit 25. In addition to this, as shown in FIG. 5, the photographed portion (shaded portion) and the unphotographed portion (outlined portion) are distinguished by color coding, and the virtual spherical surface and the current digital camera orientation (▲) The relationship may be displayed.

A-2. Operation of Embodiment Next, the operation of the above-described embodiment will be described.
FIG. 6 is a flowchart for explaining the operation of the digital camera according to this embodiment at the time of photographing in all stereoscopic directions. When the omnidirectional shooting mode is selected, first, a through image is displayed on the display unit 25 (step S10), and it is determined whether the shutter is half-pressed (step S12). If the shutter is not half-pressed, the process returns to step S10, and the through image display is continued as in the normal shooting.

On the other hand, when the shutter is half-pressed, the pitch (azimuth angle θ), yaw (elevation angle φ), and roll (rotation angle ψ) are obtained from the triaxial magnetic sensor 34 (step S14). θ), yaw (elevation angle φ), roll (rotation angle ψ) to optical axis direction (1, θ, φ), camera rotation angle (ψ), and spherical coordinates (1, θ ₀ , φ _{0 of the} captured image) ), (1, θ ₁ , φ ₁ ), (1, θ ₂ , φ ₂ ), (1, θ ₃ , φ ₃ ) are calculated (step S 16). Next, the optical axis direction (1, θ, φ), the rotation angle (ψ) of the camera, and the spherical coordinates (1, θ ₀ , φ ₀ ), (1, θ ₁ , φ ₁ ), ( 1, θ ₂ , φ ₂ ), (1, θ ₃ , φ ₃ ) are converted into memory addresses of the image memory 31 (step S 18), and the shooting data within the shooting angle of view is converted from the image memory 31 according to the memory addresses. Read (step S20).

  Next, it is determined whether or not there is a photographed portion in the read photographing data (step S22). If there is a photographed image portion, the photographed image portion is converted into a through image in monochrome (translucent). It displays so that it can be identified (step S24). Here, whether or not there is a captured part is determined by whether or not there is a data area that remains in the initial black (or white) in the read captured data. Further, if management data indicating the photographed area and the non-photographed area is separately stored, the determination may be made by comparing the coordinate data of the areas. Next, it is determined whether or not the shutter is fully pressed (step S26). If the shutter is not fully pressed, the process returns to step S10 and the above-described processing is repeated.

  On the other hand, if there is no already captured image portion, it is determined whether or not the shutter is fully pressed (step S26). If the shutter is not fully pressed, the process returns to step S10 and the above-described processing is repeated.

  Therefore, if the user shakes the digital camera in any direction with the shutter half-pressed, the captured image portion can be identified on the through image in monochrome (semi-transparent) in real time if it has been captured even once. Will be displayed. The user can determine the shooting frame so as to be continuous with the already shot image or to exclude the already shot portion while confirming the positional relationship between the through image and the already shot image in real time. . That is, in order to shoot images in all three-dimensional directions with no gaps and without overlapping as much as possible, the user sets a shooting frame so that the already-taken image portion is not displayed in monochrome (semi-transparent) on the display unit 25. You just have to decide.

  On the other hand, when the shutter is fully pressed, the photographed image data is recorded in the image memory 31 according to the memory address (step S28). In other words, if there is a pre-photographed image portion, the pre-photographed image portion, that is, the overlapping portion is overwritten, and the non-photographed portion is newly recorded. Next, it is determined whether or not all areas on the virtual spherical surface have been imaged (step S30). If all areas have not been imaged, the process returns to step 10 to repeat the above-described processing. On the other hand, when all the regions on the virtual spherical surface have been photographed, the end of photographing in all three-dimensional directions is notified on a monitor or the like (step S32), and the processing is finished. Here, whether or not all the areas on the virtual spherical surface have been photographed is determined based on whether or not there is a data area that remains black (or white) in the initial state in the entire image memory 31.

  According to the above-described embodiment, a general user can easily take an image in all three-dimensional directions without using a special device. Further, in this embodiment, the shooting order is not determined as in the prior art, and any position can be shot in any (random) order, and images in all three-dimensional directions can be easily shot without gaps. can do.

  Note that, in the above-described embodiment, photography in all three-dimensional directions should ideally be performed in all directions by moving the camera around the image sensor so as to rotate. However, since the form of the digital camera is not a dedicated one but a normal one, it is conceivable that some parallel movement components are included every time a different direction is photographed. If these parallel movements are intense, depending on the balance with the distance to the subject, the images may not be connected properly during panorama synthesis.In the present invention, however, the parallel movements do not cause a major breakdown. Ingredients may also be included.

  In this case, in order to display the photographed area for the user, when only the area is identified and displayed on the display unit 25 with reduced brightness, the four corners on the spherical coordinates obtained each time a picture is taken are displayed. If the line connecting the points is used as the boundary line with the captured area as it is, if there is a parallel movement component and there are few ways to overlap the shooting of the user, there is a possibility that an unrecorded area will occur. is there. In order to avoid this as much as possible, when the captured area is identified and displayed on the display unit 25, a quadrilateral connecting points slightly inside the four corner stake points is displayed as the captured area. That's fine.

  Alternatively, as shown in FIGS. 7A and 7B, a guide 25-1 is displayed on the display unit 25. As shown in FIG. 7A, the already photographed portions A1 and A2 that are identified and displayed are displayed. If it is located outside the guide 25-1, there is a high possibility that a left-over area will occur, so a warning is given to the user (for example, the guide 25-1 blinks in red to disable full-pressing of the shutter). ), When the already photographed portion A3 to be identified and displayed is located inside the guide 25-1, as shown in FIG. 5B, a non-warning (for example, the guide 25-1 is displayed in blue or green). It is also possible to make full pressing of the shutter effective).

  Further, in the above-described embodiment, shooting in all three-dimensional directions is continued until shooting of all areas on the virtual sphere is completed. However, user operation is performed before all areas on the virtual sphere are shot. Thus, it may be possible to interrupt the photographing in all three-dimensional directions.

10 Image Acquisition Unit 11 Lens 12 Shutter 13 LPF
14 Driver 15 Analog Signal Processing Unit 16 Imaging Sensor 17 Sampling / Signal Amplification Processing Unit 18 A / D Converter 20 Control Unit (CPU)
22 Preview Engine 23 D / A Converter 24 Driver 25 Display Unit 26 Image Buffer 27 Key Operation Unit 28 Digital Signal Processing Unit 29 Image Compression / Expansion Processing Unit 30 Program Memory 31 Image Memory 32 Card I / F
33 External recording medium 34 Triaxial magnetic sensor 38 Audio processor 36 Microphone 37 Speaker

To achieve the above object, the present invention provides an image pickup means, a display means, a detection means for detecting an azimuth angle and an elevation angle, which are optical axis directions of the image pickup means, and an arbitrary spherical coordinate indicated by the azimuth angle and the elevation angle. A storage means for storing a photographed image in association with the position of the image, and an image photographed by the imaging means corresponding to a position on the spherical coordinate indicated by an azimuth angle and an elevation angle detected by the detection means at the time of photographing. And a storage control means for additionally storing in the storage means, and a plurality of photographed images stored in the storage means on a plane corresponding to the position on the spherical coordinate to which each photographed image is associated. Display control means for displaying on the display means in a state of being synthesized at the position .

Further, as a preferred aspect, the storage means is further capable of storing a captured image in association with an arbitrary position on an all three-dimensional azimuth by spherical coordinates indicated by an azimuth angle and an elevation angle. .
Moreover, as a preferable aspect, the display control means further includes a plurality of photographing images associated with the vicinity of a position on the arbitrarily designated spherical coordinates among the plurality of photographed images stored in the storage means. The image is displayed on the display means in a synthesized state.

Further, as a preferred aspect, the detection means further detects an azimuth angle and an elevation angle that are the optical axis direction of the imaging means, and a rotation angle around the optical axis of the imaging apparatus, and the storage means A captured image is stored in association with an arbitrary position and rotation angle on a spherical coordinate indicated by an azimuth angle, an elevation angle, and a rotation angle, and the storage control unit stores an image captured by the imaging unit at the time of this shooting. In association with the position and the rotation angle on the spherical coordinates indicated by the azimuth angle, the elevation angle, and the rotation angle detected by the detection means, the storage means is additionally stored, and the display control means is the storage means Are displayed on the display means in a state where they are combined at a position and a rotation angle on a plane corresponding to the position and rotation angle on the spherical coordinates to which each captured image is associated. Specially To.

Further, as a preferred aspect, the storage means stores each captured image individually as an image file, and stores positional information on the spherical coordinates in association with each image file .

Further, as a preferred aspect, the storage means is a memory for storing a plurality of captured images in a synthesized state, and each position on the spherical coordinate and an address on the memory are mutually converted. It is characterized by being configured by a possible predetermined memory .

Moreover, as a preferable aspect, the image processing apparatus further includes an identification display unit that identifies a captured region and a non-photographed region on the spherical coordinates based on a storage state of the storage unit and displays the region on the display unit. Features.

Further, as a preferable aspect, the image display device further includes a through image display unit that sequentially displays captured images sequentially captured by the imaging unit as a through image on the display unit, and the identification display unit displays the through image display unit. The position on the spherical coordinate indicated by the azimuth and elevation detected by the detection means when the through image being taken is captured, and the sphere associated with a plurality of captured images stored in the storage means An identification display indicating the positions of a plurality of captured images is performed on the through image at a position corresponding to the positional relationship with the coordinate position, and the azimuth and elevation angles, which are the optical axis direction of the imaging means, change. The captured images displayed as through images are sequentially changed according to the position, and the positions and angles of the plurality of captured images identified and displayed on the through images are sequentially changed. And wherein the changing.

Further, as a preferred aspect , when the shutter half-press operation is performed during the display of the through image by the through image display unit, the identification display unit performs the identification display on the through image, and the shutter is fully pressed. When an operation is performed, a captured image captured by the imaging unit is added to the storage unit and stored in association with the position on the spherical coordinate indicated by the position angle and the elevation angle detected by the detection unit. Control means is further provided .

Further, as a preferred aspect, the control means further includes a plurality of predetermined photographing areas that are preliminarily determined by additionally storing new photographing images picked up in response to a shutter full-press operation in the storage means. When it is determined that shooting of a shot image has been completed, the completion of shooting is notified .

As a preferred aspect, the identification display means further changes the display state of the photographed area while maintaining the display state of the non-photographed area on the through image displayed by the through image display means. In this case, the identification display is performed .
To achieve the above object, the invention according to another aspect of the present invention provides an imaging device comprising: an imaging unit; a display unit; and a storage unit that stores a captured image in association with an arbitrary position on spherical coordinates indicated by an azimuth angle and an elevation angle. An image pickup method by the apparatus, wherein a detection process for detecting an azimuth angle and an elevation angle, which are optical axis directions of the image pickup means, and an image detected by the detection process at the time of shooting an image shot by the image pickup means Each of the captured images corresponds to a storage control process that is added and stored in the storage unit in association with the position on the spherical coordinates indicated by an angle and an elevation angle, and a plurality of captured images stored in the storage unit And a display control process for displaying on the display means in a state of being combined at a position on a plane corresponding to the position on the spherical coordinates.
To achieve the above object, the invention according to another aspect provides an imaging device and a computer of an imaging apparatus including a display unit, a detection unit that detects an azimuth angle and an elevation angle that are optical axis directions of the imaging unit, and an azimuth angle. Storage means for storing a photographed image in association with an arbitrary position on the spherical coordinates indicated by the elevation angle, and an image photographed by the imaging means at an azimuth angle and an elevation angle detected by the detection means at the time of photographing. Storage control means for adding and storing in the storage means in association with the indicated position on the spherical coordinates, and a plurality of photographed images stored in the storage means, each of the photographed images being associated with each other It is characterized by functioning as display control means for displaying on the display means in a state of being synthesized at a position on a plane corresponding to the position on the spherical coordinates.

Claims (10)

Imaging means;
Display means;
Through image display means for sequentially displaying captured images taken sequentially by the imaging means on the display means as through images;
Detection means for detecting the vertical direction, the horizontal position, and the rotation direction angle of the partial area that is currently imaged within the predetermined area that can be imaged by the imaging means;
The partial image captured by the imaging unit is additionally stored in the storage unit in association with the vertical and horizontal positions and the rotation direction angle within the predetermined area detected by the detection unit at the time of shooting. Memory control means;
Based on the storage status of the storage means, identification display means for identifying and displaying on the display means a photographed area and a non-photographed area in the predetermined area;
With
The identification display means is a positional relationship and an angular relationship between a position and angle of the through image displayed by the through image display means in the predetermined region and a position and angle of the plurality of captured images in the predetermined region. Is displayed at the position and angle corresponding to the through image, the identification display showing the position and angle of a plurality of captured images on the through image, according to the position and angle of the through image in the predetermined region sequentially changes, An imaging apparatus characterized by sequentially changing the positions and angles of a plurality of captured images that are identified and displayed on the through image.   2. The display device according to claim 1, further comprising control means for performing identification display on the through image by the identification display means when a predetermined operation is performed during display of the through image by the through image display means. The imaging device described in 1.   The control means causes the identification display means to perform identification display on the through image when the shutter half-press operation is performed while the through image is displayed by the through image display means, and the shutter full press operation is performed. 3. The imaging apparatus according to claim 2, further comprising: storing a partial image captured by the imaging unit in a storage unit in association with a position or an angle detected by the detection unit.   The control means, when it is determined that the shooting of a plurality of partial images including all of the predetermined area is completed by additionally storing a new partial image captured in response to the shutter full-press operation in the storage means, The imaging apparatus according to claim 3, wherein the imaging completion is notified.   The identification display means performs the identification display by changing the display state of the photographed area while maintaining the display state of the non-photographed area on the through image displayed by the through image display means. The imaging apparatus according to claim 1, wherein:   6. The detection unit according to claim 1, wherein the detection unit detects a vertical position, a horizontal position, and an angle in a rotation direction of a partial area that is currently captured by a three-axis sensor. The imaging device described in 1. The detection means detects all three-dimensional directions by a three-axis sensor,
The imaging apparatus according to claim 6, wherein the storage control unit additionally stores a partial image captured by the imaging unit in a storage unit in association with a position or an angle in all three-dimensional directions. . Further comprising conversion means for converting the shooting direction by the imaging means detected by the detection means into a storage area address of the storage means,
The storage means
Storing captured images in all directions captured by the imaging unit in the storage area indicated by the storage area address converted by the conversion unit;
The imaging apparatus according to claim 1, wherein the imaging apparatus is an imaging apparatus. An imaging method using an imaging device comprising: an imaging unit; a display unit that displays an image captured by the imaging unit; and a storage unit that stores the captured image.
A through image display step of sequentially displaying captured images sequentially captured by the imaging unit on the display unit as a through image;
In a predetermined area that can be imaged by the imaging means, a detection step of detecting the vertical direction, the horizontal position, and the rotational angle of the partial area that is the current imaging target;
The partial image picked up by the image pickup means is additionally stored in the storage means in association with the vertical and horizontal positions and the rotation direction angle within the predetermined area detected by the detection step at the time of shooting. A memory control step;
An identification display step for identifying and displaying on the display means the imaged area and the non-photographed area in the predetermined area based on the storage status of the storage means,
The identification display step includes a positional relationship and an angular relationship between a position and an angle of the through image displayed in the through image display step within the predetermined region and a position and an angle of a plurality of captured images within the predetermined region. Is displayed at the position and angle corresponding to the through image, the identification display showing the position and angle of a plurality of captured images on the through image, according to the position and angle of the through image in the predetermined region sequentially changes, An imaging method characterized by sequentially changing the positions and angles of a plurality of captured images that are identified and displayed on the through image. An imaging apparatus comprising: an imaging unit; a display unit that displays an image captured by the imaging unit; and a storage unit that stores the captured image.
Through image display means for sequentially displaying captured images taken sequentially by the imaging means on the display means as through images;
Detection means for detecting the vertical direction, the horizontal position, and the rotation direction angle of the partial area that is currently imaged within the predetermined area that can be imaged by the imaging means;
The partial image captured by the imaging unit is additionally stored in the storage unit in association with the vertical and horizontal positions and the rotation direction angle within the predetermined area detected by the detection unit at the time of shooting. Memory control means;
Based on the storage status of the storage means, identification display means for identifying and displaying on the display means a photographed area and a non-photographed area in the predetermined area;
To function,
The identification display means is a positional relationship and an angular relationship between a position and angle of the through image displayed by the through image display means in the predetermined region and a position and angle of the plurality of captured images in the predetermined region. Is displayed at the position and angle corresponding to the through image, the identification display showing the position and angle of a plurality of captured images on the through image, according to the position and angle of the through image in the predetermined region sequentially changes, A program characterized by sequentially changing the positions and angles of a plurality of photographed images identified and displayed on the through image.

Images