JP2005026859A - Imaging apparatus, photographed picture processing method used therefor and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus, capable of easily grasping from which turn a picture having azimuthal property, e.g., a panorama picture is photographed by recording or reproducing the picture while associating it with the turn angle of a camera. <P>SOLUTION: The apparatus comprises an azimuth sensor 18 for detecting the turn angle of a digital camera 1. When a plurality of pictures are taken by the camera 1 with varying its turn angle to make a panorama picture, the apparatus adds azimuthal information detected by the azimuth sensor 18 to the pictures and records them on a memory card 38. In reproduction of the panorama picture, it reads the pictures corresponding to the turn angles of the digital camera 1 from the memory card 38 and shows them on a display 17. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging apparatus such as a digital camera, and more particularly to an imaging apparatus suitable for use in creating a panoramic image composed of a plurality of images, and a captured image processing method and program used for the imaging apparatus.
[0002]
[Prior art]
For example, some digital cameras have a function of creating a single panoramic image by capturing a plurality of images while changing the orientation (angle) of the camera with respect to a subject, and automatically joining these images. is there. In this case, a plurality of images photographed as panoramic images are stored in a predetermined memory in the apparatus as a series of photographing data constituting the panoramic image. In general, the images are displayed one by one (for example, see Patent Document 1).
[0003]
FIG. 15 is a diagram for explaining an operation method during playback of a panoramic image in a conventional digital camera.
[0004]
In a memory (not shown) in the camera 100, a panoramic image made up of a plurality of images taken by pressing the shutter key 101 while the user changes the orientation of the camera 100 in advance is stored. On the back surface of the camera 100, an operation unit 102, a display unit 103, and the like are installed. By operating the right / left scroll keys of the operation unit 102 during playback, each image constituting the panoramic image is sequentially switched. And displayed on the display unit 103.
[0005]
[Patent Document 1]
JP-A-7-99633
[0006]
[Problems to be solved by the invention]
A panoramic image is different from a normal image taken from one direction. For example, a north image, a northeast image, an east image, etc. It consists of a single image.
[0007]
However, in a conventional digital camera, it is not known at all from which direction each image constituting the panoramic image was taken, and each image is displayed in order by the operation of the scroll key as described above during reproduction. It looks like an image that is long in one direction, and the orientation characteristics of the panoramic image cannot be used at all.
[0008]
The present invention has been made in view of the above points, and by recording and reproducing an image in association with the direction of the camera, for example, from which direction an image having azimuth characteristics such as a panoramic image was taken. It is an object of the present invention to provide an imaging apparatus that can easily grasp the above, a captured image processing method used in the imaging apparatus, and a program.
[0009]
[Means for Solving the Problems]
An electronic apparatus according to a first aspect of the present invention is an imaging apparatus that captures an image of an object, an image capturing unit that obtains a plurality of images while changing the orientation of the apparatus main body with respect to the object, and the apparatus during image capturing by the image capturing unit Orientation detection means for detecting the direction in which the main body is directed, and orientation information indicating the orientation of the apparatus main body detected by the orientation detection means is added to each of the images obtained by the photographing means, and a predetermined memory And a recording processing means for recording.
[0010]
According to such a configuration, when an image is taken by changing the orientation of the device body (that is, the orientation of the camera) in an imaging device such as a digital camera, the orientation of the device body at that time is displayed on these images. Is added and recorded in the memory, so that it is possible to know in which direction the image is taken with respect to the main body of the apparatus.
[0011]
An electronic apparatus according to a second aspect of the present invention is an imaging apparatus that captures a subject, and the apparatus body when a plurality of images captured while changing the orientation of the apparatus body with respect to the subject captures these images. A memory recorded with azimuth information indicating the orientation of the image, azimuth detecting means for detecting a direction in which the apparatus main body is directed at the time of reproduction of each image, and an azimuth indicating the orientation of the apparatus main body detected by the azimuth detecting means Based on the information, reproduction processing means for sequentially reading out the image having the orientation information from the memory in conjunction with the movement of the apparatus body, and display means for displaying the image read from the memory by the reproduction processing means It is comprised and comprises.
[0012]
According to such a configuration, in an imaging apparatus including a digital camera or the like, when a plurality of images captured in advance while changing the orientation of the apparatus body (that is, the orientation of the camera) are recorded in the memory, these images are stored. During playback, images corresponding to the direction of the main unit are displayed, and each image is displayed continuously according to the movement of the main unit, so you can feel as if you are actually looking around. Can see.
[0013]
An electronic apparatus according to a third aspect of the present invention is an imaging apparatus that captures an image of a subject, an imaging unit that obtains a plurality of images while changing the orientation of the apparatus main body with respect to the subject, and the apparatus at the time of imaging by the imaging unit First orientation detecting means for detecting the direction in which the main body is directed, and orientation information indicating the orientation of the apparatus main body detected by the first orientation detecting means is provided for each image obtained by the photographing means. In addition, recording processing means for recording in a predetermined memory, second azimuth detecting means for detecting the direction in which the apparatus main body is directed at the time of reproduction of each image, and the second azimuth detecting means detected by the second azimuth detecting means Based on azimuth information indicating the orientation of the apparatus main body, reproduction processing means for sequentially reading out images having the azimuth information from the memory in conjunction with the movement of the apparatus main body, and the reproduction processing means Constructed comprises a display means for displaying an image read from the memory by.
[0014]
According to such a configuration, in an imaging apparatus including a digital camera or the like, when each image is taken while changing the orientation of the device body (that is, the orientation of the camera), the orientation information indicating the orientation of the device body is included in each image. The image is added and recorded in the memory, and when these images are reproduced, the image corresponding to the direction in which the main body of the apparatus is directed is read from the memory and displayed. Therefore, it is possible to know in which direction the main body of the apparatus is taken, and a panoramic image can be seen as if the user is actually looking around.
[0015]
An electronic device according to a fourth aspect of the present invention is the electronic device according to the first or third aspect, wherein the photographing means displays a part of the already photographed image in a semi-transparent state so as to overlap a current photographing object. It is characterized by doing.
[0016]
According to such a configuration, a part of the already photographed image is displayed in a semi-transparent state so as to overlap the current photographing target, so that each image constituting the panoramic image is photographed while changing the orientation of the apparatus main body. In this case, these images can be taken so as to be connected in a direction.
[0017]
The electronic device according to claim 5 of the present invention is the electronic device according to claim 2 or 3, wherein the reproduction processing means displays one of the images recorded in the memory as a reference image. The image corresponding to the angle at which the apparatus main body is directed from the reference image is read from the memory and displayed.
[0018]
According to such a configuration, it is possible to switch the display of the panoramic image according to the angle directed to the apparatus main body, starting from an image taken from a certain direction.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0020]
FIG. 1 shows an external configuration when a digital camera is used as the imaging apparatus of the present invention. FIG. 1 (A) is a perspective view mainly showing a front side configuration, and FIG. 1 (B) is mainly a rear side configuration. is there.
[0021]
The digital camera 1 includes a photographing lens 2, a self-timer lamp 3, an optical viewfinder window 4, a microphone unit 5, a strobe light emitting unit 6, and a rubber grip 7 on the front surface of a substantially rectangular thin plate body. A power key 8 and a shutter key 9 are arranged on the right end side (for the user).
[0022]
The rubber grip 7 is made of rubber and is disposed so that the middle finger, the ring finger, and the little finger of the right hand can securely hold the housing when the user grips the digital camera 1 with the right hand from the right side of the housing at the time of shooting. It is a band-like projection.
[0023]
The power key 8 is a key that is operated every time the power is turned on / off, and the shutter key 9 instructs the photographing timing in the photographing mode.
[0024]
On the back of the digital camera 1, a mode switch (SW) 10, a speaker unit 11, a menu key 12, a cross key 13, a set key 14, an optical finder 15, a strobe charge lamp 16, and a display unit 17 are arranged.
[0025]
The mode switch 10 is constituted by a slide key switch, for example, and switches between a recording mode “R” and a reproduction mode “P” which are basic modes.
[0026]
The menu key 12 is operated when selecting various menu items. In this embodiment, the “panorama mode” is set by operating the menu key 12. The “panoramic mode” is a mode in which a plurality of images are captured while changing the camera direction to the horizontal direction, and these images are automatically connected to create one panoramic image.
[0027]
The cross key 13 is integrally formed with keys for moving the cursor in the up, down, left, and right directions, and is operated when moving a displayed menu item or the like.
[0028]
The set key 14 is disposed at the center position of the cross key 13 and is operated to set the contents of the menu item selected at that time.
[0029]
The strobe charge lamp 16 is an LED lamp disposed in the vicinity of the optical viewfinder 15, which is either when the user of the digital camera 1 is looking into the optical viewfinder 15 or looking at the display unit 17. However, the user can visually recognize the charge state of the strobe.
[0030]
The display unit 17 is composed of a color liquid crystal panel with a backlight, and displays a through image on the monitor as an electronic viewfinder in the recording mode, and reproduces and displays the selected image and the like in the reproduction mode.
[0031]
Although not shown, the digital camera 1 has a memory card slot for attaching / detaching a memory card used as a recording medium, a serial interface connector for connecting to an external personal computer, etc., for example, USB (Universal). Serial Bus) connector and the like are provided.
[0032]
The digital camera 1 has a direction sensor 18 incorporated therein. The user holds the device with his hand and looks into the optical viewfinder 15 so that the digital camera 1 is directed, that is, the user looks at the subject. The viewing direction is detected.
[0033]
FIG. 2 is a block diagram showing the configuration of the electronic circuit of the digital camera 1.
[0034]
In the recording mode, which is the basic mode, an imaging element disposed behind the photographic optical axis of the lens optical system 22 constituting the photographic lens 2 whose focus position and aperture position are moved by driving the motor (M) 21. The CCD 23 is scanned and driven by a timing generator (TG) 24 and a vertical driver 25, and outputs a photoelectric conversion output corresponding to a light image formed at fixed intervals for one screen.
[0035]
The photoelectric conversion output is appropriately gain-adjusted for each primary color component of RGB in the state of an analog value signal, sampled and held by the sample hold circuit 26, converted into digital data by the A / D converter 27, and color The process circuit 28 performs color process processing including pixel interpolation processing and γ correction processing to generate digital luminance signals Y and color difference signals Cb and Cr, which are output to a DMA (Direct Memory Access) controller 29.
[0036]
The DMA controller 29 once uses the luminance signal Y and the color difference signals Cb and Cr output from the color process circuit 28 by using the composite synchronization signal, memory write enable signal, and clock signal from the color process circuit 28 once. And the DMA transfer to the DRAM 31 used as the buffer memory via the DRAM interface (I / F) 30.
[0037]
The control unit 32 includes a CPU, a ROM storing a program executed by the CPU, a RAM used as a work memory, and the like. The controller 32 controls the entire digital camera 1. After the DMA transfer of the luminance and chrominance signals to the DRAM 31 is completed, the luminance and chrominance signals are read from the DRAM 31 via the DRAM interface 30, and the VRAM controller 33. Is written into the VRAM 34 via
[0038]
The digital video encoder 35 periodically reads the luminance and color difference signals from the VRAM 34 via the VRAM controller 33, generates a video signal based on these data, and outputs the video signal to the display unit 17.
[0039]
As described above, the display unit 17 functions as a monitor display unit (electronic finder) in the recording mode. By performing display based on the video signal from the digital video encoder 35, the display unit 17 receives from the VRAM controller 33 at that time. An image based on the stored image information is displayed in real time.
[0040]
When the shutter key 9 constituting the key input unit 36 is operated at a timing when still image shooting is desired in a state where the image at that time is displayed as a monitor image in real time on the display unit 17 as described above, a trigger signal is generated. Is generated. In response to this trigger signal, the control unit 32 immediately stops the path from the CCD 23 to the DRAM 31 immediately after the DMA transfer of the luminance and color difference signals for one screen captured from the CCD 23 to the DRAM 31 is completed. Transition to record save state.
[0041]
In this record storage state, the control unit 32 outputs the luminance and color difference signals for one frame written in the DRAM 31 through the DRAM interface 30 for each of Y, Cb, and Cr components of 8 pixels × 8 pixels horizontally. And is written in a JPEG (Joint Photograph Coding Experts Group) circuit 37, which is an ADCT (Adaptive Discrete Cosine Transform), an entropy coding system. Data compression is performed by processing such as conversion.
[0042]
The obtained code data is read out from the JPEG circuit 37 as a data file of one image, and is fixedly incorporated in the memory card 38 that is detachably mounted as a recording medium of the digital camera 1 or in the digital camera 1. To either one of the built-in memories 39.
[0043]
Then, along with the compression processing of the luminance and color difference signals for one frame and the completion of writing of all the compressed data to the memory card 38 or the built-in memory 39, the control unit 32 activates the path from the CCD 23 to the DRAM 31 again.
[0044]
The control unit 32 is further connected with an audio processing unit 40, a USB interface (I / F) 41, and a strobe driving unit 42, and an orientation sensor 18 for detecting a direction toward the digital camera 1. Connected.
[0045]
The audio processing unit 40 includes a sound source circuit such as a PCM sound source, digitizes an audio signal input from the microphone unit (MIC) 5 when recording audio, and performs a predetermined data file format such as MP3 (MPEG-1 audio layer). 3) Data compression is performed in accordance with the standard to create an audio data file and send it to the memory card 38 or the built-in memory 39. On the other hand, when playing back sound, the audio data file sent from the memory card 38 or the built-in memory 39 is compressed. It is solved and analogized, and the speaker unit (SP) 11 is driven to emit a loud sound.
[0046]
The USB interface 41 performs communication control when image data and other data are transmitted / received to / from other information terminal devices such as a personal computer connected via a USB connector.
[0047]
The strobe drive unit 42 charges a large-capacitance capacitor for strobe (not shown) at the time of still image shooting, and then drives the strobe light emitting unit 6 to flash based on the control from the control unit 32.
[0048]
The key input unit 36 includes the power key 8, the mode switch 10, the menu key 12, the cross key 13, the set key 14 and the like in addition to the shutter key 9 described above. The signal is sent directly to the control unit 32.
[0049]
In the playback mode, which is the basic mode, the control unit 32 selectively reads out the image data recorded in the memory card 38 or the built-in memory 39, and the procedure is completely opposite to the procedure in which the data is compressed in the recording mode by the JPEG circuit 37. The decompressed image data is decompressed, and the decompressed image data is held in the DRAM 31 via the DRAM interface 30, and then the content held in the DRAM 31 is stored in the VRAM 34 via the VRAM controller 33. Image data is read out periodically to generate a video signal, which is reproduced and output by the display unit 17.
[0050]
Next, a method for creating a panoramic image using the digital camera 1 will be described with reference to FIGS. 3 and 4.
[0051]
For example, when creating a panoramic image with a landscape or the like as a subject, a plurality of images A, B, C... Are taken while changing the orientation (angle) of the digital camera 1 with respect to the subject, as shown in FIG. In this case, the shootable range for the subject is determined by the relationship between the field angle range of the digital camera 1 and the focal position. In the example of FIG. 3, 360 is obtained by shooting the digital camera 1 toward “north”, “northeast”, “east”, “southeast”, “south”, “southwest”, “west”, “west east”. The case of creating a panoramic image of the degree is shown.
[0052]
Here, when the digital camera 1 is imaged in each direction, the direction toward the digital camera 1 is detected by the azimuth sensor 18. The direction in which the digital camera 1 is directed specifically refers to the direction in which the photographing lens 2 on the front surface of the apparatus body of the digital camera 1 is directed toward the subject.
[0053]
The direction sensor 18 is composed of, for example, a magnetic sensor, and has two sets of coils wound orthogonally inside the sensor and an MR element (magnetoresistance element) disposed at the center thereof. The MR element senses geomagnetism and has the property that the resistance value changes depending on the strength of the magnetism. Utilizing this, the resistance change of the MR element is divided into components in two directions by two sets of coils, and the direction is measured by calculating the voltage signal obtained thereby by a microprocessor. In the present embodiment, a digitized voltage signal is output from the azimuth sensor 18, and this is received by the control unit 32 and processed.
[0054]
As the azimuth sensor 18, there is a sensor that uses a GPS (global positioning system). This is called a GPS compass and is configured to measure an azimuth from the relative positional relationship between two antennas.
[0055]
When the camera direction at the time of shooting is detected by such an azimuth sensor 18, azimuth information indicating the camera direction is added to the shot image as shown in FIG. 4 and recorded in the memory card 38 or the built-in memory 39. That is, for example, if the user photographs the digital camera 1 toward the north, the orientation information indicating that it is facing north is added to the image A obtained at that time and recorded in the memory card 38 or the built-in memory 39. In this way, azimuth information is added to each of the plurality of images A, B, C,... Taken by changing the orientation of the digital camera 1 to create one continuous panoramic image.
[0056]
When the panoramic image is reproduced, an image corresponding to the direction in which the digital camera 1 is directed is read from the memory (memory card 38 or built-in memory 39) and displayed on the display unit 17. For example, when the digital camera 1 is directed to the north, the image A in the north direction among the images A, B, C... Constituting the panoramic image is displayed on the display unit 17.
[0057]
Hereinafter, the processing operation related to the panoramic image by the digital camera 1 will be described separately for (a) processing in the recording mode and (b) processing in the reproduction mode. The processes shown in the following flowcharts are executed by the control unit 32 mounted on the digital camera 1, and the control unit 32 reads the program stored in a ROM (not shown), The process is executed according to the procedure described in the program.
[0058]
(A) In recording mode
FIG. 5 is a flowchart showing the operation of the recording process of the digital camera 1, and shows a processing procedure when recording each image constituting the panoramic image.
[0059]
In order to create a panoramic image, first, the operation mode is switched to the recording mode “R” by operating the mode switch 10, and then “panoramic mode” is set by operating the menu key 12 (steps A 11 and A 12).
[0060]
In this state, when the photographing lens 2 of the digital camera 1 is pointed at the subject and the shutter key 9 is pressed (Yes in step A13), an image of the subject that is reflected on the photographing lens 2 within a predetermined field angle range is obtained as photographing data. (Step A14). During shooting, the display unit 17 functions as an electronic viewfinder, and the current shooting target is displayed on the screen.
[0061]
Here, when the user takes a picture by pressing the shutter key 9, the direction toward the digital camera 1 is detected by the direction sensor 18 (step A15), and the direction information indicating the camera direction is added to the taken image. Is recorded in a predetermined memory (step A16). The predetermined memory is one of the memory card 38 and the built-in memory 39 in the present embodiment, and will be described here as being recorded on the memory card 38 for convenience.
[0062]
Similarly, a plurality of images are taken while changing the orientation of the digital camera 1, and orientation information indicating the camera orientation at that time is added to each of these images and sequentially recorded in the memory card 38. At that time, when the second image is taken, the image taken immediately before is read from the memory card 38, and a part of the image is translucent on the screen of the display unit 17 so as to overlap the current subject. (Step A18). As a result, the next image to be photographed can be photographed so as to be connected to the previous image in a direction.
[0063]
Further, when the user explicitly instructs the end of recording by a predetermined operation such as turning off the power (Yes in step A17), the recording process here ends.
[0064]
By taking a plurality of images while changing the orientation of the digital camera 1 in this manner, these images are recorded in the memory card 38 as one continuous panoramic image.
[0065]
Specific examples are shown in FIGS.
[0066]
Now, description will be made assuming that a panoramic image made up of, for example, three captured images in the north direction, the northeast direction, and the east direction is created. FIG. 6 is an example of an image displayed on the display unit 17 during north-direction shooting, FIG. 7 is an example of an image displayed on the display unit 17 during north-east shooting, and FIG. 8 is displayed on the display unit 17 during east shooting. An example of an image is shown. FIG. 9 shows an example of a panoramic image created by connecting these captured images.
[0067]
First, as shown in FIG. 6, the shutter key 9 is pressed with the digital camera 1 facing north as shown in FIG. 6 in order to take the first image as a panoramic image. Thereby, an image in the north direction is obtained within the field angle range of the digital camera 1. At this time, the direction toward the digital camera 1 is detected by the direction sensor 18 made of, for example, a magnetic sensor, and direction information indicating the direction (in this case, the north direction) is added to the captured image and stored in the memory card 38. Is done.
[0068]
Next, the digital camera 1 is directed in the northeast direction in order to capture a second image that is a constituent image of the panoramic image. At that time, as shown in FIG. 7, a part of the image taken immediately before is displayed in a semi-transparent state on the display unit 17 functioning as an electronic view funder in accordance with the direction in which the digital camera 1 is directed. . In FIG. 7, the image indicated by the dotted line in the figure is the immediately preceding captured image. Here, after the digital camera 1 was photographed in the north direction, the digital camera 1 was subsequently directed in the northeast direction. As a captured image, an image in the north direction is read from the memory card 38, and the right end portion of the image is displayed on the display unit 17 in a translucent state.
[0069]
For example, if the display unit 17 is pointed in the west-east direction instead of the north-east direction after taking a north-direction image, the left end of the north-direction image that is the immediately previous captured image is displayed in a translucent state. become.
[0070]
In addition, it is assumed that which of the captured images is displayed in a translucent state can be arbitrarily selected by the user by operating the menu key 12 or the like, and the display range can also be arbitrarily selected by the user.
[0071]
Then, as shown in FIG. 7, in a state in which a part of the north-direction image taken immediately before is displayed, the digital camera 1 is directed to the north-east direction, and the north-east image is displayed on the part of the north-direction image. After the photographing range is determined so that the two are matched, the shutter key 9 is pressed. As a result, an image in the northeast direction that is clearly connected to the image in the north direction that has already been taken can be obtained. At this time, the direction toward the digital camera 1 is detected by the direction sensor 18 at the time of shooting, and direction information indicating the direction (in this case, the northeast direction) is added to the captured image and stored in the memory card 38. The
[0072]
The same applies to the third image, and as shown in FIG. 8, the digital camera 1 is directed eastward, and the shooting range is determined so that the eastward image is aligned with a part of the northeastern image taken immediately before. Press the shutter key 9. As a result, an image in the east direction that is clearly connected to the image in the northeast direction that has already been taken can be obtained. At this time, the direction toward the digital camera 1 is detected by the direction sensor 18 at the time of shooting, and direction information indicating the direction (in this case, the east direction) is added to the captured image and stored in the memory card 38. The
[0073]
When three images constituting a panoramic image are thus photographed, these images are recorded on the memory card 38 as one continuous panoramic image as shown in FIG.
[0074]
Here, for the sake of simplicity, a case where one type of panoramic image is created will be described. However, when a plurality of types of panoramic images are created, identification information is configured for each panoramic image. Added to each image in common. For example, if three types of panoramic images are to be created, identification information such as “NO.1” for each image constituting the first panoramic image, and each image constituting the second panoramic image. On the other hand, identification information such as “NO. 2” is added to each image constituting the third panoramic image. Thereby, at the time of reproduction, for example, when the user selects a desired panoramic image on the menu screen, for example, each image of the panoramic image is read from the memory card 38 as a display target based on the identification information of the selected panoramic image. be able to.
[0075]
(B) In playback mode
Next, the operation in the playback mode will be described.
[0076]
FIG. 10 is a flowchart showing the operation of the reproduction process of the digital camera 1, and shows a processing procedure when reproducing a panoramic image. Note that the reproduction of the panoramic image here means that each image constituting the panoramic image is read from a predetermined memory and displayed on the screen of the display unit 17. Further, it is assumed that the predetermined memory is the memory card 38 as in the recording. In the memory card 38, data of each image constituting the panoramic image is recorded together with direction information.
[0077]
As shown in FIG. 10, when a panoramic image is reproduced, first, the operation mode is switched to the reproduction mode “P” by operating the mode switch 10, and then “panoramic mode” is set by operating the menu key 12. (Steps B11 and B12).
[0078]
Here, normally, a method of sequentially displaying each image constituting the panoramic image by operating a cursor key or the like is used. However, in this embodiment, the digital camera 1 is simply pointed in an arbitrary direction, and the panoramic image is displayed. The image corresponding to that direction can be displayed.
[0079]
That is, when the user sets the digital camera 1 in an arbitrary direction with the “panorama mode” set, the direction at that time is detected by the direction sensor 18 (step B13). Then, the memory card 38 is searched based on the azimuth information indicating the camera direction detected by the azimuth sensor 18 (step B14), and a part of the panoramic image is displayed on the display unit centering on the image corresponding to the camera direction. 17 (step B15 → B17).
[0080]
If the digital camera 1 is pointed in another direction (No in step B18), the direction of the digital camera 1 at that time is detected in the same manner as described above, and the panoramic image is centered on the image corresponding to the camera direction. Is displayed on the display unit 17.
[0081]
Specifically, for example, in the case of reproducing the panoramic image shown in FIG. 9, when the user points the digital camera 1 in the north direction, in detail, the user is provided on the back surface of the digital camera 1. When the front portion of the digital camera 1 (the surface on which the photographing lens 2 is installed) is directed northward as viewed from the display unit 17, an image in the north direction as shown in FIG. 6 is displayed on the display unit 17. It will be.
[0082]
Similarly, when the digital camera 1 is pointed in the northeast direction, an image in the northeast direction as shown in FIG. 7 is displayed on the display unit 17, and when the digital camera 1 is pointed in the east direction, the east as shown in FIG. An image of the direction is displayed on the display unit 17.
[0083]
Here, when the image corresponding to the direction to which the digital camera 1 is directed is not in the memory card 38 (No in Step B15), the image in the direction is generated from the captured images in the adjacent directions (Step B16). The image is displayed on the display unit 17 as a part of the panoramic image (step B17).
[0084]
In other words, in the above example, during playback of a panoramic image composed of three images: an image shot in the north direction, an image shot in the northeast direction, and an image shot in the east direction, When the digital camera 1 is directed north-northeast, as shown in FIG. 11 (a), an image photographed facing the north-east direction with the center point Pa of the image photographed facing the north direction on the panoramic image. The midpoint Pa + b of the line segment connecting the center point Pb of the two is obtained. Then, as shown in FIG. 11B, an image within a predetermined range L centered on the midpoint Pa + b in the panoramic image is read from the memory card 38 as a north-northeast image and displayed on the display unit 17. .
[0085]
This operation is performed continuously. That is, for example, if the orientation of the digital camera 1 is continuously changed from the north direction to the northeast direction, the panorama image displayed on the display unit 17 is also panoramic from the image taken in the north direction to the image in the northeast direction. The image display changes continuously.
[0086]
Further, when the user explicitly instructs the end of the reproduction by a predetermined operation such as turning off the power (Yes in Step B18), the reproduction process here ends.
[0087]
In this way, when a panoramic image is reproduced, if the user points the digital camera 1 in an arbitrary direction, an image corresponding to the direction is displayed as a part of the panoramic image. Therefore, unlike the conventional method in which panoramic images are sequentially displayed by scroll key operations, troublesome key operations are unnecessary, and panoramic images can be viewed continuously only by changing the orientation of the digital camera 1. it can. For example, if a 360-degree panoramic image is taken as shown in FIG. 3, the digital camera 1 is rotated 360 degrees in the horizontal direction in the same way during playback, so that each person can feel as if they are actually looking around. You can see the images in the direction continuously.
[0088]
In the above-described embodiment, when a panoramic image is reproduced, an image corresponding to the direction to which the digital camera 1 is directed is displayed. It is also possible to switch the display of the panoramic image.
[0089]
That is, in the above embodiment, when the digital camera 1 is directed in the north direction, the panoramic image is displayed centering on the north-facing image. After the image at the center of the panoramic image is displayed as the reference image, the display portion is changed based on the reference image according to the angle at which the digital camera 1 is directed from the position. For example, an image in the northeast direction (FIG. 7) is first displayed as a reference image, and when the user rotates the digital camera 1 45 degrees to the right in this state, an image in the east direction (FIG. 8) is displayed. If the image is rotated 45 degrees to the left, the north image (FIG. 6) is displayed.
[0090]
FIG. 12 shows the reproduction operation at this time. First, the operation mode is switched to the reproduction mode “P” by operating the mode switch 10, and then “panoramic mode” is set by operating the menu key 12 (steps C 11 and C 12).
[0091]
Here, in a state where the user points the digital camera 1 in an arbitrary direction, a predetermined image in the panoramic image designated in advance as a display target is displayed on the display unit 17 as a reference image (step C13). The predetermined image is, for example, an image corresponding to the center portion of the panoramic image as described above.
[0092]
In this state, when the direction sensor 18 detects that the user has changed the orientation of the digital camera 1 (Yes in Step C14), the user selects the digital camera 1 based on the direction before the change and the direction after the change. It is determined how many times it has moved in the direction (step C15). In this case, for example, if the direction before the change is northeast and the direction after the change is east, it can be understood that the digital camera 1 is moved 45 degrees clockwise. If the direction after the change is north, it can be seen that the digital camera 1 has been moved 45 degrees counterclockwise.
[0093]
Thus, when the direction in which the digital camera 1 is moved and the change angle thereof are obtained, a position shifted by the change angle in the change direction of the digital camera 1 from the center position of the reference image in the panorama image is newly set. Is determined as the center of the image to be displayed (step C16), and the corresponding portion of the image is read from the memory card 38 and displayed on the display unit 17 (step C17).
[0094]
Similarly, until an instruction to end reproduction (No in Step C18), the images constituting the panoramic image are continuously displayed on the display unit 17 in conjunction with the change in the orientation of the digital camera 1.
[0095]
As described above, even if the display of the panoramic image is changed in response to the change in the orientation of the digital camera 1, it is possible to continuously view each image constituting the panoramic image without requiring the troublesome scrolling operation as in the prior art. Can do.
[0096]
(Image playback system)
Next, an image reproduction system using the digital camera 1 will be described with reference to FIG. 13 and FIG.
[0097]
FIG. 13 is a block diagram showing the configuration of the image playback system. This system includes a server 51 connected to the digital camera 1, a database (DB) 52 of the server 51, an image playback device 53, and a plurality of Display devices 54a, 54b, 54c...
[0098]
The server 51 receives each image of the panoramic image taken by the digital camera 1 together with the azimuth information added thereto and stores it in the database 52. The connection form between the server 51 and the digital camera 1 may be wired or wireless. The same applies to the connection form between the server 51 and the image reproduction device 53, which may be wired or wireless. The cable includes a communication network such as the Internet. In addition to infrared rays, the wireless includes, for example, a short-range wireless communication system such as Bluetooth (R). The Bluetooth (R) is a wireless communication system conforming to a short-range wireless communication standard, and for example, realizes wireless communication of about 10 m using a radio wave of 2.45 GHz ISM (Industrial Scientific Medical) band. is there.
[0099]
The server 51 reads each image of the panoramic image taken by the digital camera 1 from the database 52 at a predetermined timing, such as when a display request is received from the outside, and sends these to the image reproduction device 53. As described above, azimuth information is added to each image, and the image reproduction device 53 transfers these images to the display devices 54a, 54b, 54c,. Separate and display at the same time.
[0100]
That is, for example, in the case of a panoramic image composed of three directions of a north direction image, a northeast direction image, and an east direction image, the image reproduction device 53 displays the north direction image of the panorama image as the display device 54a and the northeast direction image. Is sent to the display device 54b and the east image is sent to the display device 54c, and these images are displayed simultaneously.
[0101]
As described above, according to the present system, a panoramic image photographed by the digital camera 1 can be viewed at a location apart from the digital camera 1. In this case, as shown in FIG. 14, if the display devices 54a, 54b, 54c,.
[0102]
In the above-described embodiment, the digital camera has been described as an example of the electronic device. However, the present invention is not limited to this, and for example, the camera has a photographing function such as a mobile phone with a camera or a PDA (Personal Digital Assistant). Any electronic device can be applied.
[0103]
In addition, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention.
[0104]
Further, the embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiment, the effects described in “Problems to be solved by the invention” can be solved, and are described in the “Effects of the invention” column. If the effect is obtained, a configuration from which this configuration requirement is deleted can be extracted as an invention.
[0105]
In addition, the method described in the above-described embodiment is a program that can be executed by a computer, such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD-ROM, etc.), a semiconductor memory, etc. The program can be written on a medium and applied to various apparatuses, or the program itself can be transmitted through a transmission medium such as a network and applied to various apparatuses. A computer that implements this apparatus reads a program recorded on a recording medium or a program provided via a transmission medium, and performs the above-described processing by controlling operations by this program.
[0106]
In addition to CD-ROM, DVD-ROM, and the like as recording media, for example, next-generation optical discs and red lasers using blue lasers such as Blu-ray Disc (R) and AOD (Advanced Optical Disc) are used. It is possible to implement the present invention using various large-capacity recording media that will be developed in the future, such as HD-DVD 9 to be used and Blue Laser DVD using a blue-violet laser.
[0107]
【The invention's effect】
As described in detail above, according to the present invention, when an imaging apparatus including a digital camera or the like creates a panoramic image including a plurality of images while changing the direction of the apparatus body (that is, the direction of the camera), Since azimuth information indicating the camera orientation at the time of shooting is added to each panoramic image, it is possible to know from which direction each image was taken.
[0108]
Also, when these images are played back, the device body can be pointed in any direction to display the image corresponding to that direction, so that panoramic image display switching can be performed without the need for troublesome scrolling operations as in the prior art. Can be done easily. Furthermore, since each image of the panoramic image is continuously displayed in accordance with the movement of the camera of the apparatus main body, the panoramic image can be viewed as if actually looking around.
[0109]
As described above, according to the present invention, it is possible to perform recording / playback by making full use of the characteristics of an image having an azimuth characteristic such as a panoramic image, and it is possible to enjoy an image that is not available in conventional devices.
[Brief description of the drawings]
FIG. 1 is a perspective view illustrating an external configuration when a digital camera is used as an electronic apparatus according to an embodiment of the invention.
FIG. 2 is a block diagram showing a configuration of an electronic circuit of the digital camera.
FIG. 3 is a view for explaining a method of creating a 360-degree panoramic image using the digital camera.
FIG. 4 is a view for explaining a recording method of each image of a panoramic image taken by the digital camera.
FIG. 5 is a flowchart showing an operation of panorama image recording processing by the digital camera.
FIG. 6 is a view showing an example of an image displayed on a display unit when the digital camera is in a recording mode.
FIG. 7 is a diagram illustrating an example of an image displayed on a display unit when the digital camera is in a recording mode.
FIG. 8 is a diagram illustrating an example of an image displayed on a display unit when the digital camera is in a recording mode.
FIG. 9 is a view showing an example of a panoramic image created by stitching together a plurality of images taken by the digital camera.
FIG. 10 is a flowchart showing an operation of panoramic image reproduction processing by the digital camera.
FIG. 11 is a view for explaining a method of generating an image corresponding to a direction in which the digital camera is directed.
FIG. 12 is a flowchart showing another playback processing operation of the digital camera.
FIG. 13 is a block diagram showing a configuration of an image reproduction system using the digital camera.
FIG. 14 is a diagram showing an arrangement example of each display device in the image reproduction system.
FIG. 15 is a diagram for explaining an operation method when reproducing a panoramic image in a conventional digital camera.
[Explanation of symbols]
1 ... Digital camera
2 ... Lens
3 ... Self-timer lamp
4 ... Optical finder window
5. Microphone part (MIC)
6 ... Strobe light emitting part
7 ... Rubber grip
8 ... Power key
9 ... Shutter key
10 ... Mode switch (SW)
11 ... Speaker unit (SP)
12 ... Menu key
13 ... Cross key
14 ... Set key
15 ... Optical finder
16 ... Strobe charge lamp
17 ... Display section
18 ... Direction sensor

Claims (11)

An imaging device for photographing a subject,
Photographing means for obtaining a plurality of images while changing the orientation of the apparatus main body with respect to the subject;
Azimuth detecting means for detecting the direction in which the apparatus main body is facing during photographing by the photographing means;
And recording processing means for adding azimuth information indicating the orientation of the apparatus main body detected by the azimuth detecting means to each image obtained by the photographing means and recording it in a predetermined memory. An imaging device. An imaging device for photographing a subject,
A memory in which a plurality of images taken while changing the orientation of the apparatus main body with respect to the subject are recorded together with orientation information indicating the orientation of the apparatus main body when these images are taken;
Azimuth detecting means for detecting a direction in which the apparatus main body is directed at the time of reproduction of each image;
Based on azimuth information indicating the orientation of the apparatus main body detected by the azimuth detection means, reproduction processing means for sequentially reading out images having the azimuth information from the memory in conjunction with the movement of the apparatus main body,
An image pickup apparatus comprising: display means for displaying an image read from the memory by the reproduction processing means. An imaging device for photographing a subject,
Photographing means for obtaining a plurality of images while changing the orientation of the apparatus main body with respect to the subject;
First azimuth detecting means for detecting a direction in which the apparatus main body is facing during photographing by the photographing means;
Recording processing means for adding azimuth information indicating the orientation of the apparatus main body detected by the first azimuth detecting means to each image obtained by the photographing means and recording it in a predetermined memory;
Second azimuth detecting means for detecting a direction in which the apparatus main body is directed at the time of reproduction of each image;
Based on azimuth information indicating the orientation of the apparatus main body detected by the second azimuth detection means, reproduction processing means for sequentially reading out images having the azimuth information from the memory in conjunction with the movement of the apparatus main body,
An image pickup apparatus comprising: display means for displaying an image read from the memory by the reproduction processing means. 4. The imaging apparatus according to claim 1, wherein the photographing unit displays a part of an already photographed image in a semi-transparent state so as to overlap a current photographing target. The reproduction processing means displays one of the images recorded in the memory as a reference image, and then displays an image corresponding to the angle at which the apparatus main body is directed from the reference image as a starting point. 4. The image pickup apparatus according to claim 2, wherein the image pickup apparatus is read out and displayed. A captured image processing method used for an imaging apparatus that captures a subject,
Acquiring a plurality of images while changing the orientation of the apparatus main body with respect to the subject;
Detecting the direction in which the apparatus body is facing during shooting;
An image processing method comprising: adding the detected azimuth information indicating the orientation of the apparatus main body to each image obtained at the time of photographing and recording the information in a predetermined memory. Image processing used for an imaging apparatus having a memory in which a plurality of images taken while changing the orientation of the apparatus main body with respect to a subject are recorded together with orientation information indicating the orientation of the apparatus main body when these images are taken A method,
Detecting a direction in which the apparatus main body is directed at the time of reproduction of each image;
Based on the detected orientation information indicating the orientation of the apparatus body, sequentially reading images having the orientation information from the memory in conjunction with the movement of the apparatus body;
An image processing method comprising: displaying an image read from the memory. A captured image processing method used for an imaging apparatus that captures a subject,
Acquiring a plurality of images while changing the orientation of the apparatus main body with respect to the subject;
Detecting the direction in which the apparatus body is facing during shooting;
The step of adding the detected azimuth information indicating the orientation of the apparatus main body to each image obtained at the time of photographing and recording it in a predetermined memory, and detecting the direction to which the apparatus main body is directed at the time of reproducing each image And steps to
Based on the detected orientation information indicating the orientation of the apparatus body, sequentially reading images having the orientation information from the memory in conjunction with the movement of the apparatus body;
An image processing method comprising: displaying an image read from the memory. A program executed by a computer mounted on an imaging device for photographing a subject,
In the computer,
A function of acquiring a plurality of images while changing the orientation of the apparatus main body with respect to the subject;
A function of detecting the direction in which the device body is facing during shooting;
A program for realizing a function of adding azimuth information indicating the detected orientation of the apparatus main body to each image obtained at the time of shooting and recording the information in a predetermined memory. A computer mounted on an imaging apparatus having a memory in which a plurality of images taken while changing the orientation of the apparatus main body with respect to a subject are recorded together with orientation information indicating the orientation of the apparatus main body when these images are taken A program executed by
In the computer,
A function of detecting a direction in which the apparatus main body is directed during reproduction of each image;
Based on the detected azimuth information indicating the orientation of the apparatus main body, a function of sequentially reading an image having the azimuth information from the memory in conjunction with the movement of the apparatus main body,
A program for realizing a function of displaying an image read from the memory. A program executed by a computer mounted on an imaging device for photographing a subject,
In the computer,
A function of acquiring a plurality of images while changing the orientation of the apparatus main body with respect to the subject;
A function of detecting the direction in which the device body is facing during shooting;
A function of adding the orientation information indicating the detected orientation of the apparatus main body to each of the images obtained at the time of shooting and recording the information in a predetermined memory;
A function of detecting a direction in which the apparatus main body is directed during reproduction of each image;
Based on the detected azimuth information indicating the orientation of the apparatus main body, a function of sequentially reading an image having the azimuth information from the memory in conjunction with the movement of the apparatus main body,
A program for realizing a function of displaying an image read from the memory.

Images