JPWO2019181579A1 - Cameras, imaging methods, and programs - Google Patents

Abstract

Provided are a camera, an image pickup method, and a program capable of notifying a user of charge information stored in an image pickup device during exposure without reading out the charge information. The camera (10) acquires a first captured image captured at a wide angle, acquires a second captured image in which a part of the planned imaging region immediately before the start of exposure is captured, and obtains posture information of the camera (10). Based on the detection information of the sensor (63) that detects the image and the sensor (63) from the start of exposure, the imaging range for each unit time based on the imaging range of the second captured image is detected and the unit is A first reproduced image corresponding to the imaging range for each hour is generated from the first captured image, or the first captured image and the second captured image, and the first reproduced image is integrated into the second during exposure. Generate a reproduction image of.

Description

The present invention relates to a camera, an imaging method, and a program, and more particularly to a technique for performing exposure while moving an imaging range.

Conventionally, there has been known an imaging method in which the timing of exposure is changed or the imaging range is moved during exposure to give a special effect to the obtained image. Examples of these imaging methods include long-exposure imaging and multiple-exposure imaging.

The long-exposure imaging is a method of taking an image with an exposure time longer than that of a normal exposure, and the multiple-exposure imaging is a method of acquiring a single image by performing a plurality of exposures.

Here, in the case of long-exposure or multiple exposure, if the image pickup apparatus blurs against the intention of the user (photographer), the image intended by the user cannot be obtained. Therefore, when performing long-exposure or multiple exposure, it is necessary for the user to suppress blurring of the image pickup apparatus, and a technique for assisting the suppression of blurring of the user has been considered.

Patent Document 1 describes a technique for suppressing a large movement of an imaging device by displaying a broadcast image on a display device to notify the user that multiple exposure is being performed when performing multiple exposure. Have been described.

Japanese Unexamined Patent Publication No. 2009-232227

Here, there is a method of intentionally moving an imaging device in a predetermined direction during a long exposure period to obtain an image having a special effect (moving imaging). When performing mobile imaging, the movement of the image sensor appears in the obtained image, so it is necessary to move the image sensor in the direction intended by the user, and the charge information stored in the image sensor during exposure. It is desired to confirm.

For example, even if the image pickup device is already moving in an unintended direction immediately after the start of exposure and the image pickup fails, the user cannot check the charge information stored in the image pickup device. , The exposure is completed to the end. Further, since the user cannot confirm the charge information stored in the image sensor, the charge obtained by the unintended movement of the image sensor when the image sensor moves in an unintended direction. Information is not available.

However, it is difficult to read the charge information stored in the image sensor during the exposure period, and the user cannot confirm the charge information stored in the image sensor during the exposure.

Therefore, there is a demand for a method of confirming the charge information stored in the image sensor without reading the charge information stored in the image sensor during the exposure time.

The present invention has been made in view of such circumstances, and an object of the present invention is a camera, an imaging method, and a program capable of notifying a user of charge information stored in an image sensor during exposure without reading the charge information. Is to provide.

One aspect of the present invention for achieving the above object is a camera capable of acquiring an captured image by performing exposure while moving the imaging range, at a wider angle than the display unit and the planned imaging area of the captured image. A first captured image acquisition unit that acquires the first captured image captured, and a second captured image acquisition unit that acquires a second captured image in which a part of the planned imaging region immediately before the start of exposure is captured. Based on the detection information of the sensor that detects the posture information of the camera and the sensor from the state where the exposure is started, the imaging range that detects the imaging range for each unit time based on the imaging range of the second captured image. The detection unit, the first reproduced image generation unit that generates the first reproduced image corresponding to the imaging range for each unit time from the first captured image, or the first captured image and the second captured image, and the first reproduced image generation unit. The camera includes a second reproduction image generation unit that generates a second reproduction image during exposure by integrating the first reproduction image, and a display control unit that displays the second reproduction image on the display unit.

According to this aspect, the imaging range detection unit detects the imaging range for each unit time based on the imaging range of the second captured image based on the detection information of the sensor from the state where the exposure is started. The first reproduced image corresponding to the imaging range for each unit time is generated from the first captured image or the first captured image and the second captured image by the reproduced image generation unit of 1, and the second reproduction is performed. The image generation unit generates a second reproduced image during exposure by integrating the first reproduced image. As a result, it is possible to notify the user without reading out the charge information accumulated in the image sensor during the exposure.

Preferably, the camera includes a start position detection unit that detects the start position of exposure in the first captured image by comparing the first captured image with the second captured image, and the imaging range detection unit includes the imaging range detection unit. The imaging range for each unit time is detected based on the start position.

According to this aspect, the start position detection unit detects the start position of the exposure in the first captured image by comparing the first captured image with the second captured image. As a result, it is possible to detect an accurate start position of exposure and accurately specify an imaging range for each unit time thereafter.

Preferably, the sensor detects the attitude information of the camera when acquiring the first captured image and the second captured image, and the imaging range detection unit acquires the first captured image and the second captured image. Based on the detection information of the case, the start position of the exposure is detected, and the imaging range for each unit time is detected.

According to this aspect, the sensor detects the attitude information of the camera when acquiring the first captured image and the second captured image, and the imaging range detection unit detects the first captured image and the second captured image. The start position of the exposure is detected based on the detection information in the case of acquiring the image, and the imaging range for each unit time is detected. As a result, it is possible to detect an accurate start position of exposure and accurately specify an imaging range for each unit time thereafter.

Preferably, the sensor further detects at least one of the camera's orientation information and height information from the reference plane.

According to this aspect, the sensor further detects at least one of the camera orientation information and the height information from the reference plane. This makes it possible to accurately detect the imaging range.

Preferably, the imaging range detection unit detects the moving direction of the imaging range every unit time based on the detection information of the sensor.

According to this aspect, the imaging range detection unit detects the moving direction of the imaging range every unit time based on the detection information of the sensor. This makes it possible to accurately detect the imaging range.

Preferably, the camera includes an exposure time control unit that acquires a set exposure time of exposure and controls the exposure.

According to this aspect, since the exposure time control unit acquires the set exposure time and controls the exposure, it is possible to perform the exposure for an accurate period.

Preferably, when the exposure time is longer than the threshold value, the display control unit causes the display unit to display the second reproduced image.

According to this aspect, when the exposure time is longer than the threshold value, the display control unit causes the display unit to display the second reproduced image. As a result, the charge information stored in the image sensor is efficiently notified in the long exposure time when the exposure time is longer than the threshold value.

Preferably, when the imaging range for each unit time detected by the imaging range detection unit is in a range other than the first captured image, the first reproduction image generation unit notifies the user of the notification image and the notification character. A first reproduced image including at least one of them is generated.

According to this aspect, when the imaging range for each unit time detected by the imaging range detection unit is in a range other than the first captured image by the second reproduced image generation unit, the broadcast image to be notified to the user and the notification image. A second reproduced image containing at least one of the broadcast characters is generated. As a result, it is possible to notify the user that the imaging range is in a range other than the first captured image, that is, that the movement of the imaging range is largely out of alignment.

Preferably, the display control unit switches between the first reproduced image and the second reproduced image and displays them on the display unit.

According to this aspect, since the display control unit switches between the first reproduced image and the second reproduced image and displays them on the display unit, the image currently captured by the user and the image accumulated up to the present are accumulated. You can check the image.

Preferably, the sensor is a camera shake sensor.

Preferably, a zoom control unit for fixing the zoom function is provided according to the first captured image and the imaging range for each unit time.

According to this aspect, the zoom control unit fixes the zoom function according to the first captured image and the imaging range for each unit time, so that it is possible to suppress imaging of an area other than the planned imaging area. can do.

Another aspect of the present invention is an imaging method capable of acquiring an captured image by performing exposure while moving the imaging range, and acquires a first captured image captured at a wider angle than a planned imaging region of the captured image. The sensor detects the captured image acquisition step 1 and the second captured image acquisition step of acquiring a second captured image in which a part of the planned imaging area immediately before the start of exposure is captured, and the attitude information of the camera. A detection step, an imaging range detection step that detects an imaging range for each unit time based on the imaging range of the second captured image based on the detection information of the sensor from the state where exposure is started, and an imaging range detection step for each unit time. The first reproduced image corresponding to the imaging range is integrated with the first reproduced image generation step of generating the first captured image or the first captured image and the second captured image, and the first reproduced image. This is an imaging method including a second reproduction image generation step of generating a second reproduction image during exposure and a display control step of displaying the second reproduction image on a display unit.

Another aspect of the present invention is a program for causing a computer to perform an imaging step capable of acquiring an captured image by performing exposure while moving the imaging range, and the first aspect of imaging is performed at a wider angle than the planned imaging region of the captured image. The first captured image acquisition step of acquiring the captured image, the second captured image acquisition step of acquiring a second captured image in which a part of the planned imaging area immediately before the start of exposure is captured, and the posture of the camera. A detection step of detecting information by a sensor, and an imaging range detection step of detecting an imaging range for each unit time based on the imaging range of a second captured image based on the detection information of the sensor from the state where exposure is started. The first reproduction image generation step of generating the first reproduction image corresponding to the imaging range for each unit time from the first captured image, the first captured image, and the second captured image, and the first An imaging step including a second reproduction image generation step of generating a second reproduction image during exposure by integrating the reproduction images of the above and a display control step of displaying the second reproduction image on the display unit is executed on the computer. It is a program to let you.

According to the present invention, the image pickup range detection unit detects an image pickup range for each unit time based on the image pickup range of the second captured image based on the detection information of the sensor from the state where the exposure is started. The first reproduced image corresponding to the imaging range for each unit time is generated from the first captured image or the first captured image and the second captured image by the reproduced image generation unit of 1, and the second reproduction is performed. Since the image generation unit generates the second reproduced image during the exposure by integrating the first reproduced image, the user is notified without reading the charge information accumulated in the image sensor during the exposure. Can be done.

FIG. 1 is a diagram illustrating imaging in which long-exposure is performed while moving the imaging range. FIG. 2 is a perspective view of the camera. FIG. 3 is a rear view of the camera. FIG. 4 is a block diagram showing an embodiment of the internal configuration of the camera. FIG. 5 is a block diagram showing a functional configuration example of the image processing unit. FIG. 6 is a diagram showing an example of capturing the first captured image. FIG. 7 is a diagram showing a first captured image. FIG. 8 is a diagram showing a second captured image. FIG. 9 is a diagram showing an example of the first reproduced image. FIG. 10 is a diagram showing an example of the second reproduced image. FIG. 11 is a diagram showing an example of the second reproduced image. FIG. 12 is a flow chart showing the flow of imaging by the camera. FIG. 13 is a diagram showing a first reproduced image having a zebra pattern. FIG. 14 is a diagram showing an example of the operation mode of the camera when the imaging range is moved. FIG. 15 is a diagram for explaining switching of the display of the display unit.

Hereinafter, preferred embodiments of the camera, imaging method, and program according to the present invention will be described with reference to the accompanying drawings.

[Long exposure-moving imaging]
FIG. 1 is a diagram illustrating imaging in which long-exposure is performed while moving the imaging range. FIG. 1 shows a case where a portion of the trunk 126 of the tree 125 (the pattern 145 of the outer skin of the trunk 126 is shown) is imaged as the planned imaging region H. The user (photographer) captures the planned imaging region H while moving the imaging range of the camera 10 (FIG. 2). Specifically, the camera 10 is tilted in the tilt direction (arrow L in the figure) to move the imaging range, and the entire area of the planned imaging region H is imaged.

The exposure time of the camera 10 is set so that the movement of the imaging range is completed by one exposure, and a relatively long exposure time is set. Specifically, the exposure time requires a period during which the imaging of the planned imaging region H is completed by moving the imaging range, and is at least 1 second, more preferably 5 seconds. The exposure time may be 10 seconds or 30 seconds depending on the planned imaging area H and the subject.

As described above, the present invention is preferably used in an imaging mode in which the exposure time is set to a relatively long time and the imaging range is moved during the exposure time.

[camera]
2 and 3 are a perspective view and a rear view showing the camera (digital camera) 10. The camera 10 is a digital camera that receives light that has passed through a lens by an image sensor, converts it into a digital signal, and records it on a recording medium as image data of a still image or a moving image. As shown in FIG. 2, the camera 10 is provided with a photographing lens 12, a strobe 1, and the like in front of the camera 10, and a shutter button 2, a power supply / mode switch 3, a mode dial 4, and the like are arranged on the upper surface thereof. On the other hand, as shown in FIG. 3, a liquid crystal monitor (LCD: Liquid Crystal Display) 30, a zoom button 5, a cross button 6, a MENU / OK button 7, a play button 8, a BACK button 9, and the like are arranged on the back of the camera. Has been done.

The photographing lens 12 is composed of a retractable zoom lens, and is extended from the camera body by setting the operation mode of the camera 10 to the photographing mode by the power supply / mode switch 3. The strobe 1 irradiates a main subject with strobe light.

The shutter button 2 is composed of a two-step stroke type switch composed of a so-called "half-press" and "full-press", and functions as an image pickup preparation instruction unit and an image recording instruction unit.

When the still image shooting mode is selected as the shooting mode and the shutter button 2 is "half-pressed", the camera 10 performs an imaging preparation operation for AF (Autofocus) / AE (Auto Exposure) control, and the shutter button 2 presses the shutter button 2. When "fully pressed", a still image is captured and recorded.

Further, the camera 10 starts recording a moving image when the moving image shooting mode is selected as the shooting mode and the shutter button 2 is "fully pressed", and the camera 10 starts recording when the shutter button 2 is "fully pressed" again. It stops and goes into a standby state.

The power / mode switch 3 has both a function as a power switch for turning on / off the power of the camera 10 and a function as a mode switch for setting the mode of the camera 10, and has "OFF position" and "playback position". It is slidably arranged between the image and the "imaging position". The power of the camera 10 is turned on by sliding the power / mode switch 3 to match the "reproduction position" or the "imaging position", and the power is turned off by adjusting to the "OFF position". Then, by sliding the power / mode switch 3 and adjusting it to the "reproduction position", it is set to the "reproduction mode", and by adjusting it to the "imaging position", it is set to the "shooting mode".

The mode dial 4 functions as a mode switching unit for setting the shooting mode of the camera 10, and the shooting mode of the camera 10 is set to various modes depending on the setting position of the mode dial 4. For example, there are a "still image shooting mode" for shooting a still image, a "moving image shooting mode" for shooting a moving image, a "long exposure-moving imaging mode" described with reference to FIG.

The liquid crystal monitor 30 functions as a display unit, displays a live view image in the shooting mode, displays a still image or a moving image in the playback mode, and displays a menu screen, etc., thereby partially displaying a graphical user interface. Also works as. Further, the liquid crystal monitor 30 displays the first reproduced image and the second reproduced image, which will be described later.

The zoom button 5 functions as a zoom instruction means for instructing zoom, and includes a telebutton 5T for instructing zoom to the telephoto side and a wide button 5W for instructing zoom to the wide-angle side. In the camera 10, the focal length of the photographing lens 12 changes by operating the telebutton 5T and the wide button 5W in the photographing mode. Further, in the reproduction mode, the telebutton 5T and the wide button 5W are operated to enlarge or reduce the image being reproduced.

The cross-shaped button 6 is an operation unit for inputting instructions in four directions of up, down, left, and right, and serves as a button (cursor movement operation means) for selecting an item from the menu screen or instructing selection of various setting items from each menu. Function. The left / right key functions as a frame advance (forward / reverse) button in playback mode.

The MENU / OK button 7 has both a function as a menu button for issuing a command to display a menu on the screen of the liquid crystal monitor 30 and a function as an OK button for instructing confirmation and execution of selected contents. It's a button.

The playback button 8 is a button for switching to a playback mode in which the captured still image or moving image is displayed on the liquid crystal monitor 30.

The BACK button 9 functions as a button for instructing to cancel the input operation or return to the previous operation state.

In the camera 10 according to the present embodiment, instead of providing a member unique to the buttons / switches, a touch panel is provided and the functions of the buttons / switches are realized by operating the touch panel. Good.

[Camera internal configuration]
FIG. 4 is a block diagram showing an embodiment of the internal configuration of the camera 10. The camera 10 records the captured image on the memory card 54, and the operation of the entire device is collectively controlled by the central processing unit (CPU) 40.

The camera 10 has an operation unit 38 such as a shutter button 2, a power / mode switch 3, a mode dial 4, a tele button 5T, a wide button 5W, a cross button 6, a MENU / OK button 7, a play button 8, and a BACK button 9. It is provided. The signal from the operation unit 38 is input to the CPU 40, and the CPU 40 controls each circuit of the camera 10 based on the input signal. For example, the sensor drive unit 32 controls the drive of the image sensor 16 and the lens drive unit 36 drives the lens. Control, aperture drive control, image pickup operation control, image processing control, image data recording / reproduction control by the aperture drive unit 34, and display control of the liquid crystal monitor 30 by the display control unit 61 and the like.

The luminous flux that has passed through the photographing lens 12, the diaphragm 14, the mechanical shutter (mechanical shutter) 15, and the like is imaged on the image sensor 16 which is a CMOS (Complementary Metal-Oxide Semiconductor) type color image sensor. The image sensor 16 is not limited to the CMOS type, and may be an XY address type or a CCD (Charge Coupled Device) type color image sensor.

A large number of light receiving elements (photodiodes) are two-dimensionally arranged in the image sensor 16, and the subject image formed on the light receiving surface of each photodiode has a signal voltage (or charge) of an amount corresponding to the amount of incident light. Is converted to a digital signal and output via an A / D (Analog / Digital) converter in the image sensor 16.

The CPU 40 also includes an exposure time control unit 65 and a zoom control unit 67.

The exposure time control unit 65 acquires the set exposure time and controls the end of the exposure from the start of the exposure. Specifically, when the user presses the shutter button 2, the exposure is started, and the exposure time control unit 65 controls the sensor drive unit 32 so as to end the exposure for which the acquired exposure time has elapsed.

The zoom control unit 67 controls the lens drive unit 36 to limit the zoom function. For example, the zoom function is fixed according to the first captured image and the imaging range for each unit time, which will be described later. That is, the zoom function is fixed so that the imaging range does not deviate from the planned imaging region H.

The image signal (image data) read from the image sensor 16 at the time of shooting a moving image or a still image is temporarily stored in a memory (SDRAM (Synchronous Dynamic Random Access Memory)) 48 via an image input controller 22 or temporarily stored in the memory (SDRAM (Synchronous Dynamic Random Access Memory)) 48. It is taken into the AF processing unit 42, the AE detection unit 44, and the like.

The CPU 40 controls each part of the camera 10 based on the operation of the operation unit 38, but the AF processing unit 42 always operates the AF during the shooting (display) of the live view image and the shooting (recording) of the moving image. And the AE operation by the AE detection unit 44 is performed.

When the shooting mode is the still image shooting mode, when the shutter button 2 is pressed (half-pressed) in the first stage, the CPU 40 performs the above-mentioned AF control again, and when the shutter button 2 is fully pressed, the CPU 40 performs the AF control again. The brightness of the subject (shooting Ev value) is calculated, the F value of the aperture 14 and the exposure time (shutter speed) by the mechanical shutter 15 are determined based on the calculated shooting Ev value according to the program diagram, and still images are taken (shooting). Exposure control).

On the other hand, when the shooting mode is the moving image shooting mode, when the shutter button 2 is fully pressed, the CPU 40 starts shooting and recording (recording) of the moving image. When shooting a moving image, the mechanical shutter 15 is opened, image data is continuously read from the image sensor 16 (for example, the frame rate is 30 frames / sec, 60 frames / sec), and phase difference AF is continuously performed. The brightness of the subject is calculated, and the shutter speed (charge accumulation time by the rolling shutter) and / or the aperture 14 by the aperture drive unit 34 are controlled by the shutter drive unit 33.

The CPU 40 moves the zoom lens forward and backward in the optical axis direction via the lens driving unit 36 in response to a zoom command from the zoom button 5 to change the focal length.

Further, the ROM (Read Only Memory) 47 stores various parameters and tables used for a camera control program, defect information of the image sensor 16, image processing, and the like. It can be replaced by EEPROM (Electrically Erasable Programmable Read-Only Memory).

The sensor 63 detects the posture information of the camera 10. For example, the posture information of the camera 10 may be detected by the camera shake sensor. Further, the sensor 63 may detect the orientation information of the camera 10 or the height information from the reference plane. Specific examples include GPS (Global Positioning System), gyro sensor, pressure sensor and the like.

The image processing unit 24 reads out unprocessed image data (RAW data) acquired via the image input controller 22 at the time of shooting a moving image or a still image and temporarily stored in the memory 48. The image processing unit 24 performs offset processing, pixel interpolation processing (interpolation processing of pixels for phase difference detection, scratched pixels, etc.), white balance correction, gain control processing including sensitivity correction, gamma correction processing, etc. for the read RAW data. Simultaneous processing (also referred to as "demosaic processing"), brightness and color difference signal generation processing, contour enhancement processing, color correction, and the like are performed.

The image data processed by the image processing unit 24 and processed as a live view image is input to the VRAM (Video RAM Random access memory) 50.

The VRAM 50 includes an area A and an area B for recording image data, each of which represents an image for one frame. In the VRAM 50, image data representing one frame of image is alternately rewritten in the A area and the B area. The written image data is read from an area other than the area in which the image data is rewritten among the areas A and B of the VRAM 50.

The image data read from the VRAM 50 is encoded by the video encoder 28 and output to the liquid crystal monitor 30 provided on the back surface of the camera. As a result, a live view image showing the subject image is displayed on the liquid crystal monitor 30.

The image data processed by the image processing unit 24 and processed as a still image or moving image for recording (luminance data (Y) and color difference data (Cb), (Cr)) are stored in the memory 48 again. It will be remembered.

The compression / decompression processing unit 26 processes the luminance data (Y) and the color difference data (Cb), (Cr) stored in the memory 48 by the image processing unit 24 at the time of recording a still image or a moving image. Give. In the case of a still image, it is compressed in a JPEG (Joint Photographic coding Experts Group) format, and in the case of a moving image, it is compressed in an H.264 format, for example. The compressed image data compressed by the compression / decompression processing unit 26 is recorded in the memory card 54 via the media controller 52.

Further, the compression / decompression processing unit 26 performs decompression processing on the compressed image data obtained from the memory card 54 via the media controller 52 in the playback mode. The media controller 52 records and reads compressed image data from the memory card 54.

<Functional configuration of image processing unit>
FIG. 5 is a block diagram showing a functional configuration example of the image processing unit 24.

The image processing unit 24 includes a first captured image acquisition unit 101, a second captured image acquisition unit 103, an imaging range detection unit 105, a start position detection unit 107, a first reproduction image generation unit 109, and a second reproduction. The image generation unit 111 is provided. Each process performed by the image processing unit 24 will be described below.

[First captured image]
The first captured image acquisition unit 101 is the first captured image (hereinafter referred to as A image) 121 (hereinafter referred to as A image) captured at a wider angle than the planned imaging region H of the captured image obtained by performing exposure while moving the imaging range. Acquire FIG. 7). The A image 121 is imaged in the preparatory stage of performing long exposure-moving imaging, and is imaged at a wider angle than the planned imaging area H.

FIG. 6 is a diagram showing an example of imaging of the A image 121. Note that FIG. 6 is an imaging of the A image 121 in the case of performing the long exposure-moving imaging described with reference to FIG. The A image 121 is captured by the camera 10 at a wide angle so as to include the planned imaging region H.

FIG. 7 is a diagram showing A image 121 captured by the imaging mode shown in FIG. In the A image 121, the entire tree 125 is contained within the angle of view, and naturally, the trunk 126, which is the planned imaging region H, is also contained within the angle of view. In the A image 121, the planned imaging region H is shown, and the imaging range (127A, 127B, 127C) for each unit time is shown. The imaging range will be described in detail later.

[Second captured image]
The second captured image acquisition unit 103 acquires a second captured image (hereinafter, referred to as B image) 131 (FIG. 8) in which a part of the planned imaging region H immediately before the start of exposure is captured. The B image 131 is an image captured immediately before the start of the main exposure of long exposure-moving imaging.

FIG. 8 is a diagram showing B image 131. The B image 131 is an image captured immediately before the start of long-exposure exposure, and a part of the trunk 126 is largely captured in a state where the angle of view is narrower than that of the A image 121 and the image is zoomed. The B image 131 is, for example, a B image 131 by cutting out a still image from the live view image (confirmation image) immediately before the main exposure.

[Imaging range detection]
The imaging range detection unit 105 detects the imaging range for each unit time based on the imaging range of the B image 131 based on the detection information of the sensor 63 from the state where the exposure is started. For example, the imaging range detection unit 105 uses a detection method 1 in which the imaging range of the B image 131 is detected in the A image 121, and the detected imaging range is used as the exposure start position to detect the imaging range for each unit time in the A image 121. To do. Further, for example, the imaging range detection unit 105 uses the detection method 2 for detecting the imaging range for each unit time from the detection information of the sensor 63 when the B image 131 and the A image 121 are imaged. The unit time means the frame rate (fps: frames per second) of the display of the first reproduced image or the second reproduced image, which will be described later.

≪Detection method 1≫
The imaging range detection unit 105 detects the imaging range for each unit time based on the start position detected by the start position detection unit 107.

The start position detection unit 107 detects the start position of the exposure in the A image 121 by comparing the A image 121 and the B image 131. For example, the start position detection unit 107 detects the position of the B image 131 in the A image 121 by pattern matching between the A image 121 and the B image 131. Then, the imaging range detection unit 105 acquires the detection result of the start position of the start position detection unit 107. The imaging range detection unit 105 detects the imaging range for each unit time based on the detection information of the sensor 63 with reference to the acquired start position.

≪Detection method 2≫
The imaging range detection unit 105 detects the start position based on the posture information of the camera 10 when the A image 121 and the B image 131 are captured, and detects the imaging range for each unit time.

The sensor 63 detects the posture information of the camera 10 when acquiring the A image 121 and the B image 131. For example, when the sensor 63 is a camera shake sensor, the imaging range detection unit 105 detects the imaging range by comparing the detection information of the camera shake sensor when the A image 121 and the B image 131 are imaged. Then, the imaging range detection unit 105 detects the start position of the exposure, and detects the imaging range for each unit time thereafter with reference to the detected start position of the exposure.

The imaging range detection unit 105 may detect the moving direction of the imaging range for each unit time of exposure based on the detection information of the sensor 63. The imaging range detection unit 105 can accurately detect the imaging range by acquiring the moving direction of the imaging range.

The image A image 121 of FIG. 7 described above shows the image pickup range detected by the image pickup range detection unit 105. The imaging range 127A, the imaging range 127B, and the imaging range 127C are imaging ranges that sequentially move every unit time. With the exposure start position as the imaging range 127A, the imaging range is sequentially moved to 127B and 127C by tilting the camera 10 (see FIG. 1). Although FIG. 7 shows three imaging ranges for simplification, in reality, there are many imaging ranges in the planned imaging area H.

[First reproduced image]
The first reproduced image generation unit 109 generates a first reproduced image (FIG. 9) corresponding to the imaging range for each unit time from the A image 121 or the A image 121 and the B image 131. The first reproduced image generation unit 109 generates the first reproduced image by using the image information of the imaging range for each unit time in the first reproduced image. Further, the first reproduced image generation unit 109 may generate a first reproduced image based on the B image 131 captured immediately before the start of exposure. In particular, when the change in the posture of the camera 10 is minute immediately after the start of exposure, there are many parts where the B image 131 and the region corresponding to the imaging range immediately after the start of the A image 121 overlap. Therefore, the first reproduced image generation unit 109 may generate the first reproduced image from the B image 131 and the A image 121.

FIG. 9 is a diagram showing an example of the first reproduced image. The first reproduced image 133A, the first reproduced image 133B, and the first reproduced image 133C corresponding to the imaging range 127A, the imaging range 127B, and the imaging range 127C described with reference to FIG. 7 are shown. Specifically, the imaging range 127A is set as the exposure start position, and the first reproduced image 133A corresponding thereto is shown, and the first reproduced image 133B corresponding to the imaging range 127B after the lapse of the next unit time is shown. The first reproduced image 133C corresponding to the imaging range 127C after the lapse of the next unit time is shown. That is, the first reproduced image 133 represents the charge information accumulated in the image pickup device 16 every unit time. The first reproduced image is represented by reference numeral 133, and the first reproduced image at each unit time is represented by reference numerals 133A, 133B, and 133C.

[Second reproduction image]
The second reproduced image generation unit 111 generates the second reproduced image by integrating the first reproduced image 133. Specifically, the second reproduction image generation unit 111 generates a second reproduction image by superimposing the first reproduction image 133 obtained every unit time from the start of exposure to the present.

FIG. 10 is a diagram showing an example of the second reproduced image. In the second reproduced image 135, the portion where the blur occurs is indicated by a dotted line.

The second reproduced image 135 is obtained by superimposing the first reproduced image 133A, the first reproduced image 133B, and the first reproduced image 133C shown in FIG. Since the first reproduced images (133A, 133B, 133C) in which different images are captured for each unit time are superimposed, blurring occurs in the second reproduced image 135. Specifically, in the second reproduced image 135, blurring occurs in the contour portion of the trunk 126, and blurring also occurs in the pattern 145 of the outer skin of the trunk 126.

Since the second reproduced image 135 thus obtained is generated by superimposing the first reproduced image 133 for each unit time, it represents the charge information accumulated in the image sensor 16 from the start of exposure to the present. ing.

FIG. 11 is a diagram showing another example of the second reproduced image 135. In the case of FIG. 11, the camera 10 has moved in an unintended direction, which is a failure example from the viewpoint that the image intended by the user cannot be obtained.

The second reproduced image 135 is an image of the trunk 126 of the tree 125 in the same manner as in FIG. In the case shown in FIG. 11, the posture of the camera 10 is blurred when the imaging range is moved. In this case, based on the detection information of the sensor 63 of the camera 10, the image pickup range and the first reproduced image 133 also reflect the influence of the blurring of the posture of the camera 10. Therefore, the influence of the blurring of the camera 10 is also expressed in the second reproduced image 135 generated by integrating the first reproduced image 133.

In this way, if the blurring of the posture of the camera 10 during exposure can be expressed by the second reproduced image 135, the user confirms the second reproduced image 135 to take an image before the exposure is completed. Can be stopped and retaken. In addition, a new image can be created by utilizing the blurring of the posture of the camera 10 during exposure.

[Imaging method]
Next, an imaging method (imaging step) of the camera 10 will be described. FIG. 12 is a flow chart showing the flow of imaging of the camera 10.

The captured A image 121 is captured by the camera 10 (step S10) at a wide angle including the region (scheduled imaging region H) to be imaged by long-time exposure, and the first captured image acquisition unit 101 acquires the A image 121. (First captured image acquisition step). Next, the camera 10 captures and stores the live view image until the long-exposure exposure starts (step S11).

Next, the camera 10 starts the main exposure of the mobile imaging (step S12). Then, the second captured image acquisition unit 103 acquires the B image 131 captured immediately before the start of the main exposure from the stored live view image (step S13: second captured image acquisition step).

After that, the start position detection unit 107 detects the position (exposure start position) of the B image 131 in the A image 121 (step S14: detection step). Then, the start position detection unit 107 determines whether or not the position of the B image 131 can be detected in the A image 121 (step S15). When the start position detection unit 107 cannot detect the position of the B image 131 in the A image 121, the display control unit 61 causes the liquid crystal monitor 30 to display an error message (step S23).

When the start position detection unit 107 can detect the position of the B image 131 in the A image 121, the exposure time control unit 65 determines whether the acquired exposure time is equal to or less than the threshold value (step S16). ). When the exposure time control unit 65 determines that the acquired exposure time is equal to or less than the threshold value, the display control unit 61 ends the display without displaying the second reproduced image 135 on the liquid crystal monitor 30. On the other hand, when it is determined that the exposure time acquired by the exposure time control unit 65 is larger than the threshold value, the display control unit 61 performs the following processing in order to display the second reproduced image 135 on the liquid crystal monitor 30.

The imaging range detection unit 105 acquires the current posture information (fanning, etc.) and relative position coordinates (Δx, Δy, Δz) of the camera 10 (step S17). The relative position coordinates are the coordinates of the center of the imaging range, and are the relative coordinates in the A image 121. The relative position coordinates can be calculated based on the imaging range, and the coordinates (Δx, Δy, Δz) are set based on the detection information from the sensor 63 with the center of the imaging range (center of the B image) at the start position of the exposure as the origin. It is possible to get it. After that, the imaging range detection unit 105 detects the current imaging range from the A image 121, the B image 131, and the relative position coordinates (imaging range detection step), and the first reproduction image generation unit 109 detects the first reproduction image 133. Generate (step S18: first reproduction image generation step).

Then, the image pickup range detection unit 105 determines whether or not the image pickup range is outside the range of the A image 121 (step S19). Then, when the imaging range detection unit 105 determines that the imaging range is outside the range of the A image 121, the first reproduction image generation unit 109 displays the portion outside the range with a zebra pattern. Generate 133 (step S24).

FIG. 13 is a diagram showing a first reproduced image 133 having a zebra pattern. The trunk 126 of the tree is shown in the first reproduced image 133, but it exceeds the range of the A image 121 because the angle of view is too far to the left when facing the figure. Therefore, the zebra pattern 143 notifies the user that the portion that exceeds the range of the A image 121 is out of the range, and the zebra pattern 143 is an example of the broadcast image. Further, the notification to the user may be performed not only by the notification image but also by the notification character, or may be performed by the notification image and the notification character.

Returning to FIG. 12, when the imaging range detected by the imaging range detection unit 105 is within the range of the A image 121, the second reproduction image generation unit 111 has been integrated into the image sensor 16 so far. An image is generated by integrating the first reproduced image 133 (second reproduced image generation step), and the display control unit 61 displays the generated second reproduced image 135 on the liquid crystal monitor 30 (step S20). : Display control step).

Next, the exposure time control unit 65 determines whether or not the current time is within the acquired exposure time (step S21). Then, when the current time is within the exposure time, the imaging range detection unit 105 acquires the posture information and the relative position coordinate value of the current camera 10 to detect the current imaging range. The flow indicated by the alternate long and short dash line F is performed at, for example, a normal frame rate (50 to 60 fps). After that, when the exposure time ends, the display control unit 61 ends the display of the reproduced image on the liquid crystal monitor 30 (step S22).

In the above embodiment, the hardware structure of the processing unit that executes various processes is various processors as shown below. For various processors, the circuit configuration can be changed after manufacturing the CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), etc., which are general-purpose processors that execute software (programs) and function as various processing units. Programmable Logic Device (PLD), a dedicated electric circuit, which is a processor having a circuit configuration specially designed to execute a specific process such as an ASIC (Application Specific Integrated Circuit). Is done.

One processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of a CPU and an FPGA). You may. Further, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, one processor is configured by a combination of one or more CPUs and software, as represented by a computer such as a client or a server. There is a form in which a processor functions as a plurality of processing units. Secondly, as typified by System On Chip (SoC), there is a form in which a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used. is there. As described above, the various processing units are configured by using one or more of the above-mentioned various processors as a hardware-like structure.

Further, the hardware structure of these various processors is, more specifically, an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

Each of the above configurations and functions can be appropriately realized by any hardware, software, or a combination of both. For example, for a program that causes a computer to perform the above-mentioned processing steps, a computer-readable recording medium (non-temporary recording medium) that records such a program, or a computer on which such a program can be installed. However, it is possible to apply the present invention.

As described above, in the present invention, the first reproduced image generation unit 109 makes the first reproduced image 133 corresponding to the imaging range for each unit time from the A image 121 or the A image 121 and the B image 131. The generated second reproduced image generation unit 111 generates a second reproduced image 135 during exposure by integrating the first reproduced image 133. As a result, the user can be notified by reproducing the information accumulated in the image sensor 16 during the exposure.

<Others>
[Camera movement mode]
In the above description, the camera 10 moves the imaging range by operating in the tilt direction, but the operation of the camera 10 that moves the imaging range is not limited to this.

FIG. 14 is a diagram showing an example of an operation mode of the camera 10 when the imaging range is moved. As shown in the figure, the posture of the camera 10 may be kept as it is, and the imaging range may be moved by changing the height to capture the planned imaging region H. The camera 10 moves vertically upward with respect to the ground from t = 0 to t = Δt, which is the start of the main exposure, and moves the imaging range to perform imaging of the planned imaging region H.

[Display switching between the first reproduced image and the second reproduced image]
In the above description, the mode in which the second reproduced image 135 is displayed on the liquid crystal monitor 30 of the camera 10 has been described, but the present invention is not limited to this. For example, the liquid crystal monitor 30 may switch between the first reproduced image 133 and the second reproduced image for display.

FIG. 15 is a diagram for explaining switching of the display of the liquid crystal monitor 30. In the display mode indicated by reference numeral 140, the first reproduced image 133 is displayed on the liquid crystal monitor 30. Further, in the display form indicated by reference numeral 141, the second reproduced image 135 is displayed on the liquid crystal monitor 30. For example, the user may switch between displaying the first reproduced image 133 and displaying the second reproduced image 135 on the liquid crystal monitor 30 by using the operation unit 38 according to the required information.

By controlling the display on the liquid crystal monitor 30 in this way, the user can switch between the currently captured image and the integrated image for confirmation.

[Exposure time]
In the above description, the case where the exposure time is set in advance has been described, but the present invention is not limited to this. For example, the present invention is also applied to long-exposure exposure in which the exposure ends when the user presses the shutter button 2 of the camera 10 and the user releases the press.

Although the examples of the present invention have been described above, it goes without saying that the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.

1: Strobe 2: Shutter button 3: Power / mode switch 4: Mode dial 5: Zoom button 6: Cross button 7: MENU / OK button 8: Play button 9: BACK button 10: Camera 12: Shooting lens 14: Aperture 15 : Mechanical shutter 16: Image sensor 22: Image input controller 24: Image processing unit 26: Compression / decompression processing unit 28: Video encoder 30: Liquid crystal monitor 32: Sensor drive unit 33: Shutter drive unit 34: Aperture drive unit 36: Lens drive unit 38: Operation unit 40: CPU
42: AF processing unit 44: AE detection unit 47: ROM
48: Memory 50: URAM
52: Media controller 54: Memory card 61: Display control unit 63: Sensor 65: Exposure time control unit 67: Zoom control unit 101: First captured image acquisition unit 103: Second captured image acquisition unit 105: Imaging range detection Unit 107: Start position detection unit 109: First reproduction image generation unit 111: Second reproduction image generation unit 121: First captured image 125: Tree 126: Tree trunk 127A: Imaging range 127B: Imaging range 127C: Imaging range 131: Second captured image 133: First reproduced image 133A: First reproduced image 133B: First reproduced image 133C: First reproduced image 135: Second reproduced image 145: Tree trunk Pattern H: Scheduled image area Step S10-S24: Camera operation process

Claims (14)

It is a camera that can acquire an captured image by performing exposure while moving the imaging range.
Display and
A first captured image acquisition unit that acquires a first captured image captured at a wider angle than the planned imaging region of the captured image, and a first captured image acquisition unit.
A second captured image acquisition unit that acquires a second captured image in which a part of the planned imaging region is captured immediately before the start of the exposure.
A sensor that detects the attitude information of the camera and
An imaging range detection unit that detects an imaging range for each unit time based on the imaging range of the second captured image based on the detection information of the sensor from the state where the exposure is started.
The first reproduced image corresponding to the imaging range for each unit time is generated from the first captured image, the first captured image, and the second captured image, and the first reproduced image generation unit.
A second reproduction image generation unit that generates a second reproduction image during the exposure by integrating the first reproduction image, and a second reproduction image generation unit.
A display control unit that displays the second reproduced image on the display unit,
A camera equipped with. A start position detection unit for detecting the start position of the exposure in the first captured image by comparing the first captured image with the second captured image is provided.
The camera according to claim 1, wherein the imaging range detection unit detects an imaging range for each unit time based on the start position. The sensor detects the posture information of the camera when acquiring the first captured image and the second captured image, and detects the posture information of the camera.
The imaging range detection unit detects the start position of the exposure based on the detection information when acquiring the first captured image and the second captured image, and determines the imaging range for each unit time. The camera according to claim 2 for detection. The camera according to any one of claims 1 to 3, wherein the sensor further detects at least one of the orientation information of the camera and the height information from the reference plane. The camera according to claim 4, wherein the imaging range detection unit detects a moving direction of the imaging range for each unit time based on the detection information of the sensor. The camera according to any one of claims 1 to 5, further comprising an exposure time control unit that acquires the set exposure time of the exposure and controls the exposure. The camera according to claim 6, wherein when the exposure time is longer than the threshold value, the display control unit displays the second reproduced image on the display unit. When the imaging range for each unit time detected by the imaging range detection unit is in a range other than the first captured image, the first reproduced image generation unit notifies the user of the notification image and the notification character. The camera according to any one of claims 1 to 7, which generates the first reproduced image including at least one of them. The camera according to any one of claims 1 to 8, wherein the display control unit switches between the first reproduced image and the second reproduced image and displays them on the display unit. The camera according to any one of claims 1 to 9, wherein the sensor is a camera shake sensor. The camera according to any one of claims 1 to 10, further comprising a zoom control unit that fixes a zoom function according to the first captured image and the imaging range for each unit time. It is an imaging method that can acquire an captured image by performing exposure while moving the imaging range.
The first captured image acquisition step of acquiring the first captured image captured at a wider angle than the planned imaging region of the captured image, and
A second captured image acquisition step of acquiring a second captured image in which a part of the planned imaging region is captured immediately before the start of the exposure, and
A detection step that detects camera attitude information with a sensor,
An imaging range detection step of detecting an imaging range for each unit time based on an imaging range of the second captured image based on the detection information of the sensor from the state of starting the exposure.
The first reproduced image generation step of generating the first reproduced image corresponding to the imaging range for each unit time from the first captured image, the first captured image, and the second captured image,
A second reproduction image generation step of generating the second reproduction image during the exposure by integrating the first reproduction image, and
A display control step for displaying the second reproduced image on the display unit, and
Imaging method including. It is a program that causes a computer to perform an imaging process capable of acquiring an captured image by performing exposure while moving the imaging range.
The first captured image acquisition step of acquiring the first captured image captured at a wider angle than the planned imaging region of the captured image, and
A second captured image acquisition step of acquiring a second captured image in which a part of the planned imaging region is captured immediately before the start of the exposure, and
A detection step that detects camera attitude information with a sensor,
An imaging range detection step of detecting an imaging range for each unit time based on an imaging range of the second captured image based on the detection information of the sensor from the state of starting the exposure.
The first reproduced image generation step of generating the first reproduced image corresponding to the imaging range for each unit time from the first captured image, the first captured image, and the second captured image,
A second reproduction image generation step of generating the second reproduction image during the exposure by integrating the first reproduction image, and
A display control step for displaying the second reproduced image on the display unit, and
A program that causes a computer to perform an imaging process including. A non-temporary, computer-readable recording medium when the instructions stored in the recording medium are read by a computer.
It is an imaging function that can acquire an captured image by performing exposure while moving the imaging range.
A second captured image acquisition step of acquiring a second captured image in which a part of the planned imaging region is captured immediately before the start of the exposure, and
A detection step that detects camera attitude information with a sensor,
An imaging range detection step of detecting an imaging range for each unit time based on an imaging range of the second captured image based on the detection information of the sensor from the state of starting the exposure.
The first reproduced image generation step of generating the first reproduced image corresponding to the imaging range for each unit time from the first captured image, the first captured image, and the second captured image,
A second reproduction image generation step of generating the second reproduction image during the exposure by integrating the first reproduction image, and
A display control step for displaying the second reproduced image on the display unit, and
A recording medium that enables a computer to perform imaging functions including.

Images