WO2019181579A1

WO2019181579A1 - Camera, imaging method, and program

Info

Publication number: WO2019181579A1
Application number: PCT/JP2019/009477
Authority: WO
Inventors: 一樹石田; 小林　潤; 祐樹杉原; 真彦宮田
Original assignee: 富士フイルム株式会社
Priority date: 2018-03-20
Filing date: 2019-03-08
Publication date: 2019-09-26
Also published as: JP7028960B2; JPWO2019181579A1

Abstract

The present invention provides a camera, an imaging method, and a program which make it possible to notify a user without reading out electric charge information accumulated in an imaging element during exposure. A camera (10) acquires a first captured image captured at a wide angle, acquires a second captured image obtained by capturing an image of a portion of an imaging scheduled region immediately before the start of exposure, on the basis of a sensor (63) for detecting attitude information of the camera (10), and detection information of the sensor (63) from the start of exposure, detects an imaging range per unit time using the imaging range of the second captured image as a reference, generates a first reproduced image corresponding to the imaging range per unit time from the first captured image, or the first captured image and the second captured image, and generates a second reproduced image during exposure obtained by integrating the first reproduced images.

Description

Camera, imaging method, and program

The present invention relates to a camera, an imaging method, and a program, and more particularly to a technique for performing exposure while moving an imaging range.

Conventionally, there has been known an imaging method in which a special effect is given to an obtained image by changing the timing of exposure or moving an imaging range during exposure. Examples of these imaging methods include long exposure imaging and multiple exposure imaging.

“Long exposure imaging” is a technique in which the exposure time is set longer than normal exposure, and multiple exposure imaging is a technique in which a single image is obtained by performing multiple exposures.

Here, when performing long-time exposure or multiple exposure, if the imaging device shakes against the user's (photographer's) intention, an image intended by the user cannot be obtained. Therefore, when performing long exposure or multiple exposure, it is necessary for the user to suppress blurring of the imaging apparatus, and a technique for assisting suppression of blurring of the user has been considered.

In Patent Document 1, when performing multiple exposure, there is a technique for suppressing a large movement of the imaging device by displaying a notification image on a display device and notifying the user that multiple exposure is being performed. Are listed.

JP 2009-232227 A

Here, there is a method of obtaining an image having a special effect by intentionally moving the imaging device in a predetermined direction during a long exposure period (moving imaging). When moving imaging is performed, the movement of the imaging device appears in the obtained image, so it is necessary to move the imaging device in the direction intended by the user, and the charge information accumulated in the imaging device during exposure It is hoped to confirm.

For example, the user cannot check the charge information accumulated in the image sensor even if the imaging apparatus has already moved in an unintended direction immediately after the exposure starts and imaging has failed. The exposure is completed to the end. In addition, since the user cannot check the charge information accumulated in the image sensor, when the image pickup apparatus moves in an unintended direction, the charge obtained by the unintended movement of the image pickup apparatus. Information is not available.

However, it is difficult to read out the charge information accumulated in the image sensor during the exposure period, and the user cannot check the charge information accumulated in the image sensor during the exposure.

Therefore, there is a demand for a method for confirming the charge information stored in the image sensor without reading out the charge information stored in the image sensor during the exposure time.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a camera, an imaging method, and a program capable of notifying a user without reading out charge information stored in an imaging device during exposure. Is to provide.

In order to achieve the above object, one aspect of the present invention is a camera capable of acquiring a captured image by performing exposure while moving an imaging range, and having a wider angle than an imaging scheduled area of the captured image. A first captured image acquisition unit that acquires a captured first captured image, and a second captured image acquisition unit that acquires a second captured image in which a part of the planned imaging region immediately before the start of exposure is captured And an imaging range for detecting an imaging range per unit time based on the imaging range of the second captured image, based on the detection information of the sensor from the state in which exposure is started and the sensor that detects the posture information of the camera A detection unit, and a first reproduction image generation unit that generates a first reproduction image corresponding to an imaging range for each unit time from the first imaging image, or the first imaging image and the second imaging image; Second during exposure integrating the first reproduced image A second reproduced image generating unit that generates a reproduced image, a display control unit for displaying the second reproduction image on the display unit, a camera equipped with.

According to this aspect, the imaging range detection unit detects the imaging range per unit time based on the imaging range of the second captured image based on the detection information of the sensor from the state in which the exposure is started. The first reproduction image generation unit generates a first reproduction image corresponding to the imaging range for each unit time from the first imaging image, or the first imaging image and the second imaging image, and performs the second reproduction. The image generation unit generates a second reproduced image during exposure obtained by integrating the first reproduced image. Thereby, it is possible to notify the user without reading out the charge information accumulated in the image sensor during exposure.

Preferably, the camera includes a start position detection unit that detects a start position of exposure in the first captured image by comparing the first captured image and the second captured image, and the imaging range detection unit includes: Based on the start position, an imaging range for each unit time is detected.

According to this aspect, the start position detection unit detects the exposure start position in the first captured image by comparing the first captured image and the second captured image. As a result, an accurate exposure start position can be detected, and the subsequent imaging range for each unit time can be accurately specified.

Preferably, the sensor detects posture information of the camera when acquiring the first captured image and the second captured image, and the imaging range detection unit acquires the first captured image and the second captured image. Based on the detection information, the exposure start position is detected, and the imaging range for each unit time is detected.

According to this aspect, the orientation information of the camera when the first captured image and the second captured image are acquired by the sensor is detected, and the first captured image and the second captured image are detected by the imaging range detection unit. Based on the detection information in the case of acquiring, the exposure start position is detected, and the imaging range for each unit time is detected. As a result, an accurate exposure start position can be detected, and the subsequent imaging range for each unit time can be accurately specified.

Preferably, the sensor further detects at least one of camera orientation information and height information from the reference plane.

According to this aspect, at least one of the orientation information of the camera and the height information from the reference plane is further detected by the sensor. Thereby, the imaging range can be accurately detected.

Preferably, the imaging range detection unit detects the moving direction of the imaging range per unit time based on the detection information of the sensor.

According to this aspect, the moving direction of the imaging range per unit time is detected by the imaging range detection unit based on the detection information of the sensor. Thereby, the imaging range can be accurately detected.

Preferably, the camera includes an exposure time control unit that acquires an exposure time set for exposure and controls the exposure.

According to this aspect, since the exposure time set by the exposure is acquired and the exposure is controlled by the exposure time control unit, it is possible to perform exposure for an accurate period.

Preferably, when the exposure time is longer than the threshold value, the display control unit displays the second reproduced image on the display unit.

According to this aspect, when the exposure time is longer than the threshold value, the display control unit displays the second reproduction image on the display unit. Thereby, in the long exposure with the exposure time longer than the threshold value, the charge information accumulated in the image sensor is efficiently notified.

Preferably, the first reproduction image generation unit is configured to notify the user of a notification image and a notification character when the imaging range per unit time detected by the imaging range detection unit is in a range other than the first captured image. A first reproduction image including at least one of them is generated.

According to this aspect, when the imaging range for each unit time detected by the imaging range detection unit is in a range other than the first captured image by the second reproduction image generation unit, the notification image to notify the user and A second reproduction image including at least one of the notification characters is generated. Thereby, it can be notified to the user that the imaging range is in a range other than the first captured image, that is, the movement of the imaging range has greatly deviated.

Preferably, the display control unit switches between the first reproduction image and the second reproduction image and causes the display unit to display them.

According to this aspect, since the display control unit switches between the first reproduction image and the second reproduction image and causes the display unit to display the image, the image currently captured for the user is integrated with the current image. The image can be confirmed.

Preferably, the sensor is a camera shake sensor.

Preferably, a zoom control unit is provided that fixes the zoom function according to the first captured image and the imaging range per unit time.

According to this aspect, since the zoom function is fixed by the zoom control unit according to the first captured image and the imaging range for each unit time, it is possible to suppress capturing an area other than the scheduled imaging area. can do.

Another aspect of the present invention is an imaging method capable of acquiring a captured image by performing exposure while moving an imaging range, and acquiring a first captured image captured at a wider angle than the planned imaging region of the captured image. A first captured image acquisition step, a second captured image acquisition step of acquiring a second captured image in which a part of the planned imaging region immediately before the start of exposure is captured, and camera posture information is detected by a sensor. A detection step; an imaging range detection step for detecting an imaging range per unit time based on the imaging range of the second captured image based on detection information of the sensor from a state in which exposure is started; The first reproduced image corresponding to the imaging range is generated from the first captured image, or the first reproduced image generation step for generating the first reproduced image from the first captured image and the second captured image, and the first reproduced image is integrated. exposure A second reproduced image generation step of generating a second reproduced image, and a display control step of displaying the second reproduction image on the display unit is an imaging method comprising.

Another aspect of the present invention is a program for causing a computer to execute an imaging process capable of acquiring a captured image by performing exposure while moving an imaging range. The first image is captured at a wider angle than the planned imaging region of the captured image. A first captured image acquisition step for acquiring a captured image, a second captured image acquisition step for acquiring a second captured image in which a part of the planned imaging region immediately before the start of exposure is captured, and the attitude of the camera A detection step of detecting information by a sensor, and an imaging range detection step of detecting an imaging range per unit time based on the imaging range of the second captured image based on detection information of the sensor from a state where exposure is started And a first reproduction image generation step for generating a first reproduction image corresponding to the imaging range for each unit time from the first imaging image or the first imaging image and the second imaging image. Imaging, and a second reproduction image generation step for generating a second reproduction image during exposure obtained by integrating the first reproduction image, and a display control step for displaying the second reproduction image on the display unit A program for causing a computer to execute a process.

According to the present invention, the imaging range detection unit detects the imaging range for each unit time based on the imaging range of the second captured image based on the detection information of the sensor from the state where the exposure is started. The first reproduction image generation unit generates a first reproduction image corresponding to the imaging range for each unit time from the first imaging image, or the first imaging image and the second imaging image, and performs the second reproduction. The image generation unit generates a second reproduction image during exposure obtained by integrating the first reproduction image, so that the user is notified without reading out the charge information accumulated in the image sensor during the exposure. Can do.

FIG. 1 is a diagram illustrating imaging in which long exposure is performed while moving the imaging range. FIG. 2 is a perspective view of the camera. FIG. 3 is a rear view of the camera. FIG. 4 is a block diagram showing an embodiment of the internal configuration of the camera. FIG. 5 is a block diagram illustrating a functional configuration example of the image processing unit. FIG. 6 is a diagram illustrating an example of capturing the first captured image. FIG. 7 is a diagram illustrating a first captured image. FIG. 8 is a diagram illustrating a second captured image. FIG. 9 is a diagram illustrating an example of the first reproduced image. FIG. 10 is a diagram illustrating an example of the second reproduced image. FIG. 11 is a diagram illustrating an example of a second reproduced image. FIG. 12 is a flowchart showing the flow of imaging by the camera. FIG. 13 is a diagram illustrating a first reproduced image having a zebra pattern. FIG. 14 is a diagram illustrating an example of the operation mode of the camera when moving the imaging range. FIG. 15 is a diagram for explaining switching of display on the display unit.

Hereinafter, preferred embodiments of a camera, an imaging method, and a program according to the present invention will be described with reference to the accompanying drawings.

[Long exposure-moving imaging]
FIG. 1 is a diagram illustrating imaging in which long exposure is performed while moving the imaging range. FIG. 1 shows a case where a portion of the trunk 126 of the tree 125 (the outer skin pattern 145 of the trunk 126 is shown) is imaged as the imaging scheduled region H. The user (photographer) images the imaging scheduled area H while moving the imaging range of the camera 10 (FIG. 2). Specifically, the camera 10 is tilted in the tilt direction (arrow L in the figure), the imaging range is moved, and the entire imaging planned area H is imaged.

The setting of the exposure time of the camera 10 is set so that the movement of the imaging range is completed by one exposure, and a relatively long exposure time is set. Specifically, the exposure time requires a period during which the imaging range H is completed by moving the imaging range, and is longer than at least 1 second, more preferably longer than 5 seconds. Depending on the scheduled imaging area H and subject, the exposure time may be 10 seconds or 30 seconds.

As described above, the present invention is suitably used in an imaging mode in which the exposure time is set to a relatively long time and the imaging range is moved during the exposure time.

[camera]
2 and 3 are a perspective view and a rear view showing the camera (digital camera) 10, respectively. The camera 10 is a digital camera that receives light passing through a lens with an imaging device, converts the light into a digital signal, and records it on a recording medium as still image or moving image data. As shown in FIG. 2, the camera 10 is provided with a photographing lens 12, a strobe 1 and the like on the front, and a shutter button 2, a power / mode switch 3, a mode dial 4 and the like on the top. On the other hand, as shown in FIG. 3, a liquid crystal monitor (LCD: Liquid Crystal Display) 30, a zoom button 5, a cross button 6, a MENU / OK button 7, a playback button 8, a BACK button 9 and the like are disposed on the back of the camera. Has been.

The photographing lens 12 is constituted by a retractable zoom lens, and is set out from the camera body by setting the operation mode of the camera 10 to the photographing mode by the power / mode switch 3. The strobe 1 irradiates a main subject with strobe light.

The shutter button 2 is composed of a two-stroke switch composed of so-called “half-press” and “full-press”, and functions as an imaging preparation instruction unit and also functions as an image recording instruction unit.

When the still image shooting mode is selected as the shooting mode and the shutter button 2 is “half-pressed”, the camera 10 performs an imaging preparation operation for performing AF (Autofocus) / AE (Auto-Exposure) control. When “fully pressed”, a still image is captured and recorded.

Further, the camera 10 starts recording a moving image when the moving image shooting mode is selected as the shooting mode and the shutter button 2 is “fully pressed”, and when the shutter button 2 is “fully pressed” again, the recording is performed. Stop and enter standby mode.

The power / mode switch 3 has both a function as a power switch for turning on / off the power of the camera 10 and a function as a mode switch for setting the mode of the camera 10, and “OFF position” and “reproduction position”. And “imaging position” are slidably arranged. The camera 10 is turned on by sliding the power / mode switch 3 to the “reproduction position” or “imaging position”, and turned off by setting it to the “OFF position”. Then, the power / mode switch 3 is slid to match the “reproduction position” to set the “reproduction mode”, and to the “imaging position” to set the “shooting mode”.

The mode dial 4 functions as a mode switching unit for setting the shooting mode of the camera 10, and the shooting mode of the camera 10 is set to various modes depending on the setting position of the mode dial 4. For example, there are “still image shooting mode” in which still image shooting is performed, “moving image shooting mode” in which moving image shooting is performed, and “long exposure-moving imaging mode” described in FIG.

The liquid crystal monitor 30 functions as a display unit, displays a live view image in the shooting mode, displays a still image or a moving image in the playback mode, and displays a menu screen to display a part of the graphical user interface. Also works. The liquid crystal monitor 30 displays a first reproduction image and a second reproduction image, which will be described later.

The zoom button 5 functions as zoom instruction means for instructing zooming, and includes a tele button 5T for instructing zooming to the telephoto side and a wide button 5W for instructing zooming to the wide angle side. When the camera 10 is operated in the shooting mode, the tele button 5T and the wide button 5W are operated to change the focal length of the shooting lens 12. Further, when the tele button 5T and the wide button 5W are operated in the reproduction mode, the image being reproduced is enlarged or reduced.

The cross button 6 is an operation unit for inputting instructions in four directions (up, down, left, and right). Function. The left / right key functions as a frame advance (forward / reverse feed) button in the playback mode.

The MENU / OK button 7 is an operation having a function as a menu button for instructing to display a menu on the screen of the liquid crystal monitor 30 and a function as an OK button for instructing confirmation and execution of selection contents. Button.

The playback button 8 is a button for switching to a playback mode in which a captured still image or moving image is displayed on the liquid crystal monitor 30.

The BACK button 9 functions as a button for instructing to cancel the input operation or return to the previous operation state.

In the camera 10 according to the present embodiment, the buttons / switches are not provided with specific members, but a touch panel is provided to operate the buttons / switches by operating them. Good.

[Internal structure of camera]
FIG. 4 is a block diagram showing an embodiment of the internal configuration of the camera 10. The camera 10 records captured images on a memory card 54, and the operation of the entire apparatus is centrally controlled by a central processing unit (CPU: Central Processing Unit) 40.

The camera 10 includes an operation unit 38 such as a shutter button 2, a power / mode switch 3, a mode dial 4, a tele button 5T, a wide button 5W, a cross button 6, a MENU / OK button 7, a playback button 8, and a BACK button 9. Is provided. A signal from the operation unit 38 is input to the CPU 40, and the CPU 40 controls each circuit of the camera 10 based on the input signal. For example, the sensor driving unit 32 controls the driving of the image sensor 16 and the lens driving unit 36 drives the lens. Control, aperture drive control by the aperture drive unit 34, imaging operation control, image processing control, image data recording / reproduction control, display control of the liquid crystal monitor 30 by the display control unit 61, and the like.

The luminous flux that has passed through the photographing lens 12, the diaphragm 14, the mechanical shutter (mechanical shutter) 15 and the like forms an image on the image sensor 16 that is a CMOS (Complementary Metal-Oxide Semiconductor) type color image sensor. The image sensor 16 is not limited to the CMOS type, but may be an XY address type or a CCD (Charge-Coupled Device) type color image sensor.

The image sensor 16 has a large number of light receiving elements (photodiodes) arranged two-dimensionally, and the subject image formed on the light receiving surface of each photodiode is an amount of signal voltage (or charge) corresponding to the amount of incident light. Is converted into a digital signal via an A / D (Analog / Digital) converter in the image sensor 16 and output.

The CPU 40 includes an exposure time control unit 65 and a zoom control unit 67.

The exposure time control unit 65 acquires the set exposure time and controls the end of exposure from the start of exposure. Specifically, when the user presses the shutter button 2, exposure is started, and the exposure time control unit 65 controls the sensor drive unit 32 so as to end the exposure for which the acquired exposure time has elapsed.

The zoom control unit 67 controls the lens driving unit 36 to limit the zoom function. For example, the zoom function is fixed in accordance with a first captured image to be described later and an imaging range per unit time. That is, the zoom function is fixed so that the imaging range does not deviate from the scheduled imaging area H.

An image signal (image data) read out from the image sensor 16 at the time of shooting a moving image or a still image is temporarily stored in a memory (SDRAM (Synchronous Dynamic Random Access Memory)) 48 via the image input controller 22, or The data is taken into the AF processing unit 42, the AE detection unit 44, and the like.

The CPU 40 performs overall control of each unit of the camera 10 based on the operation of the operation unit 38. During live view image shooting (display) and moving image shooting (recording), the AF processing unit 42 always performs an AF operation. The AE operation by the AE detection unit 44 is performed.

When the shooting mode is the still image shooting mode, if the shutter button 2 is first pressed (half-pressed), the CPU 40 performs the above-described AF control again, and if the shutter button 2 is fully pressed, The brightness of the subject (shooting Ev value) is calculated, and based on the calculated shooting Ev value, the F value of the aperture 14 and the exposure time (shutter speed) by the mechanical shutter 15 are determined according to the program diagram, and still image shooting ( (Exposure control).

On the other hand, when the shooting mode is the moving image shooting mode, if the shutter button 2 is fully pressed, the CPU 40 starts shooting and recording (recording) of the moving image. At the time of moving image shooting, the mechanical shutter 15 is opened, image data is continuously read from the image sensor 16 (for example, frame rates of 30 frames / second, 60 frames / second), and phase difference AF is continuously performed. The brightness of the subject is calculated, and the shutter drive unit 33 controls the shutter speed (charge accumulation time by the rolling shutter) and / or the aperture 14 by the aperture drive unit 34.

The CPU 40 moves the zoom lens forward and backward in the optical axis direction via the lens driving unit 36 in accordance with the zoom command from the zoom button 5 to change the focal length.

A ROM (Read Only Memory) 47 stores a camera control program, defect information of the image sensor 16, various parameters and tables used for image processing, and the like. It can be substituted by EEPROM (Electrically-Erasable-Programmable-Read-Only Memory).

Sensor 63 detects posture information of camera 10. For example, the posture information of the camera 10 may be detected by a camera shake sensor. Further, the sensor 63 may detect direction information of the camera 10 or height information from the reference plane. Specific examples include a GPS (Global Positioning System), a gyro sensor, a pressure sensor, and the like.

The image processing unit 24 reads out unprocessed image data (RAW data) acquired through the image input controller 22 when a moving image or a still image is captured and temporarily stored in the memory 48. The image processing unit 24 performs offset processing, pixel interpolation processing (interpolation processing of phase difference detection pixels, scratched pixels, etc.), white balance correction, gain control processing including sensitivity correction, gamma correction processing on the read RAW data, Synchronization processing (also referred to as “demosaic processing”), luminance and color difference signal generation processing, contour enhancement processing, color correction, and the like are performed.

The image data processed by the image processing unit 24 and processed as a live view image is input to a VRAM (Video RAM Random Access memory) 50.

The VRAM 50 includes an A area and a B area for recording image data each representing an image for one frame. In the VRAM 50, image data representing an image for one frame is rewritten alternately in the A area and the B area. Of the A area and B area of the VRAM 50, the written image data is read from an area other than the area where the image data is rewritten.

The image data read from the VRAM 50 is encoded by the video encoder 28 and output to the liquid crystal monitor 30 provided on the back of the camera. As a result, a live view image showing the subject image is displayed on the liquid crystal monitor 30.

The image data processed by the image processing unit 24 and processed as a still image or moving image for recording (luminance data (Y) and color difference data (Cb), (Cr)) is stored in the memory 48 again. Remembered.

The compression / decompression processing unit 26 compresses the luminance data (Y) and the color difference data (Cb), (Cr) processed by the image processing unit 24 and stored in the memory 48 when recording a still image or a moving image. Apply. In the case of a still image, it is compressed in, for example, JPEG (Joint Photographic coding Experts Group) format, and in the case of a moving image, it is compressed in, for example, H.264 format. The compressed image data compressed by the compression / decompression processing unit 26 is recorded on the memory card 54 via the media controller 52.

Also, the compression / decompression processing unit 26 performs decompression processing on the compressed image data obtained from the memory card 54 via the media controller 52 in the playback mode. The media controller 52 performs recording and reading of compressed image data with respect to the memory card 54.

<Functional configuration of image processing unit>
FIG. 5 is a block diagram illustrating a functional configuration example of the image processing unit 24.

The image processing unit 24 includes a first captured image acquisition unit 101, a second captured image acquisition unit 103, an imaging range detection unit 105, a start position detection unit 107, a first reproduction image generation unit 109, and a second reproduction. An image generation unit 111 is provided. Each process performed by the image processing unit 24 will be described below.

[First captured image]
The first captured image acquisition unit 101 captures a first captured image (hereinafter, referred to as an A image) 121 (captured at a wider angle than the planned imaging region H of the captured image obtained by performing exposure while moving the imaging range) ( FIG. 7) is acquired. The A image 121 is picked up at a preparatory stage in which long exposure-moving imaging is performed, and is picked up at a wider angle than the scheduled imaging region H.

FIG. 6 is a diagram illustrating an example of imaging of the A image 121. FIG. 6 is an image of the A image 121 when the long exposure-moving imaging described with reference to FIG. 1 is performed. The A image 121 is imaged by the camera 10 at a wide angle so as to include the imaging scheduled area H.

FIG. 7 is a diagram illustrating the A image 121 captured in the imaging mode illustrated in FIG. The A image 121 contains the entire tree 125 within the angle of view, and naturally, the trunk 126 that is the imaging-scheduled area H is also included within the angle of view. The A image 121 shows a scheduled imaging area H, and imaging ranges (127A, 127B, 127C) for each unit time. The imaging range will be described in detail later.

[Second captured image]
The second captured image acquisition unit 103 acquires a second captured image (hereinafter referred to as a B image) 131 (FIG. 8) in which a part of the planned imaging region H immediately before the start of exposure is captured. The B image 131 is a captured image that is captured immediately before the start of long exposure-moving imaging main exposure.

FIG. 8 is a diagram showing the B image 131. The B image 131 is an image that is captured immediately before the start of long exposure, and a part of the trunk 126 is enlarged in a state where the angle of view is narrower than that of the A image 121 and zoomed. As the B image 131, for example, a still image is cut out from a live view image (confirmation image) immediately before the main exposure is performed to obtain a B image 131.

[Imaging range detection]
The imaging range detection unit 105 detects an imaging range per unit time based on the imaging range of the B image 131 based on the detection information of the sensor 63 from the state where exposure is started. For example, the imaging range detection unit 105 detects the imaging range of the B image 131 in the A image 121, and uses detection method 1 for detecting the imaging range for each unit time in the A image 121 using the detected imaging range as the exposure start position. To do. Further, for example, the imaging range detection unit 105 uses the detection method 2 for detecting the imaging range for each unit time from the detection information of the sensor 63 when the B image 131 and the A image 121 are captured. The unit time means a frame rate (fps: frames per second) for displaying a first reproduction image or a second reproduction image, which will be described later.

≪Detection method 1≫
The imaging range detection unit 105 detects an imaging range for each unit time based on the start position detected by the start position detection unit 107.

The start position detector 107 detects the exposure start position in the A image 121 by comparing the A image 121 and the B image 131. For example, the start position detection unit 107 detects the position of the B image 131 in the A image 121 by pattern matching between the A image 121 and the B image 131. Then, the imaging range detection unit 105 acquires the start position detection result of the start position detection unit 107. The imaging range detection unit 105 detects the imaging range for each unit time based on the detection information of the sensor 63 with reference to the acquired start position.

≪Detection method 2≫
The imaging range detection unit 105 detects the start position based on the posture information of the camera 10 when the A image 121 and the B image 131 are captured, and detects the imaging range for each unit time.

The sensor 63 detects the posture information of the camera 10 when the A image 121 and the B image 131 are acquired. For example, when the sensor 63 is a camera shake sensor, the imaging range detection unit 105 detects the imaging range by comparing detection information of the camera shake sensor when the A image 121 and the B image 131 are captured. The imaging range detection unit 105 detects an exposure start position, and detects an imaging range for each subsequent unit time based on the detected exposure start position.

Note that the imaging range detection unit 105 may detect the moving direction of the imaging range for each unit time of exposure based on the detection information of the sensor 63. The imaging range detection unit 105 can accurately detect the imaging range by acquiring the moving direction of the imaging range.

In the A image 121 of FIG. 7 described above, the imaging range detected by the imaging range detection unit 105 is shown. The imaging range 127A, the imaging range 127B, and the imaging range 127C are imaging ranges that sequentially move every unit time. By setting the exposure start position as the imaging range 127A and tilting the camera 10, the imaging range sequentially moves to 127B and 127C (see FIG. 1). Note that, in FIG. 7, three imaging ranges are shown in a simplified manner, but there are actually a large number of imaging ranges in the imaging scheduled region H.

[First reproduction image]
The first reproduction image generation unit 109 generates a first reproduction image (FIG. 9) corresponding to the imaging range for each unit time from the A image 121 or the A image 121 and the B image 131. The first reproduction image generation unit 109 generates a first reproduction image using image information of the imaging range for each unit time in the first reproduction image. Further, the first reproduced image generation unit 109 may generate a first reproduced image based on the B image 131 captured immediately before the exposure is started. In particular, when the change in the posture of the camera 10 is small immediately after the start of exposure, the B image 131 and the area corresponding to the imaging range immediately after the start in the A image 121 have many overlapping portions. Therefore, the first reproduction image generation unit 109 may generate a first reproduction image from the B image 131 and the A image 121.

FIG. 9 is a diagram showing an example of the first reproduced image. The first reproduction image 133A, the first reproduction image 133B, and the first reproduction image 133C corresponding to the imaging range 127A, the imaging range 127B, and the imaging range 127C described in FIG. 7 are shown. Specifically, the imaging range 127A is set as the exposure start position, and a first reproduction image 133A corresponding thereto is shown. The first reproduction image 133B corresponding to the imaging range 127B after the next unit time has elapsed is shown. A first reproduction image 133C corresponding to the imaging range 127C after the next unit time has elapsed is shown. That is, the first reproduced image 133 represents charge information accumulated in the image sensor 16 for each unit time. Note that the first reproduced image is represented by reference numeral 133, and the first reproduced image at each unit time is represented by

reference numerals

133A, 133B, and 133C.

[Second reproduction image]
The second reproduction image generation unit 111 generates a second reproduction image by accumulating the first reproduction image 133. Specifically, the second reproduction image generation unit 111 generates a second reproduction image by superimposing the first reproduction images 133 obtained for each unit time obtained from the start of exposure to the present time.

FIG. 10 is a diagram showing an example of the second reproduced image. It should be noted that a portion where blur occurs in the second reproduction image 135 is indicated by a dotted line.

The second reproduction image 135 is obtained by superimposing the first reproduction image 133A, the first reproduction image 133B, and the first reproduction image 133C shown in FIG. Since the first reproduction images (133A, 133B, 133C) on which different images are copied every unit time are overlapped, the second reproduction image 135 is blurred. Specifically, in the second reproduced image 135, blur is generated in the outline portion of the trunk 126, and blur is also generated in the outer skin pattern 145 of the trunk 126.

Since the second reproduction image 135 obtained in this way is generated by overlapping the first reproduction image 133 for each unit time, it represents the charge information accumulated in the image sensor 16 from the start of exposure to the present time. ing.

FIG. 11 is a diagram showing another example of the second reproduction image 135. In the case of FIG. 11, the movement of the camera 10 has moved in an unintended direction, which is a failure example from the viewpoint that an image intended by the user cannot be obtained.

The second reproduction image 135 is an image of the trunk 126 of the tree 125 as in FIG. The case shown in FIG. 11 is a case where the posture of the camera 10 is blurred when moving the imaging range. In this case, based on the detection information of the sensor 63 of the camera 10, the imaging range and the first reproduced image 133 also reflect the influence of the shake of the posture of the camera 10. Accordingly, the influence of camera shake is also expressed in the second reproduction image 135 generated by integrating the first reproduction image 133.

As described above, when the shake of the posture of the camera 10 during the exposure can be expressed by the second reproduction image 135, the user confirms the second reproduction image 135, thereby capturing the image before the exposure is completed. Can be canceled and retaken. In addition, a new image can be created by using the shake of the posture of the camera 10 during exposure.

[Imaging method]
Next, the imaging method (imaging process) of the camera 10 will be described. FIG. 12 is a flowchart showing a flow of imaging by the camera 10.

The captured A image 121 is captured by the camera 10 at a wide angle including the region (expected imaging region H) where imaging is performed with long exposure (step S10), and the first captured image acquisition unit 101 acquires the A image 121. (First captured image acquisition step). Next, the camera 10 captures a live view image and stores and saves it until long-time exposure starts (step S11).

Next, the camera 10 starts main exposure for moving imaging (step S12). Then, the second captured image acquisition unit 103 acquires the B image 131 captured immediately before the start of the main exposure from the stored live view image (step S13: second captured image acquisition step).

After that, the start position detection unit 107 detects the position (exposure start position) of the B image 131 in the A image 121 (step S14: detection step). Then, the start position detection unit 107 determines whether or not the position of the B image 131 can be detected in the A image 121 (step S15). If the start position detection unit 107 cannot detect the position of the B image 131 in the A image 121, the display control unit 61 displays an error message on the liquid crystal monitor 30 (step S23).

When the position of the B image 131 can be detected in the A image 121, the start position detection unit 107 determines whether the acquired exposure time is equal to or less than the threshold (step S16). ). When it is determined that the exposure time acquired by the exposure time control unit 65 is equal to or less than the threshold value, the display control unit 61 ends the display without causing the liquid crystal monitor 30 to display the second reproduction image 135. On the other hand, when it is determined that the exposure time acquired by the exposure time control unit 65 is larger than the threshold value, the display control unit 61 performs the following process to display the second reproduced image 135 on the liquid crystal monitor 30.

The imaging range detection unit 105 acquires the current posture information (turning or the like) of the camera 10 and relative position coordinates (Δx, Δy, Δz) (step S17). The relative position coordinates are coordinates of the center of the imaging range and are relative coordinates in the A image 121. The relative position coordinates can be calculated based on the imaging range, and the coordinates (Δx, Δy, Δz) are detected by the detection information from the sensor 63 with the center of the imaging range (the center of the B image) at the exposure start position as the origin. It is possible to obtain. Thereafter, the imaging range detection unit 105 detects the current imaging range from the A image 121, the B image 131, and the relative position coordinates (imaging range detection step), and the first reproduction image generation unit 109 detects the first reproduction image 133. (Step S18: First reproduction image generation step).

Then, the imaging range detection unit 105 determines whether or not the imaging range is outside the range of the A image 121 (step S19). When the imaging range detection unit 105 determines that the imaging range is outside the range of the A image 121, the first reproduction image generation unit 109 displays the first reproduction image in which a portion outside the range is displayed as a zebra pattern. 133 is generated (step S24).

FIG. 13 is a diagram showing a first reproduced image 133 having a zebra pattern. Although the tree trunk 126 is shown in the first reproduced image 133, the angle of view has passed beyond the left side as viewed in the figure, so that it exceeds the range of the A image 121. Therefore, the zebra pattern 143 notifies the user that the portion that has exceeded the range of the A image 121 is out of the range, and the zebra pattern 143 is an example of the notification image. Further, the notification to the user may be performed not only by the notification image but also by the notification character, or may be performed by the notification image and the notification character.

Returning to FIG. 12, when the imaging range detected by the imaging range detection unit 105 is within the range of the A image 121, the second reproduction image generation unit 111 has been integrated into the imaging element 16 until the present time. An image is generated by integrating the first reproduction image 133 (second reproduction image generation step), and the display control unit 61 displays the generated second reproduction image 135 on the liquid crystal monitor 30 (step S20). : Display control step).

Next, the exposure time control unit 65 determines whether or not the current time is within the acquired exposure time (step S21). When the current time is within the exposure time, the imaging range detection unit 105 acquires the current posture information and relative position coordinate value of the camera 10 and detects the current imaging range. The flow indicated by the alternate long and short dash line F is performed, for example, every normal frame rate (50 to 60 fps). Thereafter, when the exposure time ends, the display control unit 61 ends the display of the reproduced image on the liquid crystal monitor 30 (step S22).

In the above-described embodiment, the hardware structure of a processing unit (processing unit) that executes various processes is the following various processors. For various processors, the circuit configuration can be changed after manufacturing a CPU (Central Processing Unit) or FPGA (Field Programmable Gate Array) that is a general-purpose processor that functions as various processing units by executing software (programs). Includes dedicated logic circuits such as programmable logic devices (Programmable Logic Devices: PLDs) and ASICs (Application Specific Specific Integrated Circuits) that have specially designed circuit configurations to execute specific processing. It is.

One processing unit may be configured by one of these various processors, or may be configured by two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of CPUs and FPGAs). May be. Further, the plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or server, one processor is configured with a combination of one or more CPUs and software. There is a form in which the processor functions as a plurality of processing units. Second, as represented by a system-on-chip (SoC), a form of using a processor that realizes the functions of the entire system including a plurality of processing units with a single IC (integrated circuit) chip. is there. As described above, various processing units are configured using one or more of the various processors as a hardware structure.

Further, the hardware structure of these various processors is more specifically an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

The above-described configurations and functions can be appropriately realized by arbitrary hardware, software, or a combination of both. For example, for a program that causes a computer to execute the above-described processing steps (processing procedure), a computer-readable recording medium (non-transitory recording medium) that records such a program, or a computer that can install such a program However, the present invention can be applied.

As described above, in the present invention, the first reproduction image generation unit 109 causes the first reproduction image 133 corresponding to the imaging range per unit time to be generated from the A image 121 or the A image 121 or the B image 131. The second reproduction image generation unit 111 generates the second reproduction image 135 that is being exposed by integrating the first reproduction image 133. Thereby, the user can be notified by reproducing the information accumulated in the image sensor 16 during exposure.

<Others>
[Camera movement type]
In the above description, the camera 10 is moved in the tilt direction to move the imaging range. However, the operation of the camera 10 that moves the imaging range is not limited to this.

FIG. 14 is a diagram illustrating an example of the operation mode of the camera 10 when moving the imaging range. As shown in the figure, the imaging area H may be imaged by moving the imaging range by changing the height while maintaining the posture of the camera 10 as it is. The camera 10 moves vertically upward with respect to the ground from t = 0 when the main exposure is started to t = Δt, and moves the imaging range to perform imaging of the scheduled imaging area H.

[Display switching between first reproduction image and second reproduction image]
In the above description, the form in which the second reproduced image 135 is displayed on the liquid crystal monitor 30 of the camera 10 has been described, but the present invention is not limited to this. For example, the first reproduction image 133 and the second reproduction image may be switched and displayed on the liquid crystal monitor 30.

FIG. 15 is a diagram for explaining switching of the display of the liquid crystal monitor 30. In the display form indicated by reference numeral 140, the first reproduced image 133 is displayed on the liquid crystal monitor 30. In the display form indicated by reference numeral 141, the second reproduced image 135 is displayed on the liquid crystal monitor 30. For example, the user may switch whether to display the first reproduction image 133 or the second reproduction image 135 on the liquid crystal monitor 30 using the operation unit 38 according to necessary information.

Thus, by controlling the display on the liquid crystal monitor 30, the user can switch between the currently captured image and the accumulated image for confirmation.

[Exposure time]
In the above description, the case where the exposure time is set in advance has been described, but the present invention is not limited to this. For example, the present invention is also applied to long-time exposure in which exposure is completed when the user presses the shutter button 2 of the camera 10 and the user releases the press.

The examples of the present invention have been described above, but the present invention is not limited to the above-described embodiments, and it goes without saying that various modifications can be made without departing from the spirit of the present invention.

1: Flash 2: Shutter button 3: Power / Mode switch 4: Mode dial 5: Zoom button 6: Cross button 7: MENU / OK button 8: Playback button 9: BACK button 10: Camera 12: Shooting lens 14: Aperture 15 : Mechanical shutter 16: Image sensor 22: Image input controller 24: Image processing unit 26: Compression / decompression processing unit 28: Video encoder 30: Liquid crystal monitor 32: Sensor driving unit 33: Shutter driving unit 34: Aperture driving unit 36: Lens driving unit 38: Operation unit 40: CPU
42: AF processing unit 44: AE detection unit 47: ROM
48: Memory 50: URAM
52: Media controller 54: Memory card 61: Display control unit 63: Sensor 65: Exposure time control unit 67: Zoom control unit 101: First captured image acquisition unit 103: Second captured image acquisition unit 105: Imaging range detection Unit 107: start position detection unit 109: first reproduction image generation unit 111: second reproduction image generation unit 121: first captured image 125: tree 126: tree trunk 127A: imaging range 127B: imaging range 127C: Imaging range 131: second captured image 133: first reproduced image 133A: first reproduced image 133B: first reproduced image 133C: first reproduced image 135: second reproduced image 145: tree trunk Pattern H: Imaging scheduled area Steps S10-S24: Camera operation process

Claims

A camera capable of performing exposure while moving an imaging range and acquiring a captured image,
A display unit;
A first captured image acquisition unit that acquires a first captured image captured at a wider angle than an imaging planned region of the captured image;
A second captured image acquisition unit that acquires a second captured image obtained by capturing a part of the planned imaging region immediately before starting the exposure;
A sensor for detecting posture information of the camera;
An imaging range detection unit that detects an imaging range per unit time based on the imaging range of the second captured image based on detection information of the sensor from the state of starting the exposure;
A first reproduction image generating unit that generates the first reproduction image corresponding to the imaging range for each unit time from the first imaging image or the first imaging image and the second imaging image;
A second reproduction image generation unit for generating the second reproduction image during the exposure obtained by integrating the first reproduction image;
A display control unit for displaying the second reproduced image on the display unit;
With a camera.
A start position detecting unit that detects a start position of the exposure in the first captured image by comparing the first captured image and the second captured image;
The camera according to claim 1, wherein the imaging range detection unit detects an imaging range for each unit time based on the start position.
The sensor detects posture information of the camera when acquiring the first captured image and the second captured image,
The imaging range detection unit detects the start position of the exposure based on the detection information when acquiring the first captured image and the second captured image, and determines the imaging range for each unit time. The camera according to claim 2 to be detected.
The camera according to any one of claims 1 to 3, wherein the sensor further detects at least one of orientation information of the camera and height information from a reference plane.
The camera according to claim 4, wherein the imaging range detection unit detects a moving direction of the imaging range for each unit time based on the detection information of the sensor.
The camera according to any one of claims 1 to 5, further comprising an exposure time control unit that acquires an exposure time set for the exposure and controls the exposure.
The camera according to claim 6, wherein when the exposure time is longer than a threshold, the display control unit displays the second reproduced image on the display unit.
The first reproduction image generation unit notifies the user of a notification image and a notification character when the imaging range for each unit time detected by the imaging range detection unit is in a range other than the first captured image. The camera according to claim 1, wherein the first reproduction image including at least one of the first reproduction image is generated.
The camera according to any one of claims 1 to 8, wherein the display control unit switches the first reproduction image and the second reproduction image to be displayed on the display unit.
10. The camera according to claim 1, wherein the sensor is a camera shake sensor.
The camera according to any one of claims 1 to 10, further comprising a zoom control unit that fixes a zoom function according to the first captured image and an imaging range for each unit time.
An imaging method capable of acquiring a captured image by performing exposure while moving an imaging range,
A first captured image acquisition step of acquiring a first captured image captured at a wider angle than the imaging planned region of the captured image;
A second captured image acquisition step of acquiring a second captured image in which a part of the planned imaging region immediately before the start of exposure is captured;
A detection step of detecting camera posture information by a sensor;
An imaging range detection step of detecting an imaging range per unit time based on an imaging range of the second captured image based on detection information of the sensor from a state of starting the exposure;
A first reproduction image generation step of generating a first reproduction image corresponding to the imaging range for each unit time from the first imaging image or the first imaging image and the second imaging image;
A second reproduction image generation step of generating a second reproduction image during the exposure obtained by integrating the first reproduction image;
A display control step of displaying the second reproduction image on the display unit;
An imaging method including:
A program for causing a computer to execute an imaging process capable of acquiring a captured image by performing exposure while moving an imaging range,
A first captured image acquisition step of acquiring a first captured image captured at a wider angle than the imaging planned region of the captured image;
A second captured image acquisition step of acquiring a second captured image in which a part of the planned imaging region immediately before the start of exposure is captured;
A detection step of detecting camera posture information by a sensor;
An imaging range detection step of detecting an imaging range per unit time based on an imaging range of the second captured image based on detection information of the sensor from a state of starting the exposure;
A first reproduction image generation step of generating a first reproduction image corresponding to the imaging range for each unit time from the first imaging image or the first imaging image and the second imaging image;
A second reproduction image generation step of generating a second reproduction image during the exposure obtained by integrating the first reproduction image;
A display control step of displaying the second reproduction image on the display unit;
A program for causing a computer to execute an imaging process including:
A non-transitory and computer-readable recording medium, when a command stored in the recording medium is read by a computer,
An imaging function capable of acquiring a captured image by performing exposure while moving the imaging range,
A second captured image acquisition step of acquiring a second captured image in which a part of the planned imaging region immediately before the start of exposure is captured;
A detection step of detecting camera posture information by a sensor;
An imaging range detection step of detecting an imaging range per unit time based on an imaging range of the second captured image based on detection information of the sensor from a state of starting the exposure;
A first reproduction image generation step of generating a first reproduction image corresponding to the imaging range for each unit time from the first imaging image or the first imaging image and the second imaging image;
A second reproduction image generation step of generating a second reproduction image during the exposure obtained by integrating the first reproduction image;
A display control step of displaying the second reproduction image on the display unit;
Recording medium that causes a computer to realize an imaging function including