WO2020158069A1 - Imaging device, imaging method, and program - Google Patents

Abstract

The purpose of the present invention is to provide an imaging device, an imaging method, and a program that make it possible to take a countermeasure against a flicker in accordance with exposure time of frames constituting a motion image. It is also the purpose of the present invention to provide an imaging device, an imaging method, and a program that make it possible to record a moving image, captured by taking a countermeasure against a flicker, at a desired frame period. The imaging device according to a first aspect of the present invention is provided with a plurality of moving image modes corresponding to the exposure time of the frames constituting the motion image. In a second moving image mode in which the exposure time of the frames constituting the motion image is set so as to be shorter than that of a first moving image mode, a first frame period is set to be an integral multiple of a flicker period in accordance with a flicker detection result, and this makes it possible to reduce a difference in brightness and a difference in color tone among frames. On the other hand, in the first moving image mode in which the exposure time of the frames is longer than that of the second moving image mode, the first frame period is set regardless of the flicker period.

Description

Imaging device, imaging method, and program

The present invention relates to an image capturing apparatus, an image capturing method, and a program that capture a moving image, and particularly to an image capturing apparatus, an image capturing method, and a program that take measures against flicker when capturing a moving image.

When a moving image is shot under an artificial light source, there are cases in which light and dark fluctuations and variations called "flicker" may occur, and techniques for taking measures against this are known. For example, Patent Document 1 describes that a plurality of means for detecting the flicker light amount change characteristic are provided, and processing for suppressing the influence of flicker is performed based on the detection results of these detection means. Further, Patent Document 2 describes that when the subject is under a flicker light source, the frame rate is set to a flicker cycle or an integral multiple thereof.

Japanese Patent Laid-Open No. 2018-133826 Japanese Patent Laid-Open No. 2012-60370

For flicker, it is preferable to take measures according to the exposure time of the frames that make up the moving image, and the moving image is recorded at an appropriate frame period (corresponding to the frame rate) according to the user's wishes. However, such points have not been taken into consideration in Patent Documents 1 and 2. Further, Patent Documents 1 and 2 are for taking a flicker-free still image by taking a flicker countermeasure in a live view of a moving image, and do not consider continuously taking and recording a moving image while taking a flicker countermeasure. It was

The present invention has been made in view of such circumstances, and an object of the present invention is to provide an imaging device, an imaging method, and a program capable of taking measures against flicker according to the exposure time of a frame forming a moving image. Another object of the present invention is to provide an image pickup apparatus, an image pickup method, and a program capable of recording a moving image picked up by taking measures against flicker at a desired frame period.

In order to achieve the above-mentioned object, an imaging device according to a first aspect of the present invention includes a cycle detection unit that detects a flicker cycle of a light source in a moving image, an imaging unit that captures the moving image at a first frame period, An image acquisition unit that acquires first moving image data indicating a captured moving image, an image processing unit that processes the first moving image data to generate second moving image data having a second frame period, and a recording device that records the second moving image data. An image pickup apparatus comprising: a recording unit for recording in a recording medium; and a control unit for controlling a first frame period and a second frame period, wherein the control unit has a first moving image mode and a moving image as a moving image capturing mode. And a second moving image mode in which the exposure time of the frames constituting the first moving image mode is set shorter than the first moving image mode, and the control unit sets the first frame period to the flicker period according to the detection result in the second moving image mode. It is set to an integral multiple, and in the first moving image mode, the first frame period is set regardless of the flicker period.

When a moving image is shot under the influence of flicker, a difference in brightness and tint due to the influence of flicker is likely to occur between frames of a moving image, and the influence becomes large when the exposure time is short. Therefore, the image pickup apparatus according to the first aspect is provided with a plurality of moving image modes (first moving image mode and second moving image mode) according to the exposure time of the frames forming the moving image. Then, in the second moving image mode (in which the influence of flicker is large) in which the exposure time of the frames forming the moving image is set shorter than that of the first moving image mode, the first frame period is set to an integral multiple of the flicker period according to the detection result of the flicker. Is set, and thereby, it is possible to reduce the difference in lightness and darkness and the difference in tint between frames. On the other hand, in the first moving image mode in which the frame exposure time is longer than that in the second moving image mode and the influence of flicker is small, the first frame period is set regardless of the flicker period.

As described above, according to the first aspect, it is possible to take countermeasures against flicker according to the exposure time of the frames forming the moving image.

In addition, in the first aspect, it is possible to record a moving image captured with flicker countermeasures at a desired frame period (second frame period). The first frame period and the second frame period may be the same or different. The recording device may be included in the imaging device or may be independent of the imaging device.

In the first aspect, the flicker can be detected before the moving image recording is started, but may be performed during the moving image recording in addition to before the moving image recording is started.

In the imaging device according to the second aspect, in the first aspect, the control unit does not change the second frame period in the second moving image mode depending on whether flicker of the light source is detected or not. In the second mode, since the second frame period (frame period of recording) is not changed in the second moving image mode regardless of whether flicker is detected or not, moving image data is recorded at a period different from a preset frame period. Never be done.

In the imaging device according to the third aspect, in the first aspect, the control unit sets the second frame period to a period longer than the first frame period when flicker of the light source is detected in the second moving image mode. In the third aspect, when the first frame period is shortened (to an integral multiple of the flicker period) as a countermeasure against flicker, the second frame period does not have to be shortened accordingly.

The image pickup apparatus according to a fourth aspect is the image pickup apparatus according to any one of the first to third aspects, wherein the image processing unit configures a plurality of first moving image data when the second frame period is longer than the first frame period. A part of the frames is selected to generate the second moving image data of the second frame period. In the fourth aspect, the moving image captured in the first frame period is adjusted to the second frame period (recording frame period) by “thinning” some frames.

The image pickup apparatus according to a fifth aspect is the image pickup device according to any one of the first to fourth aspects, wherein the image processing unit configures a plurality of first moving image data when the second frame period is longer than the first frame period. The frames that compose the second moving image data are generated by synthesizing the frames. The image processing unit may perform weighting when combining the frames.

In the imaging device according to a sixth aspect, in the first aspect, the control unit sets the second frame period to a period shorter than the first frame period when flicker of the light source is detected in the second moving image mode. The control unit may perform such processing according to a user's instruction.

In the imaging device according to a seventh aspect, in any one of the first, second, and sixth aspects, the image processing unit, when the first frame period is longer than the second frame period, outputs the first moving image data. Interpolation frames are inserted into the constituent frames to generate second moving image data of the second frame period. The interpolation frame may be a frame in which no subject is reflected, or may be the frame just before the interpolation frame is inserted. Further, it may be a frame generated by interpolation or the like from the frame before and/or after the insertion timing. When inserting an interpolation frame, it is desirable to record information such as a frame cycle of image pickup and recording, an insertion timing of an interpolation frame, etc. in a header of a moving image file.

An image pickup apparatus according to an eighth aspect is the image pickup apparatus according to the seventh aspect, further comprising a display control unit that causes the display device to display the second moving image data as an image, and the display control unit causes the second moving image data to be displayed excluding an interpolation frame. .. In the eighth aspect, since the interpolation frame is not displayed, the user can efficiently check the captured moving image.

In the imaging apparatus according to a ninth aspect, in any one of the first and third to eighth aspects, the control unit sets the second frame period to the flicker period according to the detection result in the second moving image mode. Set to an integer multiple. In the ninth aspect, the control unit may equalize the first frame period and the second frame period.

An image pickup apparatus according to a tenth aspect is the image pickup apparatus according to any one of the first to ninth aspects, further including a phase detection unit that detects flicker phase information of the light source in the first moving image data, and the control unit includes the second moving image. In the mode, the exposure timing of the frames forming the first moving image data is controlled based on the flicker phase information. According to the tenth aspect, it is possible to capture a moving image with a desired brightness, as well as to reduce variations in brightness and tint between frames.

An image pickup apparatus according to an eleventh aspect is the image pickup apparatus according to any one of the first to tenth aspects, wherein in the second moving image mode, an autofocus speed, an automatic exposure follow-up speed, and a white balance of the first moving image mode are set. At least one of the following speeds is set to a high speed. The eleventh aspect defines shooting conditions different from those in the first moving image mode in the second moving image mode.

The image pickup apparatus according to the twelfth aspect is the image pickup apparatus according to any one of the first to eleventh aspects, further including a still image extraction unit that extracts a frame forming the second moving image data as a still image. If flicker countermeasures are not taken, problems such as "a frame at the optimum timing for extracting as a still image has poor image quality" may occur. However, according to the twelfth aspect, the above-mentioned flicker countermeasures are taken. The user can select a frame to be extracted without worrying about deterioration of image quality.

In order to achieve the above-mentioned object, an imaging apparatus according to a thirteenth aspect of the present invention includes a cycle detection unit that detects a flicker cycle of a light source in a moving image, an imaging unit that captures the moving image in a first frame cycle, An image acquisition unit that acquires first moving image data indicating a captured moving image, an image processing unit that processes the first moving image data to generate second moving image data having a second frame period, and a recording device that records the second moving image data. An image pickup apparatus comprising: a recording unit for recording the first frame period and a control unit for controlling the first frame period and the second frame period, wherein the control unit sets the first frame period to an integral multiple of the flicker period according to the detection result. And the second frame period is set to a previously designated period. According to the thirteenth aspect, by setting the first frame period to be an integral multiple of the flicker period, it is possible to reduce variations in brightness and tint between frames. In addition, according to the thirteenth aspect, it is possible to record a moving image captured with flicker countermeasures at a frame cycle (second frame cycle) designated in advance. The control unit can set the second frame period according to the user's designation operation or without the user's designation operation, whereby the moving image captured by the flicker countermeasure is recorded at the desired frame period. be able to.

In the thirteenth aspect and each of the following aspects, the control unit sets and sets the second frame period (frame period of recording) according to the user's designation operation or without the user's designation operation. The first frame period (frame period of shooting) may be set according to the two-frame period.

In the thirteenth aspect of the imaging apparatus according to the fourteenth aspect, the control unit sets the second frame period to a period longer than the first frame period when the flicker of the light source is detected. The fourteenth aspect defines one aspect of the setting method of the first frame period.

The imaging device according to a fifteenth aspect is the image pickup device according to the thirteenth or fourteenth aspect, wherein the image processing unit selects one of the plurality of frames forming the first moving image data when the second frame period is longer than the first frame period. A frame is selected to generate second moving image data. In the fifteenth aspect, the moving image captured in the first frame period can be matched with the second frame period (recording frame period) by “thinning” some frames.

The imaging device according to a sixteenth aspect is the imaging device according to any one of the thirteenth to fifteenth aspects, wherein the image processing unit configures a plurality of first moving image data when the second frame period is longer than the first frame period. The frames that compose the second moving image data are generated by synthesizing the frames. Frames may be weighted during synthesis.

In the imaging device according to a seventeenth aspect, in the thirteenth aspect, the control unit sets the second frame period to a period shorter than the first frame period when the flicker of the light source is detected. The seventeenth aspect defines one aspect of the setting method of the first frame period.

In the imaging device according to an eighteenth aspect, in the thirteenth or seventeenth aspect, the image processing unit inserts an interpolation frame into a frame forming the first moving image data when the first frame period is longer than the second frame period. Then, a frame forming the second moving image data is generated. The interpolation frame may be a frame in which no subject is reflected, or may be the frame just before the interpolation frame is inserted. Further, it may be a frame generated by interpolating frames before and after the insertion timing. When inserting the interpolation frame, it is desirable to record information such as a frame period (frame rate) of image pickup and recording, an insertion timing of the interpolation frame and the like in the header of the moving image file.

An imaging apparatus according to a nineteenth aspect further comprises, in the eighteenth aspect, a display control unit that causes the display device to display the second moving image data as an image, and the display control unit causes the second moving image data to be displayed excluding an interpolation frame. .. In the nineteenth aspect, since the interpolation frame is not displayed, the user can efficiently check the recorded moving image.

An image pickup apparatus according to a twentieth aspect is the imaging apparatus according to any one of the thirteenth to nineteenth aspects, further including a phase detection unit that detects flicker phase information of the light source in the first moving image data, and the control unit sets the first moving image. The exposure timing of the frames forming the data is controlled based on the flicker phase information. According to the twentieth aspect, it is possible to not only reduce the variations in brightness and tint between frames but also shoot all the frames with a desired brightness.

The image pickup apparatus according to a twenty-first aspect is the image pickup apparatus according to any one of the thirteenth to twentieth aspects, wherein the control unit sets the first moving image mode as an image pickup mode of the moving image and the exposure time of the frames forming the moving image. The second moving image mode is set to be shorter than the first moving image mode. In the second moving image mode, the control unit sets the first frame period to an integral multiple of the flicker period according to the detection result. In the twenty-first aspect, similarly to the first aspect, the flicker detection result is obtained in the second moving image mode (the influence of the flicker is large) in which the exposure time of the frames forming the moving image is set shorter than that in the first moving image mode. Accordingly, the first frame period is set to be an integral multiple of the flicker period, so that flicker countermeasures can be taken according to the exposure time of the frame.

According to a twenty-first aspect, in the image pickup apparatus according to the twenty-first aspect, in the second moving image mode, at least one of an auto-focus speed, an automatic exposure follow-up speed, and a white balance follow-up speed is faster than the first moving-picture mode. Is set to. The twenty-second aspect defines shooting conditions different from those in the first moving image mode in the second moving image mode.

The image pickup apparatus according to the twenty-third aspect is the image pickup apparatus according to any one of the thirteenth to twenty-second aspects, further including a still image extraction unit that extracts a frame forming the second moving image data as a still image. If there is no flicker countermeasure, a problem such as "a frame with an optimal timing, but the image quality is poor" may occur. According to the twenty-third aspect, the user is concerned about the image quality deterioration due to the above-mentioned flicker countermeasure. The frame to be extracted can be selected without doing so.

In order to achieve the above-mentioned object, the imaging method according to the twenty-fourth aspect of the present invention provides a moving image capturing mode in which a first moving image mode and an exposure time of a frame forming a moving image are shorter than those in the first moving image mode. An image pickup method of an image pickup apparatus having a second moving image mode set, comprising: a period detection step of detecting a flicker period of a light source in a moving image; an image pickup step of capturing a moving image at a first frame period; An image acquisition step of acquiring first moving image data indicating a moving image, an image processing step of processing the first moving image data to generate second moving image data of a second frame period, and recording the second moving image data in a recording device. And a control step of controlling the first frame period and the second frame period. In the control step, in the second moving image mode, the first frame period is an integral multiple of the flicker period according to the detection result. And the first frame period is set regardless of the flicker period in the first moving image mode. According to the twenty-fourth aspect, as in the first aspect, it is possible to take flicker countermeasures according to the exposure time of the frames forming the moving image. Further, it is possible to record an imaged moving image with the flicker countermeasure in the second frame period. The twenty-fourth aspect may further include the same configuration as the second to twelfth aspects.

In order to achieve the above-mentioned object, an imaging method according to a twenty-fifth aspect of the present invention includes a detection step of detecting a flicker period of a light source in a moving image, an imaging step of capturing a moving image in a first frame period, and an imaging An image acquisition step of acquiring first moving image data indicating the moving image, an image processing step of processing the first moving image data to generate second moving image data of a second frame period, and the second moving image data in a recording device. A recording step for recording, and a control step for controlling the first frame period and the second frame period, wherein in the control step, the first frame period is set to an integral multiple of the flicker period according to the detection result, The second frame period is set to a previously designated period. According to the twenty-fifth aspect, similarly to the thirteenth aspect, it is possible to record a moving image captured with a countermeasure against flicker at a desired frame period. The 25th mode may further include the same configuration as the 14th to 23rd modes.

In order to achieve the above-mentioned object, a program according to a twenty-sixth aspect of the present invention causes an imaging device to execute the imaging method according to the twenty-fourth aspect. Accordingly, as in the first aspect and the twenty-fourth aspect, it is possible to take measures against flicker according to the exposure time of the frames forming the moving image. The program according to the twenty-sixth aspect may further include the same configuration as the second to twelfth aspects. Further, a non-transitory recording medium in which the computer readable code of the program according to these aspects is recorded can also be mentioned as one aspect of the present invention.

To achieve the above object, a program according to a twenty-seventh aspect of the present invention causes an imaging device to execute the imaging method according to the twenty-fifth aspect. With this, similarly to the thirteenth aspect and the twenty-fifth aspect, it is possible to record a moving image captured with a countermeasure against flicker at a desired frame period. The program according to the twenty-seventh aspect may further include the same configuration as the fourteenth to twenty-third aspects. In addition, a non-transitory recording medium in which a computer-readable code of the program of these aspects is recorded can also be cited as one aspect of the present invention.

As described above, according to the image pickup apparatus, the image pickup method, and the program of the present invention, it is possible to take measures against flicker according to the exposure time of the frames forming the moving image. Further, according to the image pickup apparatus, the image pickup method, and the program of the present invention, it is possible to record a moving image picked up with flicker countermeasures at a desired frame period.

FIG. 1 is a diagram showing a configuration of a camera according to the first embodiment. FIG. 2 is a diagram showing a functional configuration of the image processing apparatus. FIG. 3 is a flowchart showing the processing of the imaging method. FIG. 4 is a diagram showing the relationship between the imaging frame period and the influence of flicker. FIG. 5 is a diagram showing how the imaging timing is controlled. FIG. 6 is another flowchart showing the processing of the imaging method. FIG. 7 is a diagram showing how frames are thinned out. FIG. 8 is a diagram showing how a frame is inserted. FIG. 9 is a flowchart showing a reproduction process when a frame is cut out. FIG. 10 is another flowchart showing the processing of the imaging method. FIG. 11 is another flowchart showing the processing of the imaging method. FIG. 12 is still another flowchart showing the process of the imaging method. FIG. 13 is another flowchart showing the processing of the imaging method. FIG. 14 is an external view of a smartphone according to the second embodiment. FIG. 15 is a block diagram showing the configuration of the smartphone according to the second embodiment.

Hereinafter, embodiments for implementing an imaging device, an imaging method, and a program according to the present invention will be described in detail with reference to the accompanying drawings.

<First Embodiment>

<Overall structure of imaging device>

FIG. 1 is a diagram showing a configuration of a camera 10 (imaging device) according to the first embodiment. The camera 10 is composed of an interchangeable lens 100 (image pickup unit, image pickup device) and an image pickup device main body 200 (image pickup device), and forms a subject image (optical image) on the image pickup element 210 by a photographing lens including a zoom lens 110 described later. .. The interchangeable lens 100 and the imaging device main body 200 can be attached and detached via a mount (not shown).

<Structure of interchangeable lens>

The interchangeable lens 100 includes a zoom lens 110, a focus lens 120, a diaphragm 130, and a lens driving unit 140. The lens driving unit 140 drives the zoom lens 110 and the focus lens 120 forward and backward according to a command from the image processing device 240 (lens drive control unit 240H in FIG. 2) to perform zoom (optical zoom) adjustment and focus adjustment. The zoom adjustment and the focus adjustment may be performed according to a command from the image processing device 240, or may be performed according to a zoom operation or a focus operation (a zoom ring (not shown), a rotation of the focus ring, or the like) performed by the user. Good. Further, the lens driving unit 140 controls the diaphragm 130 according to a command from the image processing device 240 to adjust the exposure. On the other hand, information such as the positions of the zoom lens 110 and the focus lens 120 and the opening degree of the diaphragm 130 is input to the image processing device 240. The interchangeable lens 100 has an optical axis L.

<Structure of imaging device body>

The image pickup apparatus main body 200 includes an image pickup element 210 (image pickup section), an AFE 220 (AFE: Analog FrontEnd, image pickup section), an A/D converter 230 (A/D: Analog to Digital, image pickup section), and an image processing apparatus 240 ( (Image acquisition unit, flicker detection unit, image processing unit, recording unit, control unit, display control unit, still image extraction unit, lens drive control unit). Further, the imaging device main body 200 includes an operation unit 250, a recording device 260 (recording device), a monitor 270 (display device), and an attitude sensor 280. The imaging device main body 200 may have a shutter (not shown) for blocking the light transmitted through the imaging element 210. The shutter may be a mechanical shutter or an electronic shutter. In the case of an electronic shutter, the exposure time (shutter speed) can be adjusted by controlling the charge accumulation period of the image sensor 210 by the image processing device 240. Further, the image pickup unit 201 includes an interchangeable lens 100, an image pickup element 210, an AFE 220, and an A/D converter 230, and is controlled by an image acquisition unit 240A (image acquisition unit).

The image sensor 210 has a light receiving surface in which a large number of light receiving elements are arranged in a matrix. Then, the subject light that has passed through the zoom lens 110, the focus lens 120, and the diaphragm 130 is imaged on the light receiving surface of the image pickup element 210, and converted into an electric signal by each light receiving element. An R (red), G (green), or B (blue) color filter is provided on the light-receiving surface of the image sensor 210, and a color image of a subject can be acquired based on signals of each color. As the image sensor 210, various photoelectric conversion elements such as a CMOS (Complementary Metal-Oxide Semiconductor) and a CCD (Charge-Coupled Device) can be used. The AFE 220 performs noise removal and amplification of the analog image signal output from the image sensor 210, and the A/D converter 230 converts the captured analog image signal into a digital image signal having a gradation width.

<Structure of image processing device>

FIG. 2 is a diagram showing a functional configuration of the image processing device 240. The image processing device 240 includes an image acquisition unit 240A (image acquisition unit), a flicker detection unit 240B (cycle detection unit, phase detection unit), an image processing unit 240C (image processing unit), a recording unit 240D (recording unit), and a control unit. 240E (control unit), display control unit 240F (display control unit), still image extraction unit 240G (still image extraction unit), and lens drive control unit 240H (lens drive control unit). The image processing device 240 captures (captures) a moving image based on the digital image signal input from the A/D converter 230, generates a file, generates a still image file, processes a plurality of frames included in the moving image, and generates a still image. Performs processing such as extraction. Details of the processing using the image processing device 240 will be described later.

The functions of the image processing device 240 can be realized by using various processors. The various processors include, for example, a CPU (Central Processing Unit) that is a general-purpose processor that executes software (programs) to realize various functions. The various processors described above include a GPU (Graphics Processing Unit) that is a processor specialized for image processing. In addition, the various processors described above also include a programmable logic device (PLD) that is a processor whose circuit configuration can be changed after manufacturing such as FPGA (Field Programmable Gate Array). Further, a dedicated electric circuit, which is a processor having a circuit configuration specifically designed to execute a specific process such as an ASIC (Application Specific Integrated Circuit), is also included in the various processors described above.

When the above-described processor or electric circuit executes software (program), the processor (computer) readable code of the software to be executed is stored in a non-transitory recording medium such as a ROM (Read Only Memory). The processor then references the software. The software stored in the non-temporary recording medium includes a program for executing the imaging method according to the present invention (a program for operating the imaging device). The code may be recorded in a non-transitory recording medium such as various magneto-optical recording devices and semiconductor memories instead of the ROM. When performing processing using software, for example, a RAM (Random Access Memory) is used as a temporary storage area, and for example, data stored in an EEPROM (Electronically Erasable and Programmable Read Only Memory) (not shown) is also referred to. it can.

The image processing device 240 includes a ROM 242 (non-temporary recording medium) in addition to the above-mentioned units. The ROM 242 is read by a computer (for example, various processors forming the image processing device 240) of a program (including a program for executing the image pickup method according to the present invention) necessary for image pickup, recording, display, and the like. Possible codes are recorded.

<Operation part>

The operation unit 250 has a release button (not shown), operation buttons (for example, a cross button, a Quick button, an OK button, etc.), a dial, a switch, and the like, and the user can set an image capturing mode, a moving image capturing instruction, and a recording frame period. Various operations such as a still image extraction instruction can be performed. The control unit 240E can also accept these user instructions. The monitor 270 (touch panel type) may be used as the operation unit 250.

<Memory>

The recording device 260 (recording device) is composed of various types of magneto-optical recording media, non-transitory recording media such as semiconductor memories, and its control circuit, and stores moving images, still images, still images extracted from the moving images, and the like. The recording medium may be of a type that can be attached to and detached from the image pickup apparatus main body 200. The captured image (moving image, still image) may be transmitted to and stored in an external recording medium (other than the recording device 260) or a recording device by, for example, wired and/or wireless communication.

<Monitor and viewfinder>

The monitor 270 (display device) is composed of a touch panel type liquid crystal display panel and can display a moving image, a still image, a still image extracting frame, and the like. The monitor 270 can be arranged on the back surface side, the top surface side, or the like of the imaging device body 200. Further, the camera 10 may include a finder.

<Attitude sensor>

The camera 10 includes an attitude sensor 280 (for example, an acceleration sensor that measures acceleration in three axis directions), and can detect the attitude (direction with respect to gravity) of the camera 10 based on the measurement result of the attitude sensor 280. The display control unit 240F can change the orientation of the image displayed on the monitor 270 according to the attitude (portrait or landscape) of the camera 10.

<Shooting mode>

The camera 10 can set either a still image mode, a normal moving image mode (first moving image mode), or a still image extracting moving image mode (second moving image mode) as a shooting mode. The still image mode and the normal moving image mode are similar to those of a normal digital camera. On the other hand, the still image extraction moving image mode is a mode in which a still image can be extracted from a moving image file, and a moving image with different image capturing conditions from the normal moving image mode (a moving image with an image capturing condition that emphasizes still image extraction rather than viewing the moving image itself ) Is imaged. Specifically, in the still image extracting moving image mode, at least one of autofocus speed (focus lens drive speed for moving to a target focusing distance), automatic exposure following speed, and white balance following speed. Is set faster than the normal video mode. In the moving image mode for still image extraction, it is preferable to set the resolution to the highest value (for example, 4,000×2,000 pixels) that can be set by the camera 10 and set the color tone on the premise of still image extraction. In the still image extracting moving image mode, it is preferable that the upper limit of the ISO speed (ISO: International Organization for Standardization) is set higher than that of the normal moving image mode to increase the upper limit of the shutter speed to reduce the blurring of the subject.

The shutter speed is set to a value corresponding to the frame rate of the moving image to be recorded in the normal moving image mode (1/30 seconds when the frame rate is 30 frames/second). On the other hand, in the moving image mode for still image extraction, the speed is set higher than the frame interval (for example, less than 1/30 seconds). In the normal moving image mode, the shutter speed is set to a value corresponding to the frame rate of the moving image so that a smooth moving image is reproduced, but in this case, a moving subject may be blurred. For this reason, the shutter speed in the moving image mode for still image extraction is set higher than that in the normal moving image mode. That is, in the still image extracting moving image mode, the exposure time of the frames forming the moving image is shorter than that in the normal moving image mode. In the still image extracting moving image mode, the shutter speed (exposure time) can be set in the P mode (program mode) or the S mode (shutter speed priority), as in the still image mode. In the P mode (program mode), both the shutter speed and the aperture value are automatically controlled by the imaging device. On the other hand, in the S mode, the shutter speed is set in advance by the user, and the aperture value is automatically controlled by the imaging device.

Details of the frame period (frame rate) setting in the moving image mode for still image extraction will be described later.

According to the above-described moving image mode for still image extraction, the frames forming the moving image can be extracted as a still image afterwards. Therefore, the user can easily take a picture of an event (natural phenomenon, accident, happening, etc.) that is not known when it will occur, a picture of a subject whose state changes over time or a momentary state of a moving subject, etc. be able to. At this time, as will be described later in detail, the still image can be extracted not only at the timing at which the recording of the still image is instructed but also at other timings. Therefore, the user can acquire a still image at a desired timing. Moreover, a high-quality still image can be extracted by setting shooting conditions (shutter speed, resolution, frame period, etc.) suitable for still image extraction.

In addition to such subsequent extraction of still images, in the moving image mode for still image extraction and the normal moving image mode, still images can be captured (recorded) during moving image shooting.

<Processing in movie mode>

An image pickup method in the camera 10 having the above-described configuration will be described. FIG. 3 is a flowchart showing processing in the moving image mode (normal moving image mode and still image extracting moving image mode).

<Video mode settings>

The control unit 240E sets the camera 10 to the normal moving image mode (first moving image mode) or the still image extracting moving image mode (second moving image mode) based on the user's operation via the operation unit 250 (step S100: Mode setting step).

<Flicker detection>

The flicker detector 240B (cycle detector) detects the flicker cycle (step S110: cycle detection step). The detection can be performed, for example, by the method described in JP-A-2018-133826. Specifically, the flicker detection unit 240B can detect flicker from a plurality of frames (may be frames forming a live view image) acquired at time intervals. For example, when considering 50 Hz (power supply cycle: 20 msec) and 60 Hz (power supply cycle: about 16.7 msec) as power supply frequencies, the imaging unit 201 images a plurality of images at a frame cycle different from any power supply frequency (50 Hz, 60 Hz). Then, the image acquisition unit 240A acquires the captured image. Then, the cycle of the waveform obtained by autocorrelating the acquired image by the flicker detection unit 240B can be set as the flicker cycle. The flicker detector 240B can also detect the flicker phase from this waveform. The flicker detector 240B may detect flicker by another known method. The same applies when other power supply frequencies (other than 50 Hz and 60 Hz) are considered. Further, the flicker detection unit 240B may detect as a flicker a light source in which the light and dark changes of the light source are periodic and the light and dark difference is constant or more.

In addition, in the first embodiment and other modes (including modifications) described later, the case where flicker is detected before the start of recording a moving image is described. However, the light source of the flicker is detected after the start of recording the moving image. May be turned on or off. In consideration of such a situation, the flicker detection unit 240B detects flicker not only before the recording of the moving image but also after the recording is started, and resets the imaging frame period and/or the recording frame period based on the detection result. You may. Further, when a change in the shooting scene is detected by scene recognition or the like, flicker may be detected, and the imaging frame period and/or the recording frame period may be reset based on the detection result.

Although the waveform of the AC power supply is sinusoidal, its frequency is twice the power supply frequency when the change in brightness due to flicker (see FIG. 4) is considered. Therefore, the flicker cycle is 1/2 of the power supply cycle. Specifically, the flicker frequency when the power supply frequency is 50 Hz (cycle=1/50 sec) is 100 Hz (cycle=1/100 sec), and the flicker frequency when the power supply frequency is 60 Hz (cycle=1/60 sec) is 120 Hz (cycle=cycle). (1/120 sec).

<Setting of imaging frame cycle in moving image mode for still image extraction>

When flicker is detected (there is flicker) (Yes in step S120), the control unit 240E determines whether or not the imaging mode is the second moving image mode (still image extracting moving image mode) (step S130). In the case of the second moving image mode (Yes in step S130), the control unit 240E sets the imaging frame period according to the detection result of step S110. Specifically, for example, when the flicker cycle is 1/100 sec, the process proceeds from step S140 to step S150. The control unit 240E sets the imaging frame period (first frame period) to 1/100 sec (imaging frame rate=100 fps) (step S150: control step), and sets the recording frame period (second frame period) to 1/100 sec ( The recording frame rate=100 fps) can be set (step S160: control step). In this case, since the imaging frame cycle and the flicker cycle are equal, the imaging frame cycle is 1 times the flicker cycle (an aspect of one multiple). Further, in this case, the recording frame period and the imaging frame period are equal. Note that “fps (frame per second)” indicates the number of frames per second.

On the other hand, if the flicker cycle is 1/120 sec, the process proceeds from step S140 to step S170. The control unit 240E sets the imaging frame period (first frame period) to 1/120 sec (imaging frame rate=120 fps) (step S170: control step), and sets the recording frame period (second frame period) to 1/120 sec ( The recording frame rate can be set to 120 fps (step S180). In this case as well, since the imaging frame cycle and the flicker cycle are equal, the imaging frame cycle is 1 times the flicker cycle (an aspect of one multiple). In addition, the recording frame period and the imaging frame period are equal.

In the still image extracting moving image mode (second moving image mode), the control unit 240E does not have to change the recording frame period (second frame period) depending on whether flicker is detected or not (imaging). The frame period should be an integral multiple of the flicker period).

<Setting of frame cycle in normal video mode>

When the image capturing mode is set to the normal moving image mode (first moving image mode) (No in step S130), the control unit 240E sets the image capturing frame period (first frame period) regardless of the flicker period (step). S190: control step). In this case, the control unit 240E may set the imaging frame cycle based on a user operation, or may set the imaging frame cycle without depending on a user operation. The control unit 240E can also set the recording frame period (second frame period) to a value equal to the imaging frame period even in the normal moving image mode (step S200: control step). For example, the control unit 240E can set the imaging frame period to 1/100 sec and the recording frame period to 1/100 sec regardless of the flicker period. It should be noted that the control unit 240E can set the imaging frame period and the recording frame period as in the case of the normal moving image mode even when no flicker is detected (No in step S120).

<Recording of moving images>

The image acquisition unit 240A and the control unit 240E determine whether or not a moving image recording start instruction has been given (step S210: imaging step, image acquisition step). For example, when a release button (not shown) of the operation unit 250 is pressed down, it can be determined that “a moving image recording start instruction has been issued”. If the determination is affirmative, the process proceeds to step S220, where the image capturing unit 201 captures a moving image at the image capturing frame period (first frame period) set by the above-described processing, and the image acquiring unit 240A captures the captured moving image data (captured). First moving image data indicating a moving image) is acquired (step S220: imaging step, image acquisition step). Then, the image processing unit 240C processes the first moving image data to generate recording moving image data (second moving image data) having a recording frame period (second frame period) (step S222: image processing step). When the image capturing frame period and the recording frame period are different, the image processing unit 240C may insert or thin out the frames to generate the moving image data for recording (see an aspect described later). In addition, the image processing unit 240C may perform image processing other than frame insertion and thinning.

The recording unit 240D (recording unit) causes the recording device 260 (recording device) to record the generated second moving image data (step S224: recording step). When the image capturing frame period is 1/100 sec and the recording frame period is 1/100 sec (when the image capturing frame period and the recording frame period are the same) as in the example described above, the recording unit 240D records all the frames of the captured moving image data. Good. The recording unit 240D may record the recording moving image data in an external recording device (such as a recording device connected to the camera 10). It is preferable that the image processing unit 240C and the recording unit 240D record information such as an imaging frame period (imaging frame rate) and a recording frame period (recording frame rate) in a tag of a moving image for recording.

The image acquisition unit 240A, the image processing unit 240C, and the recording unit 240D keep the moving image recording end instruction (No in step S230) and the remaining capacity of the recording medium (recording device 260 or the like) sufficient ( (Yes in step S240), the processing from steps S220 to S224 is continued. When there is a video recording end instruction (Yes in step S230) and when the remaining capacity of the recording medium is not sufficient (No in step S240), the image acquisition unit 240A, the image processing unit 240C, and the recording unit 240D end the process. Then, the moving image file being recorded is closed (step S250).

FIG. 4 conceptually shows the relationship between the change in brightness due to the flicker light source and the image capturing timing of each frame of a moving image when the image capturing frame period (first frame period) is set to an integral multiple of the flicker period by the above-described processing. FIG. In the example shown in FIG. 4, the imaging frame period (=|t2-t1|) is set to one time (one mode of integer multiple) of the flicker period (T0). Times t1 and t2 indicate image pickup timing. As a result, a moving image can be captured in a state where the influence of the brightness of the flicker light source is constant (brightness L0), and the difference in brightness and tint between frames can be reduced.

In the moving image mode for still image extraction (second moving image mode), the exposure time of each frame forming a moving image is set shorter than that in the normal moving image mode (first moving image mode), and the influence of flicker is large, but as described above. By setting the imaging frame cycle to be an integral multiple of the flicker cycle, it is possible to take measures against flicker according to the exposure time of the frames forming the moving image.

<Setting of imaging timing>

As described above, according to one embodiment of the present invention, the difference in lightness and darkness and the difference in tint between frames can be reduced, but the brightness of the flicker light source may be different depending on the timing of capturing a moving image (the exposure timing of each frame). Each frame will be imaged in the dark. On the other hand, by setting the timing of capturing a moving image as described below, the moving image can be captured with the brightness of the flicker light source being in a desired state. The change in the brightness of the flicker light source (flicker phase information) can be detected together with the detection of the flicker cycle in step S110 (phase detecting step). That is, it is possible to grasp the relationship between the image capturing timing and the brightness from the waveforms obtained by acquiring the images a plurality of times under the control of the image obtaining unit 240A and subjecting the images to the autocorrelation processing by the flicker detecting unit 240B (phase detecting unit). Then, in the moving image mode for still image extraction (second moving image mode), the control unit 240E sets the exposure timing of the frames forming the captured moving image data (first moving image data indicating the captured moving image) based on the flicker phase information. Control. Under this control, the imaging unit 201 captures a moving image, and the image acquisition unit 240A acquires captured moving image data (step S220).

FIG. 5 shows how the brightness of the flicker light source becomes maximum (brightness L1) by imaging at the imaging timing (time t3, t4, t5, t6, t7) set by the above-described control. .. Although FIG. 5 shows a state in which imaging is performed at the exposure timing when the brightness of the flicker light source is maximized, the imaging timing (for example, t1 and t2) is set to the timing when the brightness of the flicker light source is maximized as in the example of FIG. The brightness of the moving image may be adjusted by shifting the brightness. Such setting of the imaging timing can be similarly performed in other modes and modifications thereof described below.

<Another Mode of Imaging and Recording Frame Period Setting>

(Other mode 1)

FIG. 6 is a flowchart showing another aspect 1 of the image pickup and recording frame period (first and second frame periods) setting. The same steps as those in the flowchart of FIG. 3 are designated by the same step numbers, and detailed description thereof will be omitted.

In the flowchart of FIG. 6, the control unit 240E initializes the recording frame period (second frame period) (step S102: control step). The control unit 240E has a recording frame period (1/24 sec, 1/25 sec, 1/30 sec, 1/50 sec) corresponding to a general value (24 fps, 25 fps, 30 fps, 50 fps, 60 fps, etc.) as a frame rate of a moving image. 1/60) may be set. In addition, the control unit 240E may perform the initial setting based on a user's instruction via the operation unit 250, or may perform the initial setting regardless of the user's instruction. In the following description, it is assumed that the recording frame period is set to 1/30 sec (frame rate is 30 fps) in step S102, but the same processing can be performed when another value is set to the recording frame period. ..

In another aspect 1, the recording frame cycle is left as the initial setting, and as described later in detail, by performing frame thinning, insertion, etc. on the captured moving image data (first moving image data) acquired in the capturing frame period, The recording moving image data (second moving image data) having the set frame period is generated. The imaging frame period and the recording frame period may be different. Since the recording frame period remains the default setting, steps S160 and S180 in FIG. 6 may be skipped.

When the flicker period detected in step S110 is 1/100 sec, the control unit 240E can set a value such as 1/100 sec or 1/50 sec as the imaging frame period (first frame period) (step S150: Control step). These imaging frame cycles are 1 times and 2 times (one aspect of integer multiple) the flicker cycle, respectively. Similarly, when the flicker period detected in step S110 is 1/120 sec, the control unit 240E can set values such as 1/120 sec and 1/60 sec as the imaging frame period (first frame period) ( Step S170: control step). These imaging frame cycles are 1 times and 2 times (one aspect of integer multiple) the flicker cycle, respectively. The control unit 240E may set an imaging frame cycle that is an integral multiple (three times or more) of the flicker cycle.

When the recording frame period (second frame period) is longer than the imaging frame period (first frame period) as in the above-described example, the image processing unit 240C causes the plurality of frames that form the captured moving image data (first moving image data). A part of the frames is selected to generate recording moving image data (second moving image data) having a recording frame period (second frame period) (step S222: image processing step). FIG. 7 is a diagram conceptually showing a state of such processing (frame thinning) (frames are sequentially acquired along time t). In the example of FIG. 7, the image processing unit 240C is a part of a plurality of frames (frame 506; part (a) of FIG. 16) that forms the captured moving image data (first moving image data) of the capturing frame period (T1). Frame of the recording frame period (second frame period; T2 (>T1 in FIG. 7)) constituting the recording moving image data (second moving image data) (frame 508; (b) in the same figure). Part) is being generated.

Note that the image processing unit 240C combines a plurality of frames forming the captured moving image data (first moving image data) when the recording frame period (second frame period) is longer than the captured frame period (first frame period) ( Weighted addition or the like) may be performed to generate frames that form the recording moving image data (second moving image data) (step S222: image processing step).

The control unit 240E may set the recording frame period (second frame period) to a period shorter than the imaging frame period (first frame period). For example, when the recording frame period is initially set to 1/30 sec, the control unit 240E may set the imaging frame period to 1/25 sec (when the flicker period is 1/100 sec) (step S150: control step). ). In this case, the imaging frame cycle is four times the flicker cycle (an aspect of one multiple).

When the recording frame period is shorter than the image capturing frame period (when the recording frame rate is higher than the image capturing frame rate) as in the above-described example, the image processing unit 240C inserts the interpolation frame into the frame forming the first moving image data. It is possible to generate the second moving image data of the second frame period. FIG. 8 is a diagram conceptually showing a state of such an interpolation frame. When the image capturing frame period is T1 and the recording frame period is T2 (<T1), the image processing unit 240C causes the image capturing moving image data (first moving image data) to be a frame (frame 500; part (a) of FIG. 8). Interpolation frame 502A (frame shown in gray) between the two) to generate a frame 502 having a recording frame period of T2 (see part (b) in the same figure). The frame 502 is an example in which the ratio of the imaging frame to the interpolation frame is 1:1, but the image processing unit 240C can determine the ratio according to the imaging frame period and the recording frame period. Note that examples of the interpolation frame include a black frame (dummy frame) in which the subject is not reflected, a frame in which the immediately preceding frame is used as it is, and a frame generated by interpolating frames before and/or after the insertion timing. However, the present invention is not limited to these.

When inserting the interpolation frame, the image processing unit 240C and the recording unit 240D add the shooting frame period (shooting frame rate), the recording frame period (recording frame rate), and the inserted interpolation frame to the tag of the moving image for recording. It is preferable to record information such as position. As described below, this information can be used when extracting a still image.

<Video playback and still image clipping>

FIG. 9 is a flowchart showing a state of moving image reproduction and still image cutout (still image extraction). The display control unit 240F determines whether or not the dummy frame information (information indicating the position or the like of the frame inserted by interpolation) is recorded in the tag or the like of the moving image data recorded in step S224 (step S300). When the dummy frame information is recorded (Yes in step S300), the display control unit 240F sets the information (reading: step S310) and starts reproduction of the recording moving image data (second moving image data) (step). S320). The display control unit 240F may display the image on the monitor 270 (display device), or may display the image on another display device connected to the camera 10. Note that, here, a case will be described in which a moving image is played back frame by frame (displayed one frame at a time in response to a user operation).

When the reproduction of the moving image is started, the display control unit 240F initializes the value of the counter k to 1 (step S330) and determines whether the kth frame is a dummy frame (step S340). If the k-th frame is not a dummy frame (No in step S340), the display control unit 240F displays the frame (step S360). The still image extracting unit 240G (still image extracting unit) determines whether or not there is a cutout instruction for the displayed frame (an instruction by the user to extract a frame forming the recording moving image data (second moving image data) as a still image). Is determined (step S370). If there is a cutout instruction (Yes in step S370), the still image extraction unit 240G (still image extraction unit) cuts out the instructed frame as a still image (step S380) and proceeds to step S390. Since the moving image file is stored in a moving image format such as MPEG format, the still image extraction unit 240G converts the data of the designated frame into a still image format (JPEG format or the like). The still image extraction unit 240G may perform image processing other than format conversion.

If there is no cutout instruction (No in step S370), the display control unit 240F determines whether or not there is a reproduction end instruction by the user, and if there is an instruction (Yes in step S390), the process of the flowchart of FIG. 9 is performed. finish. When there is no playback end instruction (No in step S390), the display control unit 240F determines whether or not the user has given a frame advance instruction. When there is an instruction (Yes in step S400), the display control unit 240F increments the value of the counter k by 1 (step S350), clips the value of k in the range of 1 to N (step S430), and returns to step S340. .. When there is no frame advance instruction (No in step S400) and there is a frame return instruction (instruction to display the previous frame) (Yes in step S410), the display control unit 240F decrements the value of the counter k by 1 (step S420). The value of k is clipped in the range of 1 to N (step S430), and the process returns to step S340. When there is no frame return instruction (No in step S410), the display control unit 240F clips the value of k in the range of 1 to N (step S430) and returns to step S340.

As described above, since the dummy frame is not displayed when the moving image data in which the dummy frame (one mode of the interpolation frame) is recorded is reproduced for the still image extraction, the user can efficiently perform the still image extraction work. it can.

(Modification 1 of other aspect 1)

In the flowchart of FIG. 6, the recording frame period initially set in step S102 may be changed (reset) according to the imaging frame period. In this case, the control unit 240E changes (resets) the recording frame period (second frame period) in steps S160 and S180. At this time, it is possible to set a frame cycle that conforms to the standard of a general moving image with a frame cycle that is longer than the imaging frame cycle (frame rate that is lower than the imaging frame rate). Specifically, for example, when the initially set recording frame period is 1/30 sec and the flicker period is 1/100 sec, the control unit 240E sets the imaging frame period (first frame period) to 1/100 sec (step S150). ). In addition, the control unit 240E sets the recording frame period (second frame period) to 1/60 sec, 1/50 sec, 1/25 sec, and 1/24 sec corresponding to the imaging frame period (frame rates are 60 fps, 50 fps, 25 fps, respectively). 24 fps) (step S160).

Further, for example, when the initially set recording frame period is 1/30 sec and the flicker period is 1/120 sec, the control unit 240E sets the imaging frame period to 1/120 sec (step S170). Then, the control unit 240E sets the recording frame period to 1/100 sec, 1/60 sec, 1/50 sec, 1/25 sec, and 1/24 sec corresponding to the imaging frame period (frame rates are 100 fps, 60 fps, 50 fps, 25 fps, respectively. 24 fps) (step S180).

When changing (resetting) the recording frame period, it is preferable that the display control unit 240F informs the user by displaying information on the monitor 270. In addition, the control unit 240E may allow the user to select the recording frame period to be changed (reset). Further, in the modified example 1, the recording frame cycle is longer than the imaging frame cycle. Therefore, as described above with reference to FIG. 7, the image processing unit 240C thins out the frames (selects some of the frames forming the captured moving image data according to the recording frame period to generate recording moving image data. ) And frames are combined to generate moving image data for recording (step S222).

(Modification 2 of other aspect 1)

In Modification 1 described above, the recording frame period is changed (reset) to be longer than the imaging frame period, but a recording frame period shorter than the imaging frame period (recording frame rate higher than the imaging frame rate) is set. Good. Also in this case, the recording frame period may be set in accordance with the standard of a general moving image. Specifically, for example, when the initially set recording frame period is 1/30 sec and the flicker period is 1/100 sec, the control unit 240E sets the imaging frame period (first frame period) to 1/100 sec (step S150). ), the recording frame period (second frame period) is set to 1/120 sec (frame rate is 120 fps) (step S160). The frame period can be set in the same manner even when the flicker period is 1/120 sec.

Also in the second modification, it is preferable that the display control unit 240F display the information on the monitor 270 to notify the user of the change (reset) of the recording frame period. In addition, the control unit 240E may allow the user to select the recording frame period to be changed. In the second modification, the recording frame cycle is shorter than the imaging frame cycle. Therefore, as described above with reference to FIG. 8, the image processing unit 240C inserts an interpolation frame to generate recording moving image data (step S222).

(Other mode 2)

FIG. 10 is a flowchart showing another aspect 2 of setting the imaging frame period and the recording frame period. In another aspect 2, the image is picked up at an image pickup frame period (first frame period) that is an integral multiple of the flicker period regardless of the image pickup mode (first moving image mode, second moving image mode), and a predetermined period is set as the recording frame period It is recorded as (second frame period). In FIG. 10, the same steps as those in the flowcharts of FIGS. 3 and 6 are designated by the same step numbers, and detailed description thereof will be omitted.

When the flicker cycle detected in step S110 is 1/100 sec, the control unit 240E sets the imaging frame cycle to 1/100 sec (=flicker cycle) or an integral multiple (2 times, 3 times, etc.) thereof (step S150). .. The recording frame period may remain the initial setting value (step S102). On the other hand, when the flicker cycle is 1/120 sec, the control unit 240E sets the imaging frame cycle to 1/120 sec (=flicker cycle) or an integral multiple (2 times, 3 times, etc.) thereof (step S170). The recording frame period may remain the specified value (step S102). The image processing unit 240C performs the interpolation frame insertion, the frame synthesis, and the frame thinning described above with reference to FIGS. 7 and 8 according to the relationship between the imaging frame period and the recording frame period thus set (step). S222).

If no flicker is detected (No in step S120), the control unit 240E can set the imaging frame cycle to a value equal to the recording frame set in step S102 (step S182).

(Modification 1 of other aspect 2)

FIG. 11 is a flowchart showing a modified example 1 of the other aspect 2. In Modification 1 of the other aspect 2 as well, the control unit 240E can change (reset) the recording frame period initialized in step S102 (steps S160 and S180). When changing the frame cycle, the recording frame cycle and the imaging frame cycle may be equal to each other, or one of them may be longer. Further, the specific value of the frame period can be set in the same manner as in the first embodiment, other aspect 1 and its modification. The image processing unit 240C performs the above-described interpolation frame insertion, frame combination, and frame thinning according to the relationship between the imaging frame period and the recording frame period (step S222).

(Other mode 3)

FIG. 12 is a flowchart showing another aspect 3 of setting the imaging frame period and the recording frame period. In another aspect 3, the control unit 240E sets the imaging frame period in the normal moving image mode (first moving image mode) regardless of the flicker period (as in the case of no flicker) (step S182). On the other hand, in the still image extracting moving image mode (second moving image mode), the control unit 240E sets the first frame period to an integral multiple of the flicker period according to the detection result of the flicker period. In any of the moving image modes, the control unit 240E sets the recording frame period to the previously designated frame period (step S102).

When the flicker period is 1/100 sec, the control unit 240E sets the imaging frame period to, for example, 1/100 sec, 1/50 sec, 1/25 sec, and 1/20 sec (frame rates are 100 fps, 50 fps, 25 fps, and 20 fps, respectively). be able to. In this case, the imaging frame cycle is 1 time, 2 times, 4 times, and 5 times the flicker cycle, respectively. When the flicker period is 1/120 sec, the control unit 240E sets the image capturing frame period to, for example, 1/120 sec, 1/60 sec, 1/30 sec, and 1/24 sec (frame rates are 120 fps, 60 fps, 30 fps, and 24 fps, respectively). Can be set. In this case, the imaging frame cycle is 1 time, 2 times, 4 times, and 5 times the flicker cycle, respectively. The control unit 240E may set a value that is an integral multiple of the flicker period and that conforms to a general moving image standard as the imaging frame period.

(Modification 1 of other aspect 3)

FIG. 13 is a flowchart showing Modification 1 of the other aspect 3 described above. In the first modification, the control unit 240E changes (resets) the recording frame period from the value of the initial setting (step S102) (steps S160 and S180). At this time, the control unit 240E can change the recording frame period to an integral multiple of the flicker period in accordance with the imaging frame period. The value of the imaging frame period may be the same or different. Further, when the values are different, any cycle may be long. The control unit 240E may set the image capturing frame period and/or the recording frame period according to the standard of a general moving image. The image processing unit 240C inserts interpolation frames, synthesizes frames, and thins out frames in accordance with the difference in frame period, as in the above-described aspect and modification (step S222).

Also in the above-described other aspects 1 to 3 and the modified examples thereof, it is possible to take the flicker countermeasure according to the exposure time of the frame forming the moving image as in the first embodiment. Further, it is possible to record a moving image captured by taking measures against flicker at a desired frame period.

<Second Embodiment>

Although the camera 10 which is a digital camera has been described in the first embodiment, the configuration of the imaging device is not limited to this. The other imaging device of the present invention may be, for example, a built-in or external PC camera (PC: Personal Computer), or a portable terminal device having a shooting function as described below. ..

Examples of the mobile terminal device which is an embodiment of the image pickup device of the present invention include a mobile phone, a smartphone, a PDA (Personal Digital Assistants), and a portable game machine. Hereinafter, a smartphone will be described as an example in detail with reference to the drawings.

FIG. 14 is a diagram showing an appearance of a smartphone 1 (imaging device) which is an embodiment of an imaging device of the present invention, in which (a) is a front view and (b) is a rear view. A smartphone 1 shown in FIG. 14 has a flat housing 2, and a display panel 21 (display device) as a display unit and an operation panel 22 (operation unit) as an input unit on one surface of the housing 2. Is provided with a display input unit 20. Further, the housing 2 includes a speaker 31, a microphone 32, an operation unit 40 (operation unit), camera units 41 and 42 (imaging device, imaging unit), and a strobe 43. The configuration of the housing 2 is not limited to this. For example, a configuration in which the display unit and the input unit are independent may be employed, or a configuration having a folding structure or a slide mechanism may be employed.

FIG. 15 is a block diagram showing the configuration of the smartphone 1 shown in FIG. As shown in FIG. 15, the smartphone 1 includes a wireless communication unit 11, a display input unit 20, a call unit 30, an operation unit 40, camera units 41 and 42, a flash unit 43, a storage unit 50, and an external unit. An input/output unit 60, a GPS receiving unit 70 (GPS: Global Positioning System), a motion sensor unit 80, and a power supply unit 90 are provided. The smartphone 1 also includes a main control unit 101 (image acquisition unit, flicker detection unit (cycle detection unit, phase detection unit), control unit, display control unit, still image extraction unit, lens drive control unit). In addition, as a main function of the smartphone 1, a wireless communication function for performing mobile wireless communication via a base station device and a mobile communication network is provided.

The wireless communication unit 11 performs wireless communication with the base station device accommodated in the mobile communication network according to an instruction from the main control unit 101. Using such wireless communication, various file data such as voice data and image data, electronic mail data and the like are transmitted and received, and Web data and streaming data are received.

Under the control of the main control unit 101, the display input unit 20 displays an image (still image and/or moving image), character information, and the like to visually convey the information to the user, and also to perform a user operation on the displayed information. It is a so-called touch panel for detection, and includes a display panel 21 and an operation panel 22.

In the display panel 21, an LCD (Liquid Crystal Display), an OELD (Organic Electro-Luminescence Display), or the like is used as a display device. The operation panel 22 is a device that is placed so that an image displayed on the display surface of the display panel 21 can be visually recognized, and detects one or a plurality of coordinates operated by a conductor such as a finger of a user or a pen. When such a device is operated by a user's finger or a conductor such as a pen, the operation panel 22 outputs a detection signal generated due to the operation to the main control unit 101. Next, the main control unit 101 detects the operation position (coordinates) on the display panel 21 based on the received detection signal.

As shown in FIG. 14, the display panel 21 and the operation panel 22 of the smartphone 1 exemplified as one embodiment of the image pickup apparatus of the present invention integrally configure the display input unit 20, but the operation panel The display panel 21 is arranged so as to completely cover the display panel 21. When such an arrangement is adopted, the operation panel 22 may have a function of detecting a user operation even in the area outside the display panel 21. In other words, the operation panel 22 has a detection area (hereinafter, referred to as a display area) for the overlapping portion that overlaps the display panel 21 and a detection area (hereinafter, a non-display area) for the other outer edge portion that does not overlap the display panel 21. Referred to).

The call unit 30 includes a speaker 31 and a microphone 32, converts a user's voice input through the microphone 32 into voice data that can be processed by the main control unit 101, and outputs the voice data to the main control unit 101. 11 or the audio data received by the external input/output unit 60 can be decoded and output from the speaker 31. Further, as shown in FIG. 14, for example, the speaker 31 can be mounted on the same surface as the surface on which the display input unit 20 is provided, and the microphone 32 can be mounted on the side surface of the housing 2.

The operation unit 40 is a hardware key that uses a key switch or the like, and is a device that receives an instruction from a user. For example, as shown in FIG. 14, the operation unit 40 is mounted on the side surface of the housing 2 of the smartphone 1, and is turned on when pressed by a finger or the like, and is turned off by a restoring force such as a spring when the finger is released. It is a button type switch.

The storage unit 50 (recording device) was downloaded by control program and control data of the main control unit 101, application software, address data associated with names and telephone numbers of communication partners, data of sent and received emails, and Web browsing. It stores Web data, downloaded content data, and temporarily stores streaming data and the like. Further, the storage unit 50 includes an internal storage unit 51 with a built-in smartphone and an external storage unit 52 having a detachable external memory slot. The internal storage unit 51 and the external storage unit 52 that configure the storage unit 50 are realized using known storage media.

The external input/output unit 60 serves as an interface with all external devices connected to the smartphone 1. The smartphone 1 is directly or indirectly connected to another external device via the external input/output unit 60 by communication or the like. Examples of means for communication and the like include universal serial bus (USB), IEEE 1394, and network (eg, Internet, wireless LAN). In addition, Bluetooth (registered trademark), RFID (Radio Frequency Identification), infrared communication (Infrared Data Association: IrDA) (registered trademark), and the like can be cited as means for communication and the like. Furthermore, UWB (Ultra Wide Band) (registered trademark), ZigBee (registered trademark), and the like can be cited as means for communication.

Examples of the external device connected to the smartphone 1 include a wired/wireless headset, a wired/wireless external charger, and a wired/wireless data port. In addition, a memory card (Memory card), a SIM (Subscriber Identity Module Card)/UIM (User Identity Module Card) card connected via a card socket can also be used as the external device. Also, external audio and video devices connected via audio/video I/O (Input/Output) terminals, external audio and video devices wirelessly connected, smartphones wired/wirelessly connected, and wired/wirelessly connected. External devices such as a PDA, a wired/wirelessly connected personal computer, and earphones can also be connected. The external input/output unit 60 can transmit the data transmitted from such an external device to each component inside the smartphone 1, or can transmit the data inside the smartphone 1 to the external device.

The motion sensor unit 80 includes, for example, a triaxial acceleration sensor and a tilt sensor, and detects a physical movement of the smartphone 1 according to an instruction from the main control unit 101. By detecting the physical movement of the smartphone 1, the moving direction, acceleration, and posture of the smartphone 1 are detected. The detection result is output to the main control unit 101. The power supply unit 90 supplies power stored in a battery (not shown) to each unit of the smartphone 1 according to an instruction from the main control unit 101.

The main control unit 101 includes a microprocessor, operates according to a control program and control data stored in the storage unit 50, and integrally controls each unit of the smartphone 1 including the camera unit 41. Further, the main control unit 101 includes a mobile communication control function for controlling each unit of the communication system and an application processing function for performing voice communication and data communication through the wireless communication unit 11.

Further, the main control unit 101 has an image processing function of displaying a video on the display input unit 20 based on image data (still image data or moving image data) such as received data or downloaded streaming data. The image processing function refers to a function of the main control unit 101 decoding image data, performing image processing on the decoding result, and displaying an image on the display input unit 20.

The camera units 41 and 42 are digital cameras (imaging devices) that electronically capture images using imaging devices such as CMOS and CCD. Under control of the main control unit 101, the camera units 41 and 42 convert the image data (moving image, still image) obtained by imaging into compressed image data such as MPEG or JPEG, and record it in the storage unit 50. In addition, it can be output through the external input/output unit 60 and the wireless communication unit 11. Further, the camera unit 41, under the control of the main control unit 101, performs division and combination of moving images, acquisition of high-quality still images (RAW images, etc.), frame replacement and processing, and extraction of still images from moving images. Can also In the smartphone 1 shown in FIGS. 14 and 15, one of the camera units 41 and 42 can be used for shooting, and the camera units 41 and 42 can be used simultaneously for shooting. When the camera unit 42 is used, the strobe 43 can be used.

In addition, the camera units 41 and 42 can be used for various functions of the smartphone 1. For example, the smartphone 1 can display the images acquired by the camera units 41 and 42 on the display panel 21. Further, the smartphone 1 can use the images of the camera units 41 and 42 as one of the operation inputs of the operation panel 22. When the GPS receiving unit 70 detects the position based on the positioning information from the GPS satellites ST1, ST2,..., STn, the smartphone 1 detects the position by referring to the images from the camera units 41 and 42. You can also Furthermore, the smartphone 1 refers to the images from the camera units 41 and 42 and uses the light of the camera unit 41 of the smartphone 1 without using the triaxial acceleration sensor or in combination with the triaxial acceleration sensor. It is also possible to determine the axial direction and the current usage environment. Of course, the smartphone 1 can also use the images from the camera units 41 and 42 in the application software. In addition, in the smartphone 1, the position information obtained by the GPS receiving unit 70 and the voice information obtained by the microphone 32 are added to the image data of the still image or the moving image (the main control unit or the like performs voice text conversion to obtain text information. Alternatively, the posture information acquired by the motion sensor unit 80 may be added and recorded in the storage unit 50. The smartphone 1 can also output the image data of these still images or moving images through the external input/output unit 60 or the wireless communication unit 11.

Also in the smartphone 1 having the above-described configuration, the processing of the image capturing method according to the present invention is performed similarly to the camera 10 according to the first embodiment (video image capturing, flicker cycle and phase detection, frame cycle control, recording video). Data generation and recording, reproduction display of moving image data for recording, still image extraction, etc.) can be performed. Specifically, in the smartphone 1, the camera units 41 and 42 and the main control unit 101 perform the processing (including the processing of the above-described flowchart) executed by the image processing apparatus 240 (each unit illustrated in FIG. 2) in the first embodiment. I can do it. In addition, the functions of the operation unit 250, the recording device 260, and the monitor 270 according to the first embodiment can be realized by the operation unit 40, the storage unit 50 and the operation panel 22, the display panel 21, and the operation panel 22 in the smartphone 1. it can.

As a result, also in the smartphone 1 according to the second embodiment, the same effect as that of the camera 10 according to the first embodiment (the flicker countermeasure can be performed according to the exposure time of the frame forming the moving image, and the flicker countermeasure can be taken). It is possible to obtain that a moving image captured by performing the recording can be recorded at a desired frame period.

Although the embodiments and other aspects (including modifications) of the present invention have been described above, the present invention is not limited to the above-described embodiments and other aspects, and various modifications are possible without departing from the spirit of the present invention. Deformation is possible.

1 smartphone

2 housing

10 cameras

11 Wireless communication section

20 Display input section

21 Display panel

22 Operation panel

30 call department

31 speakers

32 microphone

40 Operation part

41 camera section

42 Camera section

43 Strobe

50 memory

51 internal storage

52 external storage

60 External input/output section

70 GPS receiver

80 Motion sensor part

90 power supply

100 interchangeable lens

101 main control unit

110 zoom lens

120 focus lens

130 aperture

140 lens drive

200 Imaging device body

201 Imaging unit

210 image sensor

220 AFE

230 A/D converter

240 image processing device

240A image acquisition unit

240B Flicker detector

240C image processing unit

240D recording unit

240E control unit

240F Display control unit

240G still image extraction unit

240H Lens drive control unit

242 ROM

250 Control

260 recording device

270 monitor

280 Attitude sensor

500 frames

502 frames

502A interpolation frame

506 frames

508 frames

k counter

L optical axis

L0 brightness

L1 brightness

S100 to S430 Each step of the imaging method

ST1 GPS satellite

ST2 GPS satellite

STn GPS satellite

T0 flicker cycle

T1 imaging frame period

T2 recording frame period

t time

t1 time

t2 time

t3 time

t4 time

t5 time

t6 time

t7 time

Claims (27)

1. 
   A cycle detection unit that detects a flicker cycle of a light source in a moving image,
   
   An image capturing unit that captures a moving image having a frame period of a first frame period;
   
   An image acquisition unit that acquires first moving image data indicating the captured moving image;
   
   An image processing unit that processes the first moving image data to generate second moving image data having a frame period of a second frame period;
   
   A recording unit for recording the second moving image data in a recording device;
   
   An image pickup apparatus comprising: a control unit that controls the first frame period and the second frame period,
   
   The control unit has, as the moving image capturing mode, a first moving image mode and a second moving image mode in which an exposure time of a frame forming the moving image is set shorter than the first moving image mode,
   
   The control unit sets the first frame period to an integral multiple of the flicker period when a flicker period is detected by the period detection unit in the second moving image mode, and sets the first frame period to the flicker period in the first moving image mode. An imaging device that sets the first frame period independently of the above.
   
2. 
   The imaging device according to claim 1, wherein the control unit does not change the second frame period in the second moving image mode depending on whether a flicker period of the light source is detected or not.
   
3. 
   The imaging device according to claim 1, wherein the control unit sets the second frame period to a frame period longer than the first frame period when a flicker period of the light source is detected in the second moving image mode.
   
4. 
   When the second frame period is longer than the first frame period, the image processing unit selects a part of the plurality of frames forming the first moving image data so that the frame period is the second frame. The image pickup device according to claim 1, wherein the second moving image data having a cycle is generated.
   
5. 
   The image processing unit, when the second frame period is longer than the first frame period, synthesizes a plurality of frames forming the first moving image data to generate a frame forming the second moving image data. The imaging device according to any one of items 1 to 4.
   
6. 
   The imaging device according to claim 1, wherein the control unit sets the second frame period to a frame period shorter than the first frame period when a flicker period of the light source is detected in the second moving image mode.
   
7. 
   When the first frame period is longer than the second frame period, the image processing unit inserts an interpolation frame into a frame forming the first moving image data, and the second frame period is the second frame period. The imaging device according to claim 1, which generates moving image data.
   
8. 
   The imaging device according to claim 7, further comprising a display control unit that causes the display device to display the second moving image data as an image, wherein the display control unit displays the second moving image data excluding the interpolation frame.
   
9. 
   9. The imaging according to claim 1, wherein the control unit sets the second frame period to an integral multiple of the flicker period according to a result of the detection in the second moving image mode. apparatus.
   
10. 
    A phase detector that detects flicker phase information of the light source in the first moving image data,
    
    The image pickup apparatus according to claim 1, wherein the control unit controls the exposure timing of a frame forming the first moving image data based on the flicker phase information in the second moving image mode.
    
11. 
    11. In the second moving image mode, at least one of an autofocus speed, an automatic exposure follow-up speed, and a white balance follow-up speed is set to be higher than that in the first moving picture mode. The imaging device according to the item.
    
12. 
    The image pickup apparatus according to claim 1, further comprising a still image extraction unit that extracts a frame forming the second moving image data as a still image.
    
13. 
    A cycle detection unit that detects a flicker cycle of a light source in a moving image,
    
    An image capturing unit that captures a moving image having a frame period of a first frame period;
    
    An image acquisition unit that acquires first moving image data indicating the captured moving image;
    
    An image processing unit that processes the first moving image data to generate second moving image data having a frame period of a second frame period;
    
    A recording unit for recording the second moving image data in a recording device;
    
    An image pickup apparatus comprising: a control unit that controls the first frame period and the second frame period,
    
    An image pickup apparatus in which the control unit sets the first frame period to an integral multiple of the flicker period and the second frame period to a predesignated frame period when a flicker period is detected by the period detection unit. ..
    
14. 
    The imaging device according to claim 13, wherein the control unit sets the second frame period to a frame period longer than the first frame period when a flicker period of the light source is detected.
    
15. 
    When the second frame period is longer than the first frame period, the image processing unit selects a part of a plurality of frames forming the first moving image data to generate the second moving image data. The imaging device according to claim 13 or 14.
    
16. 
    The image processing unit, when the second frame period is longer than the first frame period, synthesizes a plurality of frames forming the first moving image data to generate a frame forming the second moving image data. Item 16. The imaging device according to any one of items 13 to 15.
    
17. 
    The imaging device according to claim 13, wherein the control unit sets the second frame period to a period shorter than the first frame period when a flicker period of the light source is detected.
    
18. 
    When the first frame period is longer than the second frame period, the image processing unit inserts an interpolation frame into a frame forming the first moving image data to generate a frame forming the second moving image data. The imaging device according to claim 13 or 17.
    
19. 
    Further comprising a display control unit for displaying the second moving image data as an image on a display device,
    
    The imaging device according to claim 18, wherein the display control unit displays the second moving image data excluding the interpolation frame.
    
20. 
    A phase detector that detects flicker phase information of the light source in the first moving image data,
    
    20. The image pickup apparatus according to claim 13, wherein the control unit controls an exposure timing of a frame forming the first moving image data based on the flicker phase information.
    
21. 
    The control unit has a first moving image mode as a moving image capturing mode, and a second moving image mode in which an exposure time of a frame forming the moving image is set shorter than the first moving image mode.
    
    The imaging device according to claim 13, wherein in the second moving image mode, the control unit sets the first frame period to an integral multiple of the flicker period according to a result of the detection.
    
22. 
    22. The imaging device according to claim 21, wherein in the second moving image mode, at least one of an autofocus speed, an automatic exposure following speed, and a white balance following speed is set to be higher than that in the first moving image mode.
    
23. 
    23. The image pickup apparatus according to claim 13, further comprising a still image extraction unit that extracts a frame that constitutes the second moving image data as a still image.
    
24. 
    A method for imaging a moving image, comprising: a first moving image mode; and a second moving image mode in which an exposure time of a frame forming the moving image is set shorter than the first moving image mode.
    
    A cycle detection step of detecting a flicker cycle of a light source in a moving image,
    
    An image capturing step of capturing a moving image having a frame period of a first frame period;
    
    An image acquisition step of acquiring first moving image data indicating the captured moving image;
    
    An image processing step of processing the first moving image data to generate second moving image data having a frame period of a second frame period;
    
    A recording step of recording the second moving image data in a recording device,
    
    A control step of controlling the first frame period and the second frame period,
    
    In the control step, in the second moving image mode, the first frame period is set to an integral multiple of the flicker period according to a result of the detection, and in the first moving image mode, the first frame period is set regardless of the flicker period. An imaging method for setting a frame period.
    
25. 
    A detection step of detecting the flicker period of the light source in the moving image,
    
    An image capturing step of capturing a moving image having a frame period of a first frame period;
    
    An image acquisition step of acquiring first moving image data indicating the captured moving image;
    
    An image processing step of processing the first moving image data to generate second moving image data having a frame period of a second frame period;
    
    A recording step of recording the second moving image data in a recording device,
    
    A control step of controlling the first frame period and the second frame period,
    
    In the control step, an imaging method in which the first frame period is set to an integral multiple of the flicker period and the second frame period is set to a predetermined period according to the detection result.
    
26. 
    A program that causes an imaging device to execute the imaging method according to claim 24.
    
27. 
    A program that causes an imaging device to execute the imaging method according to claim 25.

Images