JP2013123172A - Imaging apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To vary an imaging mode during continuous imaging.SOLUTION: An imaging apparatus includes: an imaging section having a plurality of imaging modes different in the number of imaging pixels or frame rates; a request acquisition section which acquires a change request for changing the imaging mode; and a control section which switches the imaging mode from a first imaging mode to a second imaging mode in accordance with the change request during continuous imaging by the imaging section. When switching from the first imaging mode to the second imaging mode is the change of the number of imaging pixels, the control section changes the frame rate of the second imaging mode to a frame rate corresponding to the number of changed imaging pixels. When the switching of the imaging mode is the change of the frame rate, the number of imaging pixels of the second imaging mode is changed to the number of imaging pixels corresponding to the changed frame rate.

Description

  The present invention relates to an imaging apparatus and a program.

A technique for converting the quality and amount of video data by changing both or one of the compression rate of the compression processing and the frame rate of the frame processing for the captured video signal is known (see, for example, Patent Document 1).
[Prior art documents]
[Patent Literature]
[Patent Document 1] Japanese Patent Application Laid-Open No. 2003-274360

  In the continuous shooting, for example, there is a problem that the upper limit of the frame rate at which shooting can be performed is determined by the set number of readout pixels.

  In the first aspect of the present invention, the imaging device includes an imaging unit having a plurality of imaging modes with different numbers of imaging pixels or frame rates, a request acquisition unit that acquires a change request for changing the imaging mode, and an imaging unit And a controller that switches the imaging mode from the first imaging mode to the second imaging mode in accordance with the change request during continuous shooting, and the controller changes the number of imaging pixels when switching from the first imaging mode to the second imaging mode. If it is, the frame rate in the second imaging mode is changed to the frame rate corresponding to the changed number of imaging pixels, and if the switching of the imaging mode is a change in the frame rate, the number of imaging pixels in the second imaging mode Is changed to the number of imaging pixels corresponding to the changed frame rate.

  In the second aspect of the present invention, the program obtains a change request for changing the number of imaging pixels from the imaging unit that can read out the image signal in a plurality of imaging modes having different numbers of imaging pixels; During continuous shooting by the imaging unit, the computer is caused to execute a control step of switching the imaging mode of the imaging unit from the first imaging mode to the second imaging mode according to the change request.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 shows a system configuration diagram of an imaging apparatus 100. FIG. An example of a block configuration in the image sensor 132, the camera MPU 133, and the ASIC 135 is shown. An example of an imaging mode in which the image sensor 132 operates is shown. An example of the control sequence in the image sensor 132 is shown. The processing flow in camera MPU133 is shown. An example of changing the imaging mode is shown in time series. An example of the size conversion process in the image processing unit 290 is shown. An example of a frame rate conversion process in the image processing unit 290 is shown.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a system configuration diagram of the imaging apparatus 100. The imaging apparatus 100 includes an interchangeable lens 120 and a camera body 130. The interchangeable lens 120 includes a lens side mount having a lens side mount connecting portion 121, and the camera body 130 includes a camera side mount having a camera side mount connecting portion 131. When the lens-side mount and the camera-side mount are engaged and the interchangeable lens 120 and the camera body 130 are integrated, the lens-side mount connecting portion 121 and the camera-side mount connecting portion 131 are connected. The imaging device 100 functions as a single-lens reflex camera in a state where the interchangeable lens 120 and the camera body 130 are integrated.

  The subject light passes through the lens group 122 serving as an imaging optical system along the optical axis and forms an image on the light receiving surface of the imaging element 132. The lens group 122 is controlled by the lens MPU 123. For example, the lens MPU 123 controls a focus lens motor (not shown) to move the focus lens group constituting the lens group 122.

  The lens MPU 123 is connected to the camera MPU 133 via the lens side mount connection unit 121 and the camera side mount connection unit 131, and controls the interchangeable lens 120 and the camera body 130 in cooperation with each other while performing communication. The imaging unit 150 outputs subject light as imaging data. The imaging unit 150 includes an imaging element 132 that photoelectrically converts subject light and outputs it as an image signal. As the image sensor 132, for example, a CMOS sensor can be applied. The imaging unit 150 reads the image signal output from the imaging element 132, converts it into imaging data, and outputs it. In addition, the imaging unit 150 has a plurality of imaging modes capable of imaging with different numbers of pixels or frame rates. The imaging mode has a combination of a plurality of imaging pixels and a plurality of frame rates. In other words, the imaging unit 150 has a plurality of imaging modes that can read out image signals output from the imaging element 132 at different numbers of pixels or frame rates. The camera MPU 133 allows the imaging unit 150 to image a subject with a different number of pixels or a frame rate. In each imaging mode, the number of imaging pixels, readout pixels of the imaging element 132, and readout rate of an image signal output from the imaging element 132 Then, a control signal indicating the exposure time, the charge reset of the image sensor 132 and the readout timing of the image signal, the readout gain, and the like is output to the imaging unit 150. The imaging mode in the present embodiment is determined from a combination of a plurality of imaging pixels and a plurality of frame rates. Details of the imaging mode will be described later. Note that the number of imaging pixels in each imaging mode may be arbitrarily set by the user within the range of the maximum number of pixels that the imaging element 132 has.

  The imaging data output from the imaging unit 150 is sequentially processed as image data. The ASIC 135 takes in the imaging data from the imaging unit 150. The ASIC 135 is an integrated circuit that combines circuits related to the image processing function into one. The ASIC 135 acquires information indicating the number of imaging pixels and the frame rate from the imaging unit 150. The SDRAM 136 functions as a buffer memory for temporarily storing image data during processing in the ASIC 135, image data recording and reproduction operations.

  The ASIC 135 converts the image data into data of a standardized image format and outputs it. For example, the ASIC 135 performs image processing for generating a JPEG file as still image data. In addition, the ASIC 135 performs image processing for generating an MPEG file as moving image data from a plurality of frames as time-sequential image data obtained by continuous imaging by the imaging unit 150. The camera MPU 133 transfers image data such as still image data and moving image data generated by the ASIC 135 to the external memory 160 as a nonvolatile recording medium for recording. The external memory 160 is configured by, for example, a flash memory.

  The ASIC 135 generates image data for display in parallel with the image data processed for recording in the external memory 160. The generated image data for display is displayed on the display unit 138 under the control of the display control unit 137. The display unit 138 is configured by a liquid crystal display, for example. The display unit 138 can also display various menu items related to various settings of the imaging apparatus 100 under the control of the display control unit 137.

  The imaging apparatus 100 is directly or indirectly controlled by the camera MPU 133 including each element in the image processing described above. The system memory 139 is an electrically erasable / storable nonvolatile memory, and is configured by, for example, a flash ROM. The system memory 139 stores constants, variables, programs, and the like necessary for the operation of the imaging apparatus 100 so that they are not lost even when the imaging apparatus 100 is not operating. The camera MPU 133 develops constants, variables, programs, and the like in the SDRAM 136 as appropriate, and uses them for controlling the imaging apparatus 100. The ASIC 135, SDRAM 136, system memory 139, display control unit 137, camera MPU 133, and external memory 160 in the camera main body 130 are connected to each other by a connection interface 145 such as a bus.

  The operation input unit 141 receives a user operation such as a setting operation by the user. The operation input unit 141 includes a power switch, a release button, a moving image imaging button, an imaging mode change button, various operation buttons, a touch panel integrated with the display unit 138, and the like. By operating the imaging mode change button, at least one of the number of imaging pixels and the frame rate of the imaging unit 150 can be changed. The camera MPU 133 detects that the operation input unit 141 has been operated, and executes an operation corresponding to the operation. For example, the camera MPU 133 controls each element of the imaging apparatus 100 so as to execute a release operation when a release button is operated. When the moving image capturing button is operated, the camera MPU 133 controls each element of the image capturing apparatus 100 so as to perform moving image capturing as an example of continuous image capturing. In addition, the camera MPU 133 controls each element of the imaging apparatus 100 so as to perform an operation according to the menu item and operation content displayed on the display unit 138 when the touch panel is operated.

  Each element of the camera body 130 and the external memory 160 are supplied with power from the power source 170. The power source 170 is configured by a secondary battery such as a lithium ion battery, a household AC power source, and the like that can be attached to and detached from the camera body 130. The camera MPU 133 controls power supply from the power supply 170 to each element of the imaging apparatus 100. The secondary battery is an example of a battery that drives the imaging apparatus 100. The battery includes a non-rechargeable battery that cannot be substantially charged.

  FIG. 2 shows an example of a block configuration in the imaging unit 150, the camera MPU 133, and the ASIC 135. The camera MPU 133 includes an imaging mode information acquisition unit 210 and an imaging mode selection unit 200 as functional blocks. The imaging unit 150 includes an imaging mode setting unit 220, a register setting unit 230, an internal register 240, a signal generation unit 250, and an imaging element 132 as functional blocks. The ASIC 135 includes a processing parameter setting unit 260, a register setting unit 270, an internal register 280, an image processing unit 290, and a moving image generation unit 292 as functional blocks. A part of the camera MPU 133 and a part of the ASIC 135 function as an image processing engine.

  The imaging mode information acquisition unit 210 acquires a change request for changing the imaging mode in the imaging unit 150. The imaging mode information acquisition unit 210 acquires information indicating the frame rate or the number of imaging pixels. For example, the imaging mode information acquisition unit 210 may acquire a user operation for setting the frame rate and the number of imaging pixels from the operation input unit 141 as information indicating the frame rate and the like. As will be described later, the imaging mode information acquisition unit 210 may acquire the detection result of the person from the image, the detection result of the movement of the subject from the image, and the like from the ASIC 135 as information indicating the frame rate and the like.

  The imaging mode selection unit 200 selects an imaging mode having an imaging pixel number that matches the frame rate acquired by the imaging mode information acquisition unit 210 from among a plurality of imaging modes. The imaging mode selection unit 200 can select imaging modes with different numbers of imaging pixels. The imaging mode selection unit 200 outputs information specifying the selected imaging mode to the imaging unit 150. The imaging mode selection unit 200 selects an imaging mode having a frame rate that matches the number of imaging pixels acquired by the imaging mode information acquisition unit 210 from a plurality of imaging modes. The imaging mode information acquisition unit 210 and the imaging mode selection unit 200 function as a request acquisition unit that acquires a request to change the imaging mode. The relationship between the imaging mode and the frame rate will be described with reference to FIG.

  The imaging mode selection unit 200 captures the image closest to the number of imaging pixels determined by the first imaging mode when there are a plurality of imaging modes having the number of imaging pixels matching the frame rate acquired by the imaging mode information acquisition unit 210. A second imaging mode that determines the number of pixels may be selected. Thus, by selecting an imaging mode having an imaging pixel number that is close to the imaging pixel number of the current imaging mode (first imaging mode), it is possible to prevent the pixel number from changing rapidly. Further, when the frame rate acquired by the imaging mode information acquisition unit 210 is significantly different from the current frame rate, the imaging mode selection unit 200 may select the frame rate in stages. For example, the imaging mode selection unit 200 may select a frame rate at which the frame rate acquired by the imaging mode information acquisition unit 210 gradually decreases or increases.

  Each unit of the imaging unit 150 operates according to a value set in the internal register 240. The imaging element 132 includes a photoelectric conversion element that generates an image signal, and the imaging operation of the imaging element 132 is controlled according to the internal register 240. For example, the imaging pixel is selected according to the internal register 240. As a result, imaging data is output from the imaging unit 150 in a plurality of imaging modes with different numbers of imaging pixels. The imaging data output by the imaging unit 150 is output to the image processing unit 290.

  The imaging mode setting unit 220 sets a register value prepared for each imaging mode in the register setting unit 230 based on the imaging mode information acquired from the imaging mode selection unit 200. The signal generation unit 250 generates a vertical synchronization signal and a horizontal synchronization signal for driving the image sensor 132 according to the values set in the register setting unit 230. The vertical synchronization signal generated by the signal generation unit 250 is supplied to the image sensor 132 and the register setting unit 230. The register setting unit 230 sets a register value indicating the imaging mode set by the imaging mode setting unit 220 in the internal register 240 in synchronization with the vertical synchronization signal generated by the signal generation unit 250. Therefore, the imaging mode can be switched in synchronization with the vertical synchronization signal.

  Accordingly, when the second imaging mode is selected by the imaging mode selection unit 200 while the imaging unit 150 is imaging in the first imaging mode during continuous imaging (for example, during moving image imaging), the imaging unit 150 is selected. Is switched from the first imaging mode to the second imaging mode. That is, the imaging mode setting unit 220 and the register setting unit 230 are control units that switch the imaging mode of the imaging unit 150 from the first imaging mode to the second imaging mode in accordance with a request for changing the imaging mode during continuous imaging by the imaging element 132. Function as. When the switching from the first imaging mode to the second imaging mode is a change in the number of imaging pixels, the imaging mode setting unit 220 changes the frame rate in the second imaging mode to a frame rate corresponding to the changed number of imaging pixels. If the switching of the imaging mode is a change in the frame rate, the register setting unit 230 is controlled so that the number of imaging pixels in the second imaging mode is changed to the number of imaging pixels corresponding to the changed frame rate. Under the control of the imaging mode selection unit 200 and the register setting unit 230, the imaging mode of the imaging unit 150 is switched from the first imaging mode to the second imaging mode based on the selection of the imaging mode selection unit 200.

  In the ASIC 135, the image processing unit 290 performs image processing on the image acquired from the imaging unit 150. Examples of image processing include sensitivity correction, color interpolation processing, gamma correction, white balance correction, smoothing, contour compensation, sharpening, two-dimensional filtering, and diagonal line detection. The image processing unit 290 also detects a person from the image, detects the amount of movement of the subject, and the like. As a result of the person detection, a result such as the amount of movement of the subject is output to the imaging mode information acquisition unit 210 and is input as a part of information indicating the frame rate and the like.

  Further, the image processing unit 290 converts a plurality of images obtained by continuous imaging into images having the same number of pixels based on the number of imaging pixels in the first imaging mode and the second imaging mode. For example, the image processing unit 290 enlarges or reduces the plurality of images obtained by continuous imaging into images of the same size by enlarging or reducing the number of pixels in each of the first imaging mode and the second imaging mode. Convert.

  Here, each unit of the ASIC 135 operates according to a value set in the internal register 280. For example, the image processing of the image processing unit 290 is controlled according to the value set in the internal register 280. For example, according to the internal register 280, a target size in the enlargement process or the reduction process in the image processing unit 290 is controlled. In addition, the sensitivity correction value used in the image processing unit 290 is controlled.

  Specifically, the processing parameter setting unit 260 sets a register value prepared for each imaging mode in the register setting unit 270 based on information acquired from the imaging mode selection unit 200. The register setting unit 270 sets the internal register 280 with a register value indicating the imaging mode set by the imaging mode setting unit 220 in synchronization with the vertical synchronization signal generated by the signal generation unit 250. Therefore, the image processing parameter can be switched in synchronization with the vertical synchronization signal. Therefore, the image processing in the ASIC 135 can be switched according to the switching of the imaging mode in the imaging element 132.

  The moving image generation unit 292 generates a moving image including a plurality of images obtained by continuous imaging in the first imaging mode and the second imaging mode. For example, the moving image generation unit 292 generates one moving image data such as an MPEG file. The generated moving image data is recorded in the external memory 160. In addition, the image generated by the image processing unit 290 can be applied to an image for display on the display unit 138. The display unit 138 may continuously display a plurality of images converted to the same size by the image processing unit 290.

  In addition, moving image imaging can be illustrated as continuous imaging in the above description. In this case, the frame rate may be a field rate. However, the moving image is an example of continuous imaging, and the continuous imaging is a concept including not only moving image capturing but also capturing images continuously in time. Therefore, it can be said that the frame rate corresponds to the number of frames captured per unit time or the readout rate of the image signal output from the image sensor 132.

  FIG. 3 shows an example of the imaging mode of the imaging unit 150. The imaging unit 150 can be driven in any of the full HD mode, the HD mode, and the SD mode as the imaging mode. These imaging modes differ from each other in the number of imaging pixels. The full HD mode and the HD mode have submodes with different frame rates. For example, the full HD mode has three submodes that are respectively determined by frame rates of 24 fps, 30 fps, and 60 fps. The HD mode has three submodes, each of which is defined by a frame rate of 30 fps, 60 fps, and 120 fps.

  In the full HD mode, in the sub mode with a frame rate of 60 fps, imaging data of approximately 124 megapixels is output from the imaging unit 150 per second. In the HD mode, in the sub mode with a frame rate of 120 fps, imaging data for approximately 111 megapixels per second is output from the imaging unit 150. In the SD mode, imaging data of about 127 megapixels per second is output from the imaging unit 150. For this reason, the image processing capability of the ASIC 135 or the like at the time of continuous imaging may be designed so that an image output in the SD mode can be processed.

  Different values are assigned to the imaging modes as registers to be set in the internal register 240. Also, different values are assigned to the sub mode as registers to be set in the internal register 240.

  The user can set the imaging mode before capturing a moving image by operating the operation input unit 141 or the like. For example, the user sets the imaging quality of the moving image through the setting menu. Then, the camera MPU 133 starts moving image imaging in an imaging mode corresponding to the selected imaging image quality. When the user selects from the setting menu for the user, for example, one is selected from the sub mode with the frame rate of 30 fps in the full HD mode, the sub mode with the frame rate of 120 fps in the HD mode, and the SD mode.

  A value corresponding to the imaging mode selected by the user is set in the internal register 240. For an imaging mode having a plurality of sub modes, a value corresponding to the sub mode is set in the internal register 240. Note that the combination of the number of imaging pixels and the frame rate may be referred to as a drive mode. That is, a value corresponding to the drive mode is set in the internal register 240. It should be noted that imaging modes with different numbers of imaging pixels can be realized by performing thinning-out reading in the imaging element 132. Also, an imaging mode with a different number of pixels can be realized by pixel addition.

  FIG. 4 shows an example of a control sequence in the imaging unit 150. In this example, an example in which the imaging mode of the imaging unit 150 is switched from the first drive mode to the second drive mode will be described. Here, it is assumed that the frame rate is switched in accordance with the switching of the drive mode.

  The signal generation unit 250 generates a vertical synchronization signal according to the currently set first drive mode. Here, the readout cycle (image signal readout cycle) in the first drive mode is referred to as a first readout cycle. The signal generation unit 250 generates a vertical synchronization signal at the first read cycle at times t10, t11, and t13 in the drawing.

  Here, at time t12 after time t11, when the imaging mode setting unit 220 acquires a command signal for changing the driving mode from the camera MPU 133, the imaging mode setting unit 220 corresponds to the driving mode specified by the command signal. The value is supplied to the register setting unit 230. Here, the read cycle determined in the designated drive mode is referred to as a second read cycle. When the vertical synchronization signal at time t13 is supplied, the register setting unit 230 sets the value supplied from the imaging mode setting unit 220 in the internal register 240 at time t14 when a predetermined delay time has elapsed (time). t14). Here, the delay time is a time shorter than the read cycle corresponding to the maximum frame rate. In this example, the delay time is shorter than 1/480.

  For this reason, the signal generation unit 250 generates the next vertical drive signal after the time t13 at a time after the second read cycle from the time t13. As in this example, when the frame rate in the second drive mode is high, the vertical synchronization signal is generated at a timing (time t16) earlier than the time after the first read cycle from time t13. Then, the signal generation unit 250 generates a vertical synchronization signal every time the second read cycle elapses from time t16.

  In this example, the first drive mode is set until time t13, and the second drive mode is set from time t16. Specifically, the value of the internal register 240 of the image sensor 132 that sets the drive mode is changed from a value indicating the first drive mode to a value indicating the second drive mode between successive vertical synchronization signals. As a result, the drive mode can be switched from the first drive mode to the second drive mode between successive vertical synchronization signals. Note that the vertical drive signal is an example of a signal synchronized with an image captured in continuous imaging. The signal that triggers switching of the drive mode is not limited to the vertical synchronization signal. Any signal may be used as long as the imaging mode can be switched from the first drive mode to the second drive mode in synchronization with a signal that determines the readout timing from the image sensor 132 in the continuous imaging.

  Note that the processing for setting the value of the internal register 280 in the ASIC 135 is the same as the processing for setting the internal register 240, and therefore, detailed description thereof is omitted.

  FIG. 5 shows a processing flow in the camera MPU 133. This flow is started when the moving image capturing button is pressed. This flow is executed mainly by the camera MPU 133. For example, it is executed by a task for controlling moving image capturing.

  In step S <b> 502, a command signal for starting moving image capturing is output to the image sensor 132. The imaging unit 150 initializes each unit of the imaging unit according to the command signal and starts moving image imaging.

  In step S504, the imaging mode information acquisition unit 210 waits for an event related to a frame rate change. For example, this event occurs when a predetermined condition for changing the frame rate is satisfied, such as when a person is detected from the image in the image processing unit 290. Further, this event occurs when the movement of the subject is detected. The movement of the subject is detected by the image processing unit 290 based on, for example, detection of a motion vector. In addition, this event may occur when a voice larger than a predetermined value is detected. Also, this event may occur when the user gives an instruction to change the frame rate by operating the frame rate change switch.

  When an event related to the frame rate change occurs, the imaging mode selection unit 200 determines whether or not the current frame rate needs to be changed to a new frame rate (step S506). For example, the imaging mode selection unit 200 determines that the frame rate needs to be changed when the currently set frame rate is different from the new frame rate. The new frame rate may be determined in accordance with, for example, a predetermined type of condition. For example, a frame rate when a person is detected, a frame rate when motion is detected, a frame rate when sound is detected, and the like may be determined in advance. The new frame rate may be determined according to the amount of movement of the subject, the volume of the sound, and the like. For example, the new frame rate may be set higher as the amount of motion is larger. The new frame rate may be set higher as the area occupied by the person in the image is larger.

  When it is necessary to change the frame rate, the imaging mode selection unit 200 determines whether or not there is a sub-mode in which a new frame rate is set within the currently set imaging mode (step S508).

  When there is no sub-mode with a suitable frame rate, the imaging mode selection unit 200 determines whether there is a mode that defines a new frame rate among other sub-modes or imaging modes (step S510). ). If there is a mode that defines a new frame rate, the imaging mode selection unit 200 outputs a command signal indicating the imaging mode to the imaging unit 150 in order to change the imaging mode of the imaging unit 150 (step S512). ).

  Subsequently, in step S514, a command signal for setting an image processing parameter is output to the imaging unit 150. Examples of the image processing parameters to be changed include image size change, sensitivity correction value change, filter size change, and the like.

  Subsequently, it is determined whether or not to finish moving image capturing. For example, when the moving image capturing button is pressed, it is determined that the moving image capturing ends. When it is determined that the moving image capturing is to be ended, a command signal for ending the moving image capturing is output to the image sensor 132, and this flow is ended. As described above, according to the imaging apparatus 100, the imaging mode can be changed during the imaging of the moving image from when the moving image imaging button is pressed to when it is pressed again.

  In step S506, if it is not necessary to change to a new imaging rate, the process proceeds to step S516. Also, in step S510, if there is no mode that defines a new frame rate, the process proceeds to step S516. If it is determined in step S508 that there is a submode having a new frame rate, the imaging mode selection unit 200 outputs a command signal for changing the frame rate to the imaging unit 150 (step S522). Specifically, a command signal corresponding to a sub mode that determines a new frame rate is output to the imaging unit 150. When the process of step S522 is completed, the process proceeds to step S516.

  FIG. 6 shows an example of changing the imaging mode in time series. In this example, the horizontal axis represents time, and the vertical axis represents the image size (the number of imaging pixels) of the imaging data output from the imaging unit 150.

  In this example, at time t1, switching from the 30 fps submode in the full HD mode to the moving image capturing in the 60 fps submode in the HD mode is performed. At time t2, switching from the 60 fps sub-mode in the HD mode to the moving image imaging in the SD mode occurs. In addition, at time t3, switching from the SD mode to the moving image capturing in the 60 fps sub-mode in the HD mode occurs. At time t4, the moving image capturing is switched from the 60 fps submode in the HD mode to the 30 fps submode in the full HD mode.

  FIG. 7 shows an example of size conversion processing in the image processing unit 290. Here, a description will be given using the example shown in FIG.

  The image processing unit 290 performs image processing for enlarging the size to 1920 × 1080 with respect to an image captured in the HD mode starting from time t1. The image processing unit 290 also performs image processing for enlarging the size to 1920 × 1080 even for an image captured in the SD mode starting from time t2. Further, the image processing unit 290 performs image processing for enlarging the size to 1920 × 1080 with respect to the image captured in the HD mode starting from the time t3. As a result, it is possible to generate image data having a certain size in time.

  In this example, the description has been given on the assumption that the size is enlarged, but the image size may be reduced. For example, the image processing unit 290 may perform a process of reducing the size of at least a part of the images included in the moving image so that the image has a predetermined size. The predetermined size may be a size corresponding to the number of imaging pixels set when moving image capturing is started. For example, when the image capturing mode at the start of moving image capturing is the HD mode, the image processing unit 290 reduces the image captured in the full HD mode to the same size as the HD mode, and converts the image captured in the SD mode into the HD mode. You may enlarge to the same size. Accordingly, it is possible to provide a moving image having a size designated when moving image capturing is started. Note that pixel interpolation processing can be applied as image processing of at least one of size enlargement processing and reduction processing.

  FIG. 8 shows an example of a frame rate conversion process in the image processing unit 290. Here, the description will be made using the example shown in FIG. The image processing unit 290 performs image processing for reducing the frame rate to 30 fps with respect to the moving image captured in the HD mode starting from the time t1. For example, the image processing unit 290 selects one frame every two frames and applies it to the converted moving image frame. The image processing unit 290 also performs image processing for reducing the frame rate to 30 fps even for a moving image captured in the SD mode starting from time t2. For example, the image processing unit 290 selects one frame every 16 frames and applies it to the converted moving image frame. In addition, the image processing unit 290 performs image processing for reducing the frame rate to 30 fps with respect to the moving image captured in the HD mode starting from the time t3. Thereby, it is possible to generate a moving image having a constant frame rate. In this example, the frame rate can be converted by the frame thinning process for the moving image.

  In this example, the frame rate is decreased. However, the frame rate may be increased. For example, the image processing unit 290 may apply image processing that increases the frame rate to at least some partial moving images included in the moving image in order to obtain a moving image having a predetermined frame rate. The predetermined frame rate may be a frame rate set when moving image capturing is started. For example, when the image capturing mode at the start of moving image capturing is 60 fps in the HD mode, the image processing unit 290 may convert a 30 fps moving image into a 60 fps frame rate and convert a 120 fps moving image into a 60 fps moving image. Thereby, it is possible to provide a moving image having a frame rate designated when moving image capturing is started. An example of the process for increasing the frame rate is a process of copying a previous or subsequent frame and applying it as a frame between frames. Further, a frame obtained by averaging the previous and subsequent frames may be applied as a frame between frames.

  In this example, it is assumed that image processing is performed to make the playback frame rate constant with respect to a moving image captured with the imaging rate varied. That is, when the frame rate of the first imaging mode is different from the frame rate of the second imaging mode, the image processing unit 290 is obtained by continuous imaging based on the respective frame rates of the first imaging mode and the second imaging mode. The plurality of images thus obtained are converted into images at regular time intervals. The display unit 138 continuously displays the plurality of images converted by the image processing unit 290, so that a moving image can be reproduced on the same time scale as the imaging time scale. Note that the display unit 138 may display the image at a predetermined frame rate without making the frame rate constant. In this case, a portion captured at a high frame rate is reproduced slowly. For example, when a subject moving faster than a predetermined speed is detected, the present invention can be applied to an imaging and reproduction method in which an image is captured in a high frame rate drive mode and is reproduced in slow motion during reproduction.

  According to the imaging apparatus 100 described above, by switching the imaging mode of the imaging unit 150 during continuous imaging, it is possible to widely control the frame rate of the imaging unit 150 in continuous imaging.

  The processing described in relation to the imaging apparatus 100 of the present embodiment can be realized by causing each unit of the imaging apparatus 100, for example, the camera MPU 133 or the like to operate according to a program. That is, the process can be realized by a so-called computer device. The computer device may load a program for controlling the execution of the above-described process, operate according to the read program, and execute the process. The computer device can load the program by reading a computer-readable recording medium storing the program.

  In the present embodiment, the imaging apparatus 100 as an interchangeable lens single-lens reflex camera has been described. As an imaging device, a compact digital camera, a mirrorless single-lens camera, a video camera, a mobile phone with an imaging function, a portable information terminal with an imaging function, an amusement device such as a game device with an imaging function, or a device having an imaging function is applied Can be the target of.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

DESCRIPTION OF SYMBOLS 100 Imaging device, 120 Interchangeable lens, 121 Lens side mount connection part, 130 Camera body, 131 Camera side mount connection part, 122 Lens group, 132 Image sensor, 123 Lens MPU, 133 Camera MPU, 135 ASIC, 136 SDRAM, 137 Display Control unit, 138 display unit, 139 system memory, 141 operation input unit, 145 connection interface, 150 imaging unit, 170 power source, 160 external memory, 200 imaging mode selection unit, 210 imaging mode information acquisition unit, 220 imaging mode setting unit, 230 register setting unit, 240 internal register, 250 signal generation unit, 260 processing parameter setting unit, 270 register setting unit, 280 internal register, 290 image processing unit, 292 moving image generation unit

Claims (12)

An imaging unit having a plurality of imaging modes having different imaging pixel counts or frame rates;
A request acquisition unit for acquiring a change request for changing the imaging mode;
A controller that switches the imaging mode from the first imaging mode to the second imaging mode according to the change request during continuous imaging by the imaging unit;
When the switching from the first imaging mode to the second imaging mode is a change in the number of imaging pixels, the control unit changes the frame rate of the second imaging mode to a frame rate corresponding to the changed number of imaging pixels. An imaging apparatus that changes and changes the number of imaging pixels in the second imaging mode to the number of imaging pixels corresponding to the changed frame rate when the switching of the imaging mode is a change in frame rate. The request acquisition unit
A frame rate information acquisition unit for acquiring information indicating a frame rate;
An imaging mode selection unit that selects an imaging mode having an imaging pixel number that matches the frame rate from the plurality of imaging modes based on information indicating the frame rate acquired by the information acquisition unit;
The imaging device according to claim 1, wherein the control unit switches the imaging mode of the imaging unit from the first imaging mode to the second imaging mode based on selection of the imaging mode selection unit. The request acquisition unit
An imaging pixel number information acquisition unit for acquiring information indicating the number of imaging pixels;
An imaging mode selection unit that selects an imaging mode having a frame rate suitable for the number of imaging pixels from the plurality of imaging modes based on information indicating the number of imaging pixels acquired by the information acquisition unit. And
The imaging device according to claim 1, wherein the control unit switches the imaging mode of the imaging unit from the first imaging mode to the second imaging mode based on selection by the imaging mode selection unit. The imaging device according to any one of claims 1 to 3, wherein the imaging mode has a combination of a plurality of imaging pixels and a plurality of frame rates. 5. The control unit according to claim 1, wherein the control unit switches the imaging mode from the first imaging mode to the second imaging mode in synchronization with a signal that determines a readout timing from the imaging unit in the continuous imaging. The imaging device according to item. The imaging device according to claim 5, wherein the control unit switches the imaging mode from the first imaging mode to the second imaging mode between successive vertical synchronization signals. A signal generation unit for generating the vertical synchronization signal;
The control unit changes the value of the internal register of the imaging unit that sets the imaging mode from the value indicating the first imaging mode to the value indicating the second imaging mode between the continuous vertical synchronization signals. The imaging device according to claim 6. The imaging device according to any one of claims 1 to 7, further comprising a moving image generation unit configured to generate a moving image including a plurality of images obtained by continuous imaging in the first imaging mode and the second imaging mode. An image conversion unit that converts a plurality of images obtained by the continuous shooting into an image having the same number of pixels based on the number of imaging pixels in each of the first imaging mode and the second imaging mode, and the image conversion unit. The imaging apparatus according to claim 1, further comprising a display unit that continuously displays the plurality of converted images. When the frame rate of the first imaging mode is different from the frame rate of the second imaging mode, a plurality of images obtained by the continuous imaging based on the respective frame rates of the first imaging mode and the second imaging mode An image conversion unit that converts the image of
The imaging device according to any one of claims 1 to 8, further comprising a display unit that continuously displays the plurality of images converted by the image conversion unit. The mode selection unit determines the number of imaging pixels closest to the number of imaging pixels determined by the first imaging mode when there are a plurality of imaging modes in which a frame rate suitable for the frame rate acquired by the information acquisition unit exists. The imaging apparatus according to claim 2, wherein an imaging mode to be determined is selected. A request acquisition step for acquiring a change request for changing the number of imaging pixels from the imaging unit capable of reading out image signals in a plurality of imaging modes having different imaging pixel numbers;
A program that causes a computer to execute a control step of switching an imaging mode of the imaging unit from a first imaging mode to a second imaging mode according to the change request during continuous imaging by the imaging unit.

Images