JP2013081101A - Image processing apparatus, imaging apparatus, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To avoid increase in a code amount even in a case where a speed of changing an imaging region is high.SOLUTION: An image processing apparatus comprises: a period identification unit which identifies a first period being a period in which the speed of changing an imaging region in a moving image exceeds a predetermined reference value; and a compression unit which compresses the moving image by applying higher compression intensity in the first period than in a second period being a period other than the first period. The program causes a computer to execute steps of: identifying the first period being the period in which the speed of changing the imaging region in the moving image exceeds the predetermined reference value; and compressing the moving image by applying higher compression intensity in the first period than in the second period being the period other than the first period.

Description

  The present invention relates to an image processing device, an imaging device, and a program.

A technique for determining a shooting scene and determining a compression rate for each determined scene is known (see, for example, Patent Document 1).
[Prior art documents]
[Patent Literature]
[Patent Document 1] Japanese Unexamined Patent Application Publication No. 2010-199656

  When the speed of changing the imaging region is large, the correlation between images constituting the moving image becomes small, and the code amount increases. The code amount also increases when the speed of changing the imaging condition that changes the image content is high.

  In the first aspect of the present invention, the image processing apparatus includes a period specifying unit that specifies a first period that is a period in which a change speed of an imaging region in a moving image exceeds a predetermined reference value, and a period other than the first period. A compression unit that compresses the moving image by applying a higher compression strength in the first period than in the second period.

  In the second aspect of the present invention, the image processing device includes a period specifying unit that specifies a first period in which a change speed of an imaging condition that changes image content in a moving image exceeds a predetermined reference value. A compression unit that applies a higher compression strength to the first period than the second period, which is a period other than the first period, and compresses the moving image.

  In a third aspect of the present invention, an imaging device includes an imaging unit that captures an image and the image processing device.

  In the fourth aspect of the present invention, the program specifies a first period, which is a period in which the change speed of the imaging region in the moving image exceeds a predetermined reference value, and a period other than the first period. Causing the computer to execute a step of compressing the moving image by applying a higher compression strength in the first period than in the second period.

  In the fifth aspect of the present invention, the program specifies a first period that is a period in which a change speed of an imaging condition that changes image content in a moving image exceeds a predetermined reference value; And causing the computer to execute a step of compressing the moving image by applying a higher compression strength in the first period than in the second period which is a period other than the above.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 shows a system configuration diagram of an imaging apparatus 100. FIG. An example of a block configuration in the ASIC 135 is shown. A period during which the compression strength is controlled based on the zoom magnification is schematically shown. An example of calculating the zoom magnification change speed is shown. An example of compression control based on the zoom magnification changing speed is shown. Another example of compression control based on the zoom magnification change speed is shown. Another example of compression control based on the zoom magnification change speed is shown. An example of correction data related to the focus change speed is shown in a table format. An example of the scene which judges change speed for every image area is shown typically. 2 shows a processing flow from start to finish of the imaging apparatus 100. An example of an operation flow in moving image shooting is shown. An example of the operation | movement flow in which the compression part 260 compresses a flame | frame is shown.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a system configuration diagram of the imaging apparatus 100. The imaging apparatus 100 includes an interchangeable lens 120 and a camera body 130. The interchangeable lens 120 includes a lens mount having a lens mount contact 121, and the camera body 130 includes a camera mount having a camera mount contact 131. When the lens mount and the camera mount are engaged and the interchangeable lens 120 and the camera body 130 are integrated, the lens mount contact 121 and the camera mount contact 131 are connected. The imaging device 100 functions as a single-lens reflex camera in a state where the interchangeable lens 120 and the camera body 130 are integrated.

  The subject light passes through the lens group 122 serving as a photographing optical system along the optical axis and forms an image on the light receiving surface of the image sensor 132. The lens group 122 is controlled by the lens MPU 123. For example, the lens MPU 123 controls the lens motor to move the focus lens group and the variable power lens group that constitute the lens group 122. Thereby, the focus adjustment and the focal length adjustment of the lens group 122 are performed.

  The lens MPU 123 is connected to the camera MPU 133 via the lens mount contact 121 and the camera mount contact 131, and controls the interchangeable lens 120 and the camera body 130 in cooperation with each other while performing communication. For example, the lens MPU 123 cooperates with the camera MPU 133 to adjust the focus of the lens group 122 so as to focus on the subject in at least one focus adjustment region set within the angle of view. In addition, the lens MPU 123 adjusts the position in the optical axis direction of the zoom lens that performs zooming of the lens group 122 in accordance with an instruction to change the zoom magnification acquired from the camera MPU 133. The lens MPU 123 supplies information indicating the focal position of the lens group 122 and information indicating the focal length of the lens group 122 to the camera MPU 133.

  The interchangeable lens 120 has a focus ring 25 on which a user manually performs a focus adjustment operation. By operating the focus ring 25, the focus lens group moves in the optical axis direction. For example, the focus lens group obtains a driving force that goes straight in the optical axis direction according to the rotation of the focus ring 25 around the optical axis. In addition, the interchangeable lens 120 includes a zoom ring 24 that allows a user to perform an angle of view adjustment operation. When the user operates the zoom ring 24, the focal length of the lens group 122 is adjusted. For example, the variable power lens group obtains a driving force that goes straight in the optical axis direction according to the rotation of the zoom ring 24 around the optical axis. The lens MPU 123 supplies lens information such as information indicating the focal position and information indicating the focal length to the camera MPU 133 during moving image shooting by the imaging apparatus 100.

  The imaging element 132 functions as a part of an imaging unit that captures an image using subject light that has passed through the lens group 122. The imaging element 132 includes a photoelectric conversion element that photoelectrically converts an optical image that is a subject image. As the imaging device 132, for example, a solid-state imaging device such as a CMOS sensor or a CCD sensor can be applied. The image sensor 132 supplies an analog signal as an image signal based on accumulated charges generated by receiving a subject light beam to the A / D converter 134. The A / D converter 134 converts the analog image signal output from the image sensor 132 into a digital image signal and outputs it as image data. The image sensor 132 is driven by the drive unit 140 that has received an instruction from the camera MPU 133.

  The image data output from the A / D converter 134 is stored in the SDRAM 136. At least a part of the memory area of the SDRAM 136 is used as a buffer memory that temporarily stores image data generated by imaging. When the image sensor 132 continuously captures an image with a subject light beam, the image data sequentially generated by the imaging is sequentially stored in the SDRAM 136. For example, image data of a plurality of still images obtained by continuously capturing still images with subject light flux is sequentially stored in the SDRAM 136. In addition, frame data as a plurality of images constituting a moving image obtained by capturing a moving image with the subject light flux is sequentially stored in the SDRAM 136.

  The ASIC 135 sequentially processes the image data stored in the SDRAM 136. The ASIC 135 is an integrated circuit that combines circuits related to the image processing function into one. The ASIC 135 performs image processing by using at least a part of the memory area of the SDRAM 136 as a work area for image processing.

  Specifically, the ASIC 135 converts the image data into image data of a standardized image format and outputs it. For example, the ASIC 135 performs image processing for generating a JPEG file as still image data. In addition, the ASIC 135 performs image processing for generating an MPEG file as moving image data from each image data of a plurality of images constituting the moving image. The camera MPU 133 transfers image data such as still image data and moving image data generated by the ASIC 135 to the external memory 160 as a nonvolatile recording medium for recording. As the external memory 160, for example, a semiconductor memory such as a flash memory can be exemplified.

  The ASIC 135 generates image data for display in parallel with the image data processed for recording in the external memory 160. The generated image data for display is converted into a display image signal under the control of the display control unit 137 and displayed on the display unit 138 as a display device such as a liquid crystal display. Various menu items related to various settings of the imaging apparatus 100 can be displayed on the display unit 138 under the control of the display control unit 137 with or without displaying an image. The camera MPU 133 controls the display control unit 137 to display menu items on the display unit 138.

  The imaging apparatus 100 is directly or indirectly controlled by the camera MPU 133 including each element in the image processing described above. The system memory 139 is an electrically erasable / storable nonvolatile memory, and is configured by, for example, a flash ROM. The system memory 139 stores constants, variables, programs, and the like necessary for the operation of the imaging apparatus 100 so that they are not lost even when the imaging apparatus 100 is not operating. The camera MPU 133 develops constants, variables, programs, and the like in the SDRAM 136 as appropriate, and uses them for controlling the imaging apparatus 100. The ASIC 135, SDRAM 136, system memory 139, display control unit 137, camera MPU 133, and external memory 160 in the camera main body 130 are connected to each other by a connection interface 145 such as a bus.

  The operation input unit 141 receives a user operation such as a setting operation by the user. The operation input unit 141 includes a power switch, a release button, a moving image shooting button, various operation buttons, a touch panel integrated with the display unit 138, and the like. The camera MPU 133 detects that the operation input unit 141 has been operated, and executes an operation corresponding to the operation. For example, the camera MPU 133 controls each unit of the imaging apparatus 100 to execute the release operation when the release button is operated. The camera MPU 133 controls each unit of the imaging apparatus 100 to execute moving image shooting when the moving image shooting button is operated. In addition, the camera MPU 133 controls each unit of the imaging apparatus 100 so that when a touch panel incorporated in the display unit 138 is operated, the camera MPU 133 performs an operation according to the menu item and the operation content displayed on the display unit 138. Further, the camera MPU 133 may accept a user operation for changing the zoom magnification and the focal position through the touch panel.

  The sensor unit 142 detects the change speed of the imaging region. For example, the sensor unit 142 is implemented by various sensors such as a gyro sensor for detecting a panning angle, a tilt angle, and the like of the imaging device 100, and an acceleration sensor for detecting a change in the position of the imaging device 100. The camera MPU 133 acquires a sensor output from the sensor unit 142, and calculates a panning angle, a tilt angle, a change in imaging position, and the like. The camera MPU 133 supplies the calculated information to the ASIC 135. In addition, the camera MPU 133 supplies information indicating the imaging conditions such as the zoom magnification, the focal position, the depth of field, and the aperture amount to the ASIC 135. The camera MPU 133 may calculate the zoom magnification based on the focal length information acquired from the lens MPU 123. The camera MPU 133 may calculate the focal position based on the position of the focus lens group acquired from the lens MPU 123, the rotation angle of the focus ring 25, and the like. In addition, the camera MPU 133 may calculate the depth of field based on the aperture value acquired from the lens MPU 123.

  The ASIC 135 records the information acquired from the camera MPU 133 along with the image data as incidental information of the image data. Further, the ASIC 135 may record the information acquired from the camera MPU 133 during moving image shooting in moving image data in association with the frame. Further, as will be described later, the ASIC 135 may adjust the compression strength for compressing the frames of the moving image based on the information acquired from the camera MPU 133.

  Each element of the camera body 130 and the external memory 160 are supplied with power from the power source 170. Examples of the power source 170 include a secondary battery such as a lithium ion battery that can be attached to and detached from the camera body 130, a commercial AC power source, and the like. The camera MPU 133 controls power supply from the power supply 170 to each unit of the imaging apparatus 100. The secondary battery is an example of a battery that drives the imaging device 100. The battery includes a non-rechargeable battery that cannot be substantially charged.

  FIG. 2 shows an example of a block configuration in the ASIC 135. The ASIC 135 includes a change speed specifying unit 210, a period specifying unit 220, a change region specifying unit 230, a compression strength determining unit 240, and a compression unit 260. The functional blocks of the change speed specifying unit 210, the period specifying unit 220, the change region specifying unit 230, the compression strength determining unit 240, and the compression unit 260 can function as an image processing apparatus.

  The change speed specifying unit 210 specifies the zoom change speed, the imaging direction change speed, and the imaging position change speed in moving image shooting. For example, the change speed specifying unit 210 specifies the zoom magnification change speed, the panning angle change speed, and the tilt angle change speed in moving image shooting based on information acquired from the camera MPU 133. The zoom magnification changing speed, the panning angle changing speed, and the tilt angle changing speed are examples of the imaging area changing speed in a moving image.

  The change speed specifying unit 210 specifies the change speed of the imaging condition in moving image shooting. For example, the change speed specifying unit 210 acquires information indicating imaging conditions such as the zoom magnification, the focal position, and the depth of field from the camera MPU 133, and based on the acquired information, the change speed of the zoom magnification in moving image shooting, The focal position changing speed and the depth of field changing speed are specified. The zoom magnification change speed, the focus position change speed, the depth of field change speed, and the like are examples of imaging conditions that change the image content in a moving image.

  The period specifying unit 220 specifies a first period that is a period in which the change speed of the imaging region in the moving image exceeds a predetermined reference value. Specifically, the period specifying unit 220 specifies a period in which the change speed of the imaging region specified by the change speed specifying unit 210 exceeds a predetermined reference value as the first period. In addition, the period specifying unit 220 specifies a first period that is a period in which the change speed of the imaging condition that changes the image content in the moving image exceeds a predetermined reference value. Specifically, a period in which the change speed of the imaging condition specified by the change speed specifying unit 210 exceeds a predetermined reference value is specified as the first period.

  The change area specifying unit 230 specifies a change area, which is an area where a change has occurred in the image content, based on the change speed of the imaging condition. For example, the change area specifying unit 230 specifies the change area based on the change speed and focus position of the focal position and the distance to the subject. For example, in a state in which a specific subject is focused, the image of the specific subject changes relatively greatly with only a slight change in focal length. For example, the image of the specific subject is relatively largely blurred only by a slight change in the focal length. On the other hand, an image of a subject that is not in focus at all remains blurred even if the focal length changes slightly, and does not change greatly as image content. Therefore, when the focal position changes beyond a predetermined reference value, the change area specifying unit 230 may specify the area of the subject that is more focused as a change area with higher priority.

  The depth of field is enlarged or reduced by the diaphragm. For this reason, when the aperture is changed, among the subjects located in the vicinity of the focused specific subject, the subject present in a position away from the specific subject in the optical axis direction may change the image content greatly. For example, an image of a subject that is substantially focused in a state where the aperture is stopped deviates from the depth of field by opening the aperture and becomes relatively blurred. The change area specifying unit 230 specifies a part of the image area as the change area when at least one of the focus position and the stop is changed. On the other hand, when the zoom magnification, the panning angle, the tilt angle, and the imaging position are changed, the change area specifying unit 230 specifies the entire image area as the change area.

  The compression strength determination unit 240 controls the compression strength by the compression unit 260. Specifically, the compression strength determination unit 240 determines a higher compression strength as the compression strength to be applied to the first period than the second period, which is a period other than the first period in the moving image. The compression unit 260 compresses each region according to the compression strength determined by the compression strength determination unit 240. That is, the compression unit 260 compresses the moving image by applying a higher compression strength in the first period than in the second period. The compression strength determination unit 240 determines that a filter process for increasing the frame correlation with other frames should be applied to the frames in the first period. In addition, the compression strength determination unit 240 makes the value of the quantization step applied to the frame in the first period larger than the value of the quantization step applied to the frame in the second period. As described above, the compression strength determination unit 240 determines the compression strength of irreversible compression in the compression unit 260. The compressive strength is determined by whether or not to apply the above-described filter, the quantization step, and the like.

  For example, during the period when the zoom magnification is changed at high speed, the image content changes greatly between frames. For this reason, when a moving image is reproduced, the human eye may not be able to fully recognize the quality of the frame. According to the imaging apparatus 100, high compression strength is applied during a period when the zoom magnification is changed at high speed. For this reason, it is possible to compress moving image data with a higher compression strength without greatly affecting the image quality recognized by human eyes.

  A specific operation of the compression unit 260 will be described. The compression unit 260 includes a filter unit 250, an encoding unit 270, a synthesis control unit 280, and a synthesis unit 290. The encoding unit 270 includes a first encoding unit 271 and a second encoding unit 272.

  The filter unit 250 and the encoding unit 270 function as a compression processing unit that compresses with different compression strengths in the first period and the second period. Specifically, the filter unit 250 applies a filter process that increases the degree of correlation between frames to the frames in the first period. For example, the filter unit 250 may apply a stronger noise reduction filter to the first period frame than to the second period frame. As described above, the compression unit 260 applies the filter process for increasing the degree of correlation between frames to a frame captured in the first period among a plurality of frames constituting the moving image. Note that the filter unit 250 may apply the filter process to the change region specified by the change region specifying unit 230.

  The encoding unit 270 performs moving image encoding processing such as interframe encoding, intraframe encoding, quantization, and variable length encoding. Each image region may be subjected to compression coding according to the compression strength determined by the compression strength determination unit 240. For example, the compression strength determination unit 240 may determine the value of the quantization step applied to each period. The encoding unit 270 may perform quantization using the quantization step value determined by the compression strength determination unit 240.

  The encoding unit 270 encodes the first area and the second area in the image area of the frame in parallel. As an example, an image area other than the change area may be applied as the first area, and the change area may be applied as the second area. Specifically, the first encoding unit 271 compresses the first region, and the second encoding unit 272 compresses the second region. That is, the first encoding unit 271 and the second encoding unit 272 respectively compress the first area and the second area in parallel.

  Note that the encoding unit 270 may include as many encoders as the number of image regions to which different compression strengths are applied. For example, when different compression is applied to the change region and the region other than the change region, the encoding unit 270 is provided with two encoding units.

  The composition control unit 280 supplies the first compressed data generated by compressing the first area and the second compressed data generated by compressing the second area to the composition unit 290. The combining unit 290 combines the first compressed data and the second compressed data to generate compressed data obtained by compressing one image data. The synthesis control unit 280 controls the timing at which the synthesis unit 290 synthesizes the first compressed data and the second compressed data. For example, the synthesis control unit 280 buffers the compressed data output from the first encoding unit 271 and the second encoding unit 272, and compresses the compressed data to the synthesis unit 290 when the compressed data is output from all the encoding units. Output data all at once. The composition control unit 280 can absorb the time difference required for compression of each region. Thus, the composition control unit 280 controls the composition of the compressed data in the composition unit 290.

  As described above, the compression unit 260 compresses the moving image by applying a higher compression strength in the first period than in the second period. For this reason, it is possible to reduce the entire code amount in the moving image data without greatly affecting the image quality recognized by human eyes. Further, the compression unit 260 may adjust the bit rate or the like so that the entire code amount of the moving image data matches the target code amount. By controlling so that the entire code amount is adapted to the target code amount, a large amount of code can be allocated during a period in which the zoom magnification is not changed. For this reason, it is possible to further improve the image quality during a period in which the image quality is easily recognized.

  FIG. 3 schematically shows a period during which the compression strength is controlled based on the zoom magnification. In this figure, the zoom magnification does not change from when the frame 300 is captured until the frame 310 is captured. The ASIC 135 does not control the compression strength based on the zoom magnification for the frames photographed during this period. Specifically, the filter unit 250 applies a first intensity noise reduction filter to frames taken during this period. Also, the encoding unit 270 performs quantization by applying a relatively small quantization step value.

  In this figure, the zoom magnification changes from when the frame 311 is captured until the frame 321 is captured. The ASIC 135 determines a change in zoom magnification with respect to a frame shot during this period, and variably controls the compression strength based on the zoom magnification. A period in which the compression strength is variably controlled based on the zoom magnification may be referred to as a compression strength variable period. For example, the filter unit 250 applies a noise reduction filter having a second intensity higher than the first intensity to a frame shot during a period in which the zoom magnification change speed exceeds the reference value. Also, the encoding unit 270 applies a relatively large quantization step value to a frame shot during a period in which the zoom magnification change speed exceeds the reference value, and quantizes the frame.

  FIG. 4 shows an example of calculating the zoom magnification change speed. Here, the changing speed of the focal length of the lens group 122 included in the interchangeable lens 120 is used as an index value of the changing speed of the zoom magnification. In the example of this figure, the focal length is constant until time t0, and the change of the focal length is started from time t0. Reference numeral 410 indicates a change speed when the change of the focal length is completed at time t1, and reference numeral 420 indicates a case where the change of the focal distance is completed at time t2.

  The change speed specifying unit 210 calculates the change speed of the focal length by dividing the change width of the focal distance by the time width required for the change. For example, when f1 = 18 mm, f6 = 200 mm, and t1−t0 = 2 seconds, the focal length changing speed indicated by reference numeral 410 is 91 mm / s. On the other hand, when t2−t0 = 4 seconds, the focal length changing speed indicated by reference numeral 420 is 45.5 mm / s.

  As an example, when 90 mm / s is set as the reference value for the changing speed of the focal length, the period specifying unit 220 specifies the period t0 to t1 as the first period with respect to the change of the focal length indicated by the solid line in the drawing. To do. A period in which the changing speed of the focal distance is 90 mm / s or less, for example, a period until time t0 and a period after time t1 are the second period. On the other hand, when the focal length is changed as indicated by reference numeral 420, the entire period is specified as the second period.

  FIG. 5 shows an example of compression control based on the zoom magnification change speed. This example is an example of compression control when the zoom-out operation is continued after the zoom-up operation to return to the original zoom magnification. The scales of the vertical axis and the horizontal axis in this figure are the same as those in FIG. In this example, the period from time t0 to time t10 is the compression strength variable period.

  Specifically, the focal length is increased from f1 to f6 from time t0 to time t5, and is decreased from f6 to f1 from time t5 to time t10. In this example, the increasing speed of the focal length gradually increases during the period from time t0 to time t5. Further, in the period from the time t5 to the time t10, the focal distance decreasing speed is gradually decreased. In this example, it is assumed that the temporal development of the focal length is symmetric with respect to the time t5 for the sake of easy understanding.

  In this example, when 90 mm / s is set as the reference value for the changing speed of the focal length, the period from the time t3 to the time t7 is specified as the first period in the compression strength variable period (t0 to t10). That is, the change speed specifying unit 210 specifies a period in which the focal speed change speed, for example, the focal length change speed exceeds the reference value, as the first period. The period until time t3 and the period after time t7 are the second period.

  For example, when the user performs a zoom-up operation while gradually increasing the rotation speed of the zoom ring 24 and performs a zoom-out operation while gradually decreasing the rotation speed of the zoom ring 24, the temporal development of the focal length as in this example is achieved. can get. According to the processing described in connection with this figure, the low zoom strength is applied during a period when the zoom magnification changing operation speed is low and the image quality can be easily recognized by the human eye. It is possible to prevent the image quality of the image from greatly deteriorating. On the other hand, during the period when the zoom magnification change operation is fast and the image quality cannot be easily recognized by the human eye, high compression strength is applied. Large deterioration can be prevented in advance. Here, as the zoom magnification changing speed increases, the frame correlation decreases and the generated code amount increases. According to the imaging apparatus 100, since the high compression strength is applied during a period in which the generated code amount is large, the entire code amount may be effectively suppressed.

  FIG. 6 shows another example of compression control based on the zoom magnification change speed. This example is an example of compression control when a zoom-out operation is performed slowly following a relatively high-speed zoom-up operation to slightly reduce the zoom magnification. The vertical and horizontal scales in this figure are the same as those in FIG. In this example, the period from time t0 to time t6 is the compression strength variable period.

  Specifically, the focal length is increased from f1 to f6 from time t0 to time t3, and is decreased from f6 to f5 from time t3 to time t6. In this example, it is assumed that the increasing speed of the focal length is constant during the period from time t0 to time t3. In addition, it is assumed that the focal distance reduction rate is constant during the period from time t3 to time t6.

  In this example, when 90 mm / s is set as the reference value for the changing speed of the focal length, the period from time t0 to time t3 is specified as the first period in the variable compression strength period (t0 to t5). On the other hand, the period after time t3 is the second period.

  For example, when the user rotates the zoom ring 24 at once and performs a zoom-up operation, and then slowly returns the zoom ring 24 to perform a zoom-out operation, the time evolution of the focal length as in this example can be obtained. As an image effect, an effect of emphasizing a subject photographed when the zoom magnification is maximum is obtained. In this example, a relatively low compressive strength is applied to a frame during a period when the zoom ring 24 is slowly returned to zoom out. For this reason, it is possible to prevent the image quality during the period in which the subject is particularly emphasized from being lowered. For this reason, it is possible to apply a compressive strength in accordance with the user's intention to shoot.

  In the above description, the compression strength is controlled by following the change speed of the focal length in real time. However, it may be determined whether the average value of the changing speed of the focal length within a certain period exceeds a reference value. As an example, it may be determined whether or not the average value of the changing speed of the focal length within the variable compression strength period exceeds a reference value. For example, in this example, the average value of the changing speed of the focal length is (f6-f1 + f6-f5) / (t6-t0). When 90 mm / s is set as the reference value regarding the changing speed of the focal distance, the average value of the changing speed of the focal distance is determined to be equal to or less than the reference value. Therefore, all the periods from time t0 to time t6 are specified as the second period. In this case, the compression strength in the period from time t0 to time t6 is relatively small, and a high-quality moving image is obtained. In the case of performing high-speed zoom-up and low-speed zoom-out operations as in this example, the slower the zoom-out operation and the longer the time is spent, the smaller the average value of the change speed. When the slow zooming operation period is included, there is a possibility that the image was shot aiming at the video effect by a series of zoom operations. Therefore, by controlling the compression strength based on the change speed over a period during which a series of zoom operations are occurring, the user's intention may be properly reflected in the image quality.

  FIG. 7 shows another example of compression control based on the zoom magnification changing speed. This example is an example of compression control when a zooming operation is performed at a high speed in a relatively short time. The vertical and horizontal scales in this figure are the same as those in FIG. In this example, the period from time t0 to time t1 is the variable compression strength period.

  In this example, the changing speed of the focal length in the period from time t0 to time t1 exceeds the reference value. However, since the focal length is changed for a relatively short period, it is not specified as the first period. That is, the period specifying unit 220 specifies a period in which the change speed exceeds the reference value over a time longer than a predetermined time length as the first period.

  The reduction amount of the code amount by increasing the compression strength becomes smaller as the period for increasing the compression strength is shorter. Therefore, the shorter the period in which the focal length is changed beyond the reference value, the smaller the amount of code reduction by increasing the compression strength. By controlling to increase the compression strength on the condition that the changing speed of the focal length exceeds the reference value for a time longer than a predetermined time length, the compression strength is limited to a period where the code amount reduction effect is large. Can be increased. Moreover, it is possible to prevent the compressive strength from changing frequently.

  When different compressive strength is applied to consecutive periods, the compressive strength of a part of the period in contact with the boundary is gradually increased or decreased so that the range of change in compressive strength is not more than a predetermined value at the boundary between the consecutive periods. You may let me. For example, the compression unit 260 determines in advance a difference in compression strength in the boundary region between the first period and the second period by gradually decreasing or gradually increasing the compression strength applied to the boundary region between the first period and the second period. It may be less than the specified value. For example, the compression unit 260 may gradually increase the compression strength toward the boundary in a partial region within the second period following the first period. The compression unit 260 may gradually decrease the compression strength toward the boundary in a partial region within the first period following the second period.

  From FIG. 3 to FIG. 7, the change speed of the focal length of the lens group 122 is used as an index as an example of the change speed of the zoom magnification. However, the imaging region is changed not only by changing the focal length of the lens group 122 but also by changing the enlargement magnification by image processing called so-called electronic zoom or digital zoom. Therefore, the zoom magnification changing speed is a concept including the magnification changing speed. For example, the period specifying unit 220 may specify the first period based on the zoom magnification changing speed based on the combination of the focal length of the lens group 122 and the magnification.

  From FIG. 3 to FIG. 7, the case has been described in which the first period in which the compression strength is to be increased is specified based on the zoom magnification change speed, which is an example of the imaging region change speed. As described above, examples of the changing speed of the imaging area include a changing speed of the panning angle, a changing speed of the tilt angle, and a changing speed of the imaging position in moving image shooting in addition to the changing speed of the zoom magnification. Therefore, instead of specifying the first period using the focal length change speed described with reference to FIGS. 3 to 7 as an index, the first period may be specified using the panning angle change speed as an index. Further, the first period may be specified using the change rate of the tilt angle as an index. The panning angle changing speed and the tilt angle changing speed are examples of the imaging direction changing speed. Further, the first period may be specified using the change speed of the imaging position as an index. As described above, the first period can be appropriately determined based on the so-called shooting scene in moving image shooting and the change speed of the composition.

  The zoom magnification changing speed is also an example of the imaging condition changing speed that changes the image content. As described above, examples of the changing speed of the imaging condition that changes the image content include a changing speed of the focal position, a changing speed of the depth of field, and the like in addition to the changing speed of the zoom magnification. Therefore, instead of specifying the first period using the focal length change speed described with reference to FIGS. 3 to 7 as an index, the first period may be specified using the focal position change speed as an index. Further, the first period may be specified using the change speed of the depth of field as an index. As described above, the first period can be appropriately determined based on the change speed of the imaging condition that changes the entire image content.

  The first period may be specified based on any combination of the zoom magnification changing speed, the focal position changing speed, the depth of field changing speed, the imaging direction changing speed, and the imaging position changing speed described above. . For example, the period specifying unit 220 is a period in which at least one of the zoom magnification changing speed, the focal position changing speed, the depth of field changing speed, the imaging direction changing speed, and the imaging position changing speed exceeds the reference value. May be specified as the first period.

  FIG. 8 shows an example of correction data related to the focus change speed in a table format. As an example, this data is stored in the interchangeable lens 120 as a part of lens specific information. For example, this data may be stored in the nonvolatile system memory of the lens MPU 123. This data is provided to the camera MPU 133 via the lens MPU 123.

  When the first period is specified based on the change speed of the focal position, it is desirable to use an index value that appropriately reflects the amount of change in the image content as the index value related to the focal position. For example, when the rotational speed of the focus ring 25 is used as an index value for the focal position change speed, the amount of change in image content may differ depending on the focal length of the lens group 122 even if the rotational speed is the same. For example, the image change amount corresponding to the rotation amount of the focus ring 25 may be greatly different between the tele end and the wide end.

  This data associates the focal length of the lens group 122 with the correction coefficient. For example, a correction coefficient having a larger value is associated with a longer focal length. As an example, when the rotation speed of the focus ring 25 is used as the index value of the focus position, the change speed specifying unit 210 uses the correction coefficient stored in association with the focal length of the lens group 122 as the rotation speed of the focus ring 25. The value multiplied by the speed is calculated as the change speed. The period specifying unit 220 specifies a period in which the change speed exceeds the reference value as the first period. For this reason, the period specifying unit 220 specifies the first period using an index value that appropriately reflects the degree of change in the image content in consideration of the focal length of the lens group 122 as the index value of the focal position change speed. can do.

  FIG. 9 schematically shows an example of a scene for determining the change speed for each image area. A frame 900 is one frame constituting a moving image. In this example, it is assumed that a person is focused and a subject such as a distant mountain or cloud is not focused.

  Here, when the focal position is slightly shifted from the position of the person to the extent that the distant subject is not focused, the image of the distant subject does not change much. On the other hand, the blur of the person image greatly increases. That is, by the operation of the focal position, the image content of the partial area where the focused subject is photographed changes more significantly than the image content of the partial area where the unfocused subject is photographed. Therefore, the change area specifying unit 230 specifies the partial area 910 including the focused subject image among the image areas in the frame 900 as the change area. The compression unit 260 applies higher compression strength to the change region than the non-change region which is the partial region 920 other than the change region. Therefore, according to the imaging apparatus 100, it is possible to appropriately specify a partial region whose image content changes at high speed in accordance with a change in the imaging condition, and to apply a higher compression strength to the partial region than the other partial regions. it can.

  When different compression strengths are applied to adjacent image regions, the compression strength of a part of the region in contact with the boundary is gradually increased so that the change width of the compression strength at the boundary between the adjacent image regions is equal to or less than a predetermined value. Or it may be gradually reduced. For example, the compression unit 260 may determine a difference in compressive strength between the change region and the non-change region boundary region by gradually decreasing or gradually increasing the compression strength applied to the boundary region between the change region and the non-change region. It may be less than the value. For example, the compression unit 260 may gradually increase the compression strength toward the boundary in a part of the non-change region adjacent to the change region. The compression unit 260 may gradually decrease the compressive strength toward the boundary in a part of the change area adjacent to the non-change area.

  FIG. 10 shows a processing flow from the start to the end of the imaging apparatus 100. This flow is started when a power button as a part of the operation input unit 141 is turned on. This flow is executed by controlling each part of the imaging apparatus 100 mainly by the camera MPU 133.

  In step S1000, the camera MPU 133 starts initial setting. For example, the camera MPU 133 develops various parameters for controlling the imaging apparatus 100 from the system memory 139 to the SDRAM 136. Further, lens specific information is acquired from the lens group 122 and developed as lens parameters in the SDRAM 136. The lens specific information includes the correction coefficient described with reference to FIG. Then, the camera MPU 133 sets operating conditions of each unit of the imaging apparatus 100 based on the state of the operation input unit 141 including, for example, an imaging mode switch and the developed various parameters. For example, the imaging mode is set according to the position of the imaging mode switch. In addition, imaging conditions and image processing conditions are set according to the imaging mode. For example, examples of the imaging mode include a person shooting mode suitable for person shooting, an animal shooting mode suitable for animal shooting, a landscape shooting mode suitable for landscape shooting, and the like.

  Subsequently, in step S1002, display on the display unit 138 is performed. For example, the camera MPU 133 causes the display unit 138 to display setting information that identifies the imaging mode, imaging conditions, image processing conditions, and the like.

  Subsequently, in step S1004, the camera MPU 133 determines a user instruction to the operation input unit 141. If the user instruction is an instruction to execute various settings, the camera MPU 133 performs the instructed setting process (step S1006). Examples of the setting process include a process for setting an imaging mode, a process for setting an imaging condition, a process for setting an image processing condition, a process for setting a frame rate in moving image shooting, and the like. Further, as the setting process, a process in which the user designates whether or not to perform variable control of the compression strength based on the above-described imaging region or imaging condition change speed can be exemplified.

  If it is determined in step S1004 that the user instruction is an instruction relating to imaging execution, processing relating to imaging execution is performed (step S1012). Examples of instructions related to shooting execution include operations on a live view button, a moving image shooting button, and a release button. If the user instruction in step S1004 is an instruction to execute image reproduction, reproduction processing is executed (step S1022). Examples of the reproduction process include a process of displaying an image recorded in the external memory 160 as a thumbnail, a process of displaying an image instructed by the user, and the like.

  If there is no user instruction in step S1004, a photometric process for measuring ambient light is performed (step S1032). For example, the appropriate exposure value is determined based on the photometric result of ambient light. This photometric process is executed at regular time intervals to determine the current appropriate exposure value. Information indicating the current exposure value determined in step S1032 is displayed on the display unit 138 in real time and presented to the user.

  When the processing of step S1006, step S1012, step S1022, and step S1032 is completed, the process proceeds to step S1008, and it is determined whether or not to turn off the power. For example, it is determined that the power is turned off when the power button is switched to the OFF position, or when there is no user instruction for a predetermined period after the power is turned on. If it is determined that the power is to be turned off, the operation is terminated. If it is determined that the power is not to be turned off, the process proceeds to step S1004.

  FIG. 11 shows an example of an operation flow in moving image shooting. In this example, as part of the processing in step S1012, an operation flow in the case where an instruction for moving image shooting is given is shown. This flow is started when the moving image shooting button is pressed.

  In step S <b> 1102, the camera MPU 133 instructs each unit of the image capturing apparatus 100 to shoot a moving image, thereby starting moving image shooting. When moving image shooting is started, frames are sequentially taken into the SDRAM 136. In the ASIC 135, it is determined whether or not the frame subject to image processing has been taken into the SDRAM 136 (step S1104).

  When a frame subject to image processing is captured, the ASIC 135 determines whether or not a zoom operation has been performed at the shooting timing of the frame (step S1106). For example, a period during which the zoom operation is performed is specified based on a zoom operation that instructs to change the focal length of the lens group 122 provided through the camera MPU 133, that is, a zoom operation that instructs to change the magnification by so-called digital zoom. It is determined whether or not the frame shooting timing is included.

  When the zoom operation is performed at the frame shooting timing, the period specifying unit 220 determines whether or not the zoom magnification changing speed exceeds the reference value within the period (step S1108). When it is determined that the zoom magnification changing speed exceeds the reference value, the period specifying unit 220 determines whether or not the zoom operation has continued for a predetermined period (step S1110). When the zoom operation continues for a predetermined period, the compression strength determination unit 240 sets the compression strength to a large value (step S1112). More specifically, the filter unit 250 is set to apply a strong noise reduction process.

  Subsequently, in step S1120, the compression unit 260 compresses each region according to the compression strength determined by the compression strength determination unit 240. Details of this processing will be described later with reference to FIG.

  Subsequently, in step S1122, it is determined whether or not to end the moving image shooting process. For example, when the moving image shooting button is pressed again and there is no image processing target frame, it is determined that the moving image shooting process is to be ended. When it is determined that the video shooting process is to be terminated, the operation flow is terminated. If it is determined not to end the moving image shooting process, the process advances to step S1104.

  If it is determined in step S1106 that the zoom operation has not been performed at the photographing timing, the compression strength determination unit 240 sets the compression strength to a small value (step S1114), and the process proceeds to step S1120. If it is determined in step S1108 that the zoom magnification change speed does not exceed the reference value, the compression strength determination unit 240 sets the compression strength to a small value (step S1114), and the process proceeds to step S1120.

  If it is determined in step S1110 that the zoom operation has not been continued for a predetermined period, the compression strength determination unit 240 holds the compression strength setting and compression, and the process proceeds to step S1104. If it is determined in step S1110 for the subsequent frame that the zoom operation has continued for a predetermined period, the compression strength determination unit 240 compresses the frame for which the compression strength setting is suspended. Set the strength high. On the other hand, if it is determined in step S1108 for the subsequent frame that the change speed has become the reference value or less without continuing the zoom operation for a predetermined period (step S1108: N), or the determination in step S1106 If it is determined in step S1106 that the zoom operation is not performed (step S1106: N), the compression strength is set to a small value for the frame for which the compression strength setting is suspended (step S1114), and the process proceeds to step S1120. .

  FIG. 12 shows an example of an operation flow in which the compression unit 260 compresses a frame. This flow is a detailed operation flow in step S1120 of FIG. Here, as described with reference to FIG. 9, an operation in a case where a part of an image area of one frame is specified as a change area will be described.

  In step S1200, the compression unit 260 reads a frame from the SDRAM 136. Subsequently, in step S1202, the filter unit 250 performs noise reduction processing on the read frame. The filter unit 250 applies the noise reduction process of the second strength to the frame in which the compression strength is set to be large, and the noise of the first strength that is smaller than the second strength is applied to the frame in which the compression strength is set to be small. Apply reduction processing. Here, the filter unit 250 may apply the second intensity noise reduction process to the change area, and apply the first intensity noise reduction process to an area other than the change area.

  Subsequently, data in a region other than the changed region is supplied to the first encoding unit 271 (step S1204), and data in the changed region is supplied to the second encoding unit 272 (step S1206).

  In the first encoding unit 271, for example, encoding processing including quantization is performed on data in a region other than the change region. The first encoding unit 271 may perform encoding processing such as DCT conversion, interframe compression, or intraframe compression. In the second encoding unit 272, the same encoding process as that of the first encoding unit 271 is performed on the data in the change area. The first encoding unit 271 and the second encoding unit 272 apply a quantization step value corresponding to the compression strength determined for each region by the compression strength determination unit 240, for example, and perform quantization on each region. May be done. The data of each area encoded by the first encoding unit 271 and the second encoding unit 272 is temporarily stored in the SDRAM 136 as compressed data. When the first encoding unit 271 and the second encoding unit 272 complete the encoding, the synthesis control unit 280 is notified that the encoding is completed.

  In step S1208, the synthesis control unit 280 determines whether the encoding of both the first encoding unit 271 and the second encoding unit 272 has been completed. If at least one of the first encoding unit 271 and the second encoding unit 272 has not been completed, the determination in step S1208 is repeated. When there is an encoding completion notification from both the first encoding unit 271 and the second encoding unit 272, the synthesis control unit 280 sends the compressed data of each area temporarily stored in the SDRAM 136 to the synthesis unit 290. Supply and synthesize (step S1210). Subsequently, the compressed data of one frame generated by the combining unit 290 combining the compressed data is stored in the SDRAM 136 (step S1212), and this operation flow ends.

  In particular, in FIGS. 11 and 12, compression is determined by determining the compression strength while shooting a moving image. However, for example, a frame in a period in which the compression strength is increased may be marked, and the marked frame may be compressed by applying a high compression strength. For example, after moving image shooting, the marked frame may be compressed by applying high compression. Specifically, a frame to be compressed with a high compression strength is accompanied by information indicating that the frame should be compressed with a high compression strength, and the frame with the information is compressed by applying a high compression strength. Good. Further, information specifying a frame to be compressed with a high compression strength may be temporarily stored in the SDRAM 136 or the like, and the frame specified by the stored information may be compressed by applying a high compression strength. . After the marked moving image data is once recorded in the external memory 160, the recorded moving image data may be read and compressed.

  According to the imaging apparatus 100 described above, the overall image quality can be prevented while applying a high compression strength when a user operation that changes the image content is performed, thereby preventing a significant deterioration in the visual image quality. Can be reduced. According to the imaging apparatus 100, for example, since the determination is based on a user operation or the like, it is possible to specify a period during which the compression height is increased without substantially analyzing the image.

  The processing described in relation to the imaging apparatus 100 of the present embodiment can be realized by causing each unit of the imaging apparatus 100, for example, a processor such as the camera MPU 133 to operate according to a program. The processing by the ASIC 135 can also be realized by the processor operating according to the program. That is, the process can be realized by a so-called computer device. The computer apparatus loads the program for controlling the execution of the above-described processing, and controls the hardware according to the loaded program, whereby the processing is realized by the cooperative operation of the software and the hardware. The computer device can load the program by reading a computer-readable recording medium storing the program.

  In the present embodiment, the imaging apparatus 100 as a lens interchangeable single-lens reflex camera has been described. As an imaging device, various digital cameras, mirrorless single-lens cameras, video cameras, mobile phones with an imaging function, portable information terminals with an imaging function, amusement devices such as game machines with an imaging function, etc. Equipment can be targeted. Further, as a device that executes image processing corresponding to the image processing described as processing in the ASIC 135, in addition to a personal computer, an image recording apparatus such as a television or a recorder with a recording function can be applied. Further, the image data to be processed may be not only image data that has not been subjected to substantial image processing such as a RAW image, but image data that has been subjected to various image processing such as compression.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

24 zoom ring, 25 focus ring, 100 imaging device, 120 interchangeable lens, 121 lens mount contact, 130 camera body, 131 camera mount contact, 122 lens group, 132 image sensor, 123 lens MPU, 133 camera MPU, 134 A / D Converter, 135 ASIC, 136 SDRAM, 137 Display control unit, 138 Display unit, 139 System memory, 140 Drive unit, 141 Operation input unit, 142 Sensor unit, 145 Connection interface, 170 Power supply, 160 External memory, 210 Change speed specification Unit, 220 period specifying unit, 230 change region specifying unit, 240 compression strength determining unit, 250 filter unit, 260 compressing unit, 270 encoding unit, 271 first encoding unit, 272 second encoding unit, 280 synthesis control unit , 29 Combining unit, 300,310,311,321 frame, 900 frame, 910 partial area, 920 partial area

Claims (10)

A period specifying unit that specifies a first period in which a change speed of an imaging region in a moving image exceeds a predetermined reference value;
A compression unit that compresses a moving image by applying a higher compression strength to the first period than the second period, which is a period other than the first period;
An image processing apparatus comprising: A period specifying unit that specifies a first period in which a change speed of an imaging condition that changes image content in a moving image exceeds a predetermined reference value;
A compression unit that compresses a moving image by applying a higher compression strength to the first period than the second period, which is a period other than the first period;
An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein the period specifying unit specifies a period in which the change speed exceeds the reference value over a time longer than a predetermined time length as the first period. The period specifying unit is a period in which at least one of a zoom magnification change speed, a focus position change speed, a depth of field change speed, an imaging direction change speed, and an imaging position change speed exceeds the reference value. The image processing apparatus according to claim 1, wherein the image processing apparatus is specified as the first period. The image according to any one of claims 1 to 4, wherein the compression unit applies a filter process for increasing a frame correlation to a frame captured in the first period among a plurality of frames constituting the moving image. Processing equipment. The compression unit determines in advance a difference in compression strength at the boundary between the first period and the second period by gradually decreasing or gradually increasing the compression strength applied to the boundary period between the first period and the second period. The image processing apparatus according to claim 1, wherein the image processing apparatus is set to be equal to or less than a predetermined value. The compression unit is
A compression processing unit that compresses in parallel the first image region and the second image region in the image constituting the moving image;
A combining unit that combines the first compressed data generated by compressing the first image area and the second compressed data generated by compressing the second image area;
A synthesis control unit that controls the timing at which the synthesis unit synthesizes the first compressed data and the second compressed data;
The image processing apparatus according to claim 1, comprising: An imaging unit that captures an image;
An image pickup apparatus comprising the image processing apparatus according to claim 1. Identifying a first period in which a change speed of an imaging region in a moving image exceeds a predetermined reference value;
Applying a higher compression strength to the first period than the second period, which is a period other than the first period, to compress the moving image;
A program that causes a computer to execute. Identifying a first period that is a period in which a change speed of an imaging condition that changes image content in a moving image exceeds a predetermined reference value;
Applying a higher compression strength to the first period than the second period, which is a period other than the first period, to compress the moving image;
A program that causes a computer to execute.

Images