JP5464125B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method that store image data generated by continuous imaging (continuous shooting) by performing compression encoding processing by intra-screen predictive encoding.

  In an imaging device (digital imaging device) that stores captured images as digital data (image data), the data area of the storage medium is effectively used by storing the image data after compressing and encoding the data to reduce the data capacity. It has a function to do. As one of such compression encoding methods, there is an intra-screen predictive encoding method called JPEG (Joint Photographic Experts Group) compression. This method is mainly used when compressing still images, but there is also a method called Motion JPEG that applies this to moving images. These encoding methods have poor compression efficiency at the same bit rate compared to MPEG (Moving Picture Experts Group), etc., but since difference information between screens is not used, processing at the time of encoding and decoding is light. There is an advantage that any part can be easily edited. The present invention is based on the premise that this intra-picture encoding method such as JPEG is used. Some imaging apparatuses have a function of continuously capturing images by continuously pressing a release switch.

  In such continuous imaging, a large amount of image data is generated at one time. Therefore, the compression encoding process must be appropriately controlled so that the data capacity of the generated image data is within a desired data capacity. Conventionally, a plurality of captured image data (raw data) is provisionally subjected to compression encoding processing using predetermined compression parameters, the parameters are changed according to the result, and compression encoding is performed again. The process of performing the process is repeated, and the process of bringing the data capacity of all continuously captured image data close to the desired data capacity has been performed (for example, Patent Document 1).

  In addition, the compression encoding process of the image signal is performed with the maximum value and the minimum value of the initial value of the search range of the compression rate, respectively, and the data capacity obtained by the compression encoding process with the maximum value and the compression encoding process with the minimum value are performed. Also known is a technique for approximating the correlation between the compression rate and the data volume after compression coding processing based on the obtained data volume, and estimating the compression rate candidate that provides the target data capacity after compression coding processing (For example, Patent Document 2).

  In addition, in order to realize a short-time compression encoding process, the image data is continuously extracted and the compression rate is calculated, and the next image data is compressed according to the compression rate calculated immediately before the imaging operation is performed. A technique for performing an encoding process is disclosed (for example, Patent Document 3). Similarly, there is also disclosed a technique for comparing the data capacity of one image data after compression encoding processing with a target value and determining the next compression rate so that the data capacity substantially matches the target value (for example, Patent Document 4).

JP 2008-141591 A Japanese Patent Laid-Open No. 10-150633 JP-A-11-122573 JP 2000-125255 A

  As described above, in order to store a large number of continuously captured image data in a storage medium having a limited data capacity, the captured image data must be subjected to compression encoding processing. In addition, in order to store a large number of continuously captured image data in a desired data capacity, it is necessary to appropriately control the compression rate in the compression encoding process. In the imaging apparatus, for example, compression encoding processing is performed according to a preset target compression rate. However, depending on the image data, the target compression rate and the actual compression rate may deviate significantly.

  In order to fit in the desired data capacity, it is necessary to correct such a shift in the compression ratio. For example, in the techniques of Patent Documents 1 and 2, in addition to provisional compression encoding processing, compression based on the result is performed. Since the encoding process is repeatedly performed, there is a problem that it takes a long time to determine the final compression rate. In the techniques of Patent Documents 3 and 4, since the compression rate of image data is predicted based only on the compression rate of the immediately preceding image data, irregular images are captured and the amount of movement between images is large. In some cases, the prediction accuracy of the compression rate is low.

  In view of such a problem, an object of the present invention is to provide an imaging apparatus and an imaging method capable of performing compression encoding processing at the time of continuous imaging with high accuracy and high speed.

In order to solve the above problems, the present invention provides the following imaging device and imaging method.
(1) An imaging unit that generates a plurality of image data continuous in the time direction by continuous imaging, and a compression rate of the image data is corrected based on a correction element for correcting the image data. An image processing unit that compresses and encodes the data, a data control unit that stores the compressed and encoded image data in an image storage unit, and a current compression rate that is an actual compression rate of the compressed and encoded image data A current compression rate deriving unit for deriving the current compression rate, a current compression rate holding unit for continuously holding the current compression rate in a time direction, and the plurality of held current compression rates, or the plurality of held current compression rates. A next compression rate prediction unit that predicts a next compression rate, which is a compression rate of image data to be compressed and encoded next time, from the compression rate and the compression rate of image data to be compression encoded this time, and the correction based on the next compression rate Correction element derivation for deriving elements Imaging apparatus characterized by comprising: a part, the.
(2) The imaging apparatus according to (1), wherein the next compression rate prediction unit averages a predetermined number of the current compression rates and uses the average value as the next compression rate.
(3) The next compression rate prediction unit averages a difference value between the current compression rates adjacent in the time direction of a predetermined number of the current compression rates, and adds the average value to the latest current compression rate. The image pickup apparatus according to (1), wherein the next compression rate is set.
(4) The next compression rate prediction unit averages a change rate of the current compression rate that is adjacent in a time direction to a predetermined number of the current compression rates, and multiplies the average value by the latest current compression rate. The image pickup apparatus according to (1), wherein the next compression rate is set.
(5) The next compression ratio prediction unit averages a predetermined number of the current compression ratios, and averages the difference values of the current compression ratios adjacent to each other in the time direction of the current compression ratio. 2 or 3 processes selected from the group of processes for adding to the current compression ratio, and for multiplying the latest current compression ratio by the average value of the change rates of the current compression ratio adjacent in the time direction of the current compression ratio Is switched according to a change mode of the current compression ratio, and the next compression ratio is predicted, and the imaging apparatus according to (1) above.
(6) The image processing unit is a frequency response processing unit that converts a frequency domain characteristic of image data based on a frequency response characteristic as the correction element, and the correction element derivation unit includes the next compression ratio and The imaging apparatus according to any one of (1) to (5), wherein the imaging apparatus is a frequency response deriving unit that derives the frequency response characteristics based on a target compression rate.
(7) a frequency response table in which a plurality of predetermined frequency response characteristics, a difference value between the next compression ratio and the target compression ratio, and a plurality of the frequency response characteristics are associated with each other; The frequency response deriving unit selects one frequency response characteristic using the frequency response table, wherein the frequency response table is (6).
(8) The image processing unit is a gradation processing unit that converts a gradation of image data based on a gradation characteristic as the correction element, and the correction element derivation unit includes the next compression rate and the target compression. The imaging apparatus according to any one of (1) to (5), wherein the imaging apparatus is a gradation derivation unit that derives the gradation characteristics based on a rate.
(9) a gradation table in which a plurality of gradation characteristics determined in advance, a difference value between the next compression rate and the target compression ratio, and a plurality of gradation characteristics are associated with each other; The imaging apparatus according to (8), wherein the gradation deriving unit selects one gradation characteristic using the gradation table.
(10) The image processing unit is a compression processing unit that performs compression encoding processing on image data based on a quantization table as the correction element, and the correction element derivation unit includes the next compression rate and the target The imaging apparatus according to any one of (1) to (5), wherein the imaging apparatus is a quantization table deriving unit that derives the quantization table based on a compression rate.
(11) a plurality of predetermined quantization tables, and a quantization correspondence table in which a difference value between the next compression rate and the target compression rate is associated with the plurality of quantization tables, The imaging apparatus according to (10), wherein the quantization table deriving unit selects one quantization table using the quantization correspondence table.
(12) The data control unit stores, in the image storage unit, only the image data that has been compression-encoded lastly among the plurality of image data that has been compression-encoded. The imaging device according to any one of 11).
(13) A plurality of image data continuous in the time direction is generated by continuous imaging, the compression rate of the image data is corrected based on a correction element for correcting the image data, and an intra-screen predictive encoding method is used. Compression-encoding, storing the compression-encoded image data in an image storage unit, deriving a current compression rate that is an actual compression rate of the compression-encoded image data, and setting the current compression rate in the time direction A plurality of the current compression rates held in succession, or the plurality of held current compression rates, or the plurality of held current compression rates and the compression rate of the image data to be compression encoded this time, An imaging method characterized by predicting a next compression ratio, which is a compression ratio, and deriving the correction element based on the next compression ratio.

  As described above, according to the present invention, it is possible to perform compression encoding processing at the time of continuous imaging with high accuracy and high speed.

1 is a functional block diagram illustrating a schematic configuration of an imaging apparatus according to a first embodiment. It is explanatory drawing for demonstrating operation | movement of the next compression rate estimation part. It is explanatory drawing for demonstrating operation | movement of the next compression rate estimation part. It is explanatory drawing for demonstrating operation | movement of the next compression rate estimation part. It is explanatory drawing for demonstrating correction | amendment of the compression rate by a frequency response characteristic. It is explanatory drawing for demonstrating correction | amendment of the compression rate by a gradation characteristic. It is explanatory drawing for demonstrating correction | amendment of the compression rate by a quantization table. It is the functional block diagram which showed the other example of the imaging device. It is the flowchart which showed the flow of the process of the imaging method concerning 1st Embodiment. It is the functional block diagram which showed the schematic structure of the imaging device concerning 2nd Embodiment. It is a timing chart for demonstrating the change of the production speed of image data. It is a timing chart for explaining other examples of change of the generation speed of image data. It is the flowchart which showed the flow of the process of the imaging method concerning 2nd Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  In the present embodiment, “image data” refers to data for one screen generated by one-time imaging by the imaging device of the imaging unit. “Imaging” refers to an operation of generating image data through the imaging unit, and “memory” refers to an operation of storing the generated image data in a storage medium.

  Further, “continuous imaging” refers to a series of operations for sequentially generating image data in the time direction, and for example, one or more image data is generated per second. In the present embodiment, the speed at which continuous imaging is performed and the image data is continuously stored is referred to as an imaging storage speed, and the imaging storage speed has a value of, for example, 60 frames / sec. “Single-shot imaging” refers to an operation for generating at least one piece of image data, and may include an operation for storing that single piece of image data, but before storing the image data, the image data is generated. It is not excluded to perform the operation to be performed multiple times.

(First embodiment: imaging apparatus 100)
FIG. 1 is a functional block diagram illustrating a schematic configuration of the imaging apparatus 100 according to the first embodiment. The imaging apparatus 100 includes an imaging unit 104, a synchronization signal generator (SSG) 106, a timing generator (TG) 108, a release switch 110, a Y / C processor 112, a frequency Response processing unit 114, gradation processing unit 116, compression processing unit 118, system internal bus 120, image memory 122, data control unit 124, image storage unit 126, display unit 128, and current compression rate Deriving unit 130, current compression rate holding unit 132, next compression rate prediction unit 134, frequency response deriving unit 136, frequency response holding unit 138, tone derivation unit 140, tone holding unit 142, quantum A quantization table deriving unit 144 and a quantization table holding unit 146.

  Further, the frequency response processing unit 114, the gradation processing unit 116, and the compression processing unit 118 correct the compression rate of the image data based on the correction element, and perform compression encoding using the intra prediction encoding method. The frequency response deriving unit 136, the gradation deriving unit 140, and the quantization table deriving unit 144 function as a correction element deriving unit that derives a correction element. Here, the correction element is one or more elements selected from the group of frequency response characteristics, gradation characteristics, and quantization tables used in the image processing unit. The frequency response characteristics, gradation characteristics, and quantization table will be described in detail later.

  The data control unit 124, the current compression rate deriving unit 130, the next compression rate prediction unit 134, the frequency response deriving unit 136, the gradation deriving unit 140, and the quantization table deriving unit 144 may be configured by hardware. The program may be implemented by software through a central processing unit (CPU) that operates in cooperation with a ROM storing a program, a RAM as a work area, and the like. In FIG. 1, the gradation processing unit 116 is cascaded subsequent to the frequency response processing unit 114, but this order may be reversed.

  The imaging unit 104 includes a lens and an imaging element that photoelectrically converts incident light incident through the lens and generates image data. For example, the imaging unit 104 generates and generates a plurality of image data continuous in the time direction by continuous imaging. The plurality of pieces of image data are sequentially transmitted to the Y / C processing unit 112. The synchronization signal generation unit 106 generates a horizontal synchronization signal and a vertical synchronization signal in synchronization with the internal clock in correspondence with the horizontal size and the vertical size of the generated image data, and transmits them to the timing generation unit 108.

  The timing generation unit 108 generates a control signal that changes the clock timing of the image sensor of the imaging unit 104 based on the horizontal synchronization signal and the vertical synchronization signal generated by the synchronization signal generation unit 106. The imaging unit 104 generates image data according to the control signal. The release switch 110 is configured by a button switch or the like, accepts an operation input from the photographer, and generates a control signal that serves as a trigger (trigger) for generating image data in the imaging unit 104. Further, in the imaging apparatus 100 of the present embodiment, continuous imaging can be performed by the photographer continuously holding the release switch 110.

  The Y / C processing unit 112 performs processing for separating and converting the image data into luminance data (Y) and color data (C). The frequency response processing unit 114 converts the frequency domain characteristics of the image data through the Y / C processing unit 112 based on the frequency response characteristics derived by the frequency response deriving unit 136 described later. The gradation processing unit 116 converts the gradation of the image data subjected to the frequency response characteristic conversion processing based on the gradation characteristic derived by the gradation deriving unit 140 described later.

  The compression processing unit 118 performs compression encoding processing based on the quantization table for the image data subjected to the gradation characteristic conversion processing according to the JPEG compression method, and the image data subjected to the compression encoding processing is The image data is transmitted to the image memory 122 through the system internal bus 120. Since the JPEG compression method is an existing technology, detailed description thereof is omitted here. The compression processing unit 118 can employ various existing still image compression methods other than the JPEG compression method.

  The image memory 122 is configured by a buffer such as SRAM or DRAM, and temporarily stores the transmitted image data. The data control unit 124 transfers the image data temporarily held in the image memory 122 to the image storage unit 126 through the system internal bus 120 to hold the image data, or the image data held in the image memory 122. An image based on the data is displayed on the display unit 128.

  The image storage unit 126 is configured by a storage medium such as a flash memory or an HDD (Hard Disk Drive) that is formed integrally with the imaging apparatus 100 and holds one or a plurality of image data. Further, as the image storage unit 126, an optical disk medium such as a DVD (Digital Versatile Disc) and a BD (Blu-ray Disc), a magnetic medium such as a magnetic tape and a magnetic disk, a flash memory, a portable HDD, etc. The external storage medium may be applied. Although the HDD is an apparatus accurately, in the present embodiment, it is treated as synonymous with other storage media for convenience of explanation. The display unit 128 includes a liquid crystal display, an organic EL (Electro Luminescence) display, or the like.

  The current compression rate deriving unit 130 compares the data capacity of the image data before the compression encoding process is performed by the compression processing unit 118 with the data capacity of the image data after the compression encoding process is performed, The current compression rate that is the latest and actual compression rate in the compression encoding process is derived. The derived current compression rate is sequentially held in the current compression rate holding unit 132. The current compression rate holding unit 132 holds a plurality of current compression rates derived by the current compression rate deriving unit 130 continuously in the time direction. Therefore, the current compression rate holding unit 132 holds the current compression rate for a plurality of past times.

  The next compression rate prediction unit 134 obtains the current compression rate held in the current compression rate holding unit 132 from the current compression rate holding unit 132 for the past multiple times including the most recent current compression rate, and the plurality of current compression rates. The next compression rate which is the compression rate of the image data to be compressed and encoded next time is predicted from the rate or the plurality of held current compression rates and the compression rate of the image data to be compression encoded this time. The prediction of the next compression rate is performed using (1) average value, (2) difference value, and (3) change rate of a plurality of current compression rates. Hereinafter, the derivation process of the next compression rate in each will be described in detail.

  2-4 is explanatory drawing for demonstrating operation | movement of the next compression rate estimation part 134. FIG. In particular, FIG. 2 shows a case where (1) an average value is used, FIG. 3 shows a case where (2) a difference value is used, and FIG. 4 shows a case where (3) a change rate is used. Here, the next compression rate prediction unit 134 predicts the next compression rate 152 based on the current compression rate 150 for five times, including the most recent current compression rate and the current compression rate for the past four times. However, the number of current compression rates 150 used in the next compression rate prediction unit 134 is not limited to 5 and may be 2 or more.

…（数式１）
こうして求められた平均値ｆが次圧縮率１５２となる。このように平均値を用いると、イレギュラーな画像が撮像された等、圧縮率に短時間に大きな変化があった場合であっても、その変化に安易に追従することなく、適切な次圧縮率１５２を導出することが可能となる。 When the average value is used as shown in FIG. 2, the next compression rate prediction unit 134 uses the following formula 1 to calculate the current compression rate 150 for the five times including the latest (a, b, c, d, The average value f of e) is obtained.

... (Formula 1)
The average value f thus obtained becomes the next compression rate 152. When the average value is used in this way, even if there is a large change in the compression rate in a short time, such as when an irregular image is captured, the appropriate next compression can be performed without easily following the change. The rate 152 can be derived.

…（数式２）
こうして求められた、差分値を利用した値ｇが次圧縮率１５２となる。このように差分値を用いると、画像間の移動量が大きい等、圧縮率が連続的に大小一方向に変化し続けたり、その変化方向が偏っている場合に、その変化に追従した適切な次圧縮率１５２を導出することが可能となる。 In addition, when using the difference value as shown in FIG. 3, the next compression rate predicting unit 134, as shown in the following formula 2, shows the current compression rate 150 for five times including the latest (a, b, c in FIG. 3). , D, e) are obtained (b−a, c−b, d−c, ed), and a difference value between the current compression rates 150 adjacent to each other in the time direction is calculated, and the average value is calculated as the latest current compression rate 150. Add to (here e).

... (Formula 2)
The value g using the difference value obtained in this way becomes the next compression rate 152. When the difference value is used in this way, when the compression rate continues to change in one direction, such as a large amount of movement between images, or the change direction is biased, an appropriate follow-up change is followed. The next compression rate 152 can be derived.

…（数式３）
こうして求められた、変化率を利用した値ｈが次圧縮率１５２となる。このように変化率を用いると、画像間の移動量が大きい等、圧縮率が連続的に大小一方向に変化し続けたり、その変化方向が偏っている場合に、その変化に追従した適切な次圧縮率１５２を導出することが可能となる。 Furthermore, when using the rate of change as shown in FIG. 4, the next compression rate prediction unit 134, as shown in the following Equation 3, shows the current compression rate 150 for five times including the latest (a, b, c in FIG. 4). , D, e) are obtained (b / a, c / b, d / c, e / d) between the current compression ratios 150 adjacent to each other in the time direction, and the average value is calculated as the latest current compression ratio 150. Multiply (here e).

... (Formula 3)
The value h using the change rate obtained in this way becomes the next compression rate 152. If the rate of change is used in this way, when the compression rate continues to change in one direction, such as a large amount of movement between images, or the direction of change is biased, an appropriate follow-up change is followed. The next compression rate 152 can be derived.

  As described above, the number of the current compression ratios 150 used by the next compression ratio prediction unit 134 may be two or more. For example, the number of the current compression ratios 150 is a predetermined number as in the case of starting continuous imaging. When the current compression ratio is less than the predetermined compression number, the next compression ratio prediction unit 134 uses the latest current compression ratio 150 as the next compression ratio 152 as it is, or the current compression ratio holding unit 132 holds the current compression less than a predetermined number. The next compression rate 152 may be predicted by reflecting only the rate 150.

  For example, since the current compression rate 150 of the previous image does not exist in the imaging of the first image data in the continuous imaging, the frequency response processing unit 114, the gradation processing unit 116, and the compression processing unit 118 are prepared in advance. Conversion processing or compression coding processing is performed based on the standard frequency response characteristics, gradation characteristics, and quantization table. Note that these standard frequency response characteristics, gradation characteristics, and quantization table are set based on a target compression rate set by an external operation by the user. Then, the next compression rate prediction unit 134 sets the current compression rate 150 of the image data obtained by the change processing and compression encoding processing as the next compression rate 152 as it is. In addition, while the second image data is captured, only the current compression rate of 150 is retained, but the frequency response processing unit 114, the gradation processing unit 116, and the compression processing unit 118 perform the current compression for one time. Conversion processing or compression coding processing is performed based on the new frequency response characteristics, gradation characteristics, and quantization table derived from the next compression ratio 152 by the ratio 150.

  Similarly, the next compression rate prediction unit 134 holds the current compression rate less than the predetermined number held in the current compression rate holding unit 132 until the number of the current compression rate 150 reaches the predetermined number. 150, the frequency response processing unit 114, the gradation processing unit 116, and the compression processing unit 118 predict new frequency response characteristics, gradation characteristics, and quantization derived from the next compression ratio 152. Conversion processing or compression encoding processing is performed based on the table. At this time, the data control unit 124 does not store in the image storage unit 126 the image data until the number of the current compression rate 150 reaches a predetermined number among the generated image data. You can even start remembering after you reach the number.

  Here, the frequency response processing unit 114, the gradation processing unit 116, and the compression processing unit 118 are held in the current compression rate holding unit 132 until a predetermined number is reached from the imaging of the second image data in the continuous imaging. The conversion process or the compression encoding process is performed based on the new frequency response characteristic, gradation characteristic, and quantization table using the current compression ratio 150. However, the present invention is not limited to this. The conversion process or the compression encoding process may be performed based on standard frequency response characteristics, gradation characteristics, and a quantization table prepared in advance until a predetermined number is reached.

  Further, the time position at which the plurality of current compression ratios 150 are taken out is not limited to one or more times in the past and the past as described above, and may be a predetermined number in the past excluding the latest.

  Further, the next compression rate prediction unit 134 (1) the above-described process of averaging a predetermined number of current compression rates 150, and (2) the difference value between the current compression rates 150 adjacent to each other in the time direction of the current compression rate 150. (3) A process of multiplying the latest current compression ratio 150 by the average value of the rate of change of the current compression ratio 150 adjacent in the time direction of the current compression ratio 150, The processing of 2 or 3 selected from these groups may be switched according to the change mode (direction of change and magnitude) of the current compression rate 150 to predict the next compression rate 152.

  For example, (1) a process of averaging a predetermined number of current compression ratios 150 and (2) an average value of difference values of current compression ratios 150 adjacent to each other in the time direction of the current compression ratio 150 are the latest current compression ratios. It is assumed that the process of adding to 150 is combined so as to be switchable. The next compression rate prediction unit 134 monitors the change mode of the current compression rate 150. For example, the compression rate continuously changes in one direction, such as a large movement amount between images, or the change direction is biased. (2) When the next compression rate 152 is predicted using the difference value and an irregular image is captured, or when there is a large change in the compression rate in a short time, (1) the average value is used. The next compression rate 152 is predicted. In this way, by using an effective method for deriving the next compression ratio in accordance with the change mode of the current compression ratio 150, the data capacity of the image data after compression coding can be made closer to the target value with higher accuracy. It becomes possible.

  Next, correction of the compression rate of image data will be described. The correction of the compression rate of the image data is performed by the frequency response processing unit 114, the gradation processing unit 116, and the compression processing unit 118, each performing conversion processing or compression encoding processing based on the frequency response characteristics, gradation characteristics, and quantization table. It is done by doing. Therefore, by correcting (a) frequency response characteristics, (b) gradation characteristics, and (c) quantization table, the compression ratio of the image data changes relatively, and the current compression ratio is obtained throughout the continuous imaging. 150 can be brought close to the target compression rate.

  Here, the target compression rate is a compression rate of 1 set by the photographer or selected from a plurality of options (imaging mode such as standard mode and fine mode). The target compression rate may be determined based on the data capacity per frame obtained by dividing the total data capacity that can be stored in one continuous imaging by the number of continuous imaging.

  The frequency response deriving unit 136 uses the previous frequency response characteristic held in the frequency response holding unit 138 and the next compression predicted by the next compression rate prediction unit 134 so that the predicted next compression rate 152 approaches the target compression rate. A frequency response characteristic used by the frequency response processing unit 114 is derived based on the rate 152 and a preset target compression rate in continuous imaging.

  Specifically, the frequency response deriving unit 136 first compares the next compression rate 152 predicted by the next compression rate prediction unit 134 with the target compression rate, and determines whether the compression effect should be reduced or increased. For example, when the frequency response characteristic is represented as a frequency response characteristic by an LPF (Low Pass Filter), when the frequency response deriving unit 136 determines that the compression effect should be reduced, the cutoff frequency fc (approximately 30 to about 30 to approximately) of the LPF. If it is determined that the compression effect should be increased (increase the pass bandwidth), the cut-off frequency fc of the LPF is decreased (narrow the pass bandwidth).

  FIG. 5 is an explanatory diagram for explaining the correction of the compression ratio based on the frequency response characteristics. In FIG. 5, the horizontal axis indicates the frequency, and the vertical axis indicates the response to the frequency. As described above, when the frequency response characteristic is represented as the frequency response characteristic by the LPF, the frequency response deriving unit 136 converts the previously derived frequency response characteristic to the frequency response as shown in FIG. If it is obtained from the holding unit 138 (obtains the cut-off frequency fc of the previous LPF) and determines that the compression effect should be reduced, as shown in FIG. 5B, the cut-off frequency fc of the previous LPF is The frequency is increased by a frequency determined according to the difference value between the next compression rate 152 and the target compression rate. Further, when the frequency response deriving unit 136 determines that the compression effect should be increased, as shown in FIG. 5C, the frequency response deriving unit 136 sets the previous LPF cutoff frequency fc to the difference between the next compression rate 152 and the target compression rate. Decrease by the frequency determined according to the value.

  Here, the reason why the frequency response characteristic is used for correcting the compression ratio is as follows. That is, in the image data, the larger the high frequency component, the larger the data capacity of the image data, and the compression effect must be increased accordingly. In order to bring the compression rate close to the target compression rate, it is only necessary to remove high-frequency components from the image data at a stage before the compression encoding process is performed. Therefore, in this embodiment, when the compression effect of the next compression rate 152 is larger than the compression effect of the target compression rate, it is determined that the compression effect should be reduced, and the bandwidth of the frequency response characteristic is widened. If the compression effect is smaller than the compression effect of the target compression rate, it is determined that the compression effect should be increased, and the bandwidth of the frequency response characteristic is narrowed.

  In the above description, the bandwidth of the frequency response characteristic is changed according to the difference value between the next compression ratio 152 and the target compression ratio, and the frequency response characteristic is obtained each time. However, the present invention is not limited to this case. For example, a plurality of predetermined frequency response characteristics (for example, a plurality of predetermined cut-off frequencies fc), a difference value between the next compression ratio 152 and the target compression ratio, and a plurality of frequency response characteristics are associated with each other. Alternatively, a suitable frequency response characteristic may be selected based on a difference value between the next compression ratio 152 and the target compression ratio. With this configuration, the processing load of the frequency response deriving unit 136 can be reduced, and the frequency response characteristics can be quickly obtained. Therefore, the time spent for the compression encoding process of the image data can be shortened. It is also possible to shorten the generation interval.

  The frequency response deriving unit 136 transmits the new frequency response characteristic obtained in this way to the frequency response processing unit 114, and the frequency response processing unit 114 converts the image data based on the new frequency response characteristic. . Further, the frequency response deriving unit 136 overwrites the derived frequency response characteristic on the frequency response characteristic of the frequency response holding unit 138 to prepare for the next processing.

  The gradation deriving unit 140 uses the previous gradation characteristics held in the gradation holding unit 142 and the next compression predicted by the next compression rate prediction unit 134 so that the predicted next compression rate 152 approaches the target compression rate. The gradation characteristics used by the gradation processing unit 116 are derived based on the rate 152 and a preset target compression ratio in continuous imaging.

  Specifically, the gradation deriving unit 140 first compares the next compression rate 152 predicted by the next compression rate prediction unit 134 with the target compression rate, and determines whether the compression effect should be reduced or increased. When the gradation deriving unit 140 determines that the compression effect should be reduced, for example, the overall gradient of the gradation conversion curve (here, a straight line) that is the gradation characteristic should be increased to increase the compression effect. If it is determined, the overall gradient of the gradation conversion curve is reduced.

  FIG. 6 is an explanatory diagram for explaining correction of the compression rate based on the gradation characteristics. In FIG. 6, the horizontal axis indicates the gradation value of the image data input to the gradation processing unit 116, and the vertical axis indicates the gradation value of the image data output from the gradation processing unit 116. The gradation deriving unit 140 acquires the previously derived gradation characteristic from the gradation holding unit 142 as illustrated in FIG. 6A (the inclination of the previous gradation conversion curve 160 (here, for example, the inclination). 1), and when it is determined that the compression effect should be reduced, as shown in FIG. 6B, the gradient of the previous gradation conversion curve 160 is set to a difference value between the next compression rate 152 and the target compression rate. Increase by a value determined according to When the gradation deriving unit 140 determines that the compression effect should be increased, as shown in FIG. 6C, the gradient of the previous gradation conversion curve 160 is calculated between the next compression ratio 152 and the target compression ratio. Decrease by the value determined according to the difference value. However, as shown in FIGS. 6A to 6C, when the input gradation value is 0.5, the gradation conversion curve 160 is set so that the output gradation value is also 0.5. The offset of is adjusted.

  Here, the reason why the gradation characteristic is used for correcting the compression rate is as follows. That is, in the image data, the greater the range that the gradation value can take, the greater the contrast, and the corresponding increase in the data capacity of the image data. Therefore, the compression effect must be increased accordingly. In order to bring the compression rate close to the target compression rate, the contrast of the gradation value of the image data may be changed at a stage before the compression encoding process is performed. Therefore, in this embodiment, when the compression effect of the next compression rate 152 is larger than the compression effect of the target compression rate, it is determined that the compression effect should be reduced, and the gradient of the gradation conversion curve 160 which is the gradation characteristic is increased to increase the contrast. Conversely, if the compression effect of the next compression ratio 152 is smaller than the compression effect of the target compression ratio, it is determined that the compression effect should be increased, and the gradient of the gradation conversion curve 160 is lowered to reduce the contrast. .

  Here, the gradation conversion curve 160, which is the gradation characteristic, has been described as an example of a straight line (primary curve). Can also be configured. In this case, the gradation derivation unit 140 may change the gradation conversion curve 160 so that the slope of the approximate straight line of the gradation conversion curve 160 is large or small. Moreover, although the example of 8-bit input and 8-bit output was shown as a gradation value, another bit value may be sufficient.

  In the above description, the gradient conversion curve 160, which is the gradation characteristic, is changed in accordance with the difference value between the next compression ratio 152 and the target compression ratio, and the gradation characteristic is obtained each time. However, the present invention is not limited to this, and a plurality of predetermined tone characteristics (for example, slopes of a plurality of predetermined tone conversion curves 160), a difference value between the next compression rate 152 and the target compression rate, and a plurality of A gradation table corresponding to the gradient of the gradation conversion curve 160 is prepared, and an appropriate gradient of the gradation conversion curve 160 is determined based on the difference value between the next compression rate 152 and the target compression rate. A configuration may be selected. With such a configuration, the processing load of the gradation deriving unit 140 can be reduced, and the gradation characteristics can be quickly obtained, so that the time spent for the compression encoding process of the image data can be shortened. It is also possible to shorten the generation interval.

  The gradation derivation unit 140 transmits the new gradation characteristic (the slope of the gradation conversion curve 160) obtained in this way to the gradation processing unit 116, and the gradation processing unit 116 transmits the new gradation characteristic. The image data is converted based on the characteristics. Further, the gradation deriving unit 140 overwrites the derived gradation characteristic on the gradation characteristic of the gradation holding unit 142 to prepare for the next process.

  The quantization table deriving unit 144 predicts the previous quantization table held in the quantization table holding unit 146 and the next compression rate prediction unit 134 so that the predicted next compression rate 152 approaches the target compression rate. A quantization table used by the compression processing unit 118 is derived based on the next compression rate 152 and a preset target compression rate in continuous imaging. Here, the quantization table is arranged with different coefficients for dividing the image data in order to omit relatively high frequency components of the image data after discrete cosine transform (DCT) in JPEG. For example, when an image block is represented by 8 pixels × 8 pixels, the table is similarly arranged with 8 × 8 coefficients.

  Specifically, the quantization table deriving unit 144 first compares the next compression rate 152 predicted by the next compression rate prediction unit 134 with the target compression rate, and determines whether the compression effect should be reduced or increased. When the quantization table deriving unit 144 determines that the compression effect should be reduced, for example, when each coefficient of the quantization table is set to a relatively small value (divisor is small) and it is determined that the compression effect should be increased, Each coefficient of the quantization table is set to a relatively large value (divisor is large).

  FIG. 7 is an explanatory diagram for explaining the correction of the compression rate by the quantization table 170. In FIG. 7, the quantization table 170 is represented by an array of 8 × 8 coefficients according to the image block. Further, the quantization table 170 is a coefficient corresponding to the low frequency component of the image block toward the upper left in FIG. 7, and is a coefficient corresponding to the high frequency component of the image block toward the lower right. When the quantization table deriving unit 144 obtains the previously derived quantization table 170 as illustrated in FIG. 7A from the quantization table holding unit 146 and determines that the compression effect should be reduced, FIG. As shown in (b), for example, each coefficient is multiplied by 0.5 by a value determined according to the difference value between the next compression ratio 152 and the target compression ratio, thereby reducing each coefficient. If the quantization table deriving unit 144 determines that the compression effect should be increased, as shown in FIG. 7C, the quantization table deriving unit 144 determines a value determined according to the difference value between the next compression rate 152 and the target compression rate. For example, each coefficient is multiplied by 2 to increase each coefficient.

…（数式４）
ここでＳａ／Ｓは次圧縮率１５２を示し、Ｓｂ／Ｓは目標圧縮率を示す。 Here, the reason why the quantization table 170 is used for correcting the compression rate is as follows. That is, in the image data, the compression effect increases as the coefficients of the quantization table 170 increase, and the compression effect decreases as the coefficients of the quantization table 170 decrease. More specifically, for example, the data capacity of image data before compression is S, the predicted data capacity of image data after compression is Sa, the data capacity of target image data is Sb, and quantization is performed once before. Assuming that each coefficient in the table 170 is Qorg _{i, j} (i and j are integers), the coefficient Q _{i, j} (i and j are integers) in the quantization table 170 to be obtained is obtained using Equation 4.

... (Formula 4)
Here, Sa / S indicates the next compression ratio 152, and Sb / S indicates the target compression ratio.

  For example, if the current compression rate 150 is 20% when the quantization table 170 shown in FIG. 7A is used, the coefficient of the quantization table 170 shown in FIG. 7A is multiplied by 0.5. In the quantization table 170 of FIG. 7B, the current compression rate is 40%. Similarly, the quantization table 170 of FIG. 7B obtained by multiplying the coefficient of the quantization table 170 of FIG. 7A by 2 has a current compression rate of 10%. Such compression encoding processing is an example, and the compression rate changes depending on the content of the image data that is the basis, but the amount of change in the compression rate with respect to the ratio of the coefficients in the quantization table 170 has a substantially inverse relationship.

  Therefore, in this embodiment, when the compression effect of the next compression rate 152 is larger than the compression effect of the target compression rate, it is determined that the compression effect should be reduced, and the coefficient of the quantization table 170 is lowered. If the compression effect is smaller than the compression effect of the target compression rate, it is determined that the compression effect should be increased, and the coefficient of the quantization table 170 is increased.

  In the above description, the quantization table 170 is changed according to the difference value between the next compression ratio 152 and the target compression ratio, and the quantization table 170 is obtained each time. However, the present invention is not limited to this. For example, a plurality of predetermined quantization tables 170 (for example, FIGS. 7A, 7B, and 7C), a difference value between the next compression rate 152 and the target compression rate, and a plurality of quantization tables 170 may be prepared, and an appropriate one quantization table 170 may be selected based on a difference value between the next compression rate 152 and the target compression rate. With this configuration, the processing load on the quantization table deriving unit 144 can be reduced, and the quantization table 170 can be quickly obtained. Therefore, the time spent for compression encoding processing of image data can be reduced, and as a result, images in continuous imaging can be reduced. It is also possible to shorten the data generation interval.

  The quantization table deriving unit 144 transmits the new quantization table 170 thus obtained to the compression processing unit 118, and the compression processing unit 118 compresses the image data based on the new quantization table 170. Encoding process is performed. Also, the quantization table deriving unit 144 overwrites the derived quantization table 170 on the quantization table 170 of the quantization table holding unit 146 to prepare for the next processing.

  Here, an example in which the compression rate correction is performed by all of the frequency response processing unit 114, the gradation processing unit 116, and the compression processing unit 118 has been described. However, the frequency response processing unit 114, the gradation processing unit 116, and the compression processing are performed. It is also possible to correct the compression rate with only one or two functional units selected from the unit 118. At this time, the function units to be adopted may be adopted in the order of the compression processing unit 118, the frequency response processing unit 114, and the gradation processing unit 116 in consideration of the degree of influence of the compression rate.

  For example, since the compression processing unit 118 directly performs the compression encoding process using the quantization table, it is easy to control the compression rate, and the frequency response processing unit 114 converts the existing LPF into the frequency response characteristics of this embodiment. Therefore, the gradation processing unit 116 can reduce the number of parts by using an existing amplifier circuit or gamma correction circuit for conversion of gradation characteristics. Further, when the compression rate is corrected by any one of the functional units or a combination of the two, the correction elements of the other functional units are fixed.

  FIG. 8 is a functional block diagram illustrating another example of the imaging apparatus 100. In the imaging apparatus 100 described above, the compression processing unit 118 directly compresses and encodes the image data output from the gradation processing unit 116, but the gradation processing unit 116 performs the gradation characteristic conversion processing. The image data may be temporarily held in the image memory 122 through the system internal bus 120. In this case, the compression processing unit 118 acquires image data from the image memory 122 during or after the continuously captured image data is accumulated in the image memory 122, and compresses the image data based on the quantization table. The image data is transmitted to the image memory 122 through the system internal bus 120. Such a configuration in which the compression processing unit 118 temporarily stores the image data in the image memory 122 can also be used in a second embodiment described later.

  Here, the generation of image data and the compression encoding process are made independent, only the generation of image data is performed during continuous imaging to reduce the time, and the compression encoding process is performed using a time other than during continuous imaging. To improve time efficiency. In the present embodiment, the image data can be quickly acquired from the image memory 122 during the compression encoding process, and the image data in the image memory 122 can be deleted at an early stage. Can be done.

  In the single-shot imaging according to the present embodiment, continuous imaging is executed with respect to generation of image data, and the data control unit 124 only selects the last compression-encoded image data among the plurality of compression-encoded image data. Store in the image storage unit 126. The single-shot imaging process can also be used in a second embodiment described later. With this configuration, the compression rate correction processing by continuous imaging can be used even in single-shot imaging, and the data capacity of image data in single-shot imaging can be brought close to a desired data capacity.

  In addition, a program that functions as the imaging apparatus 100 by a computer and a storage medium that stores the program are also provided. Further, the program may be read from a storage medium and taken into a computer, or may be transmitted via a communication network and taken into a computer.

(Imaging method)
Next, a description will be given of an imaging method in which the above-described imaging device 100 is used to perform compression encoding processing on images that have been continuously captured (continuous shooting) and stored.

  FIG. 9 is a flowchart showing a flow of processing of the imaging method according to the first embodiment. When performing continuous imaging by the photographer, the imaging unit 104 of the imaging apparatus 100 generates a plurality of image data continuous in the time direction by continuous imaging (S202), and the Y / C processing unit 112 converts the image data into luminance. A process of separating and converting data (Y) and color data (C) is performed (S204). Subsequently, the frequency response processing unit 114 converts the frequency domain characteristics of the image data through the Y / C processing unit 112 based on the frequency response characteristics derived by the frequency response deriving unit 136 described later (S206), The gradation processing unit 116 converts the gradation of the image data that has been subjected to the frequency response characteristic conversion processing based on the gradation characteristic derived by the gradation derivation unit 140 described later (S208), and the compression processing unit 118 performs compression encoding processing on the image data on which the gradation characteristic conversion processing has been performed based on the quantization table, and transmits the image data to the image memory 122 through the system internal bus 120 (S210).

  The current compression rate deriving unit 130 compares the data capacity of the image data before the compression encoding process is performed by the compression processing unit 118 with the data capacity of the image data after the compression encoding process is performed, The current compression rate 150 that is the latest and actual compression rate in the compression encoding process is derived (S212), and the derived current compression rate 150 is held in the current compression rate holding unit 132 (S214). Then, the next compression rate prediction unit 134 acquires the current compression rate held in the current compression rate holding unit 132 over the past multiple times including the most recent current compression rate, and the next compression rate is calculated from the plurality of current compression rates. The next compression rate, which is a rate, is predicted (S216). The prediction of the next compression rate is performed using any one of (1) average value, (2) difference value, and (3) change rate of a plurality of current compression rates.

  In addition, since there is no current compression ratio of the previous time in the first image data imaging in the continuous imaging, the frequency response processing unit 114, the gradation processing unit 116, and the compression processing unit 118 are respectively prepared in advance. Conversion processing or compression encoding processing is performed based on standard frequency response characteristics, gradation characteristics, and quantization tables. Then, the next compression rate prediction unit 134 sets the current compression rate 150 by the change process and the compression encoding process as the next compression rate 152. In the imaging of the second image data, the frequency response processing unit 114, the gradation processing unit 116, and the compression processing unit 118 use the new frequency derived from the next compression rate 152 by the current compression rate 150 for one time. Conversion processing or compression coding processing is performed based on the response characteristics, gradation characteristics, and quantization table.

  The frequency response deriving unit 136 derives a frequency response characteristic used by the frequency response processing unit 114 based on the previous frequency response characteristic held in the frequency response holding unit 138, the next compression rate 152, and the target compression rate (S218). ). The tone derivation unit 140 derives the tone characteristics used by the tone processing unit 116 based on the previous tone characteristics, the next compression rate 152, and the target compression rate held in the tone holding unit 142 (S220). ). The quantization table deriving unit 144 derives the quantization table used by the compression processing unit 118 based on the previous quantization table held in the quantization table holding unit 146, the next compression rate 152, and the target compression rate ( S222). Here, the frequency response deriving unit 136, the gradation deriving unit 140, and the quantization table deriving unit 144 use the frequency response table, the gradation table, and the quantization correspondence table, respectively, to obtain a frequency response characteristic, a gradation characteristic, and a quantum table. A conversion table may be selected. Finally, the data control unit 124 transfers the compression-encoded image data held in the image memory 122 to the image storage unit 126 through the system internal bus 120, and holds the image data (S224).

  The conventional method of repeatedly performing compression encoding processing takes a long time for compression encoding, and there has been a technique to execute without repeating compression encoding processing, but using only the compression rate of the immediately preceding image data Since the parameters for the next compression encoding were determined, the prediction accuracy was low. In the imaging apparatus 100 and the imaging method of the present embodiment, the next compression rate 152 is predicted from the current compression rate 150 of the immediately preceding plurality of image data. As a result, the current compression ratio can be made closer to the target compression ratio with higher accuracy. In particular, in the present embodiment, the imaging interval is shortened as the compression encoding speeds up, so that the compression rate can be appropriately predicted based on more similar image data. Therefore, even when irregular images are taken or the amount of movement between images is large (focus, zoom, pan / tilt operation, etc.), the compression rate does not become extremely out of value, It is possible to perform the compression encoding process at the time of continuous imaging with high accuracy and high speed.

(Second Embodiment)
In the imaging apparatus 100 described in the first embodiment, it is possible to perform compression encoding processing with high accuracy and high speed by predicting the next compression rate from the current compression rate of a plurality of immediately preceding image data. . However, since the imaging apparatus 100 refers to the current compression rate 150 of a plurality of image data over the past in order to predict the compression rate of image data to be stored, image data with an appropriate compression rate is stored. Until this is done, the imaging time required to generate a plurality of image data is required.

  For example, an imaging storage speed that is a speed aimed at storing image data (stores image data in the image storage unit 126 or leaves at least image data in the image memory 122 for storage in the image storage unit 126). In the case of 60 sheets / sec, in order to predict the compression rate of the stored image data, in order to obtain image data for two sheets of the latest image data and the previous image data, at least 1/60 X2 = 1/30 seconds are required. For this reason, it takes time to start up when taking an image, and there is a possibility that the timing of the image will be missed.

  Therefore, in the second embodiment, on the premise of the imaging apparatus 100 in the first embodiment, the compression encoding process is performed with high accuracy by controlling the generation speed of image data at the time of continuous imaging, and in a short time. The purpose is to start storing image data.

(Imaging device 300)
FIG. 10 is a functional block diagram illustrating a schematic configuration of the imaging apparatus 300 according to the second embodiment. The imaging apparatus 300 includes an imaging unit 104, a synchronization signal generation unit 306, a timing generation unit 308, a release switch 110, a Y / C processing unit 112, a frequency response processing unit 114, a gradation processing unit 116, A compression processing unit 118, a system internal bus 120, an image memory 122, a data control unit 124, an image storage unit 126, a display unit 128, a current compression rate deriving unit 130, a current compression rate holding unit 132, Next-order compression rate prediction unit 134, frequency response deriving unit 136, frequency response holding unit 138, tone deriving unit 140, tone holding unit 142, quantization table deriving unit 144, and quantization table holding unit 146 It is comprised including. Here, the synchronization signal generation unit 306 and the timing generation unit 308 function as a generation rate management unit that manages the generation rate of image data.

  The imaging unit 104, the release switch 110, the Y / C processing unit 112, the frequency response processing unit 114, the gradation processing unit 116, the compression processing unit 118, which have already been described as the constituent elements in the first embodiment, System internal bus 120, image memory 122, data control unit 124, image storage unit 126, display unit 128, current compression rate deriving unit 130, current compression rate holding unit 132, and next compression rate prediction unit 134 The frequency response deriving unit 136, the frequency response holding unit 138, the tone deriving unit 140, the tone holding unit 142, the quantization table deriving unit 144, and the quantization table holding unit 146 are substantially Since the functions are the same, a duplicate description is omitted. Here, the synchronization signal generation unit 306 and the timing generation unit 308 having different configurations are cited, and in particular, a configuration for changing the generation rate of image data. It will be described. However, in the first embodiment, a plurality of current compression ratios are required to obtain the next compression ratio. However, in the second embodiment, the next compression ratio may be obtained only from the most recent current compression ratio.

  The synchronization signal generation unit 306 generates a horizontal synchronization signal and a vertical synchronization signal in correspondence with the horizontal size and vertical size of the generated image data, and transmits them to the timing generation unit 308. The timing generation unit 308 generates a control signal for changing the clock timing of the image sensor of the imaging unit 104 based on the horizontal synchronization signal and the vertical synchronization signal generated by the synchronization signal generation unit 306. When receiving an optical signal from the outside, the imaging element of the imaging unit 104 outputs an image signal (image data) in synchronization with the clock timing. In the present embodiment, since one of the purposes is to control the generation speed of image data by the image capturing unit 104, the synchronization signal generation unit 306 has a horizontal synchronization signal and a vertical synchronization signal in order to increase the image data generation speed. In order to shorten the signal generation cycle, while reducing the generation rate of the image data, the generation cycle of the horizontal synchronization signal and the vertical synchronization signal is increased. Here, a method for controlling the generation speed of image data will be described.

  Regarding the method for controlling the generation speed of image data, when the image pickup unit 104 is driven with a predetermined clock, the upper limit of the data capacity of image data that can be read out in a unit time from one output bus of the image pickup device is determined. For this reason, generally, the higher the generation speed of image data, the smaller the data capacity of the generated image data, and the lower the generation speed of image data, the larger the data capacity of the image data. This is a technique using so-called pixel mixture, and a specific implementation method is disclosed in, for example, Japanese Patent Laid-Open No. 2006-217355. In addition, as disclosed in Japanese Patent Laid-Open No. 10-191184, a plurality of output buses are provided, and image data is obtained simultaneously from the plurality of output buses at the same timing, whereby the image data generation speed is high. Also, it is possible to obtain image data having a large data capacity.

  Therefore, unlike the case where a single output bus is provided, the above-described technique for providing a plurality of output buses does not reduce the data capacity of the image data even when the image data generation speed is increased. In other words, since the data capacity of the image data does not depend on the image data generation speed, it is possible to always set the image data generation speed high. However, if the image data generation speed is high, power is consumed correspondingly, and therefore it is desirable to switch the image data generation speed appropriately according to the imaging state at that time. In the present embodiment, a technique of providing a plurality of output buses is used, and it is assumed that the image data has the same data capacity even when the image data generation speed is switched.

  The synchronization signal generation unit 306 temporarily increases the generation speed of the image data of the imaging unit 104 according to an arbitrary trigger from the imaging storage speed for the purpose of storing the image data, and at a predetermined timing after the arbitrary trigger. Switch to imaging storage speed. Here, the arbitrary trigger is a preparation process for storing image data. For example, as a preparatory process for starting the imaging (storage), the photographer presses the release switch 110 halfway, and causes the autofocus of the imaging unit 104 to function. At this time, the imaging unit 104 moves the focus lens to focus on the subject, and also makes the image data generation speed higher than the imaging storage speed. In the imaging apparatus 300, the next compression rate 152 is predicted quickly using image data generated at a high generation speed. When the photographer fully presses the release switch 110 at a predetermined timing after half-pressing, continuous imaging and storage of the generated image data are started.

  The preparatory processing is not limited to the above-described autofocus function start operation, but is a start operation for detecting a smile of a subject, a start operation for detecting a change in video within a predetermined image range, or an image memory for image data. It may be an auto timer setting operation that uses a timer for the storage timing to 122.

  When a predetermined number of current compression ratios 150 are held and the next compression ratio is appropriately predicted, the image data generation speed is returned to the imaging storage speed, and the image memory 122 is stored. Be started.

  FIG. 11 is a timing chart for explaining a change in the generation speed of image data. In FIG. 11, as the imaging state, for example, a storage state for the purpose of storing image data, a prediction state for predicting the next compression rate 152 mainly for compression encoding of image data, and a preceding stage thereof. The three state transitions of the display state in which the image data is mainly displayed on the display unit 128 in order to browse the generated image data are shown.

  The synchronization signal generation unit 306 sets the image data generation speed to 60 frames / sec (period 1/60 sec) equal to the imaging storage speed in the display state until an arbitrary trigger occurs, and the data control unit 124 generates Image data is displayed on the display unit 128.

  Here, upon receiving an arbitrary trigger (for example, an auto focus function start operation) that is a preparation process for storing image data through the operation of the release switch 110 by the photographer, the imaging state is changed from the display state to the predicted state. The synchronization signal generator 306 controls the vertical synchronization signal VD1 to switch the image data generation speed to, for example, 240 images / sec (cycle 1/240 sec) (60 images / sec), which is higher than the imaging storage speed. sec → 240 sheets / sec).

  In addition, when a predetermined time elapses from an arbitrary trigger, or when a new trigger (for example, full press operation) is obtained through the operation of the release switch 110 of the photographer after the predetermined time elapses from the arbitrary trigger, Changes from the predicted state to the storage state, and the synchronization signal generation unit 306 controls the vertical synchronization signal VD1 to switch the image data generation speed to the imaging storage speed (240 images / sec → 60 images / sec). . Then, the data control unit 124 starts storing the image data held in the image memory 122 in the image storage unit 126. Here, since the current compression rate 150 sufficient to predict the next compression rate 152 in the prediction state is acquired, the image data is compressed and encoded at an appropriate compression rate in the storage state.

  More specifically, FIG. 11 will be described. When the imaging apparatus 300 receives an arbitrary trigger, the imaging apparatus 300 transitions the imaging state to the prediction state, and the image processing unit (frequency response processing unit 114, gradation processing unit 116, compression processing unit). 118) starts the correction of the compression rate of the image data, and the correction element deriving unit (frequency response deriving unit 136, gradation deriving unit 140, quantization table deriving unit 144) is a correction element necessary for correcting the compression rate. Begin to derive. Then, the current compression ratio 150 is held in the current compression ratio holding unit 132 by the first vertical synchronization signal VD1 in the predicted state.

  For example, when the number of the current compression ratio 150 for predicting the next compression ratio 152 is 4, the prediction state continues for four clocks of the vertical synchronization signal VD1. When four current compression ratios 150 are held in the current compression ratio holding unit 132, the imaging state transitions from the predicted state to the storage state.

  At this time, the next compression rate prediction unit 134 can appropriately predict the next compression rate 152 based on the average value of the four current compression rates 150 accumulated in the prediction state. The deriving unit 136, the gradation deriving unit 140, and the quantization table deriving unit 144) appropriately correct the correction elements based on the next compression rate 152 and the target compression rate from the first clock of the vertical synchronization signal VD1 in the storage state. And the image processing unit (frequency response processing unit 114, gradation processing unit 116, compression processing unit 118) can correct the compression rate of the image data based on the appropriate correction factor. . The image data thus compression-encoded is stored in the image storage unit 126 by the data control unit 124, for example.

  At the second clock of the vertical synchronization signal VD1 in the storage state, the next compression rate prediction unit 134 includes the three current compression rates 150 acquired immediately before in the prediction state and the current compression rate 150 of the first image data in the storage state. The next compression rate is predicted based on the average value and the like. Therefore, the compression rate between the three image data generated at 240 images / sec and the one image data generated at 60 images / sec is referred to.

  In the present embodiment, the synchronization signal generation unit 306 and the timing generation unit 308 are used to match the image data generation speed of the imaging unit 104 and the storage speed of the image data stored in the image storage unit 126 of the data control unit 124. Thus, continuous imaging can be performed smoothly.

  For example, when the upper limit of the number of image data to be stored in continuous imaging is set to two, for example, the data control unit 124 stores two image data in the image storage unit 126. Automatically changes to the display state. However, in the example of FIG. 11, the image data generation speed is maintained at the imaging storage speed (60 images / sec). Here, the upper limit of the number of image data to be stored in continuous imaging is two, but it is of course possible to set it to one or three or more. In this embodiment, the number of the current compression ratios 150 for predicting the next compression ratio 152 is four, but the number of the current compression ratios 150 may be one or more.

  By the way, in the present embodiment, it is assumed that the data capacity of the image data does not change even when the image data generation speed is switched. However, when the data capacity of the image data changes before and after the switching, pixel mixing or the like is used. For this reason, the high frequency component of the image data generated by the imaging unit 104 may be different from the high frequency component of the image data generated before that. Therefore, the correction element deriving unit (frequency response deriving unit 136, gradation deriving unit 140, quantization table deriving unit 144) takes into account the reduction of high-frequency components by switching the generation rate of image data as parameters for deriving correction elements. There is a need to.

  The compression encoding operates as follows by switching the image data generation speed described above. That is, the current compression rate 150 is stored at a high speed (a higher speed than the imaging storage speed) before the storage of the image data in the image memory 122 is started, and an appropriate next compression ratio is predicted in a short time, and thereafter Continues correction of the compression rate at the imaging storage speed. When the generation interval of image data is short, not only the time required for prediction of the next compression rate 152 is shortened, but also the continuous image data becomes an approximate image, and the prediction accuracy of the next compression rate 152 is improved.

  In addition, the synchronization signal generation unit 306 determines the generation speed of the image data of the imaging unit 104 until the arbitrary trigger occurs and after the last compression-coded image data is stored in the image storage unit. It may be lower.

  FIG. 12 is a timing chart for explaining another example of the change in the generation speed of the image data. Also in FIG. 12, three state transitions of the storage state, the prediction state, and the display state are shown as the imaging state. FIG. 12 shows a vertical synchronization signal VD1 for generating image data, compression encoding, and storing the image data in the image storage unit 126. The vertical synchronization signal VD1 changes according to the imaging state.

  The synchronization signal generation unit 306 has a generation rate of image data of 30 images / sec (period 1/60 sec), which is lower than the image capture storage speed of 60 images / sec (period 1/60 sec) in a display state until an arbitrary trigger occurs. 30 sec), the data control unit 124 causes the display unit 128 to display the generated image data. Here, power consumption and heat generation are reduced by reducing the generation speed of the image data in a display state that does not affect the storage of the image data. However, if the image data generation speed is too low, the display unit 128 may not be able to perform the image data browsing function desired by the photographer. Therefore, in this embodiment, the image data generation speed is 30 sheets / sec.

  Here, upon receiving an arbitrary trigger (for example, an auto focus function start operation) that is a preparation process for storing image data through the operation of the release switch 110 by the photographer, the imaging state is changed from the display state to the predicted state. Then, the synchronization signal generation unit 306 controls the vertical synchronization signal VD1 to switch the image data generation speed to, for example, 240 sheets / sec (cycle 1/240 sec) higher than the imaging storage speed (30 sheets / 240 sec). sec → 240 sheets / sec).

  In addition, when a predetermined time elapses from an arbitrary trigger, or when a new trigger (for example, full press operation) is obtained through the operation of the release switch 110 of the photographer after the predetermined time elapses from the arbitrary trigger, Changes from the predicted state to the storage state, and the synchronization signal generation unit 306 controls the vertical synchronization signal VD1 to switch the image data generation speed to the imaging storage speed (240 images / sec → 60 images / sec). . The data control unit 124 starts storing the image data held in the image memory 122 in the image storage unit 126 based on the vertical synchronization signal VD1 switched to the imaging storage speed. Here, since the current compression rate 150 sufficient to predict the next compression rate 152 in the prediction state is acquired, the image data is compressed and encoded at an appropriate compression rate in the storage state.

  Here, when the imaging state becomes the storage state, the synchronization signal generation unit 306 and the timing generation unit 308 control the vertical synchronization signal VD1 so that the image data generation speed becomes the imaging storage speed, and the compression processing unit 118 The image data compressed and encoded in accordance with the storage timing of the image data by the data control unit 124 is transmitted to the image memory 122. Therefore, it is possible to efficiently execute from generation to storage of image data, and to realize overall time reduction of compression encoding.

  More specifically, FIG. 12 shows that the imaging apparatus 300 displays image data on the display unit 128 at 30 sheets / sec in order to reduce power consumption and heat generation until an arbitrary trigger is received. When an arbitrary trigger is received, the imaging state of the imaging apparatus 300 becomes a predicted state, and the image processing unit (frequency response processing unit 114, gradation processing unit 116, compression processing unit 118) starts correcting the compression rate of the image data. The correction element deriving unit (frequency response deriving unit 136, gradation deriving unit 140, quantization table deriving unit 144) starts deriving correction elements necessary for correcting the compression rate. Then, the current compression ratio 150 is held in the current compression ratio holding unit 132 by the first vertical synchronization signal VD1 in the predicted state.

  For example, as in FIG. 11, when the number of current compression ratios 150 for predicting the next compression ratio 152 is 4, the prediction state continues for four clocks of the vertical synchronization signal VD1. When four current compression ratios 150 are held in the current compression ratio holding unit 132, the imaging state transitions from the predicted state to the storage state.

  At this time, the next compression rate prediction unit 134 can appropriately predict the next compression rate 152 based on the average value of the four current compression rates 150 accumulated in the prediction state. The deriving unit 136, the gradation deriving unit 140, and the quantization table deriving unit 144) appropriately correct the correction elements based on the next compression rate 152 and the target compression rate from the first clock of the vertical synchronization signal VD1 in the storage state. And the image processing unit (frequency response processing unit 114, gradation processing unit 116, compression processing unit 118) can correct the compression rate of the image data based on the appropriate correction factor. . The image data thus compression-encoded is stored in the image storage unit 126 by the data control unit 124, for example.

  At the second clock of the vertical synchronization signal VD1 in the storage state, the next compression rate prediction unit 134 includes the three current compression rates 150 acquired immediately before in the prediction state and the current compression rate 150 of the first image data in the storage state. The next compression rate is predicted based on the average value and the like. Therefore, the compression rate between the three image data generated at 240 images / sec and the one image data generated at 60 images / sec is referred to.

  The state transition of FIG. 12 will be described from the viewpoint of the photographer. The imaging state when no operation is performed is the display state, and the photographer can display the subject when the image data is generated through the display unit 128. The position and size can be confirmed. The generation speed of the image data at this time is set to a low value (for example, 30 sheets / sec) within a range that the image data update speed on the display unit 128 can accept.

  When the photographer presses the release switch 110 halfway, the image capturing apparatus 300 performs an autofocus function to focus on the subject, transitions the image capturing state to the predicted state, and generates a high image data generation speed (for example, 240 sheets / sec), the prediction of the next compression rate 152 in the compression encoding process is started. Subsequently, when the photographer fully presses the release switch 110, the imaging apparatus 300 changes the imaging state to the storage state, switches the image data generation speed to the image data storage speed (60 frames / sec), and continuously. The captured image data is stored. When the photographer opens the release switch 110, the imaging apparatus 300 returns the image data generation speed to a low value (for example, 30 images / sec).

  As described above, the imager can arbitrarily determine the image capturing and storage timing by operating the release switch 110 and the like, and can take advantage of the high accuracy and high speed compression encoding processing according to the present embodiment. Image data can be generated at a desired timing.

  In addition, a program that functions as the imaging apparatus 300 by a computer and a storage medium that stores the program are also provided. Further, the program may be read from a storage medium and taken into a computer, or may be transmitted via a communication network and taken into a computer.

(Imaging method)
Next, a description will be given of an imaging method in which the above-described imaging device 300 is used to perform compression encoding processing on images that have been continuously captured (continuous shooting) and stored.

  FIG. 13 is a flowchart illustrating a flow of processing of the imaging method according to the second embodiment. Also here, since the processing (S202 to S224) already described in the imaging method in the first embodiment is substantially the same processing, redundant description is omitted.

  In the imaging method, first, it is determined whether or not the imaging state is the predicted state or the storage state (S350), and the synchronization signal is displayed while the imaging state is the standard display state (NO in S350). The generation unit 306 sets the imaging data generation speed to be lower than the imaging storage speed (S352).

  When the image data generated by the imaging unit 104 is subjected to Y / C processing by the Y / C processing unit 112 (S204), it is determined whether the imaging state is a predicted state or a storage state (S354), and the imaging state is determined. Is in the display state (NO in S354), the imaging apparatus 300 directly stores the generated image data in the image memory 122 without performing compression encoding processing (S356). The image data held in the image memory 122 is displayed on the display unit 128. When the image data holding process S356 is completed, the process returns to the imaging state determination step S350.

  When a preparation process for storing image data as an arbitrary trigger during the display state, for example, when the photographer receives a half-press operation of the release switch 110, the imaging state transitions from the display state to the predicted state. If it is determined in the imaging state determination step S350 that the imaging state is a predicted state (YES in S350), it is determined whether or not the imaging state is a storage state (S358). If the imaging state is not the storage state, that is, if the imaging state is the prediction state (NO in S358), the synchronization signal generator 306 sets the image data generation speed higher than the imaging storage speed (S360).

  If it is determined in the imaging state determination step S354 that the imaging state is a predicted state (YES in S354), compression encoding processing (S206 to S222) is performed. Then, it is determined whether or not the imaging state is the storage state (S362). If the imaging state is not the storage state (NO in S362), the process returns to the imaging state determination step S350.

  Subsequently, when the photographer receives a full press operation of the release switch 110 after a predetermined time based on an arbitrary trigger, for example, after a predetermined time has elapsed from the arbitrary trigger, the imaging state transitions from the predicted state to the storage state. When it is determined in the imaging state determination step S350 that the imaging state is the storage state (YES in S350), and in the imaging state determination step S358, it is determined that the imaging state is the storage state (YES in S358). The synchronization signal generator 306 sets the image data generation speed to the imaging storage speed (S364).

  In the imaging state determination step S354, when it is determined that the imaging state is the storage state (YES in S354), the compression encoding process (S206 to S222) is performed as in the prediction state. When it is determined in the imaging state determination step S362 that the imaging state is the storage state (YES in S362), the image data is stored in the image storage unit 126 by the data control unit 124 (S224). When the image data holding process S224 is completed, the process returns to the imaging state determination step S350.

  As described above, according to the imaging device 300 and the imaging method described above, the next compression rate 152 is predicted from the current compression rate 150 of the immediately preceding one or a plurality of image data. The change in the time direction of the image data can be properly grasped, and the compression rate can be brought closer to the target compression rate with higher accuracy. Furthermore, by controlling the generation speed of image data, it is possible to predict the compression rate of the image data with higher accuracy and speed, and in the display state that does not affect the storage of the image data, power consumption and heat generation Can be reduced. Such an effect can be expected not only in continuous imaging but also in single-shot imaging.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  Note that each processing described as the imaging method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or by a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for an imaging apparatus and an imaging method that store image data generated by continuous imaging by performing compression encoding processing by intra-screen predictive encoding.

DESCRIPTION OF SYMBOLS 100, 300 ... Imaging device 104 ... Imaging part 114 ... Frequency response processing part 116 ... Gradation processing part 118 ... Compression processing part 124 ... Data control part 126 ... Image storage part 130 ... Current compression rate deriving part 132 ... Current compression rate holding Unit 134 ... next compression rate prediction unit 136 ... frequency response deriving unit 140 ... gradation deriving unit 144 ... quantization table deriving unit 150 ... current compression rate 152 ... next compression rate 160 ... tone conversion curve 170 ... quantization table 306 ... Synchronization signal generator 308 ... Timing generator

Claims (13)

An imaging unit that generates a plurality of image data continuous in the time direction by continuous imaging;
An image processing unit that corrects the compression rate of the image data based on a correction element for correcting the image data, and performs compression encoding using an intra-screen predictive encoding method;
A data control unit for storing the compressed and encoded image data in an image storage unit;
A current compression rate deriving unit for deriving a current compression rate which is an actual compression rate of the compression-encoded image data;
A current compression ratio holding unit for continuously holding a plurality of the current compression ratios in the time direction;
The plurality of current compression ratios held, or the next compression ratio, which is the compression ratio of the image data to be compressed and encoded next time, from the plurality of current compression ratios held and the compression ratio of the image data to be compression-encoded this time A next compression rate prediction unit that predicts
A correction element deriving unit for deriving the correction element based on the next compression ratio;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the next compression rate prediction unit averages a predetermined number of the current compression rates and uses the average value as a next compression rate.   The next compression rate prediction unit averages a difference value of the current compression rates adjacent to each other in a time direction of a predetermined number of the current compression rates, and adds the average value to the latest current compression rate. The imaging apparatus according to claim 1, wherein a compression rate is used.   The next compression rate prediction unit averages the change rate of the current compression rate that is adjacent in the time direction to a predetermined number of the current compression rates, multiplies the average value by the latest current compression rate, and then The imaging apparatus according to claim 1, wherein a compression rate is used.   The next compression rate prediction unit averages a predetermined number of the current compression rates, and calculates an average value of difference values of the current compression rates adjacent to each other in the time direction of the current compression rate as the latest current compression rate. 2 or 3 selected from the group of processes for adding to the latest current compression rate by the average value of the rate of change of the current compression rate adjacent to the current compression rate in the time direction. The imaging apparatus according to claim 1, wherein the next compression ratio is predicted by switching according to a change mode of the current compression ratio. The image processing unit is a frequency response processing unit that converts a frequency domain characteristic of image data based on a frequency response characteristic as the correction element,
The said correction element derivation | leading-out part is a frequency response derivation | leading-out part which derives | leads-out the said frequency response characteristic based on the said next compression rate and a target compression rate, The any one of Claim 1 to 5 characterized by the above-mentioned. Imaging device. A frequency response table in which a plurality of predetermined frequency response characteristics, a difference value between the next compression ratio and the target compression ratio, and a plurality of the frequency response characteristics are associated with each other;
The imaging apparatus according to claim 6, wherein the frequency response deriving unit selects one frequency response characteristic using the frequency response table. The image processing unit is a gradation processing unit that converts the gradation of image data based on gradation characteristics as the correction element,
The said correction element derivation | leading-out part is a gradation derivation | leading-out part which derives | leads-out the said gradation characteristic based on the said next compression rate and a target compression rate, The any one of Claim 1 to 5 characterized by the above-mentioned. Imaging device. A gradation table that associates a plurality of predetermined gradation characteristics, a difference value between the next compression rate and the target compression ratio, and a plurality of gradation characteristics;
The imaging apparatus according to claim 8, wherein the gradation derivation unit selects one gradation characteristic using the gradation table. The image processing unit is a compression processing unit that performs compression encoding processing on image data based on a quantization table as the correction element,
The said correction element derivation | leading-out part is a quantization table derivation | leading-out part which derives | leads-out the said quantization table based on the said next compression rate and a target compression rate, The any one of Claim 1 to 5 characterized by the above-mentioned. Imaging device. A plurality of predetermined quantization tables; and a quantization correspondence table in which a difference value between the next compression ratio and the target compression ratio is associated with the plurality of quantization tables.
The imaging apparatus according to claim 10, wherein the quantization table deriving unit selects one quantization table using the quantization correspondence table.   The said data control part memorize | stores in the said image memory | storage part only the image data by which compression encoding was carried out last among the some image data by which this compression encoding was carried out. The imaging device according to item. Generate multiple image data continuous in the time direction by continuous imaging,
Correcting the compression rate of the image data based on a correction element for correcting the image data, compression encoding using an intra-screen predictive encoding method,
Storing the compressed and encoded image data in an image storage unit;
Deriving a current compression rate which is an actual compression rate of the compression-encoded image data,
A plurality of the current compression rates are continuously held in the time direction,
The plurality of current compression ratios held, or the next compression ratio, which is the compression ratio of the image data to be compressed and encoded next time, from the plurality of current compression ratios held and the compression ratio of the image data to be compression-encoded this time Predict
An imaging method, wherein the correction element is derived based on the next compression rate.

Images