JP4124686B2 - Image data processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image data processing apparatus that processes image data captured from an image sensor such as a CCD or CMOS sensor.
[0002]
In recent years, the number of recording pixels of a digital camera has been steadily increasing, and the processing load tends to become heavier accordingly. For this reason, data processing systems in digital cameras often have a buffer memory for the purpose of increasing the continuous shooting speed. In this system, it is desired to use the buffer memory as effectively as possible in order to increase the processing capacity as much as possible, and the technology to avoid the exclusive use of the buffer by the intermediate data generated during the data processing becomes an important issue. ing.
[0003]
[Prior art]
In a digital camera, for example, when recording data of a monolithic image sensor arranged in a primary color Bayer array, raw data input from the image sensor is temporarily stored in a buffer memory, and then the data is read from the memory. Interpolation processing for versioning and JPEG compression for file compression are performed. In this case, ideally, if the input data from the image sensor is converted into JPEG data that is data to be recorded without using the buffer memory, and the conversion result can be stored in the buffer memory, the buffer memory is It is possible to make the most effective use. However, the data input from the image sensor is input in a fixed order, and the order is not necessarily a convenient order in a post-processing circuit (such as an interpolation circuit). As an example, when an image sensor that performs interlaced scanning is used, data is input in units of fields, but an interpolation circuit that interpolates the data performs processing in units of frames instead of units of fields. It is necessary to temporarily save the field data in the buffer memory. In addition, the interpolation circuit requires a large line memory and takes a form in which interpolation processing is performed by dividing into rectangles due to restrictions on the circuit amount. For this reason, even when an image sensor that can be read in frame units (specifically, an image sensor that performs progressive scanning) is used, it is necessary to save data in the buffer memory. Furthermore, in the JPEG compression process, it is determined that the compression process is performed in a rectangle (block unit) of 8 × 8 pixels. Therefore, the order of data input from the image sensor can hardly match the process. Absent.
[0004]
On the other hand, some high-quality digital cameras do not perform interpolation processing inside the camera, but instead record raw data and later perform interpolation processing by software. This recording method is called RAW recording. . In the case of RAW recording, a technique for reducing the file size by appropriately encoding the raw data before interpolation is often used. Moreover, this encoding generally encodes a difference using the fact that the correlation between adjacent data is strong. In this case, a process of re-encoding the uncompressed raw data once saved in the buffer memory is performed. This is because even a digital camera that performs RAW recording needs to check a recorded image or create a thumbnail using an attached monitor. That is, interpolation processing must be performed for data creation for image confirmation and thumbnail creation, and frame data that can be processed by the interpolation circuit needs to be once expanded on the buffer memory. In addition, if data compression is performed by simply irreversible nonlinear compression (for example, compression by γ correction) without using correlation between data, data expansion in the buffer memory is unnecessary. The problem is that the tone is lost and the image quality deteriorates.
[0005]
Incidentally, Patent Document 1 proposes a technique for dynamically changing a processing path according to required processing speed and quality. Patent Document 2 proposes a method for efficiently performing rectangular processing on data stored in a buffer memory.
[0006]
[Patent Document 1]
JP 2001-45427 A
[Patent Document 2]
JP 2000-354193 A
[0007]
[Problems to be solved by the invention]
As described above, in the conventional image data processing apparatus, since it is necessary to save a large amount of raw data (uncompressed data) input from the image sensor in the buffer memory, interpolation processing or the like using the buffer memory effectively Could not do.
[0008]
In the above-mentioned Patent Documents 1 and 2, there is no mention of the problem that RAW data having a large capacity must be temporarily saved in the buffer memory, and the problem has not been solved.
[0009]
The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an image data processing apparatus capable of effectively using a memory for storing image data and improving processing capability. It is to provide.
[0010]
[Means for Solving the Problems]
FIG. 1 is an explanatory diagram of the principle of the present invention. That is, the image data processing apparatus 10 includes an image input unit 11, an encoding unit 12, a storage unit 13, a decoding unit 14, and an interpolation unit 15. Image data is input to the image input means 11 from the image sensor. In the encoding unit 12, the image data sent from the image input unit 11 is encoded using the correlation of data between adjacent image sensors. In this encoding process, for example, data compression is performed by obtaining a difference between adjacent data and encoding the difference. When image data is classified into a plurality of types, a difference is obtained between neighboring data of the same type, and the difference is encoded. Then, the data encoded by the encoding unit 12 is stored in the storage unit 13. The decoding unit 14 takes in the code data from the storage unit 13 and performs a decoding process on the data, and the image data decoded by the decoding unit 14 is input to the interpolation unit 15. The interpolation means 15 performs image data interpolation processing to generate data necessary for image display. In this case, since the data compressed by encoding is stored in the storage unit 13 instead of storing uncompressed large-capacity data, the storage unit 13 can be used effectively, and the image data The processing capability of the processing apparatus 10 is improved.
[0011]
In addition, image data from the image sensor is sequentially input to the encoding unit 12 via the image input unit 11. The encoding means 12 encodes the image data independently for each line using only the horizontal correlation. During the encoding, the write position information (address) of the head code data on the horizontal line is stored in the storage means 13 by the write position holding means 24. The writing position information is stored every time the head data of each line is encoded, and the writing position is a position set according to the line number.
[0012]
Further, the position information (address) stored in the storage unit 13 is read out by the reading position holding unit 34, and the position information is referred to, so that the head of the line is read from the code string stored in the storage unit 13. Are specified.
[0013]
With this configuration, it is possible to perform a decoding process from the beginning of an arbitrary line. As a result, when the image is trimmed and displayed instead of the entire image, the decoding process is quickly performed.
[0014]
As shown in FIG. 2, the first storage unit 43 stores the data decoded by the decoding unit 14 for each pixel type. The data to be stored in the first storage means 43 is instructed by the instruction means. The position information of the code data input to the decoding means 14 is monitored by the monitoring means 47, and the position information of the code data corresponding to the storage data of the first storage means 43 is stored in the second storage means 45. The In this case, if the information stored in the first and second storage means 43, 45 is used, the position where decoding should be resumed is specified by the position information in the second storage means 45, and the data in the first storage means 43 is used. Thus, decoding can be resumed from an arbitrary position in the middle of the line.
[0015]
Further, by providing a plurality of first and second storage means 43 and 45, data and position information for a plurality of lines are stored in the storage means 43 and 44, respectively. Thereby, rectangular processing such as JPEG can be realized.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
(First embodiment)
A first embodiment of the present invention will be described below with reference to the drawings.
[0017]
FIG. 3 is a block circuit diagram showing the image data recording apparatus 10 of the present embodiment. The image data recording device 10 is mounted on, for example, a digital camera and records captured image data.
[0018]
The image data recording apparatus 10 includes an image data input unit 11, an encoding unit 12, a buffer memory 13, a decoding unit 14, an interpolation unit 15, a register 16, a CPU 17, and a recording medium 18. Are connected to each other. In the image data recording apparatus 10 of the present embodiment, the image data input unit 11, the encoding unit 12, the decoding unit 14, the interpolation unit 15, the register 16, and the CPU 17 are arranged on a one-chip semiconductor integrated circuit (LSI). The buffer memory 13 and the recording medium 18 are connected as external components of the LSI. Data input / output to / from the buffer memory 13 and the recording medium 18 is performed via a data bus having a 16-bit bus width.
[0019]
Image data from an image sensor (not shown) is input to the image data input unit 11. In the present embodiment, the color filter provided in the image sensor (specifically, for example, a CCD sensor) uses a primary color Bayer array as shown in FIG. 4, and is an all-pixel readout method (progressive scan). Thus, image data corresponding to each pixel is sequentially input. Specifically, a plurality of horizontal lines are sequentially scanned from top to bottom. Further, in one horizontal scanning period, the data of the leftmost element in FIG. 4 are sequentially input.
[0020]
Based on the image data from the image sensor, the image data input unit 11 performs color information extraction processing relating to brightness and hue necessary for image quality control, and sends the processed image data to the encoding unit 12. . For odd lines such as the first line and the third line in FIG. 4, color information (data R) relating to red and color information (data Gr) relating to green are extracted. For even lines such as the second line and the fourth line, color information (data Gb) regarding green and color information (data B) regarding blue are extracted.
[0021]
When receiving the image data, the encoding unit 12 performs data compression by sequentially encoding the image data. This encoding is performed by utilizing the fact that information between pixels located in the vicinity has a deep correlation. Specifically, a difference between adjacent pixels of the same type (each color) is calculated, and encoding is performed by a method of assigning a variable-length code to the calculated difference. The variable length code used for the encoding process is stored in the register 16.
[0022]
First, the first line of data R0, Gr0, R1, Gr1,... Is input to the encoding unit 12 in order. Therefore, as a data compression procedure, between these input data, the same type of continuous data is used. (Specifically, Rn−Rn−1, Grn−Grn−1) is calculated. At this time, for the data R0 and Gr0 at the beginning of the line, there is no target data for obtaining a difference, so it is necessary to obtain a difference from a fixed value. In this embodiment, the fixed value is set to “0”. That is, the difference data obtained as a calculation result is (R0-0), (Gr0-0), (R1-R0), (Gr1-Gr0),.
[0023]
Here, the CPU 17 may be configured to set an appropriate fixed value in the register 16, and this configuration can improve the data compression rate.
The encoding unit 12 compresses the data by assigning a variable-length code to the calculation result (difference data), packs the code into 16-bit data, and buffers the buffer memory 13 in units of 16-bit data. To store.
[0024]
Then, when the encoding of the data relating to the first line is completed, the image data Gb0, B0, Gb1, B1,... Of the next second line are sequentially input, so that the encoding unit 12 performs the same procedure as above. Then, encoding is performed and data is stored in the buffer memory 13. By repeating this procedure for the number of lines in one frame, the image data captured from the image sensor is captured in the buffer memory 13 in a compressed state.
[0025]
Then, when the image data for one frame has been taken into the buffer memory 13, the image data is written in the external recording medium 18 and image recording is performed.
[0026]
Further, the image data recording apparatus 10 of the present embodiment includes a monitor for confirming the captured image, and requires an operation for confirming the image. Specifically, in order to create data that can be displayed on the monitor, it is necessary to read out data once stored in the buffer memory 13 and perform interpolation processing.
[0027]
In this case, the data stored in the buffer memory 13 is variable-length encoded data, and it is necessary to perform interpolation processing on the stored data. Therefore, the decoding unit 14 reads the code data from the buffer memory 13 and Decrypt data. Then, the decoding unit 14 sends the decoded image data to the interpolation unit 15.
[0028]
The decoding process performed by the decoding unit 14 is the reverse process of the encoding process performed by the encoding unit 12. That is, the decoding unit 14 decodes the code data fetched from the buffer memory 13 into difference data, and sequentially adds the difference data. For the top data of each line, a fixed value (specifically “0”) used when calculating the difference is added.
[0029]
Further, the interpolation unit 15 performs an interpolation process on the decoded image data, creates data that can be displayed on the monitor, and sends the data to the monitor. In this interpolation processing, for the color information extracted by the image data input unit 11 (each color information for one color), information of the remaining two colors other than the color information is estimated from the color information of the surrounding pixels. As a result, data including color information of the three primary colors is generated.
[0030]
FIG. 5 is a block circuit diagram showing a specific configuration of the encoding unit 12.
The encoding unit 12 includes a shift register 21, a subtracter 22, an encoding logic circuit 23, an address counter 24, and selectors 25 and 26.
[0031]
In the encoding unit 12, the image data sent from the image data input unit 11 is sequentially stored in the shift register 21. The shift register 21 includes two registers 21a and 21b for storing image data (16-bit input data). The shift register 21 latches input data in the first-stage register 21a and shifts the data to the subsequent-stage register 21b. For example, when the shift register 21 holds the leading data R0 of the first line in the first-stage register 21a, if the second data Gr0 is input from the image data input unit 11, the data Gr0 is stored in the first-stage data 21a. While latching in the register 21a, the data R0 is shifted to the subsequent register 21b.
[0032]
The subtracter 22 receives data from the image data input unit 11 and data held in the register 21 b subsequent to the shift register 21. The subtractor 22 calculates the difference between each data using those data. Here, in the case where the data R0 is held in the subsequent register 21b of the shift register 21, the data Gr0 is held in the first register, and the data R1 is input from the image data input unit 11, the data R1 to the data R0 is subtracted to calculate difference data (R1-R0). Thus, the subtracter 22 calculates difference data between adjacent pixels for the same type of pixel.
[0033]
Difference data that is a calculation result of the subtracter 22 is input to the encoding logic circuit 23. The encoding logic circuit 23 assigns a code corresponding to the difference data. Here, the difference data is encoded by referring to the code table of the variable-length code stored in the register 16. Accordingly, since the data length of each code assigned in the encoding logic circuit 23 is different, the encoding logic circuit 23 sequentially packs each encoded code data as 16-bit data and outputs it as 16-bit unit data. .
[0034]
The address counter 24 is a counter that generates an address for writing data to the buffer memory 13. Note that an initial address input from the outside is set in the address counter 24 when writing the first code of the first line. Each time 16-bit data is output from the encoding logic circuit 23, the count value of the address counter 24 is incremented by "1".
[0035]
Based on the control signal S1 input from the CPU 17, the selector 25 selectively selects either the counter value (address value) held by the address counter 24 or the address value input from the outside as a buffer memory. 13 is output.
[0036]
The selector 26 selectively outputs either the address value of the address counter 24 or the data output from the encoding logic circuit 23 to the buffer memory 13 based on the control signal S2 input from the CPU 17. To do.
[0037]
In the buffer memory 13, an address to be written is specified based on the address value input from the selector 25, and the data from the selector 26 is written to the specified address.
[0038]
The encoding unit 12 of the present embodiment clears the data in the registers 21a and 21b in the shift register 21 to “0” before starting the encoding process for the image data for one line. Thereby, when calculating the difference between the head data of each line, the fixed data to be subtracted from the head data becomes “0”.
[0039]
In the encoding unit 12 configured as described above, the subtractor 22 always outputs difference data between image data obtained from each pixel for adjacent pixels of the same type. Therefore, in the encoding logic circuit 23, the differential data input from the subtracter 22 is packed into 16-bit data while sequentially assigning codes, and data is output. As a result, a series of encoding processes is performed. The data output from the encoding logic circuit 23 is written into the buffer memory 13 via the selector 26.
[0040]
Further, when the head code data of the code string of each line is stored in the buffer memory 13, the address (position information) is stored. That is, the buffer memory 13 is provided with a data storage area for storing encoded data and an address storage area for storing data addresses. When the head code data is stored in the data storage area, processing for writing the address value held by the address counter 24 to the address storage area is performed prior to that.
[0041]
Specifically, when the encoding logic circuit 23 encodes the first data R0 of the first line and writes the data, first, an initial value input from the outside is set in the address counter 24. Then, the selector 25 outputs an address value inputted from the outside based on the control signal S1 to the buffer memory 13, and the selector 26 counts the initial value (address value) set in the address counter 24 based on the control signal S2. ) Is output to the buffer memory 13. Here, the address value input to the selector 25 from the outside is a value indicating an address storage area in the buffer memory 13. The initial value set in the address counter 24 is a value indicating the head address of the data storage area in the buffer memory 13. Therefore, the buffer memory 13 stores the start address of the data storage area (address for storing the code string of the first line) in the address storage area specified by the address value input from the outside.
[0042]
When the data encoded by the encoding logic circuit 23 is written after the address is stored, the selector 25 outputs the count value (address value) of the address counter 24 to the buffer memory 13 based on the control signal S1, and the selector 26 Outputs the code data of the encoding logic circuit 23 to the buffer memory 13 based on the control signal S2. Therefore, the code data is stored in the buffer memory 13 in the data storage area designated by the address value of the address counter 24. Thereafter, the encoded data is sequentially output from the encoding logic circuit 23, and the count value of the address counter 24 is incremented by "1" each time the data is output. Then, the output data (code data) of the encoding logic circuit 23 is sequentially stored at the address specified by the count value of the address counter 24.
[0043]
Further, when data less than 16 bits remains when the final data of the line is processed, the encoding logic circuit 23 fills the remaining portion with “0” or “1” and outputs the data. Then, the data is written, and the encoding process of the image data of the first line is finished.
[0044]
Thereafter, the image data for the second and subsequent lines is similarly encoded, and the encoding process is repeated for the number of lines in one frame. As a result, all image data corresponding to one frame is encoded and stored in the buffer memory 13. Even in the second and subsequent lines, when processing the image data at the head of the line, the selector 25 outputs the address value input from the outside to the buffer memory 13 based on the control signal S1, and the selector 26 receives the control signal. Based on S2, the address value of the address counter 24 is output to the buffer memory 13. As a result, the head address storing the code string of each line is stored in the address storage area of the buffer memory 13.
[0045]
FIG. 6 is a block circuit diagram showing a configuration of the decoding unit 14.
The decoding unit 14 includes a decoding logic circuit 31, an adder 32, a shift register 33, an address counter 34, and a selector 35.
[0046]
In the decoding unit 14, the decoding logic circuit 31 reads code data from the buffer memory 13, decodes the data, and outputs difference data obtained by the decoding. Based on the difference data output from the decoding logic circuit 31 and the output data of the shift register 33, the adder 32 performs addition for reassembling image data before taking the difference. The shift register 33 latches the data input from the adder 32 in the first-stage register 33a, shifts the data to the subsequent-stage register 33b, and returns the data in the subsequent-stage register 33b to the adder 32.
[0047]
The address counter 34 is a counter that generates a data read address in the buffer memory 13. The selector 35 selectively outputs either the address value held by the address counter 34 or the address value input from the outside to the buffer memory 13 based on the control signal S3 input from the CPU 17. .
[0048]
As described above, in the present embodiment, the buffer memory 13 stores the head address of the code string of each line in addition to the code data. The decoding unit 14 can start the decoding process from an arbitrary line by referring to the head address.
[0049]
Specifically, when the decoding process is started from the head of the L-th line code string, first, the selector 35 outputs an address value input from the outside to the buffer memory 13 based on the control signal S3. Then, the address counter 34 takes in the data read from the buffer memory 13 according to the address value. Here, the address value inputted from the outside designates the address storing the head address of the code string of the Lth line to be decoded. That is, the address counter 34 stores the head address of the code string of the Lth line. The address for storing the head address corresponding to each line is set in advance as a system. The setting information may be a fixed value or may be stored in a memory area accessible by the CPU 17.
[0050]
Here, the selector 35 outputs the address value output from the address counter 34 to the buffer memory 13 based on the control signal S3. As a result, the code data is read from the head address storing the L-line code string in the buffer memory 13. Each time data is read from the buffer memory 13, the address counter 34 is incremented by “1”, and the count value of the address counter 34 is output to the buffer memory 13 as the next read target address value via the selector 35. Is done. As a result, the Lth line code data is sequentially read from the buffer memory 13.
[0051]
Code data read out from the buffer memory 13 is decoded by the decoding logic circuit 31 to restore difference data between adjacent pixels of the same type. The difference data is added to the immediately preceding image data of the same type in the adder 32 and restored as the original image data. As a result, it is possible to completely restore the image data from the image sensor, and the restored image data is sequentially sent from the adder 32 to the interpolation unit 15 at the subsequent stage.
[0052]
Here, the output data of the adder 32 is input to the interpolation unit 15 and also to the shift register 33. When the data is input to the shift register 33, the data is latched in the first-stage register 33a and the data held in the first-stage register 33a is shifted to the subsequent-stage register 33b. The shift register 33 returns the data of the subsequent register 33b to the adder 32 as the shift result.
[0053]
With this configuration, the data returned from the shift register 33 to the adder 32 is the image data immediately before the difference data decoded by the decoding logic circuit 31 is to be added. Prior to starting the decoding process for the image data for one line, the data in the registers 33a and 33b in the shift register 33 are cleared to “0”. By doing so, a fixed value “0” necessary for restoring the head image data of each line is automatically input from the shift register 33 to the adder 32.
[0054]
As described above, when the image data recording apparatus 10 is configured, the code data written in the buffer memory 13 can be read and decoded from an arbitrary line. As a result, the interpolation unit 15 does not target all the image data fetched into the buffer memory 13 for the interpolation process, but performs the interpolation process for the trimmed image data as necessary.
[0055]
Each image data input from the image sensor has a strong correlation not only in the horizontal direction but also in neighboring pixels in the vertical direction. Therefore, rather than taking the difference between the head data of the line and the fixed value, the compression rate can be further increased by taking the difference from the neighboring pixels of the same type that exist in the vertical direction, that is, the image data two lines before. It becomes possible. However, in the present embodiment, the correlation in the vertical direction is established, the encoding using only the correlation in the horizontal direction is performed, and the address of the head code of each line is stored, so that an arbitrary line can be easily obtained. It is possible to start decoding.
[0056]
Conventionally, in the case of compression by encoding, particularly compression by encoding using the correlation of neighboring images, the code data is also used for decoding even when the trimmed image data is to be subjected to interpolation processing. There has been a problem that decoding must be performed sequentially from the top of the. For this reason, when JPEG compression rectangular processing, which is a known image compression technique, is performed on the restored image data, redundant processing occurs.
[0057]
In the case of JPEG compression, a code called a restart marker is inserted into the code string at a certain interval, and by searching for the code of the restart marker from the code string, image data is obtained in a certain interval unit. It is possible to cut out. However, it takes a considerable time to search for the restart marker.
[0058]
In contrast, in the present embodiment, the address of the head code of each line is stored, and decoding is easily started from any line by referring to the address. Therefore, it is possible to avoid redundant processing even when JPEG compression rectangle processing is performed.
[0059]
As described above, according to the above embodiment, the following effects can be obtained.
(1) Since it is not necessary to save a large amount of uncompressed data with respect to the buffer memory 13, the buffer memory 13 can be used effectively. Thereby, the capacity of the buffer memory 13 can be reduced. Further, when the capacity of the buffer memory is not changed, the number of images that can be continuously processed in the digital camera can be increased. That is, by using the image data recording apparatus 10, it is possible to improve the system cost and processing capability of the digital camera.
[0060]
(2) Since the decoding process can be performed from the head of an arbitrary line, the decoding process when an image is trimmed can be quickly performed. Also, when JPEG compression rectangular processing is performed, it is not necessary to perform encoding from the first line, so that it is possible to avoid occurrence of redundant processing as in the prior art.
[0061]
(Second Embodiment)
In the first embodiment, the image data decoded by the decoding unit 14 is sent to the subsequent interpolation unit 15 to perform interpolation processing. The interpolation unit 15 is provided with a line memory (not shown), and interpolation processing is performed using the line memory. The reason why this line memory is required is that, when interpolation processing is performed while paying attention to a predetermined pixel, the color information of the pixels on the upper and lower lines of the target pixel is used.
[0062]
Specifically, in the case of the Bayer array shown in FIG. 4, red data R and green data Gr exist, but blue data B does not exist in odd lines such as the first line and the third line. In the even lines such as the second line and the fourth line, the green data Gb and the blue data B exist, but the red data R does not exist. In the interpolation process, all the pixels are converted into data in which red, green, and blue color information is aligned by compensating for missing information in the red, green, and blue data. Therefore, each color information cannot be supplemented by interpolation processing with only a single line of image data.
[0063]
Therefore, even when using the most primitive interpolation method, for example, the nearest neighbor method in which missing information is filled with neighboring pixel information, at least one line of image data is stored and two lines of image data are referenced at once. There is a need to. If this nearest neighbor method is used, data for one line can be stored, but in order to realize more accurate interpolation, it is necessary to widen the reference range. Interpolation processing is performed using as many as ten lines of information. Therefore, it is necessary to mount a large amount of line memory as a circuit constituting the interpolation unit 15, which increases the circuit scale.
[0064]
Therefore, as shown in FIG. 7, the size of the line memory is set to a size S20 smaller than the horizontal size of all images (size for storing data of the total number of pixels on the horizontal line) S10. The increase in scale is suppressed. When all the images are processed, interpolation processing is performed by dividing into strip-shaped areas (Block 1 to Block X) in which the horizontal width is set by the size S20 of the line memory. The specific processing order is Line 1 to Line Y of Block 1, Line 1 to Line Y of Block 2,..., Line 1 to Line Y of Block X in FIG. Therefore, it is necessary to supply data to the interpolation unit 15 in this order.
[0065]
The decoding unit 41 of this embodiment shown in FIG. 8 is configured to be able to supply data to the interpolation unit 15 in the order described above. In FIG. 8, the same symbol is attached to the same configuration as the decoding unit 14 of the first embodiment.
[0066]
The decoding unit 41 of the present embodiment includes a decoding logic circuit 31, an adder 32, shift registers 33 and 43, an address counter 34, selectors 35 and 44, a data counter 42, a register 45, and a decrementer 46. That is, the decoding unit 41 is different from the decoding unit 14 of the first embodiment in that a data counter 42, a shift register 43, a selector 44, a register 45, and a decrementer 46 are added. Hereinafter, the difference from the first embodiment will be mainly described.
[0067]
The data counter 42 stores the width information of each area (Block 1 to Block X), and outputs a permission signal to the shift register 43 based on the count value obtained by counting the read data and the block width information. Specifically, the data counter 42 outputs a permission signal to the shift register 43 when data R and data G at the right end of each area shown in FIG. At this time, the shift register 43 operates in response to the permission signal, and sequentially latches the data R and data G output from the adder 32, so that the data R and G are sent to the first stage register 43a and the subsequent stage. Stored in the register 43b.
[0068]
When the decoding logic circuit 31 decodes 16-bit code data as read data, the decoding logic circuit 31 stores 4-bit history information representing the number of bits remaining in the 16 bits that are not decoded. The history information is used to decode the data. Here, when the 4-bit history information is 0000, it means that there are no bits left undecoded in the fetched 16-bit code data, and the history information is 0001-1111. The number of bits remaining without being decoded.
[0069]
When decoding the data R and data G at the right end of each block, the decoding logic circuit 31 sends 4-bit history information to the register 45 and stores the information in the register 45. At this time, the address counter 34 sends the address value to the register 45 and causes the register 45 to store the address value. The decrementer 46 takes in the address value stored in the register 45, subtracts “1” from the address value, and outputs the subtracted address value.
[0070]
Based on the control signal S4 input from the CPU 17, the selector 35 selects the address value from the address counter 34, the address value from the register 45, the address value from the decrementer 46, and the address value input from the outside. Any one of them is selectively output to the buffer memory 13. The selector 44 selectively outputs either the data from the shift register 33 or the data from the shift register 43 to the adder 32 based on the control signal S5 input from the CPU 17.
[0071]
In the decoding unit 41 configured as described above, when decoding is resumed from the Line 1 of Block 2 after the decoding processing of Block 1 is completed, the head code address is specified from the storage information of the register 45 and the code data is read out. Here, the storage information of the register 45 indicates the code position for the pixel on the right side of the block 1 pixel (the pixel at the right end) that has already been decoded. For example, in the register 45, when the 4-bit history information stored from the decoding logic circuit 31 indicates 0000, the address value stored from the address counter 34 becomes the code address for the next pixel. Therefore, the selector 35 outputs the address value output from the register 45 to the buffer memory 13 based on the control signal S4. As a result, valid code data corresponding to the address value is read from the buffer memory 13.
[0072]
If the 4-bit history information stored in the register 45 is not 0000, it means that a part of the code data for the next pixel has already been captured. Therefore, the selector 35 outputs the address value output from the decrementer 46 to the buffer memory 13 based on the control signal S4. As a result, valid code data corresponding to the address value is read from the buffer memory 13.
[0073]
Here, although the code data of Block 2 is code data in the middle of the line, since it is data of the area following Block 1 that has already been decoded, the code data for the pixel of Block 2 based on the storage information of the register 45 The position (address) of can be specified. That is, the decoding logic circuit 31 uses the 4-bit history information acquired from the register 45 to restart decoding from the top code data of Block2.
[0074]
When the decoding of the code data is resumed, the data stored in the shift register 43 is used for the first two pixels in Block2. Specifically, the shift register 43 operates in response to the permission signal output from the data counter 42. When the data is output from the adder 32, the data stored in the registers 43a and 43b ( Data R and data G) at the right end of Block 1 are sequentially output. At this time, the selector 44 outputs the data from the shift register 43 to the adder 32 based on the control signal S5. The adder 32 adds the data to the difference data output from the decoding logic circuit 31, and restores the original image data based on the addition result. Then, the adder 32 outputs the image data to the interpolation unit 15 and the shift registers 33 and 43.
[0075]
Similarly, the decoding process after Block 3 can be resumed from the middle of the line as long as it continues to the area where the decoding has already been completed.
In the present embodiment, the data counter 42 corresponds to an instruction unit, and the shift register 43 corresponds to a first storage unit. The decryption logic circuit 31 and the address counter 34 correspond to monitoring means, and the register 45 corresponds to second storage means.
[0076]
In the decoding unit 41, shift registers 43 and registers 45 are prepared for a plurality of lines, but are not shown for convenience. For example, when a rectangular process such as JPEG is directly performed on the interpolated data, the process is performed in units of 8 × 8 pixel blocks. In this case, the shift register 43 and the register 45 are provided for 8 lines, and the number of lines (8 lines + interpolation reference line) sufficient to always output the 8 lines necessary for the JPEG processing is considered in consideration of the processing method of the interpolation circuit. It is configured so that the record of Equation -1) remains.
[0077]
As described above, according to the above embodiment, the following effects can be obtained.
(1) It is not necessary to save a large amount of uncompressed data in the buffer memory 13. Furthermore, by providing a plurality of shift registers 43 and registers 45, it is possible to solve the implementation of rectangular processing such as JPEG using the data stored in the buffer memory 13.
[0078]
The above embodiment can be modified as follows.
In the above embodiment, the address of the head code in all lines is stored, and the encoding process can be started from any line. However, the present invention is not limited to this. If the line for trimming is preset as the system, only the address of the head code of the line necessary for trimming may be stored. Also, in this case, when encoding the head data in each line that does not need to store the address of the head code, a process of taking a difference from the pixel data of the previous two lines instead of taking a difference from a fixed value ( (Encoding processing using vertical correlation) may be performed.
[0079]
In the above embodiment, the head address and code data of the code string of each line are stored in the buffer memory 13, but storage means (for example, the register 16) different from the buffer memory 13 for storing the code data The head address may be stored in the memory.
[0080]
The encoding process in the encoding unit 12 is to calculate and encode a difference between image data, but in addition to this, encoding is performed using FFT (Fourier transform), DCT (discrete cosine transform), or the like. You may do it. In short, what is necessary is just to perform an encoding process using the correlation of the data between adjacent image sensors.
[0081]
Although the present invention is embodied in an image sensor that performs all pixel readout scanning (progressive scan), it may be embodied in an image sensor that performs interlaced scanning. In addition to the CCD sensor, a CMOS sensor may be used as the image sensor.
[0082]
The image data recording apparatus 10 is configured to store the code data stored in the buffer memory 13 in the external recording medium 18, but the interpolated data subjected to the interpolation processing by the interpolation unit 15 is stored in the recording medium 18. It is good also as a structure to record.
[0083]
In the above-described embodiment, the image data recording apparatus 10 that records image data on the external recording medium 18 is embodied. However, in addition to this, an interpolation process for omitting the recording function and displaying on the monitor It may be embodied in an image data processing apparatus that performs the above.
[0084]
The various embodiments described above can be summarized as follows.
(Additional remark 1) Image provided with the image input means which inputs the image data from an image pick-up element, the memory | storage means which stores image data once, and the interpolation means which performs an interpolation process based on the image data stored in the said memory | storage means A data processing device,
Encoding means for performing encoding processing on the image data sent from the image input means using the correlation of data between the adjacent image sensors, and storing the encoded data in the storage means;
Decoding means for fetching data from the storage means, decoding the data, and inputting the decoded data to the interpolation means;
An image data processing apparatus comprising:
(Additional remark 2) The said encoding means calculates | requires the difference of the image data between the said image sensors, and compresses data by encoding this difference,
The image data processing apparatus according to appendix 1, wherein the decoding unit decodes the code data fetched from the storage unit into the difference and restores the original image data using the difference.
(Additional remark 3) The said image input means extracts the color information required for image quality control from the said image data, and sends the image data containing the extracted color information to the said encoding means. The image data processing apparatus described.
(Additional remark 4) The said interpolation means performs the interpolation process which estimates and calculates color information other than the color information of a predetermined pixel from the color information of the surrounding pixel of this predetermined pixel about the color information extracted by the said image input means The image data processing device according to attachment 3, wherein the image data processing device is described above.
(Additional remark 5) The image data processing apparatus of Additional remark 1 characterized by including the recording medium which memorize | stores the data stored in the said memory | storage means, or the data after the interpolation which the interpolation process was performed by the said interpolation means.
(Appendix 6) Image data from the image sensor is sequentially input via the image input means, and the encoding means encodes image data independently for each line using only horizontal correlation. Is,
The image data processing apparatus according to appendix 1, further comprising a writing position holding unit that stores writing position information of the code data at the head of the horizontal line.
(Additional remark 7) The said encoding means calculates | requires the difference of the image data between the said image sensors, and compresses data by encoding this difference, and when calculating | requiring the difference of the head image data in each line The image data processing apparatus according to appendix 6, wherein a fixed value is used as an initial value for obtaining the difference.
(Additional remark 8) It has the reading position holding means which specifies the code data at the head of the line from the code string stored in the storage means by reading the writing position information and referring to the position information The image data processing device according to appendix 6.
(Supplementary Note 9) First storage means for storing data decoded by the decoding means for each pixel type;
Instruction means for instructing data to be stored in the first storage means;
Monitoring means for monitoring position information of code data input to the decoding means;
Second storage means for storing position information of code data corresponding to the data stored in the first storage means;
The image data processing device according to any one of appendices 1 to 8, further comprising:
(Supplementary note 10) The image data processing apparatus according to supplementary note 9, wherein a plurality of the first and second storage units are provided, and the data and position information for a plurality of lines are stored in each storage unit.
(Supplementary note 11) The supplementary note 9 or 10, wherein the decoding process is resumed in the middle of the line based on the data stored in the first storage unit and the position information stored in the second storage unit. Image data processing apparatus.
[0085]
【The invention's effect】
As described above in detail, according to the present invention, it is possible to improve the processing capability by effectively using the memory in the image data processing apparatus.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating the principle of the present invention.
FIG. 2 is a block circuit diagram showing decoding means.
FIG. 3 is a block circuit diagram of the image data recording apparatus of the first embodiment.
FIG. 4 is an explanatory diagram showing a primary color Bayer array pattern.
FIG. 5 is a block circuit diagram showing an encoding unit.
FIG. 6 is a block circuit diagram showing a decoding unit.
FIG. 7 is an explanatory diagram showing a processing unit in an interpolation unit.
FIG. 8 is a block circuit diagram showing a decoding unit of a second embodiment.
[Explanation of symbols]
10. Image data recording device as image data processing device
11 Image data input unit as image input means
12 Encoding unit as encoding means
13 Buffer memory as storage means
14 Decoding unit as decoding means
15 Interpolation section as interpolation means
24 Address counter as writing position holding means
31 Decoding logic circuit constituting monitoring means
34 Address counter as reading position holding means
42 Data counter as instruction means
43 Shift register as first storage means
45 Register as second storage means

Claims (4)

An image data processing apparatus comprising image input means for inputting image data from an image sensor, storage means for temporarily storing image data, and interpolation means for performing interpolation processing based on the image data stored in the storage means. There,
Encoding means for performing encoding processing on the image data sent from the image input means using the correlation of data between the adjacent image sensors, and storing the encoded data in the storage means;
Decoding means for fetching data from the storage means, performing decoding processing on the data, and inputting the decoded data to the interpolation means ;
First storage means for storing data decoded by the decoding means for each pixel type;
Instruction means for instructing data to be stored in the first storage means;
Monitoring means for monitoring position information of code data input to the decoding means;
An image data processing apparatus comprising: second storage means for storing position information of code data corresponding to data stored in the first storage means . Image data from the image sensor is sequentially input via the image input means, and the encoding means encodes image data independently for each line using only horizontal correlation,
The head of the code data of horizontal lines, the image data processing apparatus according to claim 1, further comprising a writing position holding means for storing the writing position information of the code data.   The read position holding means for specifying the code data at the head of the line from the code string stored in the storage means by reading the write position information and referring to the position information. 2. The image data processing apparatus according to 2. The image according to any one of claims 1 to 3, wherein a plurality of the first and second storage units are provided, and the data and position information for a plurality of lines are stored in each storage unit. Data processing device.

Images