JP2001054054A - Picture processor and picture processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To flexibly correspond to the increase of the number of pixels in input picture data, to easily load data on ASIC and to form picture data in a block unit, which is used for a JPEG processing, without deteriorating throughput. SOLUTION: Picture data for plural rasters is stored in an H-V conversion memory 101 before the processing of a color processing part 105. Stored picture data is read for eight pixels in a direction orthogonal to a raster direction. A part of picture data which is read is stored in a reference buffer 102 and a lack buffer 103, which can store picture data for plural rasters. The picture data which is read from the two buffers and picture data which are read for eight pixels are continuously stored in a shift register. The shift register is caused to play the role of a delay element. The output of a shift register group which is cascade-connected to the shift register is supplied to a color processing part 105 as a reference data group.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method, and more particularly to a memory capable of writing (reading) compressed data to an image memory at a high speed when compressing (expanding) image data. The present invention relates to an image processing apparatus using a mapping method and a method thereof.

[0002]

2. Description of the Related Art As an image data compression / decompression method, there is an image processing apparatus using, for example, the JPEG standard (ISO / IEC 10918-1). This image processing device compresses image data for one screen captured from an image input device such as a digital still camera using a JPEG encoder and stores it on a recording medium.
Record. Further, the image processing apparatus performs processing such as reading image data recorded on a recording medium, decompressing the data by a JPEG decoder, reproducing image information, and displaying the image information on a display. Further, input devices such as digital cameras often have an encoder / decoder (hereinafter, referred to as “JPEGCODEC”) that performs JPEG encoding and decoding.

The JPEG CODEC inputs and outputs image data in blocks of 8 × 8 pixels. On the other hand, image input devices and image display devices such as cameras and televisions input and output image data in raster units. For this reason, JPEG CODEC
In order to exchange data between a camera and a television, etc., image data is temporarily stored in a memory and converted from data in block units to data in raster units or vice versa. "Raster block conversion"
).

FIG. 1 shows a system with JPEG CODEC.
FIG. 3 is a block diagram illustrating a configuration example of a main part of an image processing apparatus that performs raster / block conversion and JPEG encoding after performing color processing on a received input image signal and transfers the result to a next stage.

[0005] Input image data is stored in a line buffer group 201.
, And is input to a color processing unit 105 that performs processing in consideration of the color reproducibility of the recorded image. For example, when the image processing apparatus shown in FIG. 1 is mounted on a digital camera, this input image data can be considered as raw (raw) data from a CCD. In this case, color data in a Bayer array is input from the CCD. For example, in the case of a primary color CCD, color data is input in the order of RGRGRG ... in one line of the CCD, and in the next line, BG
Data arrangement in which color data is input in the order of BGBG... Is repeated. On the other hand, the image processing apparatus must generate RGB data for each pixel of the CCD by interpolation processing to generate one frame image. In addition, it is necessary to correct the difference in sensitivity of the image sensor for each color to maintain white balance at the time of shooting. Further, as impulse removal in color data, it is necessary to perform suppression of false colors in a high frequency region and median filter processing.

[0006] The color processing unit 105 is a part for performing the above-described image processing. In these processes, it is necessary to refer to the vertical data of the image at the same time.
1 and the reference pixel for the target pixel is
Can be supplied at the same time. Line buffer group 2 shown in Fig. 1
According to the configuration of 01, since the upper and lower two pixels can be supplied as reference pixels to the target pixel, it is possible to calculate a range of 5 × 5 pixels centered on the target pixel. Further, the line buffer group 201 is often constituted by a FIFO memory.

Reference numeral 202 denotes a block memory for temporarily storing raster image data and reading the image data in block units. Since 8 × 8 pixels are used as a unit block in JPEG, the block memory 202 needs a storage area of 8H (1H = 1 raster) or more.

Reference numeral 203 denotes a FIFO memory.
It functions as a buffer for absorbing the difference in data transfer timing with the JPEG compressor 109. The JPEG compressor 109 that inputs image data (block scan data) in units of 8 × 8 pixel blocks is controlled by the JPEG block controller 107. JPEG block controller 1
07 is JP without directly involved in JPEG processing of image data.
It monitors and controls the timing of supplying data to the EG compressor 109.

Reference numeral 110 denotes a FIFO memory for matching the data transfer timing between the output of the JPEG compressor 109 and the post-processing unit. The post-process is a process of storing the compressed data in a memory such as an SDRAM connected to a microprocessor (MPU) bus, for example. Then, generally, the compressed image data is finally stored from a SDRAM or the like to a recording medium such as a PC card memory.

Reference numeral 106 denotes a traffic manager that monitors and controls each block shown in FIG. 1 and timing of data supply to the preceding and / or subsequent stages of the block. The traffic manager 106, for example,
If usy is notified, send it to JPEG block controller 1
When the full state of the block memory 202 is received from the JPEG block controller 107, a function of notifying Busy to the preceding stage is provided. Further, when the output of the image data at the preceding stage is stopped, the image processing apparatus has a function of stopping the data reception of the line buffer group 201.

The processing executed by the former stage includes, for example, a processing for taking in data from a CCD in a digital camera, to a processing for compensating for pixel defects and shading. Further, the processing includes securing an OB value used for compensation and storing image data of one or more frames for continuous shooting or responding to a Busy notification at a later stage.

[0012]

When the configuration as shown in FIG. 1 is to be mounted on a digital still camera, the block memory 202 needs a storage capacity of 8H, and furthermore, the pixel referred to by the color processing unit 105 is required. To extend the range to 7 × 7 pixels, it is necessary to add a line buffer for 6H. Furthermore, CCDs have become increasingly pixelated, for example, 4 million pixels
1H = 2500 pixels for CCD images, 1H = 300 for 6 million pixels CCD images
The number of pixels reaches 0, and the configuration of the line buffer group 201 becomes difficult.

The configuration shown in FIG. 1 is generally realized by an ASIC. The block memory 202 stores D
It can be placed outside the ASIC using RAM or SDRAM. However, the part to be constituted by the FIFO memory is a data path system, and it is difficult to substitute a DRAM memory. For example, when Busy occurs in the subsequent stage, the line buffer group 201
In this case, it is necessary to add a complicated process for saving or discarding data to the entire data path process configured by the DRAM-based memory.

Therefore, in order to realize data path processing even when the throughput is reduced, a blank period is provided for processing 1H of data regardless of the presence or absence of Busy.
The receiving capacity of the line buffer group 201 is ensured.

In the block memory 202, the data input is in raster units, whereas the data output is in block units and the data storage and consumption modes are different. JPE until aligned
G processing cannot be started.

An object of the present invention is to solve the above-mentioned problem. The present invention can flexibly cope with an increase in the number of pixels of input image data, can be easily mounted on an ASIC, and has a block unit used for JPEG processing or the like. It is an object of the present invention to provide an image processing apparatus and a method capable of forming the image data without reducing the throughput.

[0017]

The present invention has the following configuration as one means for achieving the above object.

An image processing apparatus according to the present invention includes an input unit for inputting raster-scanned image data, a first processing unit for performing data processing based on a reference pixel area, and a second processing unit for performing data processing in units of a predetermined pixel block. An image processing apparatus having two processing means, wherein image data for a plurality of rasters are stored in a memory before data processing by the first processing means, and the image data stored in the memory is stored in a raster direction. Memory control means for reading in at least the number of pixels in the vertical direction of the block in the orthogonal direction, and storing a part of the read image data in two buffers capable of storing image data for a plurality of rasters. , The memory control unit is configured to read out the image data from the two buffers, and read out the image data for at least the number of pixels in the vertical direction. Shift register for storing sequentially,
A shift register group, which is equivalent to the shift register and cascade-connected to the shift register, wherein the shift register plays a role of a delay element, and an output of the shift register group is the first data as a reference data group. Is supplied to the processing means.

An image processing method according to the present invention is an image processing method for inputting raster scan image data and performing first data processing based on a reference pixel area and second data processing in units of a predetermined pixel block. And storing image data for a plurality of rasters in a memory before the first data processing, and converting the image data stored in the memory in a direction orthogonal to a raster direction at least in a vertical direction of the block. A few minutes of reading, a part of the read image data is stored in two buffers capable of storing image data for a plurality of rasters, and the image data read from the two buffers and the at least the number of pixels in the vertical direction are stored. The read image data is successively stored in a shift register, and the shift register serves as a delay element,
The output of a shift register group cascaded to the shift register is supplied to the first data processing as a reference data group.

[0020]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 2 is a block diagram showing a configuration example of an image processing apparatus according to an embodiment of the present invention. The same components as those shown in FIG. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted.

In the configuration shown in FIG. 2, the H_V conversion memory
Since 101 is arranged before the color processing unit 105, the block memory 202 arranged after the color processing unit 105 in FIG.
Becomes unnecessary. The H_V conversion plays a role of raster-block conversion and does not require the above-described delay of 1H. The reason will be described with reference to FIG.

FIG. 3 is a view for explaining the scanning direction in this embodiment. As shown in FIG. 3, there are a horizontal scanning direction (H scanning direction) and a vertical scanning direction (V scanning direction). When image data is supplied from a CCD of a digital camera, data rows arranged in the H scanning direction are input in the order of the V scanning direction (hereinafter, referred to as “horizontal line reading”). The following description will focus on JPEG processing of a digital still camera equipped with a horizontal line readout CCD as an image input device.

The input image data arranged in the H scanning direction is H_V
The data is sequentially stored in the H_V conversion memory 101 via the manager 104. Then, as shown in FIG. 3, data of a necessary width in the V scanning direction is read and processed. As previously mentioned,
Assuming JPEG processing, the width required in the V scanning direction is 8 pixels, and hereafter, the width required in the V scanning direction is 1 strip = 8pi
xels.

Also, in the present embodiment, it is assumed that the processing of the color processing unit 105 that requires a reference pixel is only a pixel complementing process (when a median filter or the like is not used), and a 5 × 5 pixel The following description will be given assuming that a reference pixel range is given. Therefore, the H_V manager 104
The data in the scanning direction is output for five pixels. H illustrated in FIG. 2
Five lines extending from the _V manager 104 to the color processing unit 105 mean that data in the H-scanning direction output at one time corresponds to five pixels. Note that Fig. 1
The five lines extending from the line buffer group 201 to the color processing unit 105 shown in FIG. 4 indicate that data in the V-scanning direction output once is equivalent to five pixels.

Further, as a feature of this embodiment,
This means preparing data for eight pixels at a time, that is, one strip in the V scanning direction. Arrow groups in the H scanning direction shown in FIG. 3 are intuitively illustrated Raw data input from the CCD to the image processing apparatus of the present embodiment. Color processing unit 1
When the reference pixels in 05 are 5 × 5 pixels, two lines of reference data above the target pixel line are required (see FIG. 8). However, since there is no upper line in the data of the first two lines in the H scanning direction, that is, there is no reference pixel in the upper direction, the data on the line is directly stored in the Lack buffer 103, and the end processing is performed. (End processing line shown in Fig. 3).

Then, the sequence shifts to the normal sequence from the third line. At this time, the H_V conversion memory 101
The V scanning direction data is read in strip units. And
The reading for one strip is performed by moving one pixel at a time in the H scanning direction, and moving the reading point to the next V scanning direction at the end of the 1H width.

An arrow in the V scanning direction shown in FIG. 3 indicates a strip read from the H_V conversion memory 101. And
What is actually output as reference data of the color processing unit 105 is a range enclosed by brackets in the V direction (H of extracted eight pixels).
Line).

Reading from the H_V conversion memory 101,
The difference between the H stages of ejection to the color processing block 105 is related to the processing of the Reference buffer 102 and the Lack buffer 103 at the time of reading, and depends on the reference area. This will be described in detail with reference to FIG.

FIG. 4 is a diagram showing a data array when an effective reference pixel array is created for the color processing unit 105. The upper part of FIG. 4 shows the concept of the 5 × 5 arrangement, in which the H scanning direction of CCD frame input is arranged from top to bottom, and the V scanning direction is arranged from right to left. A rectangular rectangle represents the data array, and covers (1) to (5) shown in FIG. 4 the effective pixels for the reference area in the horizontal direction. In the vertical direction, a redundant component of about four rasters is added to the next effective pixel area (for one strip). By transferring the areas (a) to (c) shown in FIG. 4 to the color processing unit 105, one strip of pixel data can be generated in the next stage.

The arrows from (a) to (c) shown in FIG.
The role of data in the V scanning direction at the time of output of the manager 104 is shown.
Is transferred by each line extending to. (a) is an effective reference area that can be calculated by the color processing unit 105, and the data after calculation corresponding to this position is finally stored in the dual port RAM 1
Stored in 08. (b) is reference pixel data read from the reference buffer 102 used for generating the current effective reference area, and is not transferred to the dual port RAM 108. (c) is reference area loss (Lack) data that cannot be calculated by the color processing unit 105 only with the data read this time among the pixel data read from the H_V conversion memory 101, and is transmitted to the color processing unit 105. At the same time, the same data is stored in the Lack buffer 103.

As shown in the lower part of FIG. 4, the H_V manager 10
Among the data output from 4, data R0 and R1 are Referen
The data is read from the ce buffer 102, and L0 and L1 are data read from the Lack buffer 103. Subsequent data D0 to D7 are data output from the H_V conversion memory 101. Of these D0 to D7, D4 and
D5 is stored in the reference buffer 102, and D6 and D7 are
The data is stored in the ck buffer 103.

FIG. 5 shows an H_V manager characteristic of this embodiment.
FIG. 2 is a block diagram showing an example of the configuration of 104. In FIG. 5, 50
Reference numeral 0 denotes a controller that controls the status transfer to the Reference buffer 102 and the Lack buffer 103 and the status transfer (including the address signal) to the H_V conversion memory 101. This controller 500 is the data select switch 501
By controlling the switching from to 503, the shift register 5
The data array input from 04 to 508 is switched to the data array shown in FIG. Referen surrounded by a broken line in FIG.
The ce buffer 102 and the Lack buffer 103 are, of course, external to the H_V manager 104.

Then, as described above, the shift register
In 504 to 508, data is sequentially stored in the arrangement shown in FIG. As shown in FIG. 5, in the reference pixel area of 5 × 5 pixels, the output of the shift register 506 becomes the target pixel column in the color processing unit 105.

The timing of these controls differs depending on the concurrent operation of the buffers and switches and the memory buffer used. Therefore, the flow of strip generation will be described below using a state machine notation instead of a flowchart or a timing chart.

FIG. 6 is a diagram showing the main operation of the present embodiment in a state machine notation. The main operation is a series of H_
This is the flow of V conversion strip generation, H_V conversion memory 101, Refe
It points to the operation center of the rence buffer 102, the Lack buffer 103, and the H_V manager 104.

When a system reset occurs in the image processing apparatus according to the present embodiment, control is shifted to an idle state (S601) and the apparatus stands by. Then, in response to an execution request such as a photographing mode, the state transits to a standby state (S602) and waits. In this state, register setting of the image processing apparatus is performed if necessary. The capture of the white balance coefficient may be performed between the states S602 and S603.

When a trigger for inputting image data occurs, H_
1H data is sequentially stored in the V conversion memory 101, but as described above, the first 1H is
The data is stored in the buffer 103 (S603). Also, as mentioned above,
In the reference pixel range of × 5 pixels, up to the second line is treated as edge processing, so the next 1H is also stored in the Lack buffer 103 (S604).

After that, when the storage of the Lack data is completed, the state transits to the state S606. The state S606 is continued while data for 8H is being stored constantly. If it is necessary to issue a Busy for the input of the image processing apparatus due to the subsequent processing or the buffer full, the process transits to the state S605, and the data is consumed, and waits until the data can be stored again. .

In the present embodiment, the storage capacity of the H_V conversion memory 101 is doubled (2 channels) of 16H to the data amount of 8H required to generate an input block corresponding to JPEG.
Separately provided, toggles storage / recall every 8H. First, data for 8H is stored in one of the channels, and the data storage channel is switched when the channel is full (S607). After switching channels, switch to that channel
After storing the data for 8H, the data of the channel that has become full starts the processing at the next stage. However, if data remains in one channel but data remains in the other channel, the state transits to the state S605 in order to enter the Busy state.

In the state S605, the traffic manager 10
6 issues Busy for the first-stage processing, and continues the data processing for the second-stage processing.

At the end of the frame, the remaining data is 8H
Then, the process goes to the termination process (S608). Of course, the end of the frame may be detected by a signal provided separately. Further, the terminating process here is a process of determining whether or not to finally consume the data stored in the Lack buffer 103, and executing it. For example, some JPEG compressors do not accept odd block units. In such a case, it is necessary to cut off excess data.

When the termination processing is completed, the completion of the processing is indicated to the entire apparatus, and the apparatus transitions to an idle state (S601).
stand by.

FIG. 7 is a diagram showing the operation of generating the reference data shown in FIG. 4 in a state machine notation, and shows the data storage control of the shift register 504 shown in FIG.

As in the main operation of FIG. 6, when a system reset occurs, the state transits to the idle state (S701) and waits. Next, after the storage of 8H of data in the state S606 is completed, when a start request for strip data processing is detected, the state transits to the state S702, and the reference
nce data for two pixels (R0 and R0 shown in FIG. 4)
1). Of course, the two pixels are the case of the reference pixel area of 5 × 5 pixels, and the first two rasters are treated as the above-described end processing because this reference area does not exist.

After setting the reference data, state S703
Then, Lack data is extracted from the Lack buffer 103 (L0 and L1 shown in FIG. 4). This Lack data part is Ref
The color processing unit 105 can perform processing as Valid data by referring to the erence data. When these buffer data are collected in the shift register 504, H_V
The Strip data is read from the conversion memory 101, and storage in the shift register 504 is started (S704). In state S704, FIG.
The storage from D0 to D3 in the Valid_data_strip shown in (a) is performed.

When the storage of D0 to D3 is completed, state S705
To D4 and D5. At this point, D4 and D5 input from the H_V conversion memory 101 become reference data at the time of the next 8H strip processing.
The result is stored in the ference buffer 102, and the state transits to the state S706.

In the state S706, D6 and D7 are stored in the shift register 504 from the H_V conversion buffer 101. These data are only referenced in this strip processing and do not have reference pixels for their own processing.
It is stored in the Lack buffer 103 as Lack data at the time of trip processing.

While continuously processing the next strip,
The state transits to the state S702 again and the next strip processing is continued, so that the strip scan moves one pixel at a time in the H direction. When the strip scanning is completed in 1H units, the number of pixels (H_
Since there is a fraction with respect to the reference area depending on length, a transition is made to state S707, and a dummy strip is inserted so that the final strip comes to the position of the shift register 506 shown in FIG. Then, when the 1H-wide strip generation scanning is completed, the state transits to the idle state S701 and waits.

The image is completed by performing the above control state for one frame scan. Further, as described above, 1H data is secured at the beginning for 8H units, and is read out in a strip shape (Strip) in the V scanning direction (the reverse V direction in the present embodiment), thereby sequentially
Color processing can be performed on 8 × 8 pixel data that requires a JPEG block, and the data can be sequentially subjected to JPEG processing. Accordingly, since the processing of the data path system is performed in a sequence, it is possible to provide an image processing apparatus that can cope with the occurrence of a subsequent Busy and that does not generate redundant timing.

As described above, according to the present embodiment, the H_V manager 104 for controlling the data array for supplying the reference pixels to the color processing unit 105 is different from the configuration of FIG.
H_V conversion memory 101, reference (Reference) buffer 102, and rack (Lack: missing data) buffer 10
3 and dual port RA before JPEG compressor 109.
By providing the M108, by performing H_V conversion at the time of the reference pixel data array for color processing, it is easy to set and cancel the Busy between the preceding and subsequent processing, and it is necessary to provide an unnecessary blank period And the throughput can be significantly improved. Further, according to the above configuration, a normal SRAM type memory cell can be applied without using a line buffer (FIFO), so that mounting in an ASIC becomes easy. In particular, by incorporating the image processing apparatus of the present embodiment into a device that requires JPEG compression such as a digital still camera, it is possible to significantly limit the design of the camera itself and improve the throughput for CCDs with higher pixels. Fulfill.

Supplementally, the FIFO memory 203 shown in FIG.
The data path in the configuration of FIG. 2 is replaced with the dual port RAM 108 shown in FIG.
This is because address conversion cannot be performed by the color processing unit 105 or the FIFO memory 203 because the data does not pass through the 02. Therefore, by making the FIFO memory 203 a dual port RAM 108, the image data input in an arbitrary sequence from the color
This enables address conversion in the block scanning direction of the compressor 109.

[0053]

[Other Embodiments] Even if the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), an apparatus (for example, a copying machine) Machine, facsimile machine, etc.).

Another object of the present invention is to supply a storage medium (or a recording medium) in which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or an apparatus, and a computer (a computer) of the system or the apparatus. It is needless to say that the present invention can also be achieved by a CPU or an MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. Also,
When the computer executes the readout program code, not only the functions of the above-described embodiments are realized, but also the operating system (OS) running on the computer based on the instructions of the program code.
It is needless to say that a case in which the functions of the above-described embodiments are implemented by performing part or all of the actual processing.

Further, after the program code read from the storage medium is written into the memory provided in the function expansion card inserted into the computer or the function expansion unit connected to the computer, the program code is read based on the instruction of the program code. Needless to say, the CPU included in the function expansion card or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

When the present invention is applied to the above-mentioned storage medium, the storage medium described above (FIG. 6 and / or FIG.
The program code corresponding to the state transition diagram is stored.

[0057]

As described above, according to the present invention,
ASIC that can flexibly respond to the increase in the number of pixels of input image data
It is possible to provide an image processing apparatus and a method thereof that can be easily mounted on a PC and that can form image data in block units used for JPEG processing or the like without lowering the throughput.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration example of a main part of a system with a JPEG CODEC;

FIG. 2 is a block diagram showing a configuration example of an image processing apparatus according to an embodiment of the present invention;

FIG. 3 is a diagram illustrating a scanning direction according to the embodiment;

FIG. 4 is a diagram showing a data array when creating an effective reference pixel array for a color processing unit;

FIG. 5 is a block diagram showing a configuration example of an H_V manager shown in FIG. 2;

FIG. 6 is a diagram illustrating a main operation of the present embodiment by a state machine,

FIG. 7 is a diagram illustrating a generation operation of the reference data illustrated in FIG. 4 in a state machine notation;

FIG. 8 is a diagram illustrating an example of a reference pixel.

Continued on front page F-term (reference) DA01 DA02 DB00 EA00 9A001 BB03 EE02 EE04 HH27 HH31 JJ35 KK42

Claims (8)

[Claims] 1. An image having input means for inputting image data of raster scanning, first processing means for performing data processing based on a reference pixel area, and second processing means for performing data processing on a predetermined pixel block basis. A processing device, wherein image data for a plurality of rasters is stored in a memory before data processing by the first processing unit, and the image data stored in the memory is at least the image data in a direction orthogonal to a raster direction. A memory control unit that reads a number of pixels in the vertical direction of the block and stores a part of the read image data in two buffers capable of storing image data for a plurality of rasters, wherein the memory control unit includes: A shift register for continuously storing image data read from the two buffers and image data read for at least the number of pixels in the vertical direction. A shift register group equivalent to the shift register and cascaded to the shift register. The shift register plays a role of a delay element, and an output of the shift register group is a reference data group. An image processing apparatus, which is supplied to a first processing unit. 2. The output of the first processing means is continuous with a width of one side of the block.
An image processing apparatus according to claim 1. 3. A storage unit for storing image data output from the first processing unit in a width of one side of the block, the order in which the image data is written in the storage unit, and the storage 3. The image processing apparatus according to claim 2, wherein a reading order of the image data from the means to the second processing unit is different. 4. The image processing apparatus according to claim 1, wherein the second processing unit performs an image compression process based on JPEG. 5. One of the two buffers buffers the same data as the image data already passed as the processable data to the first processing means as reference data for the next processing, and the other buffer buffers the other data. The buffer buffers the image data passed as the reference data to the first processing means in the current scan as missing data requiring further reference data in the processing of the image data itself. 5. The image processing device according to claim 1, wherein: 6. The image processing apparatus according to claim 1, further comprising: an imaging unit configured to generate the raster-scanned image data; and a recording control unit configured to store the image data processed by the second processing unit in a recording medium. 6. The image processing device according to claim 1, wherein: 7. A digital camera, comprising the image processing device according to claim 1. 8. An image processing method for inputting raster-scanned image data and performing first data processing based on a reference pixel area and second data processing in units of predetermined pixel blocks, wherein Before data processing, image data for a plurality of rasters is stored in a memory, and the image data stored in the memory is read in a direction orthogonal to the raster direction by at least the number of pixels in the vertical direction of the block, and the read image is read out. A part of the data is stored in two buffers capable of storing image data for a plurality of rasters, and the image data read from the two buffers and the image data read for at least the number of pixels in the vertical direction are successively stored. And stores the data in a shift register to cause the shift register to function as a delay element. An image processing method characterized by supplying to said first data processing the output of the shift register group which are cascade-connected to the static reference data group.