JP2004007511A - Apparatus and method for processing image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent processing losses from being generated when a line update and an image update are made in image processing. <P>SOLUTION: An image processor is provided with an arithmetic circuit 20 which processes mutually-adjoining pixels composed of M pixels arranged in a horizontal direction and N pixels arranged in a vertical direction, and forms an output pixel. A first temporary storage portion 21 readably stores pixels from the Mth pixel up to the last pixel of each horizontal pixel array of an image. A second temporary storage portion 22 readably stores pixels from the head pixel up to (M-1)th pixel of each horizontal pixel array of the image. A third temporary storage portion 23 delays the pixels stored in the first portion 21, has pixels input from the second portion 22, and simultaneously outputs mutually-adjoining pixels composed of M pixels arranged in the horizontal direction and N pixels arranged in the vertical direction to the arithmetic circuit 20. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing device and an image processing method. In particular, the present invention relates to an image processing apparatus and an image processing method suitable for a portable device such as a digital camera and a digital video.
[0002]
[Prior art]
In a photographing apparatus such as a digital camera or digital video, raw data (RGB data) obtained by A / D converting an output signal of an image sensor is first stored in a memory. The raw data stored in the memory is converted into YC data by the image processing circuit. The image processing circuit includes a temporary storage circuit for writing raw data read from the memory, and an arithmetic circuit for performing arithmetic processing using raw data of a plurality of pixels read from the temporary storage circuit. FIG. 38 shows an example of the order in which the raw data read from the memory for the conversion process is written in the temporary storage circuit. Specifically, raw data is sequentially read out for each horizontal pixel column (line) in one image 300. The conversion process requires a plurality of pixels that are horizontally and / or vertically adjacent. Therefore, when the writing order shown in FIG. 38 is adopted, a large-capacity line memory capable of storing pixels included in one line of the image 300 is required as a temporary storage circuit, and the image processing circuit becomes large-scale.
[0003]
Patent Literature 1 describes that an image 300 is divided into a plurality of blocks (image blocks 301) as shown in FIG. 39, and raw data is read from a memory and processed for each image block 301. If the reading method shown in FIG. 39 is adopted, the line memory only needs to have a storage capacity capable of storing the pixels included in one line of the image block 301. By reducing the storage capacity of the line memory, the scale of the image processing circuit can be reduced.
[0004]
[Patent Document 1]
JP 2000-354193 A (FIG. 3)
[0005]
[Problems to be solved by the invention]
However, when one image 300 is divided into image blocks 301 as shown in FIG. 39, the frequency of line updates increases. When one image 300 is divided into image blocks 301, the image blocks 301 need to be updated. For example, if the number of pixels in the vertical direction of the image 300 is 1218 and the total number of image blocks 301 included in the image 300 is 476, the number of line updates in the processing illustrated in FIG. 38 is 1217. In contrast, the number of line updates in the processing shown in FIG. 39 reaches 30875 times, which is the product of the number of line updates (65 times) of one image block 301 and the total number of image blocks 301 (475).
[0006]
If the reading of the raw data in each image block 301 is simply performed for each line as shown in FIG. 39, the conversion process is not performed on the raw data read from the memory to the temporary storage circuit at the time of updating the line and at the time of updating the image block. Possible combinations occur. While the raw data stored in the temporary storage circuit is a combination that cannot be converted, the arithmetic circuit cannot generate valid output pixels, and the values of the line memories and registers that constitute the temporary storage circuit simply shift. Is done.
[0007]
Taking a case where the conversion process is performed using three pixels in the horizontal direction and three pixels in the vertical direction as an example, at the time of line update, at least the third pixel 302 from the head of a new line as shown in FIG. Until is written into the temporary storage circuit, the conversion process cannot be performed. Further, at the time of updating the image block 301, the conversion process cannot be performed until at least the third pixel 302 from the head of the third line is written as shown in FIG. When the image 300 is divided into the image blocks 301 as described above, a large number of line updates and image block updates are performed. Therefore, the time loss of the processing at the time of updating the line and the image block is not negligible, and lowers the processing speed and increases the power consumption.
[0008]
Therefore, an object of the present invention is to eliminate processing loss at the time of updating a line and at the time of updating an image, thereby improving processing speed and reducing power consumption.
[0009]
[Means for Solving the Problems]
According to a first aspect of the present invention, there is provided an image storage unit for storing an image, and an output pixel is generated by performing an arithmetic operation on M horizontally and N vertically adjacent pixels included in the image, M is an integer greater than or equal to 2 and N is an integer greater than or equal to 1 and a first unit that readablely stores from the Mth pixel to the last pixel of each horizontal pixel column of the image. A temporary storage unit, a second temporary storage unit that readablely stores the first pixel to the (M-1) th pixel of each horizontal pixel column of the image in a readable manner, and a first temporary storage unit. And a third temporary pixel that receives the pixels from the second temporary storage unit and simultaneously outputs M horizontally and N vertically adjacent pixels to the arithmetic unit. An image processing apparatus comprising: a storage unit.
[0010]
The image includes both an image for one frame and an image block obtained by dividing the image for one frame.
[0011]
The image processing apparatus starts with the first pixel in the next horizontal pixel row before the pixel used to generate the last output pixel in one horizontal pixel row is stored in the third temporary storage section. Pixels used for generating the last output pixel corresponding to the one horizontal pixel column are read out from the image storage unit up to the (M-1) th pixel and written in the second temporary storage unit. When output from the third temporary storage unit to the calculation unit, the (M-1) th pixel from the head pixel of the next line is read out from the second temporary storage unit and the third temporary storage unit is read out. And a control unit that writes to the unit.
[0012]
When the line is updated, that is, after the last output pixel corresponding to one horizontal pixel column is generated, and the arithmetic unit generates the first output pixel corresponding to the next horizontal pixel column, the third temporary pixel is output. It is necessary that the storage unit has already stored the pixels from the first pixel to the Mth pixel of the next horizontal pixel row. In the first aspect of the present invention, before the line is updated, the (M-1) th pixel from the first pixel of the next horizontal pixel row is stored in the second temporary storage unit before the line is updated, and Is transferred from the temporary storage means to the third temporary storage means. Therefore, even at the time of updating the line, the third temporary storage unit can output a valid pixel set necessary for generating an output pixel to the calculation unit. In other words, it is possible to eliminate the processing loss at the time of updating the line.
[0013]
Further, the N is an integer of 2 or more, and the control unit sets the N pixels arranged in the vertical direction to the pixels belonging to the first to Nth horizontal pixel columns of the image in the image. It is preferable that the operation of sequentially reading from the storage unit and writing the data in the first temporary storage unit or the second temporary storage unit is repeated while moving the read position in the horizontal direction.
[0014]
At the time of updating the image, that is, after the last output pixel corresponding to the last horizontal pixel column in one image is generated, the output pixel corresponding to the first horizontal pixel column in the next image is generated. Sometimes, it is necessary that the pixels belonging to the first to Nth horizontal pixel columns of the next image are stored in the third temporary storage means. By reading out the first to Nth horizontal pixel columns of the image as described above and writing them in the second and third temporary storage units, the third temporary storage unit can be used even when the image is updated. Can output a set of effective pixels necessary for generating an output pixel to the arithmetic unit. In other words, the loss of processing at the time of updating the image can be eliminated.
[0015]
Further, the operation unit generates one output pixel per unit time, the first temporary storage unit includes N RAMs, and the control unit determines a previous write address of the RAM being written. Is immediately before the previous read address, one pixel is read from the image storage unit in the unit time and written in the RAM or the second temporary storage unit, and the last pixel in the RAM being written is read out. If the write address is one or more before the immediately preceding read address, two pixels are read from the image storage unit within the unit time and stored in the RAM and / or the third temporary storage unit. It is preferable to write.
[0016]
By controlling the write address and the read address of the RAM in this manner, the storage capacity of the RAM can be efficiently used without causing a problem in image processing.
[0017]
According to a second aspect of the present invention, an output pixel is generated by performing arithmetic processing on M pixels in the horizontal direction and N pixels adjacent to each other in the vertical direction included in the image stored in the image storage unit. An image processing method, wherein N is an integer of 1 or more, wherein a pixel is read out from the image storage unit while horizontally moving a readout position from the image storage unit, and is written to a first temporary storage unit; Pixels are read from the first temporary storage unit, written to the third temporary storage unit, and delayed by the third temporary storage unit, so that M pixels in the horizontal direction and N pixels in the vertical direction are adjacent to each other. To the calculation unit, the calculation circuit generates output pixels from the M × N pixels, and the pixel used to generate the last output pixel corresponding to one horizontal pixel column is the third pixel. Before the next horizontal pixel From the first pixel to the (M-1) th pixel are read from the image storage unit and written into the second temporary storage unit, and are used to generate the last output pixel corresponding to the one horizontal pixel row. Is output from the third temporary storage unit to the arithmetic unit, the pixels from the first pixel to the (M-1) th pixel of the next line are read out from the second temporary storage unit and 3. An image processing method for writing in a temporary storage unit.
[0018]
In the present invention, the (M-1) th pixel from the first pixel from the top of each horizontal pixel row stored in the second temporary storage unit in advance is stored, read out before the line is updated, and stored in the third temporary storage unit. Since the data is stored, the processing loss at the time of updating the line can be eliminated, and the processing speed can be improved and the power consumption can be reduced. Further, by reading a plurality of pixels belonging to the first to Nth horizontal pixel columns of the image in the vertical direction while moving the reading position in the horizontal direction, the processing loss at the time of updating the image is eliminated, and the processing speed is reduced. Improvement and reduction of power consumption can be achieved. Further, by making the writing speed of the pixel to the RAM, which is the first storage means, different according to the distance between the write address and the read address, the storage capacity of the RAM is efficiently used without causing a problem in image processing. can do.
[0019]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 1 shows an image processing system 1 of a digital camera including a YC processing circuit 4 which is an embodiment of an image forming apparatus according to an embodiment of the present invention, and FIG. 3 shows details of the YC processing circuit 4. The image processing system 1 includes an imaging circuit 2 having a CCD or the like, a memory 3 such as a DRAM, a YC processing circuit 4, an SRAM 5, a resolution conversion circuit 6, a compression processing circuit 7 for performing a compression process such as a JPEG compression process, and an IC. A recording medium 8 such as a card and a control circuit 9 are provided.
[0020]
The operation of the image processing system 1 will be described with reference to FIG. 2. First, raw data (RGB data) generated by the imaging circuit 2 is written to the memory 3 in step S2-1. Next, in step S2-2, the YC processing circuit 4 generates YC data based on the raw data read from the memory 3, and writes the generated YC data into the SRAM 5. If the resolution is changed in step S2-3, the resolution conversion circuit 6 converts the resolution of the YC data in step S2-4. In step S2-5, the resolution-converted YC data is written to the SRAM 5 and the memory 3. If resolution conversion is not to be performed in step S2-2, the YC data generated by the YC processing circuit 4 is written to the SRAM 5 and the memory 3. In step S2-6, the compression processing circuit 7 performs a compression process on the YC data stored in the SRAM 5. The compressed data created by the compression process is written to the memory 3. In step S2-7, the compressed data in the memory 3 is written to the recording medium 8.
[0021]
In the present embodiment, as shown in FIG. 4, the raw data stored in the memory 3 constitutes an image 11 for one frame with 1602 pixels in the horizontal direction and 1218 pixels in the vertical direction. In the following description, the position of each pixel 12 on the image 11 is represented by an X coordinate and a Y coordinate as shown in FIG. For example, the pixel 12 described as (67, 2) is the 67th pixel from the left end of the image 11 and the second pixel from the upper end. In the following description, a horizontal pixel column is called a line as necessary.
[0022]
The YC processing circuit 4 divides the image 11 for one frame into an image block 13 composed of 66 pixels 12 adjacent in the horizontal and vertical directions, as shown in FIGS. . As described later, the arithmetic circuit 20 generates YC data of an output pixel from pixel data of nine pixels 12 adjacent to each other in the horizontal direction and the vertical direction. Therefore, two pixels 12 in the horizontal direction located at the boundary between two image blocks 13 adjacent in the horizontal direction are used for processing both image blocks 13. Similarly, two pixels 12 in the vertical direction located at the boundary between two image blocks 13 adjacent in the vertical direction are used for processing both image blocks 13. The image 11 for one frame includes a total of 475 image blocks 13 including 25 in the horizontal direction and 19 in the vertical direction.
[0023]
As shown in FIG. 6, the raw data of each pixel 12 (hereinafter referred to as pixel data) includes an even / odd bit 14b and a valid bit 14c as accompanying information of the pixel data section 14a. The even-odd bit 14b indicates whether the pixel 12 belongs to an even-numbered line or an odd-numbered line. The valid bit 14c indicates whether or not the pixel data portion 14a of the pixel 12 is valid data.
[0024]
Referring to FIG. 3, the YC processing circuit 4 outputs an arithmetic circuit 20 for performing arithmetic processing on raw data of the pixel 12 stored in the memory 3 and a pixel data of the pixel 12 stored in the memory 3 in an operable combination. A first temporary storage unit 21, a second temporary storage unit 22, a third temporary storage unit 23, a first selection unit 26, and a second selection unit 27.
[0025]
As shown in FIGS. 8A to 8C, the arithmetic circuit 20 performs weighted addition on the pixel data of three pixels 12 adjacent to each other in the horizontal direction and three pixels in the vertical direction. . For example, as shown in FIG. 8A, an output pixel of (1, 1) is obtained by weighted addition of nine pixels 12 from (1, 1) to (3, 3). The arithmetic circuit 20 includes multipliers 31a, 31b, 31c and an adder 32 for summing the outputs of the multipliers 31a to 31c in order to execute the weighted addition. The arithmetic circuit 20 generates one output pixel per unit time.
[0026]
The first temporary storage unit 21 includes three RAMs 21a, 21b, and 21c that store pixel data read from the memory 3, respectively. As will be described in detail later, the three RAMs 21a to 21c store pixel data of the third pixel 12 to the last (66th) pixel 12 from the head of each line in each image block 13. Is done. As shown in FIG. 7, each of the RAMs 21a to 21c is provided with an address 33 from No. 1 to No. 64, and can store pixel data of one pixel in each address 33 in a readable and writable manner.
[0027]
The second temporary storage unit 22 includes six registers 22ad, 22ae, 22bd, 22be, 22cd, and 22ce for storing raw data of one pixel 12, respectively. As will be described in detail later, these six registers 22ad to 22ce are for storing the pixels from the head (first) pixel 12 to the second pixel 12 of each line in each image block 13. is there. The registers 22ad and 22ae correspond to the RAM 21a of the first temporary storage unit 21. The registers 22bd and 22be correspond to the RAM 21b. Further, the registers 22cd and 22ce correspond to the RAM 21c.
[0028]
The third temporary storage unit 23 includes nine registers 23aa, 23ab, 23ac, 23ba, 23bb, 23bc, 23ca, 23cb, and 23cc for storing pixel data of one pixel 12, respectively. The registers 23aa to 23bb correspond to the RAM 21a of the first temporary storage unit 21. The registers 23ba to 23bc correspond to the RAM 21b. Further, the registers 23ca to 23cc correspond to the RAM 21c.
[0029]
The first selector 26 can be switched between an upper position indicated by a solid line and a lower position indicated by a dotted line. When the first selector 26 is at the upper position, three registers 23aa to 23cc are connected in series by three. Specifically, registers 23aa to 23ac, registers 23ba to 23bc, and registers 23ca to 23cb are connected in series. Therefore, when the first selection unit 26 is at the upper position, the registers 23aa to 23ac, the registers 23ba to 23bc, and the registers 23ca to 23cb of the third temporary storage unit 23 delay the output from the RAMs 21a to 21c, respectively. And outputs it to the arithmetic circuit 20 via the second selector 27.
[0030]
When the first selection unit 26 is at the lower position, the third temporary storage unit 23 is connected to the second temporary storage unit 22. Specifically, the registers 22ad and 22ae of the second temporary storage unit 22 are connected to the registers 23ab and 23ac of the third temporary storage unit 23. The registers 22bd and 22be of the second temporary storage unit 22 are stored in the registers 23bb and 23bc of the third temporary storage unit 23. Further, the registers 22cd and 22ce of the second temporary storage unit 22 are connected to the registers 23cb and 23cc of the third temporary storage unit 23. Therefore, when the first selection unit 26 is switched to the lower position, the second temporary storage unit 22 is replaced with the registers 23ab and 23ac, the registers 23bb and 23bc, and the registers 23cb and 23cc of the third temporary storage unit 23. The pixel data is output from the registers 22ad and 22ae, the registers 22bd and 22be, and the registers 22cd and 22ce.
[0031]
The second selector 27 inputs the pixel data of the latest line to the multiplier 31a of the arithmetic circuit 20, inputs the pixel data of the line immediately before the latest line to the multiplier 31b, and The function of switching the connection between the third temporary storage unit 23 and the arithmetic circuit 20 so as to input the pixel data of the line two lines before to the multiplier 31c.
[0032]
The control circuit 9 controls the memory 9, the RAMs 21a to 21c, the registers 22ad to 22ce, the registers 23aa to 23cc, the first selection unit 26, and the second selection unit 27. Transfer of the pixel data to the unit 21 and the second temporary storage unit 22, transfer of the pixel data from the first temporary storage unit 21 and the second temporary storage unit 22 to the third temporary storage unit 23, and the third Of the pixel data from the temporary storage unit 23 to the arithmetic circuit 20. Further, the control circuit 9 controls the arithmetic circuit 20 to execute arithmetic processing and transfer generated pixel data to the SRAM 5.
[0033]
Further, the pixel data is input to the clock control unit 9 a of the control circuit 9 via the third temporary storage unit 23. The clock control unit 9a determines whether the arithmetic circuit 20 can generate a valid output pixel based on the pixel data stored in the third temporary storage unit 23 based on the even / odd bit 14b and the valid bit 14c. . When it is determined that the arithmetic circuit 20 cannot generate a valid output pixel based on the pixel data stored in the third temporary storage unit 23, the clock control unit 9a stops supplying the clock signal to the arithmetic circuit 20, and The arithmetic processing by 20 is prohibited. This prevents the arithmetic circuit 20 from wastefully consuming power when invalid pixel data is input.
[0034]
Next, the YC processing circuit 4 processes the image 11 for one frame by dividing it into a total of 475 image blocks 13 including 66 pixels 12 adjacent in the horizontal and vertical directions. The processing order of the image blocks 13 is as shown by the arrow A in FIG. Specifically, the processing is started from the image block 13 at the upper left corner of the image 11, and the processing of the image block 13 is sequentially executed for each line.
[0035]
The processing performed by the YC processing circuit 4 on each image block 13 can be roughly divided into three stages. In the first stage, the pixel data read from the memory 3 is written to the first temporary storage unit 21 or the second temporary storage unit 22. In the second stage, the pixel data read from the first temporary storage unit 21 or the second temporary storage unit 22 is written to the third temporary storage unit 23. In the third stage, the arithmetic circuit 20 generates an output pixel based on the pixel data read from the third temporary storage unit 23. As will be described in detail later, in the present embodiment, in the second stage, the provision of the second temporary storage unit 22 eliminates the time loss at the time of updating the line.
[0036]
The write destination of the pixel data read from the memory 3 in the first stage is as shown in FIG. More specifically, the pixel data of the first two pixels of each line, that is, the first to second pixels 12 are written to the second temporary storage unit 22. On the other hand, the pixel data of the pixels 12 from the third to the last (66th) of each line is written to the first temporary storage unit 21.
[0037]
In the first stage, the order in which pixel data of the pixels 12 belonging to each image block 13 are read from the memory 3 and written in the first temporary storage unit 21 or the second temporary storage unit 22 is as shown in FIG. . More specifically, for the fourth to 66th lines in the image block 13, the first temporary storage unit 21 or the second temporary storage unit 21 moves one pixel in the horizontal direction for each line as shown by the arrow B. The pixel data is written to the temporary storage unit 22 of. On the other hand, for the first to third lines in the image block 13, as shown by the arrow C, the three pixels 12 arranged in the vertical direction are sequentially read from the memory 3 and the first temporary storage unit 21 or The operation of writing to the second temporary storage unit 22 is repeated while moving the reading position in the horizontal direction. As will be described in detail later, in the present embodiment, at the start of processing of each image block 13, pixel data is read out not in the horizontal direction but in the vertical direction, thereby eliminating the time loss at the time of updating the image block.
[0038]
In the first stage, the order in which pixel data is read from the memory 3 and written into the corresponding RAMs 21a to 21c is to move the address 33 of the RAMs 21a to 21c in the horizontal direction as shown by an arrow D in FIG.
[0039]
In the second stage, the order in which pixel data is read from the RAMs 21a, 21b, and 21c of the first temporary storage unit 21 and written into the corresponding registers 23aa, 23ba, and 23ca of the third temporary storage unit 23 is indicated by an arrow in FIG. As shown in D, the address 33 of the RAMs 21a to 21c is moved in the horizontal direction.
[0040]
Here, with reference to FIG. 7, the relationship between read points and write points in the RAMs 21a to 21c of the first temporary storage unit 21 will be described. As described above, the addresses of the RAMs 21a to 21c are from No. 1 to No. 64. The read point is an address of the RAM 21a to 21c from which the pixel data has been read. The writing point is the address of the RAM where the pixel data newly read from the memory 3 is written.
[0041]
In each of the RAMs 21a to 21c, the current writing point cannot overtake the previous reading point. For example, as shown in FIG. 7, when the previous read point is No. 61, the write point needs to be before No. 60. The reason for this is that the current writing point overtakes the previous reading point, which means that among the pixel data stored in the RAMs 21a to 21c, the address at which the pixel data not yet used for generating the output pixel is stored. This is because new pixel data read from the memory 3 is overwritten on the pixel 33.
[0042]
The write point may be behind the read point as long as it does not overtake the read point. For example, when the read point is No. 61 as shown in FIG. 7, the write point may be before the No. 60. However, it is preferred that the write point be closer to the read point. Ideally, the writing point is preferably immediately before the reading point. For example, as shown in FIG. 7, when the read point is No. 61, the write point is ideally No. 60. The fact that the write point is immediately before the read point means that when pixel data is output from the address 33 in the RAMs 21a to 21c to the third temporary storage unit 23, the pixel data is immediately read from the memory 3 for that address 33. New pixel data is overwritten. In this state, the room where the writing point can be delayed from the reading point is the largest, and the storage capacity of the RAMs 21a to 21c is used most efficiently.
[0043]
The speed at which pixel data is read from the memory 3 and written to the RAMs 21a to 21c of the first temporary storage unit 21 and the registers 22ad to 22ce of the second temporary storage unit 22 is determined based on the above-described conditions regarding the write point and the read point. Is done. Specifically, when the last writing point of the RAM being written is immediately before the last reading point, one pixel 12 is stored in any one of the RAMs 21a to 21c and the registers 22ad to 22ce within a unit time. Is written. On the other hand, if the previous write point of the RAM being written is not immediately before the previous read point, any two of the RAMs 21a to 21c or any one of the RAMs 21a to 21c and any of the registers 22ad to 22ce Pixel data read from the memory 3 within a unit time is written into one pixel.
[0044]
The address management in the RAMs 21a to 21c and the adjustment of the writing speed of the number of pixels per unit time to the RAMs 21a to 21c and the registers 22ad to 22ce as described above are executed by the control circuit 9.
[0045]
In the third stage, the order in which the arithmetic circuit 20 generates output pixels and outputs the output pixels to the SRAM 5 is as shown in FIG. More specifically, as shown by an arrow E, output pixels are generated while moving one pixel in the horizontal direction for each image block 13. The arithmetic processing circuit 21 generates an output pixel from three adjacent pixels 12 in the horizontal direction and the vertical direction. Therefore, the number of pixels included in the pixel block 12 ′ of the output pixel corresponds to the number of pixels of the image 11 stored in the memory 3. It is reduced from the number of pixels in the pixel block 12 (see FIG. 4). More specifically, the pixel block 12 'of output pixels includes a total of 4096 output pixels in the horizontal and vertical directions, that is, 64 output pixels.
[0046]
Next, the operation of the YC processing circuit 4 will be described. The flowchart of FIG. 12 shows the operation of the YC processing circuit 4 for each unit time. This operation is performed by the control circuit 9. In FIG. 12, step S12-1 is the first stage, steps S12-2 to S12-5, steps S12-7 and 12-8 are the second stage, and step S12-6 is the third stage. Each corresponds. As described above, the arithmetic circuit 20 generates one output pixel for each unit time.
[0047]
In step S12-1 in FIG. 12, a method of reading pixel data from the memory 3 is determined, and pixel data is read based on the determined reading method. Specifically, the read speed, the write destination of the read pixel data, and the read direction are determined.
[0048]
Referring to FIG. 13, first, in step S13-1, it is determined whether or not the previous writing point of the RAMs 21a to 21b during writing is immediately before the previous reading point (condition 1). If it is immediately before, in step S13-2, the number of readout pixels in a unit time is determined to be one, and if it is not immediately before, in step S13-3, the number of readout pixels in a unit time is determined to be two. As described above, the reading speed is determined based on the distance between the writing point and the reading point.
[0049]
Next, in step S13-4, it is determined whether or not pixel data necessary for generating the first output pixel of the next line is stored in the registers 22ad to 22ce of the second temporary storage unit 22 (condition). 2). If the pixel data is stored, the pixel data read from the memory 3 is written to the RAMs 21a to 21c of the first temporary storage unit 21 in step S13-5. On the other hand, if the pixel data is not stored, the pixel data read from the memory 3 is written to the registers 22ad to 22ce of the second temporary storage unit 22 or the register 22ad in step S13-6. To 22ce and the RAMs 21a to 21c of the first temporary storage unit 21. As described above, the writing destination of the pixel data is determined based on the pixel data stored in the second temporary storage unit 22.
[0050]
Subsequently, in step S13-7, the pixel data previously read from the memory 3 and written in the RAMs 21a to 21c of the first temporary storage unit 21 and / or the registers 22ad to 22ce of the second temporary storage unit 22 are stored in the image data. It is determined whether or not the pixel belongs to the first three lines of the block 13 (condition 3). If the pixel data belongs to the first three lines, the reading direction is determined to be horizontal in step S13-8. On the other hand, if the pixel data does not belong to the first three lines, the vertical reading direction is determined in step S13-9. As described above, the reading direction is determined based on whether the pixel 12 being read is a pixel on the first line of the image block.
[0051]
FIG. 14 shows the relationship between the conditions 1, 2, and 3 and the read method, that is, the load method. Since there are two types of read speed, write destination, and read direction (load direction), there are a total of eight types of load methods A to H. For example, the last writing point of the RAMs 21 a to 21 c during writing is immediately before the previous reading point (condition 1), and the second temporary storage unit 22 stores the pixels necessary for generating the first output pixel of the next line. (Condition 2), and when the pixel 12 previously written from the memory 3 to the RAM 21a to 21c is not the pixel 12 of the first three lines of the image block 13 (condition 3), the loading method A is adopted and the memory 3 , The pixel data of one pixel 12 is loaded in the RAM 21a to 21c in the horizontal direction.
[0052]
After reading out from the memory 3 in step S12-1, in step S12-2, it is determined whether the output pixel generated last time is the last output pixel of the line in the image block 13 ', that is, at the time of line update. Is determined. When the output pixel is the last pixel of the line, that is, when the line is not updated, the process proceeds to step S12-7. On the other hand, if it is time to update the line, the process moves to step S12-3.
[0053]
If it is not time to update the line, steps S12-7 and S12-8 are executed. First, in step S12-7, the pixel data in the third temporary storage unit 22 is shifted by one. For example, regarding the registers 22aa to 22ac, the pixel data of the register 22aa is shifted to the register 22ab, and the pixel data of the register 22ab is shifted to the register 22ac. Next, in step S12-8, the pixel data stored in the first temporary storage unit 21 is read out and stored in the third temporary storage unit 23. Specifically, pixel data is read from the RAMs 21a to 21c and written to the registers 23aa, 23ba, and 23ca.
[0054]
Next, in step S12-6, the pixel data stored in the third temporary storage unit 23 is output to the arithmetic circuit 20 via the second selection unit 27. Specifically, the pixel data is output from the nine registers 23 aa to 23 cc included in the third temporary storage unit 23 to the multipliers 31 a to 31 c of the arithmetic circuit 20. The arithmetic circuit 20 generates an output pixel from the input pixel data.
[0055]
At the time of updating the line, steps S12-3 to S12-5 are executed. First, in step S12-3, the first selection unit 26 is switched to the lower position, the pixel data stored in the second temporary storage unit 22 is read, and stored in the third temporary storage unit 23. Specifically, the image data is output from the registers 22ad and 22ae to the registers 23ab and 23ac, from the registers 22bd and 33be to the registers 23bb and 23bc, or from the registers 22cd and 22ce to the registers 23cb and 23cc. Next, in step S12-4, the pixel data stored in the first temporary storage unit 21 is read out and stored in the third temporary storage unit 23. Specifically, pixel data is read from the RAMs 21a to 21c and written to the registers 23aa, 23ba, and 23ca. Next, the second selector 27 is switched in step S12-5. The second selector 27 inputs the pixel data of the latest line to the multiplier 31a of the arithmetic circuit 20, inputs the pixel data of the line immediately before the latest line to the multiplier 31b, and Is switched to input the pixel data of the line two lines before to the multiplier 31c. Then, in step S12-6, the pixel data stored in the third temporary storage unit 23 is output to the arithmetic circuit 20 via the second selecting unit 27, and the arithmetic circuit 20 outputs the output pixel data from the input pixel data. Generate
[0056]
FIGS. 15 to 19 show an example in which the YC processing circuit 4 repeats the processing of the flowchart in FIG. 12 at unit time intervals, and executes the YC processing on the pixel data stored in the memory 3.
[0057]
First, FIGS. 15 and 16 show the operating state of the YC processing circuit 4 from time t to time t + 143 when the (62, 63) output pixel of one frame image is generated. 15 and 16, item 1 indicates a method of loading pixel data from the memory 3 to the RAMs 21a to 21c and the registers 22ad to 22ce. Item 2 indicates a write destination of the pixel data read from the memory 3. Item 3 is a pixel to be written from the memory 3 to the RAM 21a to 21c or the like. Items 2 and 3 are the write destinations of the pixel data and the pixels to be read when reading the pixel data of the two pixels 12 per unit time. Item 6 and item 7 are the read and write points (see FIG. 7) of the RAMs 21a to 21b at that time. Item 8 is a switching state of the first selection unit 26. Item 9 is an output pixel.
[0058]
FIG. 17 shows that the pixel data is read from the memory 3 and written into the RAMs 21 a to 21 c of the first storage unit 21 or the registers 22 ad to 22 ce of the second temporary storage unit 22 by the process of step S <b> 12-1 of FIG. 12. Shows the time of day. FIG. 18 shows the time at which pixel data is read from the RAMs 21 a to 21 c and written into the registers 23 aa, 23 ba, and 23 ca of the third temporary storage unit 23 by the processes of steps S 12-4 and S 12-8 of FIG. Is shown. FIG. 19 shows the time at which an output pixel is generated by the process of step S12-6 in FIG. In FIGS. 17 to 19, the numbers in the blocks indicating the pixels 12 indicate the time. For example, the number “4” displayed on the pixel 12 at (65, 1) in FIG. 17 indicates that this pixel is read from the memory 3 at time 4. FIGS. 17 to 19 show the second image block 13b from time 1 when output pixels (63, 63) belonging to the first image blocks 13a and 13a '(see FIGS. 4 and 11) are generated. , 13b ′ until time 144 when an output pixel of (78, 2) is generated. 17 to 19, the numbers in the blocks indicate the time. The time of each pixel in FIGS. 17 to 19 corresponds to the case where t is 0 in FIGS.
[0059]
As shown in FIG. 18, writing of pixel data from each of the RAMs 21a to 21c of the first temporary storage unit 21 to the third temporary storage unit 23 simply proceeds in the horizontal direction one pixel at a time. Further, as shown in FIG. 19, the generation of output pixels by the arithmetic circuit 20 also proceeds in the horizontal direction one pixel at a time. On the other hand, writing of pixel data from the memory 3 to each of the RAMs 21a to 21c of the first temporary storage unit 21 or the registers 22ad to 22ce of the second temporary storage unit 22 (step S12-1 in FIG. 12) Since the processing is performed according to FIGS. 13 and 14, the reading direction changes as shown in FIG. 17, and the reading speed and the reading destination also change.
[0060]
When the pixels necessary for generating the first output pixel of the next line are not stored in the second temporary storage unit 22 according to Steps S13-4 to S13-6 in FIG. 13 and Condition 2 in FIG. Stores pixel data of the first pixel 12 to the second pixel 12 of the next line in the registers 22ad to 22ce of the second temporary storage unit 22. For example, in FIGS. 15 to 17, at times 4 to 9, and at times 68, 69, 132, and 133, the pixel data of the first or second pixel 12 of the line read from the memory 3 is stored in the line. The registers 22ad to 22ce are written instead of the corresponding RAMs 20a to 20c. Then, the pixel data stored in the registers 22ad to 22ce is written to the third temporary storage unit 23 in step S12-3 if the line is updated in step S12-2 in FIG. The pixel data written from the second temporary storage unit 22 to the third temporary storage unit 23 is the pixel data of the next line written from the first temporary storage unit 21 to the third temporary storage unit 23 in step S12-4. In step S12-6, the pixel data is output to the arithmetic circuit 20 together with the third pixel, and all the pixel data necessary for generating the output pixel is supplied to the arithmetic circuit 20. As described above, the second temporary storage unit 22 is provided separately from the RAMs 21a and 21b of the first temporary storage unit 21, and before the line update, the pixel data of the first and second pixels of the next line is updated. Is stored in the second temporary storage unit 22 in advance, thereby eliminating the time loss at the time of updating the line.
[0061]
The direction in which pixel data is read from the memory 3 to the first temporary storage unit 21 and the second temporary storage unit 22 is changed based on step S13-7 in FIG. 13 and condition 3 in FIG. In step S13-7, if the pixel data previously written from the memory 3 to the first temporary storage unit 21 or the second temporary storage unit 22 is not the pixel data of the pixels 12 in the first three lines or less, the next line The pixel data for two lines out of the pixel data necessary for the generation of are already stored in the RAMs 21a to 21c. Therefore, in this case, even when the reading direction is kept horizontal, no loss occurs when updating the image block. However, in step S13-7, if the pixel data previously written from the memory 3 to the first temporary storage unit 21 or the second temporary storage unit 22 is the pixel data of the pixels 12 in the first three lines or less, The generation of the next line is for the next row of the image block 13, and the pixels necessary for the generation of the next line have not yet been stored in the RAMs 21a to 21b. Therefore, if the reading direction remains horizontal, a loss occurs when updating the image block. Therefore, in this case, the reading direction is changed to the vertical direction. By making the readout direction vertical in the lines within the first three rows of the image block 13 in this way, immediately after the last output pixel of the previous image block 13 is generated, the first output pixel of the next image block 13 is It can be generated, and time loss at the time of updating an image block can be reduced. For example, as shown in FIG. 19, the last output pixel (64, 64) of the first image block 13a ′ (see FIG. 11) is generated at time 66, and at time 67 immediately after that, the second image block 13b is generated. ', The first output pixel (65, 1) has been generated. This is made possible by reading pixel data of the pixels 12 belonging to the first three lines of the second image block 13b 'in the vertical direction after time 4, as shown in FIG.
[0062]
According to steps S13-1 to S13-3 in FIG. 13 and condition 1 in FIG. 14, pixel data is read from the memory 3 and stored in the RAM 21a to 21c or the register 22ad according to the distance between the writing point and the reading point of the RAM 21a to 21c. The speed of writing to .about.22ce is changed to 1 pixel and 2 pixels per unit time. When the read point is immediately before the write point in step S13-1 in FIG. 13, it is an ideal state in which the storage capacities of the RAMs 21a to 21c are most efficiently used as described above. Accordingly, pixel data is read from the memory 3 to the RAMs 21a to 21c at one pixel per unit time (the same speed as the output pixel generation speed of the arithmetic circuit 20) so that the read point does not approach the write point any more. On the other hand, if the read point is not immediately before the write point in step S13-1 in FIG. 13, the write point is behind the read point, so that two pixels per unit time (2 pixels per unit time) so that the write point catches up with the read point. The pixel data is read from the memory 3 to the RAMs 21a to 21c at a speed twice as fast as the output pixel generation speed of the arithmetic circuit 20). The image block 13 has 66 pixels in the horizontal direction, while the RAMs 21a to 21c have 64 addresses 33, which are smaller than that. However, by controlling the writing speed to the RAMs 21a to 21c so that the writing point does not pass the reading point and the writing point approaches the reading point, the storage capacity of the RAMs 21a to 21c is efficiently used. Thus, it is possible to realize a process that does not cause a time loss at the time of updating the line or the image block.
[0063]
Next, with reference to FIG. 12 and FIGS. 20 to 37, the operation of the YC processing circuit 4 from time t + 1 to time t + 10 in FIG. 15 (time 1 to time 11 in FIGS. 17 to 19) will be described in detail. Will be described.
[0064]
FIG. 20 shows pixel data stored in the RAMs 21a to 21c of the first temporary storage unit 21 and the registers 22ad to 22ce of the second temporary storage unit 22 immediately before step S12-1 in FIG. The RAM 21a holds pixel data of the pixels (3, 64) to (66, 64). The RAM 21b holds pixel data of the pixels 12 of the pixels (3, 65) to (66, 65). Further, the RAM 21c holds pixel data of the pixels 12 of the pixels (64, 63) to (66, 63) and the pixels (3, 66) to (63, 66). The registers 22ad to 22ce hold pixel data of six pixels (1, 64) to (2, 66).
[0065]
In step S12-1 of FIG. 12, the pixel data of the pixel (64, 66) read from the memory 3 is overwritten on the address 33 of the RAM 21c storing the pixel data of the pixel (64, 63). You. FIG. 21 shows the pixel data held in the RAMs 21a to 21c and the registers 22ad to 22ce after this overwriting.
[0066]
This overwriting is determined by FIG. First, regarding the condition 1, the previous writing point of the RAM 21c during writing is an address for storing the pixel (63, 66), and the previous reading point is an address for storing the pixel (64, 63). The point is immediately before the read point. Regarding the condition 2, the pixels (1, 64) to (2, 66) necessary for generating the next line (the 64th row) are already stored in the second temporary storage unit 22. For condition 3, the pixels (64, 66) read from memory 3 are not pixels in the first three rows of image block 13. Therefore, the loading method A is selected, and the pixel data of the pixel (64, 66) is read from the memory 3 and written to the RAM 21c. By overwriting the data of the pixel (64, 66), the data of the pixel (64, 63) is erased from the RAM 21c, but this pixel (64, 63) is not used again for generating the output pixel. So there is no inconvenience.
[0067]
FIG. 22 shows the third temporary storage unit immediately after the pixel data of the pixel (64, 66) is written from the memory 3 to the RAM 21c in step S12-1 at the time t + 1, that is, immediately before shifting the third temporary storage unit 23. 23 shows the state of the pixel data stored in the storage area 23. Registers 23aa to 23ac hold pixels (62, 64) to (64, 64), registers 23ba to 23bc hold pixels (62, 65) to (64, 65), and registers 23ca to 23cc Pixels (62, 63) to (64, 63) are held. At time t, the arithmetic circuit 20 newly generates pixels (62, 63) from the pixel data of these nine pixels and outputs the pixels.
[0068]
FIG. 23 shows the state of the pixel data stored in the third temporary storage unit 23 immediately after shifting the third temporary storage unit 23 in step S12-7 at time t + 1. The pixel (62, 63) generated at the previous time (time t) does not correspond to the last pixel (64, 63) of the line (63 rows) of the image block 13 (step S12-2). Therefore, the value of the third temporary storage unit 23 is shifted in step S12-8. Specifically, the pixel data of the register 23ab is shifted to the register 23ac, and the pixel data of the register 23aa is shifted and rewritten to the register 23ab. Similarly, the pixel data of the other registers 23bb, 23bc, 23cb, and 23cc is rewritten.
[0069]
FIG. 24 shows the state of the pixel data stored in the third temporary storage unit 23 immediately after step S12-8 at time t + 1. In step S12-8, the pixel (65, 63) is read from the RAM 21c and written into the register 23ca, the pixel (65, 64) is read from the RAM 21a and written into the register 23aa, and the pixel (65, 64) is read from the RAM 21b and stored in the register 23ca. Write to 23ba. Since the six pixels (63, 64) to (64, 65) have already been stored in the third temporary storage unit 23 in step S12-7, the nine pixels (63, 64) to (65, 65) ) Is stored in the third temporary storage unit 23. This is the last time that the pixel data of the pixel (65, 63) is read from the RAM 21c, so that it is not necessary to store the pixel data of the pixel (65, 63) in the RAM 21c. That is, even if the pixel data of the pixel (65, 63) is erased by overwriting, no problem occurs in the subsequent image processing.
[0070]
FIG. 25 shows the output pixel generated at time t + 1 and the state of the pixel data held in the third temporary storage unit 23, that is, the state immediately after step S12-6. In step S12-6, the pixel data of the nine pixels (63, 65) to (65, 65) stored in the third temporary storage unit 23 is sent to the arithmetic circuit 20 via the second selection unit 27. The arithmetic circuit 20 generates the output pixels (63, 63). The registers 23aa to 23ac store pixels on the 64th row, the registers 23ba to 23bc store pixels on the 65th row, and the registers 23ca to 23cc store pixels on the 63rd row. Accordingly, the second selector 22 connects the registers 23aa to 23ac to the multiplier 31b, connects the registers 23ba to 23bc to the multiplier 31c, and connects the registers 23ca to 23cc to the multiplier 31c.
[0071]
The operation at time t + 2 is the same as the operation at time t + 1. FIG. 26 shows the state of holding the pixel data immediately after generating the pixel (64, 63) at time t + 2, that is, the state immediately before step S12-1 at time t + 3. The RAM 21a holds the pixel data of the pixels (3, 64) to (66, 64), the RAM 21b holds the pixel data of the pixels (3, 65) to (66, 65), and the RAM 21c holds the pixel (66). , 63) to (66, 63) and pixel data of pixels (3, 66) to (65, 66). The second temporary storage unit 22 holds pixel data of six pixels (1, 64) to (2, 66).
[0072]
FIG. 27 shows the state of the pixel data held in the first temporary storage unit 21 and the second temporary storage unit 22 immediately after step S12-1 at time t + 3. In step S12-1, based on the determination results of the conditions 1 to 3, the pixels (66, 63) read from the memory 3 are stored in the storage area of the RAM 21c in which the pixel data of the pixels (66, 63) are stored. 66) The pixel data is overwritten.
[0073]
FIG. 28 shows that the six pixels (1, 64) to (2, 66) stored in the second temporary storage unit 22 are read out at step S12-3 at time t + 3 and stored in the third temporary storage unit 23. The state of the pixel data stored in the third temporary storage unit 23 immediately after writing is shown. The pixel (64, 63) generated last time (time t + 2) corresponds to the last pixel (64, 63) of the line (63 rows) of the image block 13. Therefore, according to step S12-2, the pixel data is loaded from the second temporary storage unit 22 to the third temporary storage unit 23 in step S12-3. Specifically, the first selector 26 is switched to the lower position, the pixel data of the register 22ad is stored in the register 23ab, the pixel data of the register 22ae is stored in the register 23ac, the pixel data of the register 22bd is stored in the register 23bb, and the register 22be is stored. Is written to the register 23bc, the pixel data of the register 22cd is written to the register 23cb, and the pixel data of the register 22ce is written to the register 23cc.
[0074]
FIG. 29 shows that the pixel data of the three pixels (3, 64) to (3, 66) are read from the RAMs 21 a to 21 c of the first temporary storage unit 21 at step S12-4 at time t + 3, and the third temporary storage is performed. The state of the pixel data held in the third temporary storage unit 23 immediately after writing to the unit 23 is shown. After the pixel data in the third temporary storage unit 23 is in the state shown in FIG. 27, the second selection unit 27 is switched in step S12-5. Specifically, the second selector 27 is switched so that the registers 23aa to 23ac are connected to the multiplier 31a, the registers 23ba to 23bc are connected to the multiplier 31b, and the registers 23ca to 23cc are connected to the multiplier 31c. Then, in step S12-6, the pixel data of the nine pixels stored in the third temporary storage unit 23 is made to the arithmetic circuit 20, and the output pixel (1, 64) is generated. After the above operation is completed, the setting of the first selector 26 is returned to the upper position.
[0075]
FIG. 30 shows the state of holding the pixel data in the first temporary storage unit 21 and the second temporary storage unit 22 immediately after generating the pixel (1, 64) at time t + 3, that is, immediately before step S12-1 at time t + 4. Is shown. The RAM 21a holds the pixel data of the pixels (3, 64) to (66, 64), the RAM 21b holds the pixel data of the pixels (3, 65) to (66, 65), and the RAM 21c stores the pixel data of the pixel (3). , 66) to (66, 66) are held. The second temporary storage unit 22 holds pixel data of six pixels (1, 64) to (2, 66).
[0076]
FIG. 31 shows the state of the pixel data stored in the first temporary storage unit 21 and the second temporary storage unit 22 immediately after step S12-1 at time t + 4. In step S12-1, the pixel data of the pixel (65, 1) read from the memory 3 is overwritten on the register 22ae storing the pixel (1, 64). The read speed, read destination, and read direction of the pixel data are determined according to FIGS. First, regarding the condition 1, the previous writing point of the RAM 21c during writing is an address for storing the pixel data of the pixel (66, 66), and the previous reading point stores the pixel data of the pixel (3, 66). The write point is immediately before the read point. As for the condition 2, the pixels (65, 1) to (66, 3) required for generating the next line (first row) are not yet stored in the second temporary storage unit 22. Further, regarding the condition 3, the pixels (66, 66) read from the memory 3 and written into the RAM 21c are not the pixels in the first three rows of the image block 13. Therefore, the loading method B is selected, and the pixel data of the pixel (65, 1) is read from the memory 3 and written into the register 22ae.
[0077]
FIG. 32 shows the state of the pixel data stored in the third temporary storage unit 23 immediately after the output pixel (2, 64) is generated at step S12-6 at time t + 4. In step S12-2 at time t + 4, it is determined that the pixel (1, 64) generated last time does not correspond to the last pixel of the line of the image block 13. In step S12-7, the pixel data in the third temporary storage unit 23 is shifted. In step S12-8, the pixels (4, 64) to (4) are stored in the third temporary storage unit 23 from the RAMs 21a to 21c. , 66) is loaded. Thereafter, in step S12-6, the pixel data of the nine pixels stored in the third temporary storage unit 23 is input to the arithmetic circuit 20, and an output pixel (2, 64) is generated.
[0078]
FIG. 33 shows the state of the pixel data stored in the first temporary storage unit 21 and the second temporary storage unit 22 immediately before step S12-1 at time t + 5, and FIG. 34 shows the state of step S12-1 at time t + 5. 2 shows the state of the pixel data stored in the first temporary storage unit 21 and the second temporary storage unit 22 immediately after. The reading speed, the reading destination, and the reading direction of the pixel data from the memory 3 are determined according to FIGS. For condition 1, the previous write point of the RAM 21c during writing is an address for storing the pixel (66, 66), and the previous read point is an address for storing the pixel (4, 66). Is not immediately before the read point. As for the condition 2, the pixel data of the pixels (65, 1) to (66, 3) necessary for generating the next line (first row) has not been stored in the second temporary storage unit 22 yet. Further, for the condition 3, the pixel (65, 1) previously read from the memory 3 and written in the register 22ae is a pixel in the first three rows of the image block 13. Accordingly, the loading method H is selected, and the pixel data of the pixel (65, 2) is read from the memory 3 and written into the register 22be, and the pixel data of the pixel (67, 1) is read from the memory 3 and written into the RAM 21a.
[0079]
FIG. 35 shows the state of the pixel data held in the third temporary storage unit 23 immediately after the output pixel (3, 64) is generated in step S12-6 at time t + 5. In step S12-2 at time t + 5, it is determined that the previously generated pixel (2, 64) does not correspond to the last pixel of the line of the image block 13. In step S12-7, the pixel data in the first temporary storage unit 21 is shifted. Then, in step S12-8, the pixel data of the pixels (5, 64) to (5, 66) is read from each of the RAMs 21a to 21c and written to the third temporary storage unit 23. Thereafter, in step S12-6, the pixel data of the nine pixels stored in the third temporary storage unit 23 is input to the arithmetic circuit 20, and output pixels (3, 64) are generated.
[0080]
From time t + 6 to t + 9, similarly to time t + 5, the pixel data read in the vertical direction from the memory 3 to the first and second temporary storage units 21 and 22 is written (see arrow C in FIG. 10). ). Further, four output pixels (4, 67) to (7, 64) are generated between times T + 6 to T + 9. As shown in FIG. 36, immediately before step S12-1 at time t + 10, pixel data of six pixels (65, 1) to (65, 3) is stored in the second temporary storage unit 22. The pixels (67, 1) and (68, 1) are stored in the RAM 21a, the pixels (67, 2) and (68, 2) are stored in the RAM 21b, and the pixel (67, 3) is stored in the RAM 21c.
[0081]
At step S12-1 at time t + 10, the loading method G is selected according to FIGS. 13 and 14, and the pixels (68, 3) and (69, 1) read from the memory 3 are stored in the RAM 21a as shown in FIG. It is memorized. Since the pixel (7, 64) generated last time in step S12-2 is not the last pixel of the line, the value in the first temporary storage unit 21 is shifted in step S12-7, and in step S12-8, The pixel data of the pixels (10, 64) to (10, 66) are read from the RAMs 21a to 21c and stored in the registers 23aa, 23ba, and 23ca. Finally, in step S12-6, the arithmetic circuit 20 outputs the output pixel (10, 10) based on the pixel data of the nine pixels (8, 64) to (10, 66) stored in the third temporary storage unit 23. 64).
[0082]
In the above-described embodiment, the present invention has been described by taking as an example a case where output pixels are generated from three pixels in the horizontal direction and three adjacent pixels in the vertical direction (see FIGS. 8A to 8C). ). M is an integer of 2 or more, and N is an integer of 2 or more. When the arithmetic circuit generates output pixels from M pixels in the horizontal direction and N pixels in the vertical direction, the first temporary storage unit The second temporary storage unit readablely stores from the first pixel to the (M-1) th pixel of each line from the first pixel to the last pixel of each line. Should be fine. Further, in this case, for the pixels belonging to the first to Nth lines of the image block, the pixel data is vertically transferred from the memory to the first and second temporary storage units as indicated by arrow C in FIG. Just load it.
[0083]
The present invention is not limited to the above embodiment, and various modifications are possible. For example, the present invention can be applied to the resolution conversion circuit 6 in the image processing system 1 of FIG. In addition, the present invention can be applied to an image processing system provided in another device such as a digital video other than the digital camera. The present invention is also applicable to a case where one frame image is processed without being divided into image blocks.
[0084]
【The invention's effect】
As is clear from the above description, in the present invention, before the line is updated, the (M-1) th pixel from the first pixel of the next horizontal pixel row is stored in the second temporary storage unit before the line is updated. And from the second temporary storage means to the third temporary storage means. Therefore, even at the time of updating the line, the third temporary storage unit can output a valid pixel set necessary for generating an output pixel to the calculation unit. In other words, it is possible to eliminate the processing loss at the time of updating the line.
[0085]
In addition, by reading out the first to Nth horizontal pixel columns of the image as described above and writing them in the second and third temporary storage units, even when the image is updated, the third temporary pixel column is read. The storage means can output an effective set of pixels necessary for generating an output pixel to the arithmetic unit. In other words, the loss of processing at the time of updating the image can be eliminated.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating an image processing system of a digital camera including a YC processing circuit according to an embodiment of the present invention.
FIG. 2 is a flowchart for explaining the operation of the image processing system of FIG. 1;
FIG. 3 is a circuit diagram showing a YC processing circuit.
FIG. 4 is a diagram showing an image for one frame.
FIG. 5 is a diagram showing assignment of coordinates to pixels.
FIG. 6 is a schematic diagram showing a structure of pixel data.
FIG. 7 is a schematic diagram showing a RAM.
FIGS. 8A, 8B, and 8C are diagrams for explaining a method of generating an output pixel.
FIG. 9 is a diagram for explaining a write destination of pixel data of each pixel in an image block.
FIG. 10 is a diagram illustrating a writing order of pixel data of pixel data in an image block.
FIG. 11 is a diagram illustrating an output order of output pixels.
FIG. 12 is a flowchart illustrating the operation of the YC processing circuit.
FIG. 13 is a flowchart of a subroutine of step S12-1.
FIG. 14 is a table showing conditions for determining a loading method for first and second storage units;
FIG. 15 is a table illustrating the operation of the YC processing circuit from time t to time t + 143.
FIG. 16 is a table for explaining the operation of the YC processing circuit from time t to time t + 143.
FIG. 17 is a diagram illustrating a time when pixel data of each pixel is transferred from a memory to a RAM.
FIG. 18 is a diagram illustrating a time when pixel data stored in a RAM is transferred to a third temporary storage unit.
FIG. 19 is a diagram illustrating the generation time of each generated pixel.
FIG. 20 is a schematic diagram showing pixel data held in first and second temporary storage units immediately after time t.
FIG. 21 is a schematic diagram showing pixel data held in first and second temporary storage units after reading pixel data from a memory to a RAM at a time t + 1.
FIG. 22 is a schematic diagram showing pixel data held in the third temporary storage unit before the register of the third temporary storage unit is shifted at time t + 1.
FIG. 23 is a schematic diagram illustrating pixel data held in the third temporary storage unit after the register of the third temporary storage unit is shifted at time t + 1.
FIG. 24 is a schematic diagram showing pixel data stored in a third temporary storage unit after pixel data has been transferred one pixel at a time from a RAM to a third temporary storage unit at time t + 1.
FIG. 25 is a schematic diagram illustrating pixel data held in a third temporary storage unit when an output pixel is generated at time t + 1.
FIG. 26 is a schematic diagram showing pixel data held in first and second temporary storage units immediately after time t + 2.
FIG. 27 is a schematic diagram illustrating pixel data stored in first and second temporary storage units after reading pixel data from a memory to a RAM at a time t + 3.
FIG. 28 is a schematic diagram showing pixel data held in the third temporary storage unit after transferring pixel data from the second temporary storage unit to the third temporary storage unit;
FIG. 29 is a schematic diagram showing pixel data stored in the third temporary storage unit after pixel data has been transferred one pixel at a time from the RAM to the third temporary storage unit at time t + 3.
FIG. 30 is a schematic diagram showing the pixel data stored in the first and second temporary storage units immediately after time t + 3.
FIG. 31 is a schematic diagram showing pixel data held in first and second temporary storage units after reading pixel data from a memory to a RAM at time t + 4.
FIG. 32 is a schematic diagram illustrating pixel data held in a third temporary storage unit after an output pixel is generated at time t + 4.
FIG. 33 is a schematic diagram showing pixel data held in first and second temporary storage units immediately after time t + 4.
FIG. 34 is a schematic diagram showing pixel data held in first and second temporary storage units after reading pixel data from a memory to a RAM and a register at time t + 5.
FIG. 35 is a schematic diagram illustrating pixel data held in a third temporary storage unit after an output pixel is generated at time t + 5.
FIG. 36 is a schematic diagram showing pixel data held in first and second temporary storage units immediately after time t + 9.
FIG. 37 is a schematic diagram showing pixel data held in first and second temporary storage units after reading pixel data from a memory to a RAM and a register at time t + 10.
FIG. 38 is a schematic diagram showing a writing order of pixel data for one frame.
FIG. 39 is a schematic diagram showing a writing order of pixel data for each image block.
FIG. 40 is a schematic diagram for explaining a loss at the time of updating a line.
FIG. 41 is a schematic diagram for explaining a loss at the time of updating an image block.
[Explanation of symbols]
1 Image processing system
2 Imaging circuit
3 memory
4 YC processing circuit
5 SRAM
6. Resolution conversion circuit
7 Compression processing circuit
8 Recording media
9 Control circuit
9a Clock control unit
11 images
12 pixels
13 pixel block
14a Pixel data section
14b Even-odd bit
14c Valid bit
20 arithmetic circuit
21 First temporary storage unit
21a, 21b, 21c RAM
22 Second temporary storage unit
23 Third temporary storage unit
26 First Selection Unit
27 Second Selection Unit
31a, 31b, 31c Multiplier
32 adder

Claims (7)

An image storage unit for storing an image,
A computing unit that computes M horizontally and N vertically adjacent pixels contained in the image to generate output pixels, where M is an integer of 2 or more and N is an integer of 1 or more. When,
A first temporary storage unit that readablely stores from the Mth pixel to the last pixel of each horizontal pixel column of the image,
A second temporary storage unit that readablely stores the pixels from the first pixel to the (M-1) th pixel of each horizontal pixel column of the image,
The pixels stored in the first temporary storage unit are delayed, and pixels are input from the second temporary storage unit, and the M pixels in the horizontal direction and the N pixels adjacent to each other in the vertical direction are subjected to the arithmetic operation. An image processing device comprising: a third temporary storage unit that outputs the image data to the unit at the same time. Before the pixel used to generate the last output pixel of one horizontal pixel column is stored in the third temporary storage unit, the (M−1) th pixel from the first pixel of the next horizontal pixel column is stored. Pixels are read from the image storage unit and written to the second temporary storage unit, and the pixel used for generating the last output pixel corresponding to the one horizontal pixel column is the third temporary pixel. When output from the storage unit to the arithmetic unit, the (M-1) th pixel from the head pixel of the next line is read from the second temporary storage unit and written to the third temporary storage unit. The image processing device according to claim 1, further comprising a unit. N is an integer of 2 or more;
The control unit sequentially reads N pixels arranged in the vertical direction from the image storage unit for pixels belonging to the first to Nth horizontal pixel columns of the image, and stores the first temporary storage in the first temporary storage. The image processing apparatus according to claim 2, wherein an operation of writing to the storage unit or the second temporary storage unit is repeated while moving a reading position in a horizontal direction. The arithmetic unit generates one output pixel per unit time,
The first temporary storage unit includes N RAMs,
If the previous write address of the RAM being written is immediately before the previous read address, the control unit reads one pixel from the image storage unit within the unit time and reads out the one pixel from the image storage unit. If the last write address of the RAM being written to and being written to the RAM is one or more times before the immediately preceding read address, a plurality of pixels are stored in the image storage unit within the unit time. The image processing apparatus according to claim 3, wherein the image processing apparatus reads data from the memory and writes the data to the RAM and / or the third temporary storage unit. An M is an integer greater than or equal to 2 and N is greater than or equal to 1 and an output pixel is generated by arithmetically processing M horizontally and N vertically adjacent pixels included in the image stored in the image storage unit. An image processing method, which is an integer of
Pixels are read out from the image storage unit while moving the readout position in the horizontal direction, and written to the first temporary storage unit;
Reading a pixel from the first temporary storage unit and writing it to a third temporary storage unit;
Outputting the M pixels in the horizontal direction and the N pixels adjacent to each other in the vertical direction to the arithmetic unit by delaying in the third temporary storage means;
Generating an output pixel from the M × N pixels by the arithmetic circuit;
Before the pixel used for generating the last output pixel corresponding to one horizontal pixel column is stored in the third temporary storage unit, the M-th to M-th pixels are output from the first pixel in the next horizontal pixel column. Up to the first pixel is read from the image storage unit and written to the second temporary storage unit,
When a pixel used to generate the last output pixel corresponding to the one horizontal pixel row is output from the third temporary storage unit to the arithmetic unit, the next temporary storage unit outputs the next pixel from the second temporary storage unit. An image processing method of reading out from the top pixel to the (M-1) th pixel of the line and writing the readout into the third temporary storage unit. N is an integer of 2 or more;
For a plurality of pixels belonging to the first to Nth horizontal pixel columns of the image, N pixels arranged in a vertical direction are sequentially read out from the image storage unit and the first temporary storage unit or the 6. The image processing method according to claim 5, wherein the operation of writing to the second temporary storage unit is repeated while moving the read position in the horizontal direction. The arithmetic unit generates one output pixel per unit time,
The first storage unit includes N RAMs,
If the previous write address of the RAM being written is immediately before the previous read address, one pixel is read from the image storage unit within the unit time and the second temporary storage unit or the second 3 is written in the temporary storage unit,
If the last write address of the RAM being written is one or more before the immediately preceding read address, a plurality of pixels are read from the image storage unit within the unit time, and the RAM or the third pixel is read. Write to the temporary storage of
The image processing method according to claim 5.

Images