JP4264602B2 - Image processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus suitable for use in a still image capturing apparatus, and more particularly to an image processing apparatus with a reduced circuit scale.
[0002]
[Prior art]
A digital still camera takes image data obtained by a CCD image sensor into a memory or a recording medium, and then transfers the image data to a so-called personal computer or the like. Until now, most of these digital still cameras are compatible with VGA (Video Graphics Array) systems.
[0003]
For example, as shown in FIG. 16, the digital still camera 200 includes a CCD image sensor 201 that generates an image signal, an input processing / image processing circuit 202, and a memory controller 203 that performs processing for reading or writing image data. An output processing circuit 204 for converting into an output image of a predetermined method, a finder 205 for displaying the state of the subject at the time of shooting, a recording unit 207 for recording compressed image data via the CPU bus 206, A compression / decompression circuit 208 for compressing or decompressing image data, a memory 209 made of, for example, a DRAM or the like for storing image data, and a CPU 210 for controlling the entire apparatus are provided.
[0004]
Prior to the start of photographing the subject, the user needs to check the subject image displayed on the viewfinder 205 (finder mode). At this time, the CCD image sensor 201 supplies an image signal obtained by photoelectric conversion with imaging light from the subject to the input processing / image processing circuit 202. The input processing / image processing circuit 202 digitizes the image signal by performing correlated double sampling processing, for example, further performs predetermined signal processing such as gamma correction, knee processing, camera processing, and the like, and supplies the result to the memory controller 203. . The memory controller 203 supplies the image data from the input processing / image processing circuit 202 to the output processing circuit 204 under the control of the CPU 210. The output processing circuit 204 encodes the image data in, for example, the NTSC (National Television System Committee) system, further converts it to analog, and supplies it to the finder 205. As a result, the subject to be imaged is displayed on the viewfinder 205.
[0005]
On the other hand, when the user presses a shutter button (not shown) to shift to the recording mode, the memory controller 203 writes the image data supplied from the input processing / image processing circuit 202 in the memory 209. The CPU 210 reads out the image data from the memory 209, performs the JPEG (Joint Photographic Experts Group) compression processing, for example, in the compression / expansion circuit 208 and records the image data in the recording unit 207.
[0006]
When the CPU 210 shifts to the reproduction mode by a predetermined operation by the user, the CPU 210 reads the image data from the recording unit 207, performs JPEG expansion processing on the image data in the compression / decompression circuit 208, and then outputs the memory controller 203 and output processing. This is supplied to the finder 205 via the circuit 204. Thus, the photographed image is displayed on the finder 205.
[0007]
[Problems to be solved by the invention]
The input processing / image processing circuit 202 has image processing functions in the horizontal direction and the vertical direction, and converts the image data signal supplied from the CCD image sensor 201 into Y, Cb, and Cr signals. The input processing / image processing circuit 202 includes a horizontal buffer, a horizontal image processing circuit, a vertical buffer, and a vertical image processing circuit.
[0008]
The horizontal buffer buffers the image data and supplies the image data delayed by, for example, one pixel, two pixels, and three pixels to the horizontal image processing circuit. The horizontal image processing circuit performs horizontal image processing by multiplying each image data by a predetermined coefficient and combining them. The vertical buffer supplies image data delayed by one line, two lines, and three lines to the vertical image processing circuit. The vertical image processing circuit performs vertical image processing by multiplying each image data from the vertical buffer by a predetermined coefficient and combining them.
[0009]
Here, for example, as shown in FIG. 17, the vertical buffer 202a includes a number of line buffers corresponding to the number of vertical taps, and these line buffers are connected in series. In these line buffers, image data corresponding to the horizontal size of the input image data is stored.
[0010]
When the image data supplied to the input processing / image processing circuit 202 is, for example, the size of the image A shown in FIG. 18, the image data of the image A occupies about 1/2 of the line buffer as shown in FIG. Similarly, the image data of the image B occupies about 2/3 of the line buffer, and the image data of the image C occupies the entire line buffer.
[0011]
As described above, the line buffer must have a capacity capable of storing at least one line of image data. Therefore, the larger the size of the image data, the larger the circuit size of the vertical buffer and the higher the production cost.
The present invention has been proposed in view of such circumstances, and an object of the present invention is to provide an image processing apparatus capable of reducing the circuit scale and suppressing the production cost even when the size of the image data increases. To do.
[0012]
[Means for Solving the Problems]
In order to solve the above-described problems, an image processing apparatus according to the present invention stores at least one piece of image data, and stores image data for N pixels in the horizontal direction or the vertical direction from the storage unit. Reading means for reading all the image data for N pixels in the horizontal direction or the vertical direction in the vertical direction or the horizontal direction by repeatedly reading the data for each row or one column for the entire image data for one sheet. A buffer memory for storing image data for N pixels is connected in series, and the image data read out by the reading means is input to one of the buffer memories connected in series and delayed from each buffer memory. By combining the delay means for outputting image data and the weighted image data output from each buffer memory of the delay means. And a horizontal direction or a signal processing means for performing signal processing in the vertical direction of the image data, the buffer memories of the delay means, the data amount of the horizontal lines or vertical lines of one image data stored in the memory means A storage capacity for storing image data for N pixels, and the reading means stores one screen of image data for N pixels in each row or column in the horizontal direction or the vertical direction from the storage means. The image data for N pixels in the horizontal direction or the vertical direction are all read out in the vertical direction or the horizontal direction by being repeatedly read out a number of times corresponding to the size of the image and input to the signal processing means through the delay means. To do.
[0013]
In the above image processing apparatus, the storage capacity of each buffer memory constituting the delay means is a storage capacity for storing image data for N pixels which is smaller than the data amount of the horizontal line of one piece of image data stored in the storage means. , from the storage means, by repeatedly reading a number of times corresponding to one screen size in the horizontal direction or the vertical direction of image data of N pixels in each row or every column of N pixels in the horizontal direction or the vertical direction by entering the reading of all the image data in the vertical direction or the horizontal direction to the signal processing means via said delay means performs the entire image data for one sheet, from the respective buffer memories in the signal processing means Image processing is performed based on the image data.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0015]
The present invention is applied to, for example, a digital still camera 1 having the configuration shown in FIG.
The digital still camera 1 controls an image generation unit 10 that generates an image signal, a signal processing unit 20 that performs predetermined signal processing on image data, an image memory 32 that includes an SDRAM, and the signal processing unit 20. And a control unit 40.
[0016]
The image generation unit 10 is a solid-state imaging device that generates an image signal, for example, a CCD image sensor 11, and a sample / hold-analog / digital circuit that performs sample hold processing and digitization processing on the image signal and outputs image data ( (Hereinafter referred to as “S / H-A / D circuit”) 12 and a timing generator 13 for generating a timing signal. The timing generator 13 generates a horizontal synchronization signal and a vertical synchronization signal for controlling each circuit of the image generation unit 10 based on the synchronization signal supplied from the signal processing unit 20.
[0017]
The CCD image sensor 11 generates image data equivalent to, for example, XGA (eXtended Graphics Array: 1024 × 768) composed of 800,000 pixels. The CCD image sensor 11 is driven based on the synchronization signal from the timing generator 13 and outputs an image signal of 30 frames per second. The CCD image sensor 11 has a thinning function of the image signal, and the vertical component of the image signal is thinned out to 1/2, 1/3, 1/4... And output under the control of the control unit 40. be able to.
[0018]
The S / H-A / D circuit 12 also performs sample hold and A / D conversion processing at a predetermined sampling interval based on the synchronization signal from the timing generator 13 and supplies this image data to the signal processing unit 20.
[0019]
The signal processing circuit 20 is configured by a single LSI (Large Scale Integrated circuit). The signal processing unit 20 includes an input processing circuit 21 that performs input processing and camera processing on image data from the image generation unit 10, a memory controller 22 that controls reading / writing of image data to / from the image memory 32, and NTSC / PAL ( Phase Alternation by Line) An encoder 23, a D / A converter 24 that converts image data into analog form and outputs it to the outside, and a sync generator 26 that generates a synchronization signal and supplies it to the timing generator 13.
[0020]
The signal processing unit 20 also includes a memory interface 27 that is an interface of the image memory 32, a resolution conversion circuit 28 that performs resolution conversion processing of image data, and a JPEG (Joint Photographic Experts Group) that performs compression / decompression processing of image data. An encoder / decoder 29, a JPEG interface 30 that is an interface of the JPEG encoder / decoder 29, and a host interface 31 that is an interface for transmitting / receiving data to / from a CPU of the control unit 40 are provided.
[0021]
The input processing circuit 21 performs digital clamping, shading correction, aperture correction, gamma correction, color processing, and the like on the image data from the S / H-A / D circuit 12 and supplies the result to the memory controller 22. The input processing circuit 21 has a function of converting input data into Y, Cb, and Cr by signal processing. When the resolution of the image data is larger than, for example, the VGA (Video Graphics Array) format, the input processing circuit 21 can also perform processing to lower the resolution. The input processing circuit 21 performs auto-focus and auto-iris detection processing, sends the data to the control unit 40, and automatically adjusts the focus mechanism and iris mechanism. Further, the input processing circuit 21 performs auto white balance adjustment by detecting the signal levels of the three primary color signals constituting the image data.
The memory controller 22 controls the image data supplied from the input processing circuit 21 and other circuits to be written into the image memory 32 via the memory interface 27 and reads out the image data in the image memory 32 via the memory interface 27. Do. At this time, the memory controller 22 also detects whether there is a defective pixel in the CCD image sensor 11 based on the image data stored in the image memory 32.
[0022]
The memory controller 22 supplies the image data read from the image memory 32 to, for example, the NTSC / PAL encoder 23. When the image data is supplied from the memory controller 22, the NTSC / PAL encoder 23 encodes the image data using the NTSC method or the PAL method and supplies the encoded image data to the D / A converter 24. The D / A converter 24 converts the image data into analog form and outputs it through the external terminal 25.
[0023]
The memory controller 22 supplies the image data read from the memory controller 22 to the resolution conversion circuit 28 to perform resolution conversion processing, and writes the resolution-converted image data output from the resolution conversion circuit 28 into the image memory 32. .
[0024]
The memory controller 22 supplies image data to the JPEG encoder / decoder 29 via the JPEG interface 30 to perform still image compression processing, and further, the image data expanded by the JPEG encoder / decoder 29 is image memory 32. Also write to.
[0025]
The image memory 32 not only stores image data as described above, but also stores OSD (On Screen Display) data which is so-called character generator data. The OSD data referred to here is bitmap data. The memory controller 22 also controls reading / writing of the OSD data. Note that the NTSC / PAL encoder 23 combines the image data and the OSD data.
[0026]
The control unit 40 includes a CPU (Central Processing Unit) 41 that controls each circuit of the signal processing unit 20, a DRAM (Dynamic Random Access Memory) 42 that temporarily stores image data and other control data, and a control program for the CPU 41. Are stored in a ROM (Read Only Memory) 43, a flash memory interface 44 that is an interface for exchanging image data with a recording device 51 that is made of, for example, a flash memory, and a communication circuit 52 that is made of, for example, an IrLED. An IrDA interface 45 is provided.
[0027]
For example, the CPU 41 writes the image data compressed by the JPEG encoder / decoder 29 to the recording device 51 including the flash memory via the flash memory interface 44, and reads the image data from the recording device 51 to read the JPEG encoder. / Supplied to decoder 29. Further, the CPU 41 outputs the image data from the JPEG encoder / decoder 29 and the image data read from the recording device 51 to the outside as infrared light via the IrDA interface 45 and the communication circuit 52.
[0028]
Here, a simple configuration of the digital still camera 1 is shown in FIG.
[0029]
The input processing circuit 21 supplies the image data from the CCD image sensor 11 to the image memory 32 via the image data bus 33. The NTSC / PAL encoder 23 performs a predetermined encoding process on the image data from the image memory 32 and supplies the image data to the finder 36. As a result, an image of the subject is displayed on the finder 36. The finder 36 displays an image corresponding to the image data up to the VGA format.
[0030]
The memory controller 22 performs data transfer between the signal processing circuits connected to the image memory 32 and the image data bus 33. The resolution conversion circuit 28 performs resolution conversion processing of the image data from the image memory 32 and supplies the result to the image memory 32. The JPEG encoder / decoder 29 compresses the image data from the image memory 32 by the JPEG method and supplies the compressed data to the CPU 41 via the CPU bus 34. The CPU 41 writes the compressed image data into the recording device 51 via the CPU bus 34. The CPU 41 can also output the compressed image data to the outside via the CPU bus 34 and the communication circuit 52.
[0031]
As described above, according to FIG. 2, in the signal processing unit 20, each circuit is connected via the image data bus 33. The image data bus 33 is a virtual bus and indicates that there is a limit to the transfer band of image data exchanged by each circuit.
[0032]
In the signal processing unit 20, each other circuit such as the NTSC / PAL encoder 23 and the resolution conversion circuit 28 sends a request signal (request) indicating that image data is requested to the memory controller 22 before processing the image data. Send. Each of these circuits also transmits a request signal to the memory controller 22 when outputting the image data after the processing of the image data is completed.
[0033]
On the other hand, when the memory controller 22 receives a request signal from each circuit, the memory controller 22 selects a circuit having a higher priority from each circuit, and transmits an acknowledge signal to the selected circuit. Here, the acknowledge signal refers to a signal indicating that image data is supplied to a circuit that receives the signal or that image data output from the circuit that has received the signal is ready to be received. Then, the memory controller 22 reads the image data from the image memory 32 and supplies it to the circuit to which the acknowledge signal is transmitted via the image data bus 33. Further, the memory controller 22 receives the image data output from the circuit to which the acknowledge signal is transmitted and performs a process of writing the image data into the image memory 32.
[0034]
The memory controller 22 can preferentially select a circuit that needs to be processed in real time when receiving a request signal from each circuit simultaneously. For example, the memory controller 22 preferentially selects the input processing circuit 21 and the NTSC / PAL encoder 23 when displaying the subject image on the finder 36. Further, the memory controller 22 may determine the bus occupation usage rate of the image data in the image data bus 33 and determine the priority order of each circuit according to the occupation rate.
[0035]
If the memory controller 22 can supply image data to each circuit within the transfer bandwidth limit of the image data bus 33, the memory controller 22 transmits an acknowledge signal to each circuit in a time division manner, and each circuit has a predetermined value. You may control to perform the process of. As a result, the memory controller 22 effectively accesses the circuits in real time and writes the image data from each circuit to the image memory 32, or reads the image data from the image memory 32 and supplies it to each circuit. can do.
[0036]
Further, even when the memory controller 22 accesses an external circuit (not shown) via the image data bus 33, the external controller can transmit the above-described request signal or receive an acknowledge signal. For example, within the transfer bandwidth limit range of the image data bus 33, it is possible to access simultaneously in a time-sharing manner as with each circuit in the signal processing unit 20. That is, if the memory controller 22 is within the bandwidth of the image data bus 33, the memory controller 22 accesses these circuits simultaneously in a time division manner regardless of the number of circuits in the signal processing unit 20 and the number of external circuits. Can do.
[0037]
As described above, the memory controller 22 performs arbitration of the image data bus 33, control of image data writing / reading between the image memory 32 and each circuit, and data transfer to the CPU bus 34.
[0038]
Next, a specific flow of image data in the signal processing unit 20 will be described with reference to FIG.
[0039]
The input processing circuit 21 performs a CCD interface 21a that performs predetermined signal processing on the image data from the image generation unit 10, a detection circuit 21b that performs detection processing to perform the processing of the CCD interface 21a, and image data conversion processing. And a camera digital signal processor (hereinafter referred to as “camera DSP”) 21c.
[0040]
The CCD interface 21a performs digital clamp and white balance adjustment on the image data composed of the red signal, green signal, and blue signal (R, G, B) from the S / H-A / D circuit 12 shown in FIG. Processing such as gamma correction is performed, and if necessary, horizontal component thinning processing is also performed. After performing such processing, the CCD interface 21 a supplies image data to the camera DSP 21 c or supplies it to the memory controller 22 via the image data bus 33.
[0041]
The detection circuit 21b performs detection processing for autofocus, auto iris, and white balance adjustment from the image data of the CCD interface 21a.
[0042]
The camera DSP 21c converts image data composed of RGB from the CCD interface 21a into image data composed of a luminance signal Y and chroma signals (color difference signals) Cb and Cr. In addition, the camera DSP 21c includes such a simple resolution conversion circuit 21d that performs such processing and simply converts the resolution of the image data.
[0043]
The simple resolution conversion circuit 21d converts the resolution of the image data to be low when the resolution of the image data generated by the CCD image sensor 11 is larger than, for example, the VGA format.
[0044]
Specifically, as shown in FIG. 4, the simple resolution conversion circuit 21d includes a BY / RY separation circuit 61 that separates color difference signals, and a horizontal linear interpolation circuit 62 that performs horizontal interpolation processing. A BY / R-Y combining circuit 63 that combines color difference signals, a 1H delay circuit 64 that gives a delay of one horizontal scanning period (1H period) to each signal, and a vertical linear interpolation circuit 65 are provided.
[0045]
The BY / R-Y separation circuit 61 separates the color difference signals BY and RY, which are chroma signals Cb and Cr, from the image data from the camera DSP 21c, and supplies them to the horizontal linear interpolation circuit 62. The horizontal linear interpolation circuit 62 performs a horizontal interpolation process on the luminance signal Y and the color difference signals BY and RY to reduce the horizontal resolution so that the luminance signal Y and the color difference signal after the interpolation process are performed. BY and RY are supplied to the BY / RY composite circuit 63.
[0046]
The BY / R-Y combining circuit 63 combines the color difference signals BY and RY, and the luminance signal Y from the horizontal linear interpolation circuit 62 and the combined color difference signals BY and RY. The 1H delay circuit 64 and the vertical linear interpolation circuit 65 are supplied. The 1H delay circuit 64 gives a delay of 1H period to the luminance signal Y and the color difference signals BY and RY, respectively, and supplies them to the vertical linear interpolation circuit 65. The vertical linear interpolation circuit 65 performs vertical linear interpolation processing based on the luminance signal Y and the color difference signals BY and RY from the BY / RY synthesis circuit 63 and the 1H delay circuit 64, and As a result, image data composed of the luminance signal Y ′ and the color difference signals (BY) ′ and (R−Y) ′ whose horizontal and vertical resolutions are lowered is output.
[0047]
Further, the resolution conversion circuit 28 performs resolution conversion processing for converting, for example, [p × q] image data into [m × n] image data. The resolution conversion circuit 28 is a process that is performed mainly to suppress the image data generated by the CCD image sensor 11 to a predetermined resolution when the image data is high resolution. It may be processed.
[0048]
Specifically, as shown in FIG. 5, the resolution conversion circuit 28 includes an input buffer 71 for storing image data input from the image data bus 33, and a horizontal buffer for horizontally buffering image data from the input buffer 71. A direction buffer 72, a horizontal direction conversion processing circuit 73 for performing horizontal resolution conversion processing on the image data from the horizontal direction buffer 72, and a vertical direction buffer for buffering image data from the horizontal direction conversion processing circuit 73 in the vertical direction. 74, a vertical direction conversion processing circuit 75 that performs resolution conversion processing in the vertical direction, and an output buffer 76 that performs buffering at the time of output.
[0049]
When the resolution conversion circuit 28 is ready to perform the resolution conversion of the image data, the resolution conversion circuit 28 outputs a (read) request signal that requests the memory controller 22 to read the image data from the image memory 32, and further, After the data conversion process, a request signal (write) requesting the memory controller 22 to write the image data into the image memory 32 is output. Further, the resolution conversion circuit 28 receives an acknowledge signal indicating that the memory controller 22 has responded to the request signal.
[0050]
On the other hand, the horizontal buffer 72 has a configuration in which a first delay circuit 81, a second delay circuit 82, and a third delay circuit 83 that give a delay of one pixel are connected in series as shown in FIG. Is done. Therefore, the first delay circuit 81 outputs image data delayed by one pixel, the second delay circuit 82 outputs image data delayed by two pixels, and the third delay circuit 83 delays by three pixels. Output image data.
[0051]
As shown in FIG. 6, the horizontal direction conversion processing circuit 73 includes first to fourth multipliers 84, 85, 86, and 87 and first to third adders 88, 89, and 90. In some cases, a circuit for normalizing data is added after the adder 90.
[0052]
The first multiplier 84 multiplies the image data supplied from the input buffer 71 by a predetermined coefficient and supplies it to the adder 88. The second multiplier 85 multiplies the image data supplied from the first delay circuit 81 by a predetermined coefficient and supplies the result to the adder 88. The third multiplier 86 multiplies the image data supplied from the second delay circuit 82 by a predetermined coefficient and supplies the result to the adder 89. The fourth multiplier 87 multiplies the image data supplied from the third delay circuit 83 by a predetermined coefficient and supplies the result to the adder 90. The first adder 88 combines the image data and supplies the synthesized image data to the second adder 89. The second adder 89 combines the image data and supplies the synthesized image data to the third adder 90. The third adder 90 synthesizes each image data and supplies it to the vertical buffer 74 as image data that has been subjected to resolution conversion processing in the horizontal direction.
[0053]
As described above, the horizontal direction conversion processing circuit 73 performs a process of compensating for or thinning out the pixels in the horizontal direction by combining the image data having a delay of one pixel with predetermined weighting and combining them. , Convert the horizontal resolution.
[0054]
For example, as shown in FIG. 6, the vertical buffer 74 is configured by connecting first to third buffer memories 91, 92, 93 that give a delay of one line in series. Therefore, the first buffer memory 91 outputs image data delayed by one line, the second buffer memory 92 outputs image data delayed by two lines, and the third buffer memory 93 is delayed by three lines. Output image data.
[0055]
As shown in FIG. 6, the vertical conversion processing circuit 75 includes fifth to eighth multipliers 94, 95, 96, and 97 and fourth to sixth adders 98, 99, and 100. In some cases, a circuit for normalizing data is added after the adder 90.
[0056]
The fifth multiplier 94 multiplies the image data supplied from the horizontal direction conversion circuit 73 by a predetermined coefficient and supplies the result to the fourth adder 98. The sixth multiplier 95 multiplies the image data supplied from the first line memory 91 by a predetermined coefficient and supplies the result to the fourth adder 98. The seventh multiplier 96 multiplies the image data supplied from the second line memory 92 by a predetermined coefficient and supplies the result to the fifth adder 99. The eighth multiplier 97 multiplies the image data supplied from the third line memory 93 by a predetermined coefficient and supplies the result to the sixth adder 100. The fourth adder 98 synthesizes each image data and supplies it to the fifth adder 99. The fifth adder 99 synthesizes each image data and supplies it to the sixth adder 100. The sixth adder 100 synthesizes each image data and outputs it as image data that has been subjected to resolution conversion processing in the vertical direction.
[0057]
As described above, the vertical direction conversion processing circuit 75 performs processing for compensating for or thinning out pixels in the vertical direction by combining image data delayed by one line with predetermined weighting, respectively. Or convert the vertical resolution.
[0058]
In FIG. 6, the resolution conversion circuit 28 performs the resolution conversion process in the vertical direction after performing the resolution conversion process in the horizontal direction. However, as shown in FIG. 7, the resolution conversion circuit 28 performs the resolution conversion process in the vertical direction. Alternatively, horizontal resolution conversion processing may be performed. That is, the resolution conversion circuit 28 supplies the image data from the input buffer 71 to the vertical buffer 74, and in the order of the vertical buffer 74, the vertical conversion processing circuit 75, the horizontal buffer 72, and the horizontal conversion processing circuit 73. You may make it the structure which processes.
[0059]
Further, the first to third buffer memories 91, 92, 93 in the vertical buffer 74 can store image data for one line (1H). However, as shown in FIG. For example, the image data of a smaller number of N pixels (pixel length N) may be stored. At this time, the memory controller 22 needs to read the image data stored in the image memory 32 every N pixels, as shown in FIG.
[0060]
Specifically, the memory controller 22 reads the image data for one screen stored in the image memory 32 in the vertical direction by N pixels for each line. Here, as shown in FIG. 10, one screen is composed of p × q [pixels], the coordinates of the upper left pixel are (1, 1), the coordinates of the upper right pixel are (p, 1), and the lower left pixel is The coordinates are (1, q), and the coordinates of the lower right pixel are (p, q).
[0061]
As shown in FIG. 11, the memory controller 22 first reads image data for N pixels in the horizontal direction for each line in the order of the first row, the second row,. Thereby, the memory controller 22 has N pixels of image data from the left end, that is, a range (N × q pixels) surrounded by (1, 1) (1, q) (N, q) (N, 1). Image data (hereinafter referred to as “image data group (1)”) is read.
[0062]
Next, the memory controller 22 selects image data (hereinafter referred to as “image data group”) in a range surrounded by (N−1,1) (N−1, q) (N−2, q) (N−2,1). (2) "and so on) is read out. Here, when the memory controller 22 reads the column (1) and the image data group (2), the memory controller 22 reads the (N-1) th and Nth column image data twice.
[0063]
This is because the vertical direction conversion processing circuit 75 performs interpolation processing from surrounding pixels, so that the first and third pixels stored in the first to third buffer memories 91, 92, and 93 are processed results. It is because it does not make a target. For example, the pixel (N, 1) is not a target of the vertical interpolation processing result when the image data group (1) is read out. However, this (N, 1) pixel is also read out when the image data group (2) is read out, and becomes an object of the interpolation processing result at this time.
[0064]
Similarly, the memory controller 22 reads out image data for N pixels in the horizontal direction for each line so as to include the image data for the last second column of the immediately preceding image data group, and thereby the image data group Is supplied to the resolution conversion circuit 28.
[0065]
Image data matching the capacity of the first to third buffer memories 91, 92, 93 is supplied to the vertical buffer 74 of the resolution conversion circuit 28 for each line. Therefore, the first to third buffer memories 91, 92, 93 store image data shifted by one line. The vertical direction conversion processing circuit 75 can perform vertical resolution conversion processing based on the image data from the first to third buffer memories 91, 92, 93 of the vertical buffer 74.
[0066]
As described above, the memory controller 22 reads out according to the capacity of the buffer memory even if the capacity of the buffer memory required for the vertical image degree conversion is less than one line, thereby allowing the resolution conversion circuit 28 to read. It is possible to perform resolution conversion in the vertical direction.
[0067]
Here, the read overlap between the image data groups is two columns, but there may be a case where the overlap is larger than two columns or there is no overlap. Further, the present invention is not limited to resolution conversion and is also applied to image signal processing such as camera signal processing.
[0068]
In addition, here, the case where the buffer memory is used for the interpolation process in the vertical direction has been described as an example, but the same applies to the case where the buffer memory is used for the interpolation process in the horizontal direction.
[0069]
That is, for example, as shown in FIG. 12, the resolution conversion circuit 28 may perform horizontal resolution conversion using a horizontal buffer 72a composed of a buffer memory having a capacity of N pixels. As shown in FIG. 13, the memory controller 22 may read image data for N pixels in the vertical direction for each column in the order of the first column, the second column,. As in the case of the vertical interpolation process described above, the memory controller 22 needs to read the image data stored at the beginning and end of the buffer memory twice so as to be the target of the horizontal interpolation process. .
[0070]
As described above, the memory controller 22 performs the resolution conversion processing in the horizontal direction and the vertical direction on the first to third buffer memories 91, 92, 93 having a data capacity of N pixels. Image data can be read from the image memory 32. As a result, the circuit scale of the horizontal buffer 72 and the vertical buffer 74 can be reduced and the production cost can be reduced.
[0071]
The NTSC / PAL encoder 23 not only performs the above-described encoding, but also includes a simple resolution conversion circuit 23a that increases the resolution of the image data as necessary before performing the encoding process.
[0072]
The simple resolution conversion circuit 23a performs resolution conversion that matches the display standard of the finder 36 when the image data on the image memory 32 is lower than the resolution necessary for display.
[0073]
Specifically, as shown in FIG. 14, the simple resolution conversion circuit 23a includes a line memory 101 for storing image data from the image data bus 33, and a vertical linear interpolation circuit (for interpolating image data in the vertical direction). Hereinafter, it is referred to as a “V direction linear interpolation circuit”) 102 and a horizontal direction interpolation circuit 103.
[0074]
The line memory 81 stores image data from the input terminal in for one line, and supplies the image data to the vertical linear interpolation circuit 82 in the order in which the image data is stored. The vertical linear interpolation circuit 82 performs vertical linear interpolation by applying predetermined weights to the image data from the input terminal in and the image data from the vertical linear interpolation circuit 82. Next, as horizontal interpolation, Y is interpolated with a seventh-order filter, and Cb and Cr are interpolated with a third-order filter. This is only interpolation that doubles the resolution. Then, the image data is output via the output terminal out.
[0075]
For example, the image data input from the input terminal in is a, the image data read from the line memory 101 is b, the weighting coefficient is g (0 ≦ g ≦ 1), and the V-direction linear interpolation circuit 102 outputs If the image data to be processed is c, the V-direction linear interpolation circuit 102 performs the following calculation.
[0076]
c = g * a + (1-g) * b
Note that the image data output from the output terminal out is encoded as described above.
[0077]
As described above, the digital still camera 1 is configured by so-called two chips of the signal processing unit 20 and the CPU 41 in the signal processing system. Therefore, the substrate area can be reduced and the power consumption can be further reduced as compared with the case where each signal processing circuit has a chip configuration.
[0078]
In addition, since the signal processing unit 20 does not have a chip configuration including a CPU, even when an application related to the CPU 41 is changed, signal processing can be performed correspondingly. In other words, in the case of a chip configuration including a CPU, it is impossible to reconfigure the chip corresponding to a change in the application of the CPU. However, the signal processing unit 20 can perform predetermined signal processing using a CPU having an optimal configuration for each application.
[0079]
The digital still camera 1 having such a configuration is a finder mode for confirming the state and position of the subject before photographing, a recording mode for photographing the confirmed subject image, and a photograph of the photographed subject image. Playback modes, and processing is performed according to each mode.
[0080]
In the finder mode, the user needs to observe the state of the subject displayed on the finder 36 before shooting the subject by pressing a shutter button (not shown). In the finder mode, the CPU 41 controls the memory controller 22 and other circuits as follows. Note that FIG. 3 is mainly used for the description of each mode, and FIG. 15 is appropriately referred to.
[0081]
In the finder mode, the CCD image sensor 11 generates an image signal in which the vertical component is thinned down to 1/3, and the image data digitized via the S / H-A / D circuit 12 is input to the CCD interface 21a. Supply.
[0082]
The CCD interface 21a performs signal processing in synchronization with the clock shown in FIG. Specifically, as shown in FIG. 15B, the CCD interface 21a performs processing to thin out the horizontal component of the image data supplied from the image generation unit 10 to 1/3, and further performs gamma correction and the like. And supply it to the camera DSP 21c. The CCD interface 21a supplies the image data converted to 340 × 256 as a result of 1/3 decimation to the camera DSP 21c.
[0083]
As shown in FIG. 15C, the camera DSP 21c performs a data conversion process on the image data after the thinning process to convert the image data into YCrCb image data. The camera DSP 21c further converts the resolution of the image data in the simple resolution conversion circuit 21d to reduce the resolution of the image data (340 × 256 → 320 × 240), and the converted image data is stored in the memory via the image data bus 33. This is supplied to the controller 22.
[0084]
Here, the simple resolution conversion circuit 21d simply lowers the resolution to the extent necessary for subsequent processing. As a result, even if the image data generated by the CCD image sensor 11 has a high resolution, the transfer band occupied by the image data in the image data bus 33 is reduced, thereby avoiding the traffic on the image data bus 33 and the viewfinder mode. Real-time performance can be maintained.
[0085]
The memory controller 22 writes the image data into the image memory 32, and further reads the image data from the image memory 32 and supplies it to the NTSC / PAL encoder 23 via the image data bus 33 as shown in FIG. To do. At the same time, the memory controller 22 also reads the OSD data stored in the image memory 32 and supplies it to the NTSC / PAL encoder 23 via the image data bus 33 as shown in FIG. FIG. 15F shows a state of transfer on the image data bus 33 that enables the above-described real-time processing.
[0086]
The NTSC / PAL encoder 23 performs resolution conversion processing on the image data supplied from the image data bus 33 in the case of the NTSC system, 320 × 240 → 640 × 240, and in the case of the PAL system, 320 × 240 → 640 × 288. Then, the converted image data is supplied to the NTSC / PAL encoder 23. The NTSC / PAL encoder 23 further converts the image data into the NTSC system or the PAL system, synthesizes the OSD data, and supplies this to the finder 36 shown in FIG. Thereby, the subject image, subtitle information, and the like are displayed on the finder 36 in real time.
[0087]
Note that the NTSC / PAL encoder 23 performs resolution conversion so as to increase the resolution of a small resolution. For example, even when 320 × 200 image data is supplied, the NTSC / PAL encoder 23 is 640 × 240 in the NTSC format. In the case of the PAL system, the image data is converted into 640 × 288 image data and output.
[0088]
As described above, in the finder mode, the digital still camera 1 simply reduces the resolution of the image data generated by the CCD image sensor 11 at the timing shown in FIG. The image data is reduced so that it falls within the bandwidth limit of the image data bus 33, and the resolution is increased at the output stage as much as necessary for display and displayed on the finder 36.
[0089]
As a result, even if the image data has a high resolution, the digital still camera 1 does not perform a large-scale thinning process that requires a relatively long processing time, and suppresses it within the bandwidth limit of the image data bus 33 in real time. An image of the subject can be displayed on the finder 36.
[0090]
The CPU 41 sets circuits (CCD interface 21a, camera DSP 21c, NTSC / PAL encoder 23) that perform processing with priority in advance, and performs signal processing in other circuits in addition to these circuits in a time-sharing manner. In such a case, the processing of each circuit having a high priority may be performed with priority according to the data amount of the image data.
[0091]
In addition, based on the control of the CPU 41, the simple resolution conversion unit 21d performs high-speed data processing with a slight reduction in image quality in order to give priority to real-time processing when the amount of image data is large. You can also Thereby, in the finder mode, even if the amount of image data generated by the image generation unit 10 is large, processing can be performed in more real time.
[0092]
In the case of the digital still camera 1 having an electronic zoom function, the CPU 41 may control each circuit as follows.
[0093]
The memory controller 22 writes the image data supplied via the CCD interface 21a and the camera DSP 21c into the image memory 32, reads out the image data from the image memory 32, and supplies it to the resolution conversion circuit 28. The resolution conversion circuit 28 creates image data obtained by enlarging a part of the image input by the electronic zoom function, and outputs the image data to the image memory 32. This image data is read from the image memory 32 and output to the finder 36 via the NTSC / PAL encoder 23. Thereby, it is possible to generate electronically zoomed image data.
[0094]
As described above, in the finder mode, the CPU 41 gives top priority to the real-time property, and therefore does not cause each circuit to perform a process that takes a relatively long time. However, the CPU 41 may cause the memory controller 22 and other circuits to perform various processes as long as the transfer band of the image data bus 33 allows.
[0095]
For example, the memory controller 22 reads out the image data from the image memory 32 that stores the image data supplied from the CCD interface 21 a and the like, supplies the image data to the NTSC / PAL encoder 23 via the image data bus 33, and You may supply also to the decoder 29. FIG. At this time, the finder 36 displays the subject image in real time, while the JPEG encoder / decoder 29 performs JPEG compression processing of the image data.
[0096]
The JPEG encoder / decoder 29 performs compression / decompression processing of still images, and cannot process high-pixel images in real time. Therefore, the JPEG encoder / decoder 29 performs compression processing by thinning out the number of frames (number of frames or fields) of the image data supplied from the image data bus 33 by a predetermined number, or cuts out a part of the image. The compression process may be performed at a lower resolution. Accordingly, it is possible to continuously capture still images with dropped frames or continuously capture still images with low resolution.
[0097]
The user observes the state of the subject displayed on the viewfinder 36 in the above-described viewfinder mode, and determines that the subject is to be photographed, and then presses a shutter button (not shown).
[0098]
When the shutter button is pressed, the digital still camera 1 shifts to the recording mode. In the recording mode, the CPU 41 controls the memory controller 22 and other circuits as follows in order to record the photographed subject image on the recording device 51 while controlling the finder mode.
[0099]
The CCD image sensor 11 stops the thinning process in synchronization with the pressing of the shutter button, generates an XGA format image signal, and converts the digitized image data via the S / H-A / D circuit 12 to the CCD. This is supplied to the interface 21a.
[0100]
The CCD interface 21a supplies the image data supplied from the S / H-A / D circuit 12 to the memory controller 22 via the image data bus 33 instead of the camera DSP 21c. The memory controller 22 writes the image data into the image memory 32, reads the image data, and supplies it to the camera DSP 21 c via the image data bus 33. The camera DSP 21c converts image data composed of RGB into image data composed of Y, Cb, and Cr.
[0101]
Here, the image data once written in the image memory 32 is supplied to the camera DSP 21c. That is, the camera DSP 21c performs a data conversion process on the image data from the image memory 32, not the image data directly supplied from the CCD interface 21a. Therefore, the camera DSP 21c does not need to perform data conversion processing at high speed, and may execute such processing when the image data bus 33 is free. In other words, in the recording mode, since the camera DSP 21c does not need to process in real time, the image data is subjected to data conversion processing prioritizing the improvement of the image quality over the processing speed, and the converted image data is converted into the image data. The data is supplied to the memory controller 22 via the data bus 33. The memory controller 22 writes this image data into the image memory 32.
[0102]
The memory controller 22 reads the image data from the image memory 32 and supplies it to the JPEG encoder / decoder 29. The JPEG encoder / decoder 29 JPEG-compresses the image data and writes it to the recording device 51 shown in FIG.
[0103]
As described above, when it is not necessary to perform the processing in real time as in the recording mode, the CPU 41 writes the image data once in the image memory 32 and then performs a predetermined process to thereby increase the circuit scale. A high pixel image can be processed by effectively using the transfer band of the image data bus 33 while preventing the increase.
[0104]
In the recording mode, the CPU 41 records the image data in the XGA format as it is in the recording device 51. However, the CPU 41 may record the image data in the recording device 51 after converting the resolution of the image data by the resolution conversion circuit 28. . Specifically, the CPU 41 causes the resolution conversion circuit 28 to convert the resolution of the image data read from the image memory 32 via the memory controller 22 so as to correspond to VGA (1024 × 768 → 640 × 480). The image data may be compressed by the JPEG encoder / decoder 29 and then recorded on the recording device 51.
[0105]
When the user wants to check the photographed image after photographing the subject, he / she presses a reproduction button (not shown) to reproduce the photographed image.
[0106]
When the playback button is pressed, the digital still camera 1 shifts to the playback mode. In the playback mode, the CPU 41 controls each circuit as follows in order to read out the image data of the photographed subject from the recording device 51.
[0107]
When the CPU 41 detects the pressing of the playback button, it reads out the image data from the recording device 51 and stores it in the temporary DRAM 42 and then supplies it to the JPEG encoder / decoder 29 via the CPU bus 34. The JPEG encoder / decoder 29 performs JPEG expansion processing on the image data read from the recording device 51 to obtain image data in the XGA format, and supplies the image data to the memory controller 22 via the image data bus 33.
[0108]
The memory controller 22 writes the image data into the image memory 32, reads the image data from the image memory 32, and supplies the image data to the resolution conversion circuit 28 via the image data bus 33.
[0109]
The resolution conversion circuit 28 performs resolution conversion so that the image data corresponds to the VGA format (1024 × 768 → 640 × 480 in the NTSC system, 1024 × 768 → 640 × 576 in the PAL system), and the image data bus 33 is used. To the memory controller 22. The memory controller 22 writes the image data that has undergone resolution conversion processing into the image memory 32, reads out the image data from the image memory 32, and supplies it to the finder 36 via the NTSC / PAL encoder 23. As a result, an image based on the image data recorded in the recording device 51 is displayed on the finder 36.
[0110]
That is, since the resolution of the image data recorded in the recording device 51 is high, the CPU 41 supplies the image data to the finder 36 after reducing this resolution.
[0111]
Further, the CPU 41 determines the priority order of circuits to be processed with priority for each mode of the finder mode, the recording mode, and the reproduction mode, and when shifting to any mode, the CPU 41 processes each circuit according to the priority order. May be executed. Thereby, the signal processing of image data can be efficiently performed according to the processing content of each mode.
[0112]
In the above-described embodiment, the case where image data equivalent to XGA is processed has been described as an example. However, the present invention is not limited to this, and the present invention is not limited to this. Of course, it can be applied.
[0113]
【The invention's effect】
As described above in detail, according to the image processing apparatus of the present invention, the storage capacity of each buffer memory constituting the delay means is the horizontal line or vertical data of one piece of image data stored in the storage means. The storage capacity is to store image data for N pixels smaller than the amount, and the image data for N pixels in the horizontal direction or the vertical direction from the storage unit according to the size of one screen for each row or column . By repeatedly reading out the number of times, the image data for N pixels in the horizontal direction or the vertical direction are all read out in the vertical direction or the horizontal direction with respect to the entire image data for one sheet, and the signal processing means via the delay means. by entering, it is possible to perform image processing based on image data from the respective buffer memories in the signal processing means. That is, even if the storage capacity of the buffer memory is reduced, signal processing of image data of an arbitrary size can be performed. Therefore, the circuit scale of the delay means can be reduced and the cost can be reduced.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a digital still camera to which the present invention is applied.
FIG. 2 is a block diagram showing a schematic configuration of the digital still camera.
FIG. 3 is a block diagram for explaining a flow of image data in a signal processing unit of the digital still camera.
FIG. 4 is a block diagram showing a configuration of a simple resolution conversion circuit in the input processing circuit of the signal processing unit.
FIG. 5 is a block diagram illustrating a configuration of a resolution conversion circuit of the signal processing unit.
FIG. 6 is a block diagram illustrating a specific configuration of a horizontal buffer, a horizontal conversion processing circuit, a vertical buffer, and a vertical conversion processing circuit of the resolution conversion circuit.
FIG. 7 is a block diagram showing another configuration of the resolution conversion circuit.
FIG. 8 is a block diagram showing a configuration of a vertical buffer of the resolution conversion circuit.
FIG. 9 is a diagram for explaining a method when the memory controller reads image data from the image memory.
FIG. 10 is a diagram for explaining coordinate positions of pixels constituting one screen.
FIG. 11 is a diagram for explaining a method when the memory controller reads image data from the image memory.
FIG. 12 is a block diagram illustrating a configuration when a horizontal buffer of the resolution conversion circuit includes a line buffer.
FIG. 13 is a diagram for explaining a method when the memory controller reads image data from the image memory.
FIG. 14 is a block diagram showing a configuration of a simple resolution conversion circuit in the NTSC / PAL encoder of the signal processing unit.
FIG. 15 is a timing chart for explaining the contents of signal processing of each circuit in the finder mode.
FIG. 16 is a block diagram illustrating a configuration of a conventional digital still camera.
FIG. 17 is a block diagram illustrating a configuration of a vertical buffer in the resolution conversion circuit of the digital still camera.
FIG. 18 is a diagram illustrating the size of image data.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Digital still camera, 20 Signal processing part, 21 Input processing circuit, 22 Memory controller, 23 NTSC / PAL encoder, 28 Resolution conversion circuit, 32 Image memory, 33 Image data bus, 34 CPU bus, 40 Control part, 41 CPU

Claims (3)

Storage means for storing at least one piece of image data;
From the storage unit, the reading of all the image data of N pixels in the horizontal direction by repeatedly reading in the horizontal direction of image data of N pixels in each row in the vertical direction, the entire image data for one sheet of the Reading means for
A buffer memory for storing image data for N pixels is connected in series, and the image data read by the reading means is input to one of the serially connected buffer memories and delayed from each buffer memory. A delay means for outputting data;
Signal processing means for performing signal processing of the image data in the horizontal direction by weighting and synthesizing the image data output from each buffer memory of the delay means ,
Each buffer memory of the delay means has a storage capacity for storing image data for N pixels smaller than the data amount of the horizontal line of one image data stored in the storage means,
The readout means repeatedly reads out the image data for N pixels in the horizontal direction from the storage means by the number of times corresponding to the size of one screen for each row, thereby all the image data for N pixels in the horizontal direction in the vertical direction. An image processing apparatus that reads and inputs the signal to the signal processing means via the delay means . The reading means reads all the image data for N pixels in the horizontal direction in the vertical direction, and reads out the image data for at least two pixels in the horizontal direction while reading the entire image data for one sheet. The image processing apparatus according to claim 1 to be performed .   Storage means for storing at least one piece of image data;
  Reading all the image data for N pixels in the vertical direction in the horizontal direction by repeatedly reading out the image data for N pixels in the vertical direction from the storage means to the entire image data for one sheet. Reading means for
  A buffer memory for storing image data for N pixels is connected in series, and the image data read by the reading means is input to one of the serially connected buffer memories and delayed from each buffer memory. A delay means for outputting data;
  Signal processing means for performing signal processing of the image data in the vertical direction by weighting and synthesizing the image data output from each buffer memory of the delay means,
  Each buffer memory of the delay means has a storage capacity for storing image data for N pixels smaller than the data amount of the vertical line of one piece of image data stored in the storage means,
  The reading means repeatedly reads out the image data for N pixels in the vertical direction from the storage means by the number of times corresponding to the size of one screen for each column, thereby all the image data for N pixels in the vertical direction are all horizontally read. An image processing apparatus that reads and inputs the signal to the signal processing means via the delay means.

Images