JP4158245B2 - Signal processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a signal processing apparatus suitable for use in a still image capturing apparatus.
[0002]
[Prior art]
A digital still camera takes image data obtained by a CCD image sensor into a memory or a recording medium, and then transfers the image data to a so-called personal computer or the like. Until now, most of these digital still cameras are compatible with VGA (Video Graphics Array) systems.
[0003]
For example, as shown in FIG. 16, the digital still camera 200 includes a CCD image sensor 201 that generates an image signal, an input processing / image processing circuit 202, and a memory controller 203 that performs processing for reading or writing image data. An output processing circuit 204 for converting into an output image of a predetermined method, a finder 205 for displaying the state of the subject at the time of shooting, a recording unit 207 for recording compressed image data via the CPU bus 206, A compression / decompression circuit 208 for compressing or decompressing image data, a memory 209 made of, for example, a DRAM or the like for storing image data, and a CPU 210 for controlling the entire apparatus are provided.
[0004]
Prior to the start of photographing the subject, the user needs to check the subject image displayed on the viewfinder 205 (finder mode). At this time, the CCD image sensor 201 supplies an image signal obtained by photoelectric conversion with imaging light from the subject to the input processing / image processing circuit 202. The input processing / image processing circuit 202 digitizes the image signal by performing correlated double sampling processing, for example, further performs predetermined signal processing such as gamma correction, knee processing, camera processing, and the like, and supplies the result to the memory controller 203. . The memory controller 203 supplies the image data from the input processing / image processing circuit 202 to the output processing circuit 204 under the control of the CPU 210. The output processing circuit 204 encodes the image data in, for example, the NTSC (National Television System Committee) system, further converts it to analog, and supplies it to the finder 205. As a result, the subject to be imaged is displayed on the viewfinder 205.
[0005]
On the other hand, when the user presses a shutter button (not shown) to shift to the recording mode, the memory controller 203 writes the image data supplied from the input processing / image processing circuit 202 in the memory 209. The CPU 210 reads out the image data from the memory 209, performs the JPEG (Joint Photographic Experts Group) compression processing, for example, in the compression / expansion circuit 208 and records the image data in the recording unit 207.
[0006]
When the CPU 210 shifts to the reproduction mode by a predetermined operation by the user, the CPU 210 reads the image data from the recording unit 207, performs JPEG expansion processing on the image data in the compression / decompression circuit 208, and then outputs the memory controller 203 and output processing. This is supplied to the finder 205 via the circuit 204. Thus, the photographed image is displayed on the finder 205.
[0007]
[Problems to be solved by the invention]
In recent years, with the rapid technological advancement of CCD image sensors, the resolution of image data is becoming more than 1 million pixels. On the other hand, it is conceivable that the digital still camera 200 configured as described above cannot sufficiently cope with image data exceeding one million pixels.
[0008]
For example, in the finder mode, access to image data that requires real-time property and data access by the CPU 210 with an uncertain data rate are performed on one bus. Therefore, when the data amount of image data increases, real-time property is achieved. It will be difficult to secure. As a result, the CPU bus 206 becomes congested, causing a problem that the processing capability of each circuit is reduced.
[0009]
The present invention has been proposed in view of such a situation, and an object of the present invention is to provide a signal processing apparatus that can efficiently perform signal processing in each circuit by preventing congestion of an image data bus. .
[0010]
[Means for Solving the Problems]
In order to solve the above-described problems, a signal processing device according to the present invention includes an image data bus, a plurality of first signal processing means for accessing the image data bus and performing image data signal processing, and image data And a storage means for storing the image data bus, the image data supplied via the image data bus is written to the storage means, the image data stored in the storage means is read, and the image data is read A signal processing unit having a read / write means for outputting to the bus; a control bus; a plurality of second signal processing means for accessing the control bus to perform signal processing of image data; and image data in the control bus And a control means for controlling each of the second signal processing means and the reading / writing means of the signal processing unit.
[0011]
  In the signal processing device, the read / write means controls the image data bus so that the image data in the image data bus falls within a predetermined transfer band range while following the control of the control means.One of the plurality of second signal processing means that performs data transfer between the first signal processing means connected to the image data bus and data transfer to the control bus. The read / write means includes a recording means for recording image data, and the control means reads the image data stored in the storage means and supplies the read image data to the control bus via the image data bus. The image data read by the reading / writing means is recorded in the recording means, and the image data recorded in the recording means is supplied to the reading / writing means..
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0013]
The present invention is applied to, for example, a digital still camera 1 having the configuration shown in FIG.
The digital still camera 1 controls an image generation unit 10 that generates an image signal, a signal processing unit 20 that performs predetermined signal processing on image data, an image memory 32 that includes an SDRAM, and the signal processing unit 20. And a control unit 40.
[0014]
The image generation unit 10 is a solid-state imaging device that generates an image signal, for example, a CCD image sensor 11, and a sample / hold-analog / digital circuit that performs sample hold processing and digitization processing on the image signal and outputs image data ( (Hereinafter referred to as “S / H-A / D circuit”) 12 and a timing generator 13 for generating a timing signal. The timing generator 13 generates a horizontal synchronization signal and a vertical synchronization signal for controlling each circuit of the image generation unit 10 based on the synchronization signal supplied from the signal processing unit 20.
[0015]
The CCD image sensor 11 generates image data equivalent to, for example, XGA (eXtended Graphics Array: 1024 × 768) composed of 800,000 pixels. The CCD image sensor 11 is driven based on the synchronization signal from the timing generator 13 and outputs an image signal of 30 frames per second. The CCD image sensor 11 has a thinning function of the image signal, and the vertical component of the image signal is thinned out to 1/2, 1/3, 1/4... And output under the control of the control unit 40. be able to.
[0016]
The S / H-A / D circuit 12 also performs sample hold and A / D conversion processing at a predetermined sampling interval based on the synchronization signal from the timing generator 13 and supplies this image data to the signal processing unit 20.
[0017]
The signal processing circuit 20 is configured by a single LSI (Large Scale Integrated circuit). The signal processing unit 20 includes an input processing circuit 21 that performs input processing and camera processing on image data from the image generation unit 10, a memory controller 22 that controls reading / writing of image data to / from the image memory 32, and NTSC / PAL ( Phase Alternation by Line) An encoder 23, a D / A converter 24 that converts image data into analog form and outputs it to the outside, and a sync generator 26 that generates a synchronization signal and supplies it to the timing generator 13.
[0018]
The signal processing unit 20 also includes a memory interface 27 that is an interface of the image memory 32, a resolution conversion circuit 28 that performs resolution conversion processing of image data, and a JPEG (Joint Photographic Experts Group) that performs compression / decompression processing of image data. An encoder / decoder 29, a JPEG interface 30 that is an interface of the JPEG encoder / decoder 29, and a host interface 31 that is an interface for transmitting / receiving data to / from a CPU of the control unit 40 are provided.
[0019]
The input processing circuit 21 performs digital clamping, shading correction, aperture correction, gamma correction, color processing, and the like on the image data from the S / H-A / D circuit 12 and supplies the result to the memory controller 22. The input processing circuit 21 has a function of converting input data into Y, Cb, and Cr by signal processing. When the resolution of the image data is larger than, for example, the VGA (Video Graphics Array) format, the input processing circuit 21 can also perform processing to lower the resolution. The input processing circuit 21 performs auto-focus and auto-iris detection processing, sends the data to the control unit 40, and automatically adjusts the focus mechanism and iris mechanism. Further, the input processing circuit 21 performs auto white balance adjustment by detecting the signal levels of the three primary color signals constituting the image data.
[0020]
The memory controller 22 controls the image data supplied from the input processing circuit 21 and other circuits to be written into the image memory 32 via the memory interface 27 and reads out the image data in the image memory 32 via the memory interface 27. Do. At this time, the memory controller 22 also detects whether there is a defective pixel in the CCD image sensor 11 based on the image data stored in the image memory 32.
[0021]
The memory controller 22 supplies the image data read from the image memory 32 to, for example, the NTSC / PAL encoder 23. When the image data is supplied from the memory controller 22, the NTSC / PAL encoder 23 encodes the image data using the NTSC method or the PAL method and supplies the encoded image data to the D / A converter 24. The D / A converter 24 converts the image data into analog form and outputs it through the external terminal 25.
[0022]
The memory controller 22 supplies the image data read from the memory controller 22 to the resolution conversion circuit 28 to perform resolution conversion processing, and writes the resolution-converted image data output from the resolution conversion circuit 28 into the image memory 32. .
[0023]
The memory controller 22 supplies image data to the JPEG encoder / decoder 29 via the JPEG interface 30 to perform still image compression processing, and further, the image data expanded by the JPEG encoder / decoder 29 is image memory 32. Also write to.
[0024]
The image memory 32 not only stores image data as described above, but also stores OSD (On Screen Display) data which is so-called character generator data. The OSD data referred to here is bitmap data. The memory controller 22 also controls reading / writing of the OSD data. Note that the NTSC / PAL encoder 23 combines the image data and the OSD data.
[0025]
The control unit 40 includes a CPU (Central Processing Unit) 41 that controls each circuit of the signal processing unit 20, a DRAM (Dynamic Random Access Memory) 42 that temporarily stores image data and other control data, and a control program for the CPU 41. Are stored in a ROM (Read Only Memory) 43, a flash memory interface 44 that is an interface for exchanging image data with a recording device 51 that is made of, for example, a flash memory, and a communication circuit 52 that is made of, for example, an IrLED. An IrDA interface 45 is provided.
[0026]
For example, the CPU 41 writes the image data compressed by the JPEG encoder / decoder 29 to the recording device 51 including the flash memory via the flash memory interface 44, and reads the image data from the recording device 51 to read the JPEG encoder. / Supplied to the decoder 29. Further, the CPU 41 outputs the image data from the JPEG encoder / decoder 29 and the image data read from the recording device 51 to the outside as infrared light via the IrDA interface 45 and the communication circuit 52.
[0027]
Here, a simple configuration of the digital still camera 1 is shown in FIG.
[0028]
The input processing circuit 21 supplies the image data from the CCD image sensor 11 to the image memory 32 via the image data bus 33. The NTSC / PAL encoder 23 performs a predetermined encoding process on the image data from the image memory 32 and supplies the image data to the finder 36. As a result, an image of the subject is displayed on the finder 36. The finder 36 displays an image corresponding to the image data up to the VGA format.
[0029]
The memory controller 22 performs data transfer between the signal processing circuits connected to the image memory 32 and the image data bus 33. The resolution conversion circuit 28 performs resolution conversion processing of the image data from the image memory 32 and supplies the result to the image memory 32. The JPEG encoder / decoder 29 compresses the image data from the image memory 32 by the JPEG method and supplies the compressed data to the CPU 41 via the CPU bus 34. The CPU 41 writes the compressed image data into the recording device 51 via the CPU bus 34. The CPU 41 can also output the compressed image data to the outside via the CPU bus 34 and the communication circuit 52.
[0030]
  As shown in FIG.In the signal processing unit 20, each circuit is connected via the image data bus 33.It is connected.
[0031]
In the signal processing unit 20, each other circuit such as the NTSC / PAL encoder 23 and the resolution conversion circuit 28 sends a request signal (request) indicating that image data is requested to the memory controller 22 before processing the image data. Send
. Each of these circuits also transmits a request signal to the memory controller 22 when outputting the image data after the processing of the image data is completed.
[0032]
On the other hand, when the memory controller 22 receives a request signal from each circuit, the memory controller 22 selects a circuit having a higher priority from each circuit, and transmits an acknowledge signal to the selected circuit. Here, the acknowledge signal refers to a signal indicating that image data is supplied to a circuit that receives the signal or that image data output from the circuit that has received the signal is ready to be received. Then, the memory controller 22 reads the image data from the image memory 32 and supplies it to the circuit to which the acknowledge signal is transmitted via the image data bus 33. Further, the memory controller 22 receives the image data output from the circuit to which the acknowledge signal is transmitted and performs a process of writing the image data into the image memory 32.
[0033]
The memory controller 22 can preferentially select a circuit that needs to be processed in real time when receiving a request signal from each circuit simultaneously. For example, the memory controller 22 preferentially selects the input processing circuit 21 and the NTSC / PAL encoder 23 when displaying the subject image on the finder 36. Further, the memory controller 22 may determine the bus occupation usage rate of the image data in the image data bus 33 and determine the priority order of each circuit according to the occupation rate.
[0034]
If the memory controller 22 can supply image data to each circuit within the transfer bandwidth limit of the image data bus 33, the memory controller 22 transmits an acknowledge signal to each circuit in a time division manner, and each circuit has a predetermined value. You may control to perform the process of. As a result, the memory controller 22 effectively accesses the circuits in real time and writes the image data from each circuit to the image memory 32, or reads the image data from the image memory 32 and supplies it to each circuit. can do.
[0035]
Further, even when the memory controller 22 accesses an external circuit (not shown) via the image data bus 33, the external controller can transmit the above-described request signal or receive an acknowledge signal. For example, within the transfer bandwidth limit range of the image data bus 33, it is possible to access simultaneously in a time-sharing manner as with each circuit in the signal processing unit 20. That is, if the memory controller 22 is within the bandwidth of the image data bus 33, the memory controller 22 accesses these circuits simultaneously in a time division manner regardless of the number of circuits in the signal processing unit 20 and the number of external circuits. Can do.
[0036]
As described above, the memory controller 22 performs arbitration of the image data bus 33, control of image data writing / reading between the image memory 32 and each circuit, and data transfer to the CPU bus 34.
[0037]
Next, a specific flow of image data in the signal processing unit 20 will be described with reference to FIG.
[0038]
The input processing circuit 21 performs a CCD interface 21a that performs predetermined signal processing on the image data from the image generation unit 10, a detection circuit 21b that performs detection processing to perform the processing of the CCD interface 21a, and image data conversion processing. And a camera digital signal processor (hereinafter referred to as “camera DSP”) 21c.
[0039]
The CCD interface 21a performs digital clamp and white balance adjustment on the image data composed of the red signal, green signal, and blue signal (R, G, B) from the S / H-A / D circuit 12 shown in FIG. Processing such as gamma correction is performed, and if necessary, horizontal component thinning processing is also performed. After performing such processing, the CCD interface 21 a supplies image data to the camera DSP 21 c or supplies it to the memory controller 22 via the image data bus 33.
[0040]
The detection circuit 21b performs detection processing for autofocus, auto iris, and white balance adjustment from the image data of the CCD interface 21a.
[0041]
The camera DSP 21c converts image data composed of RGB from the CCD interface 21a into image data composed of a luminance signal Y and chroma signals (color difference signals) Cb and Cr. In addition, the camera DSP 21c includes such a simple resolution conversion circuit 21d that performs such processing and simply converts the resolution of the image data.
[0042]
The simple resolution conversion circuit 21d converts the resolution of the image data to be low when the resolution of the image data generated by the CCD image sensor 11 is larger than, for example, the VGA format.
[0043]
Specifically, as shown in FIG. 4, the simple resolution conversion circuit 21d includes a BY / RY separation circuit 61 that separates color difference signals, and a horizontal linear interpolation circuit 62 that performs horizontal interpolation processing. A BY / R-Y combining circuit 63 that combines color difference signals, a 1H delay circuit 64 that gives a delay of one horizontal scanning period (1H period) to each signal, and a vertical linear interpolation circuit 65 are provided.
[0044]
The BY / R-Y separation circuit 61 separates the color difference signals BY and RY, which are chroma signals Cb and Cr, from the image data from the camera DSP 21c, and supplies them to the horizontal linear interpolation circuit 62. The horizontal linear interpolation circuit 62 performs a horizontal interpolation process on the luminance signal Y and the color difference signals BY and RY to reduce the horizontal resolution so that the luminance signal Y and the color difference signal after the interpolation process are performed. BY and RY are supplied to the BY / RY composite circuit 63.
[0045]
The BY / R-Y combining circuit 63 combines the color difference signals BY and RY, and the luminance signal Y from the horizontal linear interpolation circuit 62 and the combined color difference signals BY and RY. The 1H delay circuit 64 and the vertical linear interpolation circuit 65 are supplied. The 1H delay circuit 64 gives a delay of 1H period to the luminance signal Y and the color difference signals BY and RY, respectively, and supplies them to the vertical linear interpolation circuit 65. The vertical linear interpolation circuit 65 performs vertical linear interpolation processing based on the luminance signal Y and the color difference signals BY and RY from the BY / RY synthesis circuit 63 and the 1H delay circuit 64, and As a result, image data composed of the luminance signal Y ′ and the color difference signals (BY) ′ and (R−Y) ′ whose horizontal and vertical resolutions are lowered is output.
[0046]
Further, the resolution conversion circuit 28 performs resolution conversion processing for converting, for example, [p × q] image data into [m × n] image data. The resolution conversion circuit 28 is a process that is performed mainly to suppress the image data generated by the CCD image sensor 11 to a predetermined resolution when the image data is high resolution. It may be processed.
[0047]
Specifically, as shown in FIG. 5, the resolution conversion circuit 28 includes an input buffer 71 for storing image data input from the image data bus 33, and a horizontal buffer for horizontally buffering image data from the input buffer 71. A direction buffer 72, a horizontal direction conversion processing circuit 73 for performing horizontal resolution conversion processing on the image data from the horizontal direction buffer 72, and a vertical direction buffer for buffering image data from the horizontal direction conversion processing circuit 73 in the vertical direction. 74, a vertical direction conversion processing circuit 75 that performs resolution conversion processing in the vertical direction, and an output buffer 76 that performs buffering at the time of output.
[0048]
When the resolution conversion circuit 28 is ready to perform the resolution conversion of the image data, the resolution conversion circuit 28 outputs a (read) request signal that requests the memory controller 22 to read the image data from the image memory 32, and further, After the data conversion process, a request signal (write) requesting the memory controller 22 to write the image data into the image memory 32 is output. Further, the resolution conversion circuit 28 receives an acknowledge signal indicating that the memory controller 22 has responded to the request signal.
[0049]
On the other hand, the horizontal buffer 72 has a configuration in which a first delay circuit 81, a second delay circuit 82, and a third delay circuit 83 that give a delay of one pixel are connected in series as shown in FIG. Is done. Therefore, the first delay circuit 81 outputs image data delayed by one pixel, the second delay circuit 82 outputs image data delayed by two pixels, and the third delay circuit 83 delays by three pixels. Output image data.
[0050]
As shown in FIG. 6, the horizontal direction conversion processing circuit 73 includes first to fourth multipliers 84, 85, 86, and 87 and first to third adders 88, 89, and 90. In some cases, a circuit for normalizing data is added after the adder 90.
[0051]
The first multiplier 84 multiplies the image data supplied from the input buffer 71 by a predetermined coefficient and supplies it to the adder 88. The second multiplier 85 multiplies the image data supplied from the first delay circuit 81 by a predetermined coefficient and supplies the result to the adder 88. The third multiplier 86 multiplies the image data supplied from the second delay circuit 82 by a predetermined coefficient and supplies the result to the adder 89. The fourth multiplier 87 multiplies the image data supplied from the third delay circuit 83 by a predetermined coefficient and supplies the result to the adder 90. The first adder 88 combines the image data and supplies the synthesized image data to the second adder 89. The second adder 89 combines the image data and supplies the synthesized image data to the third adder 90. The third adder 90 synthesizes each image data and supplies it to the vertical buffer 74 as image data that has been subjected to resolution conversion processing in the horizontal direction.
[0052]
As described above, the horizontal direction conversion processing circuit 73 performs a process of compensating for or thinning out the pixels in the horizontal direction by combining the image data having a delay of one pixel with predetermined weighting and combining them. , Convert the horizontal resolution.
[0053]
For example, as shown in FIG. 6, the vertical buffer 74 is configured by connecting first to third buffer memories 91, 92, 93 that give a delay of one line in series. Therefore, the first buffer memory 91 outputs image data delayed by one line, the second buffer memory 92 outputs image data delayed by two lines, and the third buffer memory 93 is delayed by three lines. Output image data.
[0054]
As shown in FIG. 6, the vertical conversion processing circuit 75 includes fifth to eighth multipliers 94, 95, 96, and 97 and fourth to sixth adders 98, 99, and 100. In some cases, a circuit for normalizing data is added after the adder 90.
[0055]
The fifth multiplier 94 multiplies the image data supplied from the horizontal direction conversion circuit 73 by a predetermined coefficient and supplies the result to the fourth adder 98. The sixth multiplier 95 multiplies the image data supplied from the first line memory 91 by a predetermined coefficient and supplies the result to the fourth adder 98. The seventh multiplier 96 multiplies the image data supplied from the second line memory 92 by a predetermined coefficient and supplies the result to the fifth adder 99. The eighth multiplier 97 multiplies the image data supplied from the third line memory 93 by a predetermined coefficient and supplies the result to the sixth adder 100. The fourth adder 98 synthesizes each image data and supplies it to the fifth adder 99. The fifth adder 99 synthesizes each image data and supplies it to the sixth adder 100. The sixth adder 100 synthesizes each image data and outputs it as image data that has been subjected to resolution conversion processing in the vertical direction.
[0056]
As described above, the vertical direction conversion processing circuit 75 performs processing for compensating for or thinning out pixels in the vertical direction by combining image data delayed by one line with predetermined weighting, respectively. Or convert the vertical resolution.
[0057]
In FIG. 6, the resolution conversion circuit 28 performs the resolution conversion process in the vertical direction after performing the resolution conversion process in the horizontal direction. However, as shown in FIG. 7, the resolution conversion circuit 28 performs the resolution conversion process in the vertical direction. Alternatively, horizontal resolution conversion processing may be performed. That is, the resolution conversion circuit 28 supplies the image data from the input buffer 71 to the vertical buffer 74, and in the order of the vertical buffer 74, the vertical conversion processing circuit 75, the horizontal buffer 72, and the horizontal conversion processing circuit 73. You may make it the structure which processes.
[0058]
Further, the first to third buffer memories 91, 92, 93 in the vertical buffer 74 can store image data for one line (1H). However, as shown in FIG. For example, the image data of a smaller number of N pixels (pixel length N) may be stored. At this time, the memory controller 22 needs to read the image data stored in the image memory 32 every N pixels, as shown in FIG.
[0059]
Specifically, the memory controller 22 reads the image data for one screen stored in the image memory 32 in the vertical direction by N pixels for each line. Here, as shown in FIG. 10, one screen is composed of p × q [pixels], the coordinates of the upper left pixel are (1, 1), the coordinates of the upper right pixel are (p, 1), and the lower left pixel is The coordinates are (1, q), and the coordinates of the lower right pixel are (p, q).
[0060]
As shown in FIG. 11, the memory controller 22 first reads image data for N pixels in the horizontal direction for each line in the order of the first row, the second row,. Thereby, the memory controller 22 has N pixels of image data from the left end, that is, a range (N × q pixels) surrounded by (1, 1) (1, q) (N, q) (N, 1). Image data (hereinafter referred to as “image data group (1)”) is read.
[0061]
Next, the memory controller 22 selects image data (hereinafter referred to as “image data group”) in a range surrounded by (N−1,1) (N−1, q) (N−2, q) (N−2,1). (2) "and so on) is read out. Here, when the memory controller 22 reads the column (1) and the image data group (2), the memory controller 22 reads the (N-1) th and Nth column image data twice.
[0062]
This is because the vertical direction conversion processing circuit 75 performs interpolation processing from surrounding pixels, so that the first and third pixels stored in the first to third buffer memories 91, 92, and 93 are processed results. It is because it does not make a target. For example, the pixel (N, 1) is not a target of the vertical interpolation processing result when the image data group (1) is read out. However, this (N, 1) pixel is also read out when the image data group (2) is read out, and becomes an object of the interpolation processing result at this time.
[0063]
Similarly, the memory controller 22 reads out image data for N pixels in the horizontal direction for each line so as to include the image data for the last second column of the immediately preceding image data group, and thereby the image data group Is supplied to the resolution conversion circuit 28.
[0064]
Image data matching the capacity of the first to third buffer memories 91, 92, 93 is supplied to the vertical buffer 74 of the resolution conversion circuit 28 for each line. Therefore, the first to third buffer memories 91, 92, 93 store image data shifted by one line. The vertical direction conversion processing circuit 75 can perform vertical resolution conversion processing based on the image data from the first to third buffer memories 91, 92, 93 of the vertical buffer 74.
[0065]
As described above, the memory controller 22 reads out according to the capacity of the buffer memory even if the capacity of the buffer memory required for the vertical image degree conversion is less than one line, thereby allowing the resolution conversion circuit 28 to read. It is possible to perform resolution conversion in the vertical direction.
[0066]
Here, the read overlap between the image data groups is two columns, but there may be a case where the overlap is larger than two columns or there is no overlap. Further, the present invention is not limited to resolution conversion and is also applied to image signal processing such as camera signal processing.
[0067]
In addition, here, the case where the buffer memory is used for the interpolation process in the vertical direction has been described as an example, but the same applies to the case where the buffer memory is used for the interpolation process in the horizontal direction.
[0068]
That is, for example, as shown in FIG. 12, the resolution conversion circuit 28 may perform horizontal resolution conversion using a horizontal buffer 72a composed of a buffer memory having a capacity of N pixels. As shown in FIG. 13, the memory controller 22 may read image data for N pixels in the vertical direction for each column in the order of the first column, the second column,. As in the case of the vertical interpolation process described above, the memory controller 22 needs to read the image data stored at the beginning and end of the buffer memory twice so as to be the target of the horizontal interpolation process. .
[0069]
As described above, the memory controller 22 performs the resolution conversion processing in the horizontal direction and the vertical direction on the first to third buffer memories 91, 92, 93 having a data capacity of N pixels. Image data can be read from the image memory 32. As a result, the circuit scale of the horizontal buffer 72 and the vertical buffer 74 can be reduced and the production cost can be reduced.
[0070]
The NTSC / PAL encoder 23 not only performs the above-described encoding, but also includes a simple resolution conversion circuit 23a that increases the resolution of the image data as necessary before performing the encoding process.
[0071]
The simple resolution conversion circuit 23a performs resolution conversion that matches the display standard of the finder 36 when the image data on the image memory 32 is lower than the resolution necessary for display.
[0072]
Specifically, as shown in FIG. 14, the simple resolution conversion circuit 23a includes a line memory 101 for storing image data from the image data bus 33, and a vertical linear interpolation circuit (for interpolating image data in the vertical direction). Hereinafter, it is referred to as a “V direction linear interpolation circuit”) 102 and a horizontal direction interpolation circuit 103.
[0073]
The line memory 81 stores image data from the input terminal in for one line, and supplies the image data to the vertical linear interpolation circuit 82 in the order in which the image data is stored. The vertical linear interpolation circuit 82 performs vertical linear interpolation by applying predetermined weights to the image data from the input terminal in and the image data from the vertical linear interpolation circuit 82. Next, as horizontal interpolation, Y is interpolated with a seventh-order filter, and Cb and Cr are interpolated with a third-order filter. This is only interpolation that doubles the resolution. Then, the image data is output via the output terminal out.
[0074]
For example, the image data input from the input terminal in is a, the image data read from the line memory 101 is b, the weighting coefficient is g (0 ≦ g ≦ 1), and the V-direction linear interpolation circuit 102 outputs If the image data to be processed is c, the V-direction linear interpolation circuit 102 performs the following calculation.
[0075]
c = g * a + (1-g) * b
Note that the image data output from the output terminal out is encoded as described above.
[0076]
As described above, the digital still camera 1 is configured by so-called two chips of the signal processing unit 20 and the CPU 41 in the signal processing system. Therefore, the substrate area can be reduced and the power consumption can be further reduced as compared with the case where each signal processing circuit has a chip configuration.
[0077]
In addition, since the signal processing unit 20 does not have a chip configuration including a CPU, even when an application related to the CPU 41 is changed, signal processing can be performed correspondingly. In other words, in the case of a chip configuration including a CPU, it is impossible to reconfigure the chip corresponding to a change in the application of the CPU. However, the signal processing unit 20 can perform predetermined signal processing using a CPU having an optimal configuration for each application.
[0078]
The digital still camera 1 having such a configuration is a finder mode for confirming the state and position of the subject before photographing, a recording mode for photographing the confirmed subject image, and a photograph of the photographed subject image. Playback modes, and processing is performed according to each mode.
[0079]
In the finder mode, the user needs to observe the state of the subject displayed on the finder 36 before shooting the subject by pressing a shutter button (not shown). In the finder mode, the CPU 41 controls the memory controller 22 and other circuits as follows. Note that FIG. 3 is mainly used for the description of each mode, and FIG. 15 is appropriately referred to.
[0080]
In the finder mode, the CCD image sensor 11 generates an image signal in which the vertical component is thinned down to 1/3, and the image data digitized via the S / H-A / D circuit 12 is input to the CCD interface 21a. Supply.
[0081]
The CCD interface 21a performs signal processing in synchronization with the clock shown in FIG. Specifically, as shown in FIG. 15B, the CCD interface 21a performs processing to thin out the horizontal component of the image data supplied from the image generation unit 10 to 1/3, and further performs gamma correction and the like. And supply it to the camera DSP 21c. The CCD interface 21a supplies the image data converted to 340 × 256 as a result of 1/3 decimation to the camera DSP 21c.
[0082]
As shown in FIG. 15C, the camera DSP 21c performs a data conversion process on the image data after the thinning process to convert the image data into YCrCb image data. The camera DSP 21c further converts the resolution of the image data in the simple resolution conversion circuit 21d to reduce the resolution of the image data (340 × 256 → 320 × 240), and the converted image data is stored in the memory via the image data bus 33. This is supplied to the controller 22.
[0083]
Here, the simple resolution conversion circuit 21d simply lowers the resolution to the extent necessary for subsequent processing. As a result, even if the image data generated by the CCD image sensor 11 has a high resolution, the transfer band occupied by the image data in the image data bus 33 is reduced, thereby avoiding the traffic on the image data bus 33 and the viewfinder mode. Real-time performance can be maintained.
[0084]
The memory controller 22 writes the image data into the image memory 32, and further reads the image data from the image memory 32 and supplies it to the NTSC / PAL encoder 23 via the image data bus 33 as shown in FIG. To do. At the same time, the memory controller 22 also reads the OSD data stored in the image memory 32 and supplies it to the NTSC / PAL encoder 23 via the image data bus 33 as shown in FIG. FIG. 15F shows a state of transfer on the image data bus 33 that enables the above-described real-time processing.
[0085]
The NTSC / PAL encoder 23 performs resolution conversion processing on the image data supplied from the image data bus 33 in the case of the NTSC system, 320 × 240 → 640 × 240, and in the case of the PAL system, 320 × 240 → 640 × 288. Then, the converted image data is supplied to the NTSC / PAL encoder 23. The NTSC / PAL encoder 23 further converts the image data into the NTSC system or the PAL system, synthesizes the OSD data, and supplies this to the finder 36 shown in FIG. Thereby, the subject image, subtitle information, and the like are displayed on the finder 36 in real time.
[0086]
Note that the NTSC / PAL encoder 23 performs resolution conversion so as to increase the resolution of a small resolution. For example, even when 320 × 200 image data is supplied, the NTSC / PAL encoder 23 is 640 × 240 in the NTSC format. In the case of the PAL system, the image data is converted into 640 × 288 image data and output.
[0087]
As described above, in the finder mode, the digital still camera 1 simply reduces the resolution of the image data generated by the CCD image sensor 11 at the timing shown in FIG. The image data is within the bandwidth limit of the image data bus 33 and is displayed on the finder 36 with a higher resolution at the output stage as necessary for display.
[0088]
As a result, even if the image data has a high resolution, the digital still camera 1 does not perform a large-scale thinning process that requires a relatively long processing time, and suppresses it within the bandwidth limit of the image data bus 33 in real time. An image of the subject can be displayed on the finder 36.
[0089]
The CPU 41 sets circuits (CCD interface 21a, camera DSP 21c, NTSC / PAL encoder 23) that perform processing with priority in advance, and performs signal processing in other circuits in addition to these circuits in a time-sharing manner. In such a case, the processing of each circuit having a high priority may be performed with priority according to the data amount of the image data.
[0090]
In addition, based on the control of the CPU 41, the simple resolution conversion unit 21d performs high-speed data processing with a slight reduction in image quality in order to give priority to real-time processing when the amount of image data is large. You can also. Thereby, in the finder mode, even if the amount of image data generated by the image generation unit 10 is large, processing can be performed in more real time.
[0091]
In the case of the digital still camera 1 having an electronic zoom function, the CPU 41 may control each circuit as follows.
[0092]
The memory controller 22 writes the image data supplied via the CCD interface 21a and the camera DSP 21c into the image memory 32, reads out the image data from the image memory 32, and supplies it to the resolution conversion circuit 28. The resolution conversion circuit 28 creates image data obtained by enlarging a part of the image input by the electronic zoom function, and outputs the image data to the image memory 32. This image data is read from the image memory 32 and output to the finder 36 via the NTSC / PAL encoder 23. Thereby, it is possible to generate electronically zoomed image data.
[0093]
As described above, in the finder mode, the CPU 41 gives top priority to the real-time property, and therefore does not cause each circuit to perform a process that takes a relatively long time. However, the CPU 41 may cause the memory controller 22 and other circuits to perform various processes as long as the transfer band of the image data bus 33 allows.
[0094]
For example, the memory controller 22 reads out the image data from the image memory 32 that stores the image data supplied from the CCD interface 21 a and the like, supplies the image data to the NTSC / PAL encoder 23 via the image data bus 33, and You may supply also to the decoder 29. FIG. At this time, the finder 36 displays the subject image in real time, while the JPEG encoder / decoder 29 performs JPEG compression processing of the image data.
[0095]
The JPEG encoder / decoder 29 performs compression / decompression processing of still images, and cannot process high-pixel images in real time. Therefore, the JPEG encoder / decoder 29 performs compression processing by thinning out the number of frames (number of frames or fields) of the image data supplied from the image data bus 33 by a predetermined number, or cuts out a part of the image. The compression process may be performed at a lower resolution. Accordingly, it is possible to continuously capture still images with dropped frames or continuously capture still images with low resolution.
[0096]
The user observes the state of the subject displayed on the viewfinder 36 in the above-described viewfinder mode, and determines that the subject is to be photographed, and then presses a shutter button (not shown).
[0097]
When the shutter button is pressed, the digital still camera 1 shifts to the recording mode. In the recording mode, the CPU 41 controls the memory controller 22 and other circuits as follows in order to record the photographed subject image on the recording device 51 while controlling the finder mode.
[0098]
The CCD image sensor 11 stops the thinning process in synchronization with the pressing of the shutter button, generates an XGA format image signal, and converts the digitized image data via the S / H-A / D circuit 12 to the CCD. This is supplied to the interface 21a.
[0099]
The CCD interface 21a supplies the image data supplied from the S / H-A / D circuit 12 to the memory controller 22 via the image data bus 33 instead of the camera DSP 21c. The memory controller 22 writes the image data into the image memory 32, reads the image data, and supplies it to the camera DSP 21 c via the image data bus 33. The camera DSP 21c converts image data composed of RGB into image data composed of Y, Cb, and Cr.
[0100]
Here, the image data once written in the image memory 32 is supplied to the camera DSP 21c. That is, the camera DSP 21c performs a data conversion process on the image data from the image memory 32, not the image data directly supplied from the CCD interface 21a. Therefore, the camera DSP 21c does not need to perform data conversion processing at high speed, and may execute such processing when the image data bus 33 is free. In other words, in the recording mode, since the camera DSP 21c does not need to process in real time, the image data is subjected to data conversion processing prioritizing the improvement of the image quality over the processing speed, and the converted image data is converted into the image data. The data is supplied to the memory controller 22 via the data bus 33. The memory controller 22 writes this image data into the image memory 32.
[0101]
The memory controller 22 reads the image data from the image memory 32 and supplies it to the JPEG encoder / decoder 29. The JPEG encoder / decoder 29 JPEG-compresses the image data and writes it to the recording device 51 shown in FIG.
[0102]
As described above, when it is not necessary to perform the processing in real time as in the recording mode, the CPU 41 writes the image data once in the image memory 32 and then performs a predetermined process to thereby increase the circuit scale. A high pixel image can be processed by effectively using the transfer band of the image data bus 33 while preventing the increase.
[0103]
In the recording mode, the CPU 41 records the image data in the XGA format as it is in the recording device 51. However, the CPU 41 may record the image data in the recording device 51 after converting the resolution of the image data by the resolution conversion circuit 28. . Specifically, the CPU 41 causes the resolution conversion circuit 28 to convert the resolution of the image data read from the image memory 32 via the memory controller 22 so as to correspond to VGA (1024 × 768 → 640 × 480). The image data may be compressed by the JPEG encoder / decoder 29 and then recorded on the recording device 51.
[0104]
When the user wants to check the photographed image after photographing the subject, he / she presses a reproduction button (not shown) to reproduce the photographed image.
[0105]
When the playback button is pressed, the digital still camera 1 shifts to the playback mode. In the playback mode, the CPU 41 controls each circuit as follows in order to read out the image data of the photographed subject from the recording device 51.
[0106]
When the CPU 41 detects the pressing of the playback button, it reads out the image data from the recording device 51 and stores it in the temporary DRAM 42 and then supplies it to the JPEG encoder / decoder 29 via the CPU bus 34. The JPEG encoder / decoder 29 performs JPEG expansion processing on the image data read from the recording device 51 to obtain image data in the XGA format, and supplies the image data to the memory controller 22 via the image data bus 33.
[0107]
The memory controller 22 writes the image data into the image memory 32, reads the image data from the image memory 32, and supplies the image data to the resolution conversion circuit 28 via the image data bus 33.
[0108]
The resolution conversion circuit 28 performs resolution conversion so that the image data corresponds to the VGA format (1024 × 768 → 640 × 480 in the NTSC system, 1024 × 768 → 640 × 576 in the PAL system), and the image data bus 33 is used. To the memory controller 22. The memory controller 22 writes the image data that has undergone resolution conversion processing into the image memory 32, reads out the image data from the image memory 32, and supplies it to the finder 36 via the NTSC / PAL encoder 23. As a result, an image based on the image data recorded in the recording device 51 is displayed on the finder 36.
[0109]
That is, since the resolution of the image data recorded in the recording device 51 is high, the CPU 41 supplies the image data to the finder 36 after reducing this resolution.
[0110]
Further, the CPU 41 determines the priority order of circuits to be processed with priority for each mode of the finder mode, the recording mode, and the reproduction mode, and when shifting to any mode, the CPU 41 processes each circuit according to the priority order. May be executed. Thereby, the signal processing of image data can be efficiently performed according to the processing content of each mode.
[0111]
In the above-described embodiment, the case where image data equivalent to XGA is processed has been described as an example. However, the present invention is not limited to this, and the present invention is not limited to this. Of course, it can be applied.
[0112]
【The invention's effect】
As described above in detail, according to the signal processing apparatus of the present invention, the control unit manages the image data in the control bus, and the second signal processing unit and the reading / writing unit of the signal processing unit are provided. By controlling the image data bus so that the image data in the image data bus falls within a predetermined transfer band range while the read / write unit follows the control of the control unit, the congestion of the image data bus is prevented. Signal processing of each signal processing means can be performed efficiently.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a digital still camera to which the present invention is applied.
FIG. 2 is a block diagram showing a schematic configuration of the digital still camera.
FIG. 3 is a block diagram for explaining a flow of image data in a signal processing unit of the digital still camera.
FIG. 4 is a block diagram showing a configuration of a simple resolution conversion circuit in the input processing circuit of the signal processing unit.
FIG. 5 is a block diagram illustrating a configuration of a resolution conversion circuit of the signal processing unit.
FIG. 6 is a block diagram illustrating a specific configuration of a horizontal buffer, a horizontal conversion processing circuit, a vertical buffer, and a vertical conversion processing circuit of the resolution conversion circuit.
FIG. 7 is a block diagram showing another configuration of the resolution conversion circuit.
FIG. 8 is a block diagram showing a configuration of a vertical buffer of the resolution conversion circuit.
FIG. 9 is a diagram for explaining a method when the memory controller reads image data from the image memory.
FIG. 10 is a diagram for explaining coordinate positions of pixels constituting one screen.
FIG. 11 is a diagram for explaining a method when the memory controller reads image data from the image memory.
FIG. 12 is a block diagram illustrating a configuration when a horizontal buffer of the resolution conversion circuit includes a line buffer.
FIG. 13 is a diagram for explaining a method when the memory controller reads image data from the image memory.
FIG. 14 is a block diagram showing a configuration of a simple resolution conversion circuit in the NTSC / PAL encoder of the signal processing unit.
FIG. 15 is a timing chart for explaining the contents of signal processing of each circuit in the finder mode.
FIG. 16 is a block diagram illustrating a configuration of a conventional digital still camera.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Digital still camera, 20 Signal processing part, 21 Input processing circuit, 22 Memory controller, 23 NTSC / PAL encoder, 28 Resolution conversion circuit, 32 Image memory, 33 Image data bus, 34 CPU bus, 40 Control part, 41 CPU

Claims (4)

An image data bus, a plurality of first signal processing means for accessing the image data bus to perform image data signal processing, a storage means for storing image data, the image data bus are managed, and the image data A signal processing unit having a read / write unit that writes image data supplied via a bus to the storage unit, reads the image data stored in the storage unit, and outputs the image data to the image data bus;
A control bus;
A plurality of second signal processing means for accessing the control bus and performing signal processing of image data;
Control means for managing the image data in the control bus, and controlling the second signal processing means and the read / write means of the signal processing unit,
The read / write means manages the image data bus so that the image data in the image data bus falls within a predetermined transfer band range, and controls each of the image data buses connected to the image data bus while following the control of the control means. Data transfer between the first signal processing means and data transfer to the control bus,
Any one of the plurality of second signal processing means includes a recording means for recording image data,
The control means controls the read / write means so as to read out the image data stored in the storage means and supply it to the control bus via the image data bus, and the read / write means reads the image data. A signal processing apparatus for controlling the recording of the image data in the recording means and supplying the image data recorded in the recording means to the reading / writing means . Any one of the plurality of second signal processing means includes compression means for compressing image data,
It said control means, the signal processing apparatus according to claim 1, wherein the image data read by the read / write means is compressed in the compression unit and performs control for supplying to said recording means. Any one of the plurality of second signal processing means includes decompression means for decompressing the compressed image data,
It said control means, the signal processing apparatus according to claim 1, wherein the reading the image data recorded on said recording means performs control to stretch to the image data to the decompression means. The signal processing apparatus according to claim 1, wherein the signal processing unit includes one integrated circuit.

Images