KR100687454B1 - Imaging apparatus and recording/reproducing apparatus - Google Patents

Abstract

In an imaging device, an image is displayed in real time on a finder. In the finder mode, the CCD interface 21a thins the horizontal components of the image data supplied from the image generating unit 10 by 1/3, and also performs gamma correction or the like to display the resulting image data. It transfers to the camera DSP 21c. The camera DSP 21c performs data conversion and resolution conversion on the image data obtained as a result of thinning, and converts the data into Y, Cb, and Cr image data transmitted to the memory controller 22. The memory controller 22 writes the image data to the image memory 32 and reads data from the memory controller 22, and transfers the read data to the NTSC / PAL encoder 23 via the image data bus 33. . The NTSC / PAL encoder 23 converts the image data into resolution, further converts the image data into data of an NTSC or PAL system, and transfers the resulting data to the finder.

Imaging Device, Finder Mode, Memory Controller, Data Conversion, Resolution Conversion

Description

Imaging apparatus and recording / reproducing apparatus

1 is a block diagram showing the structure of a conventional digital still camera.

2 is a block diagram showing the schematic structure of a digital still camera embodying the present invention;

FIG. 3 is a block diagram showing the schematic structure of the digital still camera shown in FIG.

4 is a block diagram illustrating the flow of image data in the signal processing unit of the digital still camera shown in FIG. 2;

FIG. 5 is a block diagram showing the structure of a simplified resolution conversion circuit in the input processing circuit of the signal processing unit. FIG.

6 is a block diagram showing the structure of a resolution converting circuit of the signal processing unit;

Fig. 7 is a block diagram showing a specific structure of a horizontal buffer, a horizontal converting processing circuit, a vertical buffer, and a vertical converting processing circuit of a resolution converting circuit.

8 is a block diagram showing an alternative structure of a resolution converting circuit.

9 is a block diagram showing a structure of a vertical buffer of a resolution converting circuit.

10 illustrates a technique for reading image data from an image memory by a memory controller.

11 illustrates coordinate positions of pixels constituting an image.

12 illustrates another technique for reading image data from an image memory by the memory controller.

Fig. 13 is a block diagram showing the structure of a horizontal buffer of a resolution converting circuit composed of a line buffer.

14 illustrates a technique when the memory controller reads image data from the image memory.

Fig. 15 is a block diagram showing the structure of a simplified resolution conversion circuit in the NTSC / PAL encoder of the signal processing unit.

16A-16F are timing diagrams illustrating signal processing contents of respective circuits in a finder mode.

Explanation of symbols on the main parts of the drawings

1 ... digital still camera 10 ... image generating unit

11 ... CCD image sensor

12.Sample Retention-Analog / Digital Circuit

13 ... Timing generator 20 ... Input signal processor

32.Image memory 40 ... Controller

The present invention relates to an imaging apparatus having a function of displaying an image of an object on a finder in real time.

The digital still camera recovers image data obtained by the CCD image sensor to DRAM or flash memory, and then transfers the image data to a personal computer or the like. The main part of this kind of digital still camera has been the type responsible for video graphics array (VGA) systems.

Referring to Fig. 1, this digital still camera 200 includes a CCD image sensor 201 for generating image signals, an input processing / image processing circuit 202, a memory controller 203 for recording and reading image data, and a preliminary description. An output processing circuit 204 for converting the output image of the set system, a finder 205 for displaying the state of the subject during image capturing, and a recording unit 207 for recording the compressed image data over the CPU bus 206. And a compression / extension circuit 208 that compresses / expands the image data. The digital still camera 200 also includes a memory 209 formed, for example, of DRAM, and a CPU 210 that controls the overall device.

Before starting to capture the image of the subject, the user should check the subject image displayed in the finder 205. This state is called finder mode. At this time, the CCD image sensor 201 transfers the image signal obtained by photoelectric conversion to the input processing / image processing circuit 202. The input processing / image processing circuit 202 performs dual sampling processing correlated to the image signals to digitize the image signal. The input processing / image processing circuit 202 then performs preset signal processing such as gamma correction, knee processing, or camera processing, and passes the processed image signals to the memory controller 203 to send the CPU. In response to control by 210, image data is transferred from input processing / image processing circuit 202 to output processing circuit 204. The output processing circuit 204 encodes image data according to, for example, a National Television System Committee (NTSC) system, analogizes the encoded image data, and transfers the resulting analog data to the finder 205. This allows the subject to appear in the finder 205 as the image capturing subject.

On the other hand, when the user presses a shutter button (not shown) to shift to the recording mode, the memory controller 203 causes the image data supplied from the input processing / image processing circuit 202 to be recorded in the memory 209. The CPU 210 causes the image data to be read from the memory 209 and from the recording unit 207 according to, for example, a Joint Photographic Experts Group (JPEG) system for recording the compressed image data in the recording unit 207. Is compressed by the compression / expansion circuit 208.

When the user performs the preset processing and shifts to the reproduction mode, the CPU 210 causes the image data to be read from the recording unit 207 to cause the image data to be expanded from the compression / expansion circuit 208 to the JPEG system and the resulting Data is transferred to the finder 205 through the memory controller 20 and the output processing circuit 204. The captured image is displayed in the finder 205.

In connection with recent outstanding technological advances in CCD image sensors, the resolution of image data exceeds nearly 1,000,000 pixels. On the other hand, there is a concern that the digital still camera of the above-described structure cannot sufficiently handle image data exceeding 1,000,000 pixels.

For example, when the CCD image sensor 201 outputs a high resolution image signal in the finder mode, the input processing / image processing circuit 202 performs resolution conversion of the image data in real time corresponding to the resolution of the finder 205. Should be. At the same time, memory controller 203 accesses memory 209. In addition, the output processing circuit 204 must execute a predetermined process.

Since the results are stagnant on the CPU bus 106, each circuit cannot execute the preset processing in real time and the image of the subject is displayed in the finder 205 in a frame-decimated fashion. In this case, when the subject moves, a deviation occurs between the movement of the actual subject and the subject displayed on the finder 105, which makes the imaging operation inconvenient.

On the other hand, if the image data is not high resolution, the resolution of the image data should be converted to, for example, the resolution of the NTSC system or the PAL system in consideration of the system of the finder 205. In this case, it is similarly required to display an image in the finder in real time.

Accordingly, it is an object of the present invention to provide an imaging device in which an image can be displayed in the finder in real time even if the image data of the image is high resolution.

In one aspect of the invention, the invention provides an imaging means for generating image data corresponding to imaging light from a subject, and a first resolution converting means for lowering the resolution of the image data from the imaging means by resolution conversion. Second resolution converting means for raising the resolution of the image data supplied from the first resolution converting means through the image data bus by resolution converting, and output means for outputting the image data from the second resolution converting means to the display means. It provides an imaging device comprising.

In another aspect of the present invention, the present invention provides an imaging means for generating image data corresponding to imaging light from a subject, recording / reproducing means for recording image data on a recording medium and reproducing image data recorded on the recording medium. First resolution converting means for performing resolution conversion for lowering the resolution of the image data from the imaging means, and second resolution for performing resolution conversion for raising the resolution of the image data supplied from the first resolution converting means via the image data bus. A display for displaying an image corresponding to the image data from the conversion means, the storage means for storing the image data, the third resolution conversion means for performing resolution conversion of the image data supplied from the storage means, and the second or third resolution conversion means. Playback / including means It provides a locking device.

According to the imaging apparatus of the present invention, in performing resolution conversion, the first resolution converting means lowers the resolution of the image data from the imaging means, and the second resolution converting means is supplied from the first resolution converting means via the image data bus. By increasing the resolution of the image data, the occupancy rate of the image data on the image data bus is reduced, and the subject image can be displayed in real time on the display means.

Referring to the drawings, preferred embodiments of the present invention will be described in detail.

The invention is applied to a digital still camera 1, for example, which is constructed as shown in FIG.

The digital still camera 1 includes an image generating unit 10 for generating an image signal, an input signal processor 20 for processing image data in a preset form, an image memory 32 composed of SDRAM, and an input signal processor 20. The controller 40 controls the control panel 40).

The image generating unit 10 is a solid-state imaging device that generates an image signal, such as a CCD image sensor 11, a sample holding-analog / digital circuit (S / HA / D) that holds a sample and digitizes the image signals into output image data. Circuit 12, and a timing generator 13 for generating timing signals. This timing generator 13 generates horizontal synchronization signals and vertical synchronization signals to control respective circuits of the image generating unit 10 based on the synchronization signals supplied from the signal processor input.

The CCD image sensor 11 generates image data corresponding to, for example, extended graphic array (XGA) 1024 x 768 (XGA) pixel data composed of 800,000 pixels. The CCD image sensor 11 is driven based on the synchronization signal from the timing generator 13 to output image signals at a rate of 30 frames per second. In the meantime, the CCD image sensor 11 has the function of thinning the image signals and thins the vertical components of the image signals to 1/2, 1/3, 1/4, ... and outputs the resulting signals. can do.

The S / HA / D circuit 12 also executes sample retention and A / D conversion at preset sampling intervals based on the synchronization signals from the timing generator 13 to transfer the resulting image data to the signal processor 20. Adapted to convey.

The signal processor 20 includes a single large scale integrated circuit (LSI). The signal processor 20 is an input signal processor 21 that performs input processing and camera processing on image data from the image generating unit 10, and a memory controller that controls reading / writing of image data to the image memory 32. (22), NTSC / PAL (phase alternation by line) encoder 23, D / A converter 24 for analogizing image data and outputting the resulting analog signals externally, and generating synchronization signals and synchronizing the results And a sync generator 26 for supplying a signal to the timing generator 13.

The signal processor 20 also includes a memory interface 27 as an interface for the image memory 32, a resolution conversion circuit 28 for converting the resolution of the image data, and a Joint Photographic Experts Group (JPEG) for compressing / extending the image data. An encoder / decoder 29, a JPEG interface 30 as an interface of the JPEG encoder / decoder 29, and a host interface 31 as an interface having a data transmission / reception relationship with the CPU of the controller 40 are included.

The input signal processor 21 is a digital clamp, shading correction, aperture correction, gamma correction, or color processing from the S / HA / D circuit 12. Image data is processed and the processed result signals are passed to the memory controller 22. The input signal processor 21 has a function of processing the input data to convert the input data into Y, Cb, and Cr. If the resolution of the image data is larger than the video graphics array (VGA), the input signal processor 21 can execute a process of lowering the resolution. The input signal processor 21 also transmits data to the controller 40 to auto-focussing and auto-iris detection to achieve automatic adjustment of the focusing mechanism and the iris mechanism. Run iris detection. The input signal processor 21 also detects signal levels of the three primary colors that make up the image data and adjusts the automatic white balance.

The memory controller 22 also allows image data supplied from the input signal processor 21 or other circuitry to be written to the image memory 32 via the memory interface 27 and the image memory 32 via the memory interface 27. Control to read the image data of the control unit is executed. At this time, the memory controller 22 detects whether there are defective pixels in the CCD image sensor 11 based on the image data stored in the image memory 32.

The memory controller 22 transfers the image data read from the image memory 32 to, for example, the NTSC / PAL encoder 23. When the image data is supplied from the memory controller 22, the NTSC / PAL encoder 23 encodes the image data according to the NTSC system or the PAL system and transfers the encoded data to the D / A converter 24. The D / A converter 24 analogizes the image data and outputs the resulting analog signals through the output terminal 25.

The memory controller 22 transfers the image data read from the memory controller 22 to the resolution converting circuit 28 to cause the image data to be converted in resolution, while the image data output by the resolution converting circuit 28 is stored in the image memory. To be recorded at (32).

The memory controller 22 transfers the image data to the JPEG encoder / decoder 29 via the JPEG interface 30 to perform compression of still images, while the image data expanded by the JPEG encoder / decoder 29 is stored in the image memory. To be recorded at (32).

The image memory 32 stores not only image data as described above, but also OSD data (on-screen-display data) as so-called character generator data. The OSD data consists of bit map data. The controller 22 controls the reading / writing of the OSD data. Image data and OSD data are synthesized by the NTSC / PAL encoder 23.

The controller 40 may include a central processing unit (CPU) 41, a dynamic random access memory (DRAM) 42, and a ROM in which a control program for the CPU 41 is stored, which controls respective circuits of the signal processor 20. read-only memory 43, a flash memory interface 44 that is an interface for exchanging image data with a storage device 51 such as a flash memory, and an IrDA interface 45 that is an interface of a communication circuit 52 composed of IrLEDs. It includes.

For example, the CPU 41 allows the image data compressed by the JPEG encoder / decoder 29 to be written to a storage device 51 composed of flash memory via the flash memory / interface 44, while the image data is stored. It is read from the device 51 and conveys the image data read from the JPEG encoder / decoder 29. The CPU 41 also causes the image data read out from the storage device 51 to be output to the outside as an infrared ray through the IrDA interface 45 and the communication circuit 52.

The schematic structure of the digital still camera 1 is shown in FIG.

The input signal processor 21 transfers the image data from the CCD image sensor 11 to the image memory 32 via the image data bus 33. The NTSC / PAL encoder 23 encodes the image data from the image memory 32 in a preset form and transfers the encoded result data to the finder 36. This causes the image of the subject to be displayed in a finder 36 adapted to display the image associated with the image data in VGA format.

The memory controller 22 performs data transfer between the image memory 32 and the signal processing circuits connected to the image data bus 33. The resolution conversion circuit 28 performs resolution conversion of the image data from the image memory 32 and transfers the results to the image memory 32. The JPEG encoder / decoder 29 compresses the image data from the image memory 32 according to the JPEG system, transfers the compressed image data to the CPU 41 via the CPU bus 34, and the compressed image data Write to storage 51. The CPU 41 can also output the compressed image data to the outside via the CPU bus 34 and the communication circuit 52.

Thus, in FIG. 3, respective circuits of the signal processor 20 are interconnected via an image data bus 33. The image data bus 33 is a virtual bus and indicates that there is a limitation in the transmission band for the image data exchanged between the respective circuits.

In signal processor 20, respective circuits, such as NTSC / PAL encoder 23 or resolution conversion circuit 28, transmit a request signal to memory controller 22 indicating that image data is required. These circuits also send a request signal to the memory controller 22 when outputting the image data after processing the image data.

Upon receiving the request signals from each of the circuits, the memory controller 22 selects the circuits with the highest priority and sends an acknowledgment signal to the selected circuit. The acknowledgment signal indicates that the image data can be delivered to a circuit that receives the signal or that the image data output by the circuit in which the acknowledgment signal was received is ready to be received. The memory controller 22 reads the image data from the image memory 32 and transfers the read image data through the image data bus 33 to the circuit corresponding to the destination of the acknowledgment signal. The memory controller 22 receives the image data output by the circuit which transmitted the acknowledgment signal and writes the image data to the image memory 32.

Upon receiving request signals from the plurality of circuits, the memory controller 22 may preferentially select a circuit that should perform processing in real time. For example, if an image of a subject is to be displayed in the finder 36, the memory controller 22 preferentially selects the input signal processor 21 and the NTSC / PAL encoder 23. It is also possible to decode the bus occupancy rate of the image data on the image data bus 33 so that the memory controller 22 prioritizes the respective circuits depending on the occupancy rate.

If image data can be delivered to the respective circuits within the transmission band limitation of the image data bus 33, then an acknowledgment signal is allowed to allow the memory controller 22 to perform a predetermined process in the respective circuits. It is possible to perform control that transfers time divisionally to the respective circuits. This allows the memory controller 22 to access the data in respective circuits in real time, such that the image data from each circuit is written to the image memory 32 or the image data in the image memory 32 is respectively. To be read into the circuits of the circuit.

When the memory controller 22 accesses an external circuit not shown through the image data bus 33, if the external circuit can transmit the above-described request signal or receive the transmitted acknowledgment signal, the memory controller 22 Each circuit within signal processor 20 can be simultaneously time-divisionally accessed within the transmission band limitation of the image data bus 33. That is, if within the band range of the image data bus 33, the memory controller 22 divides the circuits or signal processors in the signal processor 20 in a time-division manner regardless of the number of circuits or external circuits in the signal processor 20. Concurrent access to external circuits within 20).

As described above, the memory controller 22 controls the arbitration of the image data bus 33, the write / read control of the image data between the image memory 3s2 and the respective circuits, and the CPU bus 34. Perform data transfer to

The specific flow of image data in the signal processor 20 is described with reference to FIG. 4.

The input signal processor 21 converts the CCD interface 21a for executing signal processing preset to the image data from the image generating unit 10, the detection circuit 21b for processing the CCD interface 21a, and the conversion of the image data. The camera digital signal processor 21c (camera DSP 21c) which performs a process is included.

The CCD interface 21a performs processing such as digital clamp, white balance adjustment, or gamma correction on the image data composed of R, G, and B from the S / HA / D circuit 12 shown in FIG. Decrease components by 1/10 in the horizontal direction of the image data. After this processing, the CCD interface 21a communicates the image data to the camera DSP 21c via the image data bus 33 or to the memory controller 22.

From the image data of the CCD interface 21a, the detection circuit 21b performs detection for auto-focusing, auto-iris, or white balance adjustment.

The camera DSP 21c converts the image data of R, G, and B from the CCD interface 21a into image data composed of luminance signal Y and chrominance signals Cb, Cr. . The camera DSP 21c also has a simplified resolution converting circuit 21d which not only performs the above processing but also converts the resolution of the image data in a simplified form.

The simplified resolution converting circuit 21d operates to convert the resolution of the image data so as to lower the value if the resolution of the image data generated by the CCD image sensor 11 is larger than the VGA format.

In particular, the simplified resolution conversion circuit 21d includes a BY / RY separation circuit 61 for separating chrominance signals, a horizontal direction linear interpolation circuit 62 for interpolation in the horizontal direction, A BY / RY combining circuit 63 for synthesizing chrominance signals, a 1H delay circuit 64 for delaying each signal by a horizontal scanning period (1H period), and a vertical linear interpolation circuit 65.

The BY / RY separation circuit 61 separates the color difference signals BY and RY from the image data from the camera DSP 21c as chroma signals Cb and Cr, thereby separating the separated chroma signals in a horizontal linear direction. Transfer to interpolation circuit 62. The horizontal linear interpolation circuit 62 interpolates the luminance signals Y and the color difference signals BY and RY in the horizontal direction to lower the luminance in the horizontal direction, and interpolates the luminance signals Y and the color difference signals. (BY, RY) is passed to the BY / RY combining circuit 63.

The BY / RY synthesizing circuit 63 synthesizes the color difference signals BY and RY, and combines the luminance signals Y from the horizontal linear interpolation circuit 62 with the combined color difference signals BY.RY. To a delay circuit 64 and a vertical direction linear interpolation circuit 65. The 1H delay circuit 64 delays the luminance signals Y and the chrominance signals by 1H, and transmits the delayed signals to the vertical linear interpolation circuit 65. The vertical linear interpolation circuit 65 performs linear interpolation processing in the vertical direction based on the luminance signals Y and the chrominance signals BY and RY from the BY / RY combining circuit 63 and the 1H delay circuit 64. And outputs image data composed of luminance signals Y 'and color difference signals BY' and (RY) 'whose resolution is lowered in both the horizontal and vertical directions.

The resolution conversion circuit 28 performs resolution conversion processing for converting [p x q] image data into [m x n] image data. The resolution converting circuit 28 performs a process of suppressing the resolution to a preset value when the image data produced by the CCD image sensor 11 is a high resolution. However, it is possible to process low resolution image data into high resolution data.

Referring to FIG. 6, the resolution converting circuit 28 includes an input buffer 71 for storing image data input from the image data bus 33 and a horizontal direction for buffering image data from the input buffer 71 in a horizontal direction. Buffer 72, a horizontal deformation processing circuit 73 for converting the resolution of the image data from the horizontal buffer 72 in the horizontal direction, and buffering the image data from the horizontal deformation processing circuit 73 in the vertical direction A vertical buffer 74, a vertical deformation processing circuit 75 for converting the resolution of the image data in the vertical direction, and an output buffer 76 for buffering on output.

When the resolution of the image data is ready for conversion, the resolution conversion circuit 28 outputs a read request signal requesting the memory controller 22 to read the image data from the image memory 32, and then after the conversion processing of the image data. A write request signal for requesting the memory controller 22 to write image data to the image memory 32 is output. Resolution conversion circuit 28 also receives an acknowledgment signal indicating that memory controller 22 responds to the request signal.

Referring to FIG. 7, the horizontal buffer 72 is composed of a first delay circuit 81, a second delay circuit 82, and a third delay circuit 83, each of which creates a delay of one pixel. . Thus, the first delay circuit 81 outputs image data delayed by one pixel, and the second and third delay circuits 82 and 83 output image data delayed by two pixels and image data delayed by three pixels, respectively. .

Referring to FIG. 7, the horizontal deformation processing circuit 73 includes first to fourth multipliers 84, 85, 86, and 87 and first to third adders 88, 89, and 90. A circuit for normalizing the data is incidentally attached behind the adder 90.

The first multiplier 84 multiplies the image data supplied from the input buffer 71 by a preset coefficient and transfers the resulting data to the adder 88. The second multiplier 85 multiplies the image data supplied from the first delay circuit 81 by a predetermined coefficient and transfers the resulting data to the adder 88. The third multiplier 86 multiplies the image data supplied from the second delay circuit 82 by a predetermined coefficient and transfers the resulting data to the adder 89. The fourth multiplier 87 multiplies the image data supplied from the third delay circuit 83 by a predetermined coefficient and transfers the resulting data to the adder 90. The first adder 88 synthesizes the image data and passes the resulting data to the second adder 89. The second adder 89 synthesizes the image data and passes the resulting data to the third adder 90. The third adder 90 synthesizes the respective image data and transfers the resulting data to the vertical buffer 73 as image data whose resolution in the horizontal direction is converted.

Accordingly, the horizontal deformation processing circuit 73 weights a plurality of image data each having a pixel delay in a preset form with predetermined weights, and synthesizes the weighted image data to interpolate or reduce the pixels in the horizontal direction. To convert the resolution in the horizontal direction.

Vertical buffer 74 consists of a series connection of first to third buffers 91, 92, 93, each adapted to delay a one-line. Accordingly, the first buffer memory 91 outputs image data delayed by one line, and the second and third buffer memories 92 and 93 output image data delayed by two and three lines, respectively.

Referring to FIG. 7, the vertical deformation processing circuit 75 includes fifth to eighth multipliers 94 to 97 and fourth to sixth adders 98 to 100. Vertical deformation processing circuit 75 sometimes includes circuitry for normalizing data downstream of adder 90.

The fifth multiplier 94 multiplies the image data supplied from the horizontal direction conversion circuit 73 by a preset coefficient, and transfers the resulting data to the fourth adder 98. The sixth multiplier 95 multiplies the image data supplied from the first line memory 91 by a predetermined coefficient and transfers the resulting data to the fourth adder 98. The seventh multiplier 96 multiplies the image data supplied from the second line memory 92 by a predetermined coefficient and transfers the resulting data to the fifth adder 99. The eighth multiplier 97 multiplies the image data supplied from the third line memory 93 by a predetermined coefficient and transfers the resulting data to the sixth adder 100. The fourth adder 98 synthesizes the image data and passes the resulting data to the fifth adder 99. The fifth adder 99 synthesizes the image data and transfers the resulting data to the sixth adder 100. The sixth adder 100 synthesizes each image data, and outputs the resulting data as image data whose resolution in the horizontal direction is converted.

Accordingly, the vertical deformation processing circuit 75 weights a plurality of image data each having a line delay in a predetermined form with a predetermined weight, and synthesizes the weighted image data to interpolate or reduce the pixels in the horizontal direction. Convert the resolution in the vertical direction.

In Fig. 7, the resolution conversion circuit 28 first performs resolution conversion in the horizontal direction followed by resolution conversion in the vertical direction. However, it is possible for the resolution conversion circuit 28 to perform resolution conversion in the vertical direction followed by conversion in the horizontal direction. That is, the resolution conversion circuit 28 supplies the image data from the input buffer 71 to the vertical buffer 74, the vertical buffer 74, the vertical deformation processing circuit 75, the horizontal buffer 72, And the processing in the horizontal deformation processing circuit 73 in order.

In the above-described embodiment, the first to third buffer memories 91 to 93 in the vertical buffer 74 are configured to store 1-line 1H image data. Alternatively, the first to third buffer memories 91 to 93 may be configured to store one line or less of image data as shown in FIG. 9. At this time, the memory controller 22 needs to read the image data stored in the image memory 32 every N pixels, as shown in FIG.

In particular, the memory controller 22 reads pixel data corresponding to a viewing screen stored in the image memory 32 every N pixels based on lines in the vertical direction. Referring to FIG. 11, each observation screen is composed of pxq pixels, and the coordinates of the upper left pixel are (1,1), the coordinates of the upper right pixel are (p, 1), and the coordinates of the lower left pixel are (1 , q), and the coordinate of the lower right pixel is (p, q).

Referring to FIG. 12, the memory controller 22 causes the image data of N pixels to be read based on lines in the horizontal direction in order of rows 1, 2, ..., q. This means that the memory controller 22 has image data corresponding to N pixels from the left end, or N xq pixels, namely (1,1), (1, q), (N, q), and (N, 1). Read pixel data in the area defined by " This image data is hereinafter referred to as image data set 1.

The memory controller 22 then reads the image data within the range defined by (N-1,1), (N-1, q), (2N-2, q), (2N-2,1), which are This is referred to as image data set 2 hereafter. When the memory controller 22 reads the image data set 1 and the image data set 2, this is equivalent to reading the image data of the (N-1) th column and the Nth column twice.

The reason is that since the vertical deformation processing circuit 75 performs interpolation starting from the surrounding pixels, the pixels stored at the start end and the later end of the first to third buffer memories 91 to 93 are subject to processing. Because it is not. For example, when the image data set 1 is read, the pixels N and 1 are not subjects of the interpolation process in the vertical direction. However, this pixel (N, 1) is read out when the pixel data set 2 is read out to be the subject of the interpolation process.

In a similar manner, the memory controller 22 reads N pixel image data in the horizontal direction every line so that the image data of the last two columns of the immediately preceding image data is included. This conveys the image data set in the resolution converting circuit 28.

The vertical buffer 74 is supplied with image data in an amount corresponding to the capacities of the first to third buffers 91 to 93 based on the line. Thus, one line of image data offset is stored in each of the first to third buffer memories 91 to 93. The vertical deformation processing circuit 75 may perform resolution conversion processing in the vertical direction based on image data from the first to third buffers 91 to 93 of the vertical buffer 74.

In the memory controller 22, the memory controller 22 reads in association with the capacity of the memory buffer even if the capacity of the buffer memory required for the resolution conversion in the vertical direction does not reach one line, thereby causing the resolution conversion circuit 28 to read out. Resolution conversion can be performed in the vertical direction.

Although the read overlap between the image data sets is two columns, the overlap is more than two columns or there is no overlap. Note that the present invention is applicable to image signal processing such as camera signal processing without limitation on resolution conversion.

Although the above description relates to embodiments in which the buffer memory is used for interpolation in the vertical direction, the present invention is also applicable to embodiments in which the buffer memory is used for interpolation in the horizontal direction.

That is, the resolution conversion circuit 28 may perform resolution conversion in the horizontal direction using the vertical buffer 72a constituted by the buffer memory 72a having the capacity of N pixels. The memory controller 22 can read image data of N pixels based on the column in the order of rows 1, 2, ..., p in the vertical direction, as shown in FIG. In the meantime, the memory controller 22 needs to read the image data stored at the beginning and the end of the buffer memory twice so that these image data become the subject of the horizontal interpolation processing as in the above-described vertical interpolation processing.

Accordingly, the memory controller 22 performs an image from the image memory 32 such that resolution conversion processing in the vertical and horizontal directions is executed for the first to third buffer memories 91 to 93, each having a capacity of N pixels. Data can be read. This can cause the circuit scale of the horizontal buffer 72 and the vertical buffer 74 to be reduced to lower the manufacturing cost.

The NTSC / PAL encoder 23 which performs encoding as described above also has a simplified resolution converting circuit 23a to increase the resolution of image data, if necessary, before the encoding process.

The simplified resolution converting circuit 23a performs resolution converting to match the display standard of the finder 36 when the image data on the image memory 32 is lower than the resolution required for display.

Referring to Fig. 15, the simplified resolution converting circuit 23a includes a line memory 101 for storing image data from the image data bus 33, and a vertical linear interpolation circuit for interpolating image data in the vertical direction (V-. Directional linear interpolation circuit 102, and horizontal directional interpolation circuit 103.

The line memory 101 stores image data from the input terminal in in an amount corresponding to one line to transfer the image data to the V-direction linear interpolation circuit 102 in the order in which it is stored. The V-direction linear interpolation circuit 102 weights the image data from the V-direction linear interpolation circuit 102 and the image data from the input terminal in with a predetermined weight to perform linear interpolation in the vertical direction. The horizontal interpolation circuit 103 interpolates Cb and Cr to a third-order filter while interpolating Y with an order-seven filter. This is simply interpolation to increase the resolution by a factor of two. The horizontal interpolation circuit 103 outputs image data at an output terminal out.

For example, the image data input from the input terminal in is a, the image data read from the line memory 101 is b, the weighting factor is g (where 0 ≦ g ≦ 1), and V-direction linear interpolation. If the image data output by the circuit 102 is c, the V-direction linear interpolation circuit 102 performs the following processing:

c = g * a + (1-g) * b

The image data output by the output terminal out is encoded by the NTSC / PAL encoder 23 as described above.

In the signal processing system, the digital still camera 1 is composed of two chips, namely the signal processor 20 and the CPU 41. Therefore, since the respective signal processing circuits are respective chip configurations, the substrate surface area and power consumption may be smaller than when each signal processing circuit is a separate chip configuration.

Further, since the signal processor 20 is not a chip configuration including the CPU, the signal processing can be adaptively executed even if the application associated with the CPU 41 changes. That is, if the signal processor 20 is a chip configuration including a CPU, it is impossible to reconfigure the chip when the application of the CPU changes. However, the signal processor 20 may perform preset signal processing using a CPU which is an optimal structure based on the application.

The digital still camera 1 having the above-described structure includes a finder mode for confirming the position or state of the subject before recording the image, a recording mode for capturing the image of the confirmed subject, and a playback mode for confirming the photographed state of the subject image. Process is executed according to the dominant mode.

In the finder mode, the user must observe the state of the subject shown in the finder 36 before pressing the shutter button not shown to photograph the subject. In this finder mode, the memory controller 22 and other circuits are controlled in the following manner. In order to explain the modes, reference is mainly made to FIG. 4 and sometimes to FIG. 16.

In the finder mode, the CCD image sensor 11 generates thinning image signals 1/3 of the vertical components and sends the digitized image data through the S / HA / D circuit 12 to the CCD interface 21a. To supply.

The CCD interface 21a performs signal processing in synchronization with the clocks shown in FIG. 16A. In particular, the CCD interface 21a reduces the horizontal component of the image data supplied by the image generating unit 10 by one third, gamma corrects the reduced processed image data, and converts the corrected data into the camera DSP. To 21c. The CCD interface 21a supplies the camera DSP 21c with image data converted to 340 × 256 from the 1/3 reduction process.

The camera DSP 21c performs data conversion processing on the reduced image data to YCrCb image data. The camera DSP 21c changes the resolution of the image data in the simplified resolution conversion circuit 21d so as to lower the resolution of the image data (340 x 256 → 320 x 240), and converts the converted image data into the image data bus 33. Transfer to the memory controller 22 through.

The simplified resolution converting circuit 21d lowers the resolution in a simplified form to the extent necessary for subsequent processing. In this manner, if the image data generated by the CCD image sensor 11 is a high resolution, the transmission range taken by the image data generated by the CCD image sensor 11 may be set to an image data bus to maintain the real-time characteristics of the finder mode. 33) can be reduced to avoid congestion.

The memory controller 22 writes the image data to the image memory 32, reads the image data from the image memory 32 as shown in FIG. 16D, and reads the read image data through the image data bus 33 through the NTSC. / Pass to the PAL encoder (23). At the same time, the memory controller 22 reads the OSD data stored in the image memory 32 as shown in Fig. 16E, and transfers the OSD data stored in the image memory 32 as shown in Fig. 16E. Fig. 16F shows the delivery state on the image data bus 33 which enables the real time processing described above.

The NTSC / PAL encoder 23 performs resolution conversion of 320 x 240 → 640 x 240 or 320 x 240 → 640 x 288 for NTSC system or PAL system, respectively, and converts the converted image data into NTSC / PAL encoder 23. To pass. The NTSC / PAL encoder 23 also converts the image data into OSD data transmitted to the finder 36 shown in FIG. 3 which is data of an NTSC system or a PAL system. This allows the image of the subject, title information, etc. to be displayed on the finder 36 in real time.

On the other hand, the NTSC / PAL encoder 23 is designed to increase the resolution of low resolution data, for example, when 320 x 200 image data is supplied, this is 640 x 240 image data and 640 x for NTSC system and PAL system, respectively. The resolution is converted so that it is converted into image data.

In the digital still camera 1, the resolution of the image data generated by the CCD image sensor 11 is degraded in a simplified form in the finder mode to reduce the amount of data, so that the image data has a bandwidth limit value of the image data bus 33. And the resolution will be increased at the output stage as needed for display at the timing shown in FIG. 16F.

Thus, in the digital still camera 1, the image data is within the bandwidth limit of the image data bus 33 to allow the image of the subject to be displayed in the finder 36 without having to perform a time-consuming reduction process even if it is a high resolution. maintain.

A circuit for preferential processing, i.e., the CCD interface 21a, the camera DSP 21c, or the NTSC / PAL encoder 23 is set up earlier in the CPU 41, and the signal processing is other circuits as in the above circuits. When executed in time division at, the processing of each of the circuits having a high priority may be preferentially performed according to the data amount of the image data.

If the data amount of the image data is large in the simplified resolution converting circuit 21d, the data processing may be performed at a high processing speed to give priority to real-time processing, even if the image quality deteriorates to some extent under the control of the CPU 41. Can be. In this way, even if the data amount of the image data generated in the image generating unit 10 is large, the high speed processing can be executed in the finder mode.

In the case of a digital still camera 1 having an electronic zooming function, the CPU 41 can control each circuit in the following manner.

The memory controller 22 causes the image data supplied via the CCD interface 21a and the camera DSP 21c to be recorded in the image memory 32, and the image data is read out from the image memory 32 so that the resolution converting circuit 28 To be delivered). The resolution converting circuit 28 formulates the image data enlarged from a part of the input image by the electronic zoom function, and outputs the resulting image to the image memory 32. This image data is read out from the image memory 32 and output to the finder 36 through the NTSC / PAL encoder 23. This produces electronically zoomed image data.

The finder mode gives the highest priority to real-time characteristics, so that time consuming processing is not performed by the respective circuits. However, the CPU 41 allows the memory controller 22 and other circuits to perform various processing operations as long as they are within the range allowed by the transfer area of the image data bus 33.

For example, the memory controller 22 reads image data from the image memory 32 in which the image data supplied from the CCD interface 21a is stored, and reads the read image data through the image data bus 33 through NTSC / PAL. It can be configured to feed to the encoder 23 and also to the JPEG encoder / decoder 29. The finder 36 displays an image of the subject in real time, and the JPEG encoder / decoder 29 compresses the image data according to the JPEG system.

JPEG encoder / decoder 29 compresses / expands still images, but cannot process high-pixel images in real time. Thus, the JPEG encoder / decoder 29 slices a portion of the image so that the resolution is degraded through compression or the number of preset frames of image data supplied from the image data bus 33 via compression (frames or It is possible to reduce the number of fields. This makes it possible to continuously shoot frame-reduced still images or to continuously shoot low resolution images.

The user observes the state of the object displayed on the finder 36 in the finder mode described above. When it is determined that the object is to be photographed, the user presses a shutter button not shown.

When the shutter button is pressed, the digital still camera 1 proceeds to the recording mode. In the recording mode, the CPU 41 controls the memory controller 22 and respective circuits in the following manner to record the image of the photographed subject to the recording device 51.

The CCD image sensor 11 stops the reduction operation at the same time as the shutter is pressed, generates image signals in the XGA format, and transfers the digitized image data to the CCD interface 21a through the S / HA / D circuit 12. .

The CCD interface 21a transfers the image data supplied from the S / H-A / D circuit 12 to the memory controller 22 via the memory data bus 33 rather than the camera DSP 21c. The memory controller 22 first writes the image data to the image memory 32, and subsequently reads the image data and transfers the read image data to the camera DSP 21c via the image data bus 33. The camera DSP 21c converts image data composed of RGB into image data composed of Y, Cb, and Cr.

The image data once recorded in the image memory 32 is supplied to the camera DSP 21c. That is, the camera DSP 21c performs data conversion on the image data from the image memory 32 instead of the image data supplied directly from the CCD interface 21a. Thus, although the camera DSP 21c does not need to perform high-speed data conversion, it is sufficient that the camera DSP 21c executes this processing when the image data bus 33 is not complicated. In other words, since the camera DSP 21c does not need to perform the processing in real time, the data conversion processing can be executed by giving priority to the image quality higher than the high processing speed, and the resulting converted image data is executed by the image data bus 33. May be transferred to the memory controller 22. The memory controller 22 causes the image data to be written to the image memory 32.

The memory controller 22 causes the image data to be read from the image memory 32 and the read image data to be passed to the JPEG encoder / decoder 29. The JPEG encoder / decoder 29 compresses the image data in accordance with the JPEG system, and records the compressed image data in the recording device 51 shown in FIG.

If real time processing is not necessary, such as during recording, the CPU 41 advances the image data in the image memory 32 temporarily after using it to temporarily use the transfer band of the image data bus 33 to process the high pixel image. Allow the set process to run.

The CPU 41 directly records the image data in the XGA format to the recording device 51 in the recording mode. However, it is possible to convert the resolution of the image data before the resolution converting circuit 28 writes the image data to the recording device 51. In particular, the resolution conversion circuit 28 is associated with a VGA in association with the VGA to allow the JPEG encoder / decoder 29 to compress the image data and to write the compressed data to the recording device 51. It is possible to convert the resolution of the image data read out from (32) (1024 x 768? 640 x 480).

If desired to confirm the image taken after image capture, the operator presses a playback button, not shown, to play the image taken.

When the play button is pressed, the digital still camera 1 is moved to the play mode. In the playback mode, the CPU 41 controls the respective circuits in the following manner to read image data of the subject.

In other words, upon detecting that the playback button is pressed, the CPU 41 reads image data from the recording device 51 and reads the image before passing the data to the JPEG encoder / decoder 29 via the CPU bus 34. The data is temporarily stored in the DRAM 42. The JPEG encoder / decoder 29 expands the image data read out from the recording device 51 according to the JPEG system to produce image data in the XGA format, and stores the resulting image data through the image data bus 33 through the memory controller. To 22.

The memory controller 22 writes the image data to the image memory 32, reads the image data from the image memory 32, and transfers the read image data to the resolution converting circuit 28 via the image data bus 33. do.

The resolution conversion circuit 28 performs resolution conversion so that the image data conforms to the VGA format (1024 x 768 → 640 x 480 in NTSC system, and 1024 x 768 → 640 x 576 in PAL system), and converts the converted image data into an image. Transfers to memory controller 22 via data bus 33. The image data is then read from the image memory 32 and passed to the finder 36 through the NTSC / PAL encoder 23. The image corresponding to the image data recorded in the recording device 51 is displayed in the finder 36.

That is, since the image data recorded in the recording device 51 has a high resolution, the CPU 41 first lowers the resolution and then transfers the image data to the finder 36.

In addition, the CPU 41 sets the priority of the circuits that are preferentially processed for each of the finder mode, the recording mode, and the reproducing mode, and causes the related circuits to execute the processing in accordance with the priority for moving to one mode. It is possible. This can allow the signal processing of the image data to be effectively executed in accordance with the processing contents in each mode.

In the above-described embodiment, it is assumed that the data being processed is the same image data as the XGA. It should be noted that the present invention is not limited to this embodiment but may be applied to the processing of image data composed of, for example, one million pixels or more.

According to the imaging apparatus of the present invention, in performing the resolution conversion, the first resolution conversion means lowers the resolution of the image data from the imaging means, and the second resolution conversion means is supplied from the first resolution conversion means via the image data bus. By increasing the resolution of the image data, the occupancy ratio of the image data on the image data bus can be reduced, and the subject image can be displayed in real time on the display means.

Claims (15)

In an imaging apparatus: Imaging means for generating image data corresponding to imaging light from a subject; First resolution converting means for lowering the resolution of the image data from the imaging means by resolution conversion; Second resolution converting means for raising the resolution of the image data supplied from the first resolution converting means via an image data bus by resolution converting; And And output means for outputting the image data from the second resolution converting means to a display means. The method of claim 1, And display means for displaying an image corresponding to the image data from the output means. The method of claim 1, And the first resolution converting means executes a resolution converting which reduces the occupancy rate of the image data on the image data bus. The method of claim 1, And the second resolution converting means performs resolution converting corresponding to a display standard on the display means. In the imaging device: Imaging means for generating image data corresponding to imaging light from a subject; First resolution converting means for lowering the resolution of the image data from the imaging means by resolution converting; Second resolution converting means for raising the resolution of said image data supplied from said first resolution converting means via a resolution data bus; Storage means for storing the image data; Third resolution converting means for converting the resolution of the image data supplied from the storage means; And And output means for outputting the image data from the second or third resolution converting means to a display means. The method of claim 5, wherein And display means for displaying an image corresponding to the image data from the output means. The method of claim 5, wherein And the first resolution converting means executes a resolution converting which reduces the occupancy rate of the image data on the image data bus. The method of claim 5, wherein And the second and third resolution converting means perform resolution conversion corresponding to a display standard on the display means. The method of claim 5, wherein And the speed of resolution conversion by the first and second resolution converting means is relatively faster than the speed of the third resolution converting means. The method of claim 5, wherein The imaging device has a plurality of operating modes, The first and second resolution converting means or the third resolution converting means are selectively used according to the operating modes. In the recording / playback apparatus: Imaging means for generating image data corresponding to imaging light from a subject; Recording / reproducing means for recording the image data on a recording medium and reproducing the image data recorded on the recording medium; First resolution converting means for performing resolution converting to lower the resolution of the image data from the imaging means; Second resolution converting means for performing resolution converting to raise the resolution of said image data supplied from said first resolution converting means via an image data bus; Storage means for storing the image data; Third resolution converting means for performing resolution converting of said image data supplied from said storage means; And And display means for displaying an image corresponding to the image data from the second or third resolution converting means. The method of claim 11, And the first resolution converting means performs resolution converting to lower the occupancy ratio of the image data on the image data bus. The method of claim 11, And the second and third resolution converting means perform resolution conversion corresponding to a display standard on the display means. The method of claim 11, And the speed of resolution conversion by the first and second resolution converting means is relatively faster than the speed of the third resolution converting means. The method of claim 11, The imaging apparatus has a finder mode for displaying the image data on the display means, a recording mode for recording the image data on the recording medium, and a reproduction mode for reproducing the image data from the recording medium, If the mode is the finder mode, the first and second resolution converting means are used, And the third resolution converting means is used if the mode is the recording mode or the reproduction mode.

Images