KR20000011790A - Imaging apparatus and recording/reproducing apparatus - Google Patents

Abstract

PURPOSE: A device for imaging is provided to display an image on a finder in real time even in case of image data of a high resolution. CONSTITUTION: An imaging device contains; an imaging instrument generating image data corresponding to an imaging ray from an object; a first resolution transforming instrument lowering the resolution of the image data from the imaging instrument in response to the transformation of the resolution; a second resolution transforming instrument raising the resolution of the image data supplied from the first resolution transforming instrument through an image data bus in response to the transformation of the resolution; and an output instrument outputting the image data from the second resolution transforming instrument to a displaying instrument.

Description

Signal processing apparatus and recording / reproducing apparatus

The present invention relates to an imaging apparatus having a function of displaying an image of a subject on a finder in real time.

The digital still camera recovers the image data obtained by the CCD image sensor to DRAM or flash memory, and then transfers the image data to a personal computer or the like. The main part of this type of digital still camera has been the type responsible for video graphics array (VGA) systems.

Referring to Fig. 1, this digital still camera 200 includes a CCD image sensor 201 for generating an image signal, an input processing / image processing circuit 202, a memory controller 203 for recording and reading image data, and a preliminary description. An output processing circuit 204 for converting the output image of the set system, a finder 205 for displaying the state of the object during image capturing, and a recording unit 207 for recording the compressed image data over the CPU bus 206. And a compression / extension circuit 208 for compressing / extending the image data. The digital still camera 200 also includes a memory 209 formed for example of DRAM and a CPU 210 that controls the overall device.

Before starting to take an image of an object, the user must check the object image displayed in the finder 205. This state is called finder mode. At this time, the CCD image sensor 201 transfers the image signal obtained by photoelectric conversion to the input processing / image processing circuit 202. The input processing / image processing circuit 202 performs double sampling processing correlated to the video signal to digitize the video signal. The input processing / image processing circuit 202 then executes preset signal processing, such as gamma correction, knee processing, or camera processing, and passes the processed image signal to the memory controller 203 to provide the CPU. In response to control by 210, the image data is transferred from the input processing / image processing circuit 202 to the output processing circuit 204. The output processing circuit 204 encodes the image data according to, for example, a National Television System Committee (NTSC) system, analogizes the encoded image data, and transfers the resulting analog data to the finder 205. This allows the object to appear in the finder 205 as an image pickup object.

On the other hand, when the user presses a shutter button (not shown) to shift to the recording mode, the memory controller 203 causes the image data supplied from the input processing / image processing circuit 202 to be recorded in the memory 209. The CPU 210 causes the image data to be read from the memory 209, and records the image data from the recording unit 207 in the compression / expansion circuit 208 according to, for example, a JPEG (Joint Photographic Experts Group) system. Record at (207).

When the user executes a preset process and is shifted to the reproduction mode, the CPU 210 causes the image data to be read from the recording unit 207 so that the image data can be expanded from the compression / expansion circuit 208 to the JPEG system and the resulting Data is transferred to the finder 205 through the memory controller 20 and the output processing circuit 204. The captured image is displayed in the finder 205.

In connection with recent outstanding technological advances in CCD image sensors, the resolution of image data exceeds nearly 1,000,000 pixels. On the other hand, there is a concern that the digital still camera of the above-described structure cannot sufficiently handle image data exceeding 1,000,000 pixels.

For example, when the CCD image sensor 201 outputs a high resolution image signal in the finder mode, the input processing / image processing circuit 202 executes the resolution conversion in real time in response to the resolution of the finder 205. do. At the same time, the memory controller 203 accesses the memory 209. In addition, the output processing circuit must execute a process already set.

Since the result is stagnant in the CPU bus 106, each circuit cannot execute a preset process in real time and an image of the subject is displayed in the finder 205 in frame-reduced form. In this case, when the subject moves, a deviation occurs between the movement of the actual subject and the subject displayed on the finder 105, which makes the imaging operation inconvenient.

On the other hand, if the image data is not high resolution, the resolution of the image data should be converted to, for example, the resolution of the NTSC system or the PAL system in consideration of the system of the finder 205. In this case, it is similarly required to display an image in the finder in real time.

Accordingly, it is an object of the present invention to provide an imaging apparatus that can display an image in a finder in real time even if the image data of the image is high resolution.

In one aspect of the invention, the present invention relates to an imaging means for generating image data corresponding to an imaging ray from a subject, first resolution converting means for lowering the resolution of the image data from the imaging means by resolution conversion, and resolution conversion. A second resolution converting means for raising the resolution of the image data supplied from the first resolution converting means via the image data bus, and an output means for outputting the image data from the second resolution converting means to the display means. To provide.

In another aspect of the present invention, the present invention provides imaging means for generating image data corresponding to imaging rays from a subject, recording / reproducing means for recording image data on a recording medium, and reproducing image data recorded on the recording medium. First resolution converting means for performing resolution conversion for lowering the resolution of the image data from the imaging means, and second resolution for performing resolution conversion for raising the resolution of the image data supplied from the first resolution converting means via the image data bus. A display for displaying an image corresponding to the image data from the converting means, the storage means for storing the image data, the third resolution converting means for performing resolution conversion of the image data supplied from the storage means, and the second or third resolution converting means. It provides a reproduction / recording apparatus comprising means. .

According to the imaging device of the present invention, in performing resolution conversion, the first resolution converting means lowers the resolution of the image data from the imaging means, and the second resolution converting means is supplied from the first resolution converting means via the image data bus. By increasing the resolution of the image data, the occupancy ratio of the image data on the image data bus can be reduced, and the subject image can be displayed in real time on the display means.

1 is a block diagram illustrating the structure of a conventional digital still camera.

Fig. 2 is a block diagram showing the structure of a digital still camera that realizes the present invention.

FIG. 3 is a block diagram showing the structure of the digital still camera shown in FIG.

4 is a block diagram for explaining the flow of image data in the signal processing unit of the digital still camera shown in FIG. 2;

5 is a diagram for explaining the structure of a simplified resolution converting circuit in an input processing circuit of a signal processing unit;

6 is a block diagram showing the structure of a resolution converting circuit in the signal processing unit;

7 is a block diagram showing the structure of a horizontal buffer, a horizontal buffer processor, a vertical buffer, and a vertical buffer processor of a resolution converter circuit;

8 is a block diagram showing another structure of a resolution conversion circuit.

9 is a block diagram showing a structure of a vertical buffer of a resolution converting circuit.

Fig. 10 is a diagram explaining a technique of reading image data from the image memory by the memory controller.

11 is a diagram for explaining coordinate positions of pixels constituting an image.

FIG. 12 illustrates another technique for reading image data from the image memory by the memory controller. FIG.

Fig. 13 is a diagram showing the structure of a horizontal buffer of a resolution converting circuit composed of a line buffer.

Fig. 14 is a diagram explaining a technique when the memory controller reads image data from the image memory.

Fig. 15 is a block diagram showing the structure of a simplified resolution conversion circuit in the NTSC / PAL encoder of the signal processing unit.

16A to 16F are timing diagrams for explaining signal processing contents of respective circuits in a finder mode.

Explanation of symbols on the main parts of the drawings

1 ... digital still camera 10 ... image generating unit

11.CCD image sensor

12.Sample Retention-Analog / Digital Circuit

13 ... Timing generator 20 ... Input signal processor

32 Video Memory 40 Controller

Referring to the drawings, preferred embodiments of the present invention are described in detail.

The invention is applied to a digital still camera 1, for example, which is constructed as shown in FIG.

The digital still camera 1 includes an image generating unit 10 for generating an image signal, an input signal processor 20 for processing image data in a predetermined form, an image memory 32 composed of SDRAM, and an input signal processor 20. The controller 40 controls the control panel 40).

The image generating unit 10 is a solid-state imaging device that generates an image signal, such as a CCD image sensor 11, and a sample holding-analog / digital circuit (S / HA / D) that holds a sample and digitizes the image signal into output image data. Circuit 12) and a timing generator 13 for generating a timing signal. This timing generator 13 generates a horizontal synchronization signal and a vertical synchronization signal to control each circuit of the image generation unit 10 based on the synchronization signal supplied from the signal processor input.

The CCD image sensor 11 generates image data corresponding to, for example, extended graphic array (XGA) 1024 x 768 (XGA) pixel data composed of 8,000,000 pixels. The CCD image sensor 11 is driven based on the synchronization signal from the timing generator 13 to output image data at a rate of 30 frames per second. In the meantime, the CCD image sensor 11 has a function of thinning the video signal and thins the vertical components of the video signal into 1/2, 1/3, 1/4, ... and outputs the resulting signal. can do.

The S / HA / D circuit 12 also executes sample retention and A / D conversion at a predetermined sampling interval based on the synchronization signal from the timing generator 13 to transfer the resulting image data to the signal processor 20. Is adapted to.

The signal processor 20 includes a single large scale integrated circuit (LSI). The signal processor 20 is an input signal processor 21 which performs input processing and camera processing on the image data from the image generating unit 10, and a memory controller which controls reading / writing of image data to the image memory 32. (22), NTSC / PAL (phase alternation by line) encoder 23, D / A converter 24 for analogizing a video signal and outputting the resulting analog signal externally, and generating a synchronization signal and synchronizing the result And a sync generator 26 for supplying a signal to the timing generator 13.

The signal processor 20 also includes a memory interface 27, which is an interface for the image memory 32, a resolution conversion circuit 28 for converting the resolution of the image data, and a Joint Photographic Experts Group (JPEG) for compressing / extending the image data. The encoder / decoder 29, the JPEG interface 30 which is an interface of the JPEG encoder / decoder 29, and the host interface 31 which is an interface having a data transmission / reception relationship with the CPU of the controller 40 are included.

The input signal processor 21 receives image data from the S / HA / D circuit 12 by digital clamping, shading correction, aperture correction, gamma correction, or color processing. Processing, and delivers the processed result signal to the memory controller 22. The input signal processor 21 has a function of processing the input data to convert the input data into Y, Cb, and Cr. If the resolution of the video data is larger than the video graphics array (VGA), the input signal processor 21 can execute a process of lowering the resolution. The input signal processor 21 also delivers data to the controller 40 to perform auto-focus and auto-iris detection to achieve automatic adjustment of the focusing machine and the iris machine. The input signal processor 21 also detects signal levels of the three main colors that make up the image data and adjusts the automatic white balance.

The memory controller 22 also allows the image data supplied from the input signal processor 21 or other circuitry to be written to the image memory through the memory interface 27 and the image data of the image memory 32 through the memory interface 27. Run the control to read it. At this time, the memory controller 22 detects whether there are defective pixels in the CCD image sensor 11 based on the image data stored in the image memory 32.

The memory controller 22 transfers the image data read from the image memory 32 to the NTSC / PAL encoder 23, for example. When image data is supplied from the memory controller 22, the NTSC / PAL encoder 23 encodes the image data according to the NTSC system or the PAL system and transfers the encoded data to the D / A converter 24. The D / A converter 24 analogizes the image data and outputs the resulting analog signal through the output terminal 25.

The memory controller 22 transmits the image data read from the memory controller 22 to the resolution converting circuit 28, and the image data while the image data output by the resolution converting circuit 28 is recorded in the image memory 32. Causes the resolution conversion.

The memory controller 22 transmits the image data via the JPEG interface 30 to the JPEG encoder / 30 so as to perform compression of still images while allowing the image data expanded by the JPEG encoder / decoder 29 to be recorded in the image memory 32. It passes to the decoder 29.

The image memory 32 not only stores image data as described above, but also stores on-screen-display (OSD) data as character generator data. The OSD data consists of bit map data. The controller 22 controls the reading / writing of the OSD data. The video data and the OSD data are synthesized by the NTSC / PAL encoder 23.

The controller 40 may include a central processing unit (CPU) 41, a dynamic random access memory (DRAM) 42, and a ROM in which a control program for the CPU 41 is stored, which controls each circuit of the signal processor 20. read-only memory 43, a flash memory interface 44 that is an interface for exchanging image data with a storage device 51 such as a flash memory, and an IrDA interface 45 that is an interface of a communication circuit 52 composed of IrLEDs. It includes.

For example, the CPU 41 allows the image data compressed by the JPEG encoder / decoder 29 to be written to the storage device 51 composed of flash memory via the flash memory / interface 44, while storing the image data. The data is read from the device 51 to convey the image data read from the JPEG encoder / decoder 29. The CPU 41 also causes the image data read out from the storage device 51 to be output to the outside as an infrared ray through the IrDA interface 45 and the communication circuit 52.

The structure of the digital still camera 1 is shown in FIG.

The input signal processor 21 transfers the image data from the CCD image sensor 11 to the image memory 32 via the image data bus 33. The NTSC / PAL encoder 23 encodes the image data from the image memory 32 in a preset form and transfers the encoded result data to the finder 36. This causes the image of the object to be displayed in a finder 36 adapted to display the image associated with the image data in VGA format.

The memory controller 22 executes data transfer between the signal processing circuit connected to the image data bus 33 and the image memory 32. The resolution conversion circuit 28 performs resolution conversion on the image data from the image memory 32 and transmits the result to the image memory 32. The JPEG encoder / decoder 29 compresses the image data from the image memory 32 according to the JPEG system, passes the compressed image data to the CPU 41 via the CPU bus 34, and stores the compressed image data. To be recorded on the device 51. The CPU 41 can also output the compressed image data to the outside via the CPU bus 34 and the communication circuit 52.

Thus, in FIG. 3, each circuit of the signal processor 20 is interconnected via an image data bus 33. The image data bus 33 is a virtual bus, which indicates that there is a limitation in the transmission band for the image data exchanged between the respective circuits.

In signal processor 20, each circuit, such as NTSC / PAL encoder 23 or resolution conversion circuit 28, sends a request signal to memory controller 22 indicating that image data is required. These circuits also send a request signal to the memory controller 22 when outputting the image data after processing the image data.

Upon receiving the request signal from each circuit, memory controller 22 selects the circuit with the highest priority and sends an acknowledgment signal to the selected circuit. The acknowledgment signal indicates that the image data can be passed to the circuit receiving the signal or that the image signal output by the circuit receiving the acknowledgment signal is ready to be received. The memory controller 22 reads the image data from the image memory 32 and transfers the read image data to the circuit corresponding to the destination of the acknowledgment signal via the image data bus 33. The memory controller 22 receives the image data output by the circuit which transmitted the acknowledgment signal, and records the image data in the image memory 32.

Upon receiving the request signal from the plurality of circuits, the memory controller 22 may preferentially select a circuit that should execute a process in real time. For example, if an image of an object is to be displayed in the finder 36, the memory controller 22 preferentially selects the input signal processor 21 and the NTSC / PAL encoder 23. It is also possible to decode the bus charge ratio of the image data on the image data bus 33 so that the memory controller 22 determines the priority of each circuit depending on the charge ratio.

If image data can be delivered to each circuit within the propagation band limitation of the image data bus 33, then a grant signal is sent to each circuit to allow the memory controller 22 to execute a predetermined process in each circuit. It is possible to implement control that transfers in pieces. This allows the memory controller 22 to access the data in each circuit in real time, such that the image data from each circuit is written to the image memory 32 or that the image data in the image memory 32 is stored in each of them. To be read into the circuit and delivered.

When the memory controller 22 accesses an external circuit (not shown) via the image data bus 33, if the external circuit can transmit the above-described request signal or receive the transmitted acknowledgment signal, the memory controller 22 Each circuit in the signal processor 20 can be simultaneously time-divisionally accessed within the transmission band limitation of the video data bus 33. That is, if within the band range of the image data bus 33, the memory controller 22 divides the circuit or signal processor in the signal processor 20 into time divisions regardless of the number of circuits or external circuits in the signal processor 20. The external circuit in 20 can be accessed simultaneously.

As described above, the memory controller 22 executes the arbitration of the image data bus 33, the write / read control of the image data between the image memory 32 and each circuit, and the data transfer to the CPU bus 34. do.

Referring to FIG. 4, a specific flow of image data in the signal processor 20 is described.

The input signal processor 21 converts the CCD interface 21a for performing signal processing preset to the image data from the image generating unit 10, the detection circuit 21b for processing the CCD interface 21a, and the conversion of the image data. The camera digital signal processor 21c (camera DSP 21c) which performs a process is included.

The CCD interface 21a performs processing such as digital clamp, white balance adjustment, or gamma correction on image data composed of R, G, and B from the S / HA / D circuit 12 shown in FIG. In this case, the component is reduced by 1/10 in the horizontal direction of the image data. After this processing, the CCD interface 21a communicates the image data to the camera DSP 21c via the image data bus 33 or to the memory controller 22.

From the image data of the CCD interface 21a, the detection circuit 21b performs detection for auto-focus, auto-iris, or white balance adjustment.

The camera DSP 21c converts the image data of R, G, and B from the CCD interface 21a into image data composed of a luminance signal Y and a chrominance signal Cb, Cr. The camera DSP 21c also has a simplified resolution converting circuit 21 for performing the above processing as well as converting the resolution of the image data in a simplified form.

The simplified resolution converting circuit 21d operates to convert the resolution of the image data so as to lower the value if the resolution of the image data generated by the CCD image sensor 11 is larger than the VGA format.

Specifically, the simplified resolution conversion circuit 21d includes a BY / RY separation circuit 61 for separating the color difference signals, a horizontal linear interpolation circuit 62 for interpolation in the horizontal direction, and a BY for synthesizing the color difference signals. / RY combining circuit 63, a 1H delay circuit 64 for delaying each signal by a horizontal scanning period (1H period), and a vertical linear interpolation circuit 65.

The BY / RY separation circuit 61 separates the chrominance signals BY and RY into chroma signals Cb and Cr in the image data from the camera DSP 21c, and horizontally interpolates the separated chroma signals in the horizontal direction. Transfer to circuit 62. The horizontal linear interpolation circuit 62 interpolates the luminance signal Y and the color difference signals BY and RY in the horizontal direction to lower the luminance in the horizontal direction, and the interpolated luminance signal Y and the color difference signals BY and RY. ) Is passed to the BY / RY combining circuit 63.

The BY / RY combining circuit 63 synthesizes the color difference signals BY, RY, and combines the luminance signal Y from the horizontal linear interpolation circuit 62 and the combined color difference signal BY.RY with a 1H delay circuit ( 64) and vertical linear interpolation circuit 65. The 1H delay circuit 64 delays the luminance signal Y and the color difference signal by 1H, and transfers the delayed signal to the vertical linear interpolation circuit 65. The vertical linear interpolation circuit 65 performs linear interpolation processing in the vertical direction based on the luminance signal Y and the chrominance signals BY and RY from the BY / RY combining circuit 63 and the 1H delay circuit 64. The image data including the luminance signal Y 'and the color difference signals BY' and (RY) 'whose resolution is lowered in both the horizontal and vertical directions is output.

The resolution conversion circuit 28 performs resolution conversion processing for converting [p x q] video data into [m x n] video data. The resolution converting circuit 28 performs a process of suppressing the resolution to a preset value when the image data produced by the CCD image sensor 11 is of high resolution. However, it is possible to process low resolution video data into high resolution data.

Referring to FIG. 6, the resolution converting circuit 28 horizontally buffers an input buffer 71 for storing image data input from the image data bus 33 and a video data from the input buffer 71 in a horizontal direction. Image buffers from the horizontal buffer 72, the horizontal strain processing circuit 73 for converting the resolution of the image data from the horizontal buffer 72 to the horizontal direction, and the vertical buffers of the image data from the horizontal strain processing circuit 73 A vertical buffer 74 for processing, a vertical deformation processing circuit 75 for converting the resolution of the image data in the vertical direction, and an output buffer 76 for buffering when outputting.

When the resolution of the image data is ready to be converted, the resolution conversion circuit 28 outputs a read request signal for requesting the memory controller 22 to read the image data from the image memory 32, and then after the conversion process of the image data. A write request signal for requesting the memory controller 22 to write the image data to the image memory 32 is output. Resolution conversion circuit 28 also receives an acknowledgment signal indicating that memory controller 22 responds to the request signal.

Referring to FIG. 7, the horizontal buffer 72 is composed of a first delay circuit 81, a second delay circuit 82, and a third delay circuit 83, each of which creates a delay of one pixel. . Thus, the first delay circuit 81 outputs image data delayed by one pixel, and the second and third delay circuits 82 and 83 output image data delayed by two pixels and image data delayed by three pixels, respectively. .

Referring to FIG. 7, the horizontal deformation processing circuit 73 includes first to fourth multipliers 84, 85, 86, 87 and first to third adders 88, 89, and 90. A circuit for normalizing the data is incidentally attached behind the adder 90.

The first multiplier 84 multiplies the image data supplied from the input buffer 71 by a predetermined coefficient and transfers the resulting data to the adder 88. The second multiplier 85 multiplies the image data supplied from the first delay circuit 81 by a predetermined coefficient and transfers the resulting data to the adder 88. The third multiplier 86 multiplies the image data supplied from the second delay circuit 82 by a preset coefficient, and transfers the resulting data to the adder 89. The fourth multiplier 87 multiplies the image data supplied from the third delay circuit 83 by a predetermined coefficient and transfers the resulting data to the adder 90. The first adder 88 synthesizes the image data and transfers the resulting data to the second adder 89. The second adder 89 synthesizes the image data and transfers the resulting data to the third adder 90. The third adder 90 synthesizes each image data and transfers the resulting data to the vertical buffer 73 as image data obtained by converting the resolution in the horizontal direction.

Therefore, the horizontal deformation processing circuit 73 weights a plurality of image data each having a pixel delay in a preset form to a predetermined weight, and interpolates or reduces the weighted image data in the horizontal direction. Synthesize and convert resolution in horizontal direction.

Vertical buffer 74 consists of a series connection of first to third buffers 91, 92, 93, each adapted to delay the one-line. Accordingly, the first buffer memory 91 outputs image data delayed by one line, and the second and third buffer memories 92 and 93 output image data delayed by two and three lines, respectively.

Referring to FIG. 7, the vertical deformation processing circuit 75 includes fifth to eighth multipliers 94 to 97 and fourth to sixth adders 98 to 100. The vertical deformation processing circuit 75 sometimes includes circuitry for normalizing data on the downstream side of the adder 90.

The fifth multiplier 94 multiplies the image data supplied from the horizontal direction conversion circuit 73 by a preset coefficient, and transfers the resulting data to the fourth adder 98. The sixth multiplier 95 multiplies the image data supplied from the first line memory 91 by a predetermined coefficient and transfers the resulting data to the fourth adder 98. The seventh multiplier 96 multiplies the image data supplied from the second line memory 92 by a predetermined coefficient and transfers the resulting data to the fifth adder 99. The eighth multiplier 97 multiplies the image data supplied from the third line memory 93 by a preset coefficient and transfers the resulting data to the sixth adder 100. The fourth adder 98 synthesizes the image data and transfers the resulting data to the fifth adder 99. The fifth adder 99 synthesizes the image data and transfers the resulting data to the sixth adder 100. The sixth adder 100 synthesizes each image data, and outputs the resulting data as image data whose resolution in the horizontal direction is converted.

Accordingly, the vertical deformation processing circuit 75 weights a plurality of image data each having a line delay in a predetermined form to a predetermined weight, and interpolates or reduces the weighted image data in the horizontal direction. Compositing converts the resolution in the vertical direction.

In Fig. 7, the resolution conversion circuit 28 first performs resolution conversion in the horizontal direction followed by resolution conversion in the vertical direction. However, it is possible for the resolution conversion circuit 28 to perform resolution conversion in the vertical direction followed by conversion in the horizontal direction. That is, the resolution converting circuit 28 supplies the image data from the input buffer 71 to the vertical buffer 74, the vertical buffer 74, the vertical distortion processing circuit 75, the horizontal buffer 72, And the processing in the horizontal deformation processing circuit 73 in order.

In the above-described embodiment, the first to third buffer memories 91 to 93 in the vertical buffer 74 are configured to store 1-line (1H) image data. Alternatively, the first to third buffer memories 91 to 93 may be configured to store image data of one line or less as shown in FIG. 9. At this time, the memory controller 22 needs to read the image data stored in the image memory 32 every N pixels, as shown in FIG.

Specifically, the memory controller 22 reads out pixel data corresponding to the observation screen stored in the image memory 32 every N pixels based on the line in the vertical direction. Referring to FIG. 11, each observation screen is composed of pxq pixels, the coordinates of the upper left pixel are (1,1), the coordinates of the upper right pixel are (p, 1), and the coordinates of the lower left pixel are (1, q), and the coordinate of the lower right pixel is (p, q).

Referring to FIG. 12, the memory controller 22 causes N pixel image data to be read out based on lines in the horizontal direction in the order of rows 1, 2, ..., q. This causes the memory controller 22 to move from the left end to the image data corresponding to N pixels, or N xq pixels, i.e., (1,1), (1, q), (N, q), and (N, 1). Read pixel data within a defined area. This video data is hereinafter referred to as video data set 1.

The memory controller 22 then reads the image data within the range defined by (N-1,1), (N-1, q), (2N-2, q), (2N-2,1), which are This is called image data set 2 hereafter. When the memory controller 22 reads the image data set 1 and the image data set 2, this is equivalent to reading the image data of the (N-1) th column and the Nth column twice.

The reason is that the vertical deformation processing circuit 75 performs interpolation starting from the surrounding pixels, so that the pixels stored at the start end and the later end of the first to third buffer memories 91 to 93 are not processed. Because. For example, when the image data set 1 is read, the pixels N and 1 are not subjects of interpolation processed in the vertical direction. However, this pixel (N, 1) is read out when the pixel data set 2 is read out to be the subject of the interpolation process.

In a similar manner, the memory controller 22 reads N pixel image data in the horizontal direction every line so that the image data of the last two columns of the immediately preceding image data is included. This transfers the image data set in the resolution converting circuit 28.

The image data is supplied to the vertical buffer 74 in an amount corresponding to the capacities of the first to third buffers 91 to 93 based on the line. Thus, one line of image data offset is stored in each of the first to third buffer memories 91 to 93. The vertical deformation processing circuit 75 can perform resolution conversion processing in the vertical direction based on the image data from the first to third buffers 91 to 93 of the vertical buffer 74.

In the memory controller 22, the memory controller 22 reads in association with the capacity of the memory buffer even if the capacity of the buffer memory required for resolution conversion in the vertical direction does not reach one line, so that the resolution conversion circuit 28 Enables resolution conversion in the vertical direction.

Although the read overlap between the image data sets is two columns, there is a possibility that the overlap exceeds two columns or there is no overlap. Note that the present invention is applicable to image signal processing such as camera signal processing without limitation on resolution conversion.

Although the above description relates to embodiments in which the buffer memory is used for interpolation in the vertical direction, the present invention is also applicable to embodiments in which the buffer memory is used for interpolation in the horizontal direction.

That is, the resolution conversion circuit 28 can perform resolution conversion in the horizontal direction using the vertical buffer 72a composed of the buffer memory 72a having the capacity of N pixels as shown in FIG. The memory controller 22 can read N pixel image data based on columns in the order of rows 1, 2, ..., p in the vertical direction, as shown in FIG. In the meantime, the memory controller 22 needs to read the image data stored at the beginning and the end of the buffer memory twice so that these image data become the subject of the horizontal interpolation processing as in the above-described vertical interpolation processing.

Therefore, the memory controller 22 performs image data from the image memory 32 so that resolution conversion processing in the vertical and horizontal directions is performed for the first to third buffer memories 91 to 93, each having a capacity of N pixels. Can be read. This can cause the circuit scale of the horizontal buffer 72 and the vertical buffer 74 to be reduced to lower the manufacturing cost.

The NTSC / PAL encoder 23 that performs encoding as described above also has a simplified resolution converting circuit 23a to increase the resolution of the image data if necessary before the encoding process.

The simplified resolution conversion circuit 23a performs resolution conversion so as to conform to the display standard of the finder 36 when the image data on the image memory 32 is lower than the resolution required for display.

Referring to FIG. 15, the simplified resolution converting circuit 23a includes a line memory 101 for storing image data from the image data bus 33, and a vertical linear interpolation circuit for interpolating image data in the vertical direction (V−). Directional linear interpolation circuit 102, and horizontal directional interpolation circuit 103.

The line memory 101 stores the image data from the input terminal in in an amount corresponding to a line to transmit the image data to the V-direction linear interpolation circuit 102 in the order in which it is stored. The V-direction linear interpolation circuit 102 weights the image data from the V-direction linear interpolation circuit 102 and the image data from the input terminal in with a preset weight to perform linear interpolation in the vertical direction. The horizontal interpolation circuit 103 interpolates Y with an order-seven filter and interpolates Cb and Cr with a third order filter. This is simply interpolation to increase the resolution by a factor of two. The horizontal interpolation circuit 103 outputs image data at an output terminal out.

For example, the image data input from the input terminal in is a, the image data read from the line memory 101 is b, the weighting factor is g (where 0 ≦ g ≦ 1), and also V-direction linear interpolation. If the image data output by the circuit 102 is c, the V-direction linear interpolation circuit 102 performs the following processing:

c = g * a + (1-g) * b

The image data output by the output terminal out is encoded by the NTSC / PAL encoder 23 as described above.

In the signal processing system, the digital still camera 1 is composed of two chips, namely the signal processor 20 and the CPU 41. Therefore, since each signal processing circuit is a configuration of each chip, the substrate surface area and power consumption may be smaller than when each signal processing circuit is a separate chip configuration.

Further, since the signal processor 20 is not a chip configuration including the CPU, the signal processing can be adaptively executed even if the application associated with the CPU 41 changes. That is, if the signal processor 20 is a chip configuration including a CPU, it is impossible to reconfigure the chip when the application of the CPU changes. However, the signal processor 20 can execute preset signal processing using a CPU which is an optimal structure based on the application.

The digital still camera 1 having the above-described structure includes a finder mode for confirming the position or state of the subject before recording the image, a recording mode for capturing the image of the confirmed subject, and a playback mode for confirming the photographed state of the subject image. Process is executed according to the prevailing mode.

In the finder mode, the user must observe the state of the subject shown in the finder 36 before pressing the shutter button not shown to photograph the subject. In this finder mode, the memory controller 22 and other circuits are controlled in the following manner. In order to explain each mode, reference is mainly made to FIG. 4 and sometimes to FIG. 16.

In the finder mode, the CCD image sensor 11 generates an image signal that is thinned one third from the vertical component and sends the digitalized image data to the CCD interface 21a via the S / HA / D circuit 12. Supply.

The CCD interface 21a performs signal processing in synchronization with the clock shown in FIG. 16A. Specifically, the CCD interface 21a reduces the horizontal component of the image data supplied by the image generating unit 10 by one third, gamma corrects the processed image data, and converts the corrected data into a camera DSP. 21c). The CCD interface 21a supplies the camera DSP 21c with image data converted to 340 × 256 from the 1/3 reduction process.

The camera DSP 21c performs data conversion processing on the reduced-processed video data into YCrCb video data. The camera DSP 21c changes the resolution of the image data in the simplified resolution conversion circuit 21d so as to lower the resolution of the image data (340 x 256 → 320 x 240), and converts the converted image data into the image data bus 33. Transfer to the memory controller 22 through.

The simplified resolution converting circuit 21d reduces the resolution to the contents necessary for subsequent processing in a simplified form. In this manner, if the image data generated by the CCD image sensor 11 is a high resolution, the transmission range taken by the image data generated by the CCD image sensor 11 is used to maintain the real-time characteristics of the finder mode. 33) can be reduced to avoid congestion.

The memory controller 22 reads the image data from the image memory 32 and transfers the read image data to the NTSC / PAL encoder 23 via the image data bus 33 as shown in FIG. 16D. Is recorded in the image memory 32. At the same time, the memory controller 22 reads OSD data stored in the image memory 32 as shown in Fig. 16E, and transfers OSD data stored in the image memory 32 as shown in Fig. 16E. Fig. 16F shows a transfer state on the image data bus 33 that enables the real time processing described above.

The NTSC / PAL encoder 23 performs resolution conversion of 320 x 240 → 640 x 240 or 320 x 240 → 640 x 288 in the case of an NTSC system or a PAL system, respectively, and converts the converted image data into an NTSC / PAL encoder 23. To pass. The NTSC / PAL encoder 23 also converts the image data into OSD data transmitted to the finder 36 shown in FIG. 3 which is data of an NTSC system or a PAL system. This allows the image of the subject, title information, etc. to be displayed on the finder 36 in real time.

In the meantime, the NTSC / PAL encoder 23 increases the resolution of the low resolution data, for example when 320 x 200 image data is supplied, this is 640 x 240 image data and 640 for NTSC system and PAL system, respectively. x 288 The resolution is converted so as to be converted into image data.

In the digital still camera 1, the resolution of the image data generated by the CCD image sensor 11 is lowered in a simplified form in the finder mode to reduce the amount of data, so that the image data has a bandwidth limit value of the image data bus 33. And the resolution is increased at the output stage with the content required for display at the timing shown in FIG. 16F.

Thus, in the digital still camera, the image data is kept within the bandwidth limit of the image data bus 33 to allow the image of the subject to be displayed in the finder 36 without having to execute a time consuming reduction process even at a high resolution.

A circuit for preferential processing, i.e., the CCD interface 21a, the camera DSP 21c, or the NTSC / PAL encoder 23 is set in advance in the CPU 41, and the signal processing is timed in another circuit as in the circuit. When executed separately, the signal processing of each circuit having a high priority can be executed preferentially in accordance with the data amount of the image data.

When the data amount of the image data is large in the simplified resolution converting circuit 21d, the data processing can be executed at a high processing speed in order to give priority to the real time processing so that the image quality deteriorates to some extent under the control of the CPU 41. In this way, even if the data amount of the image data generated by the image generating unit 10 is large, the high speed processing can be executed in the finder mode.

In the case of the digital still camera 1 having the electronic zoom function, the CPU 41 can control each circuit in the following manner.

The memory controller 22 causes the image data supplied via the CCD interface 21a and the camera DSP 21c to be recorded in the image memory 32, and the image data is read out from the image memory 32 so that the resolution converting circuit 28 ). The resolution converting circuit 28 formulates the image data enlarged in a part of the input image by the electronic zoom function, and outputs the resulting image to the image memory 32. This video data is read from the video memory 32 and output to the finder 36 through the NTSC / PAL encoder 23. This produces electronically zoomed image data.

Since the finder mode gives the highest priority to the real-time characteristics, no time consuming processing is executed by each circuit. However, the CPU 41 allows to execute various processing operations if the circuits different from the memory controller 22 are within the range allowed by the transfer area of the image data bus 33.

For example, the memory controller 22 reads out the image data from the image memory 32 in which the image data supplied from the CCD interface 21a is stored, and reads the read image data through the image data bus 33 through NTSC / PAL. It is configured to supply to the encoder 23 and also to the JPEG encoder / decoder 29. The finder 36 displays the image of the subject in real time, and the JPEG encoder / decoder 29 compresses the image data according to the JPEG system.

The JPEG encoder / decoder 29 compresses / expands still images, but cannot process high pixel images in real time. Thus, the JPEG encoder / decoder 29 slices a portion of the image so that the resolution is reduced through compression or a preset number of frames (frames or fields) of the image data supplied from the image data bus 33 through compression. It is possible to reduce the number of This makes it possible to continuously shoot frame-reduced still images or to continuously shoot low resolution images.

The user observes the state of the subject displayed in the finder 36 in the finder mode described above. When it is determined that the subject is to be photographed, the user presses a shutter button not shown.

When the shutter button is pressed, the digital still camera 1 advances to the recording mode. In the recording mode, the CPU 41 controls the memory controller 22 and respective circuits in the following manner to record the image of the photographed subject to the recording device 51.

The CCD image sensor 11 stops the reduction operation at the same time as the shutter is pressed, generates an XGA format image signal, and transfers the digitalized image data to the CCD interface 21a through the S / HA / D circuit 12. .

The CCD interface 21a transfers the image data supplied from the S / H-A / D circuit 12 to the memory controller 22 via the memory data bus 33 rather than the camera DSP 21c. The memory controller 22 first records the image data in the image memory 32, and then reads the image data and transfers the read image data to the camera DSP 21c via the image data bus 33. The camera DSP 21c converts image data composed of RGB into image data composed of Y, Cb, and Cr.

The video data once recorded in the video memory 32 is supplied to the camera DSP 21c. That is, the camera DSP 21c performs data conversion on the image data from the image memory 32 instead of the image data supplied directly from the CCD interface 21a. Thus, although the camera DSP 21c does not need to perform high-speed data conversion, it is sufficient that the camera DSP 21c executes this processing when the video data bus 33 is not complicated. In other words, since the camera DSP 21c does not need to execute the processing in real time, the data conversion processing can be executed by giving priority to the image quality higher than the high processing speed, and the resulting converted video is the video data bus 33. It is communicated to the memory controller 22 via. The memory controller 22 causes the image data to be recorded in the image memory 32.

The memory controller 22 causes the image data to be read from the image memory 32 and the read image data to be passed to the JPEG encoder / decoder 29. The JPEG encoder / decoder 29 compresses the image data according to the JPEG system, and records the compressed image data in the recording device 51 shown in FIG.

If real time processing is not necessary, such as during recording, the CPU 41 preliminarily records the image data in the image memory 32 so that the transfer band of the image data bus 33 is used for processing the high pixel image. Allow the set process to run.

The CPU 41 directly records the image data in the XGA format to the recording device 51 in the recording mode. However, it is possible to convert the resolution of the image data before the resolution converting circuit 28 writes the image data to the recording device 51. In particular, resolution conversion circuitry 28 is associated with VGA to allow the JPEG encoder / decoder 29 to compress the image data and to write the compressed data to the recording device 51 via image memory ( It is possible to convert the resolution of the image data read out from (32) (1024 x 768? 640 x 480).

If it is desired to confirm the captured image after the image capturing, the operator presses a play button, not shown, to reproduce the captured image.

When the play button is pressed, the digital still camera 1 is moved to the play mode. In the reproduction mode, the CPU 41 controls each circuit in the following manner to read the image data of the subject.

That is, if it is detected that the play button is pressed, the CPU 41 reads the image data from the storage device 51 and the image read before passing the data to the JPEG encoder / decoder 29 via the CPU bus 34. The data is temporarily stored in the DRAM 42. The JPEG encoder / decoder 29 expands the image data read out from the recording device 51 according to the JPEG system to produce the image data in the XGA format, and stores the resulting image data through the image data bus 33 through the memory controller. To 22.

The memory controller 22 writes the image data to the image memory 32, reads the image data from the image memory 32, and transfers the read image data to the resolution conversion circuit 28 through the image data bus 33. do.

The resolution conversion circuit 28 performs resolution conversion so that the image data conforms to the VGA format (1024 x 768 → 640 x 480 in the NTSC system and 1024 x 768 → 640 x 576 in the PAL system), and converts the converted image data into an image. Transfers to memory controller 22 via data bus 33. The image data is then read from the image memory 32 and sent to the finder 36 through the NTSC / PAL encoder 23. This displays the image corresponding to the image data recorded in the recording device 51 in the finder 36.

That is, since the image data recorded in the recording device 51 has a high resolution, the CPU 41 first lowers the resolution and then transfers the image data to the finder 36.

In addition, the CPU 41 sets priorities of the circuits that are preferentially processed for each of the finder mode, the recording mode, and the reproducing mode, and causes the associated circuits to execute the processing according to the priority of moving to one mode. It is possible. This can allow the signal processing of the image data to be effectively executed in accordance with the processing contents in each mode.

In the above-described embodiment, it is assumed that the data being processed is the same image data as the XGA. It should be noted that the present invention is not limited to this embodiment but may be applied to, for example, processing of image data composed of one million pixels.

According to the imaging device of the present invention, in performing resolution conversion, the first resolution converting means lowers the resolution of the image data from the imaging means, and the second resolution converting means is supplied from the first resolution converting means via the image data bus. By increasing the resolution of the image data, the occupancy ratio of the image data on the image data bus can be reduced, and the subject image can be displayed in real time on the display means.

Claims (15)

Imaging means for generating image data corresponding to imaging rays from a subject; First resolution converting means for lowering the resolution of the image data from the imaging means by resolution converting; Second resolution converting means for raising the resolution of the image data supplied from said first resolution converting means via a video data bus by means of resolution converting; And And output means for outputting image data from said second resolution converting means to a display means. The method of claim 1, And display means for displaying an image corresponding to the imaging data from said output means. The method of claim 1, And said first resolution converting means performs resolution conversion for reducing the occupancy ratio of video data on said video data bus. The method of claim 1, And said second resolution converting means performs resolution conversion corresponding to a display standard on said display means. Imaging means for generating image data corresponding to imaging rays from a subject; First resolution converting means for lowering the resolution of the image data from the imaging means by resolution converting; Second resolution converting means for raising the resolution of the image data supplied from said first resolution converting means via the image data bus by means of resolution converting; Storage means for storing image data; Third resolution converting means for converting the resolution of the image data supplied from said storage means; And And output means for outputting image data from said second or third resolution converting means to a display means. The method of claim 5, And display means for displaying an image corresponding to the imaging data from said output means. The method of claim 5, And said first resolution converting means performs resolution conversion for reducing the occupancy ratio of video data on said video data bus. The method of claim 5, And the second and third resolution converting means perform resolution conversion corresponding to a display standard on the display means. The method of claim 5, And the speed of resolution conversion by the first and second resolution converting means is relatively faster than the speed of the third resolution converting means. The method of claim 5, The imaging device has a plurality of operating modes, And the first and second resolution converting means or the third resolution converting means are selectively used according to the operation mode. Imaging means for generating image data corresponding to imaging rays from a subject; Recording / reproducing means for recording the video data on the recording medium and reproducing the video data recorded on the recording medium; First resolution converting means for performing resolution converting to lower the resolution of video data from said imaging means; Second resolution converting means for performing resolution converting to raise the resolution of the image data supplied from said first resolution converting means via an image data bus; Storage means for storing image data; Third resolution converting means for performing resolution converting of the image data supplied from said storage means; And And display means for displaying an image corresponding to the image data from said second or third resolution converting means. The method of claim 11, And the first resolution converting means executes a resolution converting to lower the occupancy ratio of the video data on the video data bus. The method of claim 11, And the second and third resolution converting means perform resolution conversion corresponding to a display standard on the display means. The method of claim 11, And the speed of resolution conversion by said first and second resolution converting means is relatively faster than the speed of said third resolution converting means. The method of claim 11, The imaging apparatus has a finder mode for displaying image data on the display means, a recording mode for recording image data on the recording medium, and a reproduction mode for reproducing image data from the recording medium, If the mode is a finder mode, the first and second resolution converting means are used, And the third resolution converting means is used if the mode is a recording or reproducing mode.