JP4704525B2 - Image signal processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image signal processing apparatus that processes an image signal picked up by an image pickup device or the like to output a moving image and a still image, and is suitable for application to a digital camera having a movie mode, for example. It relates to the device.
[0002]
[Prior art]
2. Description of the Related Art In recent years, there has been known a digital camera that captures an image of an object scene with an image sensor, processes a captured moving image or still image signal, and records the processed moving image or still image signal on an information recording medium such as a memory card or magnetic tape. In such a camera, an image sensor having a larger number of pixels has been demanded. For example, a camera using a 350,000 pixel class CCD image sensor that is almost suitable for the resolution of the screen of a display device has recently been used. A camera having an image sensor with a high pixel exceeding 800,000 pixels is also supplied and supported by users who utilize the high-definition image.
[0003]
In such a camera that processes high-resolution image data, data processing is performed by a control circuit such as an on-board CPU, for example, image data is compressed and encoded, or an image signal output from an image sensor is processed. Then, by sequentially displaying the images, the display can be used instead of the viewfinder.
[0004]
[Problems to be solved by the invention]
However, with a conventional camera, for example, when using a multi-pixel imaging device having 800,000 to 1,000,000 pixels or more, the imaging data is stored in a frame memory at a cycle of, for example, 30 frames / second, and further stored. It is difficult to read out the image data and display the moving image in real time without dropping frames.
[0005]
The control circuit for controlling the imaging process and the recording process is configured by a control system such as a microcomputer, but the data bus width tends to increase according to the processing amount. However, image data that has been picked up and output is converted into digital data of about 8 to 10 bits, for example, so that natural images and the like can be expressed in full color according to the application. When using this CPU, there was a problem that the capacity could not be fully utilized and wasted.
[0006]
It is an object of the present invention to provide an image signal processing apparatus that solves the drawbacks of the prior art and that can utilize the capability of a control system without waste and that can process and output the image signal output of an image sensor. To do.
[0007]
[Means for Solving the Problems]
In order to solve the above-mentioned problems, the present invention is an image signal processing apparatus that processes and outputs an image signal obtained by an imaging means for imaging an object scene. This apparatus is synchronized with the output of the imaging means. On the one time axis, each pixel of the image signal is converted into image data of a predetermined bit length, and the signal processing is performed. The image data is connected to the output of the imaging signal processing unit and is subjected to signal processing. Bus means for transferring data, storage means connected to the bus means for storing image data output from the imaging signal processing means, and output connected to the bus means for processing and outputting image data read from the storage means Means for outputting the processed image data on a second time axis synchronized with the output side, and a control means connected to the bus means for controlling each part of the apparatus. The means transfers the image data at a bus rate corresponding to the control means, and the imaging signal processing means converts the image data of a predetermined bit length so as to correspond to the bus width in the bus means and sends the image data to the bus means. Output at the bus rate, and the output means restores the image data transferred via the bus means to image data of a predetermined bit length, converts it into a signal format for outputting the image data, and sequentially outputs it It is characterized by doing.
[0008]
In order to solve the above-mentioned problems, the present invention is an image signal processing apparatus that processes and outputs an image signal obtained by an imaging means for imaging an object scene. The apparatus is synchronized with the output of the imaging means. An image signal processing means for converting each pixel of the image signal into image data of a predetermined bit length and processing the signal, and a bus connected to the output of the image signal processing means for transferring the image processed image data Means for storing image data output from the imaging signal processing means connected to the bus means, and output means connected to the bus means for processing and outputting image data read from the storage means. The imaging signal processing means includes a outputting means for outputting the processed image data in synchronism with the output side, and a control means for controlling each part of the apparatus connected to the bus means. The image data of the bets length, and output to the bus means by bus rate corresponding to the control means, output means, and outputs to process the image data accumulated in the accumulating unit.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of an image signal processing apparatus according to the present invention will be described in detail with reference to the accompanying drawings. Referring to FIG. 1, a digital imaging / recording apparatus 10 in this embodiment is shown. This device 10 processes an image signal captured by a high pixel density imaging device (CCD) 12 in real time, writes it to the image memory 16 on the image bus 14, and stores the image signal stored in the image memory 16. Is a camera that has a movie function that can be read to the image bus 14 and played back in real-time processing. Code data obtained by compression-coding a desired still image is detachably connected to the recording / playback processing unit 18. It is a digital camera that records information on an information recording medium 20 such as a memory card. In the following description, portions that are not directly related to the present invention are not shown and described, and the reference numerals of the signals are represented by the reference numbers of the connecting lines that appear.
[0010]
As the imaging device (CCD) 10 for imaging the object scene, a solid-state imaging device such as a CCD (Charge Coupled Device) having an effective pixel having a high pixel density of about 1.3 to 1.5 million pixels is advantageously applied. Three primary color filters are arranged on the imaging surface, and the imaging device 10 is a two-dimensional image sensor that sequentially outputs RGB color pixel signals photoelectrically converted by a photodiode at a rate of 30 frames / second. This image sensor 12 is driven by a drive signal 24 such as a reset pulse, a charge transfer pulse H1, H2, V1, V2 and a synchronization signal supplied from the imaging timing control unit 22, and stores the charge accumulated in a predetermined charge accumulation time. Transfer and read horizontally and vertically. This pixel signal forms one image of 1280 x 1024 pixels in the horizontal and vertical directions in sequence, and outputs it to an output 26 via a CDS circuit (not shown) that performs correlated double sampling to reduce various noises. Is output.
[0011]
The imaging timing control unit 22 generates various timing signals based on the reference clock, and generates a driving signal 24 that drives the imaging device in accordance with the various timings. The timing control unit 22 supplies the horizontal transfer pulse H1, the horizontal synchronization signal HD, and the vertical synchronization signal VD among the generated timing signals to the imaging signal processing unit 28. As described above, the imaging timing processing unit 22 drives each unit of the imaging system by defining the first time axis in the imaging processing system.
[0012]
The imaging signal processing unit 28 connected to the output 26 of the imaging device 12 processes the input pixel signal and outputs the processed image data to the image bus 14 having a different transfer bit width in real time. It is. A detailed configuration of the imaging signal processing unit 28 is shown in FIG. The imaging signal processing unit 28 has an imaging signal processing circuit 200 that performs analog and digital processing on the pixel signal input to the input 26 and outputs the processed signal. The image signal processing circuit 200 is not specifically shown, but includes a gain control amplifier that amplifies the pixel signal to a level corresponding to each color component, a clamp circuit that adjusts the reference level of the pixel signal, and correction that performs white compression and gamma correction. Each circuit operates in accordance with a timing signal supplied from the imaging timing control unit 22. Further, the imaging signal processing circuit 200 has an analog / digital conversion circuit for converting the image signal processed by the process circuit into 10-bit digital image data for each color component (R, G, B) and outputting the digital image data. This output constitutes the output 204 of the imaging signal processing circuit 200.
[0013]
The output 204 of the imaging signal processing circuit 200 is connected to a thinning circuit 206, and the thinning circuit 206 is packed when the image data is stored in the imaging buffer 214 connected via the packing circuit 210 or from the imaging buffer 214. This is a circuit that performs a process of thinning out predetermined pixel data when reading out image data. The thinning circuit 206 in this embodiment includes a thinning processing circuit 216 and a thinning address circuit 218. The thinning address circuit 218 further includes a frequency divider 220 and an address counter 222, and outputs a captured image in real time. If the movie mode to be displayed is set in the imaging and recording apparatus 10, thinning processing is performed. In particular, when an image sensor with a high pixel density is used, the number of pixels stored in the image memory 16 (FIG. 1) is large, and the transfer rate of data output from the image signal processor 28 exceeds the bus rate of the image bus 14. In this case, thinning processing is performed by the thinning circuit 206.
[0014]
The frequency divider 220 of the thinning address circuit 218 is a circuit that divides the transfer pulse H1 according to a thinning rate b / a (a> b) for thinning the number of pixels in one line from a pixel to b pixel. In this embodiment, the input pixel timing is divided according to the thinning rate b / a, and the address counter 222 counts the address according to the output of the frequency divider 220. The thinning processing circuit 216 performs thinning signal processing on the number of pixels in one line from a pixel to b pixel, and outputs the processed data to the packing circuit 210 connected to the output 208. As the thinning signal processing, simple thinning, weighted average thinning, or the like is used. For example, image data of 1280 pixels in the horizontal direction is thinned to 640 pixels. For vertical image data, vertical 1024 lines are thinned to, for example, 640 lines in accordance with the thinning rate b / a. When the imaging / recording device 10 is set to the image recording mode, the input image data is directly output through 1024 pixels in the horizontal direction as it is without performing the thinning process. Even in the image recording mode, the thinning-out circuit 206 may thin out the input image data, and the compression / decompression processing unit 34 may compress-encode the image whose image size is reduced.
[0015]
The packing circuit 210 connected to the thinning circuit 206 is a bus width conversion circuit that outputs input 10-bit digital image data corresponding to a 16-bit bus width. The packing circuit 210 converts the input bus width m to the output bus width n, and in this embodiment, the 10-bit data processed by the imaging signal processing circuit 200 is converted to a 16-bit width equal to the bus width of the image bus 14. It is a circuit to pack.
[0016]
As shown in FIG. 3, the packing circuit 210 includes a first input register 300 for temporarily storing image data in parallel with 10 bits at the pixel timing of the image data input in synchronization with the transfer pulse H1, and a first input register. The second input register 302 temporarily stores the output of 300 and the third input register 304 temporarily stores the output of the second input register 302. The outputs of the first to third input registers are the selector circuit. 306 input I ₁ ~ I _Ten , Input I ₁₁ ~ I ₂₀ , Input I _{twenty one} ~ I ₃₀ Are connected to each.
[0017]
The selector circuit 306 receives each input I in synchronization with the select signal 310 supplied from the packing counter 308. ₁ ~ I ₃₀ Its bus output O ₁ ~ O ₁₆ The selection circuit is connected to any one of the above. The selector circuit 306 in this embodiment has an output O ₁ ~ O ₁₆ For each of the inputs I ₁ ~ I ₃₀ It is configured to select either. Specifically, the selector circuit 306 connects its input I and output O in accordance with a select signal 310 having a value corresponding to the count value of the packing counter 308, for example, as shown in FIG.
[0018]
The packing counter 308 is a counting circuit that cyclically counts the eight cycles in synchronization with the input timing of image data corresponding to the transfer pulse H1. The counter 308 supplies the count value to the selector circuit 306 as a 3-bit select signal 310. Further, the packing counter 308 supplies a significant latch enable signal 312 to the latch circuit 314 at the timing when the count values of 1, 3, 4, 6, and 7 are obtained.
[0019]
To briefly explain the example shown in FIG. 4, the selector circuit 306 does not connect the input / output when the value of the packing counter indicated by the select signal 310 is 0, 2, and 5, for example, when the counter value is 1. 16-bit input I ₁₁ ~ I ₂₆ 16-bit output each ₁ ~ O ₁₆ Connect to. The data output at this time includes all the data held in the second input register 302 and the selector input I. _{twenty one} ~ I ₂₆ 6-bit data stored in the first register 300 corresponding to When the next image data is input to the first register 300 and the stored data of each register is transferred to the next stage circuit, and the value of the packing counter 308 is incremented from 1 to 2, at this timing, the selector circuit 306 The selection operation stops. Next, when the value of the packing counter 308 is incremented to 3 and the data held in each register is transferred to the circuit of the next stage, the selector circuit 306 receives the input I for 16 bits. ₇ ~ I _{twenty two} 16-bit output each ₁ ~ O ₁₆ Connect to. Similarly, when the counter value is 4, 6 and 7, the selector circuit 306 performs input selection so that 10-bit width data is converted to 16-bit width data. ₁ ~ O ₁₆ Is output from.
[0020]
In this way, the packing circuit 210 performs input selection that is cyclically divided by the numbers obtained by dividing the least common multiple of the input m bits and the output n bits by n and m, respectively. That is, in the case of the present embodiment in which the input is 10 bits and the output is 16 bits, an operation of periodically repeating the input selection of 5 times by the selector circuit 306 according to the value obtained by dividing the least common multiple 80 by 16 at the time of packing. Do.
[0021]
Returning to FIG. 3, the 16-bit output O of the selector circuit 306 ₁ ~ O ₁₆ The latch circuit 314 connected to is latched by a flip-flop that latches input data when a significant latch enable signal is supplied from the packing counter 308, and is in a hold state otherwise. Thus, in this embodiment, when the count value of the packing counter 308 is 1,3,4,6,7, the output O of the selector circuit 306 is output. ₁ ~ O ₁₆ Is latched and output to the output 316 of the latch circuit 314. The output 212 of the latch circuit 314 is connected to the imaging buffer 214 shown in FIG.
[0022]
As described above, in this embodiment, the input bus width of 10 bits is converted to the image bus width of 16 bits. However, the present invention is not limited to this. For example, the imaging signal processing circuit 200 that processes the imaging input A processing circuit that processes data in about 8 to 12 bits may be used, and the bus width of the image bus 14 to which the control unit 30 and the image memory 16 are connected is, for example, 8, 32 or 64 depending on the configuration of the control unit 30. It may be a bit.
[0023]
The imaging buffer 214 sequentially stores each bit of the image data output from the packing circuit 210, temporarily buffers the real-time signal imaged and processed by the imaging device 12, and matches the output-side bus cycle. It is a 16-bit double buffer that outputs at high speed. In this embodiment, the 10-bit input is adapted to the bus width of 16 bits on the control unit 30 side by the packing circuit 210 and the imaging buffer 214, and the image data thinned out by the thinning circuit 206 is further packed in the imaging buffer 214. Store. The imaging buffer 214 has two storage areas for storing image data for one horizontal scanning period, and images image data for 8 words and 10 bits processed in the imaging signal processing unit 28 as 5 words and 16 bits. Store alternately in each area of the buffer. The output 14 of the imaging buffer 214 constitutes the output of the imaging signal processing circuit 28, and is connected to a control unit (CPU) 30, an image memory 16, a peripheral control unit 32, a compression / decompression processing unit 34, a recording / playback processing unit 18, and the like. Connected to a 16-bit image bus 14. Output image data from the imaging signal processing circuit 28 is accumulated in the image memory 16 by the DMA transfer control and memory control functions of the control unit 30 and the peripheral control unit 32.
[0024]
The control unit (CPU) 30 is a processing processor that controls each unit of the imaging and recording apparatus 10. In this embodiment, a RISC (Reduced Instruction Set Computer) type microprocessor is advantageously applied. . The control unit 30 performs parallel processing on instructions (instructions) such as operations and transfers through a plurality of stages such as instruction fetch, instruction decode, instruction execution, and operation result output by pipeline processing. The control unit 30 in the present embodiment dynamically controls imaging control such as automatic exposure adjustment and automatic focus adjustment at the time of imaging, and controls writing and reading of data with respect to the image memory 16. The control unit 30 performs overall control of processing operations in the compression / decompression processing unit 34 and the recording / reproduction processing unit 18. Further, the control unit 30 transfers image data transfer control to the peripheral control unit 32, and the peripheral control unit 32 converts the image data output from the imaging signal processing unit 28 to the image bus 14 into the reproduction signal processing unit 40 and It has a DMA controller and an interrupt controller that perform DMA transfer to the communication signal processing unit 42, and further to the compression / decompression processing unit 34 and the recording / reproduction processing unit 18, whereby the bus occupation time of the control unit 30 is shortened.
[0025]
In addition, the control unit 30 has a function of controlling each unit by setting operation modes of a movie mode for monitoring a captured moving image and a recording mode for recording the captured still image on the information recording medium 20. ing. The movie mode in the present embodiment further includes a function of displaying the captured moving image on the display device 50 and a function of communicating and transferring moving image data. In the movie mode in the present embodiment, the image data of a plurality of frames are sequentially processed and output without dropping frames so that the image can be reproduced in real time according to the processing timing of the moving image output destination. In contrast, in the recording mode, the processed image data is compressed and encoded in non-real time in the present embodiment.
[0026]
The compression / decompression processing unit 34 is a processing unit that compresses and encodes image data read from the image memory 16 in non-real time regardless of the output destination timing. The compression / decompression processing unit 34 supplies the processed encoded data to the recording / reproduction processing unit 18 or the communication signal processing unit 42. Further, the compression / decompression processing unit 34 decompresses the encoded data supplied from the recording / reproduction processing unit 18 and supplies the decompressed image data to the reproduction signal processing unit 40 and the communication signal processing unit 42. The compression / decompression processing unit 34 compresses image data by, for example, a two-dimensional DCT method, assigns a Huffman code thereto, and generates encoded data having a predetermined length or less.
[0027]
The recording / reproduction processing unit 18 converts the encoded data processed by the compression / encoding processing unit 34 into a predetermined recording format for the information recording medium 20 such as a memory card and a magneto-optical disk that are detachably connected. And a processing unit that reads data recorded on the information recording medium 20 and outputs the data to the image bus 14.
[0028]
A reproduction signal processing unit 40 and a communication signal processing unit 42 are further connected to the 16-bit image bus 14. These processing units 40 and 42 input image data DMA-transferred by 16 bits, and reproduce and process the image data by 10-bit internal processing, respectively, and output them.
[0029]
Specifically, as shown in FIG. 5, the reproduction signal processing unit 40 includes a reproduction buffer 500, an unpacking circuit 502, an interpolation circuit 504, and a reproduction signal processing circuit 506. A temporary storage circuit that inputs 16-bit image data stored in the image memory 16 at a rate of 30 frames / second, temporarily buffers it, and outputs it in response to a synchronization signal supplied from the reproduction timing control unit 54 is there. The output of the reproduction buffer 500 is connected to an unpacking circuit 502.
[0030]
The unpacking circuit 502 is a bus width conversion circuit that converts the bus width from n to m when image data is stored in the reproduction buffer 500 or when image data stored in the reproduction buffer 500 is read. The unpacking circuit 502 in this embodiment restores the 16-bit input to the original 10-bit image data and outputs it to the interpolation circuit 504, for example, contrary to the packing circuit 210 shown in FIG. FIG. 6 shows the internal configuration of the unpacking circuit 502. The unpacking circuit 502 temporarily stores image data input in response to a latch enable signal 600 generated in synchronization with the horizontal synchronization signal HD in bit parallel. The first input register 602 for storing and the second input register 604 for temporarily storing the output of the first input register 602, and the outputs of the first and second input registers 602 and 604 are input to the selector circuit 606. ₁ ~ I ₁₆ , Input I ₁₇ ~ I ₃₂ Are connected to each.
[0031]
The selector circuit 606 synchronizes with the select signal 610 supplied from the unpacking counter 608 for each input I ₁ ~ I ₃₂ Its output O ₁ ~ O _Ten The selection circuit is connected to any one of the above. The selector circuit 606 in this embodiment has an output O ₁ ~ O _Ten Input I for each of ₁ ~ I ₃₂ It is configured to select either. Specifically, the selector circuit 506 connects its input I and output O according to a select signal 610 indicating the count value of the unpacking counter 608 as shown in FIG.
[0032]
The unpacking counter 608 is a counting circuit that cyclically counts eight cycles of input data in synchronization with the horizontal synchronization signal HD supplied from the reproduction timing control unit 54. The unpacking counter 608 uses the count value as a 3-bit select signal 610. To the selector circuit 606. The unpacking counter 608 makes the latch enable signal 600 significant at the timing when the count values of 0, 1, 3, 4, and 6 are obtained and supplies the latch enable signal 600 to the first and second registers 602 and 604. The first and second registers 602 and 604 do not take in the data appearing at the input at other timings, that is, the timings of the count values 2, 5, and 6 at which the latch enable signal 600 is not output, and stop the data shift to the next stage circuit. To do.
[0033]
Briefly describing the example shown in FIG. 7, the selector circuit 606 receives the input I when the significant latch enable signal is supplied to each register and the value indicated by the select signal 610 is zero. ₁₇ ~ I ₂₆ Select the output of the first register 602 for each 10-bit output O ₁ ~ O _Ten Connect to. When the next data is input to the first register 602 and the data stored in each register is shifted to the next stage circuit, the counter value is incremented from 0 to 1, and the selector circuit 606 receives the input I ₁₁ ~ I ₂₀ Select each output O ₁ ~ O _Ten Connect to. Next, the counter value is incremented to 2, the output of the latch enable signal 600 is stopped, and the data stored in each register is not shifted to the next stage circuit. _{twenty one} ~ I ₃₀ Select each output O ₁ ~ O _Ten Connect to.
[0034]
In the same manner, when the counter value of the unpacking counter 608 is 3, 4 and 6, a significant latch enable signal 600 is output, and the data stored in each register is shifted to the next stage circuit. In this case, the selector circuit 606 corresponds to each of the counter values 3, 4 and 6 with the input I ₁₅ ~ I _{twenty four} , Input I ₉ ~ I ₁₈ And input I ₁₃ ~ I _{twenty two} Select and output each ₁ ~ O _Ten Connect to. When the counter values are 5 and 7, the output of the latch enable signal 600 is stopped, and the selector circuit 606 receives the input I ₁₉ ~ I ₂₈ And input I _{twenty three} ~ I ₃₂ Select. In this way, 16-bit width image data is restored to 10-bit width image data, which is a processing unit in the reproduction signal processing unit 40, and the output O of the selector circuit 606 is output. ₁ ~ O _Ten Is output in bit parallel. In this way, the unpacking circuit 502 performs input selection that is cyclically divided by the numbers obtained by dividing the least common multiple of input n bits and output m bits by m and n, respectively. In this embodiment, since the input is 16 bits and the output is 10 bits, at the time of unpacking, the selector circuit 606 periodically selects eight inputs according to the value obtained by dividing the least common multiple 80 by 10 Repeat operation.
[0035]
The latch circuit 612 connected to the output of the selector circuit 606 has its output O ₁ ~ O _Ten Is a temporary storage circuit that outputs the image data stored and held in synchronization with the horizontal synchronization signal HD. The output 614 of the latch circuit 612 constitutes the output of the unpacking circuit 502 and is connected to the interpolation circuit 504.
[0036]
The interpolation circuit 504 is a processing circuit that performs pixel interpolation on the image data temporarily stored in the reproduction buffer 500. The interpolation circuit 504 in this embodiment includes an interpolation address circuit 514 having a frequency divider 510 and an address counter 512, and an interpolation processing circuit 516. When reading out image data from the reproduction buffer 500, or an image that has not been subjected to packing processing. Interpolation processing is performed when data is stored in the reproduction buffer 500.
[0037]
For example, the frequency divider 510 divides the horizontal synchronization signal HD supplied from the reproduction timing control unit 54 into b / a (a> b), and the address counter 512 assigns an address according to the output of the frequency divider 510. Count. When the number of pixels in one line is converted from b pixels to a pixels, the interpolation processing circuit 516 performs signal processing for pixel interpolation by, for example, pre-interpolation and weighted average interpolation, and the processed data is reproduced signal processing circuit 506. Output to. The interpolation processing circuit 516 reads out from the image memory 16 data stored by thinning the image data of 1280 pixels in the horizontal direction to 640 pixels, and interpolates the image data accumulated in the reproduction buffer 500 into the original 1280 pixels. Process and output. Also, in the vertical direction of the image, the image data of 512 lines to 1024 lines is interpolated and output. When a multi-pixel system using the image sensor 10 having a high pixel density is configured as described above, and when hardware such as a display device on the reproduction side supports multi-pixels, the interpolation circuit 504 uses b pixels. To a pixel are subjected to interpolation signal processing and output. Further, when the playback-side display device or the like is compatible with, for example, the NTSC system, image data can be output without performing this interpolation processing.
[0038]
The output 518 of the interpolation processing circuit 516 constitutes the output of the interpolation circuit 504 and is connected to the reproduction signal processing circuit 506 shown in FIG.
[0039]
The reproduction signal processing circuit 506 includes a liquid crystal monitor device 50 (FIG. 1) connected to the output 520 and a television monitor device (not shown) connected to the output terminal 52 connected to the output 520 via a video cable or the like. ) And the like to generate a reproduction signal corresponding to the signal input format of the display device.
[0040]
The reproduction signal processing circuit 506 is an output circuit that reproduces moving image data for continuously displaying the image represented by the bus width and speed-converted image data in real time. The reproduction signal processing circuit 506 converts the 10-bit digital data restored by the unpacking circuit 502 into an analog signal, and adjusts the analog signal level. The reproduction signal processing circuit 506 adds a synchronization signal to the converted and adjusted image signal, and encodes the image signal into a video signal in a format corresponding to the output destination. The reproduction signal processing circuit 506 executes these signal processes in synchronization with the timing signal 522 supplied from the reproduction timing processing unit 54. On the display device 50 connected to the output of the reproduction signal processing circuit 506, the captured moving image is displayed in real time. The display device 50 may have a digital video input function, for example. In this case, the reproduction signal processing circuit 506 is configured to generate a digital video signal in which synchronization data or the like is added to the digital data.
[0041]
The reproduction timing control unit 54 generates various timing signals based on the reference clock, and generates a drive signal that drives the reproduction signal processing unit 40 in accordance with the various timings. In addition, the reproduction timing control unit 54 supplies the horizontal synchronizing signal HD and the vertical synchronizing signal VD among the generated timing signals to the interpolation circuit 504. As described above, the reproduction timing control 54 drives each part of the reproduction signal processing circuit 506 by defining the second time axis in the image reproduction system.
[0042]
The communication signal processing unit 42 shown in FIG. 8 includes a communication buffer 800, an unpacking circuit 802, an interpolation circuit 804, and a communication signal processing unit 806, and processes image data appearing on the image bus 14 in real time. Is a processing unit for outputting. The communication buffer 800, unpacking circuit 802 and interpolation circuit 804 may have the same configuration as the reproduction buffer 500, unpacking circuit 502 and interpolation circuit 504 shown in FIG. The data is converted to bus width and speed and supplied to the communication signal processing circuit 806.
[0043]
The communication signal processing circuit 806 is an interface for processing the image data in real time and transferring it to the connection destination device. This is an output circuit for adding a correction code to packetize and outputting. The communication signal processing circuit 806 in the present embodiment is a function for transmitting image data in real time at 30 frames / second in a synchronous transfer (Isochronous) mode in which the transfer rate of image data is guaranteed and transferred according to the IEEE1394 standard. have. The output 808 of the communication signal processing circuit may be constituted by, for example, a home LAN standard CEBus or USB (Universal Serial Bus), and may be connected to a personal computer or the like. When transmitting the processed packet data via a digital transmission line, a wired or wireless digital communication device is connected to the output 808 of the communication signal processing circuit, and a full-frame moving image corresponding to the imaging signal in real time. The display is made to be performed by the receiving device at the transmission destination. The communication signal processing circuit 806 also includes data encoded by the compression / decompression processing unit 34 and encoded data. Not in Raw image data can also be output. The communication timing processing unit 56 is a processing unit that generates a timing signal such as a packet synchronization signal required when the communication signal processing unit 42 performs interpolation processing, unpacking, and packetization. The communication timing processing unit 56 in the present embodiment generates a timing signal 58 corresponding to the transmission path and transmission method and supplies the timing signal 58 to the communication signal processing circuit 42.
[0044]
With the above configuration, the image data that has been 10-bit digitally processed by the imaging signal processing unit is packed into the 16-bit image bus 14 and stored in the image memory 16 under the control of the control unit 30 and the peripheral control unit 32. At this time, unused bits are not generated on the image bus 14, and the data transfer time can be shortened by utilizing the transfer rate of the image bus 14 to the maximum. The control unit 30 can reduce the processing load and perform other processing / control on the idle time. In addition, the CPU applied to the control unit 30 reduces the processing load, so a chip with a slightly lower processing capacity can be adopted, and wasteful processing can be achieved by making maximum use of the bus width. The execution of the cycle is prevented and the power consumption is reduced during the processing operation. Further, when the image data once stored in the image memory 16 is supplied to the reproduction signal processing unit 40 or the communication signal processing unit 42 operating at the output side timing via the image bus 14, the image data is processed in each process. Since it is converted into the processing bit width of the units 40 and 42 and further outputted through each buffer, the moving image can be outputted in real time without dropping frames in real time on the reproduction side. Further, even when a multi-pixel system is constructed as in the above-described embodiment, pixel thinning is performed by the thinning circuit 206 disposed in the imaging signal processing unit 28, and the transfer bus rate of the image bus 14 is controlled. By adapting to the unit 30, a moving image corresponding to real time can be displayed on the display device 50.
[0045]
In this way, real-time moving image output is performed by synchronizing the imaging processing systems, the image reproduction and image output systems, which have different processing time axes, with the packing processing, the unpacking processing, and the buffering processing of each system. be able to. As a result, smooth moving image display reproduced in real time can be performed by fully utilizing the processing capability of the control unit 30 and the image bus 14.
[0046]
In addition, even when high-resolution still image data is compressed and encoded and recorded on the information recording medium 20 without the need for real-time moving image processing, the bus width of the image bus 14 is effectively used to store the image memory. The image data can be transferred to 16, and the processing efficiency of the control unit 20 and the like can be improved.
[0047]
In addition to such a configuration, the imaging signal processing unit 28 samples the moving image input intermittently at a certain period according to the setting, and the thinning circuit 206 further reduces, for example, a 1/6 size reduced image. May be created and stored in the image memory 16. In this case, the predetermined storage area of the image memory 16 is divided into nine blocks, and the storage destination block of the reduced image is changed in the order of blocks A to I as shown in FIG. 9, for example, and each reduced image is assigned to each block. The position is sequentially changed and stored. The plurality of reduced images stored in the image memory 16 in this manner form one screen, and sequentially read out to the reproduction signal processing unit 40 in synchronization with the reproduction timing defined by the reproduction timing processing unit 54. Then, the multi-screen as shown in the figure can be sequentially updated and displayed on the display screen 90 of the display device 50. In this embodiment, since the amount of data flowing through the image bus 14 is further reduced, the processing load on the control unit 30 and the like is reduced, and the control unit 30 does not burden other processing such as special processing and imaging / recording control. Can be controlled.
[0048]
As described above, the selector circuit 306 in the packing circuit 210 has the input I ₇ ~ I ₃₀ Each output O ₁ ~ O ₁₆ Therefore, the third register 304 may have a 4-bit width register configuration, and the input of the selector circuit 306 may have a minimum 24-bit configuration. The selector circuit 606 in the unpacking circuit 502 has an input I ₉ ~ I ₃₂ Each output O ₁ ~ O _Ten Therefore, the second register 604 may have an 8-bit configuration, and the input of the selector circuit 606 may have a minimum 24-bit configuration.
[0049]
【The invention's effect】
As described above, according to the present invention, the imaging signal processing means is driven on the first time axis synchronized with the output of the imaging means, and the processed image data is converted and transferred in accordance with the bus width of the bus means. The accumulation means accumulates this image data, and the output means processes and outputs the accumulated image data on the second time axis, so that the reproduction system and the output system are synchronized in real time. Image output can be performed. In this case, since image data conversion and restoration are performed in accordance with the bus width of the bus means, the data transfer efficiency is improved, and the burden on the control means using the bus means is reduced accordingly.
[Brief description of the drawings]
FIG. 1 is a diagram showing a block configuration of an imaging recording apparatus of an embodiment to which the present invention is applied.
FIG. 2 is a block diagram illustrating an internal configuration of an imaging signal processing unit in the embodiment illustrated in FIG. 1;
3 is a block diagram showing an internal configuration of the packing circuit shown in FIG. 2. FIG.
4 is a diagram showing the correspondence between the value of the packing counter shown in FIG. 2 and the input / output of the selector circuit. FIG.
5 is a block diagram showing an internal configuration of a reproduction signal processing unit shown in FIG.
6 is a block diagram showing an internal configuration of the unpacking circuit shown in FIG. 5. FIG.
7 is a diagram showing the correspondence between the value of the unpacking counter shown in FIG. 6 and the input / output of the selector circuit.
8 is a block diagram showing an internal configuration of a communication signal processing unit shown in FIG.
FIG. 9 is a diagram showing multi-screen playback.
[Explanation of symbols]
10 Imaging and recording device
12 Image sensor (CCD)
14 Image bus
16 Image memory
18 Recording / playback processor
20 Information recording media
22 Imaging timing controller
28 Image signal processor
30 Control unit (CPU)
40 Playback signal processor
42 Communication signal processor
50 LCD monitor device
54 Playback timing controller
56 Communication timing controller

Claims (7)

In an image signal processing apparatus that processes and outputs an image signal obtained by an imaging unit that captures an object scene, the apparatus includes:
At a first time base to synchronize the output of the imaging means, and the imaging signal processing means respectively each pixel of the image signal you converted into image data of a predetermined bit length,
Connected to the output of the image pickup signal processing means, bus means for transferring front Kiga image data,
Storage means connected to the bus means for storing image data output from the imaging signal processing means;
Output means connected to the bus means for processing and outputting the image data read from the storage means, and outputting the processed image data on a second time axis synchronized with the output side Means,
Control means connected to the bus means for controlling each part of the device,
The bus means transfers the image data at a bus rate according to the control means,
The imaging signal processing means includes a thinning means for thinning out the pixels at a predetermined ratio to convert the number of pixels of the image data, and a bus for converting into image data having a bus width corresponding to the bus width of the bus means. width converting means includes an imaging buffer means for temporarily storing the output of the bus width converting means, the image pickup signal processing means, the images data converted by該間pulling means, before SL bus to said bus means Output at a rate,
The output means is an output buffer means for temporarily storing image data read from the storage means and transferred via the bus means;
Bus width restoring means for restoring the output of the output buffer means to the image data of the predetermined bit length, and restoring the image data transferred via the bus means to the image data of the predetermined bit length, respectively. An image signal processing apparatus that converts the image data into a signal format for output and sequentially outputs the image data. The image signal processing apparatus according to claim 1 , wherein the output unit includes an interpolation unit that interpolates the pixels of the image data output from the bus width restoration unit at a predetermined ratio and converts the number of output pixels. An image signal processing apparatus comprising: 2. The image signal processing apparatus according to claim 1 , wherein the output means outputs the image data so that a moving image represented by the image data read from the storage means is displayed in real time in synchronization with the display means. An image signal processing apparatus which is a reproduction signal processing means. 2. The image signal processing apparatus according to claim 1 , wherein the output means is communication signal processing means for transmitting image data read from the storage means in real time in synchronization with another apparatus. Signal processing device. The image signal processing device according to claim 1, wherein the device includes:
An image signal processing apparatus comprising data compression means for compressing and encoding image data supplied via the bus means. The image signal processing apparatus according to claim 5 , wherein the apparatus includes:
An image signal processing apparatus comprising recording processing means for recording the encoded data processed by the data compression means on an information recording medium.   2. The image signal processing apparatus according to claim 1, wherein the control means includes transfer control means for performing control for DMA transfer of the image data, and a microprocessor for controlling each unit by pipeline processing. Image processing device.

Images