JPH0737076A - Image processing device - Google Patents

Abstract

(57) [Summary] (Correction) [Purpose] To provide an image processing device which can select an arbitrary scanning method at high speed and which does not cause unnatural output calculation results. [Structure] An interface unit 22 for inputting and outputting a video bus and image data includes two input buffers for input and output, and two input buffers.
The image processing device 21 is provided with two output buffers and is synchronized with a synchronizing signal having a frequency higher than an external synchronizing frequency. The image data of one frame input to one input buffer is read while the image data of the next one frame is input to the other input buffer, and is processed at high speed in the image processing device 21. When the video bus image data transfer is interlaced, 0 is set in the even field and 1 is set in the odd field in the address pointer, and 2 is incremented every data transfer cycle to input the image data of one line-sequential frame. Store in buffer. The processing area in the XY directions is set and controlled, and only the necessary portion is processed at high speed. Also, the processing area boundary is detected and filtering processing is performed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus.

[0002]

2. Description of the Related Art Generally, there is an image processing apparatus as shown in FIG. In the image processing apparatus shown in the figure, image data is captured by the camera 1 and once captured in the frame memory 2, and then captured in the image processing apparatus via the video bus 3b and the video bus interface unit 4. The image data taken in enters the image processing unit 7 through the three multiplexers 5 (5-1, 5-2, 5-3) and the two multiplexers 6 (6-1, 6-2). . The image data entered into the image processing unit 7 is subjected to various image processing under the control of the image processing control unit 8 and then 4
Memory planes 9 (9-1, 9-2, 9-3, 9-
It is stored in the memory plane 9 selected in 4).
The image processing apparatus executes the above-described series of processing in one scan on the video bus 3b in synchronization with the synchronization signal of the camera 1.

The four memory planes 9 are controlled by a memory controller 12 for random access and serial read / write serial access, and the stored image data are respectively stored in four multiplexers 10 (10).
-1, 10-2, 10-3, 10-4). Based on the data input from the mask plane 11, the multiplexer 10 selects the memory plane 9 into which each of the image data is to be captured, captures the output image data, and switches and outputs the captured image data. The mask plane 11 is also controlled by the memory control unit 12 for random access and serial read / write serial access. Image processing controller 8
The memory control unit 12 is connected to a CPU (Central Processing Unit) (not shown) via the system bus 3a and is controlled by the CPU.

The output of the multiplexer 10-4 is image-displayed on the CRT display device 16 via the output interface 14, the video bus 3b and the frame memory 15, while the multiplexers 10-1, 10-2 and 10 are used.
The output of -3 is the multiplexer 5 (5-1, 5) described above.
-2, 5-3). These inputs are directly input to the multiplexer 5-1 and to the multiplexers 5-2 and 5-3.
Through multiplexers 6-1 and 6-2,
Output to the three input lines A, B, C of the image processing unit 7,
The image processing unit 7 executes inter-image calculation based on the input image data.

During filtering, the image data input from the video bus 3 via the video bus interface unit 4 and the image data input from the memory plane 9 via the multiplexer 10 are input to the multiplexer 5, and the multiplexer 5 Then, the input image data can be switched. Multiplexer 5-
The image data from 1 is output to the input line A of the image processing unit 7 and also to the line buffer 13-1. The line buffer 13-1 is a circuit that delays input data by 1H (horizontal: scanning line) in the horizontal direction of the image, and its output is input to the multiplexer 6-1 and the other line buffer 13-2. . The multiplexer 6-1 receives the output of the line buffer 13-1 and the output of the multiplexer 5-2, but at the time of performing filtering, the output from the line buffer 13-1 is input to the input line B of the image processing unit 7. Output. Further, the multiplexer 6-2 receives the output of the line buffer 13-2 and the output of the multiplexer 5-3, but outputs the output from the line buffer 13-2 to the input line C of the image processing unit 7. The image processing unit 7 executes the filtering process with the image data input in parallel to the three input lines A, B, and C.

The image processing unit 7 continuously processes a plurality of image data with a constant delay period as shown in FIG. 17 for both the image data calculation and the filtering calculation, and operates as if in a pipeline. . The drawing shows the clock CK, the input data of the input line A, the input data of the input line B, the input data of the input line C, and the output data after processing, respectively from the top. In the figure, the input lines A, B, and C are synchronized with the clock CK at the time t.
0 to 1H worth of valid data 0, 1, 2, ... Are respectively input, and 1H worth of valid output data 0 which is various image processing results from time t1 after a predetermined delay period (after pipeline delay), The output of 1, 2, ... Is started.

The conventional input / output timing for one image is shown in FIG. 18 (a). FIG. 6A shows the clock CK, the input data, and the output data from the top. The left side of FIG. 3B shows the input image data obtained by reading the input data, and the right side thereof shows the output image data formed by the output data.

As shown on the left side of FIG. 1B, in the input image data, the first 1H has pixels D00, D01, ..., D0.
n, and the next 1H is the pixel D10, ...
Lines composed of +1 pixels are composed of m + 1 lines from 0 line to m line. This input image data is sequentially read from the frame memory 2 or the memory plane 9 shown in FIG. 16, and any one of the input lines A selected by the image processing unit 7 as the input data shown in FIG. Input to B or C. From the image processing unit 7, the processed image data corresponding to the input data is sequentially delayed by several clocks, and the first 1H of image data O0 is output.
0, O01, ..., O0n, and the next 1H worth of image data O10, ... And n + 1 pieces of 1H worth of image data (valid data) are output. The shaded area in Fig. 6 (a) is invalid data.

FIG. 19 shows the operation timing when this image processing is processed by the image processing apparatus shown in FIG. In the figure, the synchronization signal of the camera 1 and the video bus 3 are shown from the top.
And image data transferred from the camera 1 to the image processing apparatus via the video bus interface, image data input to the image processing unit 7, image data output from the image processing unit 7, and the memory plane 9 or the video bus 3b. This is the output timing of the image data output to. In the same figure, the signal shown by the white solid color in the period shown by the arrow is valid data, and the signal shown by the diagonal line is invalid data. All of them are synchronized with the synchronization signal of the camera 1, and the processes are sequentially performed from the top.

Further, in the above-mentioned filtering process,
Arithmetic with undefined data is performed at the boundary of the processing area of the image data. The calculation with the indefinite data in this boundary portion will be described in more detail with reference to FIG. 20 showing the input image data shown in FIG. In the image data shown in the figure, in the filtering process, for one pixel to be processed, a total of nine pixels of image data including eight vertically, horizontally and obliquely surrounding the pixel are calculated.

In the case of the pixel at the center of the image, all 8 pixels surrounding the pixel are valid data, and the calculation is performed by 9 effective pixels including the own pixel. However, as shown in the figure, the 0th scan line 17 in the top row and the mth scan line in the bottom row
Each pixel forming the pixel row 19 consisting of the first pixel and the pixel row 20 consisting of the end pixel of each scanning line in the middle of the scanning line 18, the 0th scanning line 17 and the mth scanning line 18 is at the boundary of the processing area. Therefore, there is no valid image data in the area outside of these areas, and the area is an indefinite data area shown by hatching in FIG.

That is, the 0th scan line 17 has no data of the previous scan line, and the image processing section 7 receives undefined data in the undefined data area instead of the previous scan line data.
Therefore, for the image data of each pixel on the 0th scan line 17, the indefinite data for three pixels, two pixels before and after the pixel (left and right), and three pixels on the first scan line are used. Calculation is performed. In particular, for the first pixel D00 and the last pixel D0n of the 0th scan line 17, the data for one pixel that is further adjacent in the horizontal direction and the data for one pixel that is diagonally below are undefined data, and a total of 5 pixels out of the data used for calculation Minute data is calculated in the state of indefinite data.

Similarly, the m-th scanning line 18 at the bottom has no valid subsequent scanning line data, and in this case also, the indeterminate data in the hatched area is input to the image processing unit 7 as the subsequent scanning line data. Therefore, for the image data of each pixel of the m-th scanning line 18, three pixels of the m-1th scanning line, two pixels before and after the pixel (left and right), and indeterminate data for three pixels are also included. Is calculated. Also in this case, in the first pixel Dm0 and the last pixel Dmn of the m-th scanning line 18, the data of one pixel which is further adjacent in the horizontal direction and the data of one pixel which is diagonally above are indefinite data, and the data used for the calculation is also used. Of these, a total of 5 pixels of data is calculated in the state of indefinite data.

0th scan line 17 and mth scan line 18
In each scanning line in the middle of each of the pixels forming the top pixel column 19, the data of three pixels which should be adjacent to the left and the left diagonally up and down are indefinite data, and each pixel forming the end pixel column 20 is Data for three pixels that should be adjacent to the right and diagonally to the right and below are calculated in the state of indefinite data.

[0015]

Generally, image processing is roughly divided into three steps: capturing image data, preprocessing the captured image data, and measuring and recognizing using the preprocessed image data. You can proceed with. The above-mentioned pre-processing covers a wide variety of fields such as noise removal, binarization by a threshold, thinning, distortion correction, blur recovery, and feature extraction.
Various processes are performed several times on one (1 frame) input image data.

By the way, in the conventional image processing apparatus, in order to process the image data transferred from the video bus for each screen, it is necessary to perform the processing in synchronization with the processing timing of the video bus. Therefore, only one (one type) process can be executed in response to one scan of one screen of the video bus. Therefore, if multiple types of processing are required, the still screen will be scanned and executed multiple times, so not only can the imaging screen not be processed in real time, but processing on the still screen will take too much time. There was a problem.

Further, in the case of performing image processing, in order to use the image data of the preceding and following scanning lines, the transfer of the image data by the video bus is line sequential scanning, that is, the 0th line,
It is limited to the non-interlaced method in which the image data is processed by sequentially scanning the first line, the second line, ... Therefore, there is a problem that it is inconvenient because other processing performed by the interlace system cannot be used together.

Further, in the filtering process,
As described with reference to FIG. 20, in the processing area boundary portion of the image data area, the indefinite data outside the area is calculated. Therefore, there is also a drawback that the result becomes unnatural.

An object of the present invention is to provide an image processing apparatus which can perform image processing at a high speed, can select an arbitrary scanning method, and which does not cause an unnatural output of a calculation result.

[0020]

An image processing apparatus 21 according to the invention described in claim 1 (see the block diagram of FIG. 1) inputs image data fetched from the outside or image data internally processed from the outside. A video bus interface means 22 for inputting / outputting to / from the outside in synchronization with the synchronization signal and for inputting / outputting image data fetched from the outside or image data processed inside from the inside in synchronization with the internal synchronization signal, Image processing is performed at high speed by separating the external synchronizing frequency and the internal synchronizing frequency.

The video bus interface means 22 (see the block diagram of FIG. 2) has, for example, two video input buffers 22-1 and 22-2 provided in parallel with the input section as described in claim 2. .

Further, the video bus interface means 22 (see the block diagram of FIG. 3) receives the even field image data transferred from the outside in the interlace system as described in claim 3, for example. When the vertical start addresses of the video input buffers 22-1 and 22-2 are set to "0" and the image data of the odd field is input, the video input buffers 22-1 and 22-2 are input.
When the vertical start address 2-2 is set to "1" and the image data transferred from the outside in the non-interlaced mode is input, the video input buffer 2 is used.
The start address setting means 23 for setting the vertical start addresses of 2-1 and 22-2 to "0", and the video input buffer 22- when inputting image data transferred from the outside by the interlace method 1, 22
-2, the vertical address increment is switched to "2", and when the image data transferred from the outside in the non-interlaced mode is input, the video input buffers 22-1 and 22-2 in the vertical direction are input. And an increment switching means 24 for switching the increment of the address to "1".

Further, the video bus interface means 22 (see the block diagram of FIG. 4) has, for example, two video output buffers provided in parallel with the output section as described in claim 4.

The image processing apparatus according to the fifth aspect of the present invention (see the block diagram of FIG. 5) includes the video bus interface means 22, the video input buffers 22-1 and 22-2, the start address setting means 23, and the increment switching. In addition to the means 24, an XY area setting means 25 for setting a processing area in the XY directions of the image data, and an area address setting means 26 for setting a start address of the processing area of the image data are further provided, and the area address setting means is provided. The image data is processed based on the start address set by 26 and the processing area set by the XY area setting means 25.

According to a sixth aspect of the present invention, an output circuit for outputting the input image data to the image processing circuit, and a first delay for delaying the output of the output circuit by one scanning line and outputting the delayed image data to the image processing circuit. The present invention is applied to an image processing apparatus having a circuit and a second delay circuit that further delays the output of the first delay circuit by one scanning line and outputs the delayed image to the image processing circuit. .

The image processing apparatus of the present invention (see the block diagram of FIG. 6) detects the boundary of the processing area of the image data, and the detecting means 27 detects the boundary of the processing area of the image data. At this time, fixed data output means for outputting preset fixed data in place of the output of the output circuit, the first delay circuit, or the second delay circuit according to the processing area boundary of the detected image data. 28-
1, 28-2 and 28-3.

[0027]

According to the first aspect of the present invention, the video bus interface means 22 separates the external synchronizing frequency from the internal synchronizing frequency, and the image data is processed at high speed in synchronization with the internal synchronizing signal. To be done.

For example, the video bus interface means 2
When image data is input from the outside to one of the two video input buffers 22-1 and 22-2 provided in parallel to the two input units, the image data of the other video input buffer is processed at high speed. .

Further, for example, even field image data and odd field image data which are transferred from the outside to the video bus interface means 22 by an interlace method are line-sequential 1 to the video input buffer 22-1 or 22-2. It is stored in the state of frame image data. As a result, even in the case of image data transferred by the interlace method as in the case of image data transferred by the non-interlace method, the image is externally supplied to one of the video input buffers 22-1 or 22-2. When data is being input, the image data in the other video input buffer can be processed at high speed.

Further, the video bus interface means 22
With the two video output buffers provided in parallel with the output section of, the output to the outside can be performed separately from the internal processing.
In the image processing device 21 according to the fifth aspect of the present invention, in addition to the above respective operations, a processing area in the XY directions of the image data is set, and the image data in the set area can be processed at high speed.

According to the image processing apparatus of the invention described in claim 6,
At the boundary of the processing area of the image data on which the filtering processing is performed, the data outside the processing area is replaced with the preset fixed data. This stabilizes the processing result.

[0032]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 7 is a circuit block diagram of the image processing apparatus according to the embodiment. The image processing apparatus shown in the figure includes a video bus interface unit 32 for inputting and outputting image data via a video bus 31, noise removal of the image data, binarization by a threshold, thinning, distortion correction, blur recovery, An image processing unit 33 that performs processing such as feature extraction, and a storage unit 34 that temporarily stores and outputs those image data.
Composed of.

The video bus interface section 32 comprises a data input section 32-1 and a data output section 32-2. The data input section 32-1 has two input interfaces 3 that take in image data, which is picked up by the camera 35 and temporarily stored in the frame memory 36, via the video bus 31.
2-1a and 32-1a '. Then, further, these two input interfaces 32-1a, 32
-1a ', two video input buffers 32-1b, 32-1b' for receiving and storing image data respectively,
These two video input buffers 32-1b and 32-1
The image processing unit 3 reads out image data from b '.
Two output interfaces 32-1c, 3 which output to 3
2-1c ', the above two video input buffers 32-1
b, 32-1b ', an input section buffer control section 32-1d for controlling input / output, and an address register 32-for designating start addresses of input / output data of these two video input buffers 32-1b, 32-1b'. 1e. The above two video input buffers 32-1b and 32
-1b 'is composed of VRAM (Video RAM), and the input buffer control unit 32-1d continuously reads (reads) and continuously writes (writes) from the serial port side of the video input buffers 32-1b and 32-1b'. )
It is configured to be able to.

Further, the data output section 32-2 of the video bus interface section 32 is
The configuration is the same as that of the data input section 32-1. That is, the configuration of the data output unit 32-2 has two input interfaces 32-2a and 32-2a 'to which the image data output from the storage unit 34 is input, and these two input interfaces 32-2a and 32-2a'.
Two video output buffers 32-2b and 32-2b 'to which image data are respectively input from one input interface 32-2a and 32-2a', and image data of these two video output buffers 32-2b and 32-2b ' Of two output interfaces 32-2c and 32-2 for respectively outputting the data to the frame memory 37 via the video bus 31.
c ', the above two video output buffers 32-2b, 32
-2b 'input / output control section buffer control section 32
-2d and those two video output buffers 32-2
An address register 32-2e for designating a start address of input / output data of b, 32-2b 'is provided. The two video output buffers 32-2b and 32-2b '
Each of them is also composed of a VRAM, and the output buffer control section 32-2d can control continuous reading and writing from the serial port side.
Further, the image data once captured in the frame memory 37 is output to the CRT display 38 and displayed on the screen.

Next, the storage unit 34 has four interfaces 40 (40-1, 40-2, 40-3 and 4), respectively.
0-4), the memory plane 41 (41-1, 41-2,
41-3 and 41-4), and the multiplexer 42 (4
2-1, 42-2, 42-3 and 42-4). The four interfaces 40 are respectively associated with the corresponding memory planes 41 (41-1, 41-2, 41-3).
Or 41-4), the image data input from the image processing unit 33 is output, and each of the four multiplexers 42 (42-1, 42-2, 42-3 and 42-).
The image data is output to 4). In addition, the four memory planes 41 also input the input image data into four multiplexers 42 (42-1, 42-2, 42).
-3 and 42-4). Further, the storage unit 34
Is a mask plane 43 that controls the four multiplexers 42, a memory control unit 44 that controls the input and output of the mask plane 43 and the four memory planes 41, an address register 45 that specifies the start address of the mask plane 43, and four memory planes. It is provided with four address registers 46 (46-1, 46-2, 46-3, 46-4) which respectively specify the start address of the memory plane 41. The memory control unit 44 is the system bus 4
CPU (Central Processing Unit) (not shown)
Connected to, and based on the control from that CPU,
Input / output of the mask plane 43 and the four memory planes 41 is controlled by random access or serial access of continuous read / write.

The multiplexer 42 selects the memory plane 41 based on the data input from the mask plane 43, takes in the image data output from the selected memory plane 41, and fetches the taken image data. It is output as it is or switched to another circuit.
Of the four multiplexers 42, the multiplexer 4
The image data 2-4 is stored in the video bus interface unit 3
The image data of the remaining three multiplexers 42-1, 42-2, and 42-3 is output to the image processing unit 33.

The image processing unit 33 is controlled by a CPU (not shown) via the system bus 47.
0, an image processing circuit 51 that performs various image processes under the control of the image processing control unit 50, image data input from the data input unit 32-1 of the video bus interface unit 32, and a storage unit 34. Two multiplexers 42-
Three multiplexers 52 (52-1, 52-2, 52) that output the image data input from 1, 42-2 and 42-3 as they are or by switching them to the image processing circuit 51 side.
-3), and two 1H delay circuits 5 interposed between these three multiplexers 52 and the image processing circuit 51.
3 (53-1, 53-2), three fixed data output circuits 54 (54-1, 54-2, 54-3), two multiplexers 55 (55-1, 55-2), and a boundary detection unit It is composed of 56.

The output of the multiplexer 52-1 is input as image data A to the image processing circuit 51 via the fixed data output circuit 54-1 and is also branched and input to the 1H delay circuit 53-1. The output of the multiplexer 52-2 is input to the downstream multiplexer 55-1 and the output of the multiplexer 52-3 is input to the other downstream multiplexer 55-2.

The image data branched from the multiplexer 52-1 and input to the 1H delay circuit 53-1 is 1
The data is output with a time delay corresponding to the scanning line, and the downstream multiplexer 55 passes through the fixed data output circuit 54-2.
-1 and the other 1H delay circuit 53-2
Also branch to and enter. This 1H delay circuit 53-2
The image data input to the fixed data output circuit 54-3 is output with a time delay of one scanning line.
Input to the other downstream multiplexer 55-2 via.

The image data input from the upstream multiplexer 52-2 to the multiplexer 55-1 and the image data input from the 1H delay circuit 53-1 via the fixed data output circuit 54-2 depend on the image processing mode. It is selected and input to the image processing circuit 51 as image data B. Further, the image data input from the upstream multiplexer 52-3 to the multiplexer 55-2 and the image data input from the 1H delay circuit 53-2 via the fixed data output circuit 54-3 are selected according to the image processing mode. Then, the image data C is input to the image processing circuit 51. The image processing circuit 51 processes the input image data A, B, and C and outputs the processed image data to the storage unit 34.

The boundary detecting section 5 of the image processing section 33 is
6, and the memory plane 41 (41-1, 4-1,
41-2, 41-3, and 41-4) are connected to a processing area control unit 57, which will be described later in detail, and a startup circuit 58 is connected to the processing area control unit 57.

Next, in FIGS. 8A and 8B, the data input section 32-1 in the above-mentioned video bus interface section 32 is shown.
Two video input buffers 32-1b and 32-1
b ′ and the two video output buffers 32-2b and 32-2b ′ of the data output unit 32-2, and 4 of the storage unit 34.
One memory plane 41 (41-1, 41-2, 41-
3, 41-4) and the data input / output of the VRAM forming the mask plane 43 will be described.

As shown in FIG. 7A, the VRAM used in this embodiment is composed of a memory cell array 61 of 512 × 512 × 4 bits. This is an example when the number of vertical and horizontal pixels of the image data is 512 × 512. The mask plane 43 only needs to have 4 bits for each pixel in order to specify a combination of four memory planes 41 for each pixel, and thus is composed of one memory cell array 61. Further, since one pixel of normal image data has an 8-bit configuration, the video input buffer 32-1
b, 32-1b ', video output buffer 32-2b, 3
2-2b ′, memory planes 41-1, 41-2, 41
-3 and 41-4 are both constructed by arranging two memory cell arrays 61 shown in FIG. 10A so as to overlap each other. For example, the memory cell array 6 shown in FIG.
The lower 4 bits of each pixel of 512 × 512 are assigned to 1, while the memory cell array 61 which is not visible in FIG.
The upper 4 bits of each pixel are assigned to. In the following description, the data input / output of one memory cell array 61 will be described, but the same applies to the other one when two memory cells are arranged in an overlapping manner. Further, the input / output of the image data of the memory plane or the input / output buffer will be described, but the same applies to the case of the memory plane selection data of the mask plane 43.

In FIGS. 1A and 1B, the memory cell array 61 has a SAM (sequential accumulative) in the horizontal direction (X direction).
cess method) circuit 61-1 and a row decoder 61-2 are connected in the vertical direction (Y direction). Also,
SIOs (serial I / O interfaces) 1 to SIO 4 are connected to the SAM circuit 61-1 via a switch circuit 61-3.

The SAM circuit 61-1 reads / writes image data for each pixel scanning line (horizontal direction). The row decoder 61-2 decodes the row selection signals A0 to A8 input from the memory control unit 44, and a 512-bit × (lower-order or higher-order) 4-bit memory area selected corresponding to each scanning line. (Low) is designated as the read or write destination of the image data. SIO
Reference numerals 1 to SIO4 are serial input / output interfaces for serially reading and serially writing image data, and sequentially serially SAM four-bit horizontal image data for each pixel via the switch circuit 61-3. Circuit 6
Input and output with 1-1. The switch circuit 61-3 is shown in FIG.
During the serial write operation shown in (a), the switch is set to S
ON from IO1 to SIO4 toward the memory cell array 61
Although not shown in the timing chart in particular, the switch circuit 61-3 synchronizes the image data input from the SIO1 to SIO4 with the start address of the SAM circuit 61-1 which is designated in advance, in synchronization with the serial clock. I will write it.

Then, after a predetermined image data is written, a write transfer cycle to the VRAM is executed, so that one line of image data (512 bits × 4 bits) is written in the memory cell array of the selected row address. Is transferred and written.

The switch circuit 61-3 has the same structure as that shown in FIG.
During the serial read operation shown in (1), the switches are turned on from the memory cell array 61 in the directions of SIO1 to SIO4. Then, a read transfer cycle for the VRAM is executed, so that the image data for one line from the memory cell array of the selected row address is SAM.
The circuit 61-1 is transferred, and after the transfer, the switch circuit 61
-3, the image data is sequentially read serially from the start address of the SAM circuit 61-1 specified in advance in synchronization with the serial clock, and SIO1 to SIO
4 is output. In this way, the image data is VRA
Input (output) to (from) M.

As shown in FIG. 7, the data input section 32-1 and the data output section 3 of the video bus interface section 32.
2-2 has two video input buffers 3 each.
2-1b, 32-1b 'and video output buffer 32-
2b and 32-2b 'are provided, for example, in the case of the above-mentioned data input, the image data for one scan (one frame) captured from the camera 35 is stored in the video input buffers 32-1b and 32-1b'. Stored alternately.

In one video input buffer 32-1b,
While the image data is being transferred from the frame memory 36 via the video bus 31 based on the processing frequency of the video bus 31, the other video input buffer 32-1b '
Is the image processing unit 3 regardless of the processing cycle of the video bus 31.
The image processing unit 33 converts the image data based on the processing frequency of 3
Transfer to. On the contrary, the video input buffer 32-1b '
Is inputting image data, the video input buffer 32-1b
Outputs the image data to the image processing unit 33.

As described above, the video bus interface unit 32 has two video input buffers 32-1b and 32-1.
By providing b ′, on the one hand, the image data is processed in the processing cycle of the image processing unit 33 without disturbing the input of the image data from the video bus 31, and on the other hand, regardless of the processing cycle of the video bus 31. . Therefore, during one scan of the image data of the video bus 31, the image processing unit 33
Depending on the processing cycle, the image data can be processed at a processing speed of twice the processing cycle of the video bus 31 and a processing speed of three times the processing cycle of the video bus 31. it can.

FIG. 9 shows the image processing section 3 when the processing cycle of the image processing section 33 is twice as long as the processing cycle of the video bus 31.
3 is a timing chart of No. 3; In the figure, (a) is a sync signal, (b) is image data transferred from the camera 35 to the video bus interface unit 32 of the image processing apparatus via the frame memory 36 and the video bus 31, and (c) is a video bus. Image data input from the interface unit 32 to the image processing unit 33, (d) image data output from the image processing unit 33, and (e) from the video bus interface unit 32 of the image processing apparatus to the video bus 31 and the frame. Memory 3
Image data output to the CRT display 38 via (7), (f) is the processing content within the synchronization period.

As shown in FIG. 7F, first, the image data A is transferred (input) from the video bus 31 to the video bus interface section 32 in synchronization with the synchronization signal, and then in synchronization with the next synchronization signal. The image data B is transferred (input) (see (a) and (b) in the same figure). During the transfer (input) of the image data B, the image processing unit 33 performs processing 1 and processing 2 at a processing cycle twice as long as the processing cycle input to the input image data A (see FIG. (See c)), and the processing result is output to another circuit (video bus interface unit 32 or storage unit 34) in the apparatus (see (d) in the same figure). The image data A as a result of such processing is output from the video bus interface unit 32 to the CRT display 38 via the video bus 31 and the frame memory 37 in synchronization with the next synchronization signal ((e in the figure). )). Then, the input image data B is processed in the same processing cycle (see FIG.
(See (e) and (f)).

That is, for example, the image data input from the video input buffer is processed by the image processing circuit 51, the processed image data is stored in the memory plane 41, and the stored image data is serially serially output from the memory plane 41. Read out, and the read image data is read by the image processing circuit 5
1 further processes and stores again in the memory plane 41, or the video output buffer 32-2b or 32--
A plurality of types of processing of storing in 2b 'can be executed in a time corresponding to one scan of the image data of the video bus 31.

If the processing frequency of the image processing unit 33 is set as high as possible to 3 times or 4 times as high as the processing frequency of the video bus 31, accordingly, during the transfer of one scan of image data by the video bus. It enables various kinds of data processing.

Next, the video input buffers 32-1b, 3-1
The address 2-1b 'in the vertical direction (the row address described in FIGS. 8 (a) and 8 (b)) is input to the input buffer control unit 3 in FIG.
In 2-1d, it is possible to switch to either "1" increment or "2" increment.

That is, as shown in FIG. 10A, when the image data transfer on the video bus 31 is a non-interlaced (line-sequential) system, a vertical direction address pointer built in the input buffer controller 32-1d. Is set to the initial value "0" and incremented by "1" for each data transfer cycle indicated by the vertical synchronizing signal in the figure. As a result, one frame of image data is stored in the video input buffer 32-1b or 32-1b '.

On the other hand, as shown in FIG. 7B, the image data transfer on the video bus 31 is interlaced (interlaced scanning).
In the case of the system, an even field (1 screen data composed of 0th line, 2nd line, 4th line, ...)
In the case of, the initial value “0” is set in the address pointer, and the odd field (first line, third line, fifth line,
(1 screen data composed of ...), an initial value "1" is set, and "2" is incremented for each data transfer cycle. As a result, at the time of data transfer of an even field, addresses "0", "2", "4", ...
Image data is stored in, and at the time of data transfer of the next odd field, addresses "1", "3", "5", ...
Image data is stored in, and one frame of image data is completed in the video input buffer 32-1b or 32-1b '.

As described above, not only in the case of non-interlace type data transfer but also in the case of interlace type, one frame of line-sequential image data is used as the video input buffer 32-1b (or 32-1b '). ) Can be stored. Therefore, the other video input buffer 32-1
While the image data for the next one frame by the interlace method is input (stored) to b '(or 32-1b), one input buffer 32-1b is completed.
The image data processing described above can be performed using the image data for the frames. That is, it becomes possible to perform a real-time filtering process or the like which could not be performed on the input image data of the conventional interlace system.

By the way, if the processing range of the image data is limited by focusing on the face of a person or the like, the efficiency of the above-mentioned image processing is further improved. For example, when a processing range is set in the central portion of image data for processing, for example, as shown in FIG.
As shown in FIG. 1 (a), two memory planes 41 out of the four memory planes 41 having a horizontal memory size of Xmax and a vertical memory size of Ymax, respectively.
-1, 41-2 processing range specified in the central part xi xy
Each memory area A of j is set, and the image data read out from this area is subjected to predetermined processing (calculation), and as shown in FIG.
Stored in the upper left part of 3. In this case, the processing time is "(xi
Xyj) / (Xmax * Ymax) ", and the processing speed is improved.

In this embodiment, the processing range is set by setting the four address registers 46 (46-1, 46-2, 4 and 4 in FIG. 7).
6-3, 46-4) to set the start address “(a, b)” of the pre-processing data and the start address “(0, 0)” of the post-processing data in the processing area. Size of processing area x i in one area setting register
And yj by setting.

FIG. 12 shows a circuit block diagram of the processing area control unit 57. As shown in the figure, the processing area control unit 57 includes a Y size register 57-1, a Y size counter 57-2, an X size register 57-3, an X size counter 57-4, and a delay 57-5. It The Y size register 57-1 is the system bus 4 of FIG.
The size data yj of the processing area is set by the data DATA input from the CPU via 7, and the set size data yj is output to the Y size counter 57-2 at the input timing of the ** control signal YLD. The size data xi of the processing area is set in the X size register 57-3 by the input of the data DATA, and the set size data xi is sent from the image processing control section 50 to the control signal X.
Whenever LD is input, it outputs to the X size counter 57-4. The X size counter 57-4 has a start signal * EN.
Is started, the size data xi input by the X size register 57-3 is repeatedly subtracted in synchronization with the clock CLK, and the state signal * XSTATE is output while the subtraction is repeated, and the subtraction result is " When it becomes "0", the output of the status signal * XSTATE is stopped and the carry signal CY is output to the delay 57-6. As a result, from the scan line starting from the start address “(a, b)” set in the address register 46, as shown in FIG.
The unprocessed image data sequentially transferred to the SAM circuit 61-1 shown in (a) are read by the SIO1 to SIO4 via the switch circuit 61-3 in the range from the start address "a" to the size data xi. Be done. Alternatively, the processed image data is from the start address "0" to the size data xi, and from the SIO1 to SIO4 through the switch circuit 61-3, the SAM circuit 61-shown in FIG.
Written to 1.

The delay 57-5 is for the memory cell array 6
1 (memory plane 41) to SAM circuit 61-1 or SAM circuit 61-1 to memory cell array 61
The carry signal CY that has been input is output to the Y size counter 57-2 with a delay of the timing at which the data for the line is transferred. Y size counter 57-2
Subtracts the size data yj set by the Y size register 57-1 in synchronization with the clock CLK each time the carry signal CY is input, outputs the state signal * YSTATE during this period, and the subtraction result is " When it becomes "0", the output of the status signal * YSTATE is stopped and the carry signal * YCY is output. As a result, the vertical processing range starting from the start address “(a, b)” set in the address register 46 is controlled.

FIG. 13 shows an activation unit that activates the processing area control unit 57. The activation unit 58 shown in the figure is a register, and a command signal CMD input from the image processing control unit 50.
Data DA input via the system bus 47 by
When the TA is set and the predetermined bit of the set data DATA is "1", the processing area control unit 5 is activated.
The start signal * EN is output to 7 and Y of the processing area control unit 57 is output.
The carry signal * YCY from the size counter 57-2 resets the output of the start signal * EN.

In this way, it is possible to easily process image data by designating an arbitrary processing range. In the present embodiment, at the time of filtering processing, the boundary of the processing area is detected, and fixed data is output instead of indefinite data outside the boundary. The boundary of the processing area is a status signal * YSTATE that controls the Y direction output from the processing area control unit 57 described above.
And the edge before and after the state signal * XSTATE for controlling the X direction can be easily detected.

FIGS. 14 and 15 show the timing of the filtering process. FIG. 14 shows the processing timing in the vertical direction, where (a) is the processing period and (b) is the processing area control unit 5.
7 is a state signal CNT1 (state signal * YSTATE in FIG. 12), (c) is an upper boundary detection signal CNT2 output from the boundary detection unit 56, (d) is a lower boundary detection signal CNT3, e) is the multiplexer 52-
Image data A input to the image processing circuit 51 from 1;
(f) is the 1H delay circuit 53-1 to the image processing circuit 51
Image data B and (g) input to the 1H delay circuit 5
Image data C input from 3-2 to the image processing circuit 51
Indicates.

As shown in FIG. 9A, the processing period is time t.
Image data output from the video bus and the memory plane starting at 0 is selected by the multiplexer 52-1 and the selected image data A is 1V (horizontal direction 1 line) of 1V (vertical processing range). Every time period t0
... t1, t1 to t2, ..., Tn-1 to tn are sequentially output to the image processing circuit 51. As shown in (e), (f), and (g) of the same figure, the image data of continuous three lines is input to the image processing circuit 51 with a delay of 1H period and 2H period. As shown in (a ') of the figure, the filtering process of 1V is preceded by 1H with respect to the pixel of the image data B (see (f) in the figure) that is input with a delay of 1H. Image data A (see (e) in the figure) and 1H
It is calculated with the pixels of the image data C (see FIG. 9 (g)) delayed by an amount.

Therefore, in the image data A output from the multiplexer 52-1 during the filtering process of 1V, the leading 0th line is ignored, and 1H of indefinite data is added to the nth line which is the last line. Is output. The image data C output from the 1H delay circuit 53-2 has no valid image data during the first 1H period, and indefinite data is output during this period. Also,
The final nth line is ignored.

The boundary detecting section 56 in FIG. 7 detects the falling edge of the status signal CNT1 indicating the start of processing at time t0 in FIG. 14 (a) (see FIG. 14 (b)), and 1H is detected based on this detection.
After the lapse of the minute period, that is, from the time t1, the upper boundary detection signal CNT2 is output to the fixed data output circuit 54-3 for the next 1H period (see FIG. 7C). As a result, the fixed data output circuit 54-3 causes the 1H delay circuit 5 to
The fixed data of 1H is output instead of the indefinite data of the image data C output from 3-2. In this embodiment, the fixed data is set to "0" in advance.

Further, the boundary detecting section 56 detects the rising edge of the status signal CNT1 indicating the end of processing at time tn (see FIG. 9B), and based on this detection, 1 from time tn.
During the period of H minutes, the lower boundary detection signal CNT3 is output to the fixed data output circuit 54-1 (see FIG. 7D). As a result, the fixed data output circuit 54-1 outputs 1H of fixed data “0” instead of the indefinite data of the image data A output from the multiplexer 42-1.

In this way, in the filtering process, the fixed data, the 0th line data, and the 1st line data are calculated at the upper boundary of the processing region, and the n-th line is calculated at the lower boundary of the processing region. Since the -1 line data, the nth line data, and the fixed data are calculated, stable calculation results at the upper and lower boundaries can be obtained.

Next, FIG. 15 shows the processing timing in the horizontal direction of the filtering processing, and (a) shows the processing clock C.
K, (b) is 1H pixel data D0, D1, D2, ...
, Dn, (c) is the valid period of the horizontal data, (d) is the status signal CNT4 (status signal * XSTATE in FIG. 12) output from the processing area control unit 57, and (e) is the boundary detection unit 56. The left boundary detection signals CNT5 and (f) output are the same as the right boundary detection signal CNT6, and (g) is 1 before and after.
The image data A (B and C) of 1H to which the fixed data for pixels is added and which is input to the image processing circuit 51 is shown.

The boundary detecting section 56 detects the trailing edge of the state signal CNT4 indicating the start of horizontal output ((d) in the figure).
Based on this detection, the left boundary detection signal CNT5 is output for one clock period, and the rising edge of the status signal CNT4 indicating the end of horizontal output is detected. The right boundary detection signal CNT6 is output during the clock period. These outputs are fixed data output circuits 54-1, 54-2, and 5 respectively.
4-3, and these input detection signals CNT5
And CNT6 based on the fixed data output circuit 54-
1, 54-2, and 54-3 output fixed data "0" for one pixel, respectively. As a result, the image data A, B, and C to which the fixed data “0” for one pixel is added to the left and right non-processing area data are input to the image processing circuit 51.

In this way, in the filtering process, the fixed data "0", the pixel D0, and the pixel D1 are calculated for every three lines at the left boundary of the processing region, and the pixel for every three lines is calculated at the right boundary. Dn-1, pixel Dn,
Since the calculation is performed using the fixed data “0”, stable calculation results at the left and right boundaries can be obtained.

[0074]

As described in detail above, according to the present invention,
Two video buffers for input and output are provided in parallel in the interface part of the video bus, data is alternately input and output by the two video buffers, and an external synchronization frequency and an internal synchronization frequency are separated, so that the internal high frequency is high. Image data can be processed in synchronization with a frequency synchronization signal, and therefore high-speed image data can be processed without being affected by a low external data transfer rate.

Further, the image data of the even field and the odd field transferred by the interlace system are line-sequentially 1
Since the image data of the frame is stored in the buffer, the line-sequential 1 stored in the other video buffer during data transfer by the interlace method to one of the video buffers.
The image data of the frame can be processed. Therefore, the image data can be easily filtered in real time in the case of the interlace system as well as the case of the non-interlace system.

Since an arbitrary processing area can be set in the image data, the image data can be processed by limiting the range only to the necessary portion, and therefore the processing speed can be improved by the limited range. become.

Since the boundary of the processing area of the image data is detected and fixed data is output instead of the indefinite data corresponding to the outside of the area, a stable calculation result can be obtained even at the boundary of the processing area. Natural and good processing results can be obtained.

[Brief description of drawings]

FIG. 1 is a block diagram (1) of the present invention.

FIG. 2 is a block diagram (2) of the present invention.

FIG. 3 is a block diagram (No. 3) of the present invention.

FIG. 4 is a block diagram (No. 4) of the present invention.

FIG. 5 is a block diagram (No. 5) of the present invention.

FIG. 6 is a block diagram (6) of the present invention.

FIG. 7 is a circuit block diagram of an image processing apparatus according to an embodiment.

8A and 8B are diagrams illustrating writing of image data to a VRAM and reading of image data from the VRAM.

FIG. 9 is a timing chart when the processing cycle of the image processing unit is set to be twice the processing cycle of the video bus.

10A and 10B are diagrams illustrating non-interlaced (line-sequential) system image data transfer and interlaced (interlaced scanning) system image data transfer.

FIG. 11 is a diagram illustrating a method of processing image data with a limited processing range.

FIG. 12 is a circuit block diagram of a processing area control unit.

FIG. 13 is a diagram showing an activation unit that activates a processing area control unit.

FIG. 14 is a diagram illustrating vertical processing timing of pipeline processing.

FIG. 15 is a diagram illustrating a horizontal processing timing of pipeline processing.

FIG. 16 is a diagram illustrating a conventional image processing apparatus.

FIG. 17 is a diagram illustrating a pipeline operation of a conventional filtering operation.

FIG. 18 is a diagram showing input / output timing of conventional image data, and a diagram showing input image data and output image data.

FIG. 19 is a diagram showing operation timing when processing is performed by the conventional image processing apparatus.

FIG. 20 is a diagram illustrating image data used for conventional filtering processing.

[Explanation of symbols]

21 Image Processing Device 22 Video Bus Interface Means 22-1, 22-2 Video Input Buffer 23 Start Address Setting Means 24 Increment Switching Means 25 XY Area Setting Means 26 Area Address Setting Means 27 Detecting Means 28-1, 28-2, 28 -3 Fixed Data Output Means 31 Video Bus 32 Video Bus Interface Section 32-1 Data Input Section 32-1a, 32-1a 'Input Interface 32-1b, 32-1b' Video Input Buffer 32-1c, 32-1c 'Output Interface 32-1d Input section Buffer control section 32-1e Address register 32-2 Data output section 32-2a, 32-2a 'Input interface 32-2b, 32-2b' Video output buffer 32-2c, 32-2c 'Output Interface 32- d output unit buffer control unit 32-2e address register 33 image processing unit 34 storage unit 35 camera 36 frame memory 37 frame memory 38 CRT display 40 (40-1, 40-2, 40-3, 40-4) interface 41 ( 41-1, 41-2, 41-3, 41-4) Memory plane 42 (42-1, 42-2, 42-3, 42-4) Multiplexer 43 Mask plane 44 Memory control unit 45 Address register 46 (46 -1, 46-2, 46-3, 46-4) Address register 47 System bus 50 Image processing control unit 51 Image processing circuit 52 (52-1, 52-2, 52-3) Multiplexer 53 (53-1, 53-1) 53-2) 1H delay circuit 54 (54-1, 54-2, 54-3) Fixed data output circuit 55 55-1) multiplexer 56 boundary detection unit 57 processing area controller 58 the starting circuit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI technical display location G09G 5/12 9471-5G

Claims (6)

[Claims] 1. Image data externally input or externally input / output in synchronization with a synchronizing signal input externally, and externally input image data or externally processed image data. A video bus interface means (22) for inputting / outputting data to / from the inside in synchronization with an internal synchronizing signal is provided, and image processing is performed at high speed by separating an external synchronizing frequency and an internal synchronizing frequency. Image processing device. 2. The video bus interface means (2)
2. The image processing apparatus according to claim 1, wherein 2) has two video input buffers (22-1, 22-2) provided in parallel in the input section. 3. The video bus interface means (2)
2) sets the vertical start address of the video input buffers (22-1, 22-2) to "0" when inputting the image data of the even field transferred from the outside by the interlace method. , When inputting image data of odd fields, the video input buffer (22
-1, 22-2) has a vertical start address set to "1", and when inputting image data transferred from the outside in a non-interlace system, the video input buffer (22-1, 22-2) 2) start address setting means (23) for setting the vertical start address to "0", and the video input buffer (22-1) when inputting image data transferred from the outside in the interlace system. ,
The video input buffer (22-1, 22-2) is used when inputting image data transferred from the outside in the non-interlace mode by switching the vertical address increment of 22-2) to "2". 3. The image processing apparatus according to claim 2, further comprising increment switching means (24) for switching the increment of the address in the vertical direction to "1". 4. The video bus interface means (2)
The image processing apparatus according to claim 1, 2 or 3, wherein 2) has two video output buffers (22-3, 22-4) provided in parallel in the output section. 5. An XY area setting means (25) for setting a processing area in the XY directions of the image data, and an area address setting means (26) for setting a start address of the processing area of the image data. The image data is processed based on the start address set by the area address setting means (26) and the processing area set by the XY area setting means (25). The image processing device according to 3 or 4. 6. An input circuit for inputting image data to an image processing circuit, a first delay circuit for delaying the input of this input circuit by one scanning line and inputting to the image processing circuit, and this first delay circuit. A second delay circuit for further delaying the input of 1 to one scanning line and inputting it to the image processing circuit, and detecting the boundary of the processing area of the image data in the image processing apparatus for filtering the image data. Means (2
7) and, when the boundary of the processing area of the image data is detected by the detecting means (27), the input circuit, the first delay circuit, or the second delay circuit is selected according to the detected processing area boundary of the image data. Fixed data output means (28-1, 28-2, 28-3) for inputting preset fixed data to the image processing circuit instead of the input of the delay circuit of Processing equipment.