JP4056511B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus that processes an image signal in real time to detect the presence of an object and performs image recognition.

The image recognition technique is a technique for detecting the presence of an object from an image captured by a CCD camera or the like and analyzing the image content.
In ITS (Intelligent Transport Systems), image recognition technology is used for detecting vehicle stops on roads, detecting falling objects, and the like.

In addition, it is used in a wide range of fields, such as detecting short-circuit / ground faults in power facilities, and detecting fires in garbage incinerators.
On the other hand, as an algorithm for performing image recognition, for example, a background image is stored, a background difference for detecting an object from a difference from an image input at a fixed period, and a feature for detecting an object by extracting feature items from the image There are various other algorithms such as extraction.

  In general, in the image recognition technique, when the contents of these algorithms are large, the processing cannot be performed by one image processor. Therefore, after thinning out in units of frames, pipeline processing is performed in units of frames to increase the processing speed.

  In order to increase the processing speed, the processor simultaneously performs parallel processing for image data input / output and image recognition processing. Further, the image processing algorithm divides an image into several parts and is usually subjected to distributed processing.

  However, in the conventional pipeline control as described above, since the processing results are continuously obtained while causing the time delay by a plurality of processors before the unprocessed data is converted into the processed data, the delay time is increased. appear.

Since this delay time increases in proportion to the number of connected stages of the processor, it becomes a problem when an early event is detected from an image that moves at high speed.
For example, in unattended automatic driving without a driver being developed by ITS, a fallen object or an abnormal situation must be detected immediately after an event occurs in order to prevent an accident. Thus, high-speed detection is indispensable for in-vehicle image recognition.

  Furthermore, in the conventional algorithm distributed processing, since a serial communication port with low data transfer capability is used when combining images after distributed processing, real-time (1/30 second) combining is not possible, and high-speed detection is possible. It was an obstacle.

  The present invention has been made in view of these points, and an object thereof is to provide an image processing apparatus that improves the image processing speed and detects the presence of an object at high speed.

In the present invention, in order to solve the above-described problem, in the image processing apparatus 1 that processes an image signal as shown in FIG. 1 in the case where the frame is thinned out during the frame processing of the image signal, possess a line interval expanding means 11 which extends the interval, and a plurality of image processing means 12a~12n for performing line processing line interval is extended pipeline control, the interval of the image signal line, 1 The time interval from the time when the input of the image signal in units of lines is completed until the time when the input of the next image signal in units of one line is started, and the line interval expansion means 11 is the number of frames thinned out. Is n (n = 1, 2,...), The time interval between the lines is increased by the time corresponding to n lines, and the image signal line interval is expanded. Processing device 1 It is provided.

Here, the line interval extension means 11, when the frame during the frame processing of the image signal is decimated, extend the interval of the image signal line. The image processing units 12a to 12n perform line processing of lines with extended intervals by pipeline control. Also, the line interval of the image signal is a time interval from the time when the input of the image signal in units of one line is completed until the time when the input of the image signal in the next unit of one line is started. When the number of thinned frames is n (n = 1, 2,...), The means 11 sets the time interval between the lines by a time corresponding to n lines, and sets the lines of the image signal. Extend the interval.

In the image processing apparatus of the present invention, when the number of thinned frames is n (n = 1, 2,...) During frame processing of an image signal, the time interval between the lines is set to n lines. The interval between the lines is extended and the line processing is performed by pipeline control. Thereby, it is possible to improve the image processing speed and to detect the presence of the object at high speed.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a principle diagram of the image processing apparatus according to the first embodiment. The image processing apparatus 1 processes an image signal in real time (1/30 second), detects the presence of an object, and performs image recognition.

  When the frame processing of the image signal is performed, the line interval extending unit 11 extends the line interval of the image signal by the number of thinned frames when the frame is thinned out for real-time processing.

For example, in the figure, two frames are thinned out at the time of frame processing, and in that case, the line interval is extended by two lines.
The image processing units 12a to 12n perform line processing of lines with extended intervals by pipeline control.

  Here, the meaning of frame processing, line processing, and thinning will be described. The frame processing is performed on the entire image of one frame (one screen), and can be used for image processing having complicated features and detection in a state where the environment changes.

Examples of the algorithm of the image processing means that performs frame processing include feature extraction and labeling.
The line processing is basically performed on the image signal of one horizontal line (640 pixels) of the screen. The processing target pixel is one pixel unit, several peripheral pixel units, one line unit, and several neighboring line units. Etc.

Even in these small unit processes, detection by line processing is sufficiently possible if the environmental conditions are not changed so much in a simple image.
Also, the algorithm of the image processing means for performing line processing includes mask, background difference, projection, binarization and the like.

  The thinning is a thinning of image signals input every 1/30 seconds from a CCD camera or the like in units of one frame. Although the amount of information is reduced by thinning, there is no problem even if thinning is performed when a low-speed moving image has a high S / N ratio or when it may take time to detect.

  Next, the operation will be described. If each frame processing of the image processing means 12a to 12n takes 1/30 × 3 seconds, it is necessary to input an image signal obtained by thinning out two frames in order to perform frame processing in real time.

  Therefore, when pipeline processing of frame processing is performed by the image processing means 12a to 12n, 1/30 × 3 × n (n: number of image processing means 12a to 12n) seconds are processed before one frame is processed. Time delay.

  On the other hand, the line interval extending means 11 extends the line interval of the frame (in this case, two lines). Then, the line extended by the image processing means 12a to 12n is input to perform pipeline control in units of lines. As a result, the processing for each line can be completed within 1/30 × 3 seconds before one line is processed.

  In this way, for simple images that do not change much in environmental conditions, it is possible to sufficiently detect the object by line processing instead of frame processing. By performing pipeline control, the processing time can be greatly reduced, and high-speed detection of an object becomes possible.

Next, the time sequence of the waveform in which the line interval is extended by the line interval extension means 11 will be described. FIG. 2 is a diagram showing a time sequence of a line interval extended waveform.
The waveform La is a waveform having a normal line configuration. The waveform Lb is a line waveform in which the line interval is extended, and the interval for two lines is extended.

FIG. 3 is a diagram showing a time sequence of the frame waveform when the line interval is extended.
The waveform Fa is a waveform having a normal frame configuration. The waveform Fb is a frame waveform when the line interval is expanded by two lines, and is expanded to three times the original frame. In this case, the vertical blanking period is also expanded three times.

  Next, conventional image processing that performs pipeline control of frame processing after frame thinning will be described with reference to FIGS. 4 is a diagram showing an image processing configuration when frame processing is performed, and FIG. 5 is a diagram showing a time sequence of FIG.

  In FIG. 4, the input image signal is subjected to frame thinning by the frame thinning unit 101. Then, when image processing is performed by the image processing means 12a to 12e and an object is detected, a detection notification signal W1k that notifies the fact is output from the detection output unit 102.

  As image processing here, the image processing means 12a to 12e perform image recognition by performing masking, background difference, projection, binarization, and feature extraction, respectively. The frame processing time of each of the image processing units 12a to 12e is 1/30 × 3 seconds.

  In FIG. 5, the waveform W1a is a vertical synchronization signal. The waveform W1b is an event occurrence signal, and it is assumed that an event has occurred at the position shown in the figure (the interval from the event occurrence to the next frame is 0.03 seconds). Waveform W1c is a frame. A waveform W1d is a vertical synchronizing signal after thinning. Waveform W1e is frames f1 to f5 after thinning.

The frames f1 to f5 are sequentially subjected to image processing by pipeline control in the image processing means 12a to 12e.
Waveforms W1f to W1j are output signals of the image processing means 12a to 12e, respectively. When multi-stage processing of the pipeline of the frame is performed as shown in the figure, the delay time is (1/30) × 3 × 5 + 0.03≈530 mS until the detection notification signal W1k is output.

  Next, image processing for performing pipeline control of line processing after extending the line interval will be described with reference to FIGS. FIG. 6 is a diagram showing an image processing configuration when line processing is performed, and FIG. 7 is a diagram showing a time sequence of FIG.

  In FIG. 6, the line interval of the input image signal is extended by the line interval extension means 11. Then, when image processing is performed by the image processing means 12a to 12e and an object is detected, a detection notification signal W2k for notifying that is output from the detection output unit 102. The contents of the image processing here are the same as those in FIG.

  In FIG. 7, waveform W2c is an event occurrence signal, and it is assumed that an event has occurred at the position shown in the figure. A waveform W2d is a vertical synchronization signal after the line interval is expanded. A waveform W2e is frames F1 to F5 in which the line interval is extended.

The extended lines of the frames F1 to F5 are sequentially subjected to image processing by pipeline control in the image processing means 12a to 12e.
Waveforms W2f to W2j are output signals of the image processing means 12a to 12e, respectively. When multi-stage processing of the pipeline of the line is performed as shown in the figure, a delay time of (1/30) × 3 + 0.03≈130 ms is required until the detection notification signal W2k is output. In this way, processing time can be significantly reduced by performing line processing with expanded line spacing by pipeline control.

Next, a case where line interval expansion processing is performed after an image signal is stored in the frame memory will be described. FIG. 8 is a diagram showing a frame memory when line interval expansion is performed.
The frame memory 11a has a double buffer structure in which writing and reading are alternately performed with respect to the A side and the B side. After the image signal is temporarily stored in units of frames, the frame memory 11a outputs it while expanding the line interval.

For this reason, for example, a delay of one frame occurs between the time when frame A is written and the time when the frame A is read, and the delay corresponding to the extension of the line interval is added to the delay.
Therefore, the delay time when the line interval is expanded after being stored in the frame memory 11a is 1/30 (one frame time) + (1 + number of line interval extensions) / 30 (seconds).

  Next, the case where the line interval expansion process is performed after the image signal is accumulated in a first-in first-out (FIFO) of one frame will be described. FIG. 9 is a diagram showing an internal data memory when line interval expansion is performed.

  In place of the frame memory 11a shown in FIG. 8, an internal data memory 123a for one frame is provided, and the image signals input to the internal data memory 123a are sequentially output while being delayed by the time of the line interval expansion.

  Since there is no need to temporarily store in an external frame memory, there is no delay for one frame. Therefore, the delay time when the line interval extension process is performed after accumulating in the internal data memory 123a is (1 + number of line interval extensions) / 30 (seconds).

  Next, a case where line processing and frame processing are combined will be described with reference to FIGS. FIG. 10 is a diagram showing an image processing configuration when performing line processing and frame processing in combination, and FIG. 11 is a diagram showing a time sequence of FIG.

  In FIG. 10, the line interval of the input image signal is extended by the line interval extension means 11. Then, when image processing is performed by the image processing means 12a to 12e and an object is detected, a detection notification signal W3k for notifying that is output from the detection output unit 102.

  In this image processing, line processing includes masking, background difference and projection of the image processing units 12a to 12c, and binarization and feature extraction of the image processing units 12d and 12e are performed by frame processing. .

  In FIG. 11, waveform W3c is an event occurrence signal, and it is assumed that an event has occurred at the position shown in the figure. A waveform W3d is a vertical synchronization signal after the line interval is expanded. A waveform W3e is frames F1 to F5 in which the line interval is extended.

  The extended lines of the frames F1 to F5 are sequentially subjected to image processing by pipeline control in the image processing means 12a to 12e. Waveforms W3f to W3j are output signals of the image processing means 12a to 12e, respectively.

Here, the waveforms W3f to W3h are line processing, and W3i and W3j are frame processing.
Therefore, when pipeline multi-stage processing is performed as shown in the figure, a delay time of (1/30) × 3 × 3 + 0.03≈330 mS is required until the detection notification signal W3k is output.

  In this way, it is possible to efficiently perform image processing by combining algorithms such as masks and background differences suitable for line processing and algorithms such as feature extraction suitable for frame processing.

  As described above, the image processing apparatus 1 according to the first embodiment has a configuration in which line processing is performed by pipeline control by expanding the line interval of the image signal. Thereby, it is possible to improve the image processing speed and to detect the presence of the object at high speed.

  Next, an image processing apparatus according to a second embodiment will be described. FIG. 12 is a principle diagram of the image processing apparatus according to the second embodiment. The image processing apparatus 2 including the components shown in the drawing excluding the host 200 performs simultaneous parallel processing of image input processing, image processing, and image output processing.

The input image signal storage means 21 stores the input image signal and outputs it as input image data D1.
The image signal processing means 22 performs image processing on the input image data D1 to generate image processing data D2.

The output image signal storage means 23 stores the image processing data D2 and outputs it as output image data D3.
The storage area control means 24 has an input side storage area part 24-1 and an output side storage area part 24-2. The input-side storage area 24-1 is connected to the input image signal storage means 21 by the input bus Bi, and when one side stores the input image data D1, the other side stores the stored input image data D1 as an image signal. This is a double buffer memory that is output to the processing means 22. Also, storage control of setting data D4 and notification data D5 described later is performed.

In the figure, the memory A stores the input image data D1, and the memory B outputs the stored input image data D1. After switching, the processes are opposite to each other.
The output-side storage area unit 24-2 is connected to the output image signal storage means 23 by the output bus Bo, and when one stores the image processing data D2, the other stores the stored image processing data D2. This is a double buffer memory that outputs to the output image signal storage means 23.

In the figure, the memory C stores the image processing data D2, and the memory D outputs the stored image processing data D2. After switching, the processes are opposite to each other.
Further, the data switching control unit 25 performs switching control of the input image data D1 transmitted to the image signal processing unit 22 or the setting data D4 from the host 200 such as a workstation. Further, transmission control of the notification data D5 from the image signal processing means 22 to the host 200 is performed.

  The setting data D4 corresponds to, for example, a program or mask data. Further, the notification data D5 corresponds to detection result notification data or the like in which an object is detected by the image signal processing means 22.

  Next, a specific configuration and operation of the image processing apparatus 2 will be described. FIG. 13 is a diagram illustrating a configuration of the image processing apparatus 2. The image signal processing means 22 and the storage area control means 24 are included in one DSP (digital signal processor) 20.

The CPU 22 a corresponds to the image signal processing means 22, and the internal data memory 24 a, HPI (host port interface) 24 b and I / F 24 c correspond to the storage area control means 24.
The input FIFO 21 a and the input FIFO control unit 21 b correspond to the input image signal storage unit 21, and the output FIFO 23 a and the output FIFO control unit 23 b correspond to the output image signal storage unit 23. Further, the data switching control means 25 is composed of a gate circuit 25a.

The internal data memory 24a of the DSP 20 has a multi-bank configuration (memory A to D) and can be accessed independently.
Here, the memories A and B are alternately switched as image input line memories to input and output image line signals.

  That is, when the memory A receives and stores an image line signal via the HPI 24b, the memory B outputs the image line signal to the CPU 22a, and the CPU 22a performs image processing. By performing such an operation alternately in the memory A / B, the input of the image line signal and the image processing can be performed in parallel.

  In the memory C / D, when the memory C inputs and stores image processing data, the memory D outputs the image processing data to the output FIFO 23a via the I / F 24c. Such an operation is performed alternately for the memory C / D.

Further, the input FIFO control unit 21b inputs a horizontal synchronization signal serving as a reference for operation in units of lines to the DSP 20 as an interrupt signal.
In the above description, the memories A / B and C / D are memories having a double buffer structure in units of lines. However, frame processing may be performed by using memories having a double buffer structure in units of frames.

Next, the flow of the image signal will be described. The image signal is transmitted to the input FIFO 21a, the gate circuit 25a, the HPI 24b in the DSP 20, and the internal data memory A / B.
The output of the image signal is transmitted to the internal data memory C / D of the DSP 20 and the output FIFO 23a.

  Here, the image signal processing is performed by the CPU 22a in the DSP 20 reading out the image signal from one of the internal data memories A / B (accessing a memory different from the memory to which the image signal is input), and as a result. Are stored in the internal data memory C / D (accessing a memory different from the memory similarly output).

Next, the operation will be described in detail. The input FIFO 21a receives line-by-line image signals line by line.
When the input FIFO 21a receives one line, the input FIFO controller 21b raises an interrupt to the DSP 20, and transmits to the internal data memory A / B via the HPI 24b under the control of the input FIFO controller 21b. . The internal data memory A / B is designated by the transfer address generated by the input FIFO control unit 21b.

  While the input image data is being transferred to the internal data memory A, the CPU 22a performs image processing with the internal data memory B, and at the next line input timing, the input image data is transferred to the internal data memory B. Image processing is performed with the data memory A.

  The image processing result is stored in the internal data memory C or D (access to a memory different from the memory that is always output). The output from the output FIFO 23a is output to the image bus while controlling the timing of the output image data D3 with the pixel clock.

  As described above, the image processing apparatus 2 according to the second embodiment is connected via the input bus Bi and controls storage of the input image data D1, and is connected via the output bus Bo and controls storage of the image processing data D2. The storage area control means 24 is used to perform image processing. As a result, since the input / output buses can be separated, it is possible to eliminate the occurrence of conflicts and improve the quality and reliability of the apparatus.

  Next, an image processing apparatus according to a third embodiment will be described. FIG. 14 is a principle diagram of the image processing apparatus according to the third embodiment. The image processing apparatus 3 including the constituent elements in the drawing excluding the host 200 performs simultaneous parallel processing of image input processing, image processing, and image output processing.

The input image signal storage means 31 stores the input image signal and outputs it as time-division as input image data D1.
The image signal processing means 32 performs image processing on the input image data D1 in a time division manner to generate image processing data D2, and outputs the image processing data D2.

The output image signal storage means 33 stores the image processing data D2 and externally outputs it as output image data D3.
When one of the input image data storage means 36 stores the input image data D1, the other has a double buffer structure for outputting the stored input image data D1 to the image signal processing means 32.

  The input / output interface unit 37 is an input interface for input image data D1 from the input image data storage unit 36 to the image signal processing unit 32, and an output interface for image processing data D2 from the image signal processing unit 32 to the output image signal storage unit 33. I do.

  Further, the data switching control means 35 performs switching control of the input image data D1 transmitted to the image signal processing means 32 or the setting data D4 from the host 200 such as a workstation. Further, transmission control of the notification data D5 from the image signal processing means 32 to the host 200 is performed.

  Next, a specific configuration and operation of the image processing apparatus 3 will be described. FIG. 15 is a diagram showing the configuration of the image processing apparatus 3. The DSP 30 includes a CPU 32a corresponding to the image signal processing means 32 and does not have the internal data memory 24a described with reference to FIG. The frame memory A / B 36a corresponds to the input image data storage means 36.

  The input FIFO 31 a and the input FIFO control unit 31 b correspond to the input image signal storage unit 31, and the output FIFO 33 a and the output FIFO control unit 33 b correspond to the output image signal storage unit 33. Further, the data switching control means 35 is composed of a gate circuit 35a.

  In the third embodiment, instead of the internal data memory 24a, the frame memory A / B 36a that can be double-buffered in units of frames is changed to perform parallel operations in units of frames.

  In this case, since there is a possibility of conflict in the input / output interface means 37, the transfer speed of the image input / output data is increased 2 to 3 times, and the conflict is avoided by transferring in time division.

  For example, the timing is programmed so that the image input is transferred in the first 1/3 of the frame time, the DSP 30 performs image processing in the next 1/3, and is output to the output FIFO 33a in the last 1/3 time. And make it work.

  As described above, in the third embodiment, the DSP 30 that does not have the internal data memory 24a is operated in a time-sharing manner, so that it can be operated at high speed without causing a conflict in the interface unit. .

  Next, a fourth embodiment will be described. FIG. 16 is a principle diagram of an image processing apparatus according to the fourth embodiment. When the image processing speed does not meet the required performance, the image processing apparatus 4 performs image signal division processing and synthesis to increase the processing speed.

The image signal dividing unit 41 divides the image signal. Here, two systems are divided.
The first divided processing signal generation means 42a performs image processing on one of the image signals divided into two systems to generate a first divided processing signal Da.

The second divided processing signal generation means 42b performs image processing on the other of the two image signals divided into two systems to generate a second divided processing signal Db.
When one of the first divided processing signal storage means 43a stores the first divided processing signal Da, the other one outputs the first divided processing signal Da that has been stored (frame memory A1). / B1).

  When one of the second divided processing signal storage means 43b stores the second divided processing signal Db, the other outputs the second divided processing signal Db stored in the other (frame memory A2). / B2).

  The format conversion means 44 performs format conversion by synthesizing the output signals Da1 and Db1 output from the first and second divided processing signal storage means 43a and 43b with hardware.

Next, conventional image signal division processing and synthesis will be described. FIG. 17 is a diagram showing conventional image signal division processing and synthesis.
The image division signals divided into the two systems are subjected to image processing by the image processing units 420a and 420b.

  The combining unit 440 combines and outputs the signal processed by the image processing unit 420a and the signal processed by the image processing unit 420b and transmitted via the 1-bit serial communication port P.

  As described above, since the conventional synthesis processing is performed using the 1-bit serial communication port P, the data transfer capability is low, and real-time image processing cannot be performed.

Next, the first and second divided processing signal storage means 43a and 43b and the format conversion means 44 will be described.
As shown in FIG. 16, the frame memories A1 / B1 are alternately switched to input / output the first divided processing signal Da.

  For example, when the frame memory A1 receives and stores the first divided processing signal Da, the frame memory B1 outputs the stored first divided processing signal Da to the format conversion means 44. Such an operation is performed alternately on the frame memories A1 / B1.

Similarly, the frame memories A2 / B2 are alternately switched to input / output the second divided processing signal Db.
For example, when the frame memory A2 receives and stores the first divided processing signal Db, the frame memory B2 outputs the stored first divided processing signal Db to the format conversion means 44. Such an operation is performed alternately on the frame memories A2 / B2.

  The format conversion means 44 performs format conversion by synthesizing the output signals Da1 and Db1 with hardware. This is because if a DSP chip that performs image processing synthesizes an image signal, the DSP chip is burdened and the processing speed is reduced.

  In this way, the first and second divided processing signals Da and Db are respectively stored in the first and second divided processing signal storage means 43a and 43b having a double buffer structure, and the output signals Da1 and Db1 are combined and formatted. The conversion is performed by hardware. This makes it possible to perform format conversion at the time of image composition at high speed.

  Next, format conversion will be described with reference to FIGS. When the divided image signals are combined, it is necessary to perform format conversion corresponding to the division pattern to restore the original image.

FIG. 18 is a diagram showing format conversion when division is not performed. When division is not performed, the line arrangement of the output signal Da1 is as shown in the figure.
In the following description, one frame consists of 480 lines, and the line numbers of odd fields are enclosed in braces and the line numbers of even fields are enclosed in parentheses.

  The output signal Da1 from the first division processing signal storage means 43a outputs the even line after outputting the odd line. In this case, the output signal Db1 from the second divided processing signal storage means 43b is ignored.

  FIG. 19 is a diagram showing format conversion in the case of division for each line. When the odd field is divided for each line and then the even field is divided for each line, the line arrangement of the output signals Da1 and Db1 is as shown in the figure.

The output signal Da1 from the first divided processing signal storage unit 43a and the output signal Db1 from the second divided processing signal storage unit 43b are alternately output for each line.
FIG. 20 is a diagram showing format conversion when dividing into odd / even fields. The output signal Da1 from the first divided processing signal storage means 43a is output, and the output signal Db1 from the second divided processing signal storage means 43b is output and synthesized.

  FIG. 21 is a diagram showing format conversion when one frame screen is divided vertically. After the odd lines of the output signals Da1 and Db1 from the first and second divided processing signal storage means 43a and 43b are output, the even lines are output.

  Note that only one of the output signals Da1 and Db1 may be output as it is without performing the format conversion as described above. That is, output signals Da1 and Db1 from either one of the first and second divided processing signal storage means 43a and 43b are output for each line.

  As described above, the image processing apparatus 4 according to the fourth embodiment stores the two divided processed signals in the first and second divided processed signal storage means 43a and 43b having a double buffer structure, and outputs them. The signals Da1 and Db1 are synthesized and format converted by hardware. As a result, format conversion at the time of image synthesis can be performed in real time.

  Next, an image processing board to which the image processing apparatus is applied will be described with reference to FIGS. FIG. 22 is a diagram showing a video unit of the image processing board, and FIG. 23 is a diagram showing an image processing unit of the image processing board. The image processing board 100 includes a video unit 110 and an image processing unit 120.

For the video unit 110, the video decoder 111a performs A / D conversion of the video input, and the video encoder 111b performs D / A conversion, and outputs a processed signal.
The frame memory 113 alternately stores the digital image data thinned out by the frame thinning unit 112 for two screens.

  The mode selection unit 114 selects three types of modes for outputting the stored image data from the frame memory 113 to the image bus B1: normal output every 1/30 seconds / decimation output per frame / line interval extended output. And output.

The division processing signal storage unit 115 alternately accumulates image data of two systems through the image buses B2 and B3 in the frame memories A1 / B1 and A2 / B2 of two screens.
The format conversion unit 116 selects and converts the format for each line from the image data output from the division processing signal storage unit 115.

The video control unit 117 performs overall control of each processing unit constituting the video unit 110.
The input FIFO 121a inputs one line of image data from the image bus B4 to the image processing unit 120.

The input FIFO control unit 121b controls the input FIFO 121a and issues a horizontal / vertical synchronization interrupt to the DSP 123.
The output FIFO 122a inputs one line of processed image data and outputs it to the image bus B5. The output FIFO control unit 122b controls the output FIFO 122a.

The DSP 123 includes a plurality of banks of internal data memories 123a and HPIs 123b, and executes image processing.
The work / program memory 124 can be used as a program storage, work area, or frame memory A / B.

The gate circuit 125 switches the port for accessing the HPI 123b of the DSP 123 to the input FIFO 121a side or the host 200 side.
The DSP control unit 126 performs overall control of each processing unit included in the image processing unit 120.

  As described above, the image processing board 100 to which the image processing apparatus is applied can perform high-speed detection of an object, reduction of conflicts generated on the image bus, and high-speed image synthesis processing.

1 is a principle diagram of an image processing apparatus according to a first embodiment. It is a figure which shows the time sequence of a line space | interval expansion waveform. It is a figure which shows the time sequence of a frame waveform when a line space | interval is extended. It is a figure which shows the image processing structure in the case of performing a frame process. It is a figure which shows the time sequence of FIG. It is a figure which shows the image processing structure in the case of performing a line process. It is a figure which shows the time sequence of FIG. It is a figure which shows the frame memory at the time of performing line space | interval expansion. It is a figure which shows the internal data memory at the time of performing line space | interval expansion. It is a figure which shows the image processing structure in the case of performing combining line processing and frame processing. It is a figure which shows the time sequence of FIG. It is a principle figure of the image processing apparatus of 2nd Embodiment. It is a figure which shows the structure of an image processing apparatus. It is a principle figure of the image processing apparatus of 3rd Embodiment. It is a figure which shows the structure of an image processing apparatus. It is a principle figure of the image processing apparatus of 4th Embodiment. It is a figure which shows the division | segmentation process and the synthesis | combination of the conventional image signal. It is a figure which shows the format conversion when not dividing. It is a figure which shows the format conversion at the time of dividing | segmenting for every line. It is a figure which shows the format conversion at the time of dividing | segmenting into an odd / even field. It is a figure which shows the format conversion at the time of dividing | segmenting 1 frame screen up and down. It is a figure which shows the video part of an image processing board. It is a figure which shows the image processing part of an image processing board.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 11 Line space | interval expansion means 12a-12n Image processing means

Claims (4)

In an image processing apparatus that processes an image signal in real time,
When said frame during the frame processing of the image signal is decimated, and the line interval extension means extend the interval between lines of the image signal,
A plurality of image processing means for performing line processing of the line with an extended interval by pipeline control;
I have a,
The interval between the lines of the image signal is a time interval from the time when the input of the image signal in units of one line is completed to the time when the input of the image signal in units of the next one line is started,
When the number of frames thinned out is n (n = 1, 2,...), The line interval extending means leaves a time interval between the lines by a time corresponding to n lines. Extend the line spacing of the image signal,
An image processing apparatus.   2. The image processing apparatus according to claim 1, wherein the line interval expanding means stores the image signal in a frame memory having a double buffer structure, and then extends and outputs the line interval.   The image processing apparatus according to claim 1, wherein the line interval expansion unit stores the image signal in a FIFO that can store one frame, and then expands and outputs the line interval.   The image processing apparatus according to claim 1, wherein the image processing unit performs the line processing and the frame processing in combination.

Images