JPH06261288A - Moving picture processor - Google Patents

Abstract

PURPOSE:To provide the moving picture processor effectively generating a picture with an accurate refresh rate by alternately outputting the picture tempo rarily held in a temporary holding section where unprocessed picture is in exis tence. CONSTITUTION:An adjustment section 6 is synchronized with a timing signal from a timing signal generation section 1-1 of a picture input section 1 and checks whether or not processing sections 5A-5X for the picture to be displayed next complete the generation of picture. When the processing is completed, it is sent to a signal multiplexer section 3 switching the output picture to the next picture processing board. If the processing is not ended, it is not switched to the next picture processing board and the contents of a temporary holding section 9 temporarily holding the output signal from the multiplexer 3 are continuously displayed again. In this case, if the same frame is displayed two or three times in succession, it can be used instead of an unprocessed picture without so much sense of incongruity. Thus the processing efficiency can be improved without interrupting the original processing.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image processing apparatus,
In particular, when sequentially outputting moving images processed by a plurality of processing units, instead of an image that has not been processed, the image displayed immediately before is continuously displayed, and at this time, the processing unit that processed the alternative image. The present invention relates to a moving image processing apparatus that efficiently performs processing without interrupting the above operation.

[0002]

2. Description of the Related Art In recent years, it has become common to give presentations using images at presentations such as academic conferences and product announcements. I want to easily make video computer graphics, animations, and image productions. There is a request. In addition, by using parallel processing technology, pipeline processing technology, etc. to speed up and reduce the price of microprocessors, moving images can be generated on a personal computer.
It has become possible to record and reproduce moving images.

In a conventional moving image processing apparatus, especially in an apparatus in which the image processing content of its processing unit can be changed by software, the rewriting speed of one screen of a generated image, that is, the refresh rate is obtained by, for example, a video tape recorder. There is not necessarily a clear 1/30 second guarantee. Therefore, in order to guarantee this refresh rate, if image processing is not completed in 1/30 second,
Either an unprocessed image was output or a substitute image was output.

A conventional moving image processing apparatus will be described with reference to FIG. The input video signal is input to the image input unit 21,
The image signal is transmitted to the multiplexer unit 23. At this time, the image input unit 21 detects the vertical synchronization signal (VSYNC) that exists for each frame, and the control signal based on this is input to the arbitration unit 26, so that the arbitration unit 26 outputs the multiplexer unit 23 and its output. The temporary storage unit 24 of the image processing boards 20A, 20B, 20C, ... 20X for each complaint
A, 24B, 24C ... 24X are controlled to be sequentially output. In this way, the frames A, B, C, ... Of the input video signal are stored in the temporary storage units 24A, 24B, 24.
It is sequentially held in C ...

The input video signal held in the temporary storage unit 24A is stored in the mass storage device 28A via the processing unit 25A and the connection unit 27A. Other image processing board 20
The same control is performed for B, 20C ... Thus, the frames A, B, C, ... Of the input video signal are sequentially input to the mass storage devices 28A, 28B, 28C. In this way, after the input video signal is stored in each mass storage device of each image processing unit one frame at a time, the next frame is again stored in the mass storage devices 28A and 28A.
Sequentially stored in B, 28C ... In this way, the input video signal is stored in each mass storage device of each image processing board.

Next, the image processing boards 20A, 20B ...
In, the image processing is performed on the frames A, B, ... Of the input video signal. For example, when performing a process of moving an animal image of an input video signal, the image processing board 20A reads out the first stored frame A from the mass storage device 28A and passes it through the connection unit 27A and the processing unit 25A. The data is input to the temporary storage unit 24A and the processing unit 25
Based on the program, A performs a process of moving the animal image to a predetermined state, and inputs this to the temporary storage unit 24A.
This is because the other image processing boards 20B, 20C
・ ・ But also done. When the processing is completed, each processing unit outputs a processing end signal such as "1" to the arbitration unit 26, and outputs an unprocessed signal such as "0" when not processed.

The arbitration unit 26 controls the multiplexer unit 23 every 1/30 seconds by the timing signal applied from the timing signal generation unit 21-1 included in the image input unit 21, and the image processing boards 20A, 20B, 20C. The temporary storage units 24A, 24B, 24C, ... Of are sequentially taken out, and these are sequentially sent to the image output unit 22, thereby outputting the output video signal.

[0008]

By the way, in order to record the image generated by the moving image processing apparatus on the VTR or the like, it is necessary that the video is correctly set at 30 frames / sec. However, the processing units 25A, 25B, 25C ...
1 frame is 1/30 depending on the processing content.
A processing time of more than a second is required, and when the video is output at 30 frames / second, an unprocessed image is also output.

In order to prevent this, the previous image is continuously output instead of the image that has not been processed. For example, the image processing boards 20A, 20B, 20C
.. When the images A, B, C, D, ... Processed by are sequentially output, and if the image processing by the image processing board 20C is unprocessed, the image B immediately before that is output instead of C. ,
The arbitration unit 26 outputs the images A, B, B, and D.
Is working.

Therefore, the image processing board 20B continuously outputs the image B twice, so that the temporary storage unit 2
In 4B, it is necessary to hold the image B until the output control of the image D is started. That is, when the processing of the image processing board 20C is completed within the specified time, the image B is output once and then the next image from the mass storage device 28B is immediately stored and the next processing can be performed. When the image B is continuously output twice in this way because the processing is not completed, the processing of the next image is started after the second image B is output,
Since the start of processing is delayed compared to the case where the processing is completed, this may be adversely affected, and the case where the processing itself is not completed may occur. In other words, due to this, the original processing is interrupted, and there is a problem in that processing efficiency decreases.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a moving image processing apparatus capable of efficiently generating an image at an accurate refresh rate.

[0012]

In order to achieve the above object, the present invention provides a multiplexer unit 3 as shown in FIG.
The temporary holding unit 9 is connected to. And multiplexer unit 3
The output signal from is configured to be temporarily held in this.

Image processing boards 10A, 10B, 10C
.. when the processing units 5A, 5B, 5C ... Of complete the processing at the refresh rate, the arbitration unit 6 causes the multiplexer unit 3 to temporarily store images 4A, 4B, 4C.
.. are sequentially held in the temporary holding unit 9 and sequentially output from the image output unit 2 as output video signals.

However, for example, when the image processing board 10C has not completed the processing within the refresh rate, the multiplexer unit 3 outputs the image signals A and B, and then the multiplexer unit 3 sends the image signal to the temporary holding unit 9. Is not sent. However, at this time, the arbitration unit 6 sends the image B held in the temporary holding unit 9 again to the image output unit 2, so that image signals A, B, B ... Are output.

In the image processing board 10B, the processing for the next image can be started immediately after outputting the image B for which the processing is completed from the temporary storage section 4B. As in the example, the drawback of slowing the start of the next process is remedied.

Hereinafter, FIG. 1 will be briefly described. The image input section 1 is for inputting a color video signal that can be displayed on a display, and is for converting this analog input video signal into a time-series digital signal. ), A break (vertical synchronization signal VSYNC) or the like of each frame is detected.

The image output unit 2 converts the time-series digital signal transmitted from the temporary holding unit 9 into a video signal and converts the digital image into a color video signal which can be displayed on a display.

The multiplexer unit 3 sequentially inputs the time-series digital signals to the temporary storage units 4A to 4X during image processing according to the frame break signal output from the image input unit 1, that is, VSYNC, and during moving image generation. A sync signal with VSYNC output from the timing signal generator 1-1 (hereinafter, this and the actual vertical sync signal will be referred to as VSYNC).
C)), the time series digital signals are sequentially output from the temporary storage units 4A to 4X.

The temporary storage units 4A to 4X are for inputting a time-series digital signal to hold information and outputting the held information as a time-series digital signal. The processing units 5A to 5X perform numerical calculation processing for executing animation, computer graphics, image processing, etc., and in the case of moving image generation, the temporary storage units 4A to 4X.
An image is drawn on X, and when image processing is performed, temporary storage units 4A to 4A
Process the image input to X. Also, when playing a video,
The processing units 5A to 5X read the data from the mass storage devices 8A to 8X through the connection units 7A to 7X and store the data in the temporary storage unit 4.
Write to A to 4X, and if recording video, processing section 5A to 5X
Reads the contents of the temporary storage units 4A to 4X, and the connection unit 7A
Write to mass storage devices 8A-8X through ~ 7X.
It should be noted that the processing units 5A to 5X have the temporary storage units 4A to 4X of the self-image processing units 10A to 10X from the multiplexer unit 3.
Get notified if is selected.

The arbitration unit 6 receives the processing completion notification from the processing units 5A to 5X and controls the multiplexer unit 3 in accordance with the notification. The connection units 7A to 7X include temporary storage units 4A to 4X, processing units 5A to 5X, and a large capacity storage device 8A to.
The electrical connection of the connection with 8X is taken.

The mass storage devices 8A to 8X are storage devices capable of writing and reading a large amount of digital data. The temporary holding unit 9 temporarily holds the images output from the temporary storage units 4A to 4X, and outputs the substitute image from the temporary holding unit 9 when there is a processing unit that has not been processed. As a result, the temporary storage unit does not need to undesirably hold an image for which processing has been completed for a long time, and the next image processing can be performed without interrupting the work of the processing unit. In addition, when inputting an image, even if there is a shift in the time axis direction of the frame synchronization of the input image, that is, even if the VTR tape on the input side extends slightly and VSYNC is input at an interval slightly larger than the 1/30 second interval, it is temporarily held. When the data is transferred from the unit 9 to the temporary storage unit, it is buffered in units of VSYNC.
It is possible to correct the time lag at the time of input.

[0022]

In the present invention, the arbitration unit 6 synchronizes with the timing signal (VSYNC) from the timing signal generation unit 1-1 in the image input unit 1, and the processing unit for the image to be displayed next finishes the image generation. Check whether or not
If the processing is completed, a signal for switching the output image to the next image processing board is sent to the multiplexer unit 3.

If the processing is not completed, the contents of the temporary holding unit 9 are continuously displayed again without switching to the next image processing board. In this case, generally, in a moving image, even if the same frame is displayed two or three times, there is not much discomfort, so that an unprocessed image can be substituted.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT An embodiment of the present invention will be described with reference to FIGS. 2 is a configuration diagram of an embodiment of the present invention, FIG. 3 is an operation explanatory diagram of an arbitration unit in the present invention, and FIG. 4 is an operation explanatory diagram of the present invention.

In the figure, the same symbols as in FIG. 1 indicate the same parts. 1 is an image input unit, 2 is an image output unit, 3 is a multiplexer unit,
4A to 4X are temporary storage units, 5A to 5X are processing units, 6 is an arbitration unit, 7A to 7X are connection units, 8A to 8X are mass storage devices, 9 is a temporary storage unit, and 10A to 10X are image processing boards. is there.

The image input unit 1 has an NTSC-RGB converter 1A and an A / D converter 1B in addition to the timing signal generator 1-1. NTSC-RGB converter 1A is NTS
Since the video signal input as the C signal cannot be processed by the computer as the NTSC signal is, it is first converted into a time series RGB analog signal. And A /
The D converter 1B converts this time series RGB analog signal into a time series digital signal that can be processed by a computer. Further, the A / D converter 1B has a timing signal generation section 1-1 and outputs VSYNC.

The image output unit 2 is an RGB-NTSC converter 2
It has an A and D / A converter 2B. The RGB-NTSC converter 2A converts a time series RGB analog signal into a video signal of an NTSC signal, and the D / A converter 2B converts a time series digital signal into a time series RGB analog signal.

The multiplexer section 3 selectively inputs or outputs the time-series digital signal to or from one of the temporary storage sections 4A to 4X, and has, for example, three-state bidirectional buffers BA, BB, BC ... BX. It is composed of an IC and the like.

The temporary storage units 4A to 4X temporarily store information of time series digital signals transmitted to or received from the multiplexer unit 3, and are composed of, for example, dual port memories. The capacity of the temporary storage units 4A to 4X is required to hold the video data of two frames.

The processing units 5A to 5X perform arithmetic processing. During image processing, the information in the temporary storage units 4A to 4X is read out to perform processing such as noise removal, and at the time of image generation, they are stored in the temporary storage units 4A to 4X. Draw computer graphics etc. In addition, data is read / written from the mass storage devices 8A to 8X for recording / playback of moving images.

The arbitration unit 6 monitors whether or not the processes of the processing units 5A to 5X have been completed, and controls the output permission signal of the multiplexer unit 3 and is composed of, for example, an IC logic. The operation of the arbitration unit 6 will be described later.

The connection units 7A-7X are temporary storage units 4A-4.
X, which serves to electrically match the connection between the processing units 5A to 5X and the mass storage devices 8A to 8X, and is configured by, for example, a SCSI interface LSI.

The mass storage devices 8A to 8X can also hold, for example, the input video signal received by the image input unit 1 in units of frames, record and reproduce the results processed by the processing units 5A to 5X. And is composed of, for example, a hard disk unit.

The temporary holding unit 9 temporarily holds the output images from the temporary storage units 4A to 4X, substitutes the image immediately before the output image when the processing is not completed, and outputs the input image from the image input unit 1. The data is temporarily held, corrected to a correct frame period and transferred to the temporary storage units 4A to 4X, as will be described later, and is composed of, for example, a dual port memory. The capacity is required to hold one frame of video data.

The image processing boards 10A to 10X are for performing image processing on a frame-by-frame basis, and the temporary storage units 4A to 4X and the processing units 5A to 5X have the same alphabetic symbols, respectively.
5X, connection units 7A to 7X, mass storage devices 8A to 8X, and the like.

Next, the operation of the arbitration unit 6 in the present invention will be described. In the moving image processing apparatus of the present invention, when the image processing in the processing unit is unprocessed, A, the image immediately before that is output and the unprocessed one is skipped and the next one is displayed ( Example of FIG. 3 (A) and FIG. 4 (A))
And B, the previous image is displayed until the unprocessed one is completed, and is displayed after the completion (FIG. 3 (B), FIG.
(Example of (B)).

A. First, as shown in FIGS. 3 (A) and 4 (A), description will be given of a case of displaying the unprocessed items. That is, as shown in the display image of FIG. 4A, when the image processing board C has not completed processing, the image B is displayed twice consecutively, the image C is not displayed, and the next image D is displayed. Here is an example. In FIG. 4, boards A to D are image processing boards 1.
0A to 10D are shown.

The arbitration unit 6 includes a timing signal generation unit 1-
When VSYNC is applied from 1, the state of the processing signal from the processing unit 5 which is processing the image (Nth image) to be output at that time is read and it is determined whether the processing is completed.

If this processing signal is processed, the temporary storage unit and the temporary storage unit 9 processing the Nth image.
The multiplexer unit 3 is controlled so as to connect. As a result, the N-th image is held in the temporary holding unit 9, and the N-th image is simultaneously sent to the image output unit 2 to output the N-th image as a display image. Then, in the arbitration unit 6, a built-in counter (not shown) is incremented by 1
Then, the display output control for the (N + 1) th image is performed next.

If the processing signal indicates that the processing is not completed, the arbitration unit 6 controls the multiplexer unit 3 to disconnect the temporary holding unit 9 from the multiplexer unit 3 and to the image output unit 2. For example, the N−1th image held by the temporary holding unit 9 is output. Then mediation section 6
The counter is incremented by 1 to control the display of the N + 1th image.

This will be described with reference to FIG. 4A. The arbitration unit 6 sends an image processing board to which the processing signal has been transmitted as processing completion (omitted in FIG. 4A) when VSYNC V ₁ is transmitted. Image A from A is multiplexer unit 3
Then, it is sent to the image output unit 2 through the temporary holding unit 9 and the image A is output as a display image. And the next VSYNCV
_The output of the display image is continued until ₂ is output, and when V ₂ is output, the image processing board A creates the next frame.

When this VSYNC V ₂ is transmitted, the arbitration unit 6 checks whether the processing signal from the image processing board B indicates the completion of processing. In this case, since the processing is completed, the arbitration unit 6 causes the multiplexer unit 3 to operate.
And outputs the image B from the image processing board B to the temporary holding unit 9 and also to the image output unit 2. Thus, this time, the image B is output as the display image. Then, when V ₃ is output, the next frame creation is started.

When VSYNC V ₃ is transmitted to the arbitration unit 6, it is checked whether the processing signal from the image processing board C is in the completed state. However, since the image processing board C is in a non-processed state, the arbitration unit 6 forcibly interrupts the connection between the multiplexer unit 3 and the temporary holding unit 9. Then, since the image B held in the temporary holding unit 9 is output to the image output unit 2, the image B is continuously output as a display image. In the image processing board C, the next frame starts to be created after the forced interruption.

When VSYNC V ₄ is transmitted to the arbitration unit 6, the processing signal from the image processing board D (not shown in FIGS. 1 and 2) is checked. At this time, since the processing completion state is transmitted from the image processing board D, the arbitration unit 6
Controls connection switching so that the multiplexer unit 3 is connected to the image processing board D. As a result, this time, the image D from the image processing board D is held in the temporary holding unit 9 and simultaneously output to the image output unit 2, so that the image D is output as a display image. Then, when V ₅ is output, the image processing board D starts the next frame creation.

The state of the temporary holding unit 9 shown in FIG. 4A as “through” does not mean that the output image to the image output unit 2 is partially held by the holding unit 9. In this case, the image from the multiplexer unit 3 is not only output to the image output unit 2 but also held in the temporary holding unit 9 at the same time.

B. Next, as shown in FIG. 3 (B) and FIG. 4 (B), a case will be described in which when there is an unprocessed object, the previous image is displayed until the processing is completed. That is, as shown in the display image in FIG. 4B, when the image processing board C has not completed processing, the image C that has been processed is displayed after the image B has been displayed twice consecutively. .

The arbitration unit 6 includes a timing signal generation unit 1-
When 1 to VSYNC are applied, the state of the processing signal from the processing unit 5 which is processing the image (Nth image) to be output at that time is read, and it is determined whether or not the processing is completed.

If the processing signal is processed, the temporary storage unit and the temporary storage unit 9 processing the Nth image.
The multiplexer unit 3 is controlled so as to connect. As a result, the N-th image is held in the temporary holding unit 9, the N-th image is simultaneously sent to the image output unit 2, and the N-th image is output as a display image. A counter (not shown) built in the arbitration unit 6
Is incremented by 1 to control the display output for the N + 1th image.

If the processing signal indicates that the processing is incomplete, the arbitration unit 6 decrements the counter by -1 to N-1.
Further, the multiplexer unit 3 is controlled to disconnect the temporary holding unit 9 and the multiplexer unit 3 from each other, and the image output unit 2
, The N−1th image held by the temporary holding unit 9 is output. Then, the -1 counter of the arbitration unit 6 is incremented by 1 to control the display of the Nth image.

This will be described with reference to FIG. 4B. In the arbitration unit 6, when VSYNC V ₁ is transmitted, the processing signal is transmitted as a processing completed state (not shown in FIG. 4B). The image A from the processing board A is sent to the image output unit 2 through the multiplexer unit 3 and the temporary holding unit 9, and this is output as a display image. Next is VSYNCV
₂ Continue this display picture output until the output image processing board A when V ₂ is output to start creating the next frame.

When this V ₂ is transmitted, the arbitration unit 6
Checks whether the processing signal from the image processing board B indicates the completion of processing. In this case, since the processing is in the completed state, the arbitration unit 6 controls the connection switching of the multiplexer unit 3 and temporarily stores the image B from the image processing board B.
It is also output to the image output unit 2. Thus, this time, the image B is output as the display image. Then, when V ₃ is output, the next frame creation is started.

When VSYNC V ₃ is transmitted to the arbitration unit 6, it is checked whether the processing signal from the image processing board C indicates the completed state or the unfinished state. However, since the image processing board C is in a non-processed state, the arbitration unit 6 forcibly interrupts the connection between the multiplexer unit 3 and the temporary holding unit 9. Then, the image B held in the temporary holding unit 9 is output to the image output unit 2. As a result, the image B is continuously output as the display image. Image processing board C
Then, the process of creating the image C is continued.

When VSYNC V ₄ is transmitted to the arbitration unit 6, the processing signal from the image processing board C is checked. At this time, since the processing completion state has been transmitted from the image processing board C this time, the arbitration unit 6 controls the connection switching so that the multiplexer unit 3 is connected to the image processing board C. As a result, the image C from the image processing board C is output to the image output unit 2 at the same time as being held in the temporary holding unit 9. When VSYNC V ₅ is output, the image processing board C starts the next frame creation.

Further, the function of correcting the frame period for the input image, which the temporary holding unit 9 has, will be described. As the input image to the moving image processing apparatus of the present invention, the one temporarily held on the video tape is often used.

Strictly speaking, the input video signal input to the image input section 1 in such a case is not always input accurately every 1/30 seconds per frame. The cause is that, for example, the video tape may be elongated, and this one frame period may be slightly distorted.

For example, as shown in FIG.
VSYNC V indicates t₀, T ₁, T₂... and 1
/ Obtained from the input video signal at the correct cycle every 30 seconds
However, if the videotape is stretched,
t₁′, T₂As shown in '..., VSYNC V'
It is obtained with some deviation.

Even in such a case, in the present invention, the images A, B, ... Held in the temporary holding unit 9 are written in the temporary holding unit 9 based on this V ', and the temporary storage units 4A, 4B.
Are sequentially written in. However, this temporary storage unit 4
After being written in A, 4B, ..., an accurate 1/30 second VSYNC V from the timing signal generator 1-1.
Thus, for example, the data is written in the mass storage devices 8A, 8B, ... Therefore, at the time of image input, when there is a shift in the frame period of the input video in the time axis direction, it can be corrected by transferring from the temporary holding unit to the temporary storage unit.

If there is no such correction, the temporary holding unit 9
When the output of is executed exactly every 1/30 second,
(C) as shown in, 1 if the frame A is input to extend between, is output at time t _1, t ₁ 'image between -t ₁ is will not be sent to the temporary storage unit, the According to the invention, since the temporary storage is controlled at time t ₁ ′ by the extended VSYNC V ′, it becomes possible to hold all one frame A in the temporary storage unit.

[0059]

As described above, according to the present invention, even if there is an image that has not been processed, the image immediately before that which was held in the temporary holding section 9 is output instead. Since the image output can be continued without adversely affecting the start of creating the next frame of the board, the video can be efficiently processed in the correct frame cycle without deteriorating the processing performance. Moreover, even if the input video signal does not have the correct frame period, it can be corrected and processed in the correct period.

[Brief description of drawings]

FIG. 1 is a diagram illustrating the principle of the present invention.

FIG. 2 is a configuration diagram of an embodiment of the present invention.

FIG. 3 is an operation explanatory diagram of an arbitration unit in the present invention.

FIG. 4 is an explanatory diagram of the operation of the present invention.

FIG. 5 is a configuration diagram of a conventional example.

[Explanation of symbols]

 1 Image Input Section 2 Image Output Section 3 Multiplexer Section 4A to 4X Temporary Storage Section 5A to 5X Processing Section 6 Arbitration Section 7A to 7X Connection Section 8A to 8X Mass Storage Device 9 Temporary Storage Section

Claims (2)

[Claims] 1. An image input unit (1) for converting a color video signal displayable on a display into a digital image, a multiplexer unit (3) for switching a transfer destination or a transfer source of the digital image, and a digital image temporarily stored. Temporary storage (4A)
(4X), the processing units (5A) to (5X) that process the contents of the temporary storage units (4A) to (4X), and the execution statuses of the processing units (5A) to (5X) are monitored. An arbitration unit (6) that controls the multiplexer unit (3), mass storage devices (8A) to (8X) that can record and reproduce the processing results, and a color image signal that can display a digital image on a display. A moving image including an image output unit (2) for conversion and capable of processing a moving image in real time by switching a transfer destination or a transfer source of a digital image to the partial storage units (4A) to (4X) for each frame. In the image processing device, by connecting a temporary holding unit (9) for temporarily holding data to the multiplexer unit (3), the image immediately before being held by the temporary holding unit (9) can be substituted for an image that has not been processed. Image displayed on Moving image processing apparatus and outputs displayed continuously. 2. The shift of the frame period of the input video signal can be corrected by holding the input video signal at the time of video input in the temporary holding unit (9) and performing buffering control. The moving image processing apparatus according to item 1.