JP3159591B2 - Image processing method and apparatus, and computer system having image processing function - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing system, and more particularly to a compression and decompression process of a moving image which can be processed by a general-purpose computer or the like.

[0002]

2. Description of the Related Art It is well known that a still image and a moving image are displayed on the same screen by using a multi-window or a Z-buffer. The moving image in this case is displayed by taking image data from a VTR or the like into a designated area of the frame memory.

[0003] On the other hand, there are needs such as compressing and storing captured moving images and reproducing them with some processing, but high-speed processing is required for compression / decompression. For example, a normal moving image has a frame rate of 30 frames / sec, and the maximum processing period possible for one screen is about 33 ms. In order to perform input and output including compression or decompression processing during this time, a high-speed compression / decompression processor and a dedicated processor for controlling the processor are required.

Recently, a processor has been put to practical use that performs high-speed processing at 30 frames / second in accordance with the JPEG specification for compression / expansion of moving images (Image Compression Processor Data Book (Third Edition), December 1191) , Kubota C-Cube IN
C.). FIG. 13 shows the configuration of the video system described in the data book.

In the case of compression, the NTSC input device 1007
, A timing generator 1009 generates write timing for a frame buffer (FIFO) 1006, and digitally converted video data is sequentially stored in the buffer 1006. At this time, the vertical synchronization period of the image is written in the status register 1011.

The CPU 1001 always reads the status register 1011.
0 starts compression. The CL 550 sequentially compresses the image of the buffer 1006 and compresses the image in the main memory 1002 or the HD 1003.
When one frame is completed, the completion report is made to the register 1011. When the CPU 1001 reads from the register 1011 the vertical synchronization period following this end report, it starts the CL 550 for the next compression.

If the vertical synchronization period is read before reporting the end of compression, it is determined that the current compression processing has not been completed before the input of the next frame, and the stop of the current compression and the start of compression of the next frame are immediately performed at CL550. To In this way, the CPU 1001 constantly monitors the status register 1011.

On the other hand, in the case of expansion, the NTSC output device 1
008 is generated by the timing generator 10
09, the read timing of the buffer 1006 is set, and the vertical synchronization period is set in the status register 1011.
Write to. The CPU 1001 reads the register 1011 and decompresses and starts the CL 550 during the vertical synchronization period. The CL 550 reads out the compressed data, sequentially decompresses it, writes it in the buffer 1006, and reports completion to the status register 1011 when one frame is completed. CPU1
001 always monitors the status register 1011 as in the case of compression.

[0009]

The above-mentioned video system for compression / expansion processing is optionally connected to a general-purpose computer system having a graphic function, for example, a workstation (WS) or a personal computer to compress / compress moving images. In order to use it for decompression, in addition to a dedicated bus connection function, a dedicated frame memory for storing input moving images or decompressed moving images one after another and a dedicated CRT for monitoring the same are required.

For this reason, there is a problem that the configuration of the image processing unit is made redundant, and the cost is increased contrary to the downsizing of the general-purpose computer.

In particular, if the high-speed compression / decompression processor is to be directly controlled by the CPU of a general-purpose computer, it is necessary for the CPU to constantly monitor the execution status such as time management as described above. For this reason, CP
The load on U increases, and processing of other general-purpose functions can hardly be performed. On the other hand, when priority processing indispensable to a general-purpose system such as communication with other devices is executed, compression or decompression processing is interrupted or delayed, and image quality deteriorates, such as the number of frames required for moving images and synchronization cannot be maintained. There is a problem.

An object of the present invention is to provide a high-quality image processing method and apparatus capable of reducing the load on a CPU and executing a moving image compression / decompression process at a predetermined timing.

It is another object of the present invention to provide an image processing method and apparatus capable of easily generating a combined moving image by concurrently performing update drawing of a still image and compression / expansion of a moving image.

Still another object of the present invention is to use a frame memory or a CRT for a still image for processing a moving image, thereby suppressing an increase in size and cost, and reducing a processing load of a compression / decompression function. It is an object of the present invention to provide a computer system which is compatible with a general-purpose function of a CPU.

[0015]

An object of the present invention is to provide an image processing method for compressing or decompressing a moving image consisting of a predetermined number of frames (hereinafter referred to as frames) per unit time. Or a first process for instructing the start of decompression and confirming the end of the compression or decompression processing, and executing means for receiving the start instruction and inputting, compressing or decompressing and outputting a moving image, And a second process for controlling according to a vertical synchronizing signal output every time.

Also, a CP for processing the first process is provided.
U and command control means for processing the second process.

Further, the command control means outputs, for each frame, a plurality of commands for instructing input, compression or decompression, and output of a moving image for each frame, and instructions for subsequent commands arranged at the beginning of the plurality of commands. A command sequence in which a vertical synchronization use command for enabling n frames (where n is an integer of 1 or more), which is enabled after waiting for a vertical synchronization signal, is provided from the CPU, and every time the vertical synchronization signal is detected, This is achieved by transferring each of the plurality of commands to the corresponding execution means in the order of the command sequence and causing the corresponding execution means to process the commands, and by putting the subsequent command processing in a waiting state by the next synchronous signal using command in the command sequence. You.

Further, the CPU is achieved by adding an instruction for changing the still image as a change command to an arbitrary position in the command sequence.

Further, the CPU confirms whether or not there is a request for changing the still image before and during the compression / expansion processing, and gives an instruction to the execution means (image drawing processor) if there is a change request. This is achieved by:

[0020]

According to the present invention, the compression / decompression processing by the CPU does not include management of the execution status of the compression / decompression processing, so that the load on the CPU can be greatly reduced. On the other hand, the command control means for managing the execution of compression / decompression includes:
A command sequence given in advance by the CPU can be easily configured by a FIFO that first-in first-out, and the like, and high-quality moving image compression / decompression is executed at an appropriate timing synchronized with a vertical synchronization signal for each frame.

Further, since input / output and monitoring of the moving image are performed by the frame memory shared with the still image, the enlargement of the apparatus configuration is suppressed, and the still image is arbitrarily changed in parallel with the compression / decompression processing. Thus, a wide variety of combined moving images can be easily created.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail with reference to the drawings. The same elements are denoted by the same reference symbols throughout the drawings.

FIG. 1 shows a hardware configuration of the image processing apparatus according to the present embodiment, which is realized by using a graphic function of a general-purpose computer system, for example, a workstation or a personal computer.

The image processing apparatus comprises a CPU 1, a main memory 2,
A bus control unit 4, a compression / decompression processor 5 for compressing / decompressing images, an image input / output unit 6 for inputting / outputting images, a drawing processor 7 for executing drawing processing, a frame memory 8 for storing image data, and displaying images. It comprises a CRT 9. In this example, the compression / decompression processor 5 is a JPEG processor, and the image input / output unit 6 is an NTSC video input / output unit.

The CPU 1, the main memory 2, and the bus control unit 4 are connected by a system bus 3 including an address line 31 and a data line 32. The bus control unit 4, the JPEG processor 5,
The NTSC video input / output unit 6 and the drawing processor 7 are connected by an address line 44 and a data line 45.

The NTSC video input / output unit 6 and the drawing processor 7 are connected to the video data signal line 64, and the JPEG processor 5 and the drawing processor 7 are connected to the frame memory data signal line 5.
3, the drawing processor 7 and the frame memory 8 are connected by a parallel data signal line 75 and a serial data signal line 76, and the bus control unit 4 and the JPEG processor 5 are connected by a compressed data signal line 52, respectively.

Further, the JPEG processor 5 and the CPU 1
Denotes an end interrupt signal signal line 51, and the NTSC video input / output unit 6 and the bus control unit 4 are connected by a signal line of the vertical synchronizing signal 10.

The bus control unit 4 includes a bus connection unit 41 for connecting to the system bus 3, a buffer 421 for storing a command sequence from the CPU 1, a data output control unit 422 for controlling the stored commands on a first-in first-out basis, A FIFO 42 comprising a timing signal section 423 for giving output timing and a data output stop control section 43 are provided. By the FIFO 42 and the control unit 43, the CPU
Control means 40 for controlling execution of a moving image in place of
Is composed.

The JPEG processor 5 compresses the video image data (moving image) transferred from the frame memory 8 according to the command from the FIFO 42 and transfers the compressed data to the main memory 2 via the bus connection unit 41. On the other hand, the compressed video image data (compressed moving image) stored in the main memory 2 is expanded and stored in the frame memory 8 as original video image data.

The compressed data stored in the main memory 2 is CP
U1 enables storage in an auxiliary storage device (not shown), transmission to another device via communication means, or predetermined processing of a compressed moving image.

The NTSC video input / output unit 6 detects and outputs the vertical synchronizing signal 10 from the input NTSC video input signal 65, or outputs the vertical synchronizing signal 10 of one frame in 1/30 seconds. Output means 61, NT
A / D converter 62 which A / D converts an SC video signal into digital video image data 64, D / A converts video image data from frame memory 8 and outputs an analog NTSC video signal / A converter 63 is provided.

The drawing processor 7 receives a command from the CPU 1 via the FIFO 42 and processes data exchange between the JPEG processor 5 and the frame memory 8 with the drawing unit 71 for drawing characters and figures in the frame memory 8. Frame memory data transfer unit 72, NTSC
An image data input / output unit 73 for exchanging data between the video input / output unit 5 and the frame memory 8, and an input / output control unit 74 for controlling input / output between the image data input / output unit 73, the CRT 9, and the frame memory 8 are provided.

Next, the operation of the above-described image processing apparatus will be described by taking as an example a case where an NTSC video image is monitored while being compressed or decompressed by a CRT 9 and a drawing process of characters and graphics is performed in parallel with the monitoring. .

FIG. 2 is a schematic diagram for explaining an example of the screen of the CRT 9 in the image processing apparatus of the present embodiment and the storage state of the frame memory 8 at that time. The screen shows a graphic display window 91, a character display window 92, and a monitoring window 93 for compressing or expanding a video image.

The frame memory 8 is provided with a graphic image area 81, a character image area 82, and a front 83 and a back 84 of a video image area. In accordance with a command from the CPU 1, image data of figures and characters generated from the drawing unit 71 of the drawing processor 7 are stored in an area 81 and an area 82, respectively.

As the video image area (front) 83 or the video image area (back) 84, an area to be actually displayed on the screen of the CRT 9 is selected by the input / output control unit 74 of the drawing processor 7. In this case, the video image area 83 in the output area to the CRT 9 is stored in the table buffer,
The video image area 84 in the work area is defined as a back buffer. However, the relationship between the output area (or work area) of the front buffer and the back buffer is not fixed,
The actual output area is changed alternately.

FIG. 3 is a schematic diagram for explaining monitoring in the compression processing of an input video image. As shown in a block 651 of FIG.
Is captured in 30 frames per second, and outputs a vertical synchronizing signal 10 when the scanning line finishes drawing one frame as shown in a block 652 in FIG.

In this embodiment, when the vertical synchronization signal 10 immediately after the frame 1 is detected, the input / output control unit 74 stores the frame (1) in the table buffer 83 in response to a command from the FIFO 42, as shown in a block 653. . At this time, the table buffer 83 becomes an output area of the CRT 9.

Next, as shown in a block 654, when a vertical synchronization signal immediately after the frame (2) is detected, the frame (1) in the front buffer 83 is transferred to the JPEG processor 5 and compressed, and the back buffer 84 Is stored in the frame (2). At this time, the back buffer 84
9 output areas.

Subsequently, as shown in a block 655, when the next vertical synchronization is entered, the frame (3) is stored in the compressed table buffer 83, and the back buffer 84 already stored in the JPEG processor 5 has been stored. Frame (2) is compressed. Thereafter, by repeating this every time the vertical synchronization is performed, video input and compressed data can be obtained at the timing indicated by the block 656 in FIG.

The JPEG processor 5 sends an end interrupt signal 51 to the CPU 1 at the end of compression of each frame.
Is output. The CPU 1 recognizes that the JPEG processor 5 has transferred the compressed data 52 to the main memory 2 by the DMA (Direct Memory Access) method and has finished.

FIG. 4 is a schematic diagram for explaining monitoring when the NTSC video image compressed data is subjected to decompression processing. When starting the decompression processing, first, the CPU 1 gives a “vertical synchronization signal generation” instruction to the vertical synchronization signal output unit 61 of the video input / output unit 6, and outputs a 1/30 second vertical synchronization signal 10.
Start output of As a result, a command can be output from the FIFO 42.

As shown in block 661 of FIG. 4A, the main memory 2 stores NTSC video image compressed data in frame units. JPEG processor 5
Is started by a command from the FIFO 42, reads compressed data from the main memory 2 by the DMA method.

As shown in block 662 of FIG. 7B, the JPEG processor 5 decompresses the compressed frame (1) and stores the decompressed frame (1) in the table buffer 83 of the frame memory 8. At this time, the table buffer 83 is a display area, and the video image of the frame (1) is reproduced on the CRT 9.

Next, as shown in block 663, when the vertical synchronizing signal 10 is input, the FIFO 42 outputs the next command, decompresses the compressed frame (2) and stores it in the back buffer 84. The decompressed frame (1) of the table buffer 83 is output to the video input / output unit 6. At this timing, the table buffer 83 is a display area of the CRT 9.

Subsequently, as shown in a block 664, when the next vertical synchronization 10 enters, the compressed frame (3) is decompressed, the output is terminated at the previous timing, and the table buffer 83 which has become an empty buffer is used. Then, the decompressed frame (3) is stored, and the frame (2) of the back buffer 84, which has been decompressed at the previous timing, is output to the video input / output unit 6. Thereafter, by repeating every time the vertical synchronization 10 is entered,
At the timing shown in block 665 of FIG.
A video output of the compressed data can be obtained.

Next, a description will be given of the FIFO 42 for outputting a command for performing the above-described processing of the JPEG processor 5, the video input / output unit 6, and the drawing processor 7 by the detected vertical synchronizing signal 10.

FIG. 5 shows the configuration of the command control means 40 of the present embodiment composed of the FIFO 42 and the switching unit 43, and the data structure input to and output from the FIFO 42.

A command to be processed in synchronization with the input of the vertical synchronizing signal is stored in a buffer 421 in the FIFO 42 as data 1, 2,. . . Before that, the data is stored with "vertical synchronization use" data which is a command meaning "wait for a vertical synchronization signal and output subsequent data". These data are stored in the processing order from the CPU 1 to n pieces of data before the processing.

When the vertical synchronizing signal 10 is input to the reset terminal R of the timing signal unit 423 composed of a flip-flop (FF), the output Q of the data output control unit 422 at the exit of the buffer 421 becomes "0". As a result, the data in the buffer 421 is turned on (through).

When the “vertical synchronous use” data in the buffer 421 passes through the data output control section 422, the data output stop control section 43 detects this and sets “1” to the input terminal D of the timing signal section 423. give. This allows
The output Q of the FF is inverted to “1”, and the data output control unit 422 enters the data OFF state (cannot pass through).

In this state, when the next vertical synchronizing signal 10 is input to the reset terminal R of the timing signal section 423, the Q terminal of the FF becomes "0" and the data output control section 4
Reference numeral 22 indicates the ON state (pass-through) again, and each data is output to the output control unit 4 until the next "vertical synchronization use" data appears.
22 and is forwarded to the corresponding execution device to enable processing of that command.

The video input / output unit 6 sends the frame memory 8
To correctly transfer an image for one frame, it is necessary to detect the head of the image. In this embodiment, the vertical synchronization signal 10 is used in place of the detection of the head, and when the synchronization signal 10 is detected, an image for one frame is transferred to the frame memory 8. Actually, the “video input” following “use vertical synchronization” is sent from the CPU 1 to the HIFO4.
2 and the HIFO 42 temporarily stops the output with “vertical synchronization use”, resumes the output with the vertical synchronization signal detected immediately thereafter, and instructs “video input”.

By the way, each data stored in the HIFO 42 is composed of an address section 100 and a data section 101, and is passed by an execution device having the same address code. That is, each execution device of the JPEG processor 5, the video input / output unit 6, and the drawing processor 7 connected to the output address line 44 and the output data line 45 of the FIFO 42, and the data output stop control unit 43 perform address decoding of different codes. Units 5-1, 6-1 and 7-1, and 43-1.
4 is compared with its own code, and if they are the same, the data code of the data line is received. The figure shows an example in which “vertical synchronization use” data at address “1” is taken into the data output stop control unit 43.

FIG. 6 is a flow chart for explaining the schematic operation of the image processing apparatus of this embodiment which performs the compression / expansion processing of a moving image in parallel with the drawing of a still image. It is shown.

The CPU selects one of compression processing, decompression processing and other processing (general processing such as calculation) (s10).
1). Hereinafter, the operation when the compression process is selected will be described.

First, it is confirmed whether or not there is a request for changing a graphic or a character displayed on the CRT (s103), and if so, the contents of the change are instructed (s104).

The CPU in this flowchart
The instruction from 1 means that the command data is output to the FIFO 42, and the execution control is performed by the command control means 40 as described later. Of course, the drawing of the still image may be performed by directly instructing the drawing processor 7 from the CPU 1 as in the related art.

The control of the compression processing by the CPU 1 starts with an instruction to start compression for n frames of a moving image. First, n
It is checked whether or not the first compression activation for the first frame of the moving image for the frame is performed (s105). If it is the first time, commands of “use vertical synchronization” and “input video to frame buffer” are instructed (s106, s10).
7). Next, an instruction of "use vertical synchronization", an instruction of "video input to frame buffer", and an instruction of "video compression of frame buffer" are repeated for n frames (s110).
To s113).

As a result, the instruction to start compression for n frames is completed, and the data of the command sequence in the order of the instructions is loaded into the buffer 421 of the FIFO 42. Thereafter, in response to the input of the vertical synchronizing signal, the compression processing is executed in accordance with the command output from the FIFO 42 for each frame.

The processing of the CPU that has completed the above-mentioned compression start instruction returns to step s103 relating to the processing of a still image, so that it can immediately respond to a change request from the CPU 1. Furthermore, a bold loop shown in the drawing for confirming the end of the actual compression processing for n frames (s108) is repeated until the end of n frames. If an image change request is received from the operator during this time, the contents of the change are specified in step s104.

Then, the compression end for n frames is s10
If it is confirmed in step 8, it is checked whether or not the compression is completed (s1
09) If the answer is yes, the processing ends. At this time, if there is a compression request for new n frames to be continuously compressed, the compression start of s110 to s113 is instructed for the new n frames. As a result, the compression processing for the input video is continued without interruption.

By the way, the number n of frames, which is a unit of one activation, can be set to one or more. Therefore, when the timing of the still image change request is known, the number of frames up to immediately before the timing is set to n, and a still image change command is loaded in the FIFO in advance after the compression processing command for n frames. I can put it. Thereafter, by sequentially loading the same or a different number of n-frame compression processing commands and still image change commands as before, a composite compressed moving image with a still image having a desired story can be created.

As described above, the control of the compression processing by the CPU 1 is not executed for the compression processing after the compression is started to give an instruction to the command control means 40, and the confirmation of the change request of the still image (this is also performed) It is not necessary to load the commands in the FIFO in advance) and the actual end of the compression is only periodically checked, and the processing load on the CPU becomes extremely light.

Next, the operation when the decompression process is selected in step s101 will be described. Step s114
At s115, as in the case of the compression, after responding to the request for changing the still image, it is checked whether or not the first decompression activation is performed for the first frame of the compressed data of n frames (s116).

If it is the first time, "vertical synchronization signal generation" is instructed to the vertical synchronization signal output means 61 of the video input / output unit 6. This instruction is issued after the start of n frames (s113).
It may be. Alternatively, "vertical synchronization signal generation" may be arranged as a command at the beginning of a command string stored in the FIFO 42, and transferred from the command control means 40 to the video input / output unit 6 after the start-up (s113).

Subsequently, the instruction of "use vertical synchronization", the instruction of "expanded output to frame buffer", and the instruction of "output video in frame buffer as video" are repeated for n frames (s120 to s123). As a result, the decompression start instruction for n frames is completed, and the data of the command sequence in the order specified is loaded into the buffer 421 of the FIFO 42.

When the vertical synchronizing output means 61 is activated and the vertical synchronizing signal 10 is generated, the FIFO of the command control means 40
O42 outputs a command for expanding and outputting one frame to the execution means every time the vertical synchronization signal is detected.

The processing of the CPU that has completed the decompression activation returns to step s114 relating to processing of a still image.
Further, the end confirmation of the actual decompression processing for n frames (s
118) is repeated until the end of n frames. If an image change request is received from the operator during this time, the contents of the change are specified in step s104. After the confirmation of the necessity of the continuation of the decompression process (s119), it is the same as the case of the compression process.

Further, by appropriately setting the number of processing frames n, which is a unit of the start-up process, and adding a still image change command to each of the plurality of start-up processes, the changed still image is synthesized with the decompressed moving image, and a desired still image is synthesized. It is also possible to create a combined moving image with a story. In this case, the still image change command may be loaded into the FIFO in the start-up process, and the repetition process of step s114 during decompression may be omitted. Or,
It is also possible to appropriately correct the change in S114 while monitoring during execution.

As described above, also in the control of the decompression process, after the decompression activation which gives an instruction to the command control means 40, the CPU 1 does not execute the decompression process, and the confirmation of the request for changing the still image and the actual The processing load on the CPU is light only by confirming the end of decompression.

FIG. 7 shows a table buffer 8 of the frame memory 8.
7 is a flowchart for performing the compression processing shown in FIG. 6 using the third buffer and the back buffer.

In steps s106 and s107, after instructing the table buffer to start compression for the first time, it is determined whether the current processing frame is an odd number. Since the processing frame number = 1 is initially set, the first time is determined to be an odd number. The processing frame number is updated every time the startup processing is performed for each frame (+1)
Is done.

In the case of an odd number, an instruction of "use vertical synchronization" (s110-1), an instruction of "video input and display to the back buffer" (s111-1), and an instruction of "front buffer video compression" (s112-) If 1) is an even number,
An instruction of “use vertical synchronization” (s110-2), an instruction of “video input and display to table buffer” (s111-2),
The instruction of “back buffer video compression” (s112-2)
It is repeated until the processing frame reaches n (s11
3). Note that video input and display to the buffer are given by two commands as shown in FIG.

As shown in FIG. 13C, every time the activation instruction for each frame is completed in step s112, it is checked whether there is a request for changing figures or characters (s1121). For example, the change instruction may be issued (s1122), and the still image change command may be loaded into the FIFO in the startup processing.

FIG. 8 is a schematic diagram showing the operation of the command control means 40 for controlling the execution of the compression processing in accordance with a change in the command storage state of the FIFO buffer 421.

FIG. 7A shows a command sequence loaded from the CPU 1 by the activation process shown in FIG. That is, the step s for processing a graphic or character change
104, commands c103 and c105 for instructing video input to the front buffer in s106 to s1017-2 of the first compression start, and commands c106 for instructing front buffer compression and back buffer input.
To c109 are given commands c110 to c113 for instructing compression of the back buffer and input to the front buffer.

Next, in the processing flow of FIG. 7A, the number of frames is set to n = 2, the change request is confirmed again at step s104, and the change data c114 and c115 for figures and characters are changed.
Is given.

However, if the change content and timing of the still image are known in advance, graphic and character change data c114 and c115 may be given in the processing flow of FIG. 7C. In this case, the number n of processing frames is increased as needed within the range of the capacity of the buffer 423, and the number of frames to be processed is increased as shown in FIG.
The necessary change command can be loaded in the FIFO 42 corresponding to a desired frame position.

FIG. 8B shows “changed graphic data” c 101 and “changed character data” c 102 that can be output independently of the vertical synchronization signal, which are output from the buffer 421 and transferred to the drawing processor 7, followed by “vertical synchronization”. Use "c103
Is output, the data output from the buffer 421 is stopped, and a state of waiting for the vertical synchronization signal 10 is shown.

FIG. 9C shows that the FIFO 42 detects the vertical synchronizing signal 10 and “video input to the table buffer” c 10
4 and "table buffer display" c105 are output, and the output of the buffer 421 is stopped by "vertical synchronization use" c106 again.

When the next vertical synchronizing signal 10 is detected,
The “front buffer compression” c107 is transferred to the JPEG processor 5, the “back buffer video input” c108 and the “back buffer display” c109 are transferred to the drawing processor 7,
The output stops at "vertical synchronization use" c110.

Further, when the next vertical synchronizing signal 10 is detected, "back buffer compression" c111, "front buffer video input" c112, and "front buffer display" c113 are transferred to each execution means, and the second time. The “changed graphic data” c114 and the “changed character data” c115 are transferred to the drawing unit 72, and the output stops at “vertical synchronization use” c116.

As described above, in order to actually execute compression, video input to the front or back buffer must be completed before waiting control by [use vertical synchronization] immediately before "compression processing" is performed. Then, after inputting the next vertical synchronizing signal 10, the JPEG processor 5 is instructed to start compression.

Since the vertical synchronizing signal 10 is generally output long after the effective area of the image of the input frame ends, when the vertical synchronizing signal 10 is input, the input to the buffer of the effective area has already ended. are doing. Also, the JPEG processor 5 has a processing capability of compressing a moving image in real time,
Thus, while waiting for the input of the next synchronization signal, image compression for one frame stored in the “front (back) buffer” can be performed.

According to the present embodiment, the front and back frame memory buffers for storing moving image data are used, and the input and the compression are alternately repeated for each frame. The other side can be compressed when input, and the compression processing can be executed completely in synchronization with the vertical synchronization signal for each frame. The front and back buffers are alternately set as display areas, and are displayed at the same time as the input and can be monitored.

FIG. 9 shows the above operation as a vertical synchronizing signal (about 3
(3 ms period) as a reference. The video images that are sequentially transmitted can be compressed in frame units and monitored and monitored by the CRT 9 while simultaneously changing graphics and characters and storing them in the compressed data main memory 2.

[0088] In the figure, rather than a special meaning is to have started a compression process from frame 3 of t _0, when you record a video image of the runaway is, always from the middle to the compression process is started, the Until the general appearance was expressed.

[0089] In the above example, shapes and, on the command c101, c102 timing t _-1 character changing process, c command 114, c 115 at a timing t _1, respectively C
The process is executed as in the RT display, a combined moving image of a moving image (a person and a sphere) is obtained on the frame memory, and is compressed at the next timing. Note that the still image change processing interrupts at the time of confirmation in
An instruction may be given directly from the PU 1 to the drawing processor 7.

The compressed moving image or the compressed combined moving image according to the present embodiment is stored in the main memory, and necessary processing can be performed. For example, it is possible to perform processing on compressed moving image data, as well as storage in an auxiliary storage device and transmission to other devices via a communication device, which facilitates provision of a virtual visual image.

Next, details of the decompression processing will be described with reference to FIGS.
This will be described with reference to FIG. FIG. 10 is a flowchart for performing the decompression processing shown in FIG. 6 using the front buffer 83 and the back buffer 84 of the frame memory 8.

At the time of the first extension activation, step s117
After instructing the video input / output unit 6 to generate a vertical synchronizing signal, it is determined whether the current processing frame is an odd number. Since the initial value is set to frame number = 1, it is determined that the first time is an odd number, and thereafter, updated every time one frame is activated (+1).
Is done.

In the case of an odd number, an instruction for "use vertical synchronization" (s120-1), an instruction for "expansion output to the front buffer" (s121-1), and an instruction for "back buffer video output" (s122-1) On the other hand, in the case of an even number, an instruction of “use vertical synchronization” (s120-2), an instruction of “expansion output to the back buffer” (s121-2), and an instruction of “front buffer video output” (s112-2) ) Is repeated until the number of processing frames reaches n (s113). Note that, as shown in FIG. 13B, confirmation of “change of figure and character” and “change instruction” thereof may be performed immediately before step s123.

FIG. 11 is a schematic diagram showing the operation of the command control means 40 for controlling the execution of the decompression processing according to the change in the command storage state of the FIFO buffer 421.

FIG. 11A shows a command sequence loaded from the CPU 1 by the activation process shown in FIG. From this state, “changed graphic data” c131 and “changed character data” c132 irrelevant to the vertical synchronization are transferred to the drawing processor 7, and “vertical synchronization used” c13
The output of the buffer 421 stops at the output of FIG.
The state shown in FIG.

When the FIFO 42 detects the vertical synchronizing signal 10, “extended output to table buffer” c 134 is output to the JPEG processor 5. The JPEG processor 5 decompresses one of the compressed image frames read out from the main memory 2 by the DMA method and stores it in the table buffer 83. Since the expanded image is displayed in the video window 93 of the CRT 9, monitoring becomes possible.

In parallel with this, "back buffer video output" c135 is output to the drawing processor 7, but at this point, since the back buffer 84 has not yet stored the decompressed image, the actual video output is obtained. I can't. Next, when "use vertical synchronization" c136 is output, the data output of the FIFO 42 is stopped, and the state shown in FIG.

When the next vertical synchronizing signal 10 is detected,
"Expansion output to back buffer" c137 and "front buffer video output" c138 are output to the respective execution means. The video input / output device 6 obtains an actual video from this timing. Further, the "changed graphic data" c139 and the "changed character data" c140 are output, and the output of the buffer 421 is stopped again at the "use vertical synchronization" c141.

FIG. 12 is a timing chart showing the above operation based on the vertical synchronizing signal. The compressed image data from the main memory 2 is JPE
The data is decompressed by the G processor 5 and output alternately to the front buffer and the back buffer, and displayed on the CRT 9 while the decompressed data is transmitted from one buffer to the video input / output device 6 while the other buffer receives the decompressed data. I have. In addition, in parallel with the decompression processing, graphics and characters are changed at arbitrary timings t ₋₁ , t ₁ , t _3, etc., and a desired composite moving image is created. Shapes t ₀ in the figure, change of the character, by direct instruction in extended run.

As described above, the compression / decompression processing of a moving image according to the present embodiment is performed by the CPU 1 in advance of the F of the bus control unit 4.
Since a command sequence stored in the IFO 42 and instructing input / output, display (monitor), and compression / expansion of a moving image is output in synchronization with a vertical synchronization signal of the moving image, the load on the CPU in the compression / expansion processing is reduced. It can be greatly reduced.

Further, since the moving image display area of the frame memory is used as a buffer for inputting / outputting a moving image frame by frame, a special configuration for a monitor or the like is not required. In addition, this moving image display area has a double buffer structure of the front and back sides, and is alternately captured and displayed for each frame, so that when capturing and displaying are performed on one side, compression is performed on the other captured side. Since the processing can be processed in parallel,
Compression / expansion processing that is completely synchronized with the vertical synchronization signal can be performed.

Further, a frame memory is shared for processing of moving images and still images, and commands for processing for changing figures and characters are stored in advance from the CPU 1 at arbitrary timings in the above-mentioned command sequence, or from the CPU 1. Since the processing is interrupted at an arbitrary time and the change processing is performed in parallel with the compression / expansion of the moving image, the compression processing of the synthesized moving image obtained by synthesizing the moving image with the graphics and characters becomes possible. In addition, a combined moving image can be provided in parallel with a process of changing a figure or a character in parallel with the expanding process of the compressed moving image.

As described above, the compression / expansion processing of a moving image by a general-purpose computer can be easily realized. As a result, processing of the moving image itself and arbitrary synthesizing with a still image become possible. It is easy to provide various forms of screens such as screens.

In this embodiment, the FIFO is provided in the bus control unit as control means for instructing the compression / decompression processing of the moving image. However, the FIFO unit may be provided independently. Further, it may be constituted by another CPU instead of the FIFO.
Depending on the processing capacity of U, it is also possible to perform the processing by itself.

[0105]

According to the image processing method of the present invention, a command sequence required for compression / expansion of a moving image is given from the CPU in advance, and the compression / expansion is controlled according to a vertical synchronization signal of the moving image. There is no conflict with general-purpose functions of the computer system, and there is an effect that high-quality compression / expansion processing in accordance with the timing of moving images can be realized.

According to the image processing method of the present invention, a still image such as a figure or a character can be changed and combined with a moving image at an arbitrary timing in parallel with the compression process. There is an effect that the extension can be realized.

According to the image processing method of the present invention, since a still image can be changed at an arbitrary timing in parallel with the decompression process, a desired synthesized moving image can be easily created.

According to the image processing apparatus of the present invention, a moving image compression / decompression processor is provided with a frame memory, a CRT, etc., which is also used as a still image, and can perform input / output, monitoring, and compression / decompression processing of a moving image. Therefore, there is an effect that increase in size and cost of the image processing apparatus can be suppressed.

According to the computer system of the present invention,
Command sequence necessary for compression / decompression of moving images
Control means for controlling the compression / decompression of the moving image in accordance with the vertical synchronization signal of the moving image.
This is advantageous in that the processing of moving images can be easily realized even with a general-purpose computer by reducing the burden on the user.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a hardware configuration of an image processing apparatus as one embodiment of the present invention.

FIG. 2 is an explanatory diagram illustrating a display screen drawn by an image processing apparatus and a state of using a frame memory.

FIG. 3 is an explanatory diagram showing video input, compressed data, and a method of using a frame memory at the time of compression.

FIG. 4 is an explanatory view showing a method of using a frame memory at the time of compressed data video output and decompression.

FIG. 5 is an explanatory diagram showing a configuration of an image processing control unit mainly including a FIFO and an operation of the FIFO.

FIG. 6 is a flowchart illustrating an operation of the image processing apparatus according to the present exemplary embodiment.

FIG. 7 is a flowchart of parallel processing of compression of a video image and change of a still image.

FIG. 8: For video image compression and still image modification,
FIG. 5 is a schematic diagram showing a command sequence stored in a FIFO and its transition.

FIG. 9 is a timing chart showing an execution state of compression of a video image and change of a still image.

FIG. 10 is a flowchart for creating a combined moving image by performing parallel processing of expansion of a compressed video image and change of a still image.

FIG. 11 is a schematic diagram showing a command sequence stored in a FIFO and its transition for expanding a compressed video image and changing a still image.

FIG. 12 is a timing chart showing an execution state of creation of a combined moving image by expanding a compressed video image and changing a still image.

FIG. 13 is a configuration diagram of a conventional video system that performs compression and decompression exclusively.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... CPU, 2 ... main memory, 3 ... system bus, 4 ... bus control part, 40 ... image processing control means, 41 ... bus connection part, 42 ... FIFO, 421 ... buffer, 422 ... data output control part, 423 ... Timing signal section 43 data output stop control section 5 JPEG processor (compression / expansion processing means) 51 end interrupt line 52 compression data line 6 NTSC video input / output section 61 vertical synchronization detection section , 62 ... A / D converter, 63 ... D / A converter, 64 ...
Video data line, 65 ... NTSC video input line, 66 ...
NTSC video output line, 7: drawing processor, 71: drawing unit, 72: frame memory data transfer unit, 73: image data input / output unit, 74: input / output control unit, 75: frame memory parallel data, 76: frame memory serial Data, 8: Frame memory, 83: Table buffer, 84
... Back buffer, 9 ... CRT, 10 ... Vertical synchronization signal.

Continuation of the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G09G 5/36 G06T 1/00 G06T 9/00 H04N 5/06 H04N 7/24

Claims (14)

(57) [Claims] An image processing method for compressing or decompressing a moving image comprising a predetermined number of frames (hereinafter referred to as frames) per unit time, comprising: inputting, compressing or decompressing and outputting a moving image for each frame. And a vertical synchronization use command which is arranged at the head of the command and waits for a vertical synchronization signal output for each frame, and which enables the next command to be issued, for n frames (where n is 1 or more) Each time the vertical synchronizing signal is detected, each of the plurality of instruction commands is transferred to a corresponding execution unit to execute the instruction, and the command sequence is executed. An image processing method, wherein the transfer of subsequent instruction commands is stopped by the next vertical synchronization use command. 2. The image processing method according to claim 1, wherein the compressed moving image is stored in a main memory. 3. A process for drawing a still image such as a figure or a character,
An image processing method for processing a predetermined number of frames (hereinafter, referred to as frames) per unit time of a continuous moving image in parallel, comprising a plurality of instructions for inputting and compressing a moving image for each frame. A command and a vertical synchronization use command which is arranged at the top of the frame and which allows an instruction of a subsequent command to wait for detection of a vertical synchronization signal output each time the frame ends, for n frames (n is an integer of 1 or more) , A command sequence linked in the processing order is provided, the still image change content command is arranged at a desired position in the command sequence, and / or the presence or absence of the still image change request is checked during the compression processing. Then, the change content command is instructed to the corresponding execution means, and each time the vertical synchronization signal is detected, the plurality of command commands or the change command The command is transferred to the corresponding execution means to execute the instruction, and the process of stopping the transfer of the subsequent command by the next vertical synchronization use command following the transferred command is repeated for the n frames. Image processing method. 4. An image for creating a composite moving image by performing a predetermined number of frames (hereinafter referred to as a frame) of a moving image per unit time from a compressed moving image and a process of drawing a still image such as a figure or a character in parallel. In the processing method, a plurality of instruction commands for instructing decompression, display, and output of a compressed moving image for each frame, and instructions for a subsequent command arranged at the beginning of the plurality of instruction commands are detected by detecting a vertical synchronization signal output for each frame. A command sequence in which a vertical synchronization use command for waiting and enabling is provided for n frames (n is an integer of 1 or more) is provided in the processing order, and the still image change content command is arranged at a desired position in the command sequence. During the decompression process, the presence / absence of the change request of the still image is confirmed, and if there is a change request, the change content command is instructed to the corresponding execution means, and the vertical synchronization signal is detected. Each time, in the order of the command sequence, the plurality of instruction commands or the change content commands are transferred to the corresponding execution means to execute the instructions, and the subsequent vertical synchronization use commands subsequent to the transferred commands are used. An image processing method for repeating the process of stopping the transfer of the command for the n frames. 5. An image processing method for creating a composite moving image in parallel with a process of compressing or expanding a moving image input / output together with a vertical synchronizing signal for each frame and a process of drawing a still image such as a figure or a character. When the compression process is selected, the CPU instructs the command control means with a change content command for the displayed still image and a compression process command for n frames (n is an integer of 1 or more). If there is a request to change the still image during the compression processing for frames, it is instructed, and if there is a request to continue at the end of the compression processing for n frames, the above series of processing is repeated. When the decompression process is selected, a command for changing the displayed still image and a decompression process command for n frames (n is an integer of 1 or more) are instructed to the command control means, and the decompression process for the n frames is performed. If there is a request to change the still image during the process, it indicates the request, and if there is a request to continue at the end of decompression for the n frames, the above series of processes is repeated. An image processing method, wherein the change content command and the compression processing command or the decompression processing command for one frame are transferred to an execution unit every time a vertical synchronization signal is output. 6. The drawing processor according to claim 5, wherein the execution unit is a drawing processor that controls a compression / decompression processor, a frame memory and a CRT, and the drawing processor processes one frame of the compression processing command. A “moving image data input to frame memory” command, a “moving image data compression” command processed by the compression / decompression processor, and a “vertical synchronization use” command to stop output of a subsequent command processed by the command control means. One frame of the decompression processing command includes a “decompression output to frame memory” command processed by the compression / decompression processor, a “moving image data output from frame memory” processed by the drawing processor, Stop the command output for the next frame processed by the command control unit. An image processing method comprising a "use" command. 7. The image processing method according to claim 6, wherein the change content command is instructed to the drawing processor directly or via the command control means. 8. Via a CPU (Central Processing Unit) and a bus
Image input device to be connected, frame memory and CRT
With a drawing processor to control the image and draw a still image
In the image processing device, an image compression processor that compresses a moving image for each frame, and a vertical frame for each frame of the moving image input to the image input device.
Vertical synchronization detecting means for detecting a synchronization signal, the drawing processor, the image input device, and the image pressure.
Command to instruct each compression processor to perform compression processing
Command and the output of those commands are paused for each frame
Command sequence in which commands that specify
Stored by the CPU, and stores the vertical synchronization signal.
Command that outputs a command for one frame each time it is detected
Code control means , provided in the drawing processor, from the image input device.
Transferring the moving image data to the frame memory,
Moving image data from the frame memory to the image compression processor.
Image data transfer means for transferring data.
An image processing device. 9. The image compression processor according to claim 8, wherein said image compression processor is a JPEG processor.
The image input device is an NTSC video input device.
Image processing apparatus. 10. Via a CPU (Central Processing Unit) and a bus
Image output device, frame memory and CR
A drawing processor for controlling T to draw a still image
Image decompression for expanding a compressed moving image stored in a main memory in an image processing apparatus.
A processor, and a frame of the moving image to be output, which is provided in the image output device.
Vertical synchronization signal output means for outputting a vertical synchronization signal for each system
The drawing processor, the image output device, and the image decompression.
Command for instructing each of the long processors to perform decompression processing
Command and the output of those commands are paused for each frame
Command sequence in which commands that specify
The vertical synchronizing signal is stored after being given from the CPU.
A frame that outputs a command for one frame each time it is output
Command control means and the image decompression processor provided in the drawing processor.
Moving image data expanded from the memory to the frame memory
Moving image from the frame memory to the image output device.
Image data transfer means for transferring data.
Characteristic image processing device. 11. The frame memory according to claim 8, 9 or 10, wherein the frame memory receives the input or output of the moving image data.
Display and subsequent output or transfer alternately every frame
Video display area consisting of two buffers for
An image processing apparatus characterized by the above-mentioned. 12. The method according to any one of claims 8 to 11, wherein
The command control means stores the command sequence on a first-in first-out basis.
Image characterized by having a FIFO for storing therein
Processing equipment. 13. A CPU (central processing unit ) connected to a bus.
), The main memory, and the bus connected via the bus control means.
Rendering processor and image data stored and displayed
Computer system with frame memory and CRT
In a system, a moving image is input and A / D converted to digital moving image data.
The D / A conversion of the moving image data into an analog moving image
And output it, and output it in synchronization with the frame of the moving image.
An image input / output device that outputs a direct synchronization signal, and the moving image data or the compressed moving image data
Compression / decompression process that compresses or decompresses in synchronization with the
Between the image input / output device and the frame memory, and between the image input / output device and the frame memory.
The moving image between the compression / decompression processor and the frame memory
To the drawing processor that transfers image data for each frame
Provided transfer means, the drawing processor, the image input / output device, and the image
A plurality of instruction frames for instructing each of the compression / decompression processors
Command to stop the output of the command for each command and frame
Commands that list the vertical synchronization commands
And stores one frame every time the vertical synchronization signal is detected.
And outputs the instruction command for a predetermined period to predetermined execution means.
Command control means, the compression processing and the decompression realized by the CPU,
Processing function selection means for selecting any of processing and other processing
Having an image processing function characterized by comprising:
Computer system. 14. The compressed moving image data according to claim 13 , wherein the compressed moving image data is compressed or decompressed.
Direct between the compression / decompression processor and the main memory
Writing / writing by memory access (DMA) method
A core having an image processing function characterized by being read out;
Computer system.