JP3904243B2 - Image processing apparatus and method - Google Patents

Description

[0001]
[Industrial application fields]
The present invention relates to an image processing apparatus and an image processing method, and more particularly to an image processing technique for performing processing by inputting a plurality of image data. The present invention is applicable to a video game or the like that handles a plurality of image data.
[0002]
[Prior art]
Processing such as superimposing two or more image data on a display device as a single image is performed by a video game machine or the like. Conventionally, such processing is realized by assigning priorities to image data in advance and designating a predetermined area as transparent for the image data. The priority order of image data represents whether the image data is displayed on the front side or the back side when displayed on the display device. For example, in the case where an image of looking out from a train window is composed of a plurality of image data, as shown in FIG. The scenery outside the window can be set as one image data B.
[0003]
In this case, the image data F inside the train is on the front side, and the landscape image data B is on the back side. The part corresponding to the train window is designated as transparent. When the image data is configured in this way, the image data in the train with a high priority is selected for the superimposition of the two image data except for the portion of the window designated for transparency. For the portion of, background data is selected. As a result, a state where the scenery can be seen from the window portion is created.
[0004]
A configuration as shown in FIG. 11 is known as an image processing apparatus that realizes the above functions. In this configuration, there are two external devices that generate image data. From the external device 250, the image data 290, the priority information 300 on the upper surface or the lower surface with respect to the data of the external device 260, and its pixels Information 310 indicating whether or not is transparent is sent in units of pixels. Also, the external device 260 sends image data 320 and information 330 on whether the pixel is transparent or non-transparent.
[0005]
The priority order and the attribute information indicating whether it is transparent or non-transparent are input to the display image data determination circuit 280 to determine which image data is selected for each pixel. Based on the result, the image data is switched by the selector 270 and output as an image in real time.
[0006]
[Problems to be solved by the invention]
However, such an image processing apparatus can only perform a simple overlay process of two image data, and cannot perform a process of overlaying three or more image data. Of course, if a plurality of similar configurations are prepared, a large number of image data can be processed. However, high-speed processing is required and a plurality of processing units that are complex circuits must be provided, and the number of image data to be processed is small. As it increases, it becomes impossible in reality. Further, if the priority order between the image data is to be changed, all the wirings to the plurality of processing units must be switched, and this is not easy in this respect.
[0007]
In addition, it is usually impossible to perform semi-transparent superposition such that the other side of the water tank can be seen. For this reason, it has not been possible to perform advanced image processing such as multiple mixing processing by processing a large number of input image data required for a video game.
[0008]
Further, in a conventional image processing apparatus (for example, Japanese Patent Laid-Open No. 62-264096), only one type of image data is handled, and it is not considered to handle different types of image data. In particular, it was not possible to perform overlay processing between different types of image data.
[0009]
Accordingly, an image processing apparatus and an image processing method of the present invention are intended to solve these problems and to easily perform advanced image processing, particularly free superimposition processing of image data, by inputting a plurality of image data. As made.
[0010]
[Means for Solving the Problems]
In order to achieve this object, the first image processing apparatus of the present invention provides:
An image processing apparatus for inputting a plurality of image data and processing each image data,
Input means for inputting the image data in units of pixels;
Pixel processing means for dividing the input period of the pixels and performing predetermined processing on the image data of the pixels input during one divided input period;
Division processing data holding means for holding the processed image data and subjecting it to the predetermined processing in the other divided period in the pixel processing means;
Output means for outputting the result of holding the divided process data holding means as processed image data for the pixel at the end of the input period;
With,
The pixel processing unit is a unit that performs the predetermined process and performs an image overlapping process between the image data input from the input unit and the data held by the divided processing data holding unit.
This is the gist.
In this way, it is possible to easily perform overlapping processing for a plurality of image data. As the overlap, not only the context but also various overlap processes such as transmission and semi-transmission can be assumed.
[0011]
Here, it is possible to provide a selection unit that alternatively selects image data input from a plurality of external devices in accordance with the division of the input period in the pixel processing unit. This selection means selects the image data in the order corresponding to the priority order of the image data, and if the priority order is variable, the front and back of the image data can be easily changed.
[0012]
Furthermore, pixel processing meansIn addition, means for dividing the input period of the image data input from the input means into four can be provided.
[0013]
In addition to providing parameter generation means for giving parameters relating to image processing, and using pixel processing means as means for determining processing based on these parameters, various processing on the image can be realized simply by changing the parameters. Can do.
[0014]
In this case, when the processing for the image relates to the overlap of two or more images, the image processing apparatus includes an overlap parameter generating unit that gives a parameter regarding the overlap, and the pixel processing unit inputs image data input from the input unit based on the parameter. And a means for determining the ratio of mixing with the data held by the divided processing data holding means, it is possible to display the overlapping state of the images and to easily change the state.
[0015]
Here, if these parameter generating means are provided with parameter setting means capable of setting parameters for each pixel, or can be set for each period obtained by dividing the pixel input period, the processing by the parameters is performed on the pixel. It becomes possible to change for every image data which comprises a pixel.
[0016]
If the parameter generating means generates a transparent parameter determining means for determining whether or not the parameter for specifying transparency, and if this parameter is a parameter for specifying transparent, the pixel processing means If a transparent processing unit that outputs the output of the divided processing data holding unit as it is is provided, when a transparent image comes to the near side, an image lower than that or a composite image can be displayed as it is. In such a case, the display can be performed very easily.
[0017]
The selection means for selectively inputting image data allows the image data selection period to be less than the period obtained by dividing the input period of one pixel by the number of external devices. Can be completed with.
[0018]
On the other hand, the image processing method of the present invention includes:
An image processing method for inputting a plurality of image data and processing each image data,
The image data is input in pixel units,
Dividing the input period of the pixel;
The image data of pixels input during one divided input periodImage overlapProcess,
The processed image data is held, and the processed image data is stored in the divided other period.Image overlapFor the processing of
At the end of the input period, the stored image data is output as processed image data for that pixel.
This is the gist.
[0022]
  According to the present inventionSecondThe image processing apparatus is an image processing apparatus that inputs a plurality of signals for displaying a color image and performs predetermined processing between images displayed by the plurality of signals.
  Signal input means for inputting the plurality of signals in units of pixels;
  An image determination signal output means for determining whether the input signal is an image signal directly representing an image or an image signal representing an image through a predetermined conversion process, and outputting a determination signal;
  Image signal aligning means for performing the predetermined conversion process on the input signal based on the determination signal and aligning the plurality of signals in the same form capable of the predetermined process;
  Pixels for performing predetermined processing on image data of pixels in one signal input during one divided input period, by dividing an input period of pixels as a unit of the plurality of signals arranged in the same form Processing means;
  Division processing data holding means for holding the processed image data and subjecting it to the predetermined processing in the other divided period in the pixel processing means;
  Output means for outputting the result of holding the divided process data holding means as processed image data for the pixel at the end of the input period;
  The main point is that
[0023]
[Action]
In the first image processing apparatus of the present invention having such a configuration, image data is input in units of pixels. The input period for the pixel is divided, and predetermined processing is performed on the image data input during the divided input period by the pixel processing means. This process may be a process such as superposition with data held by a divided process data holding unit, which will be described later, or may be a predetermined process (for example, a filter process).
[0024]
The image data processed by the pixel processing unit is held by the divided processing data holding unit and is subjected to predetermined processing in another divided input period. That is, the input period for one pixel is divided into n, and the image data processed in the Kth period (K is an integer greater than or equal to 1 and less than n) is converted into the Lth period (L exceeds K and n). Less than an integer). If a register or the like is prepared, the image data for the Kth period can be used not only for the process of the (K + 1) th period but also for the process of any period after the (K + 2) th period.
[0025]
When the processing is performed in the divided period and the input period for the pixel ends, the output unit outputs the holding result of the divided processing data holding unit at the end of the processing as processed image data for the pixel. As a result, it is possible to obtain processed image data obtained by performing processing related to each other for at least two or more image data. Since these processes are completed during the input period for the pixel, it is obvious that the moving image can be handled as it is. Of course, it can be applied to still image processing.
[0026]
On the other hand, according to the image processing method of the present invention having the above-described configuration, a plurality of image data are input in each period obtained by dividing the input period for the pixels, and the image data after performing a predetermined process is held. For the processing in another period, the held image data is output as processed image data for the pixel at the end of the input period for the pixel.
[0029]
  In the present invention,SecondImage processing equipment,sameThe same time division processing as that of the first image processing apparatus is performed on the signals arranged in the same pattern.
  The first explained aboveOr secondThe image processing apparatus can be realized as a game machine which is an electronic device incorporating the image processing apparatus, and can use the various image processing described above.
[0030]
【Example】
In order to further clarify the effects of the present invention described above, preferred embodiments of the present invention will be described below. FIG. 1 is an external view of a video game apparatus 20 incorporating an image processing apparatus as an embodiment of the present invention, and FIG. 2 is a block diagram showing its internal configuration.
[0031]
As shown in FIG. 1, the video game apparatus 20 outputs a video signal to a color television 28 and a game machine main body 22 to which a CD-ROM 21 can be attached, game pads 24 and 26 connected to the main body 22. The video signal cable 30 and a speaker 34 for outputting sound are included. The main body 22 includes a cover 31 that can be opened and closed. The cover 31 is opened, and the CD-ROM 21 is attached to the CD-ROM drive 32. When the cover 31 is closed in this state, the turntable of the CD-ROM drive 32 rotates, and the game program, image, and sound information (hereinafter referred to as AV information) recorded on the CD-ROM 21 are stored in the microprocessor in the main body 22. And the game is started. The game progresses by operating the switch 24a of the normal game pads 24, 26, the cursor stick 24b, and the like.
[0032]
In the main body 22, various circuits for displaying a moving image using data recorded in the CD-ROM 21 are incorporated. This is shown in the block diagram of FIG. As shown in the figure, the video game apparatus 20 includes a CD-ROM drive 32 connected via a SCSI bus 36, and a microprocessor (hereinafter referred to as MPU) that performs overall image processing and all related processing. ) 40, a main memory (hereinafter referred to as M-RAM) 41 directly connected to the MPU 40, a ROM 42 that similarly stores a BIOS program, and various units connected to a bus (M-BUS) 43 of the MPU 40, That is, an image signal control unit 45, an image data decompression unit 47, a VDP unit 49 that outputs a specific image signal, a video encoder unit 50 that synthesizes and outputs a video signal, and an audio data output unit 52 that handles audio data are provided.
[0033]
In the video game apparatus 20, a memory (hereinafter referred to as K-RAM) 55 connected to a local bus (K-BUS) 54 of the image signal control unit 45 and a local bus of the image data decompression unit 47 are provided. Output signals from the connected memory (hereinafter referred to as R-RAM) 57, the video memory (hereinafter referred to as V-RAM) 59 connected to the local bus of the VDP unit 49, and the video encoder unit 50 are used as normal images. An NTSC converter 60 that converts the signal (NTSC) and outputs it to the color television 28 is provided.
[0034]
The MPU 40 is capable of high-speed operation, and includes an operation unit 40a capable of arithmetic logic operations including floating point operations and a controller 40b for dynamic memory. The MPU 40 controls each unit while performing input / output with the game pads 24 and 26, the CD-ROM 21 and the like using the BIOS pre-installed in the ROM 42 in accordance with the program developed in the main memory 41. Then, the video is displayed and the sound is output to advance the game.
[0035]
The image signal control unit 45, the image data decompression unit 47, the video encoder unit 50, and the audio data output unit 52 are each configured by a large-scale logic circuit. Explain the configuration and simple operation of each unit.
[0036]
Image signal control unit 45: This unit 45 is an MPUI / F 45a that exchanges data with the MPU 40 via the M-BUS 43, and a SCSI controller 45b that exchanges data with the CD-ROM drive 32 via the SCSI-BUS 36. AFFIN converter 45c that performs affine transformation on image data input from CD-ROM 21, etc., graphic controller 45d that controls output of image data received from CD-ROM 21, etc., and sound controller 45e that also controls output of audio data Etc. The image signal control unit 45 extracts image data and audio data from the data received from the CD-ROM 21 or the like, and temporarily stores these data in the K-RAM 55. The stored data is controlled by the MPU 40 and output to the image data decompression unit 47, the audio data output unit 52, and the like at a predetermined timing by the graphic controller 45d and the sound controller 45e. When a predetermined affine transformation is instructed to the image data from the MPU 40, the AFFIN converter 45c performs the necessary affine transformation and performs linear transformation (image deformation) of the image data.
[0037]
Image data decompression unit 47: This unit 47 is a unit 47a for performing Huffman decoding on received image data, a unit 47b for performing DCT inverse transform (IDCT) on data subjected to Huffman decoding, and a run length. A unit 47c that performs expansion and an expansion controller 47d that controls these units are provided. When these images are compressed with respect to the image data received from the image signal control unit 45 or the MPU 40, the unit 47 decodes the data using a suitable algorithm and decompresses the image data.
[0038]
VDP unit 49: This unit 49 is actually mounted with two identical chips. The functions are the same, and each has a function of generating a background image by a specific pattern and color and a block unit image called a sprite. Sprites are images specially designed to easily display moving characters in games and the like, and blocks that frequently generate and disappear, and based on signals from the MPU 40, video signals of a plurality of blocks are Output with background image. The V-RAM 59 connected to the VDP unit 49 stores sprite images, that is, necessary characters and the like. When the MPU 40 outputs a command, for example, a command to move the character from one point to another point, the VDP unit 49 analyzes the command, and uses the sprite stored in the V-RAM 59 as, for example, a movement path. It is possible to easily realize movement corresponding to the command by developing and erasing sequentially along the line.
[0039]
Video encoder unit 50: This unit 50 receives an image signal control unit 45, an image data decompression unit 47, an interface unit 50a for inputting image data from the VDP unit 49, a look-up table for determining the color of the image data, and the like. A circuit for setting priorities of a plurality of image data, and the like, an image composition unit 50b for composing a plurality of image data based on the set priorities, a DAC unit 50c for converting the composited image data into an analog signal, And a control unit 50d for controlling the image composition unit 50b. The details of each part will be described later. In this video encoder unit 50, the image signal from the image signal control unit 45, the image signal from the image data decompression unit 47, and two sets of image signals from the VDP unit 49 ( Background image and sprite), prioritize them, and synthesize these images. The degree of synthesis between the images can be set in a plurality of stages from transparent to opaque with respect to the image on the far side (high priority) image on the far side.
[0040]
Audio data output unit 52: This unit 52 is a unit for outputting audio, music, etc. from the speaker 34, and an ADPCM unit 52a for performing audio synthesis by adaptive differential pulse code modulation (ADPCM), a plurality of predetermined units. A sound source generator (PSG) 52b capable of generating the sound sources at the same time, and a mixer 50c for synthesizing these synthesized sounds and sounds from the sound sources. The unit 52 synthesizes sound in accordance with data from the image signal control unit 45 or the MPU 40 or plays music using a sound source and outputs it to the speaker 34. The main body 22 has one built-in speaker 34 and is monaural, but the output to the outside is stereo.
[0041]
Next, a detailed configuration of the video encoder unit 50 corresponding to the image processing apparatus of the present invention will be described with reference to FIGS. In FIG. 3, a thick solid line indicates a flow of YUV data, a thick two-dot chain line indicates a flow of palette data, a thin two-dot chain line indicates a flow of register information, and a broken line indicates a flow of a control signal. .
[0042]
YUV data is an 8-bit × 3 = 24-bit signal consisting of three sets of Y data, U data, and V data. Y data indicates brightness, and U data is a blue-yellow color difference. V data indicates red-green color difference information. In this embodiment, YUV data is input from the image signal control unit 45 and the image data decompression unit 47. YUV data is 8-bit data, and Y data indicates black when 00 and white when FF. U data and V data are all 8 bits in the natural color display mode (16.77 million colors display), while in the 65536 color display mode, the lower 4 bits are 0 and only the upper 4 bits are valid. is there. Since the U data and V data are signed data, both the natural color display mode and the 65536 color display mode are colorless with a value of 80.
[0043]
Palette data is data that refers to a color palette described later, and is image data that becomes YUV data after referring to the color palette. In this embodiment, the palette data is input from the VDP unit 49, the image signal control unit 45, and the image data decompression unit 47. In the case of palette data, the color of the image changes if a different color palette is referenced. The color represented by the palette data is up to 65536 colors. In the embodiment, the size of the color palette is limited to about 512, so the number of colors that can be displayed is limited to about 512 from 65536 colors.
[0044]
The register information is information that determines the operation state of the image composition unit 50b and the like, and is data that is output from the control unit 50d to the image composition unit 50b and handled inside the image composition unit 50b. The control unit 50d includes a plurality of registers that can be read and written from the MPU 40, and the image composition unit 50b performs various processes according to the values written in the registers. The control signal is a signal for controlling operation timing of each unit.
[0045]
The interface unit 50a of the video encoder unit 50 will be described in order. The interface unit 50a has a VDP unit interface (hereinafter abbreviated as first I / F) 62 for inputting two sets of image data from the VDP unit 49, and an image signal control for inputting image data from the image signal control unit 45. A unit interface (hereinafter abbreviated as second I / F) 63 and an image data expansion unit interface (hereinafter abbreviated as third I / F) 64 for inputting image data from the image data expansion unit 47 are provided. The internal configuration of each interface 62, 63, 64 will be described later.
[0046]
The image composition unit 50b includes the following circuits. That is,
A first data selector 65 for selecting an image signal of a type referring to a color palette among signals from the respective I / Fs 62, 63, 64;
An adder 67 for adding an offset value necessary for output from the first data selector 65;
A color palette 68 that is referenced by the output of the adder 67 and outputs the corresponding YUV data;
A second data selector 72 for selecting YUV data from the color palette 68 or YUV data from the second and third I / Fs;
A third data selector 73 for selecting output data of the second data selector 72 or output data of an image special processing computing unit (described later);
[0047]
A time division data latch 74 that latches the output of the third data selector 73;
An image special processing arithmetic unit 75 for performing a special superimposing operation of an image, which will be described later, between the data held in the time division data latch 74 and the output data of the second data selector 72;
A pixel image data latch 76 that holds the output of the time division data latch 74 divided into YUV signals after the superimposition processing for one pixel is completed;
D / A converters 81, 82, 83 for converting the output data of the pixel image data latch 76 into analog signals, respectively.
A MOD setting unit 84 for setting whether the output signals of the D / A converters 81 to 83 are NTSC signals or component signals;
A synchronization signal generation circuit 86 that receives a system clock SCK and an external synchronization signal and generates a dot clock DCK, horizontal and vertical synchronization signals, etc.
Is provided.
[0048]
The synchronization signal generation circuit 86 outputs a synchronization signal to the time division data latch 74, the pixel image data latch 76, and the like, although not directly shown, in addition to the MOD setter 84.
[0049]
The control unit 50d has a MPU interface 89 that controls data exchange with the MPU 40, a register that stores data written from the MPU 40, and outputs register information and control signals to control the entire video encoder unit 50. A control logic 90 and a priority chroma control circuit 96 for controlling image data priority and chroma information are provided.
[0050]
A control signal from the first I / F 62, the second I / F 63, and the third I / F 64 and register information from the system control logic 90 are input to the priority order chroma control circuit 96, and each I / F 62, 63, 64 is input. The operations of the first data selector 65, the third data selector 73, the time division data latch 74, and the like are controlled based on the priority order of image data input via the chrominance and the setting of chroma. The register information from the system control logic 90 is also input to the adder 67 and the image special processing calculator 75. The register information input to the adder 67 is the offset of the reference address of the color palette 68. Is a value that determines The register information input to the image special processing calculator 75 determines the degree of image superimposition processing in the calculator 75.
[0051]
The first I / F 62, the second I / F 63, and the third I / F 64 configuring the interface unit 50a have substantially the same configuration and function. The internal configuration is shown in the block diagram of FIG. These I / Fs 62 to 64 function to adjust differences in image signals input from the outside. That is, three kinds of image signals of 16.77 million color image data, 65536 color image data, and palette data referring to a color palette are input to the I / F. These signals are time-series signals every 8 bits, and the interface unit 50a adjusts the difference between these signals.
[0052]
As shown in the figure, the I / Fs 62 to 64 include a data buffer 101 for inputting image data from the outside, two signal adjusting units 102 and 103 for performing different processing on data temporarily stored in the data buffer 101, The first selector 104 that selects one of the signals from the signal adjustment units 102 and 103, the data buffer 105 that stores the output signal from the first selector 104, and different processing for the data stored in the data buffer 105 The two processing adjustment units 106 and 107, the second selector 108 for selecting one of the signals from the processing adjustment units 106 and 107, the data buffer 109 for storing the signal from the second selector 108, and the image data. A first control buffer 110 for inputting control data to be controlled A first selector control unit 111 that sets a selection operation of the selector 104 with reference to data stored in the buffer 110, a second control buffer 112 that stores control data in preparation for the next stage processing, and a second control A second selector control unit 113 that sets the selection operation of the selector 108 with reference to data stored in the buffer, and a third control buffer 114 that stores control data in preparation for processing from the image composition unit 50b are provided.
[0053]
The first data buffer 101 stores three types of image data, and has a function of sequentially storing data input according to a predetermined time series. Since the bus for inputting image data from the VDP unit 49 or the like is prepared for only 8 bits, an image of 65536 colors (for 16 bits) or 16.77 million colors (for 24 bits) like natural color image data. In the case of data, it cannot be taken into the video encoder unit 50 at a time. Therefore, since the image data is sent in several degrees, the data buffer inputs it and stores it for the next processing.
[0054]
The next-stage signal adjustment units 102 and 103 are means for adjusting the arrangement of such image data. FIG. 5 is a timing chart showing how various image data are input. As shown in the figure, the image data is attached with information of “surface number” indicating priority and “number of colors” indicating the color display as control data. When the number of colors specified is 16.77 million colors, as shown in the figure, the data input to the interface unit 50a is “Y1”, “Y2”, “U”, “V” by 8 bits. This signal indicates that the image data of the first dot is “Y1, U, V” and the data of the second dot is “Y2, U, V”. Therefore, the signal adjustment unit 102 stores the input data in the first data buffer 101 and adjusts the YUV data to 24 bits by 2 dots. Specifically, the signal adjustment unit 102 can be constituted by, for example, a data selector and an 8-bit × 3 × 2-stage register, and sequentially selects data input by the data selector and stores the data in each register. By doing so, the image data input in time series is arranged into complete 24-bit color digital data.
[0055]
On the other hand, as shown in FIG. 5, the configuration of the image data of 65536 colors is configured in units of 1 bit, and the image data of the upper 4 bits of U and the lower 4 bits of V follows the 8-bit Y data. Entered. Therefore, when the signal adjustment unit 102 determines that the image data is 65536 colors based on the control data, the data input by 8 bits is converted into 8 bits for Y data and 4 bits for U and V data. The data is divided and arranged into a total of 24 bits with the lower 4 bits set to 0.
[0056]
On the other hand, the signal adjustment unit 103 is a means for arranging palette data. As shown in FIG. 5, the signal adjustment unit 103 determines whether the palette data input with the pause unit interposed therebetween is 256 color display or 16 color display. In the latter case, 8-bit palette data is prepared in consideration of the palette bank number BK.
[0057]
Next, the set of processing adjustment units 106 and 107 will be described. The processing adjustment units 106 and 107 are performing special processing for an image, for example, preparation processing for image data indicating transparency. For example, if the process adjustment unit 106 determines that transparency is designated, the Y data is set to 0 for 16.770 million colors and 65536 colors, and the palette data or palette number is set to 0 for palette data. And These processes are performed by the process adjustment units 106 and 107, respectively.
[0058]
The image data (digital color data or pallet data) thus arranged is output to the image composition unit 50b. In the case of pallet data, as shown in FIG. 3, a data selector 65 and an adder 67 are used. Input to the color palette 68. The color pallet offset value is added to the other input of the adder 67 as register information from the system control logic 90. This offset value is added to the palette data as a value obtained by doubling the offset value. That is, as shown in FIG. 6, addition is performed with the offset value shifted by 1 bit to the left, and a total 9-bit color palette address is generated. As shown in FIG. 7, the color palette 68 has a size of 9 bits (512 addresses) in the address direction and 16 bits in the data direction.
[0059]
As shown in the drawing, the color palette 68 is composed of 8 bits of Y data, 4 bits of U data, and 4 bits of V data, and can output any 512 colors from 65536 colors. When this data is output from the color palette 68, the U and V data are separated, and a bit of “0000” is added to the lower order of each data to form 8 bits × 3 sets of image data. Eventually, whether 16.77 million color image data is input from the outside, 65536 color image data is input, or even when palette data is input, it is input to the second data selector 72. In this case, the signals are aligned to 8 bits × 3 sets of signals. It should be noted that U and V data are treated as signed data with “80” being 0 in binary-coded decimal notation.
[0060]
The operation of the image special processing arithmetic unit 75 that performs the superimposing process on the image data thus arranged will be described with reference to FIG. FIG. 8 is a block diagram showing in more detail a part that performs image superposition processing with the image special processing arithmetic unit 75 as the center. Compared to FIG. 3, in FIG. 8, among the registers in the system control logic 90, the coefficient register 130 that sets the degree of image superposition and the result of superimposing or superimposing specific colors on the surface with the lowest priority. A fixed color register 140 for designating the superimposition of one color with respect to is drawn as a block. The image data from the image signal control unit 45, the image data decompression unit 47, and the VDP unit 49 may be YUV data or pallet data. In the case of pallet data, 24-bit YUV is transmitted via the color palette 68. Although converted into data (hereinafter referred to as digital image data), in FIG. 8, this portion is simply described as a data selector 65 + 72. Further, the image data from each unit is accompanied by control data such as transparency information. In FIG. 8, the data flow when the control data is designated by the MPU 40 is omitted.
[0061]
In the embodiment, the transfer rate of image data from each unit is 200 nS cycles, and the transfer timing is synchronized with the dot clock (see FIG. 5) for each of the four input signals. One bit of transparent information bit is added to the digital image data as control data. The data selector 65 + 72 is a 4-input 1-output 24-bit selector that selects one input image data and outputs 24-bit digital image data. The priority order chroma control circuit 96 outputs a selection signal every 50 nS in synchronization with the system clock SLK having a frequency four times the dot clock to the data selector 65 + 72 and the coefficient register 130 according to the priority order designated by the MPU 40. To do.
[0062]
The coefficient register 130 is a register for determining a coefficient of calculation in the image special processing calculator 75 to be described later. Four coefficient registers 130 are prepared according to the number of times of superimposing digital image data. Of these, the three registers with register numbers 1, 2, and 3 are registers in which coefficients are actually set, and the register with register number 0 is a special register that specifies transparency processing when that register is specified. is there. The priority order chroma control circuit 96 outputs a selection signal to the data selector 65 + 72 in synchronization with the system clock SLK, and at the same time instructs the selection of the coefficient register 130. Here, when the transparency information is set to the value 1, the priority order chroma control circuit 96 selects 0 of the coefficient register 130. When 0 of the coefficient register 130 is selected, the third data selector 73 to which the output of the data selector 65 + 72 is connected is switched, and instead of the output of the image special processing arithmetic unit 75, the output of the data selector 65 + 72 remains as it is in time division. The data is output to the data latch 74.
[0063]
The time division data latch 74 is a circuit that latches the output data of the third data selector 73 by the system clock SLK, and as shown in FIG. 3 is a circuit for latching the output of the third data selector 73. The output of the time division data latch 74 is connected to the other input of the image special processing arithmetic unit 75 together with the pixel image data latch 76. The pixel image data latch 76 is a circuit that latches the output of the time division data latch 74 by the dot clock DCK, and is output after the time division data latch 74 latches the data of the fourth period as shown in the figure. The input data is latched by a pixel data latch signal from the system control logic 90. Accordingly, the output of the pixel image data latch 76 is updated every period of one pixel.
[0064]
The image special processing arithmetic unit 75 is an arithmetic unit that performs a hardware operation for synthesizing the output of the data selector first data selector 65 + the second data selector 72 and the output of the time division data latch 74 according to a predetermined arithmetic expression. is there. Since all the input data is 24-bit digital image data, the image special processing calculator 75 mixes the color image data at the ratio (coefficient) given by the coefficient register 130. The calculation of the image special processing calculator 75 is basically a calculation for multiplying two inputs by a coefficient designated by the coefficient register 130 to obtain the sum. That is, assuming that the output data of the data selector 65 + 72 is a, the latch data of the time division data latch 74 is b, and the coefficients designated by the coefficient register 130 are m and n, the output of the image special processing calculator 75 is The following equation (1) is obtained.
c = m · a + n · b (1)
[0065]
The actual calculation is performed on the YUV data, and the following expressions (2) to (4) are performed for each. Subscripts a, b, and c indicate correspondence with equation (1). The subscripts y, u, and v indicate coefficients for the YUV data.
Yc = my · Ya + ny · Yb (2)
Uc = mu. (Ua-80h) + nu. (Ub-80h) + 80h (3)
Vc = mv. (Va-80h) + nv. (Vb-80h) + 80h (4)
Here, the coefficients my, mu, mv, ny, nu, and nv are 9-stage values obtained by dividing the values (0 to 8) set to 1 to 3 in the coefficient register 130 by the value 8. Also, for U and V data, except that 80h is calculated in binary-coded decimal numbers, and 80h is added again, it is signed data that means a value of 0 when U and V data is 80h. This is to perform the attached operation. The calculation result by the image special processing calculator 75 is latched by the time division data latch 74 via the third data selector 73 each time.
[0066]
Next, the operation of the image composition unit 50b will be described with reference to FIGS. First, the MPU 40 sets priorities for image data from the units 45, 47, and 49 in the priority order chroma control circuit 96 via the system control logic 90. For example, the image data from the VDP unit 49, the image data from the image data decompression unit 47, and the image data from the image signal control unit 45 are set in descending order of priority. In this case, image data with a higher priority is displayed on the front side, and image data with the lowest priority is displayed on the back side. The set priority information is held until a new value is set next time.
[0067]
The image processing apparatus according to the present embodiment divides a period corresponding to one pixel into phases equal to or more than the number of image data, here, four, and processes the image data as follows by time division processing. Accordingly, the cycle of the system clock SCK is set to a value obtained by dividing the transfer rate (dot clock DCK) of the input digital image data by the number of input image data or by the number of image data or more. . In this embodiment, since the data transfer rate is 200 nS and the number of input image data is four (two sets of image data are transferred from the VDP unit 49), a value 50 nS obtained by dividing 200 nS by 4 is set to the system clock SCK. With a period of The priority chroma control circuit 96 detects the timing of phase 1 based on the dot clock DCK and the system clock SCK, and selects the image data having the lowest priority (here, the image data from the image signal control unit 45) as the data selector 65 + 72. To select.
[0068]
This data immediately enters the image special processing arithmetic unit 75, but 0 is selected as the coefficient register 130 only in the case of the phase 1 timing. In this case, the third data selector 73 displays the image special processing arithmetic unit 75. Since the output of the data selector 65 + 72 is selected instead of the output, the time division data latch 74 latches the image data with the lowest priority as it is. Instead of switching the third data selector 73 by the output of the coefficient register 130, a configuration is adopted in which an operation suppression signal is output from the system control logic 90, and the image special processing arithmetic unit 75 performs no processing, and the data selector 65 + 72. The output signal can be latched by the time division data latch 74 via the image special processing calculator 75 as it is.
[0069]
In any case, in phase 1, digital image data with the lowest priority is latched in the time division data latch 74. At the same time, the timing advances to phase 2, and the data selector 65 + 72 selects the digital image data having the second lowest priority. The selected digital image data is input to the image special processing calculator 75. The output of the time division data latch 74, that is, the digital image data with the lowest priority is added to the other input of the image special processing calculator 75. The image special processing calculator 75 performs the calculations of the above-described equations (2) to (4). Specifically, after multiplying each of the Y, U, and V data by a coefficient (an integer from 0 to 8), two data are added for each color, and this is divided by 8 (3 bits (Shift to the right) is performed.
[0070]
As a result of this calculation, the two digital image data have a mixing ratio of 0/8: 1/8, 1/8: 7/8, 2/8: 6/8, ... 7/8: 1/8, 8 / It will be superimposed at 8: 0/8. An independent coefficient is given to the coefficient register 130 for each image data by the MPU 40. The priority chroma control circuit 96 outputs an image data selection signal, selects a coefficient corresponding to the selected image data, sets the coefficient in the coefficient register 130, and outputs it to the image special processing calculator 75. The coefficient is 8 bits, the upper 4 bits are the coefficients for the digital image data a, and the lower 4 bits are the coefficients for the digital image data b. Values of 0 to 8 are set for the upper 4 bits and the lower 4 bits, respectively.
[0071]
Further, at the time of calculation, the transparent process is performed as follows. The image special processing arithmetic unit 75 checks the transparency information included in the control data. If the transparency information is in the transparent state, the digital image data output from the time division data latch 74 is time-division again via the third data selector 73 as it is. The data is output to the data latch 74. Then, since the digital image data output from the data selector 65 + 72 does not affect the output of the image special processing arithmetic unit 75 at that time, the data appears to be completely transparent, and the transparent processing is realized. In addition, the transparent processing can be realized by not outputting a latch pulse to the time division data latch 74. In this case, since it is not necessary to pass the digital image data through the image special processing calculator 75 and the third data selector 73, calculation processing of the image special processing calculator 75 and switching thereof can be facilitated.
[0072]
The output of the image special processing calculator 75 is latched by the time division data latch 74. Latching is performed every 50 nS at the rising edge of the system clock SCK. In phase 3 and phase 4, the corresponding priority digital image data is processed in the same manner, and at the end of phase 4, the time division data latch 74 latches the final result. The final result data is latched by the pixel image data latch 76 at the end of the next phase 1. The latch pulse used by the pixel image data latch 76 can be generated by delaying the dot clock DCK by one cycle of the system clock SCK by a delay circuit (not shown).
[0073]
If the image data is divided by a number larger than the number of image data in one pixel period, for example, a process of superimposing the color stored in the fixed color register 140 on the image for which the superimposition process has been completed may be performed. Is possible. If the color designated by the fixed color register 140 is set in the time division data latch 74 before selecting the image data with the lowest priority, one color is assigned to the image data with the lowest priority. It is also possible to superimpose.
[0074]
The processing for one dot is completed as described above, and the digital image data is subjected to mixing processing and transparency processing, and is output from the pixel image data latch 76 for use in the next processing (D / A conversion, etc.). These series of processes are continuously performed, and data is processed without interruption. Therefore, processing can be performed in real time, whether it is a still image or a moving image.
[0075]
As described above, in this embodiment, the case of 8-bit digital image data in the YUV format has been described. However, even in the case of digital image data of three primary colors such as RGB, the same configuration can be realized.
[0076]
According to the video game apparatus 20 of the present embodiment, it is possible to perform processing such as overlapping or mixing image data stored in the CD-ROM 21 or image data generated by the VDP unit 49. By mixing, only a specific place on the screen can be displayed brightly to produce an effect like a spotlight, or darkened to produce an effect like a shade. These are very effective for giving a stereoscopic effect to an image of a video game or the like. In addition, since the coefficient can be set freely, it is possible to switch images slowly by fade-out / in, which is often used in TV programs and the like, instead of switching images when switching images. As a result, an emotional effect can be produced at a scene change or the like, and a high-quality image can be created.
[0077]
Moreover, the image processing apparatus incorporated in the video game apparatus 20 of the present embodiment can further mix the mixed data, so that, for example, a mountain can be seen beyond the glass window of the train as viewed through the glass window. Multiple translucent processing can be easily realized. In addition, since the coefficient can be changed for each image data using the coefficient register 130, the transparency can be changed for each glass window.
[0078]
Such image superimposition processing is achieved by repeatedly using one image special processing arithmetic unit 75. Therefore, the image special processing arithmetic unit 75 can be used even if the number of image data increases. Main parts such as the divided data latch 74 and the pixel image data latch 76 need not be changed. In addition, the circuit is compact because only one image special processing arithmetic unit 75 is required, although the calculation is repeated many times for one dot.
[0079]
Further, according to the image processing apparatus of the present embodiment, whether it is 16.77 million color image data or 65536 color image data, or palette data such as 256 colors, 16 colors, or 4 colors, is overlapped in exactly the same way. The matching process can be performed. In the case of palette data, since it is converted into 24-bit data by the color palette 68 and then subjected to processing by the image special processing computing unit 75, there is no change in the process of image superposition.
[0080]
【The invention's effect】
As described above, according to the first image processing apparatus of the present invention, the image data input in units of pixels is transmitted during one input period divided for the pixels.Image overlapReceive processing.OverlappingprocessingGanaThe image data that has been saved is retained, and is divided in other divided input periods.OverlapTo be processed. Thus, the process is performed during the divided period, and when the input period for the pixel ends, the result at the end of the process is output as processed image data for the pixel. As a result, according to the image processing device of the present invention, it is possible to obtain processed image data obtained by performing processing related to each other on at least two or more image data in an input period of one pixel. Since these processes are completed during the input period for the pixel, it is obvious that the moving image can be handled as it is. Of course, it can also be applied to still image processing.
[0081]
On the other hand, according to the image processing method of the present invention having the above configuration, it is possible to perform processing such as superimposing a plurality of image data extremely easily.
[0083]
  In the present invention,SecondImage processing equipment,sameThere is an advantage that the same time-division processing as that of the first image processing apparatus is performed on the signals arranged in the same form, and predetermined processing between a plurality of images can be easily performed.
[Brief description of the drawings]
FIG. 1 is an external view of a video game apparatus 20 incorporating an image processing apparatus as an embodiment of the present invention.
FIG. 2 is a block diagram showing the internal configuration in the same manner.
FIG. 3 is a block diagram showing an internal configuration of a video encoder unit 50 which is an image processing apparatus as an embodiment.
4 is a block diagram showing an internal configuration of an interface unit 50a in the video encoder unit 50. FIG.
FIG. 5 is a timing chart showing input timing of image data.
FIG. 6 is an explanatory diagram showing how palette addresses are combined.
FIG. 7 is an explanatory diagram showing a data configuration example of a color palette 68;
FIG. 8 is a block diagram illustrating a configuration of an image composition unit 50b.
FIG. 9 is a timing chart showing an operation in the image composition unit 50b.
FIG. 10 is an explanatory diagram illustrating a state of conventional image superposition.
FIG. 11 is a block diagram illustrating a configuration example of a conventional image processing apparatus.
[Explanation of symbols]
20. Video game device
21 ... CD-ROM
22 ... Game console
24, 26 ... Gamepad
24a ... switch
24b ... Cursor stick
28 ... Color TV
30 ... Video signal cable
31 ... Cover
32 ... CD-ROM drive
34 ... Speaker
36 ... SCSI bus
40 ... MPU
40a ... Calculation unit
40b ... Controller
41 ... Main memory
42 ... ROM
43 ... BUS
45. Image signal control unit
45a ... MPUI / F
45b SCSI controller
45c ... AFFIN converter
45d: Graphic controller
45e ... Sound controller
47. Image data decompression unit
49 ... VDP unit
50 ... Video encoder unit
50a ... Interface section
50b ... Image data composition unit
50b ... Image composition unit
50c ... DAC section
50d ... Control section
52. Audio data output unit
52a ... ADPCM part
52c ... Mixer
55 ... K-RAM
59 ... V-RAM
60 ... NTSC converter
62 ... 1st I / F
63 ... 2nd I / F
64 ... 3rd I / F
65: First data selector
67 ... Adder
68 ... Color palette
72. Second data selector
73: Third data selector
74 Time-division data latch
75. Image special processing arithmetic unit
76: Pixel image data latch
81, 82, 83 ... D / A converter
84 ... MOD setting device
86. Synchronization signal generation circuit
89 ... MPU interface
90 ... System control logic
96... Priority chromach control circuit
101: First data buffer
102, 103 ... Signal adjustment unit
104... First selector
105: Data buffer
106, 107 ... processing adjustment unit
108 ... second selector
109: Data buffer
110: First control buffer
111... First selector control unit
112 ... Second control buffer
113 ... Second selector control unit
114: Third control buffer
130: Coefficient register
140: Fixed color register

Claims (11)

An image processing apparatus for inputting a plurality of image data and processing each image data,
Input means for inputting the image data in units of pixels;
Pixel processing means for dividing the input period of the pixels and performing predetermined processing on the image data of the pixels input during one divided input period;
Division processing data holding means for holding the processed image data and subjecting it to the predetermined processing in the other divided period in the pixel processing means;
An output means for outputting the retention result of the divided processing data retaining means as processed image data for the pixel at the end of the input period;
The pixel processing unit is an image processing apparatus that performs a process of overlapping an image between image data input from the input unit and data held by a divided processing data holding unit as the predetermined processing. The image processing apparatus according to claim 1,
An image processing apparatus comprising a selection unit that selectively selects image data input from a plurality of external devices according to division of an input period in the pixel processing unit and supplies the selected image data to the image processing unit.   The image processing apparatus according to claim 1, wherein the pixel processing unit includes a unit that divides the input period of the pixel into four. The image processing apparatus according to claim 1,
In addition to providing parameter generation means for giving parameters relating to image processing,
The pixel processing means is an image processing apparatus which is means for determining the processing based on the parameter. The image processing apparatus according to claim 1,
Parameter generating means for providing a parameter relating to the overlap of two or more images;
The image processing apparatus, wherein the pixel processing means includes means for determining a mixing ratio between the image data input from the input means and the retained data of the divided processing data retaining means based on the parameter.   6. The image processing apparatus according to claim 4, wherein the parameter generation means includes parameter setting means capable of specifying and setting the color component constituting the pixel.   6. The image processing apparatus according to claim 4, wherein the parameter generation means includes division parameter setting means capable of setting the parameter for each period obtained by dividing a pixel input period. The image processing apparatus according to claim 5, wherein
Transparency parameter determination means for determining whether the parameter generated by the parameter generation means is a parameter for designating transparency;
An image processing apparatus comprising: a transparent processing unit that outputs the stored content as it is to the divided processing data storage unit when it is determined that the parameter is a parameter that specifies transparency.   The image processing apparatus according to claim 2, wherein the selection means has a period for selecting image data equal to or less than a period obtained by dividing an input period of one pixel by the number of external apparatuses. An image processing method for inputting a plurality of image data and processing each image data,
The image data is input in pixel units,
Dividing the input period of the pixel;
Image overlap processing is performed on image data of pixels input during one divided input period,
Holding the processed image data, and subjecting the processed image data to the process of overlapping the images in the other divided period;
An image processing method for outputting the held image data as processed image data for the pixel at the end of the input period. Enter a plurality of signals for displaying an image of color, an image processing apparatus for performing predetermined processing between image signals of the plurality of displays,
Signal input means for inputting the plurality of signals in units of pixels;
An image determination signal output means for determining whether the input signal is an image signal directly representing an image or an image signal representing an image through a predetermined conversion process, and outputting a determination signal;
Image signal aligning means for performing the predetermined conversion process on the input signal based on the determination signal and aligning the plurality of signals in the same form capable of the predetermined process;
Pixels for performing predetermined processing on image data of pixels in one signal input during one divided input period, by dividing an input period of pixels as a unit of the plurality of signals arranged in the same form Processing means;
Division processing data holding means for holding the processed image data and subjecting it to the predetermined processing in the other divided period in the pixel processing means;
An image processing apparatus comprising: an output unit that outputs a holding result of the divided processing data holding unit as processed image data for the pixel at the end of the input period.

Images