JP4374286B2 - Image synthesizer - Google Patents

Description

  The present invention relates to an image synthesizing apparatus that synthesizes and outputs images input from a plurality of input channels.

  Such an image synthesizing apparatus is used in the following industrial fields. For example, it is possible to capture an image of 360 ° in the horizontal direction with a plurality of cameras, synthesize a plurality of input images, and output as a panoramic image. It is also possible to combine multipoint viewpoint input images and output them as single point viewpoint images, which can also be used to synthesize images from which blind spots are eliminated. Further, in a monitoring system using a plurality of monitoring cameras, input images input from a plurality of cameras are combined and output as one output image to monitor suspicious person intrusion or the like.

  In addition, it can easily add DVE (Digital Video Effect) effects that are frequently used in image (still image) filing systems, CG (computer graphics), broadcasting, etc. It is also used for (for example, car navigation display). For example, using the pixel rearrangement function, image effects such as image replacement such as wipe and roll, upside down image flip, picture in picture, multi-split screen display, superimposition of characters and characters, and dissolve by synthesis Can be given easily.

  As a prior art of such an image synthesizing apparatus, for example, an image synthesizing apparatus disclosed in Patent Document 1 below is known. This image synthesizing apparatus superimposes N (2 ≦ N) partial images on one image based on the hidden relationship between the N partial images. According to this image synthesizing apparatus, images can be synthesized at high speed, and the apparatus can be easily expanded and finely synthesized as the number of synthesized images increases.

However, the apparatus is very large and is not suitable for personal use, monitoring system, and mobile use.
JP 2003-323635 A

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image composition apparatus capable of freely composing images at high speed and at low cost.

In order to solve the above problems, an image composition device according to the present invention provides:
Multiple input channels into which images are input,
An input memory that stores image data input to each input channel in a readable manner;
Pixel rearrangement that is provided corresponding to each input channel and reads out any pixel constituting the input image from the input memory based on preset address information, arranges it at any position in the output image, and outputs it Equipment,
An image synthesis unit for synthesizing pixels read out as an output image based on synthesis ratio information set in advance;
The pixel rearrangement device is configured to perform image synthesis processing based on synthesis ratio information for each pixel in the image synthesis unit after reading out and rearranging all the pixels constituting the output image. It is.

  The operation and effect of the image composition apparatus will be described. A plurality of input channels through which images are input are provided, and an input memory for storing the input image data in a readable manner is provided in each channel. For example, a high-speed SRAM is used as the type of memory for configuring the input memory, but it is not limited to a specific type. In order to perform image synthesis, an arbitrary pixel is read out from the input memory based on preset address information, and this is output to an arbitrary position in the output image. Such pixel output is called pixel rearrangement, and a device for doing this is a pixel rearrangement device. This pixel rearrangement device is provided corresponding to each input channel.

  On the other hand, in order to perform image composition, an image is composed of a certain pixel and another pixel corresponding to the pixel position based on a predetermined composition ratio. The number of pixels to be combined when combining images can be set as appropriate, and is not limited to a specific number.

  Then, after all the pixels constituting the output image are read and rearranged by the pixel rearrangement device, the image synthesis unit performs image synthesis using the above-described synthesis ratio. As described above, since the composition is performed after the rearrangement of pixels, the channel for inputting an image can be easily expanded. Further, when a pixel is read out from the input memory and output, if a synthesis process using a synthesis ratio is performed, a synthesis circuit is required for each pixel rearrangement device, and the circuit configuration becomes complicated. Since image composition is performed at the final stage after rearrangement, there is an advantage that the circuit configuration can be simplified. As a result, it is possible to provide an image synthesizing apparatus that can synthesize images freely at high speed at low cost.

  In the present invention, a synthesis table that is a collection of synthesis information corresponding to individual pixels constituting an output image is provided, and each synthesis information corresponding to each pixel includes the address information of the pixel to be read and the pixel And the above-mentioned synthesis ratio information.

  For example, if the output image has a configuration of 640 × 480 pixels by VGA, the synthesis table similarly has 640 × 480 synthesis information. Each combination information includes at least address information of a pixel to be output at the position of the combination information and combination ratio information of the pixel. In the data structure of the address information and the composition ratio information, various methods can be considered for the address description method and the composition ratio numerical value (representing at a predetermined level or%). It is not limited to the data structure.

In the present invention, a data memory for storing a synthesis table;
A buffer for sequentially reading out and storing data of the synthesis table from the data memory;
The data stored from this buffer is read out and configured to be sent to each pixel rearrangement device as a composite stream.
Further, it is preferable that a synchronization signal generation unit is provided for transmitting the synthesized stream from the buffer in a form synchronized with the output image.

  As a set of synthesis information, the above-described synthesis table is stored in a data memory, and data is sequentially read from the data memory and stored in a buffer. This buffer adjusts the data transmission speed. Specifically, the buffer is sent from the buffer to each pixel rearrangement device (for each input channel) in synchronization with the output image. This transmitted data is called a composite stream. As described above, since the composition table has the same frame configuration as the output image, for example, the composition information elements for the horizontal period are sequentially read out, the speed is converted in the buffer, and then the composition stream is read. Can be sent out as. Thus, by synchronizing with the output image, it is not necessary to synchronize the output again with a frame buffer or the like when actually outputting as an output image. Therefore, it is possible to reduce the delay of the image between input and output.

  In the present invention, the output information is gradually generated by sequentially replacing the address information portion of the composite information of the composite stream with the pixels read by the pixel rearrangement device.

  In this way, if the configuration is such that the pixels are sequentially replaced, there is no need to send a dedicated stream for the output image, and the number of connected I / Os can be reduced, simplifying the circuit configuration and realizing low cost. can do.

In the present invention, the pixel rearrangement device includes a pixel hit determination unit that determines whether or not a pixel corresponding to the address information hits the input image;
Regardless of the determination result by the pixel hit determination unit, an address generation unit that generates an address for reading a pixel from the input memory;
A pixel output control unit that controls whether or not to output the read pixel as a pixel constituting the output image,
The pixel output control unit is configured to discard a read pixel without outputting it when a pixel that has not been hit is read.

  When trying to read a pixel based on address information, a pixel of another channel may be designated instead of a pixel of an input image of the channel. In this case, since the input image is not hit, it is not necessary to read out the pixel. However, if control is performed so that the pixel is read or not read out, the pixel cannot be read out at high speed. Therefore, the pixel is always read regardless of whether the pixel is hit or not. Thereby, high-speed processing can be performed. Since the pixel output control unit is provided, there is no problem even if a non-hit pixel is read out because it is discarded as it is not output as an output image. Therefore, it is only necessary to read out a pixel at an appropriate address for a non-hit pixel.

  A preferred embodiment of a pixel rearrangement device according to the present invention and an image composition device using the same will be described with reference to the drawings. The overall configuration of this image composition apparatus will be described with reference to the block diagram of FIG.

<Configuration of image composition device>
This image synthesizing apparatus is configured to arbitrarily combine an arbitrary pixel X of an arbitrary input image and an arbitrary pixel Y of an arbitrary input image among an input image of a plurality of channels (moving image / still image). It can be combined and placed (rearranged) at an arbitrary position in the output image. This image composition device can perform image composition at high speed and can easily expand the number of composite image inputs.

  The NTSC input block 1 can capture N channel input images and is provided with N channel NTSC decoders 10. In the present invention, the number of channels N can be set as appropriate, and can be arbitrarily increased or decreased. The image data decoded by the NTSC decoder 10 is sent to the next memory block 2 in a 4: 2: 2 format, for example, in which luminance and color signals are 8 bits each.

  The memory block 2 is provided with a frame memory 20 (corresponding to an input memory) for writing an input image in units of frames. Three frames of the frame memory 20 are provided for each channel. In other words, the input images are written in time sequence in the three-frame frame memories A, B, and C as shown in FIG. The memory block 2 of each input channel has a block configuration as shown in FIG. 10, and a write surface is designated by the switch A, and a readable surface for rearrangement is designated by the switch B. The readable surface is a surface (ready state) for which writing has been completed immediately before. In addition to this, writing is about to start or a writing surface (busy state) and a standby surface (wait state) in case the writing surface changes during relocation are prepared. Yes. These three states are cyclically moving on each surface, and rearrangement is constantly performed using pixels on the ready surface (latest input image frame).

  In this way, each input channel is provided with a frame memory for three planes in order to cyclically switch between a ready (written) memory, a busy (writing) memory, and a standby memory. Yes. By using a three-plane frame memory, the asynchronous state of each input image can be absorbed.

  Specifically, the frame memory 3 plane is configured by using six high-speed SRAMs of 4 Mbits per input image. Writing is performed at a frequency (12.2727 MHz) synchronized with the NTSC decoder 10, and reading requires simultaneous output to a plurality of combined streams (when there are a plurality of output channels and the number of pixels to be combined). It is performed several times (for example, if output to three composite streams at the same time, quadruple speed = 49.0908 MHz). The composite stream will be described later.

  The image synthesis block 3 has a function of interpreting a synthesis table (table data in which address information for pixel rearrangement is recorded), reading out corresponding pixels from the frame memory 20, and synthesizing the read pixels. Have As will be described in detail later, the synthesis table is composed of an image of the output frame image, and the pixel read from the input image is replaced (rearranged) at each pixel position of the output frame image. In this way, an output image is generated. In FIG. 1, since there are three output channels, three composite streams for generating an output image are also shown as S1, S2, and S3. A pixel rearrangement device 30 according to the present invention for rearranging pixels in the composite stream is arranged.

  Also, an overlay pattern generator 31 is provided, and an overlay image can be synthesized as with other input images. There is no difference between the input of the overlay image and the other image input in pixel rearrangement and image synthesis. Therefore, the overlay pattern generator 31 can be arranged at a position other than the position shown in FIG. 1, for example, upstream of the composite stream. The overlay pattern generator 31 is not always necessary.

  The output image synchronization signal generator 32 has a function of synchronizing the read synthesis table with the output image. In the synthesis table block 4, synthesis table data memories 40 and 41 (hereinafter referred to as data memories) for storing the synthesis table are prepared. The capacities of the data memories 40 and 41 can be set according to the data size of the synthesis table and the number of synthesis tables stored. As will be described later, if the number of synthesis information is 640 × 480 and each synthesis information is composed of 3 bytes, the data size of one synthesis table is 640 × 480 × 3 = 921,600 bytes. Become. Based on this, it is possible to set the capacity corresponding to the number of synthesis tables to be stored.

  In the present embodiment, a configuration example in which image synthesis of two pixels is performed, the data memory 40 is prepared for the A side and the data memory 41 is prepared for the B side. The same data is stored in the respective data memories 40 and 41. For example, if 64 synthesis tables are stored in the compact flash (registered trademark) 5, the 64 synthesis tables can be stored in the data memories 40 and 41 in the same manner. Of course, the present invention is not limited to this. 128 composite tables are stored in the compact flash (registered trademark) 5, 64 of which are stored in the data memory 40 for the A plane, and the remaining 64 composite tables are stored. The table may be stored in the data memory 41 for the B side.

  Of the synthesis tables stored in the respective data memories, the tables designated by the microcomputer or the like are read out on both the A side and the B side at an ultra high speed of 60 MHz, for example. The read synthesis table is sent out in synchronization with the output image by the FIFO buffers 33 and 34 (first-in first-out buffer). In the present specification, the state stored in the data memories 40 and 41, or the portion of the data string read out from the data memory 40 and 41 is defined as a synthesis table. A stream synchronized with the timing is defined as a composite stream.

  The synthesis table is input to the data memories 40 and 41 via the compact flash (registered trademark) 5. The composite table created in advance using a personal computer or the like can be stored in the compact flash (registered trademark) 5. The compact flash (registered trademark) 5 can store, for example, 64 kinds of synthesis tables. All the synthesis tables in the compact flash (registered trademark) 5 are transferred to and stored in predetermined areas in the data memories 40 and 41 by an event such as power-on.

  Of the large number of synthesis tables stored in the data memories 40 and 41, the A-side synthesis table and the B-side synthesis table are designated by a microcomputer or the like, whereby the A-side / B-side synthesis stream is specified. Can be shed as Different image synthesis can be performed by changing the designated synthesis table. An external storage medium other than CompactFlash (registered trademark) 5 may be used.

  The NTSC encoder 6 outputs the output signal synthesized by the image synthesis unit 35 in the form of an NTSC signal. Details regarding the image composition unit 35 will be described later. There are as many NTSC encoders 6 as the number of output channels. The number of output channels can be set as appropriate.

<Composite table>
Next, a synthesis table in which address information is described will be described. Using this synthesis table, any pixel can be read from the input image and rearranged at any position in the output image. First, prior to creating the synthesis table, it is necessary to map the pixels of each input image in advance. This will be described with reference to FIG. In the following description, it is assumed that the number of constituent pixels of the input image is 640 × 480. Therefore, it can be mapped as 0 to 640 × 480-1 according to image scanning. In order to deal with a plurality of image inputs, it is possible to distinguish them by mapping after this, or by adding an input image address separately. The example shown in FIG. 2 shows the mapping of the serial number configuration. Note that OVL is an offset by a primary color (RGB) designation area for overlaying a character or the like. For example, when 16-bit RGB is used for overlay, OVL = 65,536 (= 2 ¹⁶ ).

  Next, the composition of the synthesis table will be described with reference to FIG. As can be seen from this figure, the synthesis table is the same image as the output frame image. In FIG. 3, the notation L1P1 means the first pixel of the first line. This corresponds to synthesis information for one pixel. If the output image is based on VGA, it is composed of composite information of 640 × 480. Each combination information includes pixel address information (information for specifying a pixel) mapped as shown in FIG. 2 and combination ratio information. This address information and composition ratio information constitute “composition information”. For example, the combination information in L1P1 is configured by address information of an input image to be rearranged at this position and information on a combination ratio indicating the ratio of pixel combination at the pixel at the address. Then, the synthesis table shown in FIG. 3 is sent out as a synthesis stream, and the address information portion of each synthesis information is replaced by the read pixel. That is, the output image is finally generated by the replacement.

  Each combination information constituting the combination table will be described in more detail. In the present embodiment, as the synthesis table, two independent synthesis tables (A-side table / B-side table) can be set for 3-channel output. The synthesis of a pixel having an arbitrary output is performed using the synthesis information of the corresponding pixel in the A-side table and the synthesis information of the corresponding pixel in the B-side table. Note that the composition ratio information when performing image composition is determined by the value incorporated in the pixel of the B-side table, ignoring the value incorporated in the pixel of the A-side table. The level of the composition ratio can be set as appropriate, and can be, for example, eight levels from 0 (transparent) to 7 (opaque). Therefore, for example, if the composition ratio of the pixel in the B-side table is 6, the composition ratio of the pixel in the A-side table is automatically 7-6 = 1.

  Each component of the synthesis table has a data configuration of 3 bytes. This will be described with reference to FIG. As described above, the three bits D23 to D21 represent an eight-stage composition ratio. However, when the synthesis table is designated as the A-side table, these 3 bits are invalid and depend on the synthesis ratio information of the B-side table. The specific data of the composition ratio is not limited to this.

  D20 to D0 designate the read address of the input image or the overlay image 16-bit color code. As described with reference to FIG. 2, the address of the input image has the origin at the upper left of the image, the address goes up in the right / lower direction (in the manner of a scanning line of a television), and the address of the lower right pixel is 640 × 480-1. The input channel is specified by an offset of 640 × 480 (n−1).

For example, when specifying the fifth pixel of the input image 2,
Address = 640 × 480 × (2-1) +4.

For example, when the input is 5 channels and there is a 16-bit color overlay input, the following address is specified.
16-bit color: 0 to 65,535 address input image 1: 65,536 to 372,735 address input image 2: 372,736 to 679,935 address input image 3: 679,936 to 987,135 address input image 4: 987, 136 to 1,294, 335 Input image 5: 1,294,336 to 1,601,535 Invalid address: 1,601,536 to 2,097,151 If such addressing is used, It is possible to easily cope with the expansion of input channels. In other words, the addition of input channels may be performed by adding upward addressing, and can be handled without changing the address of the overlay input.

<Details of image synthesizer>
The image synthesizing apparatus according to the present invention provides an apparatus that can synthesize images freely and at high speed for personal use and business use. Composition information for performing image composition is provided in the form of a composition table as described above. This composite table can be created in advance by using a computer such as a personal computer. The composite table data memory 40 can be created via an external medium (Compact Flash (registered trademark) 5 in FIG. 1). , 41 can be stored. Data transfer from the external medium to the data memories 40 and 41 can be triggered by, for example, turning on the power to the apparatus, pressing the reload switch, or the like. As the composition table, a large number of composition tables can be prepared in the data memories 40 and 41 in accordance with the contents of the desired image composition. Which synthesis table is used can be switched using, for example, a microcomputer. This may be fixed or switched by temporal control. By such switching control, it is possible to express a synthetic motion such as DVE.

  When image synthesis is performed, the synthesis table data memories 40 and 41 are provided with the number of faces of the number of pixels desired to be synthesized for one pixel of the output image. In the embodiment of FIG. 1, the number of pixels to be combined is two, and two data memories 40 and 41 are prepared for the A side and B side. For example, when combining arbitrary pixels of the output image with arbitrary three pixels, three surfaces (A surface, B surface, C surface) are prepared.

  In FIG. 1, S1, S2, and S3 are shown as composite streams for each output channel. Conceptually, as shown in FIG. 5, when two pixels are combined, it can be considered that there are two combined streams S1-1 and S1-2. That is, there are two streams: a composite stream S1-1 from the data memory 40 for the A side and a composite stream S12 from the data memory 41 for the B side.

  In this way, the number of faces for the number of pixels to be combined is required. This is because it is necessary to specify each address of the pixel of the input image to be synthesized for one output pixel. In FIGS. 1 and 5, two surfaces (A surface and B surface) are used to synthesize two pixels, but it is not necessary to use physically different data memories. For example, it is possible to handle two virtual tables as if they exist virtually by reading two synthesis tables at high speed in a time-sharing manner from the one plane.

  If two physically separate data memories are prepared, there is a problem that I / O increases corresponding to the number of output channels set. That is, in the case of this embodiment, since the synthesis information constituting the synthesis table is composed of 3 bytes, it becomes a 3 byte wide synthesized stream. Therefore, if there are 3 output channels, I / O of 3 bytes × 2 sides × 3 channels = 18 bytes (= 144 lines) is required. Therefore, in order to reduce I / O, it is preferable to read the synthesis table from the data memory for one surface at a high speed.

  In the following description, a configuration in the case where two data memories 40 and 41 are physically prepared will be described. The synthesis table data is read from the data memories 40 and 41 in synchronization with the vertical synchronization signal and horizontal synchronization signal of the output image. This is due to the function of the output image synchronization signal generator 32 shown in FIG. By using a synchronous DRAM for the data memories 40 and 41, the data of the synthesis table can be read out at a very high speed. Therefore, one horizontal period (information for 640 pixels) of data of two or more synthesis tables can be read out in one horizontal synchronization period of the output image. If the output image is an NTSC image, since one horizontal synchronization period is 63.5 μs, data reading for 640 pixels can be performed 4 to 5 times.

  The data of the A table / B surface composite table read at high speed is speed-controlled in a form synchronized with the output image by the speed conversion unit at the subsequent stage. This speed conversion function can be realized by using FIFO buffers 33 and 34 as shown in FIG. The above points are represented by the timing chart of FIG. That is, within one horizontal period of the output image, the data of the synthesis table for 3 channels output from the A-side data memory 40 (indicated by C1TX, C2TY, C3TZ, each of data for 640 pixels in one horizontal period) Are read at high speed. Similarly, data (C1Tx, C2Ty, C3Tz) for three output channels are also read from the data memory 41 on the B surface at high speed. The combined table data for one horizontal period is stored in the FIFO buffers 33 and 34 which are line buffers. Then, the speed conversion is performed by the FIFO buffers 33 and 34, and the data is transmitted from the FIFO buffers 33 and 34 in synchronization with the output image. The length of 640 pixels of data for A-side and B-side at the time of transmission is extended so as to coincide with the horizontal effective video period. In FIG. 6, for convenience, the reading of the synthesis table and the state after the speed conversion are drawn on the same time axis. However, of course, the data is sent after the speed conversion is actually delayed by about one horizontal period. Become. In FIG. 6, the A-side composite stream in the output channel 1 is indicated by S1-1, and the B-side composite stream is indicated by S1-2. The same applies to the other output channels. In this way, the synthesis table becomes a synthesized stream by sequentially sending out data for one horizontal period.

  Furthermore, a detailed specific example of this composite stream will be described with reference to FIG. 6A. Here, data for one composite information (= 24 bits) is shown. As described above, the synthesis table is read at a high speed (60 MHz) for both the A side and the B side. Each composite stream is a stream having a width of 24 bits (= 3 bytes). However, as the number of output channels increases, the number of I / Os increases accordingly, so as shown in FIG. And B side are temporally multiplexed in 12.2727 MHz (corresponding to one pixel period of NTSC). Thereby, the number of I / O can be halved.

  Further, as shown in FIG. 6A (b), temporal multiplexing is performed by dividing into upper words AL and BL (16 bits) and lower words AU and BU (16 bits). In this case, since the synthesis information is 24 bits, unused bits are generated. This bit can be used as a management byte indicating whether or not rearrangement by the pixel rearrangement device 30 described later has been performed. If it is made like FIG. 6A (b), the speed | rate of a synthetic | combination stream will be 4 times, and I / O can be reduced further.

  With the composite stream sent out in this way, the operation of reading out the corresponding pixel from the frame memory 20 in the ready state is continuously performed in the channel of each input image according to the address information described in the composite table (address generation unit). 30A). That is, the pixel rearrangement device 30 shown in FIG. 7 sequentially rearranges the pixels. In FIG. 1, one pixel rearrangement device 30 is shown for each input image (per output channel), but there are two composite streams as shown in FIG. The number of pixel rearrangement devices 30 corresponding to the number of combined streams is provided.

  As shown in FIG. 7, in the pixel rearrangement device 30, when reading pixels from the frame memory 20, addressing is performed at high speed and randomly. Therefore, the frame memory 20 is composed of a high-speed SRAM. Further, in reading out the pixels, the address information of the synthesis table is given to the frame memory 20 regardless of whether the addressing hits. That is, regardless of whether or not a pixel read from the frame memory 20 is necessary, high-speed reading is realized by always reading the pixel.

  That is, some pixels are read from the frame memory 20 even when the pixel hit determination unit 30B determines that the pixel address is not hit. In this case, the address of the pixel to be read out may be basically any address, but for example, the upper left address of the image (the minimum physical address of the input image) can be designated. It should be noted that pixels that are read out in spite of such a hit are not rearranged as an output image under the control of the switch (function of the pixel insertion unit 30C) in FIG. From this point of view, the pixel at any address may be read out. As described above, a function of a pixel output control unit that controls whether or not to output the read pixel is provided (see the pixel insertion unit 30C).

  When high-speed reading is performed from the frame memory 20 which is an SRAM, it is difficult to control whether or not data is read or not by controlling the OE # (output enable control line) that controls reading. Therefore, the OE # control line is constantly fixed at the L (low) level, and only the address is sequentially switched to read out the pixel at the desired address. In this way, high-speed reading is realized by reading both hit pixels and unnecessary pixels.

  As described above, there are as many synthesized streams as there are pixels to be synthesized. For example, if a maximum of two pixels are combined, processing for two combined streams is required within one pixel period (for example, 12.2727 MHz for NTSC). When the number of channels of the output image becomes two, the number of pixels read out within one pixel period further increases. Since such high-speed processing is required, a configuration for reading out pixels regardless of whether or not the addressing information is hit is a very effective method.

  By performing processing within one pixel period, all pixel readout and image synthesis (which will be described later) are performed in real time, so a frame memory is prepared immediately before the image is finally output, and the output image is It is not necessary to adopt a configuration that sequentially outputs the constructed frames. This is a very important matter in reducing the delay between input and output.

  In addition, although the example which makes the speed | rate of a synthetic | combination stream 4 times was already demonstrated, this is performed in order to reduce the number of I / O crossing between IC, and in the pixel rearrangement apparatus 30, conversely, the speed | rate of 12.2727MHz. After rearrangement, the data is transferred to the next stage again at a high speed of 49.0908 MHz.

  As described above, the combined stream that has passed through the N channels of the image input is replaced with input pixels except for the address to be overlaid. In the case of 4: 2: 2, the luminance / color difference obtained by passing the addressing data through an RGB matrix circuit (primary color / color difference conversion) is replaced as image data. In the case of the RGB method, the addressing data is directly used as pixel data. As a result, the overlay is realized by replacement (see FIG. 8). Whether or not to replace data is controlled by switching control of switches in the pixel insertion unit 30C.

  As described above, all addresses of the composite stream that has arrived just before the image composition unit 35 (see FIG. 1) are replaced with pixels. That is, it is in the form of pixel data + composition ratio. Therefore, as shown in FIG. 9, the synthesis ratio information is extracted from the synthesis stream S1-1, and the synthesis ratio information is also extracted from the synthesis stream S1-2. Then, when combining the pixel G1 of the combined stream S1-1 and the pixel G2 at the same position of the combined stream S1-2, the multiplier uses the combining ratio K1 of the pixel G1 and the combining ratio K2 of the pixel G2 to multiply G1 × After calculating K1 and G2 × K2, a combined output pixel is obtained by adding both by an adder. That is, an output image synthesized by superimposing the image generated by the combined stream S1-1 and the image generated by the combined stream S1-2 is obtained. The same can be considered when there are three or more composite streams.

  As already described, it is also possible to validate only the composition ratio K1 of the composite stream S1-2 and invalidate the composition ratio K2 of the composite stream S1-1. In this case, the synthesis ratio on the side of the synthesis stream S1-2 can be obtained by 100% -K1, for example.

  According to the above configuration, when rearranging the pixels of each input image, the pixel composition processing is not performed, and the image composition is performed at the final stage after the rearrangement of the pixels is completed. With this configuration, the input channel can be easily expanded. In addition, by sending the composite stream in synchronization with the output image, it is not necessary to synchronize the output again with a frame buffer or the like before outputting the image, and therefore, the delay of the image between input and output can be shortened.

  Further, since the temporal access restriction of the frame memory 20 can be used by being distributed to the number of synthesized pixels and the number of output channels, the degree of freedom in using the image synthesizer can be increased. For example, if the frame memory 20 is a high-speed SRAM that can operate at 100 MHz, it is possible to read out eight times per output pixel speed (12.2727 MHz). The eight times can be freely distributed. Further, if a condition that pixels having the same input are not combined is added, image output of more channels can be realized.

  Further, connection I / O can be reduced by replacing the address information of the synthesized stream and the pixels. Further, read access to the input image buffer is always performed regardless of whether the pixel is hit or not, and whether or not the addressing of the composite information is replaced with pixel data. This enables the fastest access to the input image buffer.

<Another embodiment>
In the present embodiment, NTSC has been described as an example of an image input method. However, the present invention can also be applied to other arbitrary image input methods. In the present embodiment, an image of 640 × 480 has been described with respect to the number of pixels. However, the present invention can also be applied to other pixel numbers.

Block diagram showing the overall configuration of the image composition device Diagram showing input image mapping Diagram showing composition table composition The figure which shows the data structure of 1 synthetic | combination information of a synthetic | combination table. The figure explaining the synthetic stream generation process The figure explaining the read-out timing of synthetic table data A diagram showing a specific example of a composite stream The figure explaining the pixel read-out operation | movement by a pixel rearrangement apparatus. Diagram explaining overlay relocation Block diagram showing the method of image composition in the image composition unit The figure which shows the switching structure of the frame memory of 3 sides

Explanation of symbols

1 NTSC input block 2 Memory block 3 Image composition block 5 Compact Flash (registered trademark)
6 NTSC encoder 10 NTSC decoder 20 Frame memory (input memory)
30 pixel rearrangement device 30A read address generation unit 30B pixel hit determination unit 30C pixel insertion unit 31 overlay pattern generator 32 output image synchronization signal generator 33 FIFO buffer 34 FIFO buffer 35 image synthesis unit 40 synthesis table data memory 41 synthesis table Data memory

Claims (2)

Multiple input channels into which images are input,
An input memory that stores image data input to each input channel in a readable manner;
Pixel rearrangement that is provided corresponding to each input channel and reads out any pixel constituting the input image from the input memory based on preset address information, arranges it at any position in the output image, and outputs it Equipment,
A data memory that stores at least the address information of the pixel to be read and the composition ratio information of the pixel, and stores a composition table that is a collection of composition information corresponding to each pixel constituting the output image;
A buffer for sequentially reading out and storing the data of the synthesis table from this data memory;
When the data stored from the buffer is read out and sent to each pixel rearrangement device as a composite stream, a synchronization signal generator for sending out in a form synchronized with the output image;
An image composition unit that performs image composition processing based on the composition ratio information for each pixel read out as an output image after all the pixels constituting the output image are read out and rearranged by the pixel rearrangement device. ,
The pixel of the input image corresponding to the address information of the composite information of the composite stream is read by a pixel rearrangement device, and the output image is gradually replaced by sequentially replacing the read pixel with the address information portion. An image synthesizing apparatus characterized in that it is configured to be generated continuously. The input memory is composed of SRAM as a frame memory,
The pixel rearrangement device includes: a pixel hit determination unit that determines whether a pixel corresponding to the address information hits an input image;
Regardless of the determination result by the pixel hit determination unit, an address generation unit that generates an address for reading out a pixel from the SRAM ;
A pixel output control unit that controls whether or not to output the read pixel as a pixel constituting the output image,
The pixel output control unit determines that the pixel hit determination unit does not hit and reads a pixel that is not hit , discards the read pixel without using it as an output image. The image composition device according to claim 1.

Images