TW200419467A - A general purpose stereoscopic 3D format conversion system and method - Google Patents

Abstract

Because of the numerous possibilities in the capture, transmission, and display of stereoscopic image pairs during 3D format conversion, there are numerous compatibility issues between the various stereoscopic image sources and stereoscopic display systems. A general purpose stereoscopic 3D format conversion system and method is described that eliminates these compatibility issues by providing a system that performs conversion of any stereoscopic image stream within a specified set of 3D formats to any other 3D format within the set. The system also performs numerous other image processing functions that are beneficial for stereoscopic image processing such as image pan, alignment, zoom, crop, keystone correction, video format conversion, scan-rate conversion, and an array of standard 2D image enhancements including brightness, contrast, hue saturation, and sharpness.

Description

发明 Description of the invention: [Technical field to which the invention belongs] Field of the invention The present invention relates to format conversion of 3D formats, and in particular to conversion of any stereoscopic image stream in a specific set of 3D into any other in the group Device and method for 3D format conversion in 3D format. L · y BACKGROUND OF THE INVENTION In the last two decades, many stereoscopic 3D display and recording devices have been developed for multiple purposes and different customers. The main question that must be answered in the development of such devices is about which method will be used to capture, encode, and / or display stereo image pairs. All stereoscopic display devices require the display of the left-eye image to the left eye of each observer and the right-eye image to the right eye of each observer. The methods used to capture, transmit, or display such images depend on the many elements of the type that include the display or recording system, and the method by which the set of images will be presented. For example, a stereo shutter glass system generally requires a time-multiplexed sequence of left and right image pairs to be transmitted to a CRT monitor. Other display systems (for example, those based on Revove holes) display stereo image pairs in a staggered column format (column interlaced), so that, for example, eight odd lines (and transmitted images) include The left-eye image, and the even-numbered line covers eight right-eye images. Other display and recording devices also use other 31 ^ formats, including up-down, side-by-side, line-interleaved, and dual-channel (actual = separate channels). There are even more image formats that contain the left and right signals on different color channels or in the area of non-overlapping optical spectrum: 200419467 image group pair encoding. & In order to capture, transmit, and display the many moons in the stereo image group, it should be easy to understand that there are many matches between various image sources and display systems. A system with ° midnight has all the capabilities of the present invention. However, none of them-planted in a single-device-provides the versatility and flexibility of the present invention. Many devices are mentioned here, but www.stereo3d.com maintains: the latest list of recent 3D scales and displays and converter boxes. , Ι, the existing three-dimensional converter is now available in 3D multiplexer or 3D demultiplexer. The 3D multiplexer uses two sets of separated video image streams and industrializes them into a set of frame-to-frame methods for progressive video or a field-to-field single-video method for video. For the purposes of the present invention, these devices perform a field or frame sequence to dual channel conversion. In addition, the 3D demultiplexers perform reverse operations. They separate 3D image data encoded in a frame-to-frame or field-to-field 15-field manner into a single stream of information in a single video stream. For the purposes of this invention, this chain performs a field or frame sequence to a two-channel conversion. Typically, these devices are built for a specific video format (such as' NTSC or PAL video, or computer VGA mode). Some examples of such devices include those described below. 2 VRex 3D type converter 101_30 format converter 10 is an example of the embodiment of the present invention. It has the ability to receive only continuous 3D video fields from composite video or S-video sources in both NTSC or pAL formats and convert the video to a single 3D format for computer VGA video when output. The device can be configured to output 2D, dual-channel, box-sequence, field-sequence, intersecting 6 200419467 staggered, or row-interleaved, but only at manufacturing time. It is not possible for the user to change between the various 3D output formats. VRex MUX-1 ... MUX-1 is a 3D video multiplexer that uses field sequence 3D format to convert two separate NTSC video signals into a single 5 NTSC video signal. These two sets of input signals must be phase-locked. In this invention, there is no need for a genlock or an input signal that is commonly synchronized. VRex MUX-3 ... MUX-3 is the same as MUX-1 except it supports the PAL video standard.

Cyviz-Cyviz (Norwegian company) produces two 3D format conversion products, xpo.l and χρ0.2. Each of these products is a 3D video demultiplexer that converts frame-sequence computer video formats to a dual-channel format with the same resolution at half the vertical update rate when output. These products have some advanced features, including stereo edge blending, which is useful for combining two or more projection screens together. More information can be found in WWW.CVvi7Tmn 15. 3DTV—3DTV has many lines of 3D video multiplexes and video format converters. More information can be found at www.3dmagic.com. Dimensional Technology (DTI) has an auto-stereoscopic display circuit that is capable of receiving many 3D formats displayed on their devices. Most 3D display devices accept a single set of 3D formats needed to experience any sense of depth. More information on the DTI display can be found at www.dti3d.com. A 2D scan converter is a device that converts a group of video formats into another video format (eg, NTSC to PAL, or computer format to NTSC, etc.). The 2D scale changes the image resolution of the signal up or down to match the desired output. There are many 2D scan converters on the market, which include the sellers locked by RGB spectrum (www.rgb.com) and Extron (www.extron.com). Therefore, obviously, 3D stereo display and recording systems The proliferation has produced a variety of stereoscopic 3D image formats (also known as 3D styles). Because these display modes vary greatly from one another and generally do not allow them to easily switch to one another, there is a great disparity between the stereoscopic image generation system and the stereoscopic display system. There are many devices that can convert between two different sets of 3D formats (usually from a field or frame sequence to a dual channel or a two channel to a field or frame phase). However, it is necessary to return to general-purpose device technology capable of converting any 3D format to any other criminal format. [Explanation 3 Summary of the invention Disadvantages, using the present invention and the object of the present invention The above discussion and other problems of the prior art and many methods and systems are overcome or mitigated and achieved. The general three-dimensional three-dimensional lattice described here uses the system and method from t L to eliminate these compatibility issues first. The vehicle system is performed in the designated, And any 3D image stream in the 3D format is converted into any other 3D format in the minus group. This secret is the same as the foolish messenger. Your name is good for other reading and processing in the stereo image. For example, 隹 卷 丹 彡 ^ ~ 彳 Tsuzuki knows the photo, aligns, changes, ", mind, trimming , Key corrections, video format conversion, and including binocular 4kl, conversion rate, standard image enhancement. I sheep brightness series-200419467 Therefore, the invention described here allows users to completely freely connect any conventional The stereo image generating or recording system provides useful image enhancement to any conventional stereo image display device. Brief Description of the Drawings 5 These and other features of the present invention will be better understood using the following description, the scope of the appended patents, and the accompanying drawings. , Arguments, and advantages, where the graphics are:

Figure 1 shows a block diagram of an embodiment of a 3D stereo format converter; Figure 2 shows a block diagram of a front-end processor system with digital RGB output; Figure 3 shows a block diagram of a front-end processor system with digital YUV output Figure 4 shows a block diagram of a rear-end processor with digital RGB input; Figure 5 shows a block 15 diagram of a rear-end processor system with digital YUV input;

Figure 6 shows a block diagram of a 48-bit digital RGB-CHVF input data bus, Figure 7 shows a block diagram of a 60-bit digital RGB-CHVF output video data bus; 20 Figure 8 shows a 16-bit digital YHVF video A block diagram of a data bus; FIG. 9 shows a block diagram of an embodiment of a 3D data format converter; and FIG. 10 shows a block diagram of another embodiment of a 3D data formatter. I: Detailed description of the preferred embodiment of the first embodiment 9 200419467 The method and system here is to convert any 3D format to any other 30 format from a large set of 3D formats in a single electronic device. The basic set of 3D formats supported by the preferred embodiment includes: · standard 20; dual channel type; image field sequence 'image frame sequence (page view wide up and down style; column staggered style; side-by-side style; and row parent Miscellaneous other possible 3D formats can be supported, including, but not limited to, a format that is spectrally multiplexed using this device. Furthermore, the embodiments described herein provide many other useful features for use Appropriate settings, enhancements, and corrections for dual or multiplexed stereo image streams. These features include stereo image scanning, alignment, zoom photography, cropping, focus correction, aspect ratio conversion, and linear and nonlinear Scale changes, etc. Furthermore, because the stereo image is transmitted on a variety of standard video interfaces, the invention described here supports conversion from one video interface to another. The video interfaces supported by the preferred embodiment include RGB computer, component video, S video, and composite video. Each of the 15 video roar standards for each video interface is also supported by the preferred embodiment, which includes VES A GTF standard, HDTV standard, NTSC, PAL, SECAM, YUV, and RGB video, etc. Furthermore, because some embodiments of the invention described here use, in part, state-of-the-art FPGAs (field-programmable gate arrays) Technology is constituted 'Because new problems and needs may be discovered, new functions, features, 20 and 3D formats can be added to the device in the future. Format conversion and other image processing functions can be performed instantly and are excellent Performance. Figure 1 shows the basic system structure of the present invention. The 3D data generator 10 performs many operations including input channel selection, stereo demultiplexing, stereo image scale change, scan rate conversion, 3D stereo format conversion, and rotation. The main function of channel selection 10 200419467 selection. According to the input selection setting, 3D data formatter i〇2 selects

Which group of most input channels is used for 3D stereo input. Depending on the 3D format being entered, generally only one or two channels will be selected at a time. Then the 3D data formatter 102 demultiplexes or separates the 3D stereo data into two sets of separate image processing channels. Most importantly, this separation step is achieved so that the left perspective image data and the right perspective image data can be processed separately. Processing these channels together as a set of data frames will cause data errors when the image scale changes and scan conversion processing. The 3D data formatter 102 then performs an image scaling operation to adjust the image resolution required for the selected output. According to the selected data format used for the output signal, the 3D data formatter 102 can scan and convert the image data. If the output data format is input synchronously, no scan conversion is performed. This case is where the input data signal frame rate controls the internal data frame rate of the system and where the 3D output rate is directly controlled by the input k number. If the output data format is output synchronized, a scan rate conversion is achieved to synchronize the processed 3D image data with the required output frame rate. In this case, the 3D output rate is completely independent of the input signal frame rate. Each of these two methods has its advantages and disadvantages. Finally, the 3D data formatter 102 performs 3D stereo format conversion to reconstruct the processed stereo image data into a format required for output. This system can provide a large number of 20 input / output patterns and allows the internal stereo channels to operate independently, which will be described below. Another significant feature of 3D material formatter 102 is that it supports two input signal groups (A and B 'as shown in Figure 丨) and two output signal groups. Each input or output can support any 3D format that is multiplexed, regardless of ethnic group identification. The signal group mainly provides support for non-multiplexed (dual-channel) galaxies ... A set of inputs or rotations from each group (A and B) is selected for each channel. This simply indicates that two sets of inputs in the same ethnic group cannot be used in a dual-channel 3D format. But any pair of signals that contain a belief from a relative group is valid and supported. As a result, images on different video interfaces (for example, VGA computer and composite video) may become effective stereo pairs. According to the digital image data format used to transmit the original image to or from the 30 data formatter 1020, each signal group is subdivided into two groups. These two groups contain digital RGB input / output and digital Yuv input / output. In the top section, these two groups are actually represented by the group A of humans—group digital input bus A and-, group digital Υυν input bus eight. A set of similar configurations is constructed for the other two groups of input numbers (input B, output A, and output B), shown in Figure 10. Provide two types, because these two types are imaged interface

Used by electronics (for example, two sets of ADCs and DACs, video decoders / encoders, HDTV decoders / encoders, etc.). Individual digital or digital YUV buses do not actually support any number of compatible devices. The digital input bus is limited to-once only-one cent loser. All input devices within the front-end processor 20, in addition to the function, are best to have three-state output to avoid the secret of input data. However, for the output bus, any device that can support the current signal timing can work because no data message * error will occur. The peripheral processor system is the "x > 3D > material formatter 102. Figure 1 left 12 200419467 Various front-end processor systems 104, 106, 108, and 110 on the side receive image data in various formats from the outside and convert the data into digital RGB or digital YUV according to the attached input bus. Similarly, various back-end processor systems 112, 114, 116, and 118 on the right side of FIG. 1 receive digital RGB or digital YUV format image data from the 3D data format formatter 10 and convert the data for use. Various video image standards for external transmission. In addition to a set of processor systems, each digital RGB or digital YUV bus has a set of attached expansion ports. Expansion ports 120, 122, 124, 126, 128, 130, 132, 134 allow other input or output devices to be added to the system. In this way, when new video standards are created by the industry, expansion cards can be added to support new formats. Additionally, if there is a current video data format that is not supported by the preferred embodiment, an expansion card can be generated and added to the system. Front-end processor systems with RGB output 104 and 108 each represent a family

15 groups of standard video input electronic circuits, which can share the same digital RGB input bus protocol (for example, as shown in Figures 6 and 7). Similarly, the two sets of front-end processor systems 106 and 110, each with a YUV output, represent a group of private quasi-video input electronic circuits that share the same digital YUV input bus interface protocol (for example, as shown in Figure 8). Display). The standard input in the preferred embodiment includes an rgb computer, Η〇τν, composite video, video, and component video. Other possible turns include SDI (Serial Digital Interface) and DVI (Digital Video Interface). Similarly, two sets of back-end processor systems 112, 116, and * groups of quasi-image output electronic circuits with Rgb inputs share the same number. 13 200419467-bit RGB output bus protocol (for example, as in Sections 6 and 7) The two sets of back-end processor systems 114'118 with YUV input each represent the standard of a group ~ like output electronic circuits that share the same digital YUV output bus interface agreement (for example, as in Section 8 Figure shows). The standard output in the preferred embodiment includes an RGB computer, HDTV, composite video, Y / c video, and component video. Other possible inputs include SDI (Serial Digital Interface) and Tear (Digital Video Interface). 10 15 20 Except for the digital video interface which is separate from the criminal data formatter, each peripheral processing unit 4 · 118 is controlled by the group control system ⑶ via a common group peripheral control bus 138. In the preferred embodiment, the peripheral control bus 138 includes a set of two-wire serial buses, which uses a patented Ik serial surface produced by Philips Semiconductors. This is rarely common among the various video interface electronics. In addition, the peripheral control bus 138 contains a plurality of specific inputs, such as a circuit board, whose phases are used as interrupt inputs to the control system 136, for example, to receive from one or more front-end processors and / or one or more The group back-end processors get feedback. The second significant feature shown is the control system, which includes a set of microcontrollers and associated supporting circuits. In the preferred embodiment, the control system 136 includes six digital communication ports, that is, the peripheral control bus 138 described above, a group of synchronous data leg buses 14G, a group of core control ship buses 142, and a group of computers. A control interface 144, a network control interface 146, and a user panel interface 148. The control system 136 has just monitored and controlled the data format via the core control bus 142 and the synchronization information bus. The synchronization information bus has just provided the input timing information that is currently selected and used for input.

14 Information on the status of 200419467. This connection allows the control system 136 to recognize incoming video signals for proper setting of the 3D data formatter 102. In a preferred embodiment, the core control bus 142 is a set of three-line serial interfaces for the devices in the 3D data formatter 102 and specific lead lines. The 5 electronic devices within the 3D data formatter 102 have various serial communication methods that are completely different from each other. The core control bus 142 is generated to provide various serial interface patterns. The computer control interface 144 may be any suitable interface for the computer system 150. For example, a 'standard serial communication interface system can be used. In a preferred embodiment, this interface uses the RS-232 standard. Further embodiments can use USB 2.0 or other similar standards. The purpose of the computer control interface 144 is to allow the status of the control device to be collected using the local computer system 150. New hulls for various programmable components can also be uploaded via interface 144 at the same time. The network control interface 146 performs a similar function as the computer control interface 142, but allows connection to a network 152, such as a device in a 15-area network, through a standard network interface circuit. The final user panel interface 148 allows communication between the primary device and the selected user control panel 154, for example, via a set of two-line serial interfaces. The selected user control panel 154 allows zone control over devices not related to any network 152 or computer system 150. Figure 2 shows an embodiment of a front-end processor system 104 (20 or 108) with digital RGB output. The processor system 104 includes a large number of various hardware components, and its decoded image data (using various connectors and standards) has become a digital RGB format used by the 3D data formatter 102. Figure 2 shows a sample collection of this type of hardware. A preferred embodiment uses a three-channel ADC and supporting hardware to convert various computer and HDTV formats, and a video decoder chip and supporting hardware 15 200419467 to convert composite, Y / C, and component video. There are many other decoder chips that support multiple video format inputs that can be used, as indicated by the graphics. Expansion ports allow new formats supported by adding additional peripheral boards. In the preferred embodiment, the front-end analog RGB processing block 202 is implemented and supports two separate analog color channels 204 (red, green, and blue) and two sets of synchronization signals (vertical and horizontal) and 3D. Input of the ID signal of the domain. The analog input is converted to a set of 48-bit RGB (or 24-bit RGB) digital formats for transmission to a 3D data formatter 102. Other front-end processing blocks are shown in the graph. The analog HDTV processor 10 206 provides a specific set of connections for analog HDTV 208 using the γυν or 11 (}) 38 format. The digital RGB processor 210 converts digital RGB data 212 at other bit depths into a digital RGB input stream The 48-bit depth used by the row. Digital HDTV processor 214 provides a set of format support for digital HDTV 216. dv: [Processor 218 provides a set of support for digital video interface 15 standard input 220. Finally, SDI The processor 212 provides support for serial digital interface standard input 224. Each of these front-end processor blocks 202, 206, 210, 214, 218, 222 is connected to the 3D data formatter 102 via a digital RGB input bus 226 and Connected to the control system 136 via a peripheral control bus 138. 20 Front-end processor system with digital YUV output 106 (or 110) The embodiment is based on the video input hardware components that output on many digital YUV standards that replace RGB. Another set. Figure 3 shows only one set of components in this set, but other components can also be included. In a preferred embodiment, the front-end video decoder block 302 can support Three sets of separate video input 304 formats, including 16 200419467 composite video,-/ C-video, and component video (YUV or RGB). Many modern video decoder chips support this quasi-function. Front-end video decoder 3202 box At the same time, it also processes the analog to digital conversion (ADC) of the input video signal 304. Regardless of the input format, the output of the front-end video decoder block 302 is a digital form of one of the selected inputs. The 3D format of the input image data It can use any standard form. For video input signals, the 3D format is generally the image field sequence 3D (the left and right image data is transmitted on different video signal image fields) or dual input 3D (the left and right image data is Two sets of inputs are actually separated on the input connector). Other options of the front-end video processing block 302. 10 features include gain control, color and brightness control, video format decoding (NTSC, PAL, SECAM, etc.) and others Can be a feature related to video signal decoding. The front-end video decoder 302 is connected to the 3D data formatter 102 via a digital YUV input bus 306 and The edge control bus 138 is connected to the control system 136. 15 Figure 4 shows an embodiment of a back-end processor system 112 (or 116) with digital RGB input from the bus 430. This system 112 performs the inverse of the front-end processor system. The digital RGB format used by the 3D data formatter 102 (such as the digital RGB output bus 430 shown) to become a variety of video formats for output. This system is used to perform various conversions. Aggregation of standard hardware components and supporting hardware. The preferred embodiment uses a set of back-end analog RGB processors 402, for example, is made using a set of three 10-bit DACs for the analog R G B output 404. Computer video standards and HD TV standards are supported by this hardware. Figure 4 shows other hardware possibilities, including: for the analog HDTV processor 406 after the analog HDTV output 408; 17 200419467 for the digital RGB processor 410 after the digital RGB output 412; after the digital HDTV output 416 Digital HDTV processor 414; for DVI output 420 after DVI processor 418; for SDI output 424 after SDI processor 422; and other outputs. There are many other transformation hardware 5 widgets that can be used to make lists of other blocks. 1-1 Figure 5 shows a detailed back-end processor system 114 (or 118) with a digital YUV input 502. In a preferred embodiment, only one set of back-end video encoder blocks 504 is used, although other hardware may be used. The back-end video encoder block 504 converts digital YUV data into any of the various analog video formats in any NTSC, 10 PAL, or SECAM video standard, including composite, γ / c, and component (YUV and RGB). Figure 1.2 shows the representation of the 48-bit digital RGB input video bus 602 used by the 3D data formatter 102. Using this bus system 602 '24-bit or 48-bit RGB video data can be provided. In the 4M Legion 15 mode, two sets of full-image pixels are transmitted in parallel to halve the demand for the bus rate. R0 and R1 represent the first and second red pixels in the data stream. Similarly, 'G0 and G1 represent first and second green pixels and B0 and B1 represent first and second blue pixels. These color channels are each 8-bit wide as shown. The other signals in the bus constitute the video timing and 3D field ID signals. C stands for pixel clock. The thief table is synchronized horizontally. Meter reading vertical synchronization signal. F stands for video field ID. SF stands for stereo field m. 1, 3 and 7 show a representation of the 60-bit digital RGB output video bus 702 used by the 3D data formatter 102. Using this bus system 702, 60-bit, 30-bit, 48-bit, or 24-bit RGB video data can be extracted in 60-bit mode. The pixels of the two full-image groups are parallel. The need to be transmitted f halves the bus rate. R0 and R1 represent the first and second red pixels in the data. Similarly, '⑼ and ⑴ represent the first and second green pixels and bo and B1 represent the first and second blue pixels. Each of these color channels is the 10 Yuan Wei displayed by the taxi. The other signals in the s-stream constitute the video timing and 3D% ID㈣. c represents the pixel clock. H stands for horizontal synchronization signal. V stands for vertical synchronization signal. F stands for video field ID. SF stands for stereo field ID. Figure 8 shows the representation of the 16-bit digital YUV bus 802 used by the input and output of the offended data format. In ι 6-bit γυν, the -group 8-bit το channel is used for Fandu information and the second 8-bit channel is used for chroma. Additionally, if one of the input or output processor systems requires an 8-bit YUV data format, this bus system can provide redundancy and chrominance on a group of 8-bit channels. In the preferred embodiment, a 16-bit YUV in 4: 2: 2 format is used to transfer data to and from the 3D data formatter 102. For other bus systems, C represents the pixel clock, Η represents the horizontal synchronization signal, V represents the vertical synchronization symbol, F represents the video field ID, and SF represents the stereo field m. Fig. 9 shows a set of embodiments of the 3D data formatter 902 (which can be used in some embodiments, such as 3 £ shown in Fig. 丨 the data formatter 102). Some functions of the 3D data formatter 902 are described in the commonly-owned U.S. Patent No. 10/045901 and International Application Patent No. pCT / us〇2 / 〇1314 with the issue number W0 / 〇2 / 〇761〇7 In the case, its title is "Method and device for using line-interleaved data with digital light processing for stereoscopic display, and it is referenced here. The 3D data formatter 902 performs five main functions, including input channel selection, Stereo multiplexing, stereo image scale change, scan rate conversion, and 3D stereo format conversion. The 3D data formatter 902 contains five main components, including a set of microcontrollers 904, a set of four inputs and two Output RGB input data switcher / router system 906, a set of two-input two-output RGB output data switcher / router system 908, and two separate video processing units 91 with 5 associated memories 914, 916, 912. The dual processor configuration enables various functions. In some embodiments, this function includes independent image processing for left and right perspective image data. Dual processor set Provide the highest available image quality while keeping the left and right image data completely separated to prevent stereo degradation. In the other 10 cases, the dual-focused video processor system makes it possible to output video data in different 3D formats or video formats. Simultaneous processing. 4- to-2 RGB input data switcher / router system is essentially a matrix switcher for RGBHVC data signals, which has a 3D format to guide any input person to any or two groups based on the 3D format of the turn-in signal. Output capacity 15. For example, in the case where the input channel A contains left and right perspective image data, the input switcher 906 will guide the input channel a to two for the video processor cent and 912 to operate step by step. Output. In the case where the left and right perspective image data are carried on two sets of separate channels (for example, channel A and channel B), each input is directed to a single set of outputs. In a preferred embodiment: This switch is made using a high-speed CPLD integrated circuit. Two sets of video processors 910 and 912 are video processing circuits that have the ability to perform many useful functions. Can include: image weight, cat scan rate conversion, color correction, and focus correction. These processes = can also control the majority of the separate continuous rotation of the image data frame and the majority of the memory 20 200419467 output data frame These features make it possible for each video processor 910, 912 to operate on a specific set of image data corresponding to the left or right perspective image. Operation in combination with the input data switch / router 904, Virtually any 3D stereo data format can be provided. 5. Once the appropriate image data set is separated using the input box control, each video processor 910, 912 performs the required scale changes and image enhancement operations. The video processor 910 '912 also acts as a dual-port memory control, so the output & data rate can be independent of the input data rate. The input and output data rates are determined by the horizontal sync signal, vertical sync signal, and pixel clock signal. A preferred embodiment of the video processors 910 and 912 is an IP00C711 chip from ichip. Other video processor integrated circuit chips with similar functions and features can also be used at the same time. A preferred embodiment of the memories 914, 916 is a 16-megabit SDRAM device. Sufficient memory is provided to provide a large number of frame buffers (such as the four shown in Figure 9) for each video processor 9io, 912 corresponding to the majority frame control. This configuration provides the maximum control and flexibility required by this system. 2-to-2 RGB output data switcher / router 9008 is another RGBHVC digital matrix city ||, which can transport any input and output to any output according to the prescribed route. At the same time, it can also transport any color data related to the two sets of input channels to any color mismatch of the two sets of output channels in accordance with the prescribed route. This feature allows the use of a color sequential method of SD image coding. The switch works with two sets of video processors 91 and M2 to realize all possible data deduction methods, which can be used for transmission to the output device and any related processing. In a preferred embodiment, the output of each video 21 200419467 processor 910, 912 is a set of 24-bit rGb signals, which includes 8-bits for each color of red, green, and blue. To provide color multiplexing, the switch 906 can route any color input to any other color output in a prescribed route. Therefore, the switcher 906 is used as a group of 6-input 6-output matrix switchers for one of the 8-bit digital signals. In a preferred embodiment, the switch is fabricated using a high-speed CPLD integrated circuit. The M microcontroller 904 performs the setting and control functions of the 3D data generator 902. It uses a set of EEPROM memory to store register settings for each video processor 910, 912 and data exchangers 904, 906. The micro controller 10104 can also control the function interface with the user. FIG. 10 shows another embodiment of the 3D data formatter 1002 (which can be used in some embodiments, such as the 3d data format 3 | ι〇2 shown in FIG. 1). The data formatter 1002 performs The main functions of input channel selection, stereo multiplexing, stereo image scale change, scan rate conversion, and 3D stereo format conversion. In addition to these functions, it performs _ group output channel selection. A microcontroller (not shown) is shown, such as the control system in Figure 1. The output clock generator 1004 is also shown. The output clock generator 1004 generates a timing signal for outputting a video stream. The 3D data formatter 1002 of the present invention includes five main components, including package 20 including an output clock generator 1004 (OCGEN), a set of input signal paths = 1006, a set of output signal routers 1008, and a video processor A.丨 〇 丨 〇, 2 and video processor B 1012. The 3D data formatter system 1002 provides a set of dual-valued video processor configurations that enable independent image processing of left and right perspective image data. The dual processor configuration provides the highest image quality available 22 200419467 while keeping the left and right image data completely separated to prevent stereo degradation. In other embodiments, the dual processor system enables simultaneous processing of video data in different output 3D formats or video formats. Each video processor 1010, 1012 is a video processing circuit capable of performing image rescaling, scan rate conversion, color correction, emphasis correction, and many other useful functions. Video processors 1010, 1012 operating in parallel are used to achieve useful stereo scanning, alignment, zoom photography, and trimming features. These processors 1010, 1012 can control up to multiple sets of separated and continuous input image data frames and multiple sets of separated output data frames 10 (such as the four sets shown in Figure 10). These features make it possible for each video processor 1010, 1012 to operate on a specific set of image data corresponding to the left or right perspective image. This operation forms the basis of three-dimensional demultiplexing on the input. The operation in conjunction with the input signal router 1006 ’is actually supported by any conceivable 3D stereo data format. Once the appropriate stereo image data set is separated using the input box control, each video processor 1010, 1012 performs the required scale changes and image enhancement operations. The video processors 1010 and 1012 also function as dual-port memory controllers that make the output data rate independent of the input data rate. The input and output data rates are determined using their separate video data timing signals. The preferred embodiment of the video processor 1010, 1012 is the iP00C 7i5 chip of iChip Company as the main component. Other video processor integrated circuit chips with similar functions and features are also used. The preferred embodiment of the memory block 1014, 1016 is a 64-megabit SDRAM device. Sufficient memory is provided for the majority of the video processors 1010, 1012 (as shown in Figure 10) corresponding to the majority of the frame control (such as the four groups shown in Figure 10 23 200419467). (Group 4) Full frame buffer. This configuration provides the maximum control and flexibility required by the system. The following paragraphs describe some of the more important features achieved by using video processors 1010, 1012. The scan rate conversion can be achieved using the video processors 1010 and 1012, so the input image frame rate (vertical update rate) is changed to a different range on the output.

The scale of the image can be changed to 10% by using the video processors 1010 and 1012. Therefore, the signal format on the input side of the video processor signal block is converted to a larger or smaller size signal format on the output side. . Many factors control how the image changes scale, including whether to interleave input or output signals, 3D encoding methods, and whether 3D mode is in progress. Image scale changes can also use linear and non-linear methods. 15 In view of the change in image scale including the enlargement or reduction of image data and signal space, stereo zoom photography generally refers to a specific rectangular window of an input image in a fixed input signal space to a specific rectangular window of an output image in a fixed output signal space. Zoom in or out. The zoom photography function can be achieved using the video processors 1010, 1012. Zoom photography allows users to adjust and expand specific areas of the image, but cannot change the format of the output image signal. Zoom photography on the input side image causes the image to appear larger on the output side. Zoom photography on the output side results in a smaller image display. Stereoscopic scanning can be achieved using video processors 1010 and 1012, so the output 2D or 3D image can be moved vertically or horizontally on the screen. In 24 200419467 3D weighting, the stereo channels move together and the entire stereo image is shifted. This process is different from the alignment process discussed later. In the 30 alignment, the stereo channel is moved in the same direction to keep the overall image at the center of the towel. Scanning does not affect any changes in scale. 5 Stereo alignment can be achieved using video processors 1010, 1012, so equal and relative scan conversions are applied to each video processor chip. This causes the images of the 3D image streams to move away from or approach each other. This feature is very useful for stereo camera input on the output side or to correct misalignment of the dual projector system. At the same time, it is also useful for moving the zero parallax point of the image to reduce the stereo 10-window interference and adjust the maximum parallax, so stereo content designed for small screens can be displayed on the large screen (or vice versa). Stereoscopic image trimming can be achieved using video processors 1010, 1012, so unwanted edges of the input and / or output images can be eliminated. This feature is very useful to eliminate the obstruction of the stereo window or other unwanted parts of the image. The input signal router 1006 (ISR) is mainly responsible for transmitting image data and synchronization signals to the video processors 1010 and 1012 according to a prescribed route. It also has other functions that help the control system 136 monitor input video signals and control signal decoding hardware. With regard to image data and synchronization signals, the input signal path 20 acts as a 2x2 matrix switch. Data input on source data channel a (which contains RGB and YUV data streams for computer and video input respectively) can be directed to video processor A 1010 or video processor B 1012 or processors 1010, 1012 at the same time Both. Similarly, it is also applicable to data input on source data channel B. ISR 1 06 is also allowed to lose 25 200419467

The input synchronization signal group (which includes the vertical synchronization signal, the horizontal synchronization signal, and the field ID) is independently switched to the control system 136 via the core information bus 4 for analysis. The timing signal information output control does not affect the main image output multiplexer. In a preferred embodiment, the ISR 1006 also provides the control 5 system 13 6 with the ability to separately control the Sync-On_Green feature for the r G b source data channel. ISR 1006 also handles the selection of the source ID field data. In a stereo 3D system, the image field ID signal is used (especially in a frame sequence format) to completely determine the current stereo image field. This image field ID signal can be externally provided to the RGB connector 10 via a separate set of stereo synchronizing inputs, or it can be provided internally to the RGB connector on the pin 12. In the absence of one of these two sets of signals, the field ID signal can be generated from the vertical sync signal input using ISR 1006. isr 1006 provides the ability to select any of these source materials separately for the field ID signal of the source data channel. Finally, the ISR 1006 uses signals to the video processor and 15 to RGB ADC hardware to assist in the use of different pixel sampling modes for high-bandwidth RGB inputs to identify the capture of odd or even pixels. This feature also assists the father in staggered 3D multiplexing. In a preferred embodiment, the isr 1006 is fabricated using a high-speed field programmable CPLD integrated circuit. The output signal router (OSR) 1008 is mainly a set of digital switches, 20 which are used to re-multiplex the respective stereo image streams into the required 3D format. It also contains RGB and YUV processors to convert the RGB data output from each video processor 1010, 1012 into the YUV format used by the digital YUV output bus. Under normal circumstances, each output is available at all times 'so each output can simultaneously support a set of completely separated 3d formula 26 200419467 for RGB output' 〇sr 1008 can also transmit on the prescribed route with two sets of input Any color data related to the channel to any other color of the two sets of output channels (for example, exchanging red data between the stereo channels Iori to produce the output of a stereo movie). This feature allows the use of color-coded image coding methods. In a preferred embodiment, the switch is fabricated using a high-speed (field-programmable gate array) integrated circuit. This fact allows the clothes to be updated with new features. Finally, OSR 1008 was used to direct the output pixel clock and memory clock signals from OCGEN 1004. The output clock generator 1004 (〇CGEN) provides output pixel clock and memory clock signals for the video processor 1010, 1012. OCGEN 1004 operates in conjunction with the OSR surface to provide two sets of basic pixel clock configurations, which include the following configurations: Configuration 1: Output Signal Router 1.08 Boot 1 > (: 10 to? (: : 1 (: eight and PCK2 to PCKB, so the video processors 1010 and 1012 perform 15 independently of each other and generate their own output timing signals. This configuration allows the device to output two sets of different signal formats at the same time. It can be useful when the device is used to drive two sets of separate 3D displays (for example, the main projection display and operator monitor). This configuration is also useful for 2D or dual channel operation; and 20 1.4. 1 group of sad 2 · OSR 1008 guides PCK1 to PCKA and PCKB. This configuration causes the output video processors of 1010, 1012 to synchronize with each other. In this case, a group of video processors 1010 or 1012 function as if they were generated for use by the processor The main processor of the timing signals of 1010 and 1012. This configuration is the standard configuration of most multiplexed 3D format modes. 27 200419467 In a preferred embodiment of the present invention, the 3D data formatter 1002 provides a method and a device. Set to provide many 3D formats from a variety of sources. Many different methods are used by 3D content providers to encode 30 image data into video or computer data formats. Major 3D formats are supported to provide the widest possible 5 Application. The main 3D formats supported by the present invention will be described below. The input configuration of the input switch 1006 and the two sets of video processors 1010 and 1012 are also explained. Dual-channel input 3D format ... The dual-channel 3D format contains the actual Transmission of left and right perspective stereo images on separate channels. For example, when two 10 sets of separate video cameras are combined to form a single stereo camera, this format is adopted. The present invention can use a configuration input signal router 1 〇06 provides a dual-channel 3D mode to guide each input channel to a single set of separate video processors 1010 or 1012. For example, if two sets of video sources are presented on digital RGB input bus A and digital video input bus In row b, bus 15 A is directed to video processor A1010 and bus B is directed to video processor B 1012. Of course, other Combination is possible. Another main feature of the present invention is that two sets of separate video processors 1010 and 1012 are used so that the two channels of the dual-channel 3D format can be synchronized independently of each other. This ability is actually, Each input of the video processor 1010 and 1012 can be driven independently. When 20 video processors 1010 and 1012 output, the synchronization of the two channels occurs. Frame sequence input 3D format-Frame sequence 3D format basis The vertical synchronization signal output from the computer data multiplexes the time of the stereo image data. This means that the 313 image field changes with each vertical synchronization signal pulse. One method of the present invention to demultiplex this format is to guide the selected input channels to the video processors 1010, 1012. Video processor A 1010 is then configured to process only "even" frames of image data, while video processor B 1012 is configured to process only "odd," frames. "Even," and "odd," nouns The use is only a convenience measure, because the computer's RGB port does not differ between even and odd image frames. However, in the case that the computer supports the VESA standard stereo socket, the even and odd frame definitions can be slaved. The frame ID signal is derived. The% sequence input 3D format-the field sequence 3D format is very similar to the frame sequence format 'but applies to interlaced video signals. In this case, as 10 and the previous case, it is selected. The channel is directed to the video processors 1010, 1012. Because the multi-video format (for example, NTSC, PAL, etc.) differs between the even and odd fields of each video data frame, it may make the video processors 1010, 1012 Only the even or odd fields of each video frame are processed. Column parent error input 3D format-Column interleaved 3D format is another rgB 15 computer format, which multiplexes stereo image data based on the horizontal synchronization signal signal. This forms a group of multiplexed patterns that follow one another. One of the many methods that the present invention can use to demultiplex the 3D format of the column parent error is to guide a single input to the video processor 1010, 1. 〇12 and then set the memory control register of each video processor 1010, 1012, so that only even or only odd 20 columns can be used for processing. Another method is to arrange the input signal router 10 The processed column becomes blank to guide the selected input channel to the video afL processor 1010, 1012. For example, if video processor a operates on the encoded information on the even line, enter The signal router 1006 will make the odd lines blank. No matter what method is used to demultiplex the columns 29 200419467 format image, each video processor 1010 and 1012 will apply the basic scale factor of 2 in the vertical direction. Restoring the image to its full height. Other scaling factors can be applied to format the resulting image to a display-specific resolution. 5 Up and down input 3D mode ... The up and down 3D format converts left and right stereo images The image data is encoded into the top and bottom halves of each image frame. For example, a set of up and down methods encodes the right perspective data and the bottom half in the top half of each image frame. The left perspective data in the frame. This One of the methods to demultiplex the upper and lower 3D format data is to guide the selected input to the video processor 10 1010, 1012 and then set the memory control register, so that the video processor A 1010 only The top half of the frame is operated and the video processor B 1012 is operated on the bottom half of each flood frame. Other methods are also possible at the same time. Finally, each video processor 1010, 1012 will apply the basic scale factor of 2 in the vertical direction. Take the restored image to full height. Other scaling factors can be used to transform the resulting image into a display-specific resolution. Side-by-side input 3D format ... Side-by-side 3D format encodes each image. Left and right perspective image data on the left and right sides of the frame. As in the previous case, one method of the present invention for demultiplexing the stereo information in this format is to guide the selected channel to the video processors 1010 and 1012. The memory control registers for 20 video processors 1010, 1012 are then configured so that video processor A 1010 operates only on the left side of each frame and video processor B 1012 is on the right of each frame Operate on the side. Similar to the previous single-channel method, each video processor 1010, 1012 will apply the basic size factor of 2 in the horizontal direction to restore the image to its full width and maintain the appropriate appearance ratio. Other scaling factors can be applied to format the resulting image to a display-specific resolution.仃 Parent input 3D format ... line staggered 31) format coded image information Left and right perspective image data on parent line staggered lines. This format corresponds to a change in the 3D image field of the pulse wave for each pixel. As in the previous case, the present invention provides many options for demultiplexing this 3D model, including blanking the data lines on the input pixel clock or passing 4k selected channels to the video processor 1 according to a prescribed route. 10, 1012 and then set the memory control register so that only even or odd rows are processed. Just as the same 3D data formatter 1002 can receive 3D data in many different formats, it can also transmit 3Df data in many different formats—the kind being processed. To provide the widest possible range, a preferred embodiment of the present invention provides a method and device to support all of the following 31) data formats for transmitting 3D stereo information from a 3Df mixer to the desired output. Dual-channel output 3D format ... 0SR 1008 sends output from each video processor to its corresponding output or to the corresponding output. Frame sequence output 3D format ___ The emblem is encoded in the output by using the output vertical synchronization signal money as a reference, and then switching the digital output bus source data between the two sets of video processors 1G10, 1G12. Left and right perspective image data on different frames of the signal. Field sequence output 3D format ... OSR 1008 uses the output vertical synchronization signal signal material city reference to switch between two sets of video processor deletion and digital output bus source data between 1G12 in the hybrid field quasi agency to encode and interleave Output left and right perspective image data on the health field. Output 3D mode up and down ... OSR 1008 compiles a single-frame perspective image by encoding the top half of each frame:-a group of perspective images and another _ perspective image in the bottom half-the left and right sides of the image frame Perspective image data. This is achieved by using the top-bottom identification signal generated from the vertical synchronization signal to switch the digital output bus source data between the two MIMO processors, 1G12. The 14 signal has the same frequency as the vertical synchronization signal but has a 50% basic duty cycle that is modified based on the front and rear entrances. This signal is generated and used inside OSS1008. Side-by-side output 3D format-OSR 1008 encodes the left and right perspective image data in a single image frame by encoding-2 groups of perspective images on the left side of each frame and other-perspective images on the other side. This action is achieved by using the left and right side identification signals generated from the horizontal synchronization signal to switch the digital output bus source data between the two sets of video processors 1_, 1G12. This signal has the same frequency as the vertical sync signal, but has a duty cycle that is adjusted so that the switch is halfway through the visible image. This nickname was generated and used inside OSS1008. The column interleaved output 3D format 0SR 丨 008 encodes the left and right perspective image data of the Ma Shan shirt image frame by encoding a set of perspective images in each frame even number and another set of perspective images in the odd number column. This action is achieved by using the line identification signal generated from the horizontal synchronization signal to switch the digital output bus source data between the two sets of λ processing n and 1012. The line identification signal has half the frequency of the horizontal synchronization signal. Of 50. / 〇 of the shellfish cycle> [5. It was created and used internally in the erasure. Customize the wrong output 3D format _—0Sr 1008. Use the group of perspective images of each frame pair m and the other group of perspective images in the odd rows to compile. Use the left and right perspective image data of the image frame. This action is achieved by the pixel_signal generated by the image pulse to switch the digital output bus source data between 101G and 1G12 in two sets of video locations. The prime identification k number is half the frequency of the pixel clock. The duty cycle written. It is produced and used inside OSR 1GG8. 、 Color exchange dual-channel output 3D format ... This format is similar to the two-pass I-type except that one or two sets of colors are exchanged between views. The k format is used to provide a green-swap dual-channel format used by the following devices, which are described in a commonly owned U.S. patent, serial number 10 09/772128, the case _ on January 29, 2009 , Titled "System and Method for Displaying 3D Images Using Dual Projection 3D Stereoscopic Projection System" 'Its reference is here. In addition, each 3D data transmission format can be used in the input synchronization mode 15 or the output synchronization mode. The input synchronization mode indicates data that is output from the 3D data formatter 1002 and is synchronized with the external 30 signal input to the device. The output synchronization mode indicates the output data of the 3D data formatter 1002 which is independently synchronized from an external 30 input signal. The output synchronization rate is set internally using ocgen 004. 20 The production of the 3D format conversion system as shown above is just an example. Those skilled in the art will readily understand other fabrications of the present invention. All such displacements and variations are within the scope and spirit of the invention as defined by the scope of the appended patent application. [Class brief explanation ^ Ming] 33 200419467 Figure 1 shows a block diagram of an embodiment of a three-dimensional stereo format converter; Figure 2 shows a block diagram of a front-end processor system with digital RGB output, and Figure 3 shows a digital YUV output Figure 5 of the front-end processor system; Figure 4 shows the block diagram of the rear-end processor with digital RGB input; Figure 5 shows the block diagram of the rear-end processor system with digital YUV input,

Figure 6 shows the 10 block diagram of the 48-bit digital RGB_CHVF input data bus; Figure 7 shows the block diagram of the 60-bit digital RGB-CHVF output video data bus; Figure 8 shows the 16-bit digital YHVF video data bus Figure 9 shows a block diagram of an embodiment of a 3D data format converter; and Figure 9 shows a block diagram of another embodiment of a 3D data format converter.

[Symbol table of main components of the diagram] 102 ... 3 data formatter 118 ... back-end processor system 104 ... front-end processor system 120 ... expansion port 106 ... front-end processor system 122 ... expansion port 108 ··· Front-end processor system 124 ... Expansion port 110 ··· Front-end processor system 126 ... Expansion port 112 ··· Back-end processor system 128 ... Expansion 槔 114 ···· Back-end processor system 130 ... Expansion port 116 ··· Back-end processor system 132 ... Expansion port 34 200419467 134 ... Expansion port 136 ... Control system 138 ... Peripheral control bus 140 ... Synchronous information bus 142 ... Core control bus 144 ... Computer control interface 146 ... Network control interface 148 ... User panel interface 150 ... Computer system 152 ... Network 154 ... User control panel 202 ... Front-end analog RGB processor 204 ... Separate analog Color channel input 206 ... Analog HDTV processor 208 ... Analog HDTV input 210 ... Digital RGB processor 212 ... Digital RGB data input 214 ... Digital HDTV processor 216 ... Digital HDTV input 218 ... number of DVI processors 220 ... Video interface standard input 222 ... Front end processor 224 ... Serial digital interface standard input 226 ... Digital RGB input bus 228 ... Digital RGB input bus 302 ... Front end video decoder block 304 ... Video signal input 306 ... YUV input bus 402 ... back-end analog RGB processor 404 ... analog RGB output 406 ... back-end analog HDTV processor 408 ... analog HDTV output 410 ... back digital RGB processor 412 ... digital RGB Output 414 ··· Back-end digital HDTV processor 416 ··· Digital HDTV output 418 ······························································································································································ RGB output bus 502 ... digital YUV input 504 ... back-end video encoder block 602 ... 3 data formatter 702 ... 60-bit digital RGB output video bus 802 ... 16-bit digital YUV bus 902. " 3D Data formatter

35 200419467 904 ·· Microcontroller 906 ··. RGB input data switcher / router system 908—RGB output data switcher / router system 910 ··· Separate video processing unit 912 ··· Separate video processing unit 914 ... memory Body 916 ... Memory 1002 ... 3D data formatter 1004 ... Output clock generator 1006 ... Input signal router 1008 ... Output signal router 1010 ... Video processor A 1012 ... Video processor B 1014 ... Memory block 1016 ... Memory Block 36

Claims (1)

200419467 Patent application scope: 1. A stereo format conversion system, which includes: a plurality of front-end processing systems; a set of 3D data formatters; 5 a plurality of back-end processors; and a set of control systems. 2. —A method for performing stereo format conversion, which includes: Input a set of 3D data streams from one or more sets of multiple 3D formats; process the 3D data; 10 perform 3D data format conversion to generate data in the converted format; process data in the converted format for output to generate a Groups of converted 3D data streams, and outputs converted 3D data streams. 3. A stereo format conversion system, including: a set of front-end processing systems and a set of front-end expansion ports; A set of 3D data formatters; a set of back-end processors and a set of back-end expansion ports; and a set of control systems. 4. A method for performing stereo format conversion, comprising: 20 inputting a group of 3D data streams from a plurality of 3D formats; processing the 3D data in a group of front-end processors or a group of processors added to a group of front-end expansion ports Stream; perform 3D data format conversion to generate data in the converted format; 37 200419467 set of processors on a set of back-end processors or a set of back-end expansion ports to process the converted format of data to generate a set of The converted 3D data stream is used for output; and the converted 3D data stream is output, wherein the stereo format conversion method converts a plurality of 3D formats into any one of the plurality of 3D formats. 5. A stereo format conversion system, which includes: a set of front-end processing systems; A set of 3D data formatters; a set of back-end processors; and 10 sets of control systems, wherein the 3D data formatter converts stereo video and performs a set of functions, which is selected from the group including stereo image scanning, alignment, trimming, Zoom photography, focus correction, aspect ratio conversion, linear scale change, non-linear scale change, scan rate conversion, and any combination containing at least one of the above 15 functions. 6. A stereo format conversion system including: a plurality of front-end processing systems for processing most 3D input formats from one or more groups; a set of 3D data formatters; 20 majority of back-end processors, which For processing most 3D output formats to one or more groups; and a set of control systems, wherein the 3D input format of the one or more groups and the 3D output format of the one or more groups can be independently selected from the following Format family, including 38 200419467 standard 2D; dual channel type; field sequence type; frame sequence type (page turning); up and down type; column interleaving type; side by side type; row interleaving type; spectral multiplexing; and including at least A combination of the foregoing formats. 39