WO1994009595A1 - Procede et appareil comportant une architecture de systeme destinee a des communications multimedia - Google Patents

Procede et appareil comportant une architecture de systeme destinee a des communications multimedia Download PDF

Info

Publication number
WO1994009595A1
WO1994009595A1 PCT/US1992/008018 US9208018W WO9409595A1 WO 1994009595 A1 WO1994009595 A1 WO 1994009595A1 US 9208018 W US9208018 W US 9208018W WO 9409595 A1 WO9409595 A1 WO 9409595A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
processor
pixel
band width
format
Prior art date
Application number
PCT/US1992/008018
Other languages
English (en)
Inventor
Venson M. Shaw
Steven M. Shaw
Original Assignee
Shaw Venson M
Shaw Steven M
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shaw Venson M, Shaw Steven M filed Critical Shaw Venson M
Priority to PCT/US1992/008018 priority Critical patent/WO1994009595A1/fr
Priority to AU33499/93A priority patent/AU684456B2/en
Priority to GB9507794A priority patent/GB2286314B/en
Priority to CA 2147164 priority patent/CA2147164C/fr
Priority claimed from CA 2147164 external-priority patent/CA2147164C/fr
Publication of WO1994009595A1 publication Critical patent/WO1994009595A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/465—Distributed object oriented systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G06F9/4887—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues involving deadlines, e.g. rate based, periodic
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/003—Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto
    • G09G5/006—Details of the interface to the display terminal
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/363—Graphics controllers
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40—Network security protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/0035—User-machine interface; Control console
    • H04N1/00352—Input means
    • H04N1/00392—Other manual input means, e.g. digitisers or writing tablets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N1/32502—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device in systems having a plurality of input or output devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/142—Detection of scene cut or scene change
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44227—Monitoring of local network, e.g. connection or bandwidth variations; Detecting new devices in the local network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/10—Special adaptations of display systems for operation with variable images
    • G09G2320/103—Detection of image changes, e.g. determination of an index representative of the image change
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/02—Handling of images in compressed format, e.g. JPEG, MPEG
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • G09G5/393—Arrangements for updating the contents of the bit-mapped memory
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/30—Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32—Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
    • H04N1/00204—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a digital computer or a digital computer system, e.g. an internet server
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/426—Internal components of the client ; Characteristics thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
    • H04N2201/0077—Types of the still picture apparatus
    • H04N2201/0081—Image reader
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
    • H04N2201/0077—Types of the still picture apparatus
    • H04N2201/0082—Image hardcopy reproducer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
    • H04N2201/0077—Types of the still picture apparatus
    • H04N2201/0084—Digital still camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
    • H04N2201/0077—Types of the still picture apparatus
    • H04N2201/0086—Image transceiver
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
    • H04N2201/0077—Types of the still picture apparatus
    • H04N2201/0087—Image storage device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
    • H04N2201/0077—Types of the still picture apparatus
    • H04N2201/0089—Image display device

Definitions

  • the present invention relates to a method and appa ⁇ ratus for improving the efficiency of electronic communica ⁇ tion and/ in particular/ to a method and apparatus which can communicate with available electronic desk top equipment/ such as personal computers/ workstations/ video cameras/ television/ VCR'sz CD players and telephones and receive/ store/ process and send multiple forms of media information/ such as sound/ image/ graphics/ video and data/ both digital ⁇ ly and algorithmically based on a plurality of selective band widths.
  • available electronic desk top equipment such as personal computers/ workstations/ video cameras/ television/ VCR'sz CD players and telephones and receive/ store/ process and send multiple forms of media information/ such as sound/ image/ graphics/ video and data/ both digital ⁇ ly and algorithmically based on a plurality of selective band widths.
  • Technology allows the individual to communicate with others not only by the telephone/ but also by telefax machines/ personal computers and workstations utilizing modems and telephone lines and data and video information can also be stored and disseminated by means of videotapes/ compact discs and television monitors.
  • An object of the present invention is to define an integrated process architecture which can accommodate com ⁇ munications/ both transmission and retrieval/ of all di ⁇ gitally-coded or algorithmic multimedia information.
  • Another object of the invention is to provide for a novel system architecture which is flexible and allows control of the variable communications band widths and allows for flexible combinations of digitally-coded multiple media information having application to teleconferencing or educational instruction.
  • a still further object of the present invention is to provide for a novel process architecture which not only allows for digital coding techniques/ but also can inter ⁇ face with traditional analog storage or transmission tech ⁇ niques.
  • a still further object of the present invention is to provide for a novel/ process architecture which allows the user to control, program and select the appropriate media combination either before or during the communication session.
  • An apparatus and method for multimedia communications including voice/ audio/ text/ still image/ motion video and animated graphics which permits communications between multimedia transmitters and receivers and which is compatible with multiple standard or customized coding algorithmic signals such as H.261/ MPEG/ V JPEG/ EDTV or HDTV whereby multiple incompatible video coding equipment employing different video coding algorithms can now communicate with each other and which includes a reconfigurable memory for selectively adjusting the internal file format and size so as to be compatible with any available band width.
  • Figure 1 is a pictorial illustration of the communication system
  • Figure 2 is a schematic diagram illustrating the overall system methodology
  • Figure 3 is a schematic of the controller's internal operating mode for illustrating band width management
  • Figure 4 is a schematic of the internal circuitry of the multimedia communications assembly
  • Figure 5 is a schematic of the network communications processor and its design relationship to the transmission processor
  • Figure 6 is a schematic illustrating the communication between the host processor/ system memory/ pixel processor/ frame memory and display processor
  • Figure 7 is a schematic of the video codec and display subsystem
  • Figure 8 is a schematic illustration of the standard CIF and QCIF memory format
  • Figure 9 is a schematic illustration of applicant's scalable memory array reconfigurable technique
  • Figure 10 is a schematic illustrating the pixel processor flexibility to various video coding algorithms
  • Figure 11 is a schematic of the motion processor subsystems
  • Figure 12 illustrates a parallel search method
  • Figure 12A illustrates a programmable logic ' device employing cellular array logic architecture
  • Figure 12B illustrates the implementation of cellular logic processing
  • Figure 13 is a schematic of the multimedia assembly.
  • FIG. 1/ there is shown a pictorial illustration depicting the communication devices available presently for the home or office. These include a VCR 102 / CD player 103/ telephone 104/ television 106/ personal computer 108 and fax machine 110. Each of these communication devices has a distinct function. The telephone can transmit and receive audio and data; a fax machine can transmit and receive text documents/ a television can receive video broadcasts and audio; and a personal computer can be used for many data processing applications. It is Applicant's intention to disclose an assembly which can physically communicate with these electronic devices to permit them to function complimentary with each other and .to communicate with other electronic devices regardless of whether the other communication devices were digital or algorithmic and to code and decode automatically to the available band width.
  • the communication is accomplished by a multimedia communications assembly 112/ being of size and shape/ similar to that of a VCR.
  • the aforementioned electronic devices would interconnect with the multimedia communications assembly 112 to allow the user/operator to control/ complement and utilize the functions of the electronic devices by means of the multi ⁇ media communications assembly 112.
  • Figure 2 illustrates the overall system operation and methodology for the multimedia communications assembly 112.
  • Assembly 112 makes it possible to exchange a multitude of different forms of media objects over a wide range of communication networks.
  • Prior art has shown methods and apparatus to improve compression and decompression techniques for individual media types and individual band width ranges.
  • video coding algorithms are intrinsically incompatible with each other/ there is need for an assembly 112 to provide a common interface whereby incompatible equip ⁇ ment can freely exchange media objects through interfacing with assembly 112.
  • the schematic methodology illustrated in Figure 2 comprises the following major system components. They are a network communications processor 202; a transmission pro ⁇ cessor 204; a pixel processor 206; a motion processor 208; a transform processor 210; a display processor 212; a capture processor 220; a frame memory 214 and a host processor 218.
  • graphics input might possibly be in the form of an RGB format 224; VGA format 226; XGA format 228; or SVGA format 230.
  • Text media objects could be either in the form of a Group 3 format 232; Group 4 format 234; or ASCI format 236.
  • Motion media objects may conform either to H.261 format 238; MPEG format 240; or other specialized formats 242.
  • Still background media objects could be con ⁇ forming either to JPEG format 244 or other specialized formats 234.
  • Input audio media objects could be conformfing to CD audio format 246; voice grade audio 248 or FM audio format 250.
  • Each media object within a category/ namely/ audio/ still image/ motion video, text and graphics would be im ⁇ ported to a multiplexer 252 dedicated to each category in order to identify the input signal and then be directed to a dedicated overlay 254 for each category of media ob ⁇ ject.
  • the overlay 254 provides the ability for the assembly/ disassembly/ deletion/ addition and modification of a selected group of multimedia objects.
  • the input signals/ be they audio, still image/ motion video/ text or graphics/ are converted into computer object-oriented language format for encoding into a frame memory 214 as described hereafter.
  • the system architecture provides for an interface which will enable multiple incompatible video coding equip ⁇ ment employing different video coding algorithms to communi ⁇ cate. This is accomplished through a scalable frame memory architecture reconfigurable technique (SMART) described in Figure 9.
  • SMART scalable frame memory architecture reconfigurable technique
  • Assembly 112 also optimizes the video coding for specific compression ratios in order to meet specific band width requirements.
  • the band width controller 260 receives the band width requirement from the network communication processor 202/ the band width controller 260 will then instruct the host processor 218 to develop the appropriate compression ratio in order to meet the real time performance requirements.
  • Band width controller 260 will also interface with transmission pro ⁇ cessor 204 in order to import and export the media article at the appropriate band width.
  • Assembly 112 can program the network communication processor 202, the transmission processor 204 and the display processor 212 to provide the various types of communications interfaces.
  • the internal operation modes of host processor 218 permit it to adapt to different compression ratio require ⁇ ments and network band width requirements.
  • network band width inter ⁇ faces 1. Communicating over an analog phone line employing V.32 mode / 9/600 bit per second (bps) band width is required, a quarter common immediate frame (QCIF) format is displayed at 7.5 frames per second (fps).
  • V.32 mode / 9/600 bit per second (bps) band width is required, a quarter common immediate frame (QCIF) format is displayed at 7.5 frames per second (fps).
  • QCIF quarter common immediate frame
  • ISDN HI network/ common intermediate frame will be transmitted at 15 frames per second.
  • Tl network/ common intermediate frames (CIF) will be trans ⁇ mitted at 30 frames per. second.
  • the multimedia assembly As a result of the aforesaid plurality of band widths/ it is necessary for the multimedia assembly to continuously monitor the processor and network band width availability and to simultaneously determine the amount of compression or decompression that is required with respect to the data in frame memory 314 to be transmitted. Due to the variable band width or throughput requirement for each transmission network/ only dedicated processor approaches have been shown in the prior art to meet a specific band width performance. For example/ three video conferencing techniques are required at the 112 Kbs, 384 Kbs and 1.544 Mbs band width range.
  • the multimedia assembly disclosed herein/ includes different transceiver pairs for each specific network type.
  • the system architecture disclosed herein/ and in particular/ host pro ⁇ cessor 218 in conjunction with band width controller 260 unit/ sealer 258 and reconfiguration unit 256/ can continu ⁇ ously adapt to a variety of network and processor band width changing situations/ for example/ noisy local line condition and network traffic congestion. This is possible as a result of the scalable memory architecture which permits the con ⁇ tinuous reprogramming of the internal file format of frame memory 214 so that it is suitable for the specific band width requirement at that moment.
  • the appropriate frame size 262 will be adjusted first/ frame by frame difference 264 will then be calculated.
  • an appropriate motion vector 270 can be derived.
  • interpolation techniques 266 can be employed to simulate frame difference signal.
  • Decision logic 272 is employed to analyze situation and make a final decision. In case of scene changes, system will be reset to intraframe codng mode for further processing.
  • a detailed design of the motion processor 208 is further shown in Figure 11.
  • FIG 3 is a schematic illustration of the controller's operating modes for band width management based upon the international compression standard CCITT H.261.
  • each common intermediate format frame (CIF frame) 302 consists of twelve (12) groups of blocks and each group of blocks would consist of thirty-three (33) macro-blocks with each macro-block consisting of six (6) blocks (4 ⁇ 's and 2 U/V's). Each block would consist of 8X8 pixels and each pixel would consist of an 8 bit value.
  • the quarter common intermediate format frame (QCIF frame) 304 would consist of three groups of blocks and these would be identical to those of the CIF's 302.
  • the first interface mode 310 is for 9.6 Kbs analog modems.
  • the second interface mode 312 is for 16 Kbs ISDN D channel.
  • the third network interface mode 314 is for 19.2 Kbs high speed analog modems.
  • the fourth network interface mode 316 is for 56 Kbs digital network (PSDN).
  • the fifth network interface mode 318 is for 64 Kbs ISDN single B channel.
  • the sixth network interface mode 320 is for dual B channel 128 Kbs ISDN BRI network.
  • the seventh network interface mode 322 is for 384 Kbs ISDN HI network and the eighth network in ⁇ terface mode 324 is for 1.544 Mbs ISDN PRI or Tl network.
  • Host processor 218 also has programmable frame updating rate capability 326.
  • Frame updating rate 326 provides host processor 218 with five options. They can be either 30 frame per second (fps); 15 fps; 10 fps; 7.5 fps or 1 fps.
  • the standard frame update rate 326 for each network interface mode 306 would be 1 fps for first network inter ⁇ face mode 310; 1.5 fps for second network interface mode 312; 2 fps for third network interface mode 314; 6.5 fps for fourth network interface mode 316; 7.5 fps for fifth interface mode 318; 15 fps for sixth and seventh interface mode 320 and 322/ respectively and 30 fps for eighth inter ⁇ face mode 324.
  • the CIF format 302 system throughput requires 4.6 mega bytes per second (MBS).
  • the QCIF formal 304 requires 288 kilo bytes per second. Assuming we use 8 kilo bytes per second as the measuring base for. real time video trans ⁇ mission over fifth network interface mode 318/ the CIF format 302 system would require a compression ratio of 576:1 based upon the CCITT H.261 compression standard.
  • the QCIF format 304 would require a 36:1 compression ratio.
  • single QCIF format 304 will be employed for the first through fifth network interface modes 310, 312, 314, 316 and 318, respectively.
  • Double OCIF format will be employed for sixth network interface mode 320 and single CIF format 302 or quadruple QCIF format 304 sequences will be utilized for the seventh and eighth network interface modes 322 and 324.
  • Figure 4 illustrates a schematic view of the multimedia communications assembly 112. It consists of the following major system components. They are a network communications processor 202; a transmission processor 204; a pixel processor 206; a motion processor 208; a transform processor 210; a display processor 212; a capture processor 220; a frame memory 214 and a host processor 218. These system components can be imple ⁇ mented either using custom integrated circuit devices/ a programmable integrated circuit; microprocessor; micro ⁇ controller; digital signal processor or software/ depending upon the specific ⁇ system performance requirement.
  • the system components are interconnected through a system host bus 418 and a high speed video bus 422.
  • the system host bus 418 allows the host processor 218 to control access and communicate with the system components such as the network communication processor 202/ the transmission processor 204/ the pixel processor 206/ and the frame memory 214.
  • the video bus 422 interconnects the frame memory 214 with such components as the capture processor 220/ the display processor 212/ the transform processor 210, the pixel proces ⁇ sor 206 and the motion processor 208 to perform high speed video signal processing functions.
  • Both the system host bus 418 and the video bus 422 are bi-directional parallel buses.
  • the first is the video pipeline 424 consisting of a direct interconnection between the cap ⁇ ture processor 220/ pixel processor 206/ motion processor 208 / transform processor 210/ frame memory 214 and display processor 212.
  • the second system interconnect 342 consists of the direct interconnection between the network communication processor 202/ transmission processor 204, host processor 218 and pixel processor 206. In order to facilitate these interconnect operations, first in, first out memory devices 428 are inserted where appropriate.
  • the frame memory 214 can be implemented either in static random access memory 430 or video random access memory 434.
  • the static random access memory 430 is easier to imple ⁇ ment, but at a higher cost.
  • the video random access memory (VRAM) 434 is less expensive, but slower than the static random access memory 430 and requires a controller 434 to update the memory array.
  • the video random access memory 434 is provided with two access ports 436 and 437 providing access to the random accessible memory array. This is done since many video coding algorithms employ frequent use of the interframe coding 440 to reduce band widths. Namely/ only the frame difference signal 442 will be transmitted. Therefore/ the twin memory accesses are required to store both the new frame 444 and the old frame 448 and to facilitate frame differencing operations 450.
  • the pixel processor 206 serves as the bus master 420 for the video bus 422 by having the video random access memory (VRAM) controller 434 function positioned within the pixel processor 206 core.
  • VRAM video random access memory
  • Pixel processor 206 also is equipped with the bit level manipulation functions 456 such as variable length coder and decoder (VLC) 458/ scan format converter 460 and quantization converter 462. These permit the pixel processor to utilize international video coding algorithms for communicating as discussed hereafter.
  • VLC variable length coder and decoder
  • the capture processor 220 can decode various types of analog video input formats and convert them (e.g./ NTSC 464, PAL 466, SCAM 468, or SVHS 469) to CCIR 601 470 U V 471 4:2:2 472.
  • the ability of the capture processor 220 to de ⁇ code the aforesaid formats provide for a convenient inter ⁇ face between the multimedia communications assembly 112 and the television 106, VCR 102 or video camera 465.
  • the CIF 302 formulated YUV 471 signals will first transfer out of the capture processor 220 and store in the frame memory 214.
  • the luminance (Y) signal 474 will be loaded into the motion processor 208 to perform motion estimation 475.
  • a motion vector 476 will be developed for each macro block 477 and store in the associated frame memory 214 location.
  • the difference between the new and old macro blocks will also be coded in discrete cosine transform (DCT) coefficients 478 using the transform processor 210.
  • Pixel processor 206 will perform a raster to zig-zag scan conversion 460, quanti ⁇ zation 462 and VLC coding 458 of the DCT coefficients 478 for each macro block 477 of luminance 474 and chrominance 473.
  • the transmission processor 204 will format the CIF 302 frames into the CCITT H.261 238 format and attach the appropriate header 481 information.
  • a CIF frame 302 will partition into twelve groups of blocks 482 and each group of blocks 482 will consist of thirty-three macro blocks 477 and each macro block 477 will be composed of four luminance signals 474/ and one U & V signal 473.
  • the network communica ⁇ tion processor 202 will provide the control interface to the telecommunications network 480 or to a microwave link 483.
  • the serial compressed video bit stream 484 will be received from the network communica ⁇ tion processor 202.
  • the bit stream will be converted from serial to parallel and decode the appropriate header message 481 using the transmission processor 204.
  • the information will then be sent to the frame memory 214 through pixel pro ⁇ cessor 206.
  • Pixel processor 206 will then perform a variable length decoder 458/ zig-zag-to-raster scan conversion 460 and dequantization 463.
  • the YUV 471 macro block 477 of DCT coefficients 478 will be sent to frame memory 214 through pixel processor 206.
  • Pixel processor 206 will then send YUV 471 macro blocks 477/ one at a time to the transform processor 210 to perform inverse DCT operation 485.
  • the YUV 471 difference 450 will then be added to the old signal 452 to conform to a new YUV pixel 446 for each macro block 477.
  • the display processor 212 will then perform YUV 471 to RGB 224 conversion and generate analog signal from the RGB 224 or thence generate an 8 bit VGA 226 color image through color mapping 486.
  • the display processor 212 will then provide a convenient interface to various displays such as television 106/ personal computers 108 or monitor.
  • a high speed small computer system interface (SCSI) 488 with the external host 487 such as a personal computer or work station.
  • SCSI small computer system interface
  • the advantage of the small computer system interface 488 is that it provides a system independent in ⁇ terface between the external host 487 and the multimedia communications assembly 112. Since only simplified control messages 489 are required to pass between the two hosts, modi ⁇ fications to the system to provide for various operation for ⁇ mats such as DOS 491/ UNIX 490 or Macintosh 492 can easily be accomplished.
  • the high speed small computer system inter ⁇ face 488 will also allow the transmission of video sequences between the two hosts.
  • the communication pipeline is employed to facilitate real time frame formatting 410/ protocol controlling 412/ transmission and decoding.
  • the host processor 218 is the bus master 420 for the system bus 418. Consequently/ host processor 218 will be able to access to the frame memory 214 and/or system memory 216, and monitor progress through a windowing operation 494.
  • the windowing operation 494 essen ⁇ tially allows a portion of the system memory 216 to be memory mapped 495 to the frame memory 214 so that the host processor 218 can use it as a window to view frame memory 214 status and operations in real time.
  • Figure 5 illustrates the network communication pro ⁇ cessor 202 and its design relationship to transmission processo 204.
  • Network communication processor 202 is comprised of an analog front end transceiver 514/ digital signal processor modem 516 and a buffer memory 518. These network communication processor 202 components are interconnected through a private NCP bus 520.
  • the transmission processor 204 consists of a frame formatter 522/ a protocol controller 524 and an error
  • SUBSTITUTE SHEET processor 526 The transmission processor 204 components and the buffer memory 518 are interconnected through another private X bus 528.
  • the bit-serial D bus 530 facilitates the network communication processor 202 and transmission processor 204 communication through digital signal processor modem 516 and frame formatter 522 sub-systems.
  • the private NCP bus 520 / D bus 530 and X bus 528 are designed to facilitate effective data addressing and transfer in between the sub-system blocks.
  • digital signal processor modem 516 and protocol controller 524 are interconnected to the host processor 218 through system bus 418.
  • the specific requirement of the bus design which may include address 510, data 512 and control 502 sections is dependent upon the data throughput 504, word size 506 and bus contention 508 considerations.
  • the network communications processor 202 implements the DTE 536 function while the host processor 218/ and transmission processor 204, perform the DCE 532 function. This allows the proper pairing of the DCE 532 and DTE 536 interfaced to a local customer premises equipment 534 so as to perform conference control 538, store and forward 540 or band width management 542.
  • digital signal processor modem 516 is the local host controller 544.
  • Analog front end 514 consists of an analog to digital converter (ADC) 546 and a digital to analog conver ⁇ ter (DAC) 548.
  • ADC analog to digital converter
  • DAC digital to analog conver ⁇ ter
  • the analog-to-digital converter 546 samples and holds the analog input signal 550 and converts it to a digital bit stream.
  • the digital-to-analog converter 548 buffers the digital output bit streams and converts them into an analog output signal.
  • the analog front end is the front end interface to the telephone network 480 from the system.
  • the output digital bit stream from the analog-to-digital converter 546 is then transferred to the buffer memory 518 for temporary storage.
  • the digital signal processor modem 516 will access this information ⁇ through buffer memory 518 to perform line coding functions. Inside the digital signal processor modem 516 is a programmable digital signal processor 552. Digital signal processor 552 is programmable allowing for easy imple ⁇ mentation of line coding 554 and control 556 functions for many of the analog front end 514 functions.
  • the frame formatter 522 first received the incoming information frame 558 and header message 481 from the digital signal pro ⁇ cessor modem 516 and identifies the proper receiving video coding algorithm types 560. Protocol controller 524 then takes over and starts the appropriate protocol decoding 562 procedures. Once the control frame 564 and information frame 558 header information are fully decoded, the information frame 558 is sent to the error processor for error checking and correction. Corrected bit streams are then converted from serial to parallel form using serial to parallel con ⁇ verter 568 and are stored in the first in and first out buffer 428 for furthet processing ⁇ The first in, first out buffer 428 is designed into four 32K bits section.
  • Each section allows for storage of 32K bits which is the maximum allow ⁇ ance of a compressed CIF frame. Therefore, 128K bits in the first in, first out buffer allows double buffering and simul ⁇ taneous transmitting and receiving of the incoming and out ⁇ going video information frames.
  • the network communications processor is designed to operate in the following specific speeds.
  • Host processor 218 has three options to lower the bit rate in order to accommodate and correct the degradation.
  • Option 1 would be for the host processor 218 to notify the pixel processor 206 and select a coarser quantization level 572.
  • Option 2 would be to drop the frame update rate and increase the in ⁇ terpolation rate 574.
  • Option 3 would be to drop from CIF to QCIF 576.
  • the error processor 526 detects more than two single bit errors/ the error processor 526 will notify the pixel processor 206 and host processor 218.
  • Host proces ⁇ sor 218 again has two options. Either pixel processor 206 can request for an retransmission or host processor 218 can delete the complete macro block 477 and wait until the next macro block is sent. Meanwhile host processor 218 will send the old macro block 308 from the frame memory 214 and use it to update the display.
  • Figure 6 illustrates the interactions between the front end communication systems and the host processor 218/ system memory 216/ pixel processor 206/ frame memory 214 and display processor 212. These interactions are performed through system bus 418.
  • the incoming video sequence 602 is first received by a front end demodulator 515.
  • Network communica ⁇ tions processor 202 and transmission processor 204 will decode the control message and header information 606.
  • the pixel processor 206 and transform processor 210 will then transform these sequences from frequency domain to pixel domain and store same in the frame memory 214.
  • the display processor 212 performs the appropriate interpolation to display the output video sequence at the selected frame rate.
  • the outgoing video sequence 603 can be prepared through coding of the frame difference 442 for each macro block 477 to con ⁇ vert from pixel to frequency domain. to transmit out through front end modulators 514.
  • a self-contained micro controller 608 could provide the frame formatting 610/ error processing 612 and protocol control functions 524. This would provide service at low bit rate applications up to 64 Kbs range. For higher speed applica ⁇ tions 16 bit or 32 bit high performance embedded micro con ⁇ trollers could be employed.
  • Figure 7 illustrates a block diagram of the design of the video codec and display subsystem 702 and its inter- v action with the transmission processor 204 and host processor
  • the video codec and display subsystem 702 consists of pixel processor 206/ transform processor 210/ frame memory 214 and display processor 212.
  • Pixel processor 206 is the host controller for the video codec and display sub-system 702.
  • Pixel processor 206 is also the controller for the video bus 422.
  • Pixel processor 206 communicates with the host proces ⁇ sor 218 through system bus 418 using its internal host inter ⁇ face circuit 704.
  • Pixel processor 206 also interconnects to transmission processor 204 through a first in, first out memory buffer 706 using its internal serial interface 708.
  • Pixel processor 206 interfaces and controls frame memory 214 through video bus 422 using its internal VRAM controller circuit 434.
  • Pixel processor 206 interfaces with motion processor 208 through video bus 422 and with display processor 212 through private DP bus using its internal display processor decoder 714.
  • the pixel processor 206 also interfaces with transform processor 210 through first in, first out memory 707 and input multiplexer 716.
  • SUBSTITUTESHEET Pixel processor 206 is also required to perform time critical pixel domain video coder and decoder functions 718. These include variable length coder and decoder, run level coder and decoder, quantization and dequantization, zig-zag to raster or rastar to zig-zag scan conversion.
  • Frame memory 214 is designed to store old frames 714 and new frames 712 at two discrete section. Old frame 714 being stored as the reference model while the difference between the new and old frames are being updated via a dif ⁇ ferencing signal 442 which will be either coded for transmis ⁇ sion or decoded and added back to the old frame 714 for the reconstruction of new frame 309.
  • pixel processor 206 will retrieve from the frame memory 214 these frame differencing signals 442 in macro blocks 477. Transform processor 210 will per ⁇ form the DCT (discrete cosine transform) function 716 to trans ⁇ late each of the Y, U, and V block from pixel to frequency domain. The pixel processor 206 will apply these discrete cosine transforms to the decoder or encoder function before
  • SUBSTITUTE SHEET forwarding the coded bit stream to the transmission processor 204 for transmission.
  • pixel processor 206 will retrieve these frame difference bit streams 442 from the transmission processor 204 first in, first out buffer 706, apply the de ⁇ coding procedures, and then communicate with the transform processor 210 through its input first in, first out buffer 707.
  • Transform processor 210 will perform the inverse DCT (discrete cosine transform) operation 485 to derive the pixel domain values for each Y, U and V block 471.
  • These pixel values will be stored in the transform processor output first in, first out 710 until the pixel processor 206 retrieves the old pixel block from frame memory 214.
  • the signal dif ⁇ ferential will then be forwarded to the pixel processor to update the new values of Y, U and V.
  • Transform processor 210 also performs matrix trans ⁇ position 736, two-dimensional filter 738, matrix multiplica ⁇ tion 740 and matrix addition 742. These are required since whenever motion compensation techniques are applied, the old frame 714 must be filtered before it can be added to the new frame difference 442. Additionally, the Inverse (Discrete Cosine Transform) 485 output must be transposed before
  • SUBSTITUTE SHEET final addition The double buffered input 707 and output 710 first in/ first out memories and the input multiplexer 716 are employed to allow the four stage pipeline required for the discrete cosine transform operation. Additional speed may be obtained through the use of additional transform pipeline processor 744 arranged in parallel.
  • CIF format 302 and QCIF format 304 are designed for the transportation of video information over a telecommunication network. They are commonly applied by international coding algorithms such as CCITT H.261 238 and MPEG 240 standards.
  • the CIF format 302 consists of 352 pixels for each horizontal scan line with 288 scan lines on the vertical di ⁇ mension.
  • the CIF format 302 is further partitioned into twelve groups of blocks 482.
  • Each group of block consists of 33 macro blocks 477 and each macro block consists of 4 Y blocks 474/ 1 U block 473 and 1 V block 473 and each block consists of 64 8-bit pixels.
  • the QCIF format 304 consists of 176 pixels for each horizontal scan line with 144 scan lines on the vertical di ⁇ mension.
  • the QCIF format 304 is further partitioned into three groups of blocks 482/ each group of block 410 consist ⁇ ing of 33 macro blocks 477 with each macro block consisting of 4 Y blocks 474, 1 U block 473 and 1 V block 473.
  • Each macro block 477 comprises 384 bytes of YUV data since the frame rate for CIF format 302 is 30 fps (frame per second) and each CIF format 302 frame consists of 396 macro blocks.
  • the band width required to send uncompressed CIF format 149 frames would be 4.6 mega bytes per second which is the equivalent to a total of 576 channels of 64 Kbs B channel
  • Each QCIF format 304 has 99 macro blocks 477 and frame updates at 7.5 fps.
  • the system throughput requires 288 KBs which is the equivalent of 36 channels of 64 KBs based B channel 802. Therefore, an uncompressed CIF format 302 frame transmitti at 30 fps requires 24 Tl lease lines 804 and the QCIF format 304 transmitting at 7.5 fps requires 1.5 Tl lines 804. As such, 75 micro seconds would be required to code an incoming CIF format 304, 1.2 milliseconds would be required for each macro block at 7.5 fps. ⁇ ⁇
  • the CCITT H.261 standard 238 requires a switch from inter to intra frame mode after every 132 frames of transmission in order to avoid accumulative error. This means that in a 30 fps transmission, every 4.4 seconds intra CIF format
  • intra frame coding will be engaged every 17.6 seconds.
  • Figure 9 is a schematic illustration of the scalable memory array reconfigurable technique utilized by Applicant's multimedia assembly 112 in order to optimize the performance for encoding CIF format 302. To achieve 30 fps updates, the time required to encode a macro block 404 is 75 microseconds.
  • a single 8 x 8 DCT operation will consume 6.4 microseconds.
  • pipeline 902 and parallel processing 904 techniques can be applied to improve system performance such as multiple DCT transform pipeline processors 744 can be cas ⁇ caded in parallel as shown in Figure 7, this solution is not acceptable for the consumer based mass market.
  • the scalable memory array reconfigurable technique reduces the standard CIF format 302 to a modified CIF format 906 with slightly coarser resolution and yet retain all of the integrity of the standard CIF format 302 and QCIF format 304.
  • the scalable memory array has the option to choose between the CIF format 302 or QCIF format 304.
  • the modified CIF format 906 provides a 288h X 192v resolution 908 and the modified QCIF format 907 provides a 144h X 96v resolution 910. This provides close to the original CIF and QCIF 302 and 304 quality respectively and also maintains the 4:1:1 integrity of the YUV signal 471.
  • Each CIF format 302 will still retain twelve (12) groups of blocks 482 and each QCIF format 151 will still maintain three (3) groups
  • CCIR 601 image 916 input which consists of 720h X 480v resolution can be downsampled (5:2) 918 to the 288h X 192v Y resolution and further down- sampled 5:1 920 to the 144h X 96v U, V resolution.
  • the Y, U, V can perform 2:5 upsampling 922 for the Y and 1:5 upsampling 924 for the U and V.
  • the signi ⁇ ficance of this modified CIF format 908 design is that the internal processing performance requirement is reduced by 46% which means v we are now allowed to use slower and more economical hardware for encoder processing.
  • memory subsystems such as frame memory 214 and first-in, first-out memory 428, can employ slower memory devices that reduce costs.
  • scalable memory array 926 permits the further scaling down of our modified CIF format 908 to meet either application requirements or cost production require ⁇ ments or to simply drop from a higher resolution format to
  • SUBSTITUTESHEET a coarser resolution format to meet the real time and coding requirement.
  • the CIF frame format could be implemented at 144h X 96v resolution and a QCIF frame format in 72h X 48v resolution. Consequently, the multimedia as ⁇ sembly 112 can employ the standard CIF format 302 or QCIF format 304 when cost and performance are acceptable.
  • the scalable memory array 926 would be adopted so that the CIF and QCIF formats would be adapted as per the following frame selection examples. Mode CIF QCIF TYPE
  • the scalable memory array also allows the parti ⁇ tion of frame memory 214 into sections of modified frames to allow multiple processes to run in each frame section.
  • a frame memory 214 of 352h X 288v size can be scaled down to either a single 288h X 192v section; 4 144h X 98v sections; 16 72h X 48v sections; 64 36 x 24v sections or any of the mixed combinations/ all of the sections being processed in parallel.
  • the scalable memory array can also provide remote MPEG 240 video playback.
  • Standard MPEG provides four times the resolution improvement over the existing CCIR 601
  • SUBSTITUTE SHEET standard Namely/ the standard MPEG 188 can provide 1440h X 960v resolution. The significance is now we are not only able to run each memory section as a parallel process/ but we are also able to provide compatibility between the two standards MPEG 240 and H.261 238. Now, the MPEG standard 240 designed originally only to provide high resolution motion video playback locally can now be used to transmit compressed MPEG programs across the network employing the widely available H.261 video codec facilities.
  • the scalable memory array also enables the user to manage and provide the remote transmission of MPEG 240 video programs employing conference controller 928/ store and N forward 930 nd video distribution 932.
  • a compressed MPEG frame 240 into one of the modified CIF for ⁇ mat 908 or simply send multiple compressed MPEG subframes by partition.
  • a 1440h X 960v MPEG frame 240 can downsample 5:1 into a 288h X 192v modified CIF frame 908 for transmission and decode and upsample at 1:5 to display it at standard MPEG resolution at the corresponding output.
  • the scalable memory array formats have significance in that due to their compact size, they become useful in repre ⁇ senting moving objects in the foreground when the background information is still.
  • the background information would be pretransmitted during the intra frame coding mode 936, while the different moving objects would be transmitted during the interfra e coding mode 938. ⁇ Depending upon the size of the moving object/ the appropriate size of the modified format will be employed. At the decoder end/ the moving objects will be overlayed with the still background context to provide motion sequence.
  • the scalable memory array is particularly suitable to progressive encoding of images when band width needs to be conserved.
  • the scalable memory array will choose the coarser modified CIF format to transmit the initial frames and then utilize a larger modified CIF format to send sub ⁇ sequent frames such that the complete image sequence will gradually be upgraded to the original CIF quality.
  • Pixel processor 206 is the local host controller for the video codec and display subsystem 702 and the host processor 218 is the global host controller for the overall system.
  • the pixel processor 206 serves as the bus master for video bus 422 and host processor 218 serves as the bus master for the system bus 418.
  • Both the video bus 422 and the system bus 418 are system-wide parallel inter ⁇ connects.
  • Video bus 422 is specifically designed to facili ⁇ tate the high speed video information transfer among subsystem components.
  • Figure 10 illustratesa the Pixel processor 206 designing to meet the flexible performance for various types of popular video coding algorithms such as the MPEG/ H.261 or JPEG. Meanwhile, pixel processor 206 can also perform other pixel domain-based proprietary methods. While most pixel algorithms are either inter 936 or intra 938 frame coding, the CCITT and ISO standard algorithms (MPEG, JPEG and H.261) are transformed domain coding methods employing fast DCT im ⁇ plementation and inter frame differencing techniques. Addi ⁇ tionally, MPEG and H.261 also apply motion compensation tech ⁇ niques.
  • the pixel processor 206 is equipped with a 24 bit address line 1002 to permit it to access 16 mega bytes of program memory.
  • the program memory can further be partitioned into separate segments with each segment designated for a specific coding algorithm. Since pixel processor 306 is micro- programmable, it is relatively easy to update the changes while MPEG 240, H.261 238 and JPEG 244 standards are still evolving.
  • the pixel processor 206 is also designed with par ⁇ allel processing in mind.
  • the micro programmable architecture allows multiple pixel processors 206 to couple over video bus 420 to provide concurrent program execution for an extremely high throughput. This will allow each pixel pro ⁇ cessor 206 to be dedicated to a coder 1008 function or a de ⁇ coder 1010 function. If 6 pixel processors 206 are employed, this will allow the concurrent execution of an entire macro block 477. Similarly, the multiplicity of pixel processors depending upon cost and size could permit the process of an entire group of block 482 simultaneously.
  • host processor 218 is somewhat critical in that it must be able to provide an interface with the external host 1006, it must be able to execute the popular DOS 491
  • HEET or UNIX program 490 such as word processing or spread sheet programs and it must be economical.
  • a suggested choice is intel 80286 or 80386 microprocessors. These provide a convenient bus interface with the AT bus which has sufficient bus band width to be used as the system bus 418 of the system.
  • the aforesaid micro-processors also provide compatibility with a wide variety of DOS 491 based software application programs.
  • the small computer system interface 488 is readily available and capable of providing high speed interface between the internal system bus and the external host 1006.
  • FIG 11 is a schematic illustration of motion processor 208 subsystems.
  • motion processor 208 is designed to iden ⁇ tify and specify a motion vector 1102 for each macro block 477 within the existing luminance (Y) frame 474.
  • the motion vector 1102 for the color difference for (U, V) frames 473 can then be derived as either 50% or the truncated integer value of the Y frame.
  • the principle is that for each 16h X 16v source macro block 1108, the surrounding 48h X 48v area 1106 of updated new frame 712 will be needed to be searched and compared.
  • the new macro block 477 having the least dis ⁇ tortion will be identified as the destination macro block 1104 and the distance between the source and destination macro block will be defined as the motion vector 1102.
  • motion processor 208 requires that for each of the four blocks 1109 residing within the old source macro block 1108 of the existing frame, the corresponding destination macro block 1104 centered within the new frame must be identified. Therefore, every correspond ⁇ ing, surrounding 6h X 6v area 1106 of blocks in the new frame must be searched and compared with the old macro block reference in order to derive the best match with least distortion. This approach will require 589, 824 cycles of search and compare operations. Provided the search and compare operations can be fully pipelined, an instruction cycle time of .13 nano seconds is still required which is too time-consuming for the 75 microsecond per macro block real time requirement at 30 fps updates.
  • the motion processor 208 must employ parallel processing and multi-processing techniques.
  • the multimedia assembly 112 incorporates a ⁇ ine grain, tightly coupled, paral ⁇ lel pixel processor architecture 1112 which provides faster speed and better results. This is accomplished by partition ⁇ ing existing macro block 477 into 4 8 X 8 blocks 1109. Four parallel processing arrays 1116 consisting of 24h X 24v pro ⁇ cessor elements are configured into nine (9) regions.
  • SUBSTITUTE SHEET nine regions of macro processor elements 1114 are tightly coupled together. Each region of the existing frame can have direct interconnection and simultaneous access to its eight (8) nearest neighboring regions from the corresponding new frame. Each region of macro processing elements 1114 is des ⁇ ignated to perform various types of pixel domain processing functions for the 8 8 block extracted from the old source macro block 1108.
  • Figure 11 illustrates a parallel search method for 8 X 8 blocks residing within the old source macro block 1108.
  • Each can conduct simultaneous match and compare operations with all of their nine nearest neighboring blocks.
  • the out ⁇ puts of the nine matching operations are first locally stored at the corresponding regional pixel processor arrays 1116. They are then shifted out and summed at the output accumula ⁇ tor 1118 and adder circuits 1120. The results are then compared using the comparator circuit 1122 to obtain the best match.
  • the physical distance between the new macro block which results in the best match and the old reference macro block will be applied as the motion vector for the old luminance macro block.
  • the regional pixel processor array 1116 can be recon- figurable and is designed based upon nine banks of processor element arrays 1126.
  • Each processor element array 882 consists
  • SUBSTITUTE SHEET of sixty-four processor elements 1128 The nine banks of processor element arrays 1126 are interconnected through shift registers 1130 and switches 1132.
  • a vertically-cascaded processor array 1138 crossbar switch array 1134 and shift register array 1136 can be implemented. Additional layer such as storage array can be added to provide the additional functionality. This array will be extremely powerful when multi-layered packaging becomes available for the chip level modules and intergrated circuit technologies.
  • a two-dimensional pixel processor array 1116 can also be designed using nine banks of processor element arrays 1126 equipped with peripheral switches 1132 and shift registers 1130.
  • the switches 1132 can be reconfigurable to guide direction about the date of flow where the shift registers 1130 can transfer data from any processor element array 1126 or input to any other processor element array 1126 or output. Both switches 1132 and shift registers 1130 are byte wide to facili ⁇ tate parallel data flow.
  • the processor element arrays 1126 were designed based upon an 8 X 8 array of simple processor elements 1128.
  • the processor element arrays 1126 are designed for interconnection among the processor elements so that recon ⁇ figuration can be accomplished to meet different application
  • the processor elements 1128 are designed so that each can be programmed to execute simple instructions.
  • Each pro ⁇ cessor element 1128 consists of a simple ALU 1140 which can execute simple instructions such as add, subtract, load, store, compare, etc.
  • Figure 12A illustrates the design example of a program ⁇ mable logic device 1201 which employs a cellular array logic architecture. This figure is used to demonstrate the function ⁇ ality and physical design of the device. The practical size for an N X N array is dependent upon the application require ⁇ ments and the state of the art of the implementing technology.
  • Figure 12B illustrates the practical implementation of a cellular logic processor element 1204 using a charge couple device 970 technology.
  • the objective is to provide an intergrated image sensor array 1206 with the digital pre- processing capabilities so that image coding for the macro blocks and pixel domain image coding functions can be performed.
  • the other objective is to allow the implementation of on-chip parallel image sensing and parallel image processing 976 utili ⁇ zing the same or compatible technology.
  • the cellular array logic architecture illustrated in Figure 12B are useful that they can implement fine grain, tightly-coupled parallel processing systems. They employ single-instruction-multiple-
  • the design illustrated in Figure 12B is based upon a much simpler architecture, the design being dedicated only to image processing and coding applications.
  • the major ob ⁇ jective is to meet real time performance requirements for macro block pixel domain processing. functions or motion pro ⁇ cessing.
  • Figure 12A is employed to demonstrate how frame differencing functions can be performed for each of the in ⁇ coming sub-image macro blocks 477.
  • 3 X 3 array is used to represent macro block sub-image 477 which, from the current frame, is first shifted into the processor element; the corresponding macro block sub-image of the previous frame 1218 is then loaded into the processor element and the comparison functions are performed between the two macro blocks
  • the macro blocks will be marked and the macro block marker 1242 and macro block difference 1244 between the two frames will be stored in frame memory 214. If there is no difference, the current frame macro blockvvalue 1216 will be deleted and the previous frame macro block value 1218 will be used for display updates.
  • macro block processor 1220 will notify host processor and pixel processor 206 and switch the operation from interframe coding 1227 to intraframe coding 1228.
  • the significance is that while incoming images sensed from the camera, the specific macro blocks with the frame differencing can be identified and stored. Consequently, in the interframe coding modes 1227, only those macro blocks 477 requiring motion estimation and compensation 1222, trans ⁇ form coding 1229 or quantization 1226 will be marked and stored in the frame memory 214 to represent the image sequence of the current frame. In the case of scene or illumination changes enough macro blocks will be detected with frame differencing that the system will automatically switch to intraframe coding mode 1228.
  • SUBSTITUTE SHEET Figure 12B illustrates additional pixel domain pro ⁇ cessing functions including low pass filtering 1230; high pass filtering 1232 and quantization 1226.
  • the variable quantization 1226 can be performed by presetting the threshold value .1234 and then shifting and quantisizing the corresponding transform domain coefficients based upon the zig-zag scan format at each low, medium and high frequency regions.
  • the threshold value can be reprogrammed to adjust the quantization level.
  • the differencing macro blocks will be sent to transform processor 210 to perform DCT operation 1224, the output of the DCT coefficients can further be reloaded into the processor element array to perform quantization.
  • band width reduction control 260 is required, initial thresholding is combined with a coarser quantization 1226 level to reduce the image resolution.
  • multiple parallel processor element arrays can be cascaded to perform concurrent macro block operatio such as frame differencing, motion processing and quantization.
  • charge couple device technology 1202 is its suitability for image processing/ multiplexing, and storage operations. This can be done both in the analog and digital domain. Therefore, depending upon the applica ⁇ tion requirement, both analog processing 1238, digital pro ⁇ cessing 1240 and memory functions using these processor ele ⁇ ment arrays 1126 can be accomplished.
  • Figure 13 is a schematic illustration of the func ⁇ tional model architecture in order to simplify the functional processes covered out by the hardware previously discussed.
  • the principal functional elements comprise a band width manager 1300, a formatter 1302, a pixel-domain-codec encoder 1304 coupled with a pixel-domain-codec decoder 1306, a transform- domain-codec encoder 1308 coupled with a transform-domain- codec decoder 1310/ a network-domain-codec encoder 1312 coupled with a network-domain-codec decoder 1314 and a controller 1316.
  • the band width manager 1300 provides band width control capability wherein a two-dimensional band width-over ⁇ lay-lookup-table (BOLUT) can be constructed to map the specific band width ranges/ i.e./ 2.4 Kbs to 100 Mbs/ et al/ into selecti options of media combinations such as overlay in the audio/ video/ text and graphics with various types of quality and resolution.
  • BOLUT two-dimensional band width-over ⁇ lay-lookup-table
  • the band width manager 1300 function is to constant ⁇ ly monitor the network to detect abrupt network band width changes caused by local line degradation or network traffic congestion.
  • the band width manager 1300 will respond by ad ⁇ justing the media combinations to accommodate the available band width.
  • band width manager 1300 operates to reconfigure the different band widths specified by the network providing upgradability and paral ⁇ lelism for time-sharing.
  • the formatter 1302 communicates with the band width manager 1300 to ascertain the band width availability for incoming or outgoing signals.
  • the formatter translates this external information into an internally-operating format.
  • the scalable memory array reconfigurable technique will re ⁇ configure the internal processer and frame memory structure pursuant to the directions of the formatter. This allows the external format to be translated into a suitable internal format to provide system compatibility.
  • the scalable-me ory- array-reconfigurable-technique (SMART) as discussed in Figure 9 is capable of translating a programmable internal format
  • Formatter 1302 identifies the transmit ⁇ ting or receiving coding algorithms, derives their specific format requirements and if these external format requirements are different from the current internal formats, the formatter reformats the horizontal and vertical resolution which results in a separate internal format which is compatible with the external format. These internal format operations, such as the reduction of the horizontal and vertical resolution, are performed by employing interpolation and downsampling tech ⁇ niques or upsampling techniques.
  • the formatter 1302 also communicates with the frame memory so that the frame memory is aware of the internal format to be stored.
  • the network-domain-codec encoder 1312 and decoder 1314 are used to provide line coding and decoding functions.
  • SUBSTITUTE SHEET Network domain codec decoder 1314 would receive network trans- missions via its front end transceiver 1320. It would then perform protocol procedures 1322, network communication pro ⁇ cedures 1324, variable length coding 1326, run length coding 1328 and filtering 1330. The resultant transform coefficients and pixel data will then be forwarded to either pixel-domain- codec decoder 1306 or transform-domain-codec decoder 1310.
  • the network-domain-codec encoder 1312 would receive encoded pixel data or transform coefficients from the other encoders and convert them into serial codes for network transmission performing functions similar to the network domain codec decoder 1314. Simultaneously, band width manager 1300 will interface with encoder 1312 and decoder 1314 to exchange protocol con ⁇ trol and applications information regarding band width avail ⁇ ability.
  • the pixel-domain-codec encoder 1304 and decoder 1306 are designed for custom coding algorithms such as vector quantization, pixel domain operations for the DCT transform based standard coding algorithms such as MPEG, et al, pixel domain operations for motion compensation and image post ⁇ processing functions and analysis and preprocessing techniques for video coding.
  • the pixel-domain-codec provides for pixel domain preprocessing 1332, pixel domain coding 1334,
  • SUBSTITUTE SHEET i age processing 1336 SUBSTITUTE SHEET i age processing 1336, color space conversion 1338, pixel interpolation 1340, vector quantization 1342 and color lookup mapping 1344.
  • the transform-domain-codec encoder 1308 and decoder 1310 are specifically designed for forward and inverse trans ⁇ formation operations required by the international standard coding algorithms such as MPEG, et al.
  • Transform-domain-codec encoder and decoder 1308 and 1310 also provide forward and inverse transform-based operations such as Harr transform and Hadamard transform. Additionally, generic matrix opera ⁇ tions and post-matrix operations, such as scan conversion, quantization and normalization techniques, are performed by the transform-domain-codec.
  • the controller 1316 comprised of either a single or plurality of local host processors which manage the in ⁇ struction sequencing and system control functions for data transfer, memory management, input/output interfacing and processor pipelining.
  • a second controller is a single or plurality of pixel processors used to manage the video pipeline, the scalable memory array reconfigurable technique, frame memories, formatters and display processing. Additional ⁇ ly, the pixel processor is used to perform pixel-domain-codec encoding and decoding functions and can be used in multiples in order to facilitate macro block and group of block process ⁇ ing. Similarly, a single or plurality of transform processors can be employed as coprocessor for the pixel processors, in performing transform-domain-codec encoding and decoding functions.
  • ⁇ v be directed to the pixel-domain-codec or transform-domain- codec after suitable formatting.
  • the media information could then be displayed via the pixel-domain-codec decoder 1306.
  • Origination signals from either storage, camera, TV or CD would be subjected to frame differencing 1364 and frame image capture 1366 before being encoded by pixel-domain-codec en ⁇ coder 1304.
  • These origination signals could then be trans ⁇ mitted via network-domain-codec, encoder 1312 dependent upon the band width manager 1300 and controller 1360 monitoring of band width availability.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • General Engineering & Computer Science (AREA)
  • Discrete Mathematics (AREA)
  • Computing Systems (AREA)
  • Computer Graphics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Appareil de commande et procédé qui permettent d'exécuter une pluralité de fonctions de commande en vue de la communication d'unités multimédia comprenant la voix, le son, le texte, l'image fixe, la vidéo et les graphiques animés. Ledit appareil comporte un système de mémoire d'images qui est compatible avec des signaux algorithmiques de codage multiples standard ou personnalisés tels que H.261, MPEG, JPEG, EDTV ou HDTV. Ledit appareil permet à des équipements de codage vidéo multiples incompatibles employant des algorithmes de codage vidéo différents de communiquer librement entre eux. L'appareil de commande comporte en outre une fonction de gestion de la largeur de bandes destinée à mettre automatiquement à l'échelle le signal de l'unité multimédia pour qu'il s'adapte à la largeur de bande disponible.
PCT/US1992/008018 1991-09-20 1992-10-16 Procede et appareil comportant une architecture de systeme destinee a des communications multimedia WO1994009595A1 (fr)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/US1992/008018 WO1994009595A1 (fr) 1991-09-20 1992-10-16 Procede et appareil comportant une architecture de systeme destinee a des communications multimedia
AU33499/93A AU684456B2 (en) 1992-10-16 1992-10-16 Method and apparatus including system architecture for multimedia communications
GB9507794A GB2286314B (en) 1992-10-16 1992-10-16 Method and apparatus including system architecture for multimedia communications
CA 2147164 CA2147164C (fr) 1991-09-20 1992-10-16 Methode et appareil pour les communications multimedia

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US76345191A 1991-09-20 1991-09-20
PCT/US1992/008018 WO1994009595A1 (fr) 1991-09-20 1992-10-16 Procede et appareil comportant une architecture de systeme destinee a des communications multimedia
CA 2147164 CA2147164C (fr) 1991-09-20 1992-10-16 Methode et appareil pour les communications multimedia

Publications (1)

Publication Number Publication Date
WO1994009595A1 true WO1994009595A1 (fr) 1994-04-28

Family

ID=25677909

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1992/008018 WO1994009595A1 (fr) 1991-09-20 1992-10-16 Procede et appareil comportant une architecture de systeme destinee a des communications multimedia

Country Status (2)

Country Link
US (1) US5600844A (fr)
WO (1) WO1994009595A1 (fr)

Cited By (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2299237A (en) * 1995-03-21 1996-09-25 Samsung Electronics Co Ltd Video decoding apparatus
FR2735318A1 (fr) * 1995-06-08 1996-12-13 Thomson Broadcast Systems Architecture a memoire unique centralisee pour le transfert d'images video
WO1997000579A1 (fr) * 1995-06-14 1997-01-03 Thomson Consumer Electronics, Inc. Systeme d'interface et de bus simple pour appareils numeriques grand public
EP0782333A3 (fr) * 1995-12-25 1998-07-29 Hitachi, Ltd. Appareil d'affichage d'image
US5841987A (en) * 1994-08-19 1998-11-24 Thomson Consumer Electronics, Inc. Simple bus and interface system for consumer digital equipment
EP0883286A2 (fr) * 1997-06-04 1998-12-09 Nec Corporation Méthode de communication à rélais de trames et réseau pour des signaux téléphoniques à largeur de bande fixe et des signaux fac-simile à largeur de bande modifiable
WO1999035788A2 (fr) * 1997-12-31 1999-07-15 Koninklijke Philips Electronics N.V. Infrastructure universelle et sans fil d'information pour immeuble
WO2000036585A1 (fr) * 1998-12-11 2000-06-22 Hitachi, Ltd. Procedes et dispositif de decodage et d'affichage d'images multiples au moyen d'un seul et meme processeur
EP1143694A2 (fr) * 2000-03-27 2001-10-10 Symagery Microsystems Inc. Appareil de capture et traitement d'images
EP1927949A1 (fr) * 2006-12-01 2008-06-04 Thomson Licensing Réseau d'éléments de traitement avec registres locaux
EP2581861A4 (fr) * 2010-06-13 2016-01-27 Shenzhen Mpr Technology Co Ltd Système de plateforme de réseau et son procédé de gestion
WO2016171846A1 (fr) * 2015-04-23 2016-10-27 Google Inc. Compilateur pour effectuer une traduction entre une architecture d'ensemble d'instructions (isa) de processeur d'image virtuelle et un matériel cible ayant une structure de réseau à décalage bidimensionnelle
US9749548B2 (en) 2015-01-22 2017-08-29 Google Inc. Virtual linebuffers for image signal processors
US9756268B2 (en) 2015-04-23 2017-09-05 Google Inc. Line buffer unit for image processor
US9769356B2 (en) 2015-04-23 2017-09-19 Google Inc. Two dimensional shift array for image processor
US9772852B2 (en) 2015-04-23 2017-09-26 Google Inc. Energy efficient processor core architecture for image processor
US9830150B2 (en) 2015-12-04 2017-11-28 Google Llc Multi-functional execution lane for image processor
US9965824B2 (en) 2015-04-23 2018-05-08 Google Llc Architecture for high performance, power efficient, programmable image processing
US9978116B2 (en) 2016-07-01 2018-05-22 Google Llc Core processes for block operations on an image processor having a two-dimensional execution lane array and a two-dimensional shift register
US9986187B2 (en) 2016-07-01 2018-05-29 Google Llc Block operations for an image processor having a two-dimensional execution lane array and a two-dimensional shift register
US10095479B2 (en) 2015-04-23 2018-10-09 Google Llc Virtual image processor instruction set architecture (ISA) and memory model and exemplary target hardware having a two-dimensional shift array structure
US10204396B2 (en) 2016-02-26 2019-02-12 Google Llc Compiler managed memory for image processor
US10284744B2 (en) 2015-04-23 2019-05-07 Google Llc Sheet generator for image processor
US10313641B2 (en) 2015-12-04 2019-06-04 Google Llc Shift register with reduced wiring complexity
US10380969B2 (en) 2016-02-28 2019-08-13 Google Llc Macro I/O unit for image processor
US10387988B2 (en) 2016-02-26 2019-08-20 Google Llc Compiler techniques for mapping program code to a high performance, power efficient, programmable image processing hardware platform
US10546211B2 (en) 2016-07-01 2020-01-28 Google Llc Convolutional neural network on programmable two dimensional image processor
US10915773B2 (en) 2016-07-01 2021-02-09 Google Llc Statistics operations on two dimensional image processor

Families Citing this family (90)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5706290A (en) * 1994-12-15 1998-01-06 Shaw; Venson Method and apparatus including system architecture for multimedia communication
US5983004A (en) * 1991-09-20 1999-11-09 Shaw; Venson M. Computer, memory, telephone, communications, and transportation system and methods
US6424989B1 (en) * 1991-09-20 2002-07-23 Venson M. Shaw Object-oriented transaction computing system
US6441842B1 (en) * 1992-02-19 2002-08-27 8×8, Inc. Video compression/decompression processing and processors
US5434850A (en) 1993-06-17 1995-07-18 Skydata Corporation Frame relay protocol-based multiplex switching scheme for satellite
US6771617B1 (en) 1993-06-17 2004-08-03 Gilat Satellite Networks, Ltd. Frame relay protocol-based multiplex switching scheme for satellite mesh network
JP3528214B2 (ja) * 1993-10-21 2004-05-17 株式会社日立製作所 画像表示方法及び装置
NL9401262A (nl) * 1994-08-01 1996-03-01 Sony Telecom Europ Nv Systeem voor telecommunicatie.
CA2205023C (fr) 1994-11-11 2000-02-29 Istvan Sebestyen Procede et dispositif pour la transmission de donnees telecopieur codees entre des terminaux de communication a capacite multimedia
US7133845B1 (en) * 1995-02-13 2006-11-07 Intertrust Technologies Corp. System and methods for secure transaction management and electronic rights protection
EP1526472A3 (fr) * 1995-02-13 2006-07-26 Intertrust Technologies Corp. Systèmes et procédés de gestion de transactions sécurisées et de protection de droits électroniques
US7095854B1 (en) * 1995-02-13 2006-08-22 Intertrust Technologies Corp. Systems and methods for secure transaction management and electronic rights protection
US6948070B1 (en) * 1995-02-13 2005-09-20 Intertrust Technologies Corporation Systems and methods for secure transaction management and electronic rights protection
JP3375245B2 (ja) * 1995-04-26 2003-02-10 インターナショナル・ビジネス・マシーンズ・コーポレーション 耐障害性マルチメディア・プログラム配布のための装置
US7720672B1 (en) * 1995-12-29 2010-05-18 Wyse Technology Inc. Method and apparatus for display of windowing application programs on a terminal
KR100427136B1 (ko) * 1996-05-30 2004-08-11 마츠시타 덴끼 산교 가부시키가이샤 데이터변환장치
JPH10111792A (ja) * 1996-10-03 1998-04-28 Nec Corp 画像処理装置
US6349297B1 (en) * 1997-01-10 2002-02-19 Venson M. Shaw Information processing system for directing information request from a particular user/application, and searching/forwarding/retrieving information from unknown and large number of information resources
US5852819A (en) * 1997-01-30 1998-12-22 Beller; Stephen E. Flexible, modular electronic element patterning method and apparatus for compiling, processing, transmitting, and reporting data and information
US6247068B1 (en) * 1997-03-07 2001-06-12 Advanced Micro Devices Inc. Winsock-data link library transcoder
GB9706198D0 (en) * 1997-03-25 1997-05-14 Msu Uk Limited Compact disk players and semiconductor chips for the same
US6266091B1 (en) * 1997-07-31 2001-07-24 Lsi Logic Corporation System and method for low delay mode operation video decoding
WO1999023762A1 (fr) 1997-11-03 1999-05-14 Harris Corporation Architecture de systeme radio reconfigurable
EP1223637B1 (fr) 1999-09-20 2005-03-30 Fractus, S.A. Antennes multiniveau
AU7583000A (en) * 1999-09-21 2001-04-24 Wyse Technology, Inc. Displaying windowing application programs on a terminal
US6915508B2 (en) * 2000-06-12 2005-07-05 Sun Microsystems, Inc. Method and apparatus for eliminating C recursion from a Java™ bytecode interpreter
US6778972B2 (en) * 2000-08-10 2004-08-17 Gustavo S. Leonardos′ System and method for providing integrated management of electronic information
US7292968B2 (en) * 2000-09-29 2007-11-06 Cadence Design Systems, Inc. Hot carrier circuit reliability simulation
US7567891B1 (en) 2000-09-29 2009-07-28 Cadence Design Systems, Inc. Hot-carrier device degradation modeling and extraction methodologies
US7225320B2 (en) * 2000-12-28 2007-05-29 Koninklijke Philips Electronics N.V. Control architecture for a high-throughput multi-processor channel decoding system
AUPR329501A0 (en) * 2001-02-22 2001-03-22 Worldlingo, Inc Translation information segment
US7752419B1 (en) 2001-03-22 2010-07-06 Qst Holdings, Llc Method and system for managing hardware resources to implement system functions using an adaptive computing architecture
US7962716B2 (en) * 2001-03-22 2011-06-14 Qst Holdings, Inc. Adaptive integrated circuitry with heterogeneous and reconfigurable matrices of diverse and adaptive computational units having fixed, application specific computational elements
US8843928B2 (en) 2010-01-21 2014-09-23 Qst Holdings, Llc Method and apparatus for a general-purpose, multiple-core system for implementing stream-based computations
US7489779B2 (en) * 2001-03-22 2009-02-10 Qstholdings, Llc Hardware implementation of the secure hash standard
US7653710B2 (en) 2002-06-25 2010-01-26 Qst Holdings, Llc. Hardware task manager
US6836839B2 (en) 2001-03-22 2004-12-28 Quicksilver Technology, Inc. Adaptive integrated circuitry with heterogeneous and reconfigurable matrices of diverse and adaptive computational units having fixed, application specific computational elements
US20040133745A1 (en) 2002-10-28 2004-07-08 Quicksilver Technology, Inc. Adaptable datapath for a digital processing system
US7400668B2 (en) * 2001-03-22 2008-07-15 Qst Holdings, Llc Method and system for implementing a system acquisition function for use with a communication device
US6577678B2 (en) 2001-05-08 2003-06-10 Quicksilver Technology Method and system for reconfigurable channel coding
US6839882B2 (en) * 2001-06-01 2005-01-04 Virtual Silicon Technology, Inc. Method and apparatus for design of integrated circuits
US6681287B2 (en) * 2001-07-02 2004-01-20 Nanoamp Solutions, Inc. Smart memory
US7046635B2 (en) * 2001-11-28 2006-05-16 Quicksilver Technology, Inc. System for authorizing functionality in adaptable hardware devices
US6986021B2 (en) 2001-11-30 2006-01-10 Quick Silver Technology, Inc. Apparatus, method, system and executable module for configuration and operation of adaptive integrated circuitry having fixed, application specific computational elements
US8412915B2 (en) * 2001-11-30 2013-04-02 Altera Corporation Apparatus, system and method for configuration of adaptive integrated circuitry having heterogeneous computational elements
US7602740B2 (en) * 2001-12-10 2009-10-13 Qst Holdings, Inc. System for adapting device standards after manufacture
US7088825B2 (en) * 2001-12-12 2006-08-08 Quicksilver Technology, Inc. Low I/O bandwidth method and system for implementing detection and identification of scrambling codes
US7215701B2 (en) 2001-12-12 2007-05-08 Sharad Sambhwani Low I/O bandwidth method and system for implementing detection and identification of scrambling codes
US7231508B2 (en) * 2001-12-13 2007-06-12 Quicksilver Technologies Configurable finite state machine for operation of microinstruction providing execution enable control value
US7403981B2 (en) * 2002-01-04 2008-07-22 Quicksilver Technology, Inc. Apparatus and method for adaptive multimedia reception and transmission in communication environments
US20030144892A1 (en) * 2002-01-29 2003-07-31 International Business Machines Corporation Method, system, and storage medium for providing knowledge management services
US20070169073A1 (en) * 2002-04-12 2007-07-19 O'neill Patrick Update package generation and distribution network
US7493375B2 (en) * 2002-04-29 2009-02-17 Qst Holding, Llc Storage and delivery of device features
US7328414B1 (en) * 2003-05-13 2008-02-05 Qst Holdings, Llc Method and system for creating and programming an adaptive computing engine
US7660984B1 (en) 2003-05-13 2010-02-09 Quicksilver Technology Method and system for achieving individualized protected space in an operating system
US8108656B2 (en) 2002-08-29 2012-01-31 Qst Holdings, Llc Task definition for specifying resource requirements
US7937591B1 (en) 2002-10-25 2011-05-03 Qst Holdings, Llc Method and system for providing a device which can be adapted on an ongoing basis
US8276135B2 (en) 2002-11-07 2012-09-25 Qst Holdings Llc Profiling of software and circuit designs utilizing data operation analyses
US7225301B2 (en) 2002-11-22 2007-05-29 Quicksilver Technologies External memory controller node
US7609297B2 (en) * 2003-06-25 2009-10-27 Qst Holdings, Inc. Configurable hardware based digital imaging apparatus
US7886093B1 (en) * 2003-07-31 2011-02-08 Hewlett-Packard Development Company, L.P. Electronic device network supporting compression and decompression in electronic devices
US7200837B2 (en) * 2003-08-21 2007-04-03 Qst Holdings, Llc System, method and software for static and dynamic programming and configuration of an adaptive computing architecture
CN1317649C (zh) * 2003-09-19 2007-05-23 联想(北京)有限公司 一种基于安全芯片的计算机终端安全系统
US20050071656A1 (en) * 2003-09-25 2005-03-31 Klein Dean A. Secure processor-based system and method
US8413155B2 (en) 2004-03-13 2013-04-02 Adaptive Computing Enterprises, Inc. System and method for a self-optimizing reservation in time of compute resources
US8782654B2 (en) 2004-03-13 2014-07-15 Adaptive Computing Enterprises, Inc. Co-allocating a reservation spanning different compute resources types
US7904895B1 (en) * 2004-04-21 2011-03-08 Hewlett-Packard Develpment Company, L.P. Firmware update in electronic devices employing update agent in a flash memory card
US20070266388A1 (en) 2004-06-18 2007-11-15 Cluster Resources, Inc. System and method for providing advanced reservations in a compute environment
US8526940B1 (en) 2004-08-17 2013-09-03 Palm, Inc. Centralized rules repository for smart phone customer care
US8176490B1 (en) 2004-08-20 2012-05-08 Adaptive Computing Enterprises, Inc. System and method of interfacing a workload manager and scheduler with an identity manager
US7743377B2 (en) * 2004-10-25 2010-06-22 Microsoft Corporation Cooperative threading in a managed code execution environment
WO2006053093A2 (fr) 2004-11-08 2006-05-18 Cluster Resources, Inc. Systeme et procede fournissant des executions de systeme au sein d'un environnement informatique
US8863143B2 (en) 2006-03-16 2014-10-14 Adaptive Computing Enterprises, Inc. System and method for managing a hybrid compute environment
US9231886B2 (en) 2005-03-16 2016-01-05 Adaptive Computing Enterprises, Inc. Simple integration of an on-demand compute environment
US9015324B2 (en) 2005-03-16 2015-04-21 Adaptive Computing Enterprises, Inc. System and method of brokering cloud computing resources
WO2006107531A2 (fr) 2005-03-16 2006-10-12 Cluster Resources, Inc. Integration simple d'un environnement informatique sur demande
US8782120B2 (en) 2005-04-07 2014-07-15 Adaptive Computing Enterprises, Inc. Elastic management of compute resources between a web server and an on-demand compute environment
CA2603577A1 (fr) 2005-04-07 2006-10-12 Cluster Resources, Inc. Acces a la demande a des ressources informatiques
CN101131695B (zh) * 2006-08-25 2010-05-26 北京书生国际信息技术有限公司 一种文档库系统及其实现方法
US20070207800A1 (en) * 2006-02-17 2007-09-06 Daley Robert C Diagnostics And Monitoring Services In A Mobile Network For A Mobile Device
US20070288909A1 (en) * 2006-06-07 2007-12-13 Hong Kong Applied Science and Technology Research Institute Company Limited Hardware JavaTM Bytecode Translator
US8209676B2 (en) 2006-06-08 2012-06-26 Hewlett-Packard Development Company, L.P. Device management in a network
US8752044B2 (en) 2006-07-27 2014-06-10 Qualcomm Incorporated User experience and dependency management in a mobile device
US8761394B2 (en) * 2006-12-28 2014-06-24 Echostar Technologies L.L.C. System and method for secure broadcast communication
US8041773B2 (en) 2007-09-24 2011-10-18 The Research Foundation Of State University Of New York Automatic clustering for self-organizing grids
US11720290B2 (en) 2009-10-30 2023-08-08 Iii Holdings 2, Llc Memcached server functionality in a cluster of data processing nodes
US10877695B2 (en) 2009-10-30 2020-12-29 Iii Holdings 2, Llc Memcached server functionality in a cluster of data processing nodes
US20110125554A1 (en) * 2009-11-23 2011-05-26 At&T Mobility Ii Llc System and method for implementing a dynamic market
DE102011080169A1 (de) * 2011-08-01 2013-02-07 Robert Bosch Gmbh Kommunikationsanbindung für Sensorik in Fahrzeug-Regelsystemen
CN107113394B (zh) * 2015-01-06 2019-10-18 麦克赛尔株式会社 影像显示装置、影像显示方法以及影像显示系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4295154A (en) * 1978-08-04 1981-10-13 Nippon Telegraph, Telephone Public Corp. Digital video and audio file system
US4477809A (en) * 1982-06-18 1984-10-16 General Electric Company Method for random-access radio-frequency data communications
EP0395266A2 (fr) * 1989-04-27 1990-10-31 Sony Corporation Traitement de signal vidéo dépendant du mouvement
EP0422795A2 (fr) * 1989-10-13 1991-04-17 Gec Plessey Telecommunications Limited Moyen pour la détection d'erreurs et la surveillance de la largeur de bande pour un commutateur de paquets
US5029163A (en) * 1988-03-18 1991-07-02 At&T Bell Laboratories Synchronous protocol data formatter

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4541012A (en) * 1982-01-04 1985-09-10 Compression Labs, Inc. Video bandwidth reduction system employing interframe block differencing and transform domain coding
US4849810A (en) * 1987-06-02 1989-07-18 Picturetel Corporation Hierarchial encoding method and apparatus for efficiently communicating image sequences
US5261080A (en) * 1987-08-21 1993-11-09 Wang Laboratories, Inc. Matchmaker for assisting and executing the providing and conversion of data between objects in a data processing system storing data in typed objects having different data formats
US5369778A (en) * 1987-08-21 1994-11-29 Wang Laboratories, Inc. Data processor that customizes program behavior by using a resource retrieval capability
US5057932A (en) * 1988-12-27 1991-10-15 Explore Technology, Inc. Audio/video transceiver apparatus including compression means, random access storage means, and microwave transceiver means
US5091782A (en) * 1990-04-09 1992-02-25 General Instrument Corporation Apparatus and method for adaptively compressing successive blocks of digital video
US5457780A (en) * 1991-04-17 1995-10-10 Shaw; Venson M. System for producing a video-instruction set utilizing a real-time frame differential bit map and microblock subimages
US5339434A (en) * 1992-12-07 1994-08-16 Trw Inc. Heterogeneous data translation system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4295154A (en) * 1978-08-04 1981-10-13 Nippon Telegraph, Telephone Public Corp. Digital video and audio file system
US4477809A (en) * 1982-06-18 1984-10-16 General Electric Company Method for random-access radio-frequency data communications
US5029163A (en) * 1988-03-18 1991-07-02 At&T Bell Laboratories Synchronous protocol data formatter
EP0395266A2 (fr) * 1989-04-27 1990-10-31 Sony Corporation Traitement de signal vidéo dépendant du mouvement
EP0422795A2 (fr) * 1989-10-13 1991-04-17 Gec Plessey Telecommunications Limited Moyen pour la détection d'erreurs et la surveillance de la largeur de bande pour un commutateur de paquets

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
1988 INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS, vol.1, Espoo, Finland, June 7-9, 1988, pages 227-230, T. Murakami et al.: 'A DSP ARCHITECTURE FOR 64KBPS MOTION VIDEO CODEC' *
COMMUNICATIONS OF THE ACM vol. 34, no. 4, April 1991, NEW YORK, US pages 103 - 112 , XP228793 M. LIEBHOLD ET AL. 'Toward an Open Envirenment for Digital Video' *
ELEKTRONIK vol. 38, no. 13, 23 June 1989, MüNCHEN, DE pages 76 - 86 , XP120497 I. RUGE ET AL. 'Schlüsseltechnologie Mikroelektronik' *
Fourth International Colloquium on Advanced Television Systems Ottawa, CA, June 25-29, 1990; pages 2A.1.1-2A.1.17, Richard S. Prodan: 'SPACE STATION HDTV APPLICATIONS FOR SCIENCE, OPERATIONS, AND MEDIA DISSEMINATION' *
IEEE International Conference on Communications ICC '90 Georgia, US, April 15-19, 1990, pages 439 442, W. Wang et al.: 'BANDWIDTH ALLOCATION FOR ATM NETWORKS' *
International Zurich Seminar on Digital Communications, Zürich, CH, 1984, pages 57-61, Hans C. Bergmann: 'MOTION ADAPTIVE FRAME INTERPOLATION' *
PTR PHILIPS TELECOMMUNICATION AND DATA SYSTEMS REVIEW vol. 47, no. 2, June 1989, HILVERSUM, NL pages 20 - 35 , XP54482 N. HAHN ET AL. 'Video conferencing' *
RTM RUNDFUNKTECHNISCHE MITTEILUNGEN vol. 34, no. 4, July 1990, NORDERSTEDT, DE pages 157 - 164 , XP160597 P. HOU ET AL. 'VERBESSERTE BEWEGUNGSSCHÄTZUNG FÜR FERNSEHBILDER' *

Cited By (76)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5841987A (en) * 1994-08-19 1998-11-24 Thomson Consumer Electronics, Inc. Simple bus and interface system for consumer digital equipment
GB2299237B (en) * 1995-03-21 1997-02-26 Samsung Electronics Co Ltd Apparatus having decoding capability of a higher level for decoding video data of a plurality of channels coded at a lower level
GB2299237A (en) * 1995-03-21 1996-09-25 Samsung Electronics Co Ltd Video decoding apparatus
FR2735318A1 (fr) * 1995-06-08 1996-12-13 Thomson Broadcast Systems Architecture a memoire unique centralisee pour le transfert d'images video
WO1996042077A1 (fr) * 1995-06-08 1996-12-27 Thompson Broadcast Systems Architecture a memoire unique centralisee pour le transfert d'images video
WO1997000579A1 (fr) * 1995-06-14 1997-01-03 Thomson Consumer Electronics, Inc. Systeme d'interface et de bus simple pour appareils numeriques grand public
EP0782333A3 (fr) * 1995-12-25 1998-07-29 Hitachi, Ltd. Appareil d'affichage d'image
US6359903B1 (en) 1997-06-04 2002-03-19 Nec Corporation Frame relay communication method and network for fixed-bandwidth telephone signals and alterable-bandwidth facsimile signals
EP0883286A2 (fr) * 1997-06-04 1998-12-09 Nec Corporation Méthode de communication à rélais de trames et réseau pour des signaux téléphoniques à largeur de bande fixe et des signaux fac-simile à largeur de bande modifiable
EP0883286A3 (fr) * 1997-06-04 1999-07-28 Nec Corporation Méthode de communication à rélais de trames et réseau pour des signaux téléphoniques à largeur de bande fixe et des signaux fac-simile à largeur de bande modifiable
WO1999035788A3 (fr) * 1997-12-31 1999-09-16 Koninkl Philips Electronics Nv Infrastructure universelle et sans fil d'information pour immeuble
WO1999035788A2 (fr) * 1997-12-31 1999-07-15 Koninklijke Philips Electronics N.V. Infrastructure universelle et sans fil d'information pour immeuble
WO2000036585A1 (fr) * 1998-12-11 2000-06-22 Hitachi, Ltd. Procedes et dispositif de decodage et d'affichage d'images multiples au moyen d'un seul et meme processeur
US6204887B1 (en) 1998-12-11 2001-03-20 Hitachi America, Ltd. Methods and apparatus for decoding and displaying multiple images using a common processor
EP1143694A2 (fr) * 2000-03-27 2001-10-10 Symagery Microsystems Inc. Appareil de capture et traitement d'images
EP1143694A3 (fr) * 2000-03-27 2003-11-19 Symagery Microsystems Inc. Appareil de capture et traitement d'images
EP1927949A1 (fr) * 2006-12-01 2008-06-04 Thomson Licensing Réseau d'éléments de traitement avec registres locaux
US7941634B2 (en) 2006-12-01 2011-05-10 Thomson Licensing Array of processing elements with local registers
EP2581861A4 (fr) * 2010-06-13 2016-01-27 Shenzhen Mpr Technology Co Ltd Système de plateforme de réseau et son procédé de gestion
US10277833B2 (en) 2015-01-22 2019-04-30 Google Llc Virtual linebuffers for image signal processors
US10791284B2 (en) 2015-01-22 2020-09-29 Google Llc Virtual linebuffers for image signal processors
US9749548B2 (en) 2015-01-22 2017-08-29 Google Inc. Virtual linebuffers for image signal processors
US10516833B2 (en) 2015-01-22 2019-12-24 Google Llc Virtual linebuffers for image signal processors
US10284744B2 (en) 2015-04-23 2019-05-07 Google Llc Sheet generator for image processor
CN107430760B (zh) * 2015-04-23 2021-01-12 谷歌有限责任公司 用于图像处理器的二维移位阵列
US11190718B2 (en) 2015-04-23 2021-11-30 Google Llc Line buffer unit for image processor
CN107430760A (zh) * 2015-04-23 2017-12-01 谷歌公司 用于图像处理器的二维移位阵列
GB2554204A (en) * 2015-04-23 2018-03-28 Google Inc Compiler for translating between a virtual image processor instruction set architecture (ISA) and target hardware
US9965824B2 (en) 2015-04-23 2018-05-08 Google Llc Architecture for high performance, power efficient, programmable image processing
US11182138B2 (en) 2015-04-23 2021-11-23 Google Llc Compiler for translating between a virtual image processor instruction set architecture (ISA) and target hardware having a two-dimensional shift array structure
US11153464B2 (en) 2015-04-23 2021-10-19 Google Llc Two dimensional shift array for image processor
US10095479B2 (en) 2015-04-23 2018-10-09 Google Llc Virtual image processor instruction set architecture (ISA) and memory model and exemplary target hardware having a two-dimensional shift array structure
US10095492B2 (en) 2015-04-23 2018-10-09 Google Llc Compiler for translating between a virtual image processor instruction set architecture (ISA) and target hardware having a two-dimensional shift array structure
US11138013B2 (en) 2015-04-23 2021-10-05 Google Llc Energy efficient processor core architecture for image processor
US11140293B2 (en) 2015-04-23 2021-10-05 Google Llc Sheet generator for image processor
US10216487B2 (en) 2015-04-23 2019-02-26 Google Llc Virtual image processor instruction set architecture (ISA) and memory model and exemplary target hardware having a two-dimensional shift array structure
US10275253B2 (en) 2015-04-23 2019-04-30 Google Llc Energy efficient processor core architecture for image processor
US9772852B2 (en) 2015-04-23 2017-09-26 Google Inc. Energy efficient processor core architecture for image processor
US9769356B2 (en) 2015-04-23 2017-09-19 Google Inc. Two dimensional shift array for image processor
US10291813B2 (en) 2015-04-23 2019-05-14 Google Llc Sheet generator for image processor
GB2554204B (en) * 2015-04-23 2021-08-25 Google Llc Compiler for translating between virtual image processor instruction set architecture (ISA) and target hardware having a two-dimensional shift array structure
CN110149802B (zh) * 2015-04-23 2021-04-16 谷歌有限责任公司 用于在虚拟图像处理器指令集架构（isa）与具有二维移位阵列结构的目标硬件之间进行转译的编译器
US10321077B2 (en) 2015-04-23 2019-06-11 Google Llc Line buffer unit for image processor
US9785423B2 (en) 2015-04-23 2017-10-10 Google Inc. Compiler for translating between a virtual image processor instruction set architecture (ISA) and target hardware having a two-dimensional shift array structure
WO2016171846A1 (fr) * 2015-04-23 2016-10-27 Google Inc. Compilateur pour effectuer une traduction entre une architecture d'ensemble d'instructions (isa) de processeur d'image virtuelle et un matériel cible ayant une structure de réseau à décalage bidimensionnelle
US10754654B2 (en) 2015-04-23 2020-08-25 Google Llc Energy efficient processor core architecture for image processor
US10719905B2 (en) 2015-04-23 2020-07-21 Google Llc Architecture for high performance, power efficient, programmable image processing
CN110149802A (zh) * 2015-04-23 2019-08-20 谷歌有限责任公司 用于在虚拟图像处理器指令集架构（isa）与具有二维移位阵列结构的目标硬件之间进行转译的编译器
US10397450B2 (en) 2015-04-23 2019-08-27 Google Llc Two dimensional shift array for image processor
US10417732B2 (en) 2015-04-23 2019-09-17 Google Llc Architecture for high performance, power efficient, programmable image processing
US10638073B2 (en) 2015-04-23 2020-04-28 Google Llc Line buffer unit for image processor
US10599407B2 (en) 2015-04-23 2020-03-24 Google Llc Compiler for translating between a virtual image processor instruction set architecture (ISA) and target hardware having a two-dimensional shift array structure
US9756268B2 (en) 2015-04-23 2017-09-05 Google Inc. Line buffer unit for image processor
US10560598B2 (en) 2015-04-23 2020-02-11 Google Llc Sheet generator for image processor
US10313641B2 (en) 2015-12-04 2019-06-04 Google Llc Shift register with reduced wiring complexity
US9830150B2 (en) 2015-12-04 2017-11-28 Google Llc Multi-functional execution lane for image processor
US10185560B2 (en) 2015-12-04 2019-01-22 Google Llc Multi-functional execution lane for image processor
US10477164B2 (en) 2015-12-04 2019-11-12 Google Llc Shift register with reduced wiring complexity
US10998070B2 (en) 2015-12-04 2021-05-04 Google Llc Shift register with reduced wiring complexity
US10204396B2 (en) 2016-02-26 2019-02-12 Google Llc Compiler managed memory for image processor
US10685422B2 (en) 2016-02-26 2020-06-16 Google Llc Compiler managed memory for image processor
US10387988B2 (en) 2016-02-26 2019-08-20 Google Llc Compiler techniques for mapping program code to a high performance, power efficient, programmable image processing hardware platform
US10304156B2 (en) 2016-02-26 2019-05-28 Google Llc Compiler managed memory for image processor
US10387989B2 (en) 2016-02-26 2019-08-20 Google Llc Compiler techniques for mapping program code to a high performance, power efficient, programmable image processing hardware platform
US10504480B2 (en) 2016-02-28 2019-12-10 Google Llc Macro I/O unit for image processor
US10380969B2 (en) 2016-02-28 2019-08-13 Google Llc Macro I/O unit for image processor
US10733956B2 (en) 2016-02-28 2020-08-04 Google Llc Macro I/O unit for image processor
US10334194B2 (en) 2016-07-01 2019-06-25 Google Llc Block operations for an image processor having a two-dimensional execution lane array and a two-dimensional shift register
US10915773B2 (en) 2016-07-01 2021-02-09 Google Llc Statistics operations on two dimensional image processor
US10546211B2 (en) 2016-07-01 2020-01-28 Google Llc Convolutional neural network on programmable two dimensional image processor
US10789505B2 (en) 2016-07-01 2020-09-29 Google Llc Convolutional neural network on programmable two dimensional image processor
US9986187B2 (en) 2016-07-01 2018-05-29 Google Llc Block operations for an image processor having a two-dimensional execution lane array and a two-dimensional shift register
US9978116B2 (en) 2016-07-01 2018-05-22 Google Llc Core processes for block operations on an image processor having a two-dimensional execution lane array and a two-dimensional shift register
US10531030B2 (en) 2016-07-01 2020-01-07 Google Llc Block operations for an image processor having a two-dimensional execution lane array and a two-dimensional shift register
US11196953B2 (en) 2016-07-01 2021-12-07 Google Llc Block operations for an image processor having a two-dimensional execution lane array and a two-dimensional shift register
US12020027B2 (en) 2016-07-01 2024-06-25 Google Llc Convolutional neural network on programmable two dimensional image processor

Also Published As

Publication number Publication date
US5600844A (en) 1997-02-04

Similar Documents

Publication Publication Date Title
US5706290A (en) Method and apparatus including system architecture for multimedia communication
WO1994009595A1 (fr) Procede et appareil comportant une architecture de systeme destinee a des communications multimedia
US6404928B1 (en) System for producing a quantized signal
AU657510B2 (en) Improved image encoding/decoding method and apparatus
US5461679A (en) Method and apparatus for encoding/decoding image data
WO1994001824A1 (fr) Architecture de circuit integre monopuce pour le calcul par jeux d'instructions video
EP1025692B1 (fr) Affectation de ressources de calcul dans un decodeur de trains d'informations
US5689800A (en) Video feedback for reducing data rate or increasing quality in a video processing system
CA2171727C (fr) Compression video realisee au moyen d'un codage iteratif de donnees d'erreur
US6184936B1 (en) Multi-function USB capture chip using bufferless data compression
EP0720374A1 (fr) Appareil pour le décodage en parallele de signaux vidéo numériques
US4888795A (en) Videotelephone apparatus for transmitting high and low resolution video signals over telephone exchange lines
JPH09247679A (ja) スケーラブルｍｐｅｇ２適合ビデオ・エンコーダ
US5905815A (en) Decoding encoded image signals encoded by further transforming transformed DC signals
US6256350B1 (en) Method and apparatus for low cost line-based video compression of digital video stream data
JP2001500688A (ja) 逐次画像圧縮用のビット割当方法
US5434567A (en) Encoding image signals by further transforming transformed DC signals
US5764802A (en) Encoding image signals using a blur image
US6560280B1 (en) Video transmission system
CN1126411A (zh) 并行编码/解码数字视频信号的装置
JPH0787491A (ja) 量子化装置、逆量子化装置及び画像処理装置並びに量子化方法、逆量子化方法及び画像処理方法
AU684456B2 (en) Method and apparatus including system architecture for multimedia communications
CA2147164C (fr) Methode et appareil pour les communications multimedia
EP0720372A1 (fr) Appareil pour le codage/décodage en parallèle de signaux vidéo numériques
JPH07231445A (ja) 画像符号化方法およびその装置

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AT AU BB BG BR CA CH DE DK ES FI GB HU JP KP KR LK LU MG MW NL NO RO RU SD SE US

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LU MC NL SE BF BJ CF CG CI CM GA GN ML MR SN TD TG

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 9507794.7

Country of ref document: GB

Ref document number: 2147164

Country of ref document: CA

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase