KR20240007142A - 5g 네트워크들을 통한 확장 현실 데이터의 분할 렌더링 - Google Patents

5g 네트워크들을 통한 확장 현실 데이터의 분할 렌더링 Download PDF

Info

Publication number
KR20240007142A
KR20240007142A KR1020237038106A KR20237038106A KR20240007142A KR 20240007142 A KR20240007142 A KR 20240007142A KR 1020237038106 A KR1020237038106 A KR 1020237038106A KR 20237038106 A KR20237038106 A KR 20237038106A KR 20240007142 A KR20240007142 A KR 20240007142A
Authority
KR
South Korea
Prior art keywords
data
virtual objects
sessions
scene
media
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
KR1020237038106A
Other languages
English (en)
Korean (ko)
Inventor
이메드 부아지지
토마스 슈토크하머
니콜라이 콘라드 렁
Original Assignee
퀄컴 인코포레이티드
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/742,168 external-priority patent/US12035020B2/en
Application filed by 퀄컴 인코포레이티드 filed Critical 퀄컴 인코포레이티드
Publication of KR20240007142A publication Critical patent/KR20240007142A/ko
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/765—Media network packet handling intermediate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23412—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/61—Network physical structure; Signal processing
    • H04N21/6106—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
    • H04N21/6131—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via a mobile phone network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/631—Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8455—Structuring of content, e.g. decomposing content into time segments involving pointers to the content, e.g. pointers to the I-frames of the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/11—Positioning of individual sound objects, e.g. moving airplane, within a sound field

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Computer Graphics (AREA)
  • Acoustics & Sound (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Image Generation (AREA)
  • Information Transfer Between Computers (AREA)
KR1020237038106A 2021-05-12 2022-05-12 5g 네트워크들을 통한 확장 현실 데이터의 분할 렌더링 Pending KR20240007142A (ko)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US202163187840P 2021-05-12 2021-05-12
US63/187,840 2021-05-12
US17/742,168 US12035020B2 (en) 2021-05-12 2022-05-11 Split rendering of extended reality data over 5G networks
US17/742,168 2022-05-11
PCT/US2022/028995 WO2022241119A1 (en) 2021-05-12 2022-05-12 Split rendering of extended reality data over 5g networks

Publications (1)

Publication Number Publication Date
KR20240007142A true KR20240007142A (ko) 2024-01-16

Family

ID=81927514

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020237038106A Pending KR20240007142A (ko) 2021-05-12 2022-05-12 5g 네트워크들을 통한 확장 현실 데이터의 분할 렌더링

Country Status (6)

Country Link
US (2) US12489958B2 (https=)
EP (1) EP4338424A1 (https=)
JP (1) JP2024519747A (https=)
KR (1) KR20240007142A (https=)
BR (1) BR112023022995A2 (https=)
WO (1) WO2022241119A1 (https=)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230050360A1 (en) * 2021-08-12 2023-02-16 Tencent America LLC Procedures for providing ar/mr applications to 5g devices by running ar/mr processing on 5g edge servers/cloud including dynamic scene updates
US12597174B2 (en) * 2021-11-03 2026-04-07 Tencent America LLC Method and apparatus for delivering 5G AR/MR cognitive experience to 5G devices
US12309216B2 (en) * 2023-02-14 2025-05-20 Tencent America LLC Method and apparatus for split-rendering embedded in a media streaming architecture for 5G networks
EP4690817A1 (en) * 2023-04-07 2026-02-11 InterDigital CE Patent Holdings, SAS Multi-renderer and computation data in scene description
US20250063079A1 (en) * 2023-08-15 2025-02-20 Nokia Technologies Oy Seamless split rendering session relocation between split rendering servers supporting extended reality applications in mobile communication networks
US20250299410A1 (en) * 2024-03-19 2025-09-25 Qualcomm Incorporated Distributing communication and computing of split rendering of media data

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU761202B2 (en) 1997-09-22 2003-05-29 Sony Corporation Generation of a bit stream containing binary image/audio data that is multiplexed with a code defining an object in ascii format
WO2000001154A1 (en) 1998-06-26 2000-01-06 General Instrument Corporation Terminal for composing and presenting mpeg-4 video programs
US10165577B2 (en) 2012-06-15 2018-12-25 At&T Intellectual Property I, L.P. Dynamic response management leveraging dynamic quality of service allocation
US10986155B2 (en) * 2014-09-29 2021-04-20 Avaya Inc. Segmented video codec for high resolution and high frame rate video
US20170249785A1 (en) 2016-02-29 2017-08-31 Vreal Inc Virtual reality session capture and replay systems and methods
GB2554877B (en) 2016-10-10 2021-03-31 Canon Kk Methods, devices, and computer programs for improving rendering display during streaming of timed media data
US20180276891A1 (en) 2017-03-24 2018-09-27 Pcms Holdings, Inc. System and method for providing an in-context notification of a real-world event within a virtual reality experience
CN110313187B (zh) * 2017-06-15 2022-06-07 杜比国际公司 处理媒体内容以供第一装置再现的方法、系统和装置
WO2019190722A1 (en) 2018-03-28 2019-10-03 Pearson Education, Inc. Systems and methods for content management in augmented reality devices and applications
WO2020139766A2 (en) 2018-12-28 2020-07-02 Pcms Holdings, Inc. System and method for optimizing spatial content distribution using multiple data systems
GB2585479B (en) * 2019-05-10 2024-01-31 Victor Kennedy Roderick Reduction of the effects of latency for extended reality experiences
US11310560B2 (en) 2019-05-17 2022-04-19 Samsung Electronics Co., Ltd. Bitstream merger and extractor
US11405699B2 (en) * 2019-10-01 2022-08-02 Qualcomm Incorporated Using GLTF2 extensions to support video and audio data
US11184417B2 (en) 2019-10-08 2021-11-23 Qualcomm Incorporated System and apparatus for providing network assistance for traffic handling in downlink streaming
US11297116B2 (en) * 2019-12-04 2022-04-05 Roblox Corporation Hybrid streaming
US12035020B2 (en) 2021-05-12 2024-07-09 Qualcomm Incorporated Split rendering of extended reality data over 5G networks

Also Published As

Publication number Publication date
EP4338424A1 (en) 2024-03-20
TW202304216A (zh) 2023-01-16
WO2022241119A1 (en) 2022-11-17
US12489958B2 (en) 2025-12-02
US20240414415A1 (en) 2024-12-12
JP2024519747A (ja) 2024-05-21
BR112023022995A2 (pt) 2024-01-23
US20250317631A1 (en) 2025-10-09

Similar Documents

Publication Publication Date Title
US12035020B2 (en) Split rendering of extended reality data over 5G networks
US11405699B2 (en) Using GLTF2 extensions to support video and audio data
US12489958B2 (en) Split rendering of extended reality data over 5G networks
KR102342274B1 (ko) 이미지에서 가장 관심있는 영역의 진보된 시그널링
KR102252238B1 (ko) 이미지에서의 가장 관심있는 영역
US20190104326A1 (en) Content source description for immersive media data
US20220335694A1 (en) Anchoring a scene description to a user environment for streaming immersive media content
US11665219B2 (en) Processing media data using a generic descriptor for file format boxes
US10567734B2 (en) Processing omnidirectional media with dynamic region-wise packing
JP2020526982A (ja) メディアコンテンツのためのリージョンワイズパッキング、コンテンツカバレッジ、およびシグナリングフレームパッキング
JP2017022715A (ja) コード化ビデオデータのネットワークストリーミング
KR20230173096A (ko) 몰입형 미디어 콘텐츠를 스트리밍하기 위한 사용자 환경에 대한 장면 디스크립션의 앵커링
AU2018301313B2 (en) Processing media data using an omnidirectional media format
KR102654999B1 (ko) 강화된 영역별 패킹 및 뷰포트 독립적 hevc 미디어 프로파일
TWI918936B (zh) 經由５ｇ網路對擴展現實資料的分離渲染
WO2022192885A1 (en) Camera control data for virtual camera in virtual interactive scene defined by streamed media data
CN117256154A (zh) 通过5g网络对扩展现实数据的拆分渲染
JP2024509524A (ja) ストリーミングされるメディアデータによって定義される仮想インタラクティブシーンにおける仮想カメラのためのオブジェクト衝突データ
HK40017804A (en) Enhanced region-wise packing and viewport independent hevc media profile
HK40017031A (en) Processing media data using a generic descriptor for file format boxes
HK40009749A (en) Signaling important video information in network video streaming using mime type parameters
HK40009749B (en) Signaling important video information in network video streaming using mime type parameters

Legal Events

Date Code Title Description
PA0105 International application

Patent event date: 20231103

Patent event code: PA01051R01D

Comment text: International Patent Application

PG1501 Laying open of application
A201 Request for examination
PA0201 Request for examination

Patent event code: PA02012R01D

Patent event date: 20250422

Comment text: Request for Examination of Application