TWI723743B - Multimedia system using time of flight and operating method thereof - Google Patents

Multimedia system using time of flight and operating method thereof Download PDF

Info

Publication number
TWI723743B
TWI723743B TW109101328A TW109101328A TWI723743B TW I723743 B TWI723743 B TW I723743B TW 109101328 A TW109101328 A TW 109101328A TW 109101328 A TW109101328 A TW 109101328A TW I723743 B TWI723743 B TW I723743B
Authority
TW
Taiwan
Prior art keywords
time
flight ranging
electronic devices
flight
time length
Prior art date
Application number
TW109101328A
Other languages
Chinese (zh)
Other versions
TW202041881A (en
Inventor
印秉宏
王佳祥
游騰健
Original Assignee
大陸商廣州印芯半導體技術有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 大陸商廣州印芯半導體技術有限公司 filed Critical 大陸商廣州印芯半導體技術有限公司
Priority to US16/850,024 priority Critical patent/US11418274B2/en
Publication of TW202041881A publication Critical patent/TW202041881A/en
Application granted granted Critical
Publication of TWI723743B publication Critical patent/TWI723743B/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H04N25/75—Circuitry for providing, modifying or processing image signals from the pixel array

Abstract

A multimedia system using time-of-flight and an operating method thereof. The multimedia system includes a plurality of electronic devices. The plurality of electronic devices each include a processing module, a time-of-flight module, and a communication module. The time-of-flight module is configured to perform a time-of-flight ranging operation. The communication module is configured to perform wireless communication. The plurality of electronic devices communicate via respective communication modules to formulate an operation protocol and respective unique identifiers, and to perform a time slot synchronization between different electronic devices. The plurality of electronic devices sequentially perform the time-of-flight ranging operation according to the operation protocol and the respective unique identifiers.

Description

應用飛時測距的多媒體系統及其操作方法Multimedia system using time-of-flight ranging and its operation method

本發明是有關於一種測距技術，且特別是有關於一種應用飛時測距（Time-of-Flight，ToF）的多媒體系統及其操作方法。The present invention relates to a ranging technology, and particularly relates to a multimedia system using Time-of-Flight (ToF) and an operating method thereof.

在一般具有由多個穿戴式電子裝置進行互動操作的虛擬實境（Virtual Reality，VR）系統、擴增實境（Augmented Reality，AR）系統或其他多媒體系統當中，所述多個穿戴式電子裝置之間的距離資訊的取得是由所述多個穿戴式電子裝置各別回傳定位資料至主控伺服器來進行分析及運算後，主控伺服器接著再分別回傳對應的距離資訊至所述多個穿戴式電子裝置。對此，所述多個穿戴式電子裝置之間的距離資訊的取得需要花費大量的資料運算時間以及資料傳送時間，而導致在進行互動操作的過程中容易產生延遲，並且持續佔有主控伺服器的一部分的運算資源。有鑑於此，以下將提出幾個實施例的解決方案。Generally, in a Virtual Reality (VR) system, Augmented Reality (AR) system or other multimedia systems that are interactively operated by multiple wearable electronic devices, the multiple wearable electronic devices The distance information is obtained by the multiple wearable electronic devices respectively returning positioning data to the master server for analysis and calculation, and then the master server will then return corresponding distance information to all of them. A number of wearable electronic devices are described. In this regard, the acquisition of distance information between the plurality of wearable electronic devices requires a lot of data calculation time and data transmission time, which may cause delays in the process of interactive operations and continue to occupy the main control server. Part of the computing resources. In view of this, the following will propose solutions in several embodiments.

本發明提供一種應用飛時測距的多媒體系統及其操作方法，可使多媒體系統當中的多個電子裝置的每一個皆能夠有效地進行飛時測距功能。The present invention provides a multimedia system using time-of-flight distance measurement and an operating method thereof, so that each of a plurality of electronic devices in the multimedia system can effectively perform the time-of-flight distance measurement function.

本發明的應用飛時測距的多媒體系統包括多個電子裝置。所述多個電子裝置各別包括處理模組、飛時測距模組以及通信模組。飛時測距模組耦接處理模組，並且用以執行飛時測距操作。通信模組耦接處理模組，並且用以進行無線通信。所述多個電子裝置經由各別的通信模組進行通信，以制訂操作協議以及各別的唯一標識符，並且進行不同電子裝置間時段（time slot）的同步。所述多個電子裝置依據操作協議以及各別的唯一標識符來依序經由各別的飛時測距模組執行飛時測距操作。The multimedia system using time-of-flight ranging of the present invention includes a plurality of electronic devices. The multiple electronic devices each include a processing module, a time-of-flight ranging module, and a communication module. The time-of-flight ranging module is coupled to the processing module and used to perform the time-of-flight ranging operation. The communication module is coupled to the processing module and used for wireless communication. The multiple electronic devices communicate via respective communication modules to formulate operating protocols and respective unique identifiers, and perform time slot synchronization between different electronic devices. The plurality of electronic devices sequentially perform time-of-flight ranging operations via respective time-of-flight ranging modules according to the operating protocol and the respective unique identifiers.

在本發明的一實施例中，上述的操作協議包括所述多個電子裝置的多個飛時測距時段的排序，並且所述多個飛時測距時段彼此未重疊。In an embodiment of the present invention, the aforementioned operating protocol includes a sequence of multiple time-of-flight ranging periods of the multiple electronic devices, and the multiple time-of-flight ranging periods do not overlap with each other.

在本發明的一實施例中，上述的多個電子裝置各別的飛時測距模組透過間接飛時測距法進行飛時測距操作。所述多個電子裝置各別執行飛時測距操作的一個操作周期長度大於一個間接飛時測距周期的時間長度。所述一個間接飛時測距周期的時間長度等於一個光感測的時間長度與一個資料傳輸的時間長度相加。所述一個光感測的時間長度大於所述一個資料傳輸的時間長度。In an embodiment of the present invention, the respective time-of-flight ranging modules of the multiple electronic devices described above perform the time-of-flight ranging operation through the indirect time-of-flight ranging method. The length of one operation cycle for each of the plurality of electronic devices to perform the time-of-flight ranging operation is longer than the time length of one indirect time-of-flight ranging period. The time length of the one indirect time-of-flight ranging period is equal to the time length of a light sensing plus the time length of a data transmission. The time length of the one light sensing is greater than the time length of the one data transmission.

在本發明的一實施例中，上述的多個電子裝置各別的飛時測距模組透過直接飛時測距法進行飛時測距操作。所述多個電子裝置各別執行飛時測距操作的一個操作周期長度等於一個直接飛時測距周期的時間長度。所述一個直接飛時測距周期的時間長度等於一個光感測的時間長度與一個資料傳輸的時間長度相加。所述一個光感測的時間長度小於所述一個資料傳輸的時間長度。In an embodiment of the present invention, the respective time-of-flight ranging modules of the multiple electronic devices described above perform the time-of-flight ranging operation through the direct time-of-flight ranging method. The length of one operation period for the multiple electronic devices to perform time-of-flight ranging operations is equal to the length of a direct time-of-flight ranging period. The time length of the direct time-of-flight ranging period is equal to the time length of a light sensing plus the time length of a data transmission. The time length of the one light sensing is less than the time length of the one data transmission.

在本發明的一實施例中，上述的多媒體系統為虛擬實境系統或擴增實境系統。In an embodiment of the present invention, the aforementioned multimedia system is a virtual reality system or an augmented reality system.

本發明的應用飛時測距的多媒體系統的操作方法包括以下步驟：藉由多個電子裝置經由各別的通信模組進行通信，以制訂操作協議以及各別的唯一標識符，並且進行不同電子裝置間時段（time slot）的同步；以及藉由所述多個電子裝置依據操作協議以及各別的該唯一標識符來依序經由各別的飛時測距模組執行飛時測距操作。The operating method of the multimedia system using time-of-flight distance measurement of the present invention includes the following steps: multiple electronic devices communicate through separate communication modules to formulate operating protocols and separate unique identifiers, and perform different electronic devices. Synchronization of time slots between devices; and the multiple electronic devices sequentially perform time-of-flight ranging operations through respective time-of-flight ranging modules according to the operating protocol and the respective unique identifiers.

基於上述，本發明的應用飛時測距的多媒體系統的操作方法可使多媒體系統中的多個電子裝置依序地進行飛時測距而不會發生信號衝突及誤判的情況發生。Based on the above, the operating method of the multimedia system using the time-of-flight ranging of the present invention enables multiple electronic devices in the multimedia system to perform the time-of-flight ranging in sequence without signal conflict and misjudgment.

為讓本發明的上述特徵和優點能更明顯易懂，下文特舉實施例，並配合所附圖式作詳細說明如下。In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

為了使本發明之內容可以被更容易明瞭，以下特舉實施例做為本發明確實能夠據以實施的範例。另外，凡可能之處，在圖式及實施方式中使用相同標號的元件/構件/步驟，係代表相同或類似部件。In order to make the content of the present invention more comprehensible, the following embodiments are specifically cited as examples on which the present invention can indeed be implemented. In addition, wherever possible, elements/components/steps with the same reference numbers in the drawings and embodiments represent the same or similar parts.

圖1是依照本發明的一實施例的電子裝置的示意圖。參考圖1，電子裝置100包括處理模組110、飛時測距（Time-of-Flight，ToF）模組120以及通信模組130。處理模組110耦接飛時測距模組120以及通信模組130。在本實施例中，電子裝置100可先藉由通信模組130與另一電子裝置進行通信，以制定操作協議（Protocol）以及各自的唯一標識符（Unique identifier, UID），並且進行不同電子裝置間時段（time slot）的同步。唯一標識符用於識別電子裝置100的身分，並且操作協議包括基於不同唯一標識符的多個飛時測距時段（Time slot）的排序。因此，在本實施例中，電子裝置100的處理模組110可接著依據操作協議以及各別的唯一標識符，來判斷對應於自身的唯一標識符的飛時測距時段在操作協議中的排序，以決定飛時測距模組120執行飛時測距的時間。FIG. 1 is a schematic diagram of an electronic device according to an embodiment of the invention. 1, the electronic device 100 includes a processing module 110, a time-of-flight (ToF) module 120 and a communication module 130. The processing module 110 is coupled to the time-of-flight ranging module 120 and the communication module 130. In this embodiment, the electronic device 100 may first communicate with another electronic device through the communication module 130 to formulate an operating protocol (Protocol) and respective unique identifiers (Unique identifier, UID), and perform different electronic devices Time slot synchronization. The unique identifier is used to identify the identity of the electronic device 100, and the operating protocol includes a sequence of multiple time slots based on different unique identifiers. Therefore, in this embodiment, the processing module 110 of the electronic device 100 can then determine the order of the time-of-flight ranging period corresponding to its own unique identifier in the operating protocol according to the operating protocol and the respective unique identifier. , To determine the time when the time-of-flight ranging module 120 executes the time-of-flight ranging.

在本實施例中，處理模組110可例如包括中央處理單元（Central Processing Unit，CPU），或是其他可程式化之一般用途或特殊用途的微處理（Microprocessor）、數位訊號處理器（Digital Signal Processor，DSP）、可程式化控制器、特殊應用積體電路（Application Specific Integrated Circuits，ASIC）、可程式化邏輯裝置（Programmable Logic Device，PLD）、其他類似處理裝置或這些裝置的組合。在本實施例中，通信模組130為無線通信模組，例如WiFi模組。In this embodiment, the processing module 110 may include, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose micro-processing (Microprocessor), digital signal processor (Digital Signal Processor), etc. Processor, DSP), programmable controller, application specific integrated circuit (Application Specific Integrated Circuits, ASIC), programmable logic device (Programmable Logic Device, PLD), other similar processing devices or a combination of these devices. In this embodiment, the communication module 130 is a wireless communication module, such as a WiFi module.

圖2是依照本發明的一實施例的多媒體系統的示意圖。參考圖2，多媒體系統200可例如是虛擬實境（Virtual Reality，VR）系統或擴增實境（Augmented Reality，AR）系統等，本發明並不限於此。多媒體系統200可包括多個電子裝置210~240，並且電子裝置210~240操作在同一虛擬實境應用程式或同一擴增實境程式，以進行互動操作。在本實施例中，電子裝置210~240可例如是穿戴式電子裝置。電子裝置210~240可各別包括虛擬實境或擴增實境的顯示模組以及相關控制電路等，並且還各別包括如圖1實施例的電子裝置100當中的多個模組。Fig. 2 is a schematic diagram of a multimedia system according to an embodiment of the present invention. Referring to FIG. 2, the multimedia system 200 may be, for example, a Virtual Reality (VR) system or an Augmented Reality (AR) system, etc. The present invention is not limited thereto. The multimedia system 200 may include a plurality of electronic devices 210-240, and the electronic devices 210-240 operate in the same virtual reality application or the same augmented reality program for interactive operations. In this embodiment, the electronic devices 210 to 240 may be wearable electronic devices, for example. The electronic devices 210 to 240 may each include virtual reality or augmented reality display modules and related control circuits, etc., and also each include a plurality of modules in the electronic device 100 in the embodiment of FIG. 1.

在本實施例中，電子裝置210~240可藉由各自的通信模組來進行通信以制定操作協議以及各自的唯一標識符。唯一標識符用於識別彼此的身分，並且操作協議包括基於不同唯一標識符的多個飛時測距時段的排序。因此，電子裝置210~240的處理模組110可接著依據操作協議以及各別的唯一標識符，來各自判斷對應於自身的唯一標識符的飛時測距時段在操作協議中的排序，以決定各自執行飛時測距的時間。In this embodiment, the electronic devices 210 to 240 can communicate through their respective communication modules to formulate operating protocols and their unique identifiers. Unique identifiers are used to identify each other's identities, and the operating protocol includes a sequence of multiple time-of-flight ranging periods based on different unique identifiers. Therefore, the processing modules 110 of the electronic devices 210 to 240 can then determine the order of the time-of-flight ranging periods corresponding to their unique identifiers in the operating protocol according to the operating protocol and the respective unique identifiers to determine The time for each to perform time-of-flight ranging.

舉例而言，如圖2所示，電子裝置210~240已制定依序執行飛時測距的順序。因此，電子裝置210的飛時測距模組首先發射感測光201至當前朝向的電子裝置220的穿戴者，並且接收對應回傳的反射光後，即可經運算而取得電子裝置210的穿戴者與電子裝置220的穿戴者之間的距離。以此類推，電子裝置220的飛時測距模組接著發射感測光202至當前朝向的電子裝置230的穿戴者，以進行測距。電子裝置230的飛時測距模組接著發射感測光203至當前朝向的電子裝置220的穿戴者，以進行測距。電子裝置240的飛時測距模組接著發射感測光204至當前朝向的電子裝置220的穿戴者，以進行測距。由於電子裝置210~240可持續依序反覆進行測距，因此依據操作協議的測距時段的排序，電子裝置210的飛時測距模組再次進行測距，以發射感測光205至當前朝向的電子裝置240的穿戴者（電子裝置210的穿戴者可能轉向），以取得電子裝置210的穿戴者與電子裝置240的穿戴者之間的當前距離。For example, as shown in FIG. 2, the electronic devices 210 to 240 have established a sequence for performing time-of-flight ranging. Therefore, the time-of-flight ranging module of the electronic device 210 first emits the sensing light 201 to the wearer of the electronic device 220 that is currently facing, and after receiving the reflected light corresponding to the return, the wearer of the electronic device 210 can be obtained by calculation. The distance from the wearer of the electronic device 220. By analogy, the time-of-flight ranging module of the electronic device 220 then emits the sensing light 202 to the wearer of the electronic device 230 that is currently facing to perform distance measurement. The time-of-flight distance measurement module of the electronic device 230 then emits the sensing light 203 to the wearer of the electronic device 220 currently facing to perform distance measurement. The time-of-flight distance measurement module of the electronic device 240 then emits the sensing light 204 to the wearer of the electronic device 220 that is currently facing to perform distance measurement. Since the electronic devices 210 to 240 can continuously perform ranging in sequence, the time-of-flight ranging module of the electronic device 210 performs ranging again according to the sequence of the ranging period of the operating protocol to emit the sensing light 205 to the current orientation The wearer of the electronic device 240 (the wearer of the electronic device 210 may turn) to obtain the current distance between the wearer of the electronic device 210 and the wearer of the electronic device 240.

據此，本實施例的多媒體系統200的電子裝置210~240可有效且快速地取得彼此的距離，並且還可透過通信模組上傳給彼此或主控伺服器，以便於進行中的應用操作可即時地取得電子裝置210~240之間的距離資訊，而進行相對應的操作。Accordingly, the electronic devices 210 to 240 of the multimedia system 200 of this embodiment can effectively and quickly obtain the distance between each other, and can also upload to each other or the master server through the communication module, so that the application operation in progress can be easily performed. The distance information between the electronic devices 210-240 is obtained in real time, and the corresponding operation is performed.

圖3是依照本發明的一實施例的間接飛時測距（Indirect Time-of-Flight，I-ToF）的信號時序圖。參考圖2以及圖3，時序I-ToF表示單一飛時測距模組進行週期性的測距操作的時序。依據時序I-ToF可知，一個間接飛時測距周期的時間長度P0等於一個光感測的時間長度PA（斜線處）與一個資料傳輸的時間長度PB（非斜線處）相加。在本實施例中，光感測的時間長度PA係指飛時測距模組當中的光發射單元發射感測光至飛時測距模組當中的光感測單元接收相應的反射光的時間差的時間長度。資料傳輸時間長度PB係指飛時測距模組當中的類比至數位轉換器（Analog-to-Digital Converter，ADC）電路輸出距離資料的時間長度。在本實施例中，時序T1~T4為分別對應於電子裝置210~240各別的飛時測距模組進行週期性的測距操作的時序。FIG. 3 is a signal timing diagram of Indirect Time-of-Flight (I-ToF) according to an embodiment of the present invention. Referring to FIG. 2 and FIG. 3, the timing I-ToF represents the timing of the periodic ranging operation performed by a single time-of-flight ranging module. According to the timing I-ToF, the time length P0 of an indirect time-of-flight ranging period is equal to the time length PA of a light sensing (at the diagonal line) and the time length PB (not at the diagonal line) of data transmission added. In this embodiment, the time length PA of light sensing refers to the time difference between the light emitting unit in the time-of-flight ranging module emitting sensing light to the time difference between the light sensing unit in the time-of-flight ranging module receiving the corresponding reflected light length of time. The data transmission time length PB refers to the length of time for the analog-to-digital converter (ADC) circuit in the time-of-flight ranging module to output distance data. In this embodiment, the time sequences T1 to T4 respectively correspond to the time sequences of the time-of-flight ranging modules of the electronic devices 210 to 240 performing periodic ranging operations.

對此，電子裝置210~240各別的飛時測距模組是透過間接飛時測距法進行飛時測距操作。間接飛時測距法是透過計算感測光的波形與反射光的波形間的相位差來換算距離，因此其響應所需時間較長，而使所述一個光感測的時間長度PA大於所述一個資料傳輸的時間長度PB。換言之，由於所述一個光感測的時間長度PA大於所述一個資料傳輸的時間長度PB，因此當電子裝置210~240各別執行飛時測距操作的一個操作周期長度P1必然會大於一個間接飛時測距周期的時間長度P0。In this regard, the respective time-of-flight ranging modules of the electronic devices 210 to 240 perform the time-of-flight ranging operation through the indirect time-of-flight ranging method. The indirect time-of-flight ranging method is to calculate the phase difference between the waveform of the sensed light and the waveform of the reflected light to convert the distance. Therefore, the response time is longer, and the time length PA of the one light sensing is greater than the The length of time PB for a data transmission. In other words, since the time length PA of the one light sensing is greater than the time length PB of the one data transmission, when the electronic devices 210 to 240 each perform the time-of-flight ranging operation, the length of one operation period P1 will inevitably be longer than the one indirect. The time length P0 of the time-of-flight ranging period.

詳細而言，參照時序T1~T4，電子裝置220需等待電子裝置210的光感測結束後才接續進行光感測。以此類推，當電子裝置240的光感測結束後，電子裝置210才可重新進行下一輪的光感測。也就是說，電子裝置210~240可依據間接飛時測距法來依序進行測距，但更新頻率（Refresh Rate）將會下降。另外，上述的飛時測距時段的順序係指在時序T1~T4中各別的光感測期間（斜線處）的順序。In detail, referring to the time sequence T1 to T4, the electronic device 220 needs to wait for the light sensing of the electronic device 210 to finish before continuing to perform the light sensing. By analogy, after the light sensing of the electronic device 240 ends, the electronic device 210 can perform the next round of light sensing again. In other words, the electronic devices 210 to 240 can perform ranging in sequence according to the indirect time-of-flight ranging method, but the refresh rate (Refresh Rate) will decrease. In addition, the sequence of the time-of-flight ranging period mentioned above refers to the sequence of the respective light sensing periods (hatched lines) in the time sequence T1 to T4.

圖4是依照本發明的一實施例的直接飛時測距（Direct Time-of-Flight，D-ToF）的信號時序圖。參考圖2以及圖4，時序D-ToF表示單一飛時測距模組進行週期性的測距操作的時序。依據時序D-ToF可知，一個直接飛時測距周期的時間長度P0’等於一個光感測的時間長度PA’（斜線處）與一個資料傳輸的時間長度PB’（非斜線處）相加。在本實施例中，光感測的時間長度PA’係指飛時測距模組當中的光發射單元發射感測光至飛時測距模組當中的光感測單元接收相應的反射光的時間差的時間長度。資料傳輸時間長度PB’係指飛時測距模組當中的類比至數位轉換器電路輸出距離資料的時間長度。在本實施例中，時序T1’~T4’為分別對應於電子裝置210~240各別的飛時測距模組進行週期性的測距操作的時序。FIG. 4 is a signal timing diagram of Direct Time-of-Flight (D-ToF) according to an embodiment of the present invention. Referring to FIG. 2 and FIG. 4, the timing D-ToF represents the timing of the periodic ranging operation performed by a single time-of-flight ranging module. According to the time sequence D-ToF, the time length P0' of a direct time-of-flight ranging period is equal to the time length of a light sensing PA' (at the diagonal line) plus the time length of a data transmission PB' (at the non-slashed line). In this embodiment, the time length of light sensing PA' refers to the time difference between the light emitting unit in the time-of-flight ranging module emitting sensing light to the time difference between the light sensing unit in the time-of-flight ranging module receiving corresponding reflected light The length of time. The data transmission time length PB' refers to the time length of the distance data output from the analog-to-digital converter circuit in the time-of-flight ranging module. In this embodiment, the time sequences T1' to T4' are the time sequences corresponding to the periodical ranging operations performed by the time-of-flight ranging modules of the electronic devices 210-240, respectively.

對此，電子裝置210~240各別的飛時測距模組是透過直接飛時測距法進行飛時測距操作時。直接飛時測距法是透過計算發射感測光與接收反射光之間的時間差來換算距離，因此其響應較快，而使所述一個資料傳輸的時間長度PB’大於所述一個光感測的時間長度PA’。換言之，由於所述一個光感測的時間長度PA’遠小於所述一個資料傳輸的時間長度PB’，因此當電子裝置210~240各別執行飛時測距操作的一個操作周期長度P1’可等於一個直接飛時測距周期的時間長度P0’。In this regard, the respective time-of-flight ranging modules of the electronic devices 210 to 240 perform the time-of-flight ranging operation through the direct time-of-flight ranging method. The direct time-of-flight ranging method calculates the distance by calculating the time difference between the emitted sensing light and the received reflected light, so its response is faster, and the time length PB' of the one data transmission is greater than that of the one light sensing The length of time PA'. In other words, since the time length PA' of the one light sensing is much smaller than the time length PB' of the one data transmission, the length of one operation period P1' of the time-of-flight ranging operation performed by the electronic devices 210 to 240 can be It is equal to the length of time P0' of a direct time-of-flight ranging period.

詳細而言，參照時序T1’~T4’，電子裝置220需等待電子裝置210的光感測結束後才接續進行光感測。以此類推，當電子裝置240的光感測結束後，電子裝置210可剛好結束輸出距離資料，而無須等待以直接接續進行下一輪的光感測。也就是說，電子裝置210~240可依據直接飛時測距法來依序進行測距，但相較於圖3實施例，其更新頻率不會降低。另外，上述的飛時測距時段的順序係指在時序T1’~T4’中各別的光感測期間（斜線處）的順序。In detail, referring to the time sequence T1'~T4', the electronic device 220 needs to wait for the light sensing of the electronic device 210 to finish before continuing to perform the light sensing. By analogy, when the light sensing of the electronic device 240 ends, the electronic device 210 can just finish outputting the distance data without waiting to directly continue the next round of light sensing. In other words, the electronic devices 210 to 240 can perform ranging in sequence according to the direct time-of-flight ranging method, but compared with the embodiment in FIG. 3, the update frequency thereof will not be reduced. In addition, the sequence of the time-of-flight ranging period mentioned above refers to the sequence of the respective light sensing periods (hatched lines) in the time sequence T1'~T4'.

圖5是依照本發明的一實施例的多媒體系統的操作方法的流程圖。參考圖2以及圖5，本實施例的操作方法可適用於圖2的多媒體系統200。在步驟S510中，電子裝置210~240經由各別的通信模組進行通信，以制訂操作協議以及各別的唯一標識符，並且進行不同電子裝置間時段（time slot）的同步。在步驟S520中，電子裝置210~240依據該操作協議以及各別的所述唯一標識符來依序經由各別的飛時測距模組執行飛時測距操作。因此，本實施例的操作方法可使多媒體系統200中的多個電子裝置210~240依序地進行飛時測距而不會發生信號衝突及誤判的情況發生。Fig. 5 is a flowchart of an operating method of a multimedia system according to an embodiment of the present invention. Referring to FIG. 2 and FIG. 5, the operation method of this embodiment can be applied to the multimedia system 200 of FIG. 2. In step S510, the electronic devices 210 to 240 communicate via respective communication modules to formulate operating protocols and respective unique identifiers, and perform time slot synchronization between different electronic devices. In step S520, the electronic devices 210 to 240 sequentially execute the time-of-flight ranging operation via the respective time-of-flight ranging module according to the operating protocol and the respective unique identifiers. Therefore, the operating method of the present embodiment enables the multiple electronic devices 210 to 240 in the multimedia system 200 to sequentially perform time-of-flight ranging without signal conflict and misjudgment.

另外，關於本實施例的多媒體系統200以及電子裝置210~240的其他元件特徵、實施細節以及技術特徵，可參考上述圖1至圖4的各實施例的說明而獲致足夠的教示、建議以及實施說明，因此在此不多加贅述。In addition, regarding other component features, implementation details, and technical features of the multimedia system 200 and the electronic devices 210 to 240 of this embodiment, please refer to the description of each embodiment in FIG. 1 to FIG. 4 to obtain sufficient teachings, suggestions, and implementations. Explain, so I won’t repeat it here.

綜上所述，本發明的應用飛時測距的多媒體系統及其操作方法可藉由飛時測距的方式來提供有效且即時的測距功能，並且本發明的多媒體系統及其操作方法可使多媒體系統中的多個電子裝置先經由通信模組進行通信，以制定操作協議以及各自的唯一標識符，再接著依序地各自進行飛時測距而不會發生信號衝突及誤判的情況發生。In summary, the multimedia system and operating method of the present invention using time-of-flight ranging can provide effective and real-time ranging functions by means of time-of-flight ranging, and the multimedia system and operating method of the present invention can Enable multiple electronic devices in the multimedia system to communicate through the communication module to formulate operating protocols and their own unique identifiers, and then perform time-of-flight ranging in sequence without signal conflicts and misjudgments. .

雖然本發明已以實施例揭露如上，然其並非用以限定本發明，任何所屬技術領域中具有通常知識者，在不脫離本發明的精神和範圍內，當可作些許的更動與潤飾，故本發明的保護範圍當視後附的申請專利範圍所界定者為準。Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be determined by the scope of the attached patent application.

100、210~240:電子裝置 110:處理模組 120:飛時測距模組 130:通信模組 200:多媒體系統 201~205:感測光 PA、PB、P0、P1、PA’、PB’、P0’、P1’:時間長度 I-ToF、D-ToF、T1、T2、T3、T4、T1’、T2’、T3’、T4’:時序 S510、S520:步驟100, 210~240: electronic device 110: Processing module 120: On-the-fly ranging module 130: Communication module 200: Multimedia system 201~205: Sensing light PA, PB, P0, P1, PA’, PB’, P0’, P1’: length of time I-ToF, D-ToF, T1, T2, T3, T4, T1’, T2’, T3’, T4’: Timing S510, S520: steps

圖1是依照本發明的一實施例的電子裝置的示意圖。 圖2是依照本發明的一實施例的多媒體系統的示意圖。 圖3是依照本發明的一實施例的間接飛時測距（Indirect Time-of-Flight，I-ToF）的信號時序圖。 圖4是依照本發明的一實施例的直接飛時測距（Direct Time-of-Flight，D-ToF）的信號時序圖。 圖5是依照本發明的一實施例的多媒體系統的操作方法的流程圖。FIG. 1 is a schematic diagram of an electronic device according to an embodiment of the invention. Fig. 2 is a schematic diagram of a multimedia system according to an embodiment of the present invention. FIG. 3 is a signal timing diagram of Indirect Time-of-Flight (I-ToF) according to an embodiment of the present invention. FIG. 4 is a signal timing diagram of Direct Time-of-Flight (D-ToF) according to an embodiment of the present invention. Fig. 5 is a flowchart of an operating method of a multimedia system according to an embodiment of the present invention.

100:電子裝置 100: electronic device

110:處理模組 110: Processing module

120:飛時測距模組 120: On-the-fly ranging module

130:通信模組 130: Communication module

Claims (8)

一種應用飛時測距的多媒體系統，包括：多個電子裝置，各別包括：一處理模組；一飛時測距模組，耦接該處理模組，並且用以執行一飛時測距操作；以及一通信模組，耦接該處理模組，並且用以進行無線通信，其中該些電子裝置經由各別的該通信模組進行通信，以制訂一操作協議以及各別的一唯一標識符，並且進行不同電子裝置間時段(time slot)的同步，並且該些電子裝置依據該操作協議以及各別的該唯一標識符來依序經由各別的該飛時測距模組執行該飛時測距操作，其中該操作協議包括該些電子裝置的多個飛時測距時段的排序，並且該些飛時測距時段彼此未重疊。 A multimedia system using time-of-flight ranging, including: a plurality of electronic devices, each including: a processing module; a time-of-flight ranging module, coupled to the processing module, and used to perform a time-of-flight ranging Operation; and a communication module coupled to the processing module and used for wireless communication, wherein the electronic devices communicate through the respective communication modules to formulate an operating protocol and a unique identifier Time slot synchronization between different electronic devices, and the electronic devices execute the flight through the respective time-of-flight ranging modules in sequence according to the operating protocol and the respective unique identifiers. Time-of-flight ranging operation, wherein the operating protocol includes a sequence of multiple time-of-flight ranging periods of the electronic devices, and the time-of-flight ranging periods do not overlap with each other. 如申請專利範圍第1項所述的多媒體系統，其中該些電子裝置各別的該飛時測距模組透過一間接飛時測距法進行該飛時測距操作，並且該些電子裝置各別執行該飛時測距操作的一個操作周期長度大於一個間接飛時測距周期的時間長度，其中該一個間接飛時測距周期的時間長度等於一個光感測的時間長度與一個資料傳輸的時間長度相加，並且該一個光感測的時間長度大於該一個資料傳輸的時間長度。 For example, in the multimedia system described in claim 1, wherein the time-of-flight ranging module of each of the electronic devices performs the time-of-flight ranging operation through an indirect time-of-flight ranging method, and each of the electronic devices Do not perform the operation period of the time-of-flight ranging operation that is longer than the time length of an indirect time-of-flight ranging period, where the time length of an indirect time-of-flight ranging period is equal to the time length of a light sensing and a data transmission The time length is added together, and the time length of the one light sensing is greater than the time length of the data transmission. 如申請專利範圍第1項所述的多媒體系統，其中該些電子裝置各別的該飛時測距模組透過一直接飛時測距法進行該飛時測距操作，並且該些電子裝置各別執行該飛時測距操作的一個操作周期長度等於一個直接飛時測距周期的時間長度，其中該一個直接飛時測距周期的時間長度等於一個光感測的時間長度與一個資料傳輸的時間長度相加，並且該一個光感測的時間長度小於該一個資料傳輸的時間長度。 For example, in the multimedia system described in claim 1, wherein the time-of-flight ranging module of each of the electronic devices performs the time-of-flight ranging operation by a direct time-of-flight ranging method, and each of the electronic devices Do not perform the time-of-flight ranging operation. The length of an operation cycle is equal to the time length of a direct time-of-flight ranging cycle, where the time length of the direct time-of-flight ranging cycle is equal to the time length of a light sensing and a data transmission time. The time length is added together, and the time length of the one light sensing is less than the time length of the data transmission. 如申請專利範圍第1項所述的多媒體系統，其中該多媒體系統為一虛擬實境系統或一擴增實境系統。 The multimedia system described in claim 1, wherein the multimedia system is a virtual reality system or an augmented reality system. 一種應用飛時測距的多媒體系統的操作方法，包括：藉由多個電子裝置經由各別的一通信模組進行通信，以制訂一操作協議以及各別的一唯一標識符，並且進行不同電子裝置間時段(time slot)的同步；以及藉由該些電子裝置依據該操作協議以及各別的該唯一標識符來依序經由各別的一飛時測距模組執行一飛時測距操作，其中該操作協議包括該些電子裝置的多個飛時測距時段的排序，並且該些飛時測距時段彼此未重疊。 An operating method of a multimedia system using time-of-flight ranging includes: multiple electronic devices communicate via a separate communication module to formulate an operating protocol and a separate unique identifier, and perform different electronic Synchronization of time slots between devices; and by the electronic devices according to the operating protocol and the respective unique identifiers to sequentially perform a time-of-flight ranging operation via a respective time-of-flight ranging module , Wherein the operating protocol includes a sequence of a plurality of time-of-flight ranging periods of the electronic devices, and the time-of-flight ranging periods do not overlap with each other. 如申請專利範圍第5項所述的操作方法，其中該些電子裝置各別的該飛時測距模組透過一間接飛時測距法進行該飛時測距操作，並且該些電子裝置各別執行該飛時測距操作的一個操作周期長度大於一個間接飛時測距周期的時間長度，其中該一個間接飛時測距周期的時間長度等於一個光感測的時間長度與一個資 料傳輸的時間長度相加，並且該一個光感測的時間長度大於該一個資料傳輸的時間長度。 For the operation method described in item 5 of the scope of patent application, the time-of-flight ranging module of each of the electronic devices performs the time-of-flight ranging operation through an indirect time-of-flight ranging method, and each of the electronic devices Do not perform the operation period of the time-of-flight ranging operation longer than the time length of an indirect time-of-flight ranging period, where the time length of an indirect time-of-flight ranging period is equal to the time length of a light sensing and a data. The time length of the data transmission is added together, and the time length of the one light sensing is greater than the time length of the one data transmission. 如申請專利範圍第5項所述的操作方法，其中該些電子裝置各別的該飛時測距模組透過一直接飛時測距法進行該飛時測距操作，並且該些電子裝置各別執行該飛時測距操作的一個操作周期長度等於一個直接飛時測距周期的時間長度，其中該一個直接飛時測距周期的時間長度等於一個光感測的時間長度與一個資料傳輸的時間長度相加，並且該一個光感測的時間長度小於該一個資料傳輸的時間長度。 For the operation method described in item 5 of the scope of patent application, the time-of-flight ranging module of each of the electronic devices performs the time-of-flight ranging operation through a direct time-of-flight ranging method, and each of the electronic devices Do not perform the time-of-flight ranging operation. The length of an operation cycle is equal to the time length of a direct time-of-flight ranging cycle, where the time length of the direct time-of-flight ranging cycle is equal to the time length of a light sensing and a data transmission time. The time length is added together, and the time length of the one light sensing is less than the time length of the data transmission. 如申請專利範圍第5項所述的操作方法，其中該多媒體系統為一虛擬實境系統或一擴增實境系統。 The operating method described in item 5 of the scope of patent application, wherein the multimedia system is a virtual reality system or an augmented reality system.
TW109101328A 2019-05-02 2020-01-15 Multimedia system using time of flight and operating method thereof TWI723743B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/850,024 US11418274B2 (en) 2019-05-02 2020-04-16 Multimedia system applying time of flight ranging and operating method thereof

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201962842448P 2019-05-02 2019-05-02
US62/842,448 2019-05-02

Publications (2)

Publication Number Publication Date
TW202041881A TW202041881A (en) 2020-11-16
TWI723743B true TWI723743B (en) 2021-04-01

Family

ID=73154264

Family Applications (3)

Application Number Title Priority Date Filing Date
TW109101328A TWI723743B (en) 2019-05-02 2020-01-15 Multimedia system using time of flight and operating method thereof
TW109101327A TWI732424B (en) 2019-05-02 2020-01-15 Image sensor with distance sensing function and operating method thereof
TW109103470A TWI745852B (en) 2019-05-02 2020-02-05 Time of flight module and operating method thereof and multimedia system

Family Applications After (2)

Application Number Title Priority Date Filing Date
TW109101327A TWI732424B (en) 2019-05-02 2020-01-15 Image sensor with distance sensing function and operating method thereof
TW109103470A TWI745852B (en) 2019-05-02 2020-02-05 Time of flight module and operating method thereof and multimedia system

Country Status (2)

Country Link
CN (3) CN111965655A (en)
TW (3) TWI723743B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11637977B2 (en) * 2020-07-15 2023-04-25 Corephotonics Ltd. Image sensors and sensing methods to obtain time-of-flight and phase detection information

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103026691A (en) * 2010-06-16 2013-04-03 高通股份有限公司 RF ranging-assisted local motion sensing
TWI535318B (en) * 2013-02-19 2016-05-21 英特爾Ｉｐ公司 Improved wireless network location techniques
CN107912061A (en) * 2015-03-06 2018-04-13 伊利诺斯工具制品有限公司 Sensor for welding aids in head-mounted display

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4200328B2 (en) * 2005-04-18 2008-12-24 パナソニック電工株式会社 Spatial information detection system
US7944548B2 (en) * 2006-03-07 2011-05-17 Leica Geosystems Ag Increasing measurement rate in time of flight measurement apparatuses
US8542348B2 (en) * 2010-11-03 2013-09-24 Rockwell Automation Technologies, Inc. Color sensor insensitive to distance variations
EP2703836B1 (en) * 2012-08-30 2015-06-24 Softkinetic Sensors N.V. TOF illuminating system and TOF camera and method for operating, with control means for driving electronic devices located in the scene
EP2992357A4 (en) * 2013-04-29 2017-01-18 Nokia Technologies OY A method and apparatus for fusing distance data from a distance sensing camera with an image
US10291329B2 (en) * 2013-12-20 2019-05-14 Infineon Technologies Ag Exchanging information between time-of-flight ranging devices
JP2016012904A (en) * 2014-06-02 2016-01-21 ソニー株式会社 Imaging element, imaging method, and electronic equipment
EP2955544B1 (en) * 2014-06-11 2020-06-17 Sony Depthsensing Solutions N.V. A TOF camera system and a method for measuring a distance with the system
US9502457B2 (en) * 2015-01-29 2016-11-22 Semiconductor Components Industries, Llc Global shutter image sensor pixels having centralized charge storage regions
ES2879277T3 (en) * 2015-03-06 2021-11-22 Gatekeeper Systems Inc Low-power location of moving objects
US9749556B2 (en) * 2015-03-24 2017-08-29 Semiconductor Components Industries, Llc Imaging systems having image sensor pixel arrays with phase detection capabilities
JPWO2016151982A1 (en) * 2015-03-26 2018-01-25 パナソニックＩｐマネジメント株式会社 Solid-state imaging device and imaging apparatus equipped with the same
WO2017089596A1 (en) * 2015-11-26 2017-06-01 Odos Imaging Ltd. Imaging system, distance measuring device, method for operating the imaging system and the distance measuring device
US10075640B2 (en) * 2015-12-31 2018-09-11 Sony Corporation Motion compensation for image sensor with a block based analog-to-digital converter
CN105866783A (en) * 2016-04-15 2016-08-17 中国科学院上海技术物理研究所 Laser ranging method and laser ranging device utilizing semi-continuous square wave modulation and flight time measurement
CN106911381A (en) * 2017-01-20 2017-06-30 西南电子技术研究所（中国电子科技集团公司第十研究所） Laser link communication measurement hybrid system
CN107167792A (en) * 2017-06-09 2017-09-15 中国电子科技集团公司第三十四研究所 A kind of Type of Atmospheric Laser Communication and its distance-finding method with distance measurement function
GB201713092D0 (en) * 2017-08-15 2017-09-27 Purelifi Ltd Optical wireless communication device
CN108415025A (en) * 2018-05-10 2018-08-17 成都九壹通智能科技股份有限公司 The system for carrying out two-way flight time ranging and communication based on wireless pulses radiofrequency signal
CN108646214A (en) * 2018-05-10 2018-10-12 成都九壹通智能科技股份有限公司 Range-measurement system based on customed ranging protocol
CN109032179A (en) * 2018-08-08 2018-12-18 西安电子科技大学 Unmanned plane flying control equipment based on laser intensity detection

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103026691A (en) * 2010-06-16 2013-04-03 高通股份有限公司 RF ranging-assisted local motion sensing
TWI535318B (en) * 2013-02-19 2016-05-21 英特爾Ｉｐ公司 Improved wireless network location techniques
CN107912061A (en) * 2015-03-06 2018-04-13 伊利诺斯工具制品有限公司 Sensor for welding aids in head-mounted display

Also Published As

Publication number Publication date
CN111885322A (en) 2020-11-03
TWI732424B (en) 2021-07-01
TW202041881A (en) 2020-11-16
TW202101968A (en) 2021-01-01
TW202041884A (en) 2020-11-16
CN111965655A (en) 2020-11-20
CN111885322B (en) 2022-11-22
TWI745852B (en) 2021-11-11
CN111965656A (en) 2020-11-20

Similar Documents

Publication Publication Date Title
US20120072626A1 (en) Automatic Addressing Protocol for a Shared Bus
CN110430100A (en) Network connectivty detection method and device
TWI723743B (en) Multimedia system using time of flight and operating method thereof
US20180041375A1 (en) Communication device, communication method, and computer readable medium
JP7382298B2 (en) Methods, systems, devices and vehicle controllers for parking processing
CN105308570A (en) Method and apparatus for data transfer to the cyclic tasks in a distributed real-time system at the correct time
US9928184B2 (en) Microcomputer
US10530562B2 (en) Correlating local time counts of first and second integrated circuits
US20150078399A1 (en) Method and Apparatus for Consistent Modification of the Schedules in a Time-Controlled Switch
CN111611767A (en) Verification method and device
US11418274B2 (en) Multimedia system applying time of flight ranging and operating method thereof
CN116306408B (en) Verification environment determination method, device, equipment and storage medium of System On Chip (SOC)
WO2020150068A1 (en) Multichip timing synchronization circuits and methods
TWI543040B (en) Compensation time computing method and device for clock difference
CN108241141B (en) Positioning system and method
JP4080397B2 (en) Parallel computer
CN106411564B (en) For detecting the device and method of ethernet frame
US9176554B2 (en) Power management in a bridge based on a second derivative value derived from a mathematical derivative of plurality of values corresponding to the received packets
US10216670B2 (en) Synchronization of a network of sensors
TWI480737B (en) Device expanding system and device expanding method thereof
CN109510682B (en) Method, device, terminal and storage medium for synchronizing BMC (baseboard management controller) clocks of pooling server system
US20130107878A1 (en) Method for node communication
CN113721703B (en) Clock synchronization control device, system and control method in multi-path CPU system
JP5035416B2 (en) Information processing apparatus, information processing apparatus control method, and semiconductor device
US20180136955A1 (en) Simulation apparatus, simulation method, and computer readable medium