WO2025218474A1 - 用于设备控制的方法、装置、设备、存储介质和程序产品 - Google Patents

用于设备控制的方法、装置、设备、存储介质和程序产品

Info

Publication number
WO2025218474A1
WO2025218474A1 PCT/CN2025/085492 CN2025085492W WO2025218474A1 WO 2025218474 A1 WO2025218474 A1 WO 2025218474A1 CN 2025085492 W CN2025085492 W CN 2025085492W WO 2025218474 A1 WO2025218474 A1 WO 2025218474A1
Authority
WO
WIPO (PCT)
Prior art keywords
locators
group
locator
timing
timings
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2025/085492
Other languages
English (en)
French (fr)
Inventor
丁双春
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025218474A1 publication Critical patent/WO2025218474A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/141—Control of illumination
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes

Definitions

  • Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, apparatuses, devices, computer-readable storage media, and computer program products for device control.
  • Extended Reality has been widely researched and applied.
  • XR integrates virtual content and real scenes by leveraging a combination of hardware devices and various technical means, providing users with a unique sensory experience.
  • XR includes, for example, Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR).
  • VR uses computers to simulate a virtual world in three-dimensional space, providing users with an immersive experience in vision, hearing, touch, and other aspects.
  • AR allows real environments and virtual objects to be superimposed in real time on the same space and exist simultaneously.
  • MR is a new visualization environment that merges the real and virtual worlds, where objects in physical real-world scenes coexist with objects in the virtual world in real time.
  • the virtual environment can be used to provide users with an immersive interactive experience by having them wear XR devices.
  • a method for controlling a device includes: controlling a light source at each of a first group of locators to illuminate according to a first set of timings; controlling a light source at each of a second group of locators to illuminate according to a second set of timings, wherein the first set of timings and the second set of timings are configured to synchronize the illumination timing of the light source at at least one of the first group of locators with the illumination timing of the light sources at some of the second group of locators; and controlling an image collector to perform image acquisition according to at least a third timing, wherein the third timing is configured to enable the image collector to perform image acquisition when at least one of the first group of locators and some of the second group of locators are illuminated synchronously.
  • a device for controlling a device includes: a first lighting control module configured to control a light source at each locator in a first group of locators to be lit according to a first set of timings; a second lighting control module configured to control a light source at each locator in a second group of locators to be lit according to a second set of timings, wherein the first set of timings and the second set of timings are configured to synchronize the lighting time of the light sources at some of the locators in the first group with the lighting time of the light sources at at least one of the locators in the second group; and an image acquisition module configured to control an image acquisition device to perform image acquisition according to at least a third timing, wherein the third timing is configured to enable the image acquisition device to perform image acquisition when at least one of the locators in the first group of locators and some of the locators in the second group of locators are synchronously illuminated.
  • an electronic device in a third aspect of the present disclosure, includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.
  • a computer-readable storage medium wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.
  • a computer program product is provided, which is tangibly stored in a computer storage medium and includes computer-executable instructions, which, when executed by a device, cause the device to perform the method of the first aspect.
  • FIG1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented
  • FIG2 shows a flow chart of a process of device control according to some embodiments of the present disclosure
  • FIG. 3 is a schematic diagram illustrating an example of a first timing sequence of a first group of locators, a second timing sequence of a second group of locators, and a third timing sequence of an image collector according to some embodiments of the present disclosure
  • FIG4 shows a flowchart of a process of device control according to some embodiments of the present disclosure
  • FIG5 is a schematic diagram showing an image set collected by an image collector according to some embodiments of the present disclosure.
  • FIG6 shows a block diagram of an apparatus for device control according to some embodiments of the present disclosure.
  • a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.
  • the prompt information in response to receiving a user's active request, may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form.
  • the pop-up window may also contain a selection control for the user to select "agree” or “disagree” to provide personal information to the electronic device.
  • FIG1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented.
  • the environment 100 can be a physical scene, that is, the environment 100 can be an example of an actual scene in the real world.
  • the user 110 can implement interactive operations by wearing at least one XR device.
  • the at least one XR device may include a head-mounted XR device 120, an XR manipulator 122-1 and an XR manipulator 122-2 held in the left and right hands respectively (which may be individually or collectively referred to as XR manipulators 122), and a plurality of locators 124-1 and locators 124-2 that can be used to be worn on other parts of the body, such as the lower limbs.
  • the locator may also be referred to as a tracker.
  • the XR manipulator 122-1 may include a locator 123-1
  • the XR manipulator 122-2 may include a locator 123-2
  • the locator 123-1 and the locator 123-2 may belong to a first group of locators (each of which is referred to as a first locator 123).
  • locator 124-1 and locator 124-2 may belong to a second group of locators (each of which is referred to as a second locator 124).
  • the handheld XR manipulator 122 may be, for example, any suitable type of device controlled by hand, such as a handle.
  • the second locator 124 may be any suitable type of tracker for wearing on multiple parts of the body (such as the waist, arm, etc.).
  • the head-mounted XR device 120 may communicate with the XR manipulator 122 and each second locator 124 to achieve interaction with the user in a collaborative manner.
  • the XR device may include a device for capturing images, such as an image collector such as a camera or an infrared camera, to capture images of the physical space.
  • the image collectors may be respectively provided on one or more XR devices.
  • the image collector may be provided on the head-mounted XR device 120 to capture images within a wider spatial range based on a higher spatial position.
  • an infrared camera is provided on the head-mounted XR device 120, correspondingly, at least one infrared light source may be provided on the XR manipulator 122 and the second locator 124, respectively, so that the infrared camera can capture the infrared light source on the XR manipulator 122 and the second locator 124, thereby identifying and locating the XR manipulator 122 and the second locator 124 based on the obtained image.
  • the image collector used for image acquisition can be independent of the XR device (e.g., independent of the head-mounted XR device 120). Such an image collector can be independently deployed to perform image perception of the first locator 123, the second locator 124, and other devices and environments in the environment.
  • the user 110 can use multiple XR devices to communicate with the electronic device 130, so as to use the electronic device 130 to process the data obtained by the XR device and send data to the XR device.
  • the XR device 120 may not communicate with the electronic device 130 (or the environment 100 may not include the electronic device 130 for example).
  • the head-mounted XR device 120 can be used to interact with other XR devices such as the XR manipulator 122-1 (and the first locator therein), the XR manipulator 122-2 (and the first locator therein), the second locator 124-1, and the second locator 124-2.
  • the XR control device 121 in the head-mounted XR device 120 can be used to interact with other XR devices. It should be understood that the XR control device 121 is not limited to being set in the head-mounted XR device 120. It can be set in other XR devices or exist as an independent control device.
  • user 110 can use an XR device to access a virtual environment 140.
  • Virtual environment 140 can, for example, be a space.
  • virtual objects 150 of user 110 can be generated in virtual environment 140, allowing user 110 to interact immersively with virtual objects 150 within virtual environment 140, such as adjusting the movements of virtual objects 150.
  • virtual tools and/or virtual objects of other users can be provided within virtual environment 140, allowing user 110 to interact with virtual tools within virtual environment 140 through virtual objects 150, as well as interact with virtual objects of other users through virtual objects 150.
  • the electronic device 130 can be a separate device capable of communicating with the XR device and/or other image capture devices, such as a server or computing node for image or data processing, or can be integrated with the XR device and/or other image capture devices.
  • the electronic device 130 can be implemented as an XR device, that is, in this case, the XR device can implement all the functions of the electronic device 130. It should be understood that the above description of the electronic device 130 is merely exemplary and non-restrictive.
  • the electronic device 130 can be implemented as a variety of forms, structures, or categories of devices, and the embodiments of the present disclosure are not limited in this regard.
  • positioning tracking of trackers i.e. locators
  • handheld manipulators such as handles
  • specific light sources such as optical infrared lights
  • the camera shoots the infrared light of the handle or other tracker and combines it with the IMU (Inertial Measurement Unit) posture for tracking.
  • IMU Inertial Measurement Unit
  • the embodiments of the present disclosure propose a scheme for device control.
  • the light sources at each locator in the first group of locators are controlled to be illuminated according to a first set of timings.
  • the light sources at each locator in the second group of locators are controlled to be illuminated according to a second set of timings, and the first set of timings and the second set of timings are configured to synchronize the lighting time of the light sources at at least one locator in the first group of locators with the lighting time of the light sources at some locators in the second group of locators.
  • the image collector is controlled to perform image acquisition according to at least a third timing
  • the third timing is configured to enable the image collector to perform image acquisition when at least one locator in the first group of locators and some locators in the second group of locators are synchronously illuminated.
  • the scheme can realize the time-sharing illumination of the locators in the second group of locators, while being compatible with the illumination of the first group of locators, thereby achieving the distinction between different locators, avoiding positioning confusion when subsequent detection is performed based on the captured images, and thereby improving the efficiency and accuracy of positioning tracking.
  • FIG2 shows a flowchart of a process 200 for device control according to some embodiments of the present disclosure.
  • these embodiments will be described with reference to the environment 100 of FIG1 .
  • these embodiments may be implemented in the XR control device 121 of FIG1 .
  • these embodiments may also be implemented in a remote controller, which may be provided in the electronic device 130 of FIG1 , for example.
  • the following specific embodiments are implemented in the XR control device 121 as an example, but this is merely exemplary and not intended to be limiting.
  • the XR control device 121 may control the light source at each first locator 123 in the first group of locators to illuminate according to the first set of timing sequences.
  • the light source includes, but is not limited to, an infrared light source.
  • the image collector may identify and capture the light source, and identify and locate the first locator 123 and the locator 124 based on the captured image.
  • each first locator 123 is deployed at the XR manipulator 122 . In such an embodiment, the XR manipulator 122 can be identified and located by each first locator 123 .
  • the first group of locators includes multiple first locators 123, and the multiple first timings corresponding to the multiple first locators 123 are configured to cause the light sources at the multiple first locators 123 to be illuminated synchronously.
  • At least one first locator 123 can be deployed at each XR manipulator 122. In such an embodiment, each time the image collector captures the light sources at the multiple first locators 123, it can directly distinguish the corresponding XR manipulator 122 based on the captured image. Therefore, the light sources at the multiple first locators 123 can be controlled to illuminate synchronously.
  • the first group of locators may also include only a single first locator, and in some examples the locator may be included in any one of the XR manipulators 122.
  • the light source patterns of the multiple first locators 123 are different. In such embodiments, when the image collector captures the light sources of the multiple first locators 123, the corresponding light source patterns (e.g., light spots) on the image are different, thereby enabling the manipulators corresponding to the multiple first locators 123 to be quickly distinguished.
  • the light source in the locator may include a single light source having a corresponding light spot.
  • the light source in the locator may include multiple sub-light sources, which may have a certain positional relationship, so that corresponding light source spots can be formed by controlling these sub-light sources.
  • the XR control device 121 may control the light sources at each of the second locators 124 in the second group of locators to illuminate according to a second set of timings, where the first set of timings and the second set of timings are configured to synchronize the illumination timing of the light sources at at least one of the first locators 123 in the first group of locators with the illumination timing of the light sources at some of the second locators 124 in the second group of locators.
  • the light source pattern of the first group of locators differs from the light source pattern of the second group of locators, for example, the light source pattern of any first locator 123 differs from the light source pattern of any second locator 124.
  • the image collector can simultaneously capture images of the first group of locators and the second group of locators.
  • the image collector in order to achieve better recognition of the posture and movement of the user 110, can simultaneously capture images of the first group of locators and the second group of locators. Based on this, the light source pattern of the first group of locators can be made different from the light source pattern of the second group of locators. In this way, when the image collector simultaneously captures the images of the first locator 123 and part of the second locator 124, the two can be easily distinguished.
  • the positioning and tracking effect can be improved.
  • the second group of positioners includes a plurality of second positioners 124, and the plurality of second timing sequences corresponding to the plurality of second positioners 124 are configured to sequentially illuminate the light sources at the plurality of second positioners 124.
  • the light source patterns of the plurality of second positioners 124 are the same.
  • the appearance and internal configuration of the multiple second locators 124 can be the same. Accordingly, the light source patterns of the multiple second locators 124 can be the same. It should be understood that the multiple second locators 124 can be the same type of locators. Therefore, in such an embodiment, by setting multiple second timings so that the light sources at the corresponding multiple second locators 124 are illuminated in sequence, the time-sharing lighting of the multiple second locators 124 can be achieved. This enables the image collector to light up only one second locator 124 when capturing an image, thereby avoiding the abnormal situation where the simultaneous lighting of multiple second locators 124 leads to recognition confusion.
  • the XR control device 121 can control the image collector to perform image acquisition according to at least a third timing, and the third timing is configured to enable the image collector to perform image acquisition when at least one first locator 123 (or all first locators 123) in the first group of locators and some second locators 124 in the second group of locators are synchronously illuminated.
  • the XR manipulator 122 where the first locator 123 is located and the second locator 124 can be used simultaneously, and tracking of at least one XR manipulator 122 can be achieved while tracking and positioning of multiple second locators 124 are also achieved, and the tracking and positioning of the XR manipulator 122 and the second locator 124 do not interfere with each other.
  • FIG3 illustrates a schematic diagram of an example of a first timing sequence for a first group of positioners, a second timing sequence for a second group of positioners, and a third timing sequence for an image collector, according to some embodiments of the present disclosure.
  • Example 300 may be implemented in environment 100 of FIG1 .
  • Example 300 may be applied to some steps of process 200 .
  • example 300 includes a third timing sequence 310 of the image collector, a first timing sequence 320-1 of the first locator 123-1 of the XR manipulator 122-1, and a first timing sequence 320-2 of the first locator 123-2 of the XR manipulator 122-2 (which may be individually or collectively referred to as the first timing sequence 320), a second timing sequence 330-1 of the second locator 124-1, and a second timing sequence 330-2 of the second locator 124-2 (which may be individually or collectively referred to as the second timing sequence 330).
  • the third timing sequence 310 includes multiple high pulses 301, which can represent exposure of the head-mounted XR device 120 and the XR manipulator 122.
  • the first timing sequence 320 and the second timing sequence 330 both include multiple high pulses 302, each of which indicates that the corresponding light source is illuminated.
  • the light sources at the XR manipulator 122-1 and the XR manipulator 122-2 are synchronously illuminated.
  • the illumination times of the light sources at the second locator 124-1 and the second locator 124-2 are different, and the illumination times of the light sources at each second locator and the respective XR manipulator 122 are synchronized.
  • controlling the "synchronous illumination” and “illumination time synchronization" of different locators may include controlling these locators to illuminate at the same time, or allowing these locators to have a common illumination state within a specific time period, or allowing these locators to be illuminated so that the image acquisition device captures these locators during one exposure period of performing image acquisition, etc.
  • the light sources at the second locator 124-1 and the second locator 124-2 are sequentially illuminated (i.e., illuminated in a time-sharing manner).
  • the image collector Based on the third timing sequence 310, the image collector performs image acquisition when the XR manipulator 122-1 and the XR manipulator 122-2, as well as one of the second locators 124-1 and the second locator 124-2, are illuminated.
  • the periods of the time sequences in the first set of time sequences are the same, and the periods of the time sequences in the second set of time sequences are the same. In some embodiments, the periods corresponding to the first set of time sequences are different from the periods corresponding to the second set of time sequences.
  • the first group of timings includes the first timing 320-1 and the first timing 320-2
  • the second group of timings includes the second timing 330-1 and the second timing 330-2.
  • the periods of the first timing 320-1 and the first timing 320-2 can be the same, and the periods of the second timing 330-1 and the second timing 330-2 can also be the same. However, the periods of the first timing 320 and the second timing 330 can be different.
  • the XR manipulator 122-1 and the XR manipulator 122-2 can be distinguished, so the periods of the first timing 320-1 and the first timing 320-2 can be the same, and the first timing 320-1 and the first timing 320-2 can be the same.
  • the light sources at the plurality of second positioners 124 can be lit sequentially, that is, the second timing 330-1 and the second timing 330-2 are staggered, the periods of the second timing 330-1 and the second timing 330-2 can be the same, while the periods of the first timing 320 and the second timing 330 can be different.
  • the second locator 124 is illuminated in a time-sharing manner, which can improve the tracking effect of the second locator 124.
  • the time-sharing illumination can reduce the power consumption of each second locator 124, thereby extending the service life of the product.
  • the XR control device 121 may determine the timing corresponding to each second locator 124 in the second group of locators based on the attribute information of the second group of locators.
  • the timing corresponding to each second locator 124 in the second group of locators is based on the order in which each second locator 124 in the second group of locators is illuminated and/or the cycle in which the second group of locators are sequentially polled and illuminated.
  • the attribute information of the second group of locators includes at least the number of second locators 124 in the second group of locators.
  • the attribute information of at least one second locator 124 may also include a unique identification number for each second locator 124.
  • the XR control device 121 can determine the order in which each second locator 124 is illuminated based on the number of locators and their identification numbers. Furthermore, the XR control device 121 can determine the cycle in which each second locator 124 is sequentially illuminated based on the number of locators in the second group. Then, based on this cycle and the order in which each second locator 124 is illuminated, the timing of each second locator 124 can be determined. This facilitates the time-sharing illumination of each second locator 124.
  • the period corresponding to the second set of timing sequences is an integer multiple of the period corresponding to the first set of timing sequences, and the integer multiple is determined based on the number of second locators 124 in the second set of locators. For example, if the number of locators in the second set of locators is 2, then the period of the corresponding second set of timing sequences is twice the period of the first set of timing sequences. Referring to FIG3 , the period of the second timing sequence 330 can be twice that of the first timing sequence 320. This allows the light sources of each second locator 124 in the second set of locators to be illuminated sequentially.
  • the XR control device 121 may, for a first image acquisition cycle, assign each timing in the second set of timings to each second locator 124 in the second group of locators, and receive a response regarding the configuration of each timing in the second set of timings from each second locator 124 in the second group of locators.
  • the first image acquisition cycle may correspond to a complete cycle in which the second group of locators completes a round of time-sharing illumination, and there may be multiple acquisition points in this image acquisition cycle.
  • the XR control device 121 when controlling the image collector to perform image acquisition according to at least the third timing sequence, if a response is received from each second locator 124 in the second group of locators regarding the configuration of each timing sequence in the second group of timing sequences within a predetermined time, the XR control device 121 may control the image collector to perform image acquisition for at least the first image acquisition cycle according to the third timing sequence. Furthermore, if a response is not received from one or more second locators 124 in the second group of locators within the predetermined time, the XR control device 121 may determine that the configuration of the second group of timing sequences in the first image acquisition cycle has failed.
  • the XR control device 121 can configure the corresponding timing for the next second locator 124 of the current second locator, so that the light source of the next second locator 124 performs time-sharing lighting according to the corresponding timing.
  • the successful configuration of the second locator 124 is determined by the second locator 124's response to its timing configuration and the time it takes to do so. If the second locator 124 is successfully configured, image acquisition can proceed during the current image acquisition cycle. If the configuration fails, this indicates a timing synchronization failure during the current image acquisition cycle, requiring reconfiguration of the second timing during the next image acquisition cycle. This ensures the accuracy of the time-sharing activation of the second locator 124, improving tracking efficiency for the second locator 124.
  • the XR control device 121 may perform image analysis on the first image captured by the image collector to determine the position information and/or posture information corresponding to the portion of the second locators 124 that were illuminated in the second group of locators at the time of capture of the first image. In some embodiments, based on the second set of timings, the XR control device 121 may determine the portion of the second locators 124 that were illuminated in the second group of locators at the time of capture of the first image, and write the sequence numbers of the determined portion of the second locators 124 into the frame information of the first image. Furthermore, when determining the position information and/or posture information corresponding to the portion of the locators, the XR control device 121 may determine that the frame information of the first image contains the position information and/or posture information corresponding to the portion of the locators.
  • the XR control device 121 can write the sequence number of the next second locator 124 into the image collector's configuration information before controlling the image collector to capture an image of the next second locator 124. After the XR control device 121 writes the sequence number of the next second locator 124 into the image collector's configuration information, the frame information of the image of the next second locator 124 captured by the image collector will include the corresponding sequence number.
  • the upper layer when it acquires this frame of image, it will determine from the image information that the current image is illuminated by the next second locator 124, and that the current image also includes an image of the light source of the first locator 123 of at least one XR manipulator 122, thereby performing image analysis.
  • This allows for convenient identification of the specific illuminated second locator 124 through the frame information, facilitating location tracking and corresponding acquisition of posture information of the user's 110 body part.
  • the XR control device 121 may receive the timing used by each second locator 124 in the second group of locators, and determine whether the timing received from each second locator 124 in the second group of locators is the same as the second group of timings. If it is determined that at least one timing received from at least one second locator 124 in the second group of locators is different from at least one timing in the second group of timings, each timing in the second group of timings may be reassigned to each second locator 124 in the second group of locators.
  • the next second locator 124 that needs to be lit can immediately report its own serial number to the XR control device 121.
  • the XR control device 121 will detect whether the configuration information of the serial number to be written corresponds one-to-one with the serial number of the reported second locator 124. If an abnormality occurs, it will be considered that the timing of the next second locator 124 that needs to be lit is disordered. Then, the timing of the second locator 124 can be reconfigured in the next image acquisition cycle to ensure the stability of the timing. Therefore, through this timing detection mechanism, it is possible to detect in real time whether the timing of the second locator 124 is abnormal to ensure that the time-sharing lighting of each second locator 124 can proceed normally.
  • the XR control device 121 can detect the connection status of each second locator 124 in the second group of locators to determine whether the number of second locators 124 in the second group of locators has changed. If it is determined that the number of second locators in the second group of locators has changed, the second group of timing sequences corresponding to the changed second group of locators is determined based on the changed number. In some embodiments, the XR control device 121 determines that the first and second groups of locators are connected to the XR control device 121 before controlling the first and second groups of locators to light up.
  • the timing of the corresponding second locators 124 will be determined based on the changed number.
  • the XR control device 121 is deployed at the head-mounted XR device 120.
  • the head-mounted XR device 120 can detect the connection status between it and each second locator 124 to determine whether the number of second locators 124 connected to the head-mounted XR device has changed. If so, the timing corresponding to each second locator 124 can be determined based on the changed number of second locators 124.
  • the flexibility and adaptability of tracking and positioning of multiple second locators 124 can be improved.
  • the second group of locators can be illuminated in a time-sharing manner while being compatible with the illumination of the first group of locators, thereby avoiding positioning confusion during subsequent detection based on the captured images, thereby improving the efficiency and accuracy of positioning tracking in XR scenarios.
  • FIG4 shows a flow chart of a process for device control according to some embodiments of the present disclosure.
  • Process 400 may be implemented in environment 100 of FIG1 .
  • Process 400 may be shown as a specific embodiment of the steps of process 200.
  • Process 400 may be implemented according to the timing of example 300 of FIG3 .
  • process 400 involves relevant processes of the head-mounted XR device 120, the second locator 124 (including the second locator 124-1 and the second locator 124-2), and the XR manipulator 122 (including the XR manipulator 122-1 and the XR manipulator 122-2).
  • Each second locator 124 performs similar operations
  • each XR manipulator 122 performs similar operations.
  • Process 400 can be described in detail by taking the XR control device 121 and the image collector as an example, which are deployed at the head-mounted XR device 120.
  • the XR manipulator 122 needs to be connected to the head-mounted XR device 120 (particularly, the XR control device 121 therein), and the second locator 124 needs to be connected to the head-mounted XR device 120 (particularly, the XR control device 121 therein).
  • the head-mounted XR device 120 can detect whether it is connected to the second locator 124. If it is determined that it is connected to the second locator 124 (410), the head-mounted XR device 120 sends a respective second timing sequence to each second locator 124 (411).
  • the XR manipulator 122 After being connected to the head-mounted XR device 120 (430), the XR manipulator 122 lights up the infrared light source of its own first locator 123 according to a predetermined first timing (431). That is, in each cycle, the head-mounted device 120 does not need to send the first timing to the XR manipulator 122 every time.
  • the first timing can be configured or defined at the XR manipulator 122.
  • the XR manipulator 122 After being connected to the head-mounted device 120, the XR manipulator 122 can perform lighting according to the predetermined first timing. In addition, since the lighting time of multiple XR manipulators 122 can be the same, the first timing configured at each XR manipulator 122 can also be the same.
  • each second locator 124 After connecting to the head-mounted XR device 120 (420), each second locator 124 receives the second timing and configures it (421), and then replies to the head-mounted XR device 120 that the configuration is complete (422). After receiving the reply (412), the head-mounted XR device 120 determines whether the second locator 124 has completed the configuration within the preset time (413). If the second locator 124 has not completed the configuration within the preset time, the second timing is re-sent to the second locator 124 in the next cycle (411). If the second locator 124 completes the configuration within the preset time, the image collector is configured according to the corresponding second timing (414). Specifically, the serial number of the second locator 124 can be written into the image collector.
  • the second locator 124 will also light up its own infrared light source according to the second timing, realizing time-sharing lighting of the infrared light source (423). Then, the head-mounted XR device 120 can obtain the serial number of the lit locator according to the frame information of the image captured by the image collector (415) and analyze the image.
  • the head-mounted XR device 120 can determine in real time whether the number of the second locators 124 has changed (416). If it has changed, the second timing is updated again and the second timing is issued (411).
  • the second locator 124 can report the current second timing regularly, and then the head-mounted XR device 120 can determine whether the reported second timing is correct when the number of the second locators 124 has not changed (417). If it is correct, the image collector can continue to be configured according to the second timing (414). If it is not correct, it will be considered that the second timing is disordered, and the second timing will be issued again in the next cycle (411) to ensure the stability of the second timing.
  • the head-mounted XR device 120 writes the sequence number of the second locator 124-1 into the image acquisition device configuration.
  • the infrared light source of the second locator 124-1 is illuminated, while the infrared light source of the second locator 124-2 is turned off.
  • the upper layer acquires this frame of image, it will obtain through image information that the second locator 124-1 is lit in the current image, and obtain images of the synchronously lit second locator 124-1 and the locator 123 of the XR manipulator 122 for analysis.
  • the head-mounted XR device 120 writes the serial number of the second locator 124-2 into the image acquisition device configuration.
  • the second locator 124-2 is illuminated, and the second locator 124-1 is turned off.
  • the upper layer acquires this frame of image, it will use the image information to obtain that the second locator 124-2 is currently illuminated, and it will also obtain and analyze the image of the infrared light source that synchronously illuminates the second locator 124-2 and the left and right XR manipulators 122. In this reciprocating process, the tracking effect of the two XR manipulators 122 and the two second locators 124 can be achieved.
  • FIG5 illustrates a schematic diagram of an image set 500 captured by an image collector according to some embodiments of the present disclosure.
  • Image set 500 may be implemented in environment 100 of FIG1 .
  • Image set 500 may be illustrated as an example of the result of executing process 200.
  • Image set 500 may be illustrated as an example of the result of executing the method implemented using example 300 of FIG3 .
  • Image set 500 may be illustrated as an example of the result of executing process 400.
  • image set 500 includes image 510 and image 520.
  • Image 510 is captured by the image collector when the infrared light source of second locator 124-1 is on.
  • the serial number of second locator 124-1 is recorded in the frame information of image 510.
  • Image 520 is captured by the image collector when the infrared light source of second locator 124-2 is on.
  • the serial number of second locator 124-2 is recorded in the frame information of image 520.
  • the two XR manipulators 122-1 and 122-2 are also illuminated. Accordingly, the three light spots in image 510 represent the exposure of the infrared light sources of the second locator 124-1, the left-hand XR manipulator 122-1, and the right-hand XR manipulator 122-2. Since the infrared light source of the second locator 124-2 is illuminated at the same time, the two XR manipulators 122-1 and 122-2 are also illuminated.
  • the three light spots in image 520 represent the exposure of the infrared light sources of the second locator 124-2, the left-hand XR manipulator 122-1, and the right-hand XR manipulator 122-2.
  • the light spots are arranged in a certain space. Therefore, the process for device control of the disclosed embodiment realizes time-sharing lighting control of the locator, and realizes tracking and positioning of the locator.
  • FIG. 6 shows a schematic structural block diagram of an apparatus 600 for device control according to certain embodiments of the present disclosure.
  • Apparatus 600 may be implemented as or included in an XR control device 121.
  • Each module/component in apparatus 600 may be implemented by hardware, software, firmware, or any combination thereof.
  • the device 600 includes a first lighting control module 610, which is configured to control the light sources at each locator in the first group of locators to be lit according to a first set of timings; a second lighting control module 620, which is configured to control the light sources at each locator in the second group of locators to be lit according to a second set of timings, the first set of timings and the second set of timings being configured to synchronize the lighting time of the light sources at at least one locator in the first group of locators with the lighting time of the light sources at some locators in the second group of locators; and an image acquisition module 630, which is configured to control the image acquirer to perform image acquisition at least according to a third timing, the third timing being configured to enable the image acquirer to perform image acquisition when at least one locator in the first group of locators and some locators in the second group of locators are synchronously illuminated.
  • a first lighting control module 610 which is configured to control the light sources at each locator in the first group of locators to be lit according
  • the period corresponding to the first set of timing sequences is different from the period corresponding to the second set of timing sequences.
  • the periods of the respective timing sequences in the first group of timing sequences are the same, and the periods of the respective timing sequences in the second group of timing sequences are the same.
  • the light source pattern of the first group of positioners is different than the light source pattern of the second group of positioners.
  • the second group of locators includes a plurality of second locators, and the plurality of second timings corresponding to the plurality of second locators are configured to cause the light sources at the plurality of second locators to be illuminated in sequence.
  • the light source patterns of the plurality of second positioners are the same.
  • the first group of locators includes a plurality of first locators, and the plurality of first timings corresponding to the plurality of first locators are configured to cause the infrared light sources at the plurality of first locators to be illuminated synchronously.
  • the light source patterns of the plurality of first positioners are different.
  • the device 600 also includes a timing determination module, which is configured to determine the timing corresponding to each locator in the second group of locators based on the attribute information of the second group of locators in response to detecting that the connection of the second group of locators is completed, and the timing corresponding to each locator in the second group of locators is based on the order in which each locator in the second group of locators is illuminated and/or the cycle in which the second group of locators are polled and illuminated in sequence.
  • a timing determination module which is configured to determine the timing corresponding to each locator in the second group of locators based on the attribute information of the second group of locators in response to detecting that the connection of the second group of locators is completed, and the timing corresponding to each locator in the second group of locators is based on the order in which each locator in the second group of locators is illuminated and/or the cycle in which the second group of locators are polled and illuminated in sequence.
  • the attribute information of the second set of locators includes at least the number of locators in the second set of locators.
  • the period corresponding to the second set of timings is an integer multiple of the period corresponding to the first set of timings, and wherein the integer multiple is determined based on the number of locators in the second set of locators.
  • the second lighting control module 620 is further configured to configure each timing in the second group of timings to each locator in the second group of locators for the first image acquisition cycle; and receive a reply for the configuration of each timing in the second group of timings from each locator in the second group of locators.
  • the image acquisition module 630 is further configured to control the image acquisition device to perform image acquisition for the first image acquisition cycle at least according to the third timing in response to receiving a reply from each locator in the second group of locators regarding the configuration of each timing in the second group of timing within a predetermined time; and the device 600 also includes a configuration failure module, which is configured to determine that the configuration of the second group of timings in the first image acquisition cycle has failed in response to not receiving a reply from one or more locators in the second group of locators within a predetermined time.
  • the device 600 also includes an image analysis module configured to perform image analysis on the first image captured by the image collector to determine the position information and/or posture information corresponding to some of the locators in the second group of locators that are lit at the time of capture of the first image.
  • an image analysis module configured to perform image analysis on the first image captured by the image collector to determine the position information and/or posture information corresponding to some of the locators in the second group of locators that are lit at the time of capture of the first image.
  • the device 600 also includes a serial number writing module, which is configured to determine the partial locators in the second group of locators that are lit at the acquisition time of the first image based on the second group of timings; and write the serial numbers of the determined partial locators in the frame information of the first image;
  • the image analysis module includes an information determination module, which is configured to determine that the frame information of the first image contains position information and/or posture information corresponding to the partial locators.
  • the apparatus 600 further includes a number determination module configured to detect a connection status of each locator in the second group of locators to determine whether the number of locators in the second group of locators has changed; and in response to determining that the number of locators in the second group of locators has changed, determine a second group of timings corresponding to the changed second group of locators based on the changed number.
  • a number determination module configured to detect a connection status of each locator in the second group of locators to determine whether the number of locators in the second group of locators has changed; and in response to determining that the number of locators in the second group of locators has changed, determine a second group of timings corresponding to the changed second group of locators based on the changed number.
  • the apparatus 600 further includes a timing reconfiguration module configured to receive a respective timing for use from each locator in the second group of locators; determine whether the timing received from each locator in the second group of locators is the same as the second group of timings; and in response to determining that at least one timing received from at least one locator in the second group of locators is not the same as at least one timing in the second group of timings, reconfigure each timing in the second group of timings to each locator in the second group of locators.
  • a timing reconfiguration module configured to receive a respective timing for use from each locator in the second group of locators; determine whether the timing received from each locator in the second group of locators is the same as the second group of timings; and in response to determining that at least one timing received from at least one locator in the second group of locators is not the same as at least one timing in the second group of timings, reconfigure each timing in the second group of timings to each locator in the second group of locators.
  • the apparatus 600 may be implemented at an extended reality (XR) control device, and the apparatus 600 further includes a connection determination module configured to determine that the first group of locators and the second group of locators are connected to the XR control device before controlling the first group of locators and the second group of locators to perform lighting.
  • XR extended reality
  • the XR control device includes a head-mounted XR device, and/or wherein the first set of positioners is deployed at the XR manipulator.
  • the units and/or modules included in the device 600 can be implemented in various ways, including software, hardware, firmware, or any combination thereof.
  • one or more units and/or modules can be implemented using software and/or firmware, such as machine executable instructions stored on a storage medium.
  • some or all of the units and/or modules in the device 500 can be implemented at least in part by one or more hardware logic components.
  • exemplary types of hardware logic components include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FIG7 shows a block diagram of an electronic device 700 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 700 shown in FIG7 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 700 shown in FIG7 can be used to implement the XR control device 121 of FIG1 .
  • electronic device 700 is in the form of a general-purpose computing device.
  • Components of electronic device 700 may include, but are not limited to, one or more processors or processing units 710, memory 720, storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760.
  • Processing unit 710 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 720. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of electronic device 700.
  • the electronic device 700 typically includes a plurality of computer storage media. Such media can be any accessible media that can be obtained by the electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media.
  • the memory 720 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory) or some combination thereof.
  • the storage device 530 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk or any other medium that can be used to store information and/or data and can be accessed within the electronic device 700.
  • the electronic device 700 may further include additional removable/non-removable, volatile/non-volatile storage media.
  • a disk drive for reading from or writing to a removable, non-volatile disk (e.g., a “floppy disk”) and an optical drive for reading from or writing to a removable, non-volatile optical disk may be provided.
  • each drive may be connected to a bus (not shown) by one or more data media interfaces.
  • the memory 720 may include a computer program product 725 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.
  • the communication unit 740 enables communication with other computing devices via a communication medium. Additionally, the functions of the components of the electronic device 700 can be implemented as a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 700 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.
  • PCs network personal computers
  • Input device 750 may be one or more input devices, such as a mouse, keyboard, or trackball.
  • Output device 760 may be one or more output devices, such as a display, a speaker, or a printer.
  • Electronic device 700 may also communicate with one or more external devices (not shown) via communication unit 740, as needed, such as storage devices, display devices, or the like, with one or more devices that allow a user to interact with electronic device 700, or with any device that allows electronic device 700 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input/output (I/O) interface (not shown).
  • I/O input/output
  • a computer-readable storage medium on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above.
  • a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.
  • These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and/or other device to operate in a specific manner.
  • the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
  • each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification.
  • the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each box in the block diagram and/or flow chart, and the combination of the boxes in the block diagram and/or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Studio Devices (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

根据本公开的实施例，提供了用于设备控制的方法、装置、设备、存储介质和程序产品。该方法包括：控制第一组定位器中各个定位器处的光源根据第一组时序执行点亮；控制第二组定位器中各个定位器处的光源根据第二组时序执行点亮，第一组时序和第二组时序被配置为使第一组定位器中至少一个定位器处的光源的点亮时间与第二组定位器中部分定位器处的光源的点亮时间同步；以及控制图像采集器至少根据第三时序执行图像采集，第三时序被配置为使图像采集器在第一组定位器中的至少一个定位器和第二组定位器中的部分定位器被同步点亮时执行图像采集。

Description

用于设备控制的方法、装置、设备、存储介质和程序产品
本申请要求2024年4月15日递交的、标题为“用于设备控制的方法、装置、设备、存储介质和程序产品”、申请号为202410454996.7的中国发明专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开的示例实施例总体涉及计算机领域，特别地涉及用于设备控制的方法、装置、设备、计算机可读存储介质和计算机程序产品。
背景技术
扩展现实(Extended Reality，简称XR)得到广泛研究和应用。XR通过利用硬件设备和多种技术手段的结合，将虚拟内容和真实场景进行融合，为用户提供独特的感官体验。XR例如包括虚拟现实(Virtual Reality，简称VR)、增强现实(Augmented Reality，简称AR)、混合现实(Mix Reality，简称MR)等。VR是利用计算机模拟三维空间的虚拟世界，为用户提供视觉、听觉、触觉等方面的沉浸式体验。AR使得真实环境和虚拟对象实时叠加到同一个空间并同时存在。MR是融合现实世界和虚拟世界的新的可视化环境，物理真实世界场景中的对象与虚拟世界中的对象实时共存。
由此，可通过使用户佩戴XR设备来利用虚拟环境为用户提供一种身临其境的交互体验。
发明内容
在本公开的第一方面，提供了一种用于设备控制的方法。该方法包括：控制第一组定位器中各个定位器处的光源根据第一组时序执行点亮；控制第二组定位器中各个定位器处的光源根据第二组时序执行点亮，第一组时序和第二组时序被配置为使第一组定位器中至少一个定位器处的光源的点亮时间与第二组定位器中部分定位器处的光源的点亮时间同步；以及控制图像采集器至少根据第三时序执行图像采集，第三时序被配置为使图像采集器在第一组定位器中的至少一个定位器和第二组定位器中的部分定位器被同步点亮时执行图像采集。
在本公开的第二方面，提供了一种用于设备控制的装置。该装置包括：第一点亮控制模块，被配置为控制第一组定位器中各个定位器处的光源根据第一组时序执行点亮；第二点亮控制模块，被配置为控制第二组定位器中各个定位器处的光源根据第二组时序执行点亮，第一组时序和第二组时序被配置为使第一组定位器中部分定位器处的光源的点亮时间与第二组定位器中至少一个定位器处的光源的点亮时间同步；以及图像采集模块，被配置为控制图像采集器至少根据第三时序执行图像采集，第三时序被配置为使图像采集器在第一组定位器中的至少一个定位器和第二组定位器中的部分定位器被同步点亮时执行图像采集。
在本公开的第三方面，提供了一种电子设备。该设备包括至少一个处理单元；以及至少一个存储器，至少一个存储器被耦合到至少一个处理单元并且存储用于由至少一个处理单元执行的指令。指令在由至少一个处理单元执行时使设备执行第一方面的方法。
在本公开的第四方面，提供了一种计算机可读存储介质。该计算机可读存储介质上存储有计算机程序，计算机程序可由处理器执行以实现第一方面的方法。
在本公开的第五方面，提供了一种计算机程序产品。该计算机程序产品被有形地存储在计算机存储介质中并且包括计算机可执行指令，计算机可执行指令在由设备执行时使设备执行第一方面的方法。
应当理解，该部分中所描述的内容并非旨在限定本公开的实施例的关键特征或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的描述而变得容易理解。
附图说明
结合附图并参考以下详细说明，本公开各实施例的上述和其他特征、优点及方面将变得更加明显。在附图中，相同或相似的附图标记表示相同或相似的元素，其中：
图1示出了本公开的实施例能够在其中实现的示例环境的示意图；
图2示出了根据本公开的一些实施例的设备控制的过程的流程图；
图3示出了根据本公开的一些实施例的第一组定位器的第一组时序、第二组定位器的第二组时序及图像采集器的第三时序的示例的示意图；
图4示出了根据本公开的一些实施例的设备控制的过程的流程图；
图5示出了根据本公开的一些实施例的图像采集器采集到的图像集的示意图；
图6示出了根据本公开的一些实施例的用于设备控制的装置的框图；以及
图7示出了能够实施本公开的一个或多个实施例的电子设备的框图。
具体实施方式
可以理解的是，在使用本公开各实施例公开的技术方案之前，均应当依据相关法律法规通过恰当的方式对本公开所涉及个人信息的类型、使用范围、使用场景等告知用户并获得用户的授权。
例如，在响应于接收到用户的主动请求时，向用户发送提示信息，以明确地提示用户，其请求执行的操作将需要获取和使用到用户的个人信息。从而，使得用户可以根据提示信息来自主地选择是否向执行本公开技术方案的操作的电子设备、应用程序、服务器或存储介质等软件或硬件提供个人信息。
作为一种可选的但非限定性的实现方式，响应于接收到用户的主动请求，向用户发送提示信息的方式例如可以是弹窗的方式，弹窗中可以以文字的方式呈现提示信息。此外，弹窗中还可以承载供用户选择“同意”或者“不同意”向电子设备提供个人信息的选择控件。
可以理解的是，上述通知和获取用户授权过程仅是示意性的，不对本公开的实现方式构成限定，其它满足相关法律法规的方式也可应用于本公开的实现方式中。
可以理解的是，本技术方案所涉及的数据(包括但不限于数据本身、数据的获取或使用)应当遵循相应法律法规及相关规定的要求。
下面将参照附图更详细地描述本公开的实施例。虽然附图中示出了本公开的某些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反，提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实施例仅用于示例性作用，并非用于限制本公开的保护范围。
需要注意的是，本文中所提供的任何节/子节的标题并不是限制性的。本文通篇描述了各种实施例，并且任何类型的实施例都可以包括在任何节/子节下。此外，在任一节/子节中描述的实施例可以以任何方式与同一节/子节和/或不同节/子节中描述的任何其他实施例相结合。
在本公开的实施例的描述中，术语“包括”及其类似用语应当理解为开放性包含，即“包括但不限于”。术语“基于”应当理解为“至少部分地基于”。术语“一个实施例”或“该实施例”应当理解为“至少一个实施例”。术语“一些实施例”应当理解为“至少一些实施例”。下文还可能包括其他明确的和隐含的定义。术语“第一”、“第二”等可以指代不同的或相同的对象。下文还可能包括其他明确的和隐含的定义。
图1示出了本公开的实施例能够在其中实现的示例环境100的示意图。环境100可以是一个物理场景，即，环境100可以是一个真实世界的实际场景示例。在环境100中，用户110可以通过佩戴至少一个XR设备来实现交互操作。至少一个XR设备可以包括头戴式XR设备120，左右手分别手持的XR操纵器122-1和XR操纵器122-2(可以单独或统一称为XR操纵器122)，以及多个可以用于佩戴在身体其他部位如下肢上的定位器124-1和定位器124-2。定位器也可以称为追踪器。在一些实施例中，XR操纵器122-1可以包括定位器123-1，XR操纵器122-2可以包括定位器123-2，定位器123-1和定位器123-2可以属于第一组定位器(其中的各个定位器称为第一定位器123)。相应地，定位器124-1和定位器124-2可以属于第二组定位器(其中的各个定位器称为第二定位器124)。手持XR操纵器122例如可以是手柄等通过手部控制的任意适当类型的设备。第二定位器124可以是用于佩戴在身体多个部位(如腰上、手臂上等)的任何适当类型的追踪器。在这样的示例中，头戴式XR设备120可以与XR操纵器122和各第二定位器124进行通信，以通过协作的方式实现与用户的交互。
在一些实施例中，XR设备可以包括用于采集图像的设备，如相机、红外摄像头等的图像采集器，以拍摄物理空间的图像。图像采集器可以分别设置在一个或多个XR设备上。示例性地，图像采集器可以设置于头戴式XR设备120上，以基于较高的空间位置来采集更广的空间范围内的图像。若头戴式XR设备120上设置有红外摄像头，相应地，XR操纵器122和第二定位器124上可以分别设置有至少一个红外光源，以使红外摄像头可以拍摄到XR操纵器122和第二定位器124上的红外光源，从而根据得到的图像对XR操纵器122和第二定位器124进行识别定位。
在另一些场景中，用于图像采集的图像采集器可以独立于XR设备(例如，独立于头戴式XR设备120)。这样的图像采集器可以被独立部署以用于对环境中的第一定位器123、第二定位器124以及其他设备和环境等进行图像感知。
在一些实施例中，用户110可以利用多个XR设备与电子设备130进行通信，以利用电子设备130对通过XR设备获取的数据进行处理，并为XR设备下发数据。应当理解的是，在部分情况下，XR设备120也可以不与电子设备130进行通信(或者说，环境100也可以示例性地不包括电子设备130)。此时，可以通过头戴式XR设备120来与XR操纵器122-1(以及其中的第一定位器)、XR操纵器122-2(以及其中的第一定位器)、第二定位器124-1、第二定位器124-2等其他XR设备进行交互。具体地，可以通过头戴式XR设备120中的XR控制设备121来与其他XR设备进行交互。应当理解，XR控制设备121可以不限于设置在头戴式XR设备120中，其可以设置在其他XR设备中，也可以作为一个独立的控制设备存在。
在环境100中，用户110可以利用XR设备来获取虚拟环境140。虚拟环境140示例性地可以是一个空间。在一些实施例中，在虚拟环境140中可以产生用户110的虚拟对象150，以使得用户110可以在虚拟环境140中通过虚拟对象150进行沉浸式地互动体验，例如可以调整虚拟对象150的动作等。在一些实施例中，虚拟环境140中可以提供虚拟工具和/或其他用户的虚拟对象，以使得用户110可以在虚拟环境140中通过虚拟对象150与虚拟环境140中的虚拟工具进行互动，还可以通过虚拟对象150与其他用户的虚拟对象进行互动。
电子设备130可以是能够与XR设备和/或其他图像捕获设备进行通信的单独设备，比如用于图像或数据处理的服务器、计算节点等，也可以与XR设备和/或其他图像捕获设备集成在一起。在一些实施例中，电子设备130可以实现为XR设备，也即，在此情况下，XR设备可以实现电子设备130的所有功能。应当理解，上述关于电子设备130的描述仅仅是示例性而非限制性的，电子设备130可以实现为多种形式、结构或类别的设备，本公开的实施例对此没有限制。
应当理解，仅出于示例性的目的描述环境100中各个元素的结构和功能，而不暗示对于本公开的范围的任何限制。
如前文所简要提及的，用户可以通过佩戴XR设备来实现在虚拟环境中的身临其境的体验。因此，用户在佩戴XR设备进行操作时，对于部分XR设备，需要实现定位追踪。目前在XR领域对于追踪器(即定位器)或者手持操纵器(如手柄)的定位追踪，主要采用部署具有特定光源(例如光学红外灯)的定位器来实现定位追踪。具体通过相机拍摄手柄或者其他追踪器的红外灯来结合IMU(Inertial Measurement Unit，惯性测量单元)姿态进行追踪。
通常，期望生产和使用统一的体感追踪器，这不仅便于生成制造、而且也方便使用和扩展。然而，如果体感追踪器的外观以及其中设置的光源位置、光斑等均是完全一致的，那么将难以区分不同的追踪器。此外，根据不同场景，追踪器的使用数量也会不定，这样对于追踪器的区别只能依赖于追踪器的相对位置或者IMU姿态进行判断，这将导致对于检测和识别追踪器存在很大的干扰和困难。因此，在多个追踪器和手柄同时使用的时候，如何能够准确区分其中的不同定位器，避免误判和干扰，是一项待解决的重要任务。
本公开的实施例提出了一种用于设备控制的方案。根据本公开的各种实施例，控制第一组定位器中各个定位器处的光源根据第一组时序执行点亮。然后，控制第二组定位器中各个定位器处的光源根据第二组时序执行点亮，第一组时序和第二组时序被配置为使第一组定位器中至少一个定位器处的光源的点亮时间与第二组定位器中部分定位器处的光源的点亮时间同步。接下来，控制图像采集器至少根据第三时序执行图像采集，第三时序被配置为使图像采集器在第一组定位器中的至少一个定位器和第二组定位器中的部分定位器被同步点亮时执行图像采集。由此，该方案可以实现对第二组定位器中定位器的分时点亮，且同时兼容第一组定位器的点亮，实现了对不同定位器的区分，避免了后续根据拍摄的图像进行检测时发生定位混淆，进而提高了定位追踪的效率和准确性。
以下将继续参考附图描述本公开的一些示例实施例。
图2示出了根据本公开的一些实施例的设备控制的过程200的流程图。为便于讨论，将参考图1的环境100来描述这些实施例。在一些示例中，这些实施例可以被实现在图1的XR控制设备121中。在另一些示例中，这些实施例也可以被实现在远程控制器中，远程控制器例如可以被设置在图1的电子设备130中。以下具体实施例以被实现在XR控制设备121中作为示例，但这仅是示例性的，而无意任何限制。
还应当理解，虽然参考图1所示的涉及XR设备的环境100来描述示例实施例，但本公开的实施例所提供的设备控制方案可以同样被应用到非XR环境中，只要需要在这样的环境中需要利用图像采集器对环境中部署的定位器进行追踪和定位。
如图2所示，在框210，XR控制设备121可以控制第一组定位器中各个第一定位器123处的光源根据第一组时序执行点亮。在一些实施例中，光源包括但不限于红外光源。图像采集器可以识别并拍摄该光源，根据拍摄到的图像来实现对第一定位器123和定位器124的识别定位。在一些实施例中，各个第一定位器123被部署在XR操纵器122处。在这样的实施例中，XR操纵器122可以通过各个第一定位器123被识别定位。
在一些实施例中，第一组定位器包括多个第一定位器123，并且其中多个第一定位器123对应的多个第一时序被配置为使多个第一定位器123处的光源被同步点亮。每个XR操纵器122处可以部署至少一个第一定位器123。在这样的实施例中，图像采集器在每次拍摄到多个第一定位器123处的光源时，可以根据拍摄的图像直接区分出对应的XR操纵器122。因此，多个第一定位器123处的光源可以被控制为同步点亮。在一些实施例中，第一组定位器也可以仅包括单个第一定位器，并且在一些示例中该定位器可以被包括任一个XR操纵器122中。
结合图1，以第一组定位器包括由用户左手持有的XR操纵器122-1的第一定位器123-1和右手持有的XR操纵器122-2的第一定位器123-2为例，图像采集器同时拍摄到第一定位器123-1和第一定位器123-2处的光源时，可以在图像上直接区分出分别对应于左手XR操纵器122-1和右手XR操纵器122-2的光斑，从而识别出左手XR操纵器122-1和右手XR操纵器122-2的位置和姿态。
在一些实施例中，多个第一定位器123的光源图案不同。在这样的实施例中，图像采集器采集到多个第一定位器123的光源时，图像上的对应的光源图案(如光斑)即不同，从而能够快捷地分辨出多个第一定位器123对应的操纵器。在一些实施例中，定位器中的光源可以包括单个光源，该光源具有对应的光斑。在一些实施例中，定位器中的光源可以包括多个子光源，这些子光源可以具有一定的位置关系，从而通过控制这些子光源可以形成对应的光源光斑。
在框220，XR控制设备121可以控制第二组定位器中各个第二定位器124处的光源根据第二组时序执行点亮，第一组时序和第二组时序被配置为使第一组定位器中至少一个第一定位器123处的光源的点亮时间与第二组定位器中部分第二定位器124处的光源的点亮时间同步。在一些实施例中，第一组定位器的光源图案与第二组定位器的光源图案不同，例如任意一个第一定位器123的光源图案与任意一个第二定位器124的光源图案不同。
在一些场景中，图像采集器能够同时采集第一组定位器和第二组定位器的图像，在另一些场景中，为了实现对用户110的姿态和动作的更好地识别，可以使图像采集器同时采集第一组定位器和第二组定位器的图像。基于此，可以使第一组定位器的光源图案与第二组定位器的光源图案不同。这样，在图像采集器同时采集到第一定位器123和部分第二定位器124的图像时，可以方便地将二者区分出来。因此，通过使第一组定位器的光源图案与第二组定位器的光源图案不同，而且至少一个第一定位器123处的光源的点亮时间与第二组定位器中部分第二定位器124处的光源的点亮时间同步，可以提高定位追踪效果。
在一些实施例中，第二组定位器包括多个第二定位器124，并且其中多个第二定位器124对应的多个第二时序被配置为使多个第二定位器124处的光源被依次点亮。在一些实施例中，多个第二定位器124的光源图案相同。
由于第二定位器124可以被用户110同时使用多个来进行追踪定位，因此多个第二定位器124的外形和内部配置可以是相同的。相应地，多个第二定位器124的光源图案可以是相同的。应当理解，多个第二定位器124可以是同一种定位器。因此，在这样的实施例中，通过设置多个第二时序使对应的多个第二定位器124处的光源被依次点亮，能够实现多个第二定位器124的分时点亮。这样能够使图像采集器采集图像时只点亮一个第二定位器124，从而避免同步点亮多个第二定位器124导致识别混乱的异常情况。
在框230，XR控制设备121可以控制图像采集器至少根据第三时序执行图像采集，第三时序被配置为使图像采集器在第一组定位器中的至少一个第一定位器123(或全部第一定位器123)和第二组定位器中的部分第二定位器124被同步点亮时执行图像采集。
在这样的实施例中，通过设置第三时序来使图像采集器在第一组定位器中的至少一个第一定位器123(或全部第一定位器123)和第二组定位器中的部分第二定位器124被同步点亮时执行图像采集，可以实现第一定位器123所处的XR操纵器122与第二定位器124的同时使用，并实现对于至少一个XR操纵器122追踪的同时也实现对于多个第二定位器124的追踪和定位，而且可以实现XR操纵器122和第二定位器124的追踪定位互不干扰。
图3示出了根据本公开的一些实施例的第一组定位器的第一组时序、第二组定位器的第二组时序及图像采集器的第三时序的示例的示意图。示例300可以被实现在图1的环境100中。示例300可以被应用在过程200的部分步骤中。
如图3所示，示例300包括图像采集器的第三时序310，还包括XR操纵器122-1的第一定位器123-1的第一时序320-1和XR操纵器122-2的第一定位器123-2的第一时序320-2(可以单独或统一称为第一时序320)，第二定位器124-1的第二时序330-1和第二定位器124-2的第二时序330-2(可以单独或统一称为第二时序330)。第三时序310中包含多个拉高的脉冲301，可以表示头戴式XR设备120和XR操纵器122的曝光。第一时序320和第二时序330中均包含多个拉高的脉冲302，每个拉高的脉冲302表示对应的光源被点亮。
具体地，基于第一时序320-1和第一时序320-2，XR操纵器122-1和XR操纵器122-2处的光源被同步点亮。基于第二时序330-1、第二时序330-2和第一时序320，第二定位器124-1和第二定位器124-2处的光源的点亮时间不同，并且每个第二定位器与各个XR操纵器122处的光源的点亮时间分别同步。在本文中，控制不同定位器的“同步点亮”和“点亮时间同步”可以包括控制这些定位器在相同的时间点亮、或者让这些定位器在特定时间段内具有共同的点亮状态、或者让这些定位器被点亮从而使得图像采集装置在执行图像采集的一个曝光期间采集到这些定位器等。
进一步地，基于第二时序330-1和第二时序330-2，第二定位器124-1和第二定位器124-2处的光源被依次点亮(即分时点亮)。基于第三时序310，图像采集器在XR操纵器122-1和XR操纵器122-2、以及第二定位器124-1和第二定位器124-2中的一个第二定位器124均被点亮时执行图像采集。
在一些实施例中，第一组时序中各个时序的周期相同，并且第二组时序中各个时序的周期相同。在一些实施例中，第一组时序对应的周期与第二组时序对应的周期不同。
以图3为例，第一组时序包括第一时序320-1和第一时序320-2，第二组时序包括第二时序330-1和第二时序330-2。第一时序320-1和第一时序320-2的周期可以是相同的，第二时序330-1和第二时序330-2的周期也可以是相同的。而第一时序320和第二时序330的周期可以是不同的。因为第一时序320-1和第一时序320-2分别对应的第一定位器123-1和第一定位器123-2的光源在被图像采集器同时拍摄到时，可以区别出XR操纵器122-1和XR操纵器122-2，因此第一时序320-1和第一时序320-2的周期可以是相同的，且第一时序320-1和第一时序320-2可以是相同的。另外，因为多个第二定位器124处的光源可以被依次点亮，即第二时序330-1和第二时序330-2先后错开，因此第二时序330-1和第二时序330-2的周期可以是相同的，而第一时序320和第二时序330的周期可以是不同的。
由此，通过第一定位器123的第一时序320、第二定位器124的第二时序330及图像采集器的第三时序310，实现了第二定位器124的分时点亮，可以提高第二定位器124的追踪效果。并且，实现分时点亮能够降低各第二定位器124的功耗，从而提高产品的使用时长。
在一些实施例中，若检测到第二组定位器的连接完成，XR控制设备121可以基于第二组定位器的属性信息确定第二组定位器中各个第二定位器124对应的时序，第二组定位器中各个第二定位器124对应的时序基于第二组定位器中各个第二定位器124被点亮的次序和/或第二组定位器被依次轮询点亮的周期。在一些实施例中，第二组定位器的属性信息至少包括第二组定位器中第二定位器124的数目。在一些实施例中，至少一个第二定位器124的属性信息还可以包括其中每个第二定位器124的唯一识别号。
具体地，XR控制设备121在检测到连接的第二组定位器后，可以根据第二组定位器的定位器数目和识别号来确定其中每个第二定位器124被点亮的次序，并可以根据第二组定位器的定位器数目来确定各个第二定位器124被依次轮询点亮的周期。然后，基于该周期和每个第二定位器124被点亮的次序，可以确定各个第二定位器124的时序。由此，有利于实现各个第二定位器124的分时点亮。
在一些实施例中，第二组时序对应的周期是第一组时序对应的周期的整数倍数，并且其中整数倍数基于第二组定位器中的第二定位器124数目来确定。例如，如果第二组定位器的数目为2，那么其对应的第二组时序的周期是第一组时序的周期的2倍。参照图3，第二时序330的周期可以是第一时序320的2倍。由此，可以实现第二组定位器中的各个第二定位器124的光源被依次点亮。
在一些实施例中，在控制第二组定位器中各个第二定位器124处的光源根据第二组时序执行点亮时，针对第一图像采集周期，XR控制设备121可以将第二组时序中的各个时序分别配置到第二组定位器中的各个第二定位器124，并从第二组定位器中的各个第二定位器124分别接收针对第二组时序中的各个时序的配置的回复。第一图像采集周期可以对应于第二组定位器被执行完一轮分时点亮的完整周期，在这个图像采集周期中会有多个采集点。
在一些实施例中，在控制图像采集器至少根据第三时序执行图像采集时，若在预定时间内接收到从第二组定位器中的各个第二定位器124分别针对第二组时序中的各个时序的配置的回复，XR控制设备121可以控制图像采集器至少根据第三时序执行第一图像采集周期的图像采集。并且，若在预定时间内未从第二组定位器中的一个或多个第二定位器124接收到回复，XR控制设备121可以确定在第一图像采集周期中第二组时序的配置失败。
示例性地，在配置第二组时序时，若第二组定位器中的当前第二定位器124的光源根据对应的时序完成分时点亮，会发出关于时序配置的回复消息。然后，基于该回复消息，XR控制设备121可以为当前第二定位器的下一第二定位器124配置对应的时序，以使该下一第二定位器124的光源根据对应的时序执行分时点亮。
在这样的实施例中，根据第二定位器124对其时序配置的回复以及回复时间，才能确定第二定位器124是否配置成功。在第二定位器124配置成功的情况下，当前图像采集周期才能执行图像采集。若配置失败，即表示当前图像采集周期的时序同步失败，要等下一图像采集周期重新配置第二时序。由此，可以确保第二定位器124分时点亮的准确性，有利于提高对第二定位器124的追踪效率。
在一些实施例中，XR控制设备121可以对图像采集器采集到的第一图像执行图像分析，以确定在第一图像的采集时间处第二组定位器中被点亮的部分第二定位器124对应的位置信息和/或姿态信息。在一些实施例中，基于第二组时序，XR控制设备121可以确定在第一图像的采集时间处第二组定位器中被点亮的部分第二定位器124，并在第一图像的帧信息中写入所确定的部分第二定位器124的序号。并且，在确定部分定位器对应的位置信息和/或姿态信息时，XR控制设备121可以确定第一图像的帧信息中包含部分定位器对应的位置信息和/或姿态信息。
示例性地，若当前第二定位器124的下一第二定位器124所对应的时序配置完成，基于配置完成的时序，XR控制设备121可以在控制图像采集器采集该下一第二定位器124的图像之前，将该下一第二定位器124的序号写入图像采集器的配置信息中。XR控制设备121在将该下一第二定位器124的序号写入图像采集器的配置信息中后，图像采集器所采集到的该下一第二定位器124的图像的帧信息中便包含了对应的序号。然后，上层在获取这帧图像的时候，通过图像信息会获取到当前图像点亮的是该下一第二定位器124，并且当前图像中还包含至少一个XR操纵器122的第一定位器123的光源的图像，从而进行图像分析。由此，可以通过帧信息方便地识别到具体点亮的第二定位器124，有利于实现定位追踪，并相应获悉用户110的身体部分部位的姿态信息。
在一些实施例中，XR控制设备121可以从第二组定位器中的各个第二定位器124接收各自使用的时序，并确定从第二组定位器中的各个第二定位器124接收到的时序与第二组时序是否相同。若确定从第二组定位器中的至少一个第二定位器124接收到的至少一个时序与第二组时序中的至少一个时序不相同，将第二组时序中的各个时序分别重新配置给第二组定位器中的各个第二定位器124。
示例性地，在当前第二定位器124的光源点亮后，下一个需要点亮的第二定位器124可以立马上报自身的序号给到XR控制设备121，XR控制设备121会检测即将写入的序号的配置信息是否和上报的第二定位器124的序号一一对应。如果出现异常，会认为该下一个需要点亮的第二定位器124的时序错乱。然后可以在下一个图像采集周期重新进行该第二定位器124的时序的配置，来保证时序的稳定。由此，通过该时序检测机制，可以实时检测第二定位器124的时序是否异常，以确保各第二定位器124的分时点亮可以正常进行。
在一些实施例中，XR控制设备121可以检测第二组定位器中各个第二定位器124的连接状态，以确定第二组定位器中的第二定位器124数目是否发生变化。若确定第二组定位器中第二定位器数目发生变化，基于变化后的数目来确定变化后的第二组定位器对应的第二组时序。在一些实施例中，XR控制设备121在控制第一组定位器和第二组定位器执行点亮之前，确定第一组定位器和第二组定位器已连接到XR控制设备121。
具体地，若XR控制设备121连接到的第二定位器124的数目减少或者增多，都会基于变化后的数目来确定对应的第二定位器124的时序。在一些实施例中，XR控制设备121被部署在头戴式XR设备120处。示例性地，头戴式XR设备120可以检测其与各个第二定位器124的连接状态，以确定连接到头戴式XR设备的第二定位器124的数目是否发生变化，若发生变化，可以基于变化后的数目的第二定位器124来确定各个第二定位器124对应的时序。由此，通过第二定位器124的连接状态的检测机制，可以提高对多个第二定位器124追踪定位的灵活性和适应性。
通过以上公开的实施例，可以实现对第二组定位器的分时点亮，且同时兼容第一组定位器的点亮，避免了后续根据拍摄的图像进行检测时发生定位混淆，进而可以提高XR场景下定位追踪的效率和准确性。
图4示出了根据本公开的一些实施例的设备控制的过程的流程图。过程400可以被实现在图1的环境100中。过程400可以被示出为过程200的步骤的具体实施例。过程400可以根据图3的示例300的时序来实现。
如图4所示，过程400涉及头戴式XR设备120、第二定位器124(包括第二定位器124-1和第二定位器124-2)及XR操纵器122(包括XR操纵器122-1和XR操纵器122-2)的相关流程。每个第二定位器124执行的操作类似，并且每个XR操纵器122执行的操作类似。过程400可以以XR控制设备121和图像采集器被部署在头戴式XR设备120处为示例来详细描述。
XR操纵器122需要连接到头戴式XR设备120(特别是连接到其中的XR控制设备121)，且第二定位器124需要连接到头戴式XR设备120(特别是连接到其中的XR控制设备121)。头戴式XR设备120可以检测是否确定连接到第二定位器124。若确定连接到第二定位器124(410)，头戴式XR设备120向各个第二定位器124下发各自的第二时序(411)。
在连接到头戴式XR设备120(430)后，XR操纵器122根据预定的第一时序对自身第一定位器123的红外光源进行点亮(431)。也就是说，在每个周期中，头戴式设备120不需要每次都向XR操纵器122下发第一时序。第一时序可以被配置或定义在XR操纵器122处。在连接到头戴式设备120后，XR操纵器122可以根据预定的第一时序来执行点亮。此外，由于多个XR操纵器122的点亮时间可以是相同的，因此在每个XR操纵器122处配置的第一时序也可以是相同的。
在连接到头戴式XR设备120(420)后，每个第二定位器124接收到第二时序并进行配置(421)，然后向头戴式XR设备120回复配置完成(422)。头戴式XR设备120收到回复(412)后，判断第二定位器124是否是在预设时间内完成配置(413)。若第二定位器124未在预定时间内完成配置，则下一周期重新向第二定位器124下发第二时序(411)。若第二定位器124在预定时间内完成配置，则按照对应的第二时序进行图像采集器的配置(414)，具体可以将第二定位器124的序号写入图像采集器中。第二定位器124也会按照第二时序点亮自身的红外光源，实现红外光源的分时点亮(423)。然后，头戴式XR设备120可以根据图像采集器拍摄的图像的帧信息来获取点亮的定位器的序号(415)，并对该图像进行分析。
接下来，头戴式XR设备120可以实时判断第二定位器124的数目是否发生变化(416)。若发生变化，则重新更新第二时序，并下发第二时序(411)。第二定位器124可以定时上报当前的第二时序，然后头戴式XR设备120可以在第二定位器124的数目未发生变化时，判断上报的第二时序是否正确(417)。若正确，可以继续按照第二时序进行图像采集器的配置(414)。若不正确，会认为第二时序错乱，则在下一周期重新下发第二时序(411)，以保证第二时序的稳定。
以图1为示例，头戴式XR设备120拍摄XR操纵器122和第二定位器124的曝光时刻的时候，两个XR操纵器122中的第一定位器123总是同步点亮，因此每次拍摄时会一直捕捉到两个XR操纵器122中的第一定位器123，但是多个第二定位器124的红外光源是分时点亮的。在所有拍摄XR操纵器122的帧图像曝光时刻，左手的XR操纵器122-1和右手的XR操纵器122-2会同步点亮各自的红外光源。当前帧曝光之前，头戴式XR设备120会在图像采集器的配置中写入第二定位器124-1的序号。在曝光的时刻，第二定位器124-1的红外光源点亮，第二定位器124-2的红外光源关闭。上层在获取这帧图像的时候，通过图像信息会获取到当前图像点亮的是第二定位器124-1，并且获取到同步点亮的第二定位器124-1和XR操纵器122的定位器123的图像进行分析。
在下一帧曝光之前，头戴式XR设备120会在图像采集器的配置中写入第二定位器124-2的序列号。在曝光的时刻，第二定位器124-2点亮，第二定位器124-1关闭。上层在获取这帧图像的时候，通过图像信息会获取到当前图像点亮的是第二定位器124-2，并且获取到同步点亮第二定位器124-2和左右XR操纵器122的红外光源的图像进行分析。以此往复，可以实现对两个XR操纵器122和两个第二定位器124的追踪效果。
图5示出了根据本公开的一些实施例的图像采集器采集到的图像集500的示意图。图像集500可以被实现在图1的环境100中。图像集500可以被示出为过程200的执行结果的示例。图像集500可以被示出为采用图3的示例300来实现的方法的执行结果的示例。图像集500可以被示出为过程400的执行结果的示例。
如图5所示，图像集500包括图像510和图像520。图像510为在第二定位器124-1的红外光源点亮时由图形采集器采集到的图像，在图像510的帧信息中会记录第二定位器124-1的序号。图像520为在第二定位器124-2的红外光源点亮时由图像采集器采集到的图像，在图像520的帧信息中会记录第二定位器124-2的序号。
由于在第二定位器124-1的红外光源点亮的同时，两个XR操纵器122-1和122-2也被点亮。相应地，图像510中的3个光斑分别表示第二定位器124-1、左手的XR操纵器122-1和右手的XR操纵器122-2的红外光源点亮的曝光。由于在第二定位器124-2的红外光源点亮的同时，两个XR操纵器122-1和122-2也被点亮。相应地，图像520中的3个光斑分别表示第二定位器124-2、左手的XR操纵器122-1和右手的XR操纵器122-2的红外光源点亮的曝光。另外，可以看出光斑是在空间中有一定排布的。由此，该公开的实施例的用于设备控制的过程实现了对定位器的分时点亮控制，实现对定位器的追踪和定位。
图6示出了根据本公开的某些实施例的用于设备控制的装置600的示意性结构框图。装置600可以被实现为或者被包括在XR控制设备121中。装置600中的各个模块/组件可以由硬件、软件、固件或者它们的任意组合来实现。
如图所示，装置600包括第一点亮控制模块610，被配置为控制第一组定位器中各个定位器处的光源根据第一组时序执行点亮；第二点亮控制模块620，被配置为控制第二组定位器中各个定位器处的光源根据第二组时序执行点亮，第一组时序和第二组时序被配置为使第一组定位器中至少一个定位器处的光源的点亮时间与第二组定位器中部分定位器处的光源的点亮时间同步；以及图像采集模块630，被配置为控制图像采集器至少根据第三时序执行图像采集，第三时序被配置为使图像采集器在第一组定位器中的至少一个定位器和第二组定位器中的部分定位器被同步点亮时执行图像采集。
在一些实施例中，第一组时序对应的周期与第二组时序对应的周期不同。
在一些实施例中，第一组时序中各个时序的周期相同，并且第二组时序中各个时序的周期相同。
在一些实施例中，第一组定位器的光源图案与第二组定位器的光源图案不同。
在一些实施例中，第二组定位器包括多个第二定位器，并且其中多个第二定位器对应的多个第二时序被配置为使多个第二定位器处的光源被依次点亮。
在一些实施例中，多个第二定位器的光源图案相同。
在一些实施例中，第一组定位器包括多个第一定位器，并且其中多个第一定位器对应的多个第一时序被配置为使多个第一定位器处的红外光源被同步点亮。
在一些实施例中，多个第一定位器的光源图案不同。
在一些实施例中，装置600还包括时序确定模块，被配置为响应于检测到第二组定位器的连接完成，基于第二组定位器的属性信息确定第二组定位器中各个定位器对应的时序，第二组定位器中各个定位器对应的时序基于第二组定位器中各个定位器被点亮的次序和/或第二组定位器被依次轮询点亮的周期。
在一些实施例中，第二组定位器的属性信息至少包括第二组定位器中定位器的数目。
在一些实施例中，第二组时序对应的周期是第一组时序对应的周期的整数倍数，并且其中整数倍数基于第二组定位器中的定位器数目来确定。
在一些实施例中，第二点亮控制模块620进一步被配置为针对第一图像采集周期，将第二组时序中的各个时序分别配置到第二组定位器中的各个定位器；以及从第二组定位器中的各个定位器分别接收针对第二组时序中的各个时序的配置的回复。
在一些实施例中，图像采集模块630进一步被配置为响应于在预定时间内接收到从第二组定位器中的各个定位器分别针对第二组时序中的各个时序的配置的回复，控制图像采集器至少根据第三时序执行第一图像采集周期的图像采集；并且装置600还包括配置失败模块，被配置为响应于在预定时间内未从第二组定位器中的一个或多个定位器接收到回复，确定在第一图像采集周期中第二组时序的配置失败。
在一些实施例中，装置600还包括图像分析模块，被配置为对图像采集器采集到的第一图像执行图像分析，以确定在第一图像的采集时间处第二组定位器中被点亮的部分定位器对应的位置信息和/或姿态信息。
在一些实施例中，装置600还包括序号写入模块，被配置为基于第二组时序，确定在第一图像的采集时间处第二组定位器中被点亮的部分定位器；以及在第一图像的帧信息中写入所确定的部分定位器的序号；图像分析模块包括信息确定模块，被配置为确定第一图像的帧信息中包含部分定位器对应的位置信息和/或姿态信息。
在一些实施例中，装置600还包括数目确定模块，被配置为检测第二组定位器中各个定位器的连接状态，以确定第二组定位器中的定位器数目是否发生变化；以及响应于确定第二组定位器中定位器数目发生变化，基于变化后的数目来确定变化后的第二组定位器对应的第二组时序。
在一些实施例中，装置600还包括时序重新配置模块，被配置为从第二组定位器中的各个定位器接收各自使用的时序；确定从第二组定位器中的各个定位器接收到的时序与第二组时序是否相同；以及响应于确定从第二组定位器中的至少一个定位器接收到的至少一个时序与第二组时序中的至少一个时序不相同，将第二组时序中的各个时序分别重新配置给第二组定位器中的各个定位器。
在一些实施例中，装置600可以被实现在扩展现实(XR)控制设备处，并且装置600还包括连接确定模块，被配置为在控制第一组定位器和第二组定位器执行点亮之前，确定第一组定位器和第二组定位器已连接到XR控制设备。
在一些实施例中，XR控制设备包括头戴式XR设备，和/或其中第一组定位器被部署在XR操纵器处。
装置600中所包括的单元和/或模块可以利用各种方式来实现，包括软件、硬件、固件或其任意组合。在一些实施例中，一个或多个单元和/或模块可以使用软件和/或固件来实现，例如存储在存储介质上的机器可执行指令。除了机器可执行指令之外或者作为替代，装置500中的部分或者全部单元和/或模块可以至少部分地由一个或多个硬件逻辑组件来实现。作为示例而非限制，可以使用的示范类型的硬件逻辑组件包括现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准品(ASSP)、片上系统(SOC)、复杂可编程逻辑器件(CPLD)，等等。
图7示出了其中可以实施本公开的一个或多个实施例的电子设备700的框图。应当理解，图7所示出的电子设备700仅仅是示例性的，而不应当构成对本文所描述的实施例的功能和范围的任何限制。图7所示出的电子设备700可以用于实现图1的XR控制设备121。
如图7所示，电子设备700是通用计算设备的形式。电子设备700的组件可以包括但不限于一个或多个处理器或处理单元710、存储器720、存储设备730、一个或多个通信单元740、一个或多个输入设备750以及一个或多个输出设备760。处理单元710可以是实际或虚拟处理器并且能够根据存储器720中存储的程序来执行各种处理。在多处理器系统中，多个处理单元并行执行计算机可执行指令，以提高电子设备700的并行处理能力。
电子设备700通常包括多个计算机存储介质。这样的介质可以是电子设备700可访问的任何可以获取的介质，包括但不限于易失性和非易失性介质、可拆卸和不可拆卸介质。存储器720可以是易失性存储器(例如寄存器、高速缓存、随机访问存储器(RAM))、非易失性存储器(例如，只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、闪存)或它们的某种组合。存储设备530可以是可拆卸或不可拆卸的介质，并且可以包括机器可读介质，诸如闪存驱动、磁盘或者任何其他介质，其可以能够用于存储信息和/或数据并且可以在电子设备700内被访问。
电子设备700可以进一步包括另外的可拆卸/不可拆卸、易失性/非易失性存储介质。尽管未在图7中示出，可以提供用于从可拆卸、非易失性磁盘(例如“软盘”)进行读取或写入的磁盘驱动和用于从可拆卸、非易失性光盘进行读取或写入的光盘驱动。在这些情况中，每个驱动可以由一个或多个数据介质接口被连接至总线(未示出)。存储器720可以包括计算机程序产品725，其具有一个或多个程序模块，这些程序模块被配置为执行本公开的各种实施例的各种方法或动作。
通信单元740实现通过通信介质与其他计算设备进行通信。附加地，电子设备700的组件的功能可以以单个计算集群或多个计算机器来实现，这些计算机器能够通过通信连接进行通信。因此，电子设备700可以使用与一个或多个其他服务器、网络个人计算机(PC)或者另一个网络节点的逻辑连接来在联网环境中进行操作。
输入设备750可以是一个或多个输入设备，例如鼠标、键盘、追踪球等。输出设备760可以是一个或多个输出设备，例如显示器、扬声器、打印机等。电子设备700还可以根据需要通过通信单元740与一个或多个外部设备(未示出)进行通信，外部设备诸如存储设备、显示设备等，与一个或多个使得用户与电子设备700交互的设备进行通信，或者与使得电子设备700与一个或多个其他计算设备通信的任何设备(例如，网卡、调制解调器等)进行通信。这样的通信可以经由输入/输出(I/O)接口(未示出)来执行。
根据本公开的示例性实现方式，提供了一种计算机可读存储介质，其上存储有计算机可执行指令，其中计算机可执行指令被处理器执行以实现上文描述的方法。根据本公开的示例性实现方式，还提供了一种计算机程序产品，计算机程序产品被有形地存储在非瞬态计算机可读介质上并且包括计算机可执行指令，而计算机可执行指令被处理器执行以实现上文描述的方法。
这里参照根据本公开实现的方法、装置、设备和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理单元，从而生产出一种机器，使得这些指令在通过计算机或其他可编程数据处理装置的处理单元执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
可以把计算机可读程序指令加载到计算机、其他可编程数据处理装置、或其他设备上，使得在计算机、其他可编程数据处理装置或其他设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其他可编程数据处理装置、或其他设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实现的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
以上已经描述了本公开的各实现，上述说明是示例性的，并非穷尽性的，并且也不限于所公开的各实现。在不偏离所说明的各实现的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实现的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其他普通技术人员能理解本文公开的各个实现方式。

Claims (23)

  1. 一种用于设备控制的方法，包括：
    控制第一组定位器中各个定位器处的光源根据第一组时序执行点亮；
    控制第二组定位器中各个定位器处的光源根据第二组时序执行点亮，所述第一组时序和所述第二组时序被配置为使所述第一组定位器中至少一个定位器处的光源的点亮时间与所述第二组定位器中部分定位器处的光源的点亮时间同步；以及
    控制图像采集器至少根据第三时序执行图像采集，所述第三时序被配置为使所述图像采集器在所述第一组定位器中的至少一个定位器和所述第二组定位器中的部分定位器被同步点亮时执行图像采集。
  2. 根据权利要求1所述的方法，其中所述第一组时序对应的周期与所述第二组时序对应的周期不同。
  3. 根据权利要求1所述的方法，其中所述第一组时序中各个时序的周期相同，并且所述第二组时序中各个时序的周期相同。
  4. 根据权利要求1所述的方法，其中所述第一组定位器的光源图案与所述第二组定位器的光源图案不同。
  5. 根据权利要求1所述的方法，其中所述第二组定位器包括多个第二定位器，并且其中所述多个第二定位器对应的多个第二时序被配置为使所述多个第二定位器处的光源被依次点亮。
  6. 根据权利要求5所述的方法，其中所述多个第二定位器的光源图案相同。
  7. 根据权利要求1所述的方法，其中所述第一组定位器包括多个第一定位器，并且其中所述多个第一定位器对应的多个第一时序被配置为使所述多个第一定位器处的光源被同步点亮。
  8. 根据权利要求7所述的方法，其中所述多个第一定位器的光源图案不同。
  9. 根据权利要求1所述的方法，还包括至少通过以下方式确定所述第二组时序：
    响应于检测到所述第二组定位器的连接完成，基于所述第二组定位器的属性信息确定所述第二组定位器中各个定位器对应的时序，所述第二组定位器中各个定位器对应的时序基于所述第二组定位器中各个定位器被点亮的次序和/或所述第二组定位器被依次轮询点亮的周期。
  10. 根据权利要求9所述的方法，其中所述第二组定位器的属性信息至少包括所述第二组定位器中定位器的数目。
  11. 根据权利要求10所述的方法，其中所述第二组时序对应的周期是所述第一组时序对应的周期的整数倍数，并且其中所述整数倍数基于所述第二组定位器中的定位器数目来确定。
  12. 根据权利要求1所述的方法，其中控制所述第二组定位器中各个定位器处的光源根据第二组时序执行点亮包括：
    针对第一图像采集周期，将所述第二组时序中的各个时序分别配置到所述第二组定位器中的各个定位器；以及
    从所述第二组定位器中的各个定位器分别接收针对所述第二组时序中的各个时序的所述配置的回复。
  13. 根据权利要求12所述的方法，其中控制所述图像采集器至少根据第三时序执行图像采集包括：
    响应于在预定时间内接收到从所述第二组定位器中的各个定位器分别针对所述第二组时序中的各个时序的所述配置的回复，控制所述图像采集器至少根据第三时序执行所述第一图像采集周期的图像采集；并且
    其中所述方法还包括：
    响应于在所述预定时间内未从所述第二组定位器中的一个或多个定位器接收到所述回复，确定在所述第一图像采集周期中所述第二组时序的配置失败。
  14. 根据权利要求1所述的方法，还包括：
    对所述图像采集器采集到的第一图像执行图像分析，以确定在所述第一图像的采集时间处所述第二组定位器中被点亮的所述部分定位器对应的位置信息和/或姿态信息。
  15. 根据权利要求14所述的方法，还包括：
    基于所述第二组时序，确定在所述第一图像的采集时间处所述第二组定位器中被点亮的所述部分定位器；以及在所述第一图像的帧信息中写入所确定的所述部分定位器的序号；
    其中所述确定所述部分定位器对应的位置信息和/或姿态信息，包括：确定所述第一图像的帧信息中包含所述部分定位器对应的位置信息和/或姿态信息。
  16. 根据权利要求1所述的方法，还包括：
    检测所述第二组定位器中各个定位器的连接状态，以确定所述第二组定位器中的定位器数目是否发生变化；以及
    响应于确定所述第二组定位器中定位器数目发生变化，基于变化后的数目来确定变化后的所述第二组定位器对应的所述第二组时序。
  17. 根据权利要求1所述的方法，还包括：
    从所述第二组定位器中的各个定位器接收各自使用的时序；
    确定从所述第二组定位器中的各个定位器接收到的时序与所述第二组时序是否相同；以及
    响应于确定从所述第二组定位器中的至少一个定位器接收到的至少一个时序与所述第二组时序中的至少一个时序不相同，将所述第二组时序中的各个时序分别重新配置给所述第二组定位器中的各个定位器。
  18. 根据权利要求1所述的方法，其中所述方法被实现在扩展现实(XR)控制设备处，并且所述方法还包括：
    在控制所述第一组定位器和所述第二组定位器执行点亮之前，确定所述第一组定位器和所述第二组定位器已连接到所述XR控制设备。
  19. 根据权利要求18所述的方法，其中所述XR控制设备被部署在头戴式XR设备处，和/或
    其中所述第一组定位器被部署在XR操纵器处。
  20. 一种用于设备控制的装置，包括：
    第一点亮控制模块，被配置为控制第一组定位器中各个定位器处的光源根据第一组时序执行点亮；
    第二点亮控制模块，被配置为控制第二组定位器中各个定位器处的光源根据第二组时序执行点亮，所述第一组时序和所述第二组时序被配置为使所述第一组定位器中至少一个定位器处的光源的点亮时间与所述第二组定位器中部分定位器处的光源的点亮时间同步；以及
    图像采集模块，被配置为控制图像采集器至少根据第三时序执行图像采集，所述第三时序被配置为使所述图像采集器在所述第一组定位器中的至少一个定位器和所述第二组定位器中的部分定位器被同步点亮时执行图像采集。
  21. 一种电子设备，包括：
    至少一个处理单元；以及
    至少一个存储器，所述至少一个存储器被耦合到所述至少一个处理单元并且存储用于由所述至少一个处理单元执行的指令，所述指令在由所述至少一个处理单元执行时使所述电子设备执行根据权利要求1至19中任一项所述的方法。
  22. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序可由处理器执行以实现根据权利要求1至19中任一项所述的方法。
  23. 一种计算机程序产品，所述计算机程序产品被有形地存储在计算机存储介质中并且包括计算机可执行指令，计算机可执行指令在由设备执行时使设备执行根据权利要求1至19中任一项所述的方法。
PCT/CN2025/085492 2024-04-15 2025-03-27 用于设备控制的方法、装置、设备、存储介质和程序产品 Pending WO2025218474A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202410454996.7A CN120833453A (zh) 2024-04-15 2024-04-15 用于设备控制的方法、装置、设备、存储介质和程序产品
CN202410454996.7 2024-04-15

Publications (1)

Publication Number Publication Date
WO2025218474A1 true WO2025218474A1 (zh) 2025-10-23

Family

ID=97393962

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2025/085492 Pending WO2025218474A1 (zh) 2024-04-15 2025-03-27 用于设备控制的方法、装置、设备、存储介质和程序产品

Country Status (2)

Country Link
CN (1) CN120833453A (zh)
WO (1) WO2025218474A1 (zh)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106569337A (zh) * 2016-10-21 2017-04-19 北京小鸟看看科技有限公司 一种虚拟现实系统及其定位方法
CN206270589U (zh) * 2016-10-21 2017-06-20 北京小鸟看看科技有限公司 一种虚拟现实系统
CN111752386A (zh) * 2020-06-05 2020-10-09 深圳市欢创科技有限公司 一种空间定位方法、系统及头戴式设备

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106569337A (zh) * 2016-10-21 2017-04-19 北京小鸟看看科技有限公司 一种虚拟现实系统及其定位方法
CN206270589U (zh) * 2016-10-21 2017-06-20 北京小鸟看看科技有限公司 一种虚拟现实系统
CN111752386A (zh) * 2020-06-05 2020-10-09 深圳市欢创科技有限公司 一种空间定位方法、系统及头戴式设备

Also Published As

Publication number Publication date
CN120833453A (zh) 2025-10-24

Similar Documents

Publication Publication Date Title
CN110782492B (zh) 位姿跟踪方法及装置
JP2009093612A (ja) 光学ナビゲーションのために画像データのフレーム内の特徴クラスタにラベル付けするシステムおよび方法
US9430138B2 (en) Flexible room controls
US9223677B2 (en) Generation of trace data in a multi-processor system
CN111090255A (zh) 可编程逻辑控制器和主单元
JP7070153B2 (ja) 外観検査システム、外観検査結果の表示方法、および、外観検査結果の表示プログラム
JP2023509291A (ja) ジョイント赤外および可視光視覚慣性オブジェクト追跡
US11209790B2 (en) Actuator control system, actuator control method, information processing program, and storage medium
US20190005636A1 (en) Methods and systems for operating an apparatus through augmented reality
CN109388096A (zh) 信息处理装置及方法及非易失性的计算机可读记录介质
JP2020179441A (ja) 制御システム、情報処理装置および制御方法
JP2021024052A (ja) 画像処理装置、画像処理方法およびプログラム
JP6915288B2 (ja) 画像処理システム、画像処理装置、ＦＰＧＡ（Ｆｉｅｌｄ Ｐｒｏｇｒａｍｍａｂｌｅ Ｇａｔｅ Ａｒｒａｙ）における回路の再構成方法、および、ＦＰＧＡにおける回路の再構成プログラム
WO2025218474A1 (zh) 用于设备控制的方法、装置、设备、存储介质和程序产品
CN108803861B (zh) 一种交互方法、设备及系统
WO2022137580A1 (ja) 制御システム、サポート装置およびラベル付与方法
CN111752386A (zh) 一种空间定位方法、系统及头戴式设备
JP2021079464A (ja) 設定装置、設定方法およびプログラム
WO2020003887A1 (ja) 外観検査システム、外観検査結果の表示方法、および、外観検査結果の表示プログラム
JP2017211739A (ja) ユーザインターフェース装置およびユーザインターフェースプログラム
CN110471577B (zh) 一种360度全方位虚拟触控方法、系统、平台及存储介质
JP2018185563A (ja) 情報処理装置、情報処理方法、コンピュータプログラム、及び記憶媒体
JP7067869B2 (ja) 画像処理システム、情報処理装置、情報処理方法、および情報処理プログラム
JP2021182303A (ja) 操作領域の規定方法、操作領域の規定プログラム、情報処理装置、空間表示システム
WO2025237080A1 (zh) 用于设备通信的方法、装置、设备、存储介质和程序产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25789343

Country of ref document: EP

Kind code of ref document: A1