WO2018072593A1 - 虚拟现实系统及其空间定位装置、定位方法 - Google Patents

虚拟现实系统及其空间定位装置、定位方法 Download PDF

Info

Publication number
WO2018072593A1
WO2018072593A1 PCT/CN2017/102757 CN2017102757W WO2018072593A1 WO 2018072593 A1 WO2018072593 A1 WO 2018072593A1 CN 2017102757 W CN2017102757 W CN 2017102757W WO 2018072593 A1 WO2018072593 A1 WO 2018072593A1
Authority
WO
WIPO (PCT)
Prior art keywords
light source
infrared light
wireless controller
infrared
spatial positioning
Prior art date
Application number
PCT/CN2017/102757
Other languages
English (en)
French (fr)
Inventor
张秀志
曲婧
邓雪冰
Original Assignee
北京小鸟看看科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201610922097.0A external-priority patent/CN106569337B/zh
Priority claimed from CN201611073079.6A external-priority patent/CN106547458A/zh
Application filed by 北京小鸟看看科技有限公司 filed Critical 北京小鸟看看科技有限公司
Publication of WO2018072593A1 publication Critical patent/WO2018072593A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/02—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/04—Interpretation of pictures
    • G01C11/06—Interpretation of pictures by comparison of two or more pictures of the same area
    • G01C11/08—Interpretation of pictures by comparison of two or more pictures of the same area the pictures not being supported in the same relative position as when they were taken
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays

Definitions

  • the present invention relates to the field of virtual reality technologies, and in particular, to a virtual reality system, a spatial positioning device thereof, and a positioning method.
  • the existing space locating device is usually directly connected to the host through a cable, communicates with the host through the cable, and is powered by the host.
  • the space locating device is inconvenient to move, limiting the scope of use of the virtual reality device, and the line
  • the cable also limits the user's movements.
  • the spatial positioning is based on the infrared light source and the visible light source.
  • the specific working principle is that the headlight display and the wireless controller are provided with an infrared light source and a visible light source, and the infrared light source is positioned by the infrared light source.
  • the position of the display and the position of the wireless controller determine whether the positioned object is a head mounted display device or a wireless controller by different colors or different shapes of the visible light source.
  • the device is distinguished according to the color and shape of the visible light source. Objects with similar colors and shapes in the environment may cause interference, and interference may occur between the visible light sources on different devices when they are occluded or close to each other.
  • the invention provides a virtual reality system, a spatial positioning device and a positioning method thereof, so as to solve the problem that the existing space positioning device is inconvenient to move, and the cable can limit the motion of the user.
  • the present invention provides a spatial positioning device for a virtual reality system, the virtual reality system comprising a host, a head mounted display device provided with an infrared light source, and a wireless controller;
  • the spatial positioning device Includes: a microcontroller, a central processing unit, a first infrared image sensor, a second infrared image sensor, and a radio frequency chip;
  • the first infrared image sensor and the second infrared image sensor are configured to acquire images of infrared light sources disposed on the head mounted display device and/or the wireless controller from two different angles at the same time, and collect the images
  • the received image is sent to the central processor;
  • the central processor is configured to identify the position of the infrared light source according to the two images acquired by the first infrared image sensor and the second infrared image sensor at the same time, and extract the motion track of the infrared light source, and send the motion track to The microcontroller;
  • the microcontroller is configured to control a wireless connection between the radio frequency chip and a host of the virtual reality system; and control the radio frequency chip to send a motion track of the infrared light source to the host.
  • the present invention provides a virtual reality system including a host, a head mounted display device, a plurality of wireless controllers, infrared light sources respectively disposed on the head mounted display device and the wireless controller, And a plurality of the above spatial positioning devices;
  • the spatial positioning device is disposed around the head mounted display device and the wireless controller, and is wirelessly connected to the host through a radio frequency chip;
  • the host sends a first synchronization signal to each infrared light source and each spatial positioning device according to a preset frequency, and controls all the infrared light sources and the spatial positioning device to work synchronously;
  • the spatial positioning device collects an image according to a preset frequency, locates the position of each infrared light source from the image, and extracts a motion track of each infrared light source, and sends the image to the host through a radio frequency chip;
  • the host performs corresponding transformation on the image according to the received motion trajectory of each infrared light source, and sends the transformed image to the head mounted display device, and the head mounted display device displays the transformed image to the user. .
  • the present invention provides a virtual reality system positioning method, the method comprising:
  • the embodiment of the present invention uses two infrared image sensors to collect images of the infrared light source, and the central processor extracts the motion trajectory of the infrared light source according to the image, and then wirelessly transmits the signal to the host of the virtual reality system through the radio frequency chip, thereby realizing High-precision spatial positioning of the motion trajectory; and the space locating device provided by the present invention can supply power to itself through the built-in battery, so that it is not necessary to connect to the host of the virtual reality system through a cable, so that the space locating device can be freely moved, and the user has more A large range of activities allows you to experience virtual reality anytime, anywhere.
  • the distinction between the respective devices can be realized, thereby completing the tracking and positioning of the head mounted display device, the first wireless controller, and the second wireless controller.
  • FIG. 1 is a schematic structural diagram of a spatial positioning apparatus of a virtual reality system according to some embodiments of the present invention
  • FIG. 2 is a schematic structural diagram of a spatial positioning apparatus of a virtual reality system according to another embodiment of the present invention.
  • FIG. 3 is a functional block diagram of a virtual reality system according to some embodiments of the present invention.
  • FIG. 4 is a functional block diagram of a virtual reality system according to another embodiment of the present invention.
  • FIG. 5 is a functional block diagram of a wireless controller in some embodiments of the present invention.
  • FIG. 6 is a timing diagram of a synchronization signal in some embodiments of the present invention.
  • FIG. 7 is a flowchart of a method for locating a virtual reality system according to some embodiments of the present invention.
  • the design concept of the present invention is that the existing spatial positioning device needs to be connected to the host through a cable to communicate with the host through the cable, and the host provides power for the space, and the spatial positioning device is inconvenient to move, which limits the virtual reality device.
  • the range is used and the cable also limits the user's movements.
  • the present invention utilizes two infrared image sensors to acquire an image of the infrared light source, and the central processor extracts the motion trajectory of the infrared light source according to the image, and then wirelessly transmits the signal to the host of the virtual reality system through the radio frequency chip to achieve high motion trajectory.
  • the spatial positioning device provided by the present invention can supply power to itself through the built-in battery, so there is no need to connect to the host of the virtual reality system through a cable, so that the spatial positioning device can be freely moved, and the user has a larger range of activities. , you can experience virtual reality anytime, anywhere.
  • the existing virtual reality system distinguishes devices according to the color and shape of the visible light source. Objects with similar colors and shapes in the environment are prone to interference, and interference occurs when the visible light sources on different devices block or approach each other. .
  • the present invention provides only an infrared light source on the head mounted display device, the first wireless controller, and the second wireless controller, and does not provide a visible light source, and can control each infrared light source to work in different manners. Location and differentiation of devices. Since the present invention does not use a visible light source, the problem that the visible light source is likely to interfere with the positioning of the positioned device is avoided.
  • FIG. 1 is a schematic structural diagram of a spatial positioning apparatus of a virtual reality system according to some embodiments of the present invention
  • the virtual reality system includes a host, a head mounted display device provided with an infrared light source, and a wireless controller, as shown in FIG.
  • the spatial positioning device provided by this embodiment includes a microcontroller MCU 110, a central processing unit CPU 120, a first infrared image sensor CMOS1 130, a second infrared image sensor COMS2 140, and a radio frequency chip RF 150.
  • CMOS1 130 and a second infrared image sensor COMS2 140 for acquiring images of infrared light sources disposed on the head mounted display device and/or the wireless controller from two different angles at the same time, and collecting The received image is sent to the central processing unit 120;
  • the central processing unit CPU 120 is configured to identify the position of the infrared light source according to the two images acquired by the first infrared image sensor CMOS1 130 and the second infrared image sensor COMS2 140 at the same time, and extract the motion track of the infrared light source, and send the motion track to the infrared light source.
  • the microcontroller MCU 110 is configured to control the radio connection between the radio frequency chip RF 150 and the host of the virtual reality system; and control the radio frequency chip RF 150 to send the motion track of the infrared light source to the host.
  • the embodiment of the invention uses two infrared image sensors to collect images of the infrared light source, and the central processor extracts the motion track of the infrared light source according to the image, and then wirelessly transmits the signal to the host of the virtual reality system through the radio frequency chip to realize the motion track.
  • the high-precision spatial positioning eliminates the need to connect to the host of the virtual reality system through a cable, so that the spatial positioning device can be moved at will, and the user has a larger range of activities.
  • the central processing unit CPU is connected to the microcontroller through the SPI bus, the I/O port, and the URAT interface; the central processing unit CPU connects the first infrared image sensor CMOS1 and the second infrared through the IIC bus and the MIPI interface.
  • the microcontroller MCU reserves a URAT interface and a JTAG interface.
  • the central processing unit CPU and the radio frequency chip RF respectively reserve a URAT interface for debugging of the spatial positioning device.
  • the first infrared image sensor CMOS1 and the second infrared image sensor COMS2 are externally provided with a structural component LENS including a lens, and the first infrared image sensor CMOS1 and the second infrared image sensor COMS2 are collected from two different angles and disposed on the head mounted display device and / or an image of the infrared source on the wireless controller and send the captured image to the central processing unit CPU.
  • the central processing unit CPU is a master control scheme that supports the simultaneous operation of two image sensors. The central processing unit CPU recognizes the position of the infrared light source according to the two images acquired by the first infrared image sensor CMOS1 and the second infrared image sensor COMS2 at the same time.
  • the moving track of the infrared light source can be represented by Euler angle or quaternary number, and then the CPU of the central processor sends the motion track to the MCU of the microcontroller.
  • the microcontroller MCU controls the wireless connection between the RF chip RF and the host of the virtual reality system, and controls the RF chip RF to convert the motion track of the infrared light source into a radio frequency signal and send it to the host, thereby realizing high-precision spatial positioning of the motion track.
  • the spatial positioning device further includes: a power management chip PMU, a charging chip Charger, a charging interface, and a built-in battery battery.
  • the charging interface is a 9-pin Micro USB 3.0 interface.
  • the charging chip Charger takes power from the charging interface, and supplies power to the microcontroller MCU and the central processing unit CPU through the power management chip PMU, and simultaneously charges the built-in battery Battery; when the charging interface Micro USB is not When connected to an external power supply, the charging chip Charger takes power from the internal battery Battery and supplies power to the microcontroller MCU and the central processing unit CPU through the power management chip PMU.
  • the space locating device communicates with the host through the radio frequency chip RF, and can be powered by the built-in battery, so that the host of the virtual reality system does not need to be connected by a cable, so that the space locating device can move at random, and the user has more A large range of activities allows you to experience virtual reality anytime, anywhere.
  • the spatial positioning device further includes a multi-layer package chip eMCP chip and a charged erasable programmable read-only memory EEPROM chip, and the eMCP chip is connected to the central processing unit CPU for storing instructions required for the central processing unit CPU to work normally. And data; the EEPROM chip is connected to the RF chip RF for storing the software information of the RF chip RF. EEPROM core The chip is also connected to the microcontroller MCU through the IIC bus; when the spatial positioning device is in the normal working mode, the microcontroller MCU controls the EEPROM chip to open the write protection; when the spatial positioning device is in the firmware upgrade mode, the microcontroller MCU controls the EEPROM chip to be disabled. Write protection, and write data to the EEPROM chip through the IIC bus to realize the update and upgrade of the software information of the RF chip RF.
  • eMCP chip is connected to the central processing unit CPU for storing instructions required for the central processing unit CPU to work normally.
  • data the EEPROM chip is connected
  • the spatial positioning device provided in this embodiment further includes a tact switch KEY and a light emitting diode LED, the tact switch KEY and the light emitting diode LED are respectively connected to the power management chip PMU, and the tact switch KEY is used to control the space positioning device to be powered on and off.
  • the power-off diode LED is used to indicate the working state of the spatial positioning device, and the light-emitting diode LED can indicate that the device is wirelessly connected successfully, powered on, and charged by emitting light of different colors.
  • the clocks of the first infrared image sensor CMOS1 and the second infrared image sensor COMS2 can be obtained by the following two methods: In the first manner, the clock is obtained from the central processing unit CPU, and the first infrared image sensor CMOS1 and the second infrared image are in this case.
  • the clock of sensor COMS2 is identical to the clock of the central processor CPU.
  • an external crystal oscillator Xtal is provided, and the external crystal oscillator Xtal is connected to the first infrared image sensor CMOS1 and the second infrared image sensor COMS2, respectively, and the external crystal oscillator Xtal provides clocks for the first infrared image sensor CMOS1 and the second infrared image sensor COMS2.
  • the clocks of the first infrared image sensor CMOS1 and the second infrared image sensor COMS2 may not coincide with the clock of the central processing unit CPU.
  • the microcontroller MCU is further configured to control the radio frequency chip RF to receive the first synchronization signal from the host of the virtual reality system, and send the signal to the central processing unit CPU, where the first synchronization signal is used to control each device in the virtual reality system.
  • the central processing unit CPU transmits second synchronization signals, such as FSIN and VSYNC signals, to the first infrared image sensor CMOS1, the second infrared image sensor COMS2, and the microcontroller MCU according to the received first synchronization signal.
  • Each chip in the spatial positioning system is synchronized by the second synchronization signals FSIN and VSYNC signals, which can realize low delay of image acquisition and data back transmission, thereby realizing low delay of spatial positioning.
  • the microcontroller MCU receives the first synchronization signal from the host through the radio frequency chip RF and sends it to the central processing unit CPU, and the first synchronization signal is used to control the synchronization of each device in the virtual reality system;
  • the CPU sends a second synchronization signal to the first infrared image sensor CMOS1 according to the received first synchronization signal.
  • the first infrared image sensor CMOS1 receives the second synchronization signal, the infrared image is collected.
  • the first infrared image sensor CMOS1 collects each time.
  • the infrared image of one frame is delayed by a preset time interval, and the third synchronization signal is sent to the second infrared image sensor CMOS2; when the second infrared image sensor CMOS2 receives the third synchronization signal, the infrared image of the same frame is started to be collected.
  • a wireless display device includes two photosensitive chips, which are equivalent to two cameras, when there is only one set of chips for control and processing, if two photosensitive chips are taken at the same time, the two photosensitive chips will be at the same time.
  • the image data collected and sent out may conflict with each other. Therefore, there is a delay in the time when the two photosensitive chips start to collect images in the embodiment, for example, it may be 100 microseconds, and does not bring too much error to the positioning result. In addition, you can avoid conflicts in data transmission.
  • the charging interface Micro USB is connected to the central processing unit CPU, and the charging interface Micro USB supports the OTG.
  • the central processing unit CPU detects that the device connected to the charging interface Micro USB is an external storage device, the central processing unit CPU Images received from the first infrared image sensor CMOS1 and the second infrared image sensor COMS2 are written to the external storage device and power is supplied to the external storage device.
  • the spatial positioning device further includes an indicator LED for indicating an operational status and a button KEY for controlling the spatial positioning device.
  • FIG. 3 is a functional block diagram of a virtual reality system according to some embodiments of the present invention.
  • the virtual reality system provided in this embodiment includes a host 310, a head mounted display device 320, and a plurality of wireless controllers 330.
  • An infrared light source 340 disposed on the head mounted display device 320 and the wireless controller 330, and a plurality of spatial positioning devices 350 as described in the first embodiment above.
  • the spatial positioning device 350 is disposed around the head mounted display device 320 and the wireless controller 330 and is wirelessly connected to the host 310 via a radio frequency chip.
  • the host 310 transmits a first synchronization signal to each of the infrared light sources 340 and each of the spatial positioning devices 350 according to a preset frequency, and controls all of the infrared light sources 340 and the spatial positioning device 350 to operate in synchronization.
  • the spatial positioning device 350 acquires an image according to a preset frequency, locates the position of each infrared light source 340 from the image, and extracts a motion trajectory of each infrared light source 340, and transmits it to the host 310 through the built-in radio frequency chip.
  • the host 310 performs corresponding transformation on the image according to the received motion trajectory of each infrared light source 340, and transmits the transformed image to the head mounted display device 320, and the converted display image is displayed to the user by the head mounted display device 320.
  • the host of the virtual reality system may be an independent device, such as a PC or a mobile phone; the host of the virtual reality system may also be integrated with the head mounted display device, such as a virtual reality integrated machine.
  • the embodiment of the invention uses two infrared image sensors to collect images of the infrared light source, and the central processor extracts the motion trajectory of the infrared light source according to the image, and then wirelessly transmits the signal to the host of the virtual reality system through the radio frequency chip to realize high-precision spatial positioning of the motion track. There is no need to connect to the host of the virtual reality system through a cable, so that the space locating device can move freely, and the user has a larger range of activities.
  • the virtual reality system provided in this embodiment includes a host 410, a head mounted display device 420 wirelessly connected to the host 410, The first wireless controller 430, the second wireless controller 440, and a plurality of spatial positioning devices 450.
  • a plurality of spatial positioning devices 450 are disposed around the space where the head mounted display device 420, the first wireless controller 430, and the second wireless controller 440 are located, the head mounted display device 420, the first wireless controller 430, and the second wireless control.
  • the infrared light source is respectively disposed on the upper portion, and the spatial positioning device 450 can acquire the position information of each infrared light source by collecting the infrared image of the space. In fact, an infrared light source can be positioned using only two infrared images acquired simultaneously from different angles.
  • FIG. 4 shows the surrounding head mounted display device 420, the first wireless controller 430, and the second wireless controller 440.
  • Four spatial positioning devices are arranged in the space, which can effectively prevent the occlusion of the infrared light source during the movement of the user, such as occlusion of the body or occlusion of certain obstacles.
  • the host 410 can obtain initial position information of the head mounted display device 420, the first wireless controller 430, and the second wireless controller 440 through the spatial positioning device 450, and perform initial setting.
  • the head mounted display device 420, the first wireless controller 430, and the second wireless controller 440 may be sequentially illuminated in a certain order, or may be blinked according to a certain frequency, and the spatial positioning device 450 may acquire each device after acquiring the infrared image.
  • the sampling frequency of the spatial positioning device 450 is relatively high, and the moving speed of the user's head and hands is not too fast.
  • the position information of one device does not exhibit jumping changes in two consecutive frames, and the motion track is continuous.
  • the smooth curve after the system completes initialization to obtain the initial position information of each device, it can obtain the position information of each device in real time according to its motion track.
  • the distance between the head mounted display device 420, the first wireless controller 430, and the second wireless controller 440 may be too close, mutual occlusion, and trajectory crossing. In this case, it is necessary to re-determine which device each infrared light source corresponds to.
  • the infrared light source on the head-mounted display device 420 is always lit, and the first wireless controller The infrared light sources on the 430 and the second wireless controller 440 are alternately extinguished at a preset frequency.
  • the spatial positioning device 450 collects the infrared image at the preset frequency, it can determine that the infrared light source corresponds to the head mounted display device 420 according to the situation that the infrared light source is lit.
  • the wireless controller 430 is also the second wireless controller 440. After completing the distinction between the devices, the spatial positioning device transmits the location information corresponding to each device to the host 410, and the host 410 processes the image according to the location information, and then displays the processed image by the head mounted display device 420. user.
  • the virtual reality system provided in this embodiment can determine the position of the infrared light source by setting an infrared light source on the head mounted display device, the first wireless controller, and the second wireless controller, and acquiring the infrared image by using the spatial positioning device; and controlling each infrared
  • the light source works in different manners, and the distinction between the various devices can be realized, thereby completing the tracking and positioning of the head mounted display device, the first wireless controller, and the second wireless controller.
  • the virtual reality system provided by the embodiment does not need to set a visible light source on the head mounted display device, the first wireless controller, and the second wireless controller, thereby avoiding performing different operations on the positioned device according to different colors or different shapes of the visible light source. Distinguish between problems that are prone to interference.
  • each spatial positioning device 450 needs to collect the same frame image at the same time. If the time difference of the collected image is too large, the accuracy of the positioning may be affected; and the sampling frequency of the spatial positioning device 450 needs to be The frequencies of the infrared light sources on the first wireless controller 430 and the second wireless controller 440 are alternately extinguished, so that it is necessary to keep the clock synchronization of all the infrared light sources and the spatial positioning device.
  • the host 410 The first synchronization signal is periodically sent to all of the infrared light source and the spatial positioning device 450 during the operation of the virtual reality system to keep the clocks of all the infrared light sources and the spatial positioning device synchronized.
  • FIG. 5 is a functional block diagram of a wireless controller in some embodiments of the present invention.
  • the microcontroller 510 sends a control pulse to the infrared light source 530.
  • the signal causes the infrared source 530 to flash at a certain frequency.
  • the wireless controller also includes a button 540 for receiving user control commands, and an inertial measurement unit 550 that measures the wireless controller's own attitude data.
  • the infrared light source is illuminated when receiving a high level control signal.
  • the spatial positioning device collects an odd frame image
  • the infrared light source on the wireless controller lights up
  • the spatial positioning device collects the even frame image
  • the infrared light source on the second wireless controller lights up.
  • the odd-numbered frame image collected by the spatial positioning device has two infrared light sources, a head-mounted display device and a first wireless controller
  • the collected even-numbered frame images have two infrared light sources: a head-mounted display device and a second wireless controller.
  • the first infrared image sensor and the second infrared image sensor of the spatial positioning device are respectively connected to the central processor through the serial interface of the camera, and the collected infrared image is sent to the central processor, and the central processor transmits the infrared image of the previous frame with the current
  • the frame infrared image is compared: if an infrared light source is lit in both frames, the central processor determines that the infrared light source is an infrared light source disposed on the head mounted display device; if an infrared light source is only in an odd number When the frame image is lit, the central processor determines that the infrared light source is an infrared light source disposed on the first wireless controller; if an infrared light source is only lit in an even frame image, the central processor determines The infrared light source is an infrared light source disposed on the second wireless controller.
  • FIG. 7 is a flowchart of a method for locating a virtual reality system according to some embodiments of the present invention. As shown in FIG. 7 , the method for locating a virtual reality system provided by this embodiment includes:
  • Step S710 wirelessly connect the head mounted display device, the first wireless controller, the second wireless controller, and the plurality of spatial positioning devices to the host, and arrange the spatial positioning device around the head mounted display device and the first wireless controller. The space around the space where the second wireless controller is located.
  • Step S720 respectively setting an infrared light source on the head mounted display device, the first wireless controller, and the second wireless controller.
  • Step S730 Acquire initial position information of the head mounted display device, the first wireless controller, and the second wireless controller by using the spatial positioning device.
  • Step S740 controlling the infrared light source on the head mounted display device to be always lit, controlling the first wireless controller and the second wireless
  • the infrared light source on the controller alternately extinguishes according to the preset frequency, uses the spatial positioning device to collect the infrared image according to the preset frequency, determines the position of each infrared light source according to the collected infrared image, and distinguishes the head mounted display device and the first wireless controller.
  • the second wireless controller then sends the corresponding location information to the host.
  • the virtual reality system positioning method provided by the preferred embodiment of the present invention further includes: using the host timing to send the first synchronization signal to all the infrared light sources and the spatial positioning device, and maintaining the clocks of all the infrared light sources and the spatial positioning device. Synchronize.
  • step S740 “controlling the infrared light source on the first wireless controller and the second wireless controller to alternately extinguish according to the preset frequency” specifically includes:
  • the host sends a periodic control signal to the infrared light source on the first wireless controller and the second wireless controller, so that when the spatial positioning device collects the odd frame image, the infrared light source on the first wireless controller lights up when spatial positioning When the device collects an even frame image, the infrared light source on the second wireless controller lights up.
  • step of distinguishing the head mounted display device, the first wireless controller, and the second wireless controller in step S740 specifically includes:
  • the first infrared image sensor and the second infrared image sensor are used to compare the previous frame infrared image with the current frame infrared image:
  • an infrared light source is in a light state in two frames of images, determining that the infrared light source is an infrared light source disposed on the head mounted display device;
  • the infrared light source is an infrared light source disposed on the second wireless controller.
  • the present invention has the following advantages: the embodiment of the present invention uses two infrared image sensors to acquire an image of an infrared light source, and the central processor extracts a motion trajectory of the infrared light source according to the image, and then wirelessly transmits the image to the virtual reality system through the radio frequency chip.
  • the host device realizes high-precision spatial positioning of the motion trajectory; and the space locating device provided by the present invention can supply power to itself through the built-in battery, so that the space locating device can be freely moved without connecting to the host of the virtual reality system through a cable. Users have a larger range of activities and can experience virtual reality anytime, anywhere.
  • the distinction between the respective devices can be realized, thereby completing the tracking and positioning of the head mounted display device, the first wireless controller, and the second wireless controller.
  • various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof.
  • a microcontroller or digital signal processor may be used in practice to implement some or all of the functionality of some or all of the components in accordance with embodiments of the present invention.
  • the invention can also be implemented as a device or device program (e.g., a computer program and a computer program product) for performing some or all of the methods described herein.
  • a program implementing the invention may be stored on a computer readable medium or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
  • the spatial locating device of the virtual reality system of the present invention conventionally includes a processor and a computer program product or computer readable medium in the form of a memory.
  • the memory may be an electronic memory such as a flash memory, an EEPROM, an EPROM, a hard disk, or a ROM.
  • the memory has a memory space for program code for performing any of the method steps described above.
  • the storage space for the program code may include various program codes for implementing the various steps in the above methods, respectively.
  • the program code can be read from or written to one or more computer program products.
  • These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such computer program products are typically portable or fixed storage units.
  • the storage unit can be similarly arranged storage segments, storage spaces, and the like.
  • the program code can be compressed, for example, in an appropriate form.
  • the storage unit comprises computer readable code for performing the steps of the method according to the invention, ie code readable by, for example, a processor, which, when executed, causes the spatial locating device of the virtual reality system to perform the above described The various steps in the method.

Abstract

提供一种虚拟现实系统及其空间定位装置、定位方法，在空间定位装置（350）中，两个红外图像传感器（130，140）采集红外光源（340）的图像，中央处理器（120）从采集的图像中提取红外光源（340）的运动轨迹，微控制器（110）控制射频芯片（150）将红外光源（340）的运动轨迹无线发送给主机（310），使得空间定位装置（350）不需要连接线缆即可正常工作，可以随意移动，用户拥有了更大的活动范围，可以随时随地进行虚拟现实体验。并且控制各红外光源以不同的方式工作，完成对头戴显示设备（420）、第一无线控制器（430）、第二无线控制器（440）的跟踪定位。不需要在头戴显示设备（420）、第一无线控制器（430）、第二无线控制器（440）上设置可见光源，避免了根据可见光源的不同颜色或不同形状对被定位设备进行区分定位容易产生干扰的问题。

Description

虚拟现实系统及其空间定位装置、定位方法
交叉引用
本申请引用于2016年10月21日提交的专利名称为“一种虚拟现实系统及其定位方法”的第201610922097.0号以及专利名称为“一种虚拟现实系统及其空间定位装置”的第201611073079.6号中国专利申请，其通过引用被全部并入本申请。
技术领域
本发明涉及虚拟现实技术领域，具体涉及一种虚拟现实系统及其空间定位装置、定位方法。
发明背景
随着虚拟现实技术的发展，虚拟现实设备与体感游戏的结合也日渐白热化，需要对肢体动作的高精度空间定位。
现有的空间定位设备通常是通过线缆直接连接到主机上，通过线缆与主机通信，并由主机为其提供电源，空间定位设备不便于移动，限制了虚拟现实设备的使用范围，并且线缆也会限制用户的动作。
现有的虚拟现实系统，空间定位都是基于红外光源和可见光源来完成的，具体的工作原理是：头戴显示器和无线控制器上均设置有红外光源和可见光源，通过红外光源定位头戴显示器的位置以及无线控制器的位置，通过可见光源的不同颜色或不同形状判断定位到的物体是头戴显示设备还是无线控制器。根据可见光源的颜色和形状对设备进行区分，环境中颜色和形状近似的物体会产生干扰，并且在不同设备上的可见光源相互遮挡或者接近时相互之间也会产生干扰。
发明内容
本发明提供了一种虚拟现实系统及其空间定位装置、定位方法，以解决现有的空间定位设备不便于移动，并且线缆会限制用户的动作的问题。
根据本发明的第一实施例，本发明提供了一种虚拟现实系统的空间定位装置，所述虚拟现实系统包括主机、设置有红外光源的头戴显示设备和无线控制器；所述空间定位装置包括： 微控制器、中央处理器、第一红外图像传感器、第二红外图像传感器和射频芯片；
所述第一红外图像传感器和所述第二红外图像传感器，用于在同一时刻从两个不同的角度采集设置在头戴显示设备和/或无线控制器上的红外光源的图像，并将采集到的图像发送给所述中央处理器；
所述中央处理器，用于根据所述第一红外图像传感器和所述第二红外图像传感器在同一时刻采集的两帧图像，识别出红外光源的位置，并提取红外光源的运动轨迹，发送给所述微控制器；
所述微控制器，用于控制所述射频芯片与虚拟现实系统的主机无线连接；以及控制所述射频芯片将红外光源的运动轨迹发送给主机。
根据本发明的第二实施例，本发明提供了一种虚拟现实系统，包括主机、头戴显示设备、若干无线控制器、分别设置在所述头戴显示设备和无线控制器上的红外光源、以及若干上述的空间定位装置；
所述空间定位装置，布置在所述头戴显示设备和无线控制器周围，通过射频芯片无线连接到所述主机；
所述主机按照预设的频率向每个红外光源和每个空间定位装置发送第一同步信号，控制所有的红外光源和空间定位装置同步工作；
所述空间定位装置按照预设的频率采集图像，从所述图像中定位出每个红外光源的位置，并提取出每个红外光源的运动轨迹，通过射频芯片发送给所述主机；
所述主机根据接收到的每个红外光源的运动轨迹对图像进行相应的变换，并将变换后的图像发送给所述头戴显示设备，所述头戴显示设备将变换后的图像显示给用户。
根据本发明的第三实施例，本发明提供了一种虚拟现实系统定位方法，所述方法包括：
将头戴显示设备、第一无线控制器、第二无线控制器和多个空间定位装置分别无线连接到主机，将所述空间定位装置环绕布置在所述头戴显示设备、第一无线控制器、第二无线控制器所在空间的周围；
在所述头戴显示设备、第一无线控制器、第二无线控制器上分别设置红外光源；
通过所述空间定位装置分别获取所述头戴显示设备、第一无线控制器、第二无线控制器的初始位置信息；
控制所述头戴显示设备上的红外光源一直点亮，控制所述第一无线控制器和第二无线控制器上的红外光源按照预设频率交替明灭，利用所述空间定位装置按照所述预设频率采集所 述空间的红外图像，根据采集的红外图像确定每一个红外光源的位置，并区分所述头戴显示设备、第一无线控制器、第二无线控制器，之后将相应的位置信息发送给所述主机。
本发明的有益效果是：本发明实施例利用两个红外图像传感器采集红外光源的图像，中央处理器根据图像提取红外光源的运动轨迹，之后通过射频芯片无线发送给虚拟现实系统的主机，实现对运动轨迹的高精度空间定位；并且本发明提供的空间定位装置可以通过内置电池为自身供电，因此无需通过线缆连接到虚拟现实系统的主机，使得该空间定位装置可以随意移动，用户拥有了更大的活动范围，可以随时随地进行虚拟现实体验。
并且控制各红外光源以不同的方式工作，可以实现对各个设备的区分，从而完成对头戴显示设备、第一无线控制器、第二无线控制器的跟踪定位。不需要在头戴显示设备、第一无线控制器、第二无线控制器上设置可见光源，避免了根据可见光源的不同颜色或不同形状对被定位设备进行区分定位容易产生干扰的问题。
附图简要说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1是本发明一些实施例提供的一种虚拟现实系统的空间定位装置的结构示意图；
图2是本发明另一些实施例提供的一种虚拟现实系统的空间定位装置的结构示意图；
图3是本发明一些实施例提供的一种虚拟现实系统的功能框图；
图4是本发明另一些实施例提供的一种虚拟现实系统的功能框图；
图5是本发明一些实施例中无线控制器的功能框图；
图6是本发明一些实施例中同步信号的时序图；
图7是本发明一些实施例提供的一种虚拟现实系统定位方法的流程图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明实施方式作进一步地详细描述。
应当明确，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本 发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其它实施例，都属于本发明保护的范围。
本发明的设计构思是：现有的空间定位设备需要通过线缆连接到主机，以便通过线缆与主机通信，并由主机为其提供电源，空间定位设备不便于移动，限制了虚拟现实设备的使用范围，并且线缆也会限制用户的动作。针对这种情况，本发明利用两个红外图像传感器采集红外光源的图像，中央处理器根据图像提取红外光源的运动轨迹，之后通过射频芯片无线发送给虚拟现实系统的主机，实现对运动轨迹的高精度空间定位，并且本发明提供的空间定位装置可以通过内置电池为自身供电，因此无需通过线缆连接到虚拟现实系统的主机，使得该空间定位装置可以随意移动，用户拥有了更大的活动范围，可以随时随地进行虚拟现实体验。
现有的虚拟现实系统根据可见光源的颜色和形状对设备进行区分，环境中颜色和形状近似的物体容易产生干扰，并且在不同设备上的可见光源相互遮挡或者接近时相互之间也会产生干扰。针对这种情况，本发明在头戴显示设备、第一无线控制器、第二无线控制器上仅设置红外光源，不设置可见光源，通过控制各红外光源以不同的方式工作，可以实现对各个设备的定位和区分。由于本发明没有使用可见光源，因而也就避免了根据可见光源对被定位设备进行区分定位容易产生干扰的问题。
实施例一
图1是本发明一些实施例提供的一种虚拟现实系统的空间定位装置的结构示意图，该虚拟现实系统包括主机、设置有红外光源的头戴显示设备和无线控制器，如图1所示，本实施例提供的空间定位装置包括微控制器MCU 110、中央处理器CPU 120、第一红外图像传感器CMOS1 130、第二红外图像传感器COMS2 140、射频芯片RF 150。
第一红外图像传感器CMOS1 130和第二红外图像传感器COMS2 140，用于在同一时刻从两个不同的角度采集设置在头戴显示设备和/或无线控制器上的红外光源的图像，并将采集到的图像发送给中央处理器120；
中央处理器CPU 120，用于根据第一红外图像传感器CMOS1 130和第二红外图像传感器COMS2 140在同一时刻采集的两帧图像，识别出红外光源的位置，并提取红外光源的运动轨迹，发送给微控制器MCU 110；
微控制器MCU 110，用于控制射频芯片RF 150与虚拟现实系统的主机无线连接；以及控制射频芯片RF 150将红外光源的运动轨迹发送给主机。
本发明实施例利用两个红外图像传感器采集红外光源的图像，中央处理器根据图像提取红外光源的运动轨迹，之后通过射频芯片无线发送给虚拟现实系统的主机，实现对运动轨迹 的高精度空间定位，无需通过线缆连接到虚拟现实系统的主机，使得该空间定位装置可以随意移动，用户拥有了更大的活动范围。
具体地，如图2所示，中央处理器CPU通过SPI总线、I/O口和URAT接口连接微控制器；中央处理器CPU通过IIC总线和MIPI接口连接第一红外图像传感器CMOS1和第二红外图像传感器COMS2；中央处理器CPU通过IIC总线和IO口连接电源管理芯片PMU；微控制器MCU通过SPI总线、IIC总线和I/O口连接射频芯片RF。
微控制器MCU预留有一个URAT接口和一个JTAG接口，中央处理器CPU和射频芯片RF分别预留有一个URAT接口，用于空间定位装置的调试。
第一红外图像传感器CMOS1和第二红外图像传感器COMS2外设置有包含镜头的结构组件LENS，第一红外图像传感器CMOS1和第二红外图像传感器COMS2从两个不同的角度采集设置在头戴显示设备和/或无线控制器上的红外光源的图像，并将采集到的图像发送给中央处理器CPU。中央处理器CPU为支持两个图像传感器同时工作的主控方案，中央处理器CPU根据第一红外图像传感器CMOS1和第二红外图像传感器COMS2在同一时刻采集的两帧图像，识别出红外光源的位置，并提取红外光源的运动轨迹，红外光源运动轨迹可以用欧拉角或四元数来表示，之后中央处理器CPU将运动轨迹发送给微控制器MCU。微控制器MCU控制射频芯片RF与虚拟现实系统的主机无线连接，并控制射频芯片RF将红外光源的运动轨迹转换成射频信号发送给主机，从而实现对运动轨迹的高精度空间定位。
在一些可选的实施例中，该空间定位装置还包括：电源管理芯片PMU、充电芯片Charger、充电接口和内置电池Battery。
本实施例中，充电接口为9pin Micro USB3.0接口。当充电接口Micro USB连接到外部电源时，充电芯片Charger从充电接口取电，通过电源管理芯片PMU为微控制器MCU和中央处理器CPU供电，同时为内置电池Battery充电；当充电接口Micro USB未连接到外部电源时，充电芯片Charger从内置电池Battery取电，通过电源管理芯片PMU为微控制器MCU和中央处理器CPU供电。
本实施例提供的空间定位装置通过射频芯片RF与主机进行通信，并且可以通过内置电池供电，因此无需通过线缆连接到虚拟现实系统的主机，使得该空间定位装置可以随意移动，用户拥有了更大的活动范围，可以随时随地进行虚拟现实体验。
本实施例中，空间定位装置还包括多制层封装芯片eMCP芯片和带电可擦可编程只读存储器EEPROM芯片，eMCP芯片连接中央处理器CPU，用于存储中央处理器CPU正常工作所需要的指令和数据；EEPROM芯片连接射频芯片RF，用于存储射频芯片RF的软件信息。EEPROM芯 片还通过IIC总线连接微控制器MCU；当空间定位装置处于正常工作模式时，微控制器MCU控制EEPROM芯片开启写保护；当空间定位装置处于固件升级模式时，微控制器MCU控制EEPROM芯片禁用写保护，并通过IIC总线向EEPROM芯片写入数据，实现对射频芯片RF的软件信息的更新升级。
本实施例提供的空间定位装置还包括轻触开关KEY和发光二极管LED，轻触开关KEY和发光二极管LED分别连接电源管理芯片PMU，轻触开关KEY用于控制空间定位装置上电开机和断电关机，发光二极管LED用于指示空间定位装置的工作状态，发光二极管LED可以通过发出不同颜色的光指示设备无线连接成功、开机启动、正在充电等状态。
第一红外图像传感器CMOS1和第二红外图像传感器COMS2的时钟可以通过以下两种方式获取：第一种方式，从中央处理器CPU中获取时钟，此时第一红外图像传感器CMOS1、第二红外图像传感器COMS2的时钟与中央处理器CPU的时钟一致。第二种方式，设置一个外部晶振Xtal，外部晶振Xtal分别连接第一红外图像传感器CMOS1和第二红外图像传感器COMS2，由外部晶振Xtal为第一红外图像传感器CMOS1和第二红外图像传感器COMS2提供时钟，此时第一红外图像传感器CMOS1、第二红外图像传感器COMS2的时钟可以与中央处理器CPU的时钟不一致。
在一个实施例中，微控制器MCU还用于控制射频芯片RF从虚拟现实系统的主机接收第一同步信号，并发送给中央处理器CPU，第一同步信号用于控制虚拟现实系统中各个设备保持同步。中央处理器CPU根据接收到的第一同步信号向第一红外图像传感器CMOS1、第二红外图像传感器COMS2和微控制器MCU发送第二同步信号，如FSIN及VSYNC信号。空间定位系统中的各个芯片通过第二同步信号FSIN及VSYNC信号保持同步，可以实现图像采集及数据回传的低延时，从而实现空间定位的低延时性。
在另一些实施例中，微控制器MCU通过射频芯片RF从主机接收第一同步信号，并发送给中央处理器CPU，第一同步信号用于控制虚拟现实系统中各个设备保持同步；中央处理器CPU根据接收到的第一同步信号向第一红外图像传感器CMOS1发送第二同步信号，当第一红外图像传感器CMOS1接收到第二同步信号时，开始采集红外图像；第一红外图像传感器CMOS1每采集一帧红外图像，延时一个预设时间间隔后，向第二红外图像传感器CMOS2发送第三同步信号；当第二红外图像传感器CMOS2接收到第三同步信号时，开始采集同一帧红外图像。由于一个无线显示设备中包括了两个感光芯片，相当于两个相机，当负责控制和处理的芯片只有一套，若使两个感光芯片在同一时间拍摄，两个感光芯片会在同一时间向外发送采集的图像数据，有可能会相互冲突，因此本实施例中两个感光芯片开始采集图像的时间有一个延时，例如可以是100微秒，不会给定位结果带来过多的误差，又可以避免数据传输的冲突。
在另一个优选实施例中，充电接口Micro USB连接到中央处理器CPU，充电接口Micro USB支持OTG，当中央处理器CPU检测连接到充电接口Micro USB的设备为外部存储设备时，中央处理器CPU将从第一红外图像传感器CMOS1和第二红外图像传感器COMS2接收到的图像写入该外部存储设备，并为该外部存储设备供电。
该空间定位装置还包括用于指示工作状态的指示灯LED以及用于控制空间定位装置的按键KEY。
实施例二
图3是本发明一些实施例提供的一种虚拟现实系统的功能框图，如图3所示，本实施例提供的虚拟现实系统包括主机310、头戴显示设备320、若干无线控制器330、分别设置在头戴显示设备320和无线控制器330上的红外光源340、以及若干如上述实施例一所述的空间定位装置350。
空间定位装置350布置在头戴显示设备320和无线控制器330周围，通过射频芯片无线连接到主机310。主机310按照预设的频率向每个红外光源340和每个空间定位装置350发送第一同步信号，控制所有的红外光源340和空间定位装置350同步工作。
空间定位装置350按照预设的频率采集图像，从图像中定位出每个红外光源340的位置，并提取出每个红外光源340的运动轨迹，通过内置的射频芯片发送给主机310。主机310根据接收到的每个红外光源340的运动轨迹对图像进行相应的变换，并将变换后的图像发送给头戴显示设备320，由头戴显示设备320将变换后的图像显示给用户。
本实施例中，虚拟现实系统的主机可以为一个独立的设备，如PC机、手机等；虚拟现实系统的主机也可以与头戴显示设备集成为一个整体，如虚拟现实一体机。
本发明实施例利用两个红外图像传感器采集红外光源的图像，中央处理器根据图像提取红外光源的运动轨迹，之后通过射频芯片无线发送给虚拟现实系统的主机，实现对运动轨迹的高精度空间定位，无需通过线缆连接到虚拟现实系统的主机，使得该空间定位装置可以随意移动，用户拥有了更大的活动范围。
实施例三
图4是本发明另一些实施例提供的一种虚拟现实系统的功能框图，如图4所示，本实施例提供的虚拟现实系统包括主机410、与主机410无线连接的头戴显示设备420、第一无线控制器430、第二无线控制器440和多个空间定位装置450。
多个空间定位装置450环绕布置在头戴显示设备420、第一无线控制器430、第二无线控制器440所在空间的周围，头戴显示设备420、第一无线控制器430、第二无线控制器440 上分别设置有红外光源，空间定位装置450通过采集该空间的红外图像可以获取每一个红外光源的位置信息。事实上，仅利用从不同角度同时采集的两幅红外图像即可对一个红外光源进行定位，图4示出了环绕头戴显示设备420、第一无线控制器430、第二无线控制器440所在空间布置了4个空间定位装置，可以有效防止用户运动过程中对红外光源的遮挡，例如身体的遮挡或某些障碍物的遮挡。
在该虚拟现实系统启动时，主机410通过空间定位装置450可以分别获取头戴显示设备420、第一无线控制器430、第二无线控制器440的初始位置信息，进行初始化设置。例如可以控制头戴显示设备420、第一无线控制器430、第二无线控制器440按照一定顺序依次点亮，或者按照一定的频率闪烁，空间定位装置450采集红外图像后可分别获取每一个设备的初始位置。空间定位装置450的采样频率较高，而用户头部和双手的运动速度不会过快，通常一个设备的位置信息不会在连续两帧图像中出现跳跃性变化，其运动轨迹是连续的、平滑的曲线，在系统完成初始化获取每个设备的初始位置信息后，即可根据其运动轨迹实时获取每个设备的位置信息。
但是，在该虚拟现实系统运行过程中，头戴显示设备420、第一无线控制器430、第二无线控制器440之间可能会出现距离过近、相互遮挡、轨迹交叉等情形，在这种情况下就必须重新判断每一个红外光源对应的是哪一个设备。为了区分头戴显示设备420、第一无线控制器430、第二无线控制器440，本实施例提供的虚拟现实系统中，头戴显示设备上420的红外光源一直点亮，第一无线控制器430和第二无线控制器440上的红外光源按照预设频率交替明灭。由于每一个红外光源工作的时序不同，空间定位装置450以这一预设频率采集红外图像后，就可以根据红外光源点亮的情况，判断一个红外光源对应的是头戴显示设备420、第一无线控制器430还是第二无线控制器440。在完成设备之间的区分之后，空间定位装置将每一个设备对应的位置信息发送给主机410，由主机410根据位置信息对图像进行处理，再由头戴显示设备420将处理后的图像显示给用户。
本实施例提供的虚拟现实系统，通过在头戴显示设备、第一无线控制器、第二无线控制器上设置红外光源，利用空间定位装置采集红外图像可以确定红外光源的位置；并且控制各红外光源以不同的方式工作，可以实现对各个设备的区分，从而完成对头戴显示设备、第一无线控制器、第二无线控制器的跟踪定位。本实施例提供的虚拟现实系统不需要在通过在头戴显示设备、第一无线控制器、第二无线控制器上设置可见光源，避免了根据可见光源的不同颜色或不同形状对被定位设备进行区分容易产生干扰的问题。
本发明提供的虚拟现实系统，各空间定位装置450需要在同一时间采集同一帧图像，如果采集图像的时间差过大，则会影响定位的精度；并且空间定位装置450的采样频率需要与 第一无线控制器430和第二无线控制器440上的红外光源交替明灭的频率保持一致，因此需要保持所有红外光源和空间定位装置的时钟同步，在本发明的一个优选实施例中，主机410在虚拟现实系统运行过程中定时向所有红外光源和空间定位装置450发送第一同步信号，保持所有红外光源和空间定位装置的时钟同步。
图5是本发明一些实施例中无线控制器的功能框图，如图5所示，当无线通信模块520接收到主机发送的第一同步信号后，微控制器510会向红外光源530发送控制脉冲信号，使红外光源530按照一定的频率闪烁。无线控制器还包括用于接收用户控制指令的按键540，以及测量无线控制器自身姿态数据的惯性测量单元550。
图6是本发明一些实施例中同步信号的时序图，如图6所示，红外光源接收到高电平的控制信号时会点亮，本实施中当空间定位装置采集奇数帧图像时，第一无线控制器上的红外光源点亮，当空间定位装置采集偶数帧图像时，第二无线控制器上的红外光源点亮。空间定位装置采集的奇数帧图像上会有头戴显示设备和第一无线控制器两个红外光源，采集的偶数帧图像上会有头戴显示设备和第二无线控制器两个红外光源。空间定位装置的第一红外图像传感器和第二红外图像传感器分别通过相机串行接口连接到中央处理器，将采集到的红外图像发送给中央处理器，中央处理器将上一帧红外图像与当前帧红外图像进行对比：若某一红外光源在两帧图像中均处于点亮状态，则中央处理器判断该红外光源为设置在头戴显示设备上的红外光源；若某一红外光源仅在奇数帧图像中处于点亮状态，则中央处理器判断该红外光源为设置在第一无线控制器上的红外光源；若某一红外光源仅在偶数帧图像中处于点亮状态，则中央处理器判断该红外光源为设置在第二无线控制器上的红外光源。从而实现了对头戴显示设备、第一无线控制器、第二无线控制器之间的区分。
实施例四
图7是本发明一些实施例提供的一种虚拟现实系统定位方法的流程图，如图7所示，本实施例提供的虚拟现实系统定位方法包括：
步骤S710：将头戴显示设备、第一无线控制器、第二无线控制器和多个空间定位装置分别无线连接到主机，并将空间定位装置环绕布置在头戴显示设备、第一无线控制器、第二无线控制器所在空间的周围。
步骤S720：在头戴显示设备、第一无线控制器、第二无线控制器上分别设置红外光源。
步骤S730：通过空间定位装置分别获取头戴显示设备、第一无线控制器、第二无线控制器的初始位置信息。
步骤S740：控制头戴显示设备上的红外光源一直点亮，控制第一无线控制器和第二无线 控制器上的红外光源按照预设频率交替明灭，利用空间定位装置按照预设频率采集红外图像，根据采集的红外图像确定每一个红外光源的位置，并区分头戴显示设备、第一无线控制器、第二无线控制器，之后将相应的位置信息发送给主机。
为了使各设备协同工作，本发明一优选实施例提供的虚拟现实系统定位方法还包括：利用主机定时向所有红外光源和空间定位装置发送第一同步信号，保持所有红外光源和空间定位装置的时钟同步。
优选地，步骤S740中“控制第一无线控制器和第二无线控制器上的红外光源按照预设频率交替明灭”具体包括：
利用主机向第一无线控制器和第二无线控制器上的红外光源发送周期性控制信号，使得当空间定位装置采集奇数帧图像时，第一无线控制器上的红外光源点亮，当空间定位装置采集偶数帧图像时，第二无线控制器上的红外光源点亮。
优选地，步骤S740中“区分头戴显示设备、第一无线控制器、第二无线控制器”具体包括：
将第一红外图像传感器和第二红外图像传感器采集上一帧红外图像与当前帧红外图像进行对比：
若某一红外光源在两帧图像中均处于点亮状态，则判断该红外光源为设置在头戴显示设备上的红外光源；
若某一红外光源仅在奇数帧图像中处于点亮状态，则判断该红外光源为设置在第一无线控制器上的红外光源；
若某一红外光源仅在偶数帧图像中处于点亮状态，则判断该红外光源为设置在第二无线控制器上的红外光源。
综上所述，本发明的有益效果是：本发明实施例利用两个红外图像传感器采集红外光源的图像，中央处理器根据图像提取红外光源的运动轨迹，之后通过射频芯片无线发送给虚拟现实系统的主机，实现对运动轨迹的高精度空间定位；并且本发明提供的空间定位装置可以通过内置电池为自身供电，因此无需通过线缆连接到虚拟现实系统的主机，使得该空间定位装置可以随意移动，用户拥有了更大的活动范围，可以随时随地进行虚拟现实体验。
并且控制各红外光源以不同的方式工作，可以实现对各个设备的区分，从而完成对头戴显示设备、第一无线控制器、第二无线控制器的跟踪定位。不需要在头戴显示设备、第一无线控制器、第二无线控制器上设置可见光源，避免了根据可见光源的不同颜色或不同形状对被定位设备进行区分定位容易产生干扰的问题。
以上所述，仅为本发明的具体实施方式，在本发明的上述教导下，本领域技术人员可以在上述实施例的基础上进行其他的改进或变形。本领域技术人员应该明白，上述的具体描述只是更好的解释本发明的目的，本发明的保护范围应以权利要求的保护范围为准。
需要说明的是：本发明的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的软件模块实现，或者以它们的组合实现。本领域的技术人员应当理解，可以在实践中使用微控制器或者数字信号处理器(DSP)来实现根据本发明实施例中的一些或者全部部件的一些或者全部功能。本发明还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如，计算机程序和计算机程序产品)。这样的实现本发明的程序可以存储在计算机可读介质上，或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到，或者在载体信号上提供，或者以任何其他形式提供。
本发明的虚拟现实系统的空间定位装置传统上包括处理器和以存储器形式的计算机程序产品或者计算机可读介质。存储器可以是诸如闪存、EEPROM、EPROM、硬盘或者ROM之类的电子存储器。存储器具有用于执行上述方法中的任何方法步骤的程序代码的存储空间。例如，用于程序代码的存储空间可以包括分别用于实现上面的方法中的各种步骤的各个程序代码。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。这些计算机程序产品包括诸如硬盘，紧致盘(CD)、存储卡或者软盘之类的程序代码载体。这样的计算机程序产品通常为便携式或者固定存储单元。该存储单元可以类似布置的存储段、存储空间等。程序代码可以例如以适当形式进行压缩。通常，存储单元包括用于执行根据本发明的方法步骤的计算机可读代码，即可以由例如处理器读取的代码，这些代码被运行时，导致该虚拟现实系统的空间定位装置执行上面所描述的方法中的各个步骤。
应该注意的是上述实施例对本发明进行说明而不是对本发明进行限制，并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。单词“包括”不排除存在未列在权利要求中的元件或步骤。本发明可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本发明的实施例可以在没有这些具体细节的情况下被实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。本说明书中使用的语言主要是为了可读性和教导的目的而选择的，而不是为了解释或者限定本发明的主题而选择的。

Claims (20)

  1. 一种虚拟现实系统的空间定位装置，所述虚拟现实系统包括主机、设置有红外光源的头戴显示设备和无线控制器；所述空间定位装置包括：微控制器、中央处理器、第一红外图像传感器、第二红外图像传感器和射频芯片；
    所述第一红外图像传感器和所述第二红外图像传感器，用于在同一时刻从两个不同的角度采集设置在头戴显示设备和/或无线控制器上的红外光源的图像，并将采集到的图像发送给所述中央处理器；
    所述中央处理器，用于根据所述第一红外图像传感器和所述第二红外图像传感器在同一时刻采集的两帧图像，识别出红外光源的位置，并提取红外光源的运动轨迹，发送给所述微控制器；
    所述微控制器，用于控制所述射频芯片与虚拟现实系统的主机无线连接；以及控制所述射频芯片将红外光源的运动轨迹发送给所述主机。
  2. 如权利要求1所述的空间定位装置，其中，
    所述微控制器，还用于通过所述射频芯片从主机接收第一同步信号，并发送给所述中央处理器；
    所述中央处理器根据接收到的第一同步信号同时向所述第一红外图像传感器、所述第二红外图像传感器、所述微控制器发送第二同步信号。
  3. 如权利要求1所述的空间定位装置，其中，
    所述微控制器，还用于通过所述射频芯片从主机接收第一同步信号，并发送给所述中央处理器；
    所述中央处理器根据接收到的第一同步信号向所述第一红外图像传感器发送第二同步信号，当所述第一红外图像传感器接收到所述第二同步信号时，开始采集红外图像；
    所述第一红外图像传感器每采集一帧红外图像，延时一个预设时间间隔后，向所述第二红外图像传感器发送第三同步信号；当所述第二红外图像传感器接收到所述第三同步信号时，开始采集同一帧红外图像。
  4. 如权利要求1所述的空间定位装置，其中，还包括：电源管理芯片、充电芯片、充电接口和内置电池；
    当所述充电接口连接到外部电源时，所述充电芯片从所述充电接口取电，通过所述电源管理芯片为所述微控制器和所述中央处理器供电，同时为所述内置电池充电；
    当所述充电接口未连接到外部电源时，所述充电芯片从所述内置电池取电，通过所述电源管理芯片为所述微控制器和所述中央处理器供电。
  5. 如权利要求4所述的空间定位装置，其中，所述充电接口为Micro USB接口；所述Micro USB接口连接所述中央处理器，
    当所述中央处理器检测连接到所述Micro USB接口的设备为外部存储设备时，所述中央处理器将从所述第一红外图像传感器和所述第二红外图像传感器接收到的图像写入所述外部存储设备，并为所述外部存储设备供电。
  6. 如权利要求4所述的空间定位装置，其中，还包括轻触开关和发光二极管，所述轻触开关和所述发光二极管分别连接所述电源管理芯片；
    所述轻触开关，用于控制所述空间定位装置上电开机和断电关机；
    所述发光二极管，用于指示所述空间定位装置的工作状态。
  7. 如权利要求1所述的空间定位装置，其中，
    所述中央处理器通过SPI总线、I/O口和URAT接口连接微控制器；
    所述中央处理器通过IIC总线和MIPI接口连接所述第一红外图像传感器和所述第二红外图像传感器；
    所述中央处理器通过IIC总线和IO口连接所述电源管理芯片；
    所述微控制器通过SPI总线、IIC总线和I/O口连接所述射频芯片。
  8. 如权利要求1所述的空间定位装置，其中，还包括多制层封装芯片eMCP芯片；所述eMCP芯片连接所述中央处理器，用于存储所述中央处理器正常工作所需要的指令和数据。
  9. 如权利要求1所述的空间定位装置，其中，还包括带电可擦可编程只读存储器EEPROM芯片；
    所述EEPROM芯片连接所述射频芯片，用于存储所述射频芯片的软件信息；
    所述EEPROM芯片还通过IIC总线连接所述微控制器；当所述空间定位装置处于正常工作模式时，所述微控制器控制所述EEPROM芯片开启写保护；当所述空间定位装置处于固件升级模式时，所述微控制器控制所述EEPROM芯片禁用写保护，并通过IIC总线向所述EEPROM芯片写入数据。
  10. 如权利要求1所述的空间定位装置，其中，所述微控制器预留有一个URAT接口和一个JTAG接口，所述中央处理器和所述射频芯片分别预留有一个URAT接口，用于所述空间定位装置的调试。
  11. 如权利要求1所述的空间定位装置，其中，还包括外部晶振；
    所述外部晶振分别连接所述第一红外图像传感器和所述第二红外图像传感器，用于为所述第一红外图像传感器和所述第二红外图像传感器提供时钟。
  12. 一种虚拟现实系统，其中，包括主机、头戴显示设备、若干无线控制器、分别设置在所述头戴显示设备和无线控制器上的红外光源、以及若干如权利要求1-11任一项所述的空间定位装置；
    所述空间定位装置，布置在所述头戴显示设备和无线控制器周围，通过射频芯片无线连接到所述主机；
    所述主机按照预设的频率向每个红外光源和每个空间定位装置发送第一同步信号，控制所有的红外光源和空间定位装置同步工作；
    所述空间定位装置按照预设的频率采集图像，从所述图像中定位出每个红外光源的位置，并提取出每个红外光源的运动轨迹，通过射频芯片发送给所述主机；
    所述主机根据接收到的每个红外光源的运动轨迹对图像进行相应的变换，并将变换后的图像发送给所述头戴显示设备，所述头戴显示设备将变换后的图像显示给用户。
  13. 如权利要求12所述的虚拟现实系统，其中，所述若干无线控制器包括第一无线控制器和第二无线控制器；
    在所述虚拟现实系统启动时，所述主机通过所述空间定位装置分别获取所述头戴显示设备、第一无线控制器、第二无线控制器的初始位置信息；
    在所述虚拟现实系统运行过程中，所述头戴显示设备上的红外光源一直点亮，所述第一无线控制器和第二无线控制器上的红外光源按照预设频率交替明灭；所述空间定位装置按照所述预设频率采集所述空间的红外图像，根据采集的红外图像确定每一个红外光源的位置，并区分所述头戴显示设备、第一无线控制器、第二无线控制器，之后将相应的位置信息发送给所述主机。
  14. 如权利要求12所述的虚拟现实系统，其中，在所述虚拟现实系统运行过程中，所述主机定时向所有红外光源和空间定位装置发送第一同步信号，保持所有红外光源和无线摄像设备的时钟同步。
  15. 如权利要求12所述的虚拟现实系统，其中，在所述虚拟现实系统运行过程中，所述主机向所述第一无线控制器和第二无线控制器上的红外光源发送周期性控制信号，当所述空间定位装置采集奇数帧图像时，控制所述第一无线控制器上的红外光源点亮，当所述空间 定位装置采集偶数帧图像时，控制所述第二无线控制器上的红外光源点亮。
  16. 如权利要求15所述的虚拟现实系统，其中，所述空间定位装置的第一红外图像传感器和第二红外图像传感器将采集到的红外图像发送给中央处理器；
    所述中央处理器将上一帧红外图像与当前帧红外图像进行对比：
    若某一红外光源在两帧图像中均处于点亮状态，则所述中央处理器判断所述红外光源为设置在所述头戴显示设备上的红外光源；
    若某一红外光源仅在奇数帧图像中处于点亮状态，则所述中央处理器判断所述红外光源为设置在所述第一无线控制器上的红外光源；
    若某一红外光源仅在偶数帧图像中处于点亮状态，则所述中央处理器判断所述红外光源为设置在所述第二无线控制器上的红外光源。
  17. 一种虚拟现实系统定位方法，其中，所述方法包括：
    将头戴显示设备、第一无线控制器、第二无线控制器和多个空间定位装置分别无线连接到主机，将所述空间定位装置环绕布置在所述头戴显示设备、第一无线控制器、第二无线控制器所在空间的周围；
    在所述头戴显示设备、第一无线控制器、第二无线控制器上分别设置红外光源；
    通过所述空间定位装置分别获取所述头戴显示设备、第一无线控制器、第二无线控制器的初始位置信息；
    控制所述头戴显示设备上的红外光源一直点亮，控制所述第一无线控制器和第二无线控制器上的红外光源按照预设频率交替明灭，利用所述空间定位装置按照所述预设频率采集所述空间的红外图像，根据采集的红外图像确定每一个红外光源的位置，并区分所述头戴显示设备、第一无线控制器、第二无线控制器，之后将相应的位置信息发送给所述主机。
  18. 如权利要求17所述的虚拟现实系统定位方法，其中，所述方法还包括：
    利用所述主机定时向所有红外光源和空间定位装置发送第一同步信号，保持所有红外光源和空间定位装置的时钟同步。
  19. 如权利要求17所述的虚拟现实系统定位方法，其中，所述控制所述第一无线控制器和第二无线控制器上的红外光源按照预设频率交替明灭，具体包括：
    利用所述主机向所述第一无线控制器和第二无线控制器上的红外光源发送周期性控制信号，当所述空间定位装置采集奇数帧图像时，控制所述第一无线控制器上的红外光源点亮， 当所述空间定位装置采集偶数帧图像时，控制所述第二无线控制器上的红外光源点亮。
  20. 如权利要求19所述的虚拟现实系统定位方法，其中，所述区分所述头戴显示设备、第一无线控制器、第二无线控制器，具体包括：
    将所述第一红外图像传感器和第一红外图像传感器采集上一帧红外图像与当前帧红外图像进行对比：
    若某一红外光源在两帧图像中均处于点亮状态，则判断所述红外光源为设置在所述头戴显示设备上的红外光源；
    若某一红外光源仅在奇数帧图像中处于点亮状态，则判断所述红外光源为设置在所述第一无线控制器上的红外光源；
    若某一红外光源仅在偶数帧图像中处于点亮状态，则判断所述红外光源为设置在所述第二无线控制器上的红外光源。
PCT/CN2017/102757 2016-10-21 2017-09-21 虚拟现实系统及其空间定位装置、定位方法 WO2018072593A1 (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201610922097.0 2016-10-21
CN201610922097.0A CN106569337B (zh) 2016-10-21 2016-10-21 一种虚拟现实系统及其定位方法
CN201611073079.6 2016-11-29
CN201611073079.6A CN106547458A (zh) 2016-11-29 2016-11-29 一种虚拟现实系统及其空间定位装置

Publications (1)

Publication Number Publication Date
WO2018072593A1 true WO2018072593A1 (zh) 2018-04-26

Family

ID=62018922

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/102757 WO2018072593A1 (zh) 2016-10-21 2017-09-21 虚拟现实系统及其空间定位装置、定位方法

Country Status (1)

Country Link
WO (1) WO2018072593A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109655789A (zh) * 2018-12-26 2019-04-19 青岛小鸟看看科技有限公司 一种头戴显示系统及其空间定位追踪装置、方法
CN114020097A (zh) * 2022-01-10 2022-02-08 北京鲸鲮信息系统技术有限公司 信号同步方法、装置、电子设备及存储介质
CN114510149A (zh) * 2022-02-15 2022-05-17 深圳市恒必达电子科技有限公司 一种带力量感应的vr/ar体感装置和力度识别系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104090660A (zh) * 2014-05-30 2014-10-08 贺杰 一种基于立体视觉的动作采集和反馈方法及系统
CN105359063A (zh) * 2013-06-09 2016-02-24 索尼电脑娱乐公司 利用追踪的头戴式显示器
WO2016109232A2 (en) * 2014-12-31 2016-07-07 Sony Computer Entertainment Inc. Signal generation and detector systems and methods for determining positions of fingers of a user
CN106019265A (zh) * 2016-05-27 2016-10-12 北京小鸟看看科技有限公司 一种多目标定位方法和系统
CN106131419A (zh) * 2016-07-21 2016-11-16 北京小鸟看看科技有限公司 一种用于同步多个无线摄像设备的方法和系统及虚拟现实系统
CN106547458A (zh) * 2016-11-29 2017-03-29 北京小鸟看看科技有限公司 一种虚拟现实系统及其空间定位装置
CN106569337A (zh) * 2016-10-21 2017-04-19 北京小鸟看看科技有限公司 一种虚拟现实系统及其定位方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105359063A (zh) * 2013-06-09 2016-02-24 索尼电脑娱乐公司 利用追踪的头戴式显示器
CN104090660A (zh) * 2014-05-30 2014-10-08 贺杰 一种基于立体视觉的动作采集和反馈方法及系统
WO2016109232A2 (en) * 2014-12-31 2016-07-07 Sony Computer Entertainment Inc. Signal generation and detector systems and methods for determining positions of fingers of a user
CN106019265A (zh) * 2016-05-27 2016-10-12 北京小鸟看看科技有限公司 一种多目标定位方法和系统
CN106131419A (zh) * 2016-07-21 2016-11-16 北京小鸟看看科技有限公司 一种用于同步多个无线摄像设备的方法和系统及虚拟现实系统
CN106569337A (zh) * 2016-10-21 2017-04-19 北京小鸟看看科技有限公司 一种虚拟现实系统及其定位方法
CN106547458A (zh) * 2016-11-29 2017-03-29 北京小鸟看看科技有限公司 一种虚拟现实系统及其空间定位装置

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109655789A (zh) * 2018-12-26 2019-04-19 青岛小鸟看看科技有限公司 一种头戴显示系统及其空间定位追踪装置、方法
CN114020097A (zh) * 2022-01-10 2022-02-08 北京鲸鲮信息系统技术有限公司 信号同步方法、装置、电子设备及存储介质
CN114020097B (zh) * 2022-01-10 2022-06-17 北京鲸鲮信息系统技术有限公司 信号同步方法、装置、电子设备及存储介质
CN114510149A (zh) * 2022-02-15 2022-05-17 深圳市恒必达电子科技有限公司 一种带力量感应的vr/ar体感装置和力度识别系统

Similar Documents

Publication Publication Date Title
US11009941B2 (en) Calibration of measurement units in alignment with a skeleton model to control a computer system
WO2018072593A1 (zh) 虚拟现实系统及其空间定位装置、定位方法
CN103529929B (zh) 手势识别系统及可识别手势动作的眼镜
KR101763151B1 (ko) 모바일 디바이스 및 그의 센서 제어 방법
US9472964B2 (en) Interchangeable intelligent accessories for mobile devices
TWI697655B (zh) 深度感測裝置、用於組態其之方法及機器可讀取媒體
WO2017120624A1 (en) Task-oriented feedback using a modular sensing device
WO2016184107A1 (zh) 用于视线焦点定位的可穿戴设备及视线焦点定位方法
US20160165220A1 (en) Display apparatus and method of controlling display apparatus
CN107800959A (zh) 电子设备及其控制方法
US10775946B2 (en) Universal handheld controller of a computer system
CN106569337B (zh) 一种虚拟现实系统及其定位方法
CN103309437B (zh) 基于摄像机的姿势的缓存机制
US20180164983A1 (en) Display system, display apparatus, control method for display apparatus
CN105573697A (zh) 可拆卸的电子设备及其操作方法
CN106131419B (zh) 一种用于同步多个无线摄像设备的方法和系统及虚拟现实系统
CN109613983A (zh) 头戴显示系统中手柄的定位方法、装置和头戴显示系统
CN110622108B (zh) 提供触觉反馈的方法以及执行该方法的电子装置
CN106507083B (zh) 一种无线通信装置、虚拟现实光球及虚拟现实系统
CN104887327B (zh) 一种用于手术导航的触敏式配准工具及其配准方法
US20170276941A1 (en) Head mounted display
CN106375638A (zh) 一种视觉追踪监控与自动照明控制系统
KR20170047603A (ko) 전자 장치의 센서 데이터 획득 방법 및 이를 사용하는 전자 장치
US11216066B2 (en) Display device, learning device, and control method of display device
CN104965588B (zh) 基于nfc可扩展的智能手环或智能手表

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17861363

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17861363

Country of ref document: EP

Kind code of ref document: A1