Disclosure of Invention
The invention provides a split type AR device communication method for solving the technical problems that an existing integrated head-wearing type AR device is complex in structure and inconvenient to wear due to large weight, and the problems can be solved.
In order to solve the technical problems, the invention adopts the following technical scheme to realize:
a split type AR device communication method comprises the following steps that the split type AR device comprises an AR head-mounted display and a controller which are arranged in a split mode, the AR head-mounted display is connected with the controller through a data line, and the split type AR device communication method comprises the following steps:
virtualizing an AR head-mounted display into a number of standard devices;
the method comprises the steps that a controller detects that an external device is inserted into a communication interface, and sends an enumeration request to the external device;
the AR head-mounted display feeds back equipment information to the controller, wherein the equipment information at least comprises information of all standard equipment;
the controller searches a corresponding device driver from a local support device list according to the device information, and loads the driver;
the controller communicates with each standard device.
Further, the communication interface between the AR head mounted display and the controller is a USB interface.
Further, the AR head-mounted display comprises two cameras, each camera is virtualized to be a standard USB device, each standard USB device is communicated with the controller through a USB endpoint, and camera data are transmitted by adopting a USB UVC protocol.
Further, the two cameras are an RGB Camera and a TOF Camera respectively.
Further, the AR head-mounted display includes a plurality of sensors, all sensors are virtualized as a standard USB device, the standard USB device uses a USB endpoint to communicate with the controller, and sensor data is transmitted via a USB hid raw protocol.
Furthermore, the control signal and the sensor data share the same USB endpoint for transmission.
Further, the controller monitors the communication interface and is used for monitoring the plugging and unplugging state of the external equipment.
Furthermore, when each standard device packages the data sent to the controller, a timestamp is added at the same time, and when the controller analyzes the data, the data of all the standard devices are synchronized according to the timestamp.
Further, the device information includes a device descriptor, a configuration descriptor, an interface descriptor, and an endpoint descriptor, and the controller loads a driver according to the device information and communicates with each standard device through a corresponding endpoint and interface.
The invention also provides AR equipment, which comprises an AR head-mounted display and a controller which are arranged in a split mode, wherein the AR head-mounted display is connected with the controller through a data line, the AR head-mounted display at least comprises a display screen, a camera and a sensor, the controller is connected with a battery, and the AR equipment carries out communication according to any one of the split type AR equipment communication methods.
Compared with the prior art, the invention has the advantages and positive effects that: according to the split type AR device, the controller and the battery are separated from the head-mounted display, the weight of the head-mounted display can be greatly reduced due to the absence of the devices, so that the head-mounted display is lighter, the AR head-mounted display is virtualized into a plurality of standard devices in a communication mode of the head-mounted display and the controller, the controller loads corresponding driving programs for the standard devices, the standard devices can directly communicate with each other, the data of the sensor and the camera does not need to be re-packaged, software of the android is completely compatible and supported, the standard devices are used for transmitting different data, the data transmission rate requirement can be met, the real-time performance of data transmission is guaranteed, and blocking is prevented.
Other features and advantages of the present invention will become more apparent from the detailed description of the embodiments of the present invention when taken in conjunction with the accompanying drawings.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
In a first embodiment, the present embodiment provides a split AR device communication method, as shown in fig. 1, the split AR device includes an AR head-mounted display and a controller that are separately arranged, the AR head-mounted display is connected to the controller through a data line, and the split AR device communication method includes the following steps:
virtualizing an AR head-mounted display into a number of standard devices; the head-mounted display in this embodiment is composed of the following main modules: the left display screen and the right display screen are used for displaying pictures for enhancing reality; the two cameras are respectively an RGB camera and a TOF camera, wherein the RGB camera is used for shooting a real-time scene, and the TOF camera is used for measuring the depth of field to locate the spatial position of an object; the system comprises an acceleration sensor, a gyroscope, a gravity sensor, a backlight sensor and a proximity sensor, wherein the acceleration sensor, the gyroscope and the gravity sensor are integrated in an IMU chip and used for measuring the self-position information of a user; backlight sensor and proximity sensor integrate in ALS + PS chip, spaker and mic, be used for input and output speech signal respectively, because the module of AR wear display is more, through virtualizing into a plurality of standard equipment with the module in this scheme, can adopt standard interface to adopt standard agreement and control module to communicate, need not to encapsulate again the data of each module to wear display, can reach real-time transmission and handle user's space geographical position and scene information. And a better augmented reality effect is realized.
Wear-type display adopts traditional communication interface connection communication with the controller in this scheme: the scheme of data transmission between the head-mounted display and the controller needs to be developed in a matching mode at two ends, the controller serves as a host end, and the head-mounted display serves as an equipment end.
After the head-mounted display and the controller are powered on, as shown in fig. 2, when the head-mounted display is connected with the controller by using a data line, the controller detects that an external device is inserted into the communication interface, and sends an enumeration request to the external device;
the AR head-mounted display feeds back equipment information to the controller according to the received enumeration request, wherein the equipment information at least comprises information of all virtual standard equipment;
the controller searches a corresponding device driver from a local support device list according to the device information fed back by the AR head-mounted display, and loads the driver;
because the controller loads the driver for each standard device, the controller can directly communicate with each standard device through the communication interface.
In the split type AR equipment of this embodiment, the battery sets up at the controller end, be connected for its power supply with the controller, because the weight of battery is heavier, through separating controller and battery and head mounted display, the weight of head mounted display will have alleviateed greatly, make head mounted display lighter, thereby it is stronger to user's head oppression sense when current integral type head mounted display wears to have solved, the technical problem of effect is worn in the influence, head mounted display and controller communication mode are through wearing the virtual a plurality of standard equipment of AR head mounted display, the controller is the corresponding driver of each standard equipment loading, both sides can directly communicate, need not to encapsulate sensor and camera's data again, software to android is compatible completely, each standard equipment is used for transmitting different data, can satisfy data transmission rate demand, data transmission real-time has been guaranteed, prevent to block.
The USB interface has wide communication application, supports hot plugging, is convenient to plug and pull, and is a USB interface as a preferred embodiment in the embodiment, wherein the communication interface between the AR head-mounted display and the controller is the USB interface. Wherein, the controller can select a high-pass 845 chip. The head-mounted display acquires corresponding data in real time by using the 6-axis sensor and the camera, transmits the acquired data to the controller through the USB, calculates the spatial geographic position of the user in real time through a 6DOF algorithm by the controller, blends the spatial information of the user into a specific scene, and transmits the spatial information to the head-mounted display through the USB so as to realize augmented reality.
The data volume that the camera needs to transmit in wearing the display is the biggest, and the mode of virtualizing each camera as a standard USB device respectively is adopted in this embodiment, and each standard USB device communicates with the controller through a USB endpoint respectively, and the camera data adopts the USB UVC protocol to transmit.
The AR head-mounted display comprises a plurality of sensors, all the sensors are virtualized to be standard USB equipment, the standard USB equipment adopts a USB endpoint to communicate with the controller, and sensor data are transmitted through a USB hid raw protocol.
In the scheme, the basis for virtualizing each module in the head-mounted display into the standard equipment is according to the type and the amount of data transmitted by the standard equipment, and the USB UVC protocol and the USB hid raw protocol which are adopted for transmission between the standard equipment and the controller are standard protocols of a USB bottom layer, and when the USB communication interface is used for transmitting data, the data of the sensor and the camera do not need to be re-packaged, and the android software is completely compatible and supported.
The head-mounted display and the controller are communicated through the USB communication interface, and the head-mounted display and the controller can be compatible with a traditional integrated head-mounted display charger and can be charged through the traditional charger.
That is, in this embodiment, the modules in the head-mounted display are virtualized to be 3 standard devices, and 3 endpoint (end points) of the USB are used between the corresponding head-mounted display and the controller to output data, and each standard device uses one endpoint to transmit data, where 2 endpoint is used to transmit the camera data of the UVC class, and 1 endpoint is used to transmit the sensor data of the hid raw class.
For example, three standard devices respectively use three endpoints EP0, EP1, EP2 of the USB to transmit data:
EP0: the sensor data is transmitted, 6DOF and control commands are included, and the data volume is predicted to be 48kbs-22Mbps.
EP 1: for transmitting RGB Camera data, 1080/30fps or 1080/60fps preview, the image format is MJPEG, and the data amount is 300-995Mbs.
EP 2: for transmitting 60fps TOF Camera data.
Since the data amount of the control signal is small, the control signal and the sensor data share the same USB endpoint for transmission in the present embodiment. And an end point does not need to be occupied for the control signal, so that the channel is saved, and the occupation of the bandwidth is reduced.
After the controller is powered on, the method also comprises the step of monitoring the communication interface in real time, and the step is used for monitoring the plugging state of the external equipment. If the external device insertion is not detected, monitoring is continued, if the external device insertion is detected, a command is sent to the external device to establish connection with the head-mounted display, and corresponding drivers (UVC and hid raw) are loaded according to enumeration of the USB. If the head-mounted display responds normally, usb communication will be performed between the controller and the head-mounted display, and the display will send sensor and camera data to the controller. The controller performs normal data processing.
In order to ensure the synchronism of the sensor data and the camera data, when the data sent to the controller are packaged by each standard device, a timestamp is added, and when the controller analyzes the data, the data of all the standard devices are synchronized according to the timestamp.
The AR head-mounted display feeds back device information to the controller, the device information comprises a device descriptor, a configuration descriptor, an interface descriptor and an endpoint descriptor, the controller loads a driver according to the configuration descriptor in the device information, and the controller accords with the endpoint descriptor according to the interface descriptor and communicates with each standard device through corresponding endpoints and interfaces.
The second embodiment provides an AR device, as shown in fig. 1, which includes an AR head-mounted display and a controller that are separately arranged, where the AR head-mounted display is connected to the controller through a data line, and the AR head-mounted display at least includes a display screen, a camera and a sensor, where the display screen includes a left display screen and a right display screen, and is used to display augmented reality pictures; the two cameras are respectively an RGB camera and a TOF camera, wherein the RGB camera is used for shooting a real-time scene, and the TOF camera is used for measuring the depth of field to locate the spatial position of an object; the number of the sensors is 6, and the sensors are respectively an acceleration sensor, a gyroscope, a gravity sensor, a backlight sensor and a proximity sensor and are used for measuring the position information of the user; the head-mounted display further comprises a spreader and a mic which are used for inputting and outputting voice signals respectively, the battery and the controller are arranged at the same end, the battery and the controller are connected and used for supplying power to the battery, the weight of the head-mounted display is greatly reduced by separating the controller and the battery from the head-mounted display, the head-mounted display is light and convenient to use, the AR device communicates according to the split type AR device communication method recorded in the first embodiment, the communication method can be recorded in the first embodiment specifically, and the description is omitted herein.
It is to be understood that the above description is not intended to limit the present invention, and the present invention is not limited to the above examples, and those skilled in the art may make modifications, alterations, additions or substitutions within the spirit and scope of the present invention.