WO2020073334A1 - 扩展内容显示方法、装置、系统及存储介质 - Google Patents

扩展内容显示方法、装置、系统及存储介质 Download PDF

Info

Publication number
WO2020073334A1
WO2020073334A1 PCT/CN2018/110138 CN2018110138W WO2020073334A1 WO 2020073334 A1 WO2020073334 A1 WO 2020073334A1 CN 2018110138 W CN2018110138 W CN 2018110138W WO 2020073334 A1 WO2020073334 A1 WO 2020073334A1
Authority
WO
WIPO (PCT)
Prior art keywords
display device
screen
extended
virtual
see
Prior art date
Application number
PCT/CN2018/110138
Other languages
English (en)
French (fr)
Inventor
周梁
冯智勇
季佳松
Original Assignee
北京小米移动软件有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京小米移动软件有限公司 filed Critical 北京小米移动软件有限公司
Priority to PCT/CN2018/110138 priority Critical patent/WO2020073334A1/zh
Priority to CN201880001631.1A priority patent/CN109496293B/zh
Publication of WO2020073334A1 publication Critical patent/WO2020073334A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display

Definitions

  • the present disclosure relates to the field of display control technology, and in particular, to an extended content display method, device, system, and storage medium.
  • the multi-screen expansion technology can realize the display of content on multiple different screens for display, thereby providing users with a richer interactive experience.
  • the terminal may be connected to the physical expansion screen through the video output interface, so that the extended display content is dropped into the physical expansion screen for display.
  • the terminal can be connected to multiple different physical expansion screens through multiple different video output interfaces to achieve multi-screen expansion functions.
  • An embodiment of the present disclosure provides an extended content display method, device, system, and storage medium.
  • the technical solution is as follows:
  • an extended content display method including:
  • the visible virtual expansion screen refers to the virtual expansion screen within the FOV of the see-through near-eye display device
  • the acquiring the extended display content corresponding to the visible virtual extended screen includes:
  • the content acquisition request further includes position information of a visible area of the visible virtual expansion screen, and the visible area of the visible virtual expansion screen refers to the visible virtual expansion screen being displayed on the see-through near-eye display device The display area within the FOV;
  • the receiving extended display content corresponding to the visible virtual extended screen sent by the main display device includes:
  • the method further includes:
  • the visible virtual The visible area of the extended screen refers to the display area of the visible virtual extended screen within the FOV of the see-through near-eye display device;
  • the sending the extended display content to the see-through near-eye display device includes:
  • the method further includes:
  • the acquiring the posture information of the fluoroscopic near-eye display device includes:
  • the posture information of the see-through near-eye display device is determined according to the sensor data.
  • the posture information of the see-through near-eye display device includes: distance offset information of the current position of the see-through near-eye display device relative to a reference position, and the current orientation of the see-through near-eye display device relative to Information about the angle offset of the reference orientation.
  • an extended content display method which is applied to an extended content display system, the system including: a near-eye display device, a computing component, and a main display device; the method includes:
  • the see-through near-eye display device creates n virtual expansion screens, where n is a positive integer
  • the computing component determines the visibility among the n virtual expansion screens based on the posture information of the fluoroscopic near-eye display device, the FOV of the fluoroscopic near-eye display device, and the posture information of the n virtual expansion screens A virtual extended screen; wherein, the visible virtual extended screen refers to a virtual extended screen within the FOV of the see-through near-eye display device;
  • the computing component obtains the extended display content corresponding to the visible virtual extended screen from the main display device;
  • the computing component sends the extended display content to the see-through near-eye display device
  • the see-through near-eye display device displays the extended display content in the visible virtual extended screen.
  • the computing component acquiring the extended display content corresponding to the visible virtual extended screen from the main display device includes:
  • the computing component sends a content acquisition request to the main display device, where the content acquisition request includes identification information of the visible virtual expansion screen;
  • the main display device According to the content acquisition request, the main display device generates extended display content corresponding to the visible virtual extended screen;
  • the main display device sends the extended display content to the computing component.
  • the content acquisition request further includes position information of a visible area of the visible virtual expansion screen, and the visible area of the visible virtual expansion screen refers to the visible virtual expansion screen being displayed on the see-through near-eye display device The display area within the FOV;
  • the main display device According to the content acquisition request, the main display device generates extended display content corresponding to the visible virtual extended screen, including:
  • the main display device generates extended display content corresponding to the visible area of the visible virtual extended screen according to the position information of the visible area of the visible virtual extended screen.
  • the method further includes:
  • the computing component extracts the extended display content corresponding to the visible area of the visible virtual extended screen from the extended display content corresponding to the visible virtual extended screen according to the position information of the visible area of the visible virtual extended screen; wherein, The visible area of the visible virtual extended screen refers to the display area of the visible virtual extended screen in the FOV of the see-through near-eye display device;
  • the computing component sending the extended display content to the see-through near-eye display device includes:
  • the computing component sends the expanded display content corresponding to the visible area of the visible virtual expanded screen to the see-through near-eye display device.
  • the method further includes:
  • the computing component obtains a pose adjustment parameter corresponding to the i-th virtual expansion screen among the n virtual expansion screens, where i is a positive integer less than or equal to n;
  • the calculation component determines the adjusted posture information of the i-th virtual expansion screen according to the posture information of the i-th virtual expansion screen and the posture adjustment parameters of the i-th virtual expansion screen;
  • the computing component sends the adjusted posture information of the i-th virtual expansion screen to the see-through near-eye display device;
  • the see-through near-eye display device displays the i-th virtual expansion screen according to the adjusted posture information of the i-th virtual expansion screen.
  • the method further includes:
  • the perspective near-eye display device collects sensor data through a pose sensor, and sends the sensor data to the computing component;
  • the calculation component determines the pose information of the see-through near-eye display device based on the sensor data.
  • the method further includes:
  • the main display device generates a main display content corresponding to the main screen, and the main screen refers to a screen of the main display device;
  • the main display device displays the main display content on the main screen.
  • the computing component is integrated into the see-through near-eye display device
  • the computing component is integrated in the main display device
  • the computing component is a device independent of the see-through near-eye display device and the main display device.
  • an extended content display device including:
  • a posture acquisition module configured to obtain posture information of the see-through near-eye display device and posture information of n virtual expansion screens created by the see-through near-eye display device, where n is a positive integer;
  • the visible screen determination module is configured to determine the n virtual extensions based on the posture information of the see-through near-eye display device, the FOV of the see-through near-eye display device, and the posture information of the n virtual extension screens A visible virtual expansion screen in the screen; wherein, the visible virtual expansion screen refers to a virtual expansion screen within the FOV of the see-through near-eye display device;
  • the content acquisition module is configured to acquire extended display content corresponding to the visible virtual extended screen
  • the content sending module is configured to send the extended display content to the see-through near-eye display device, so that the see-through near-eye display device displays the extended display content in the visible virtual extended screen.
  • the content acquisition module includes:
  • the request sending unit is configured to send a content acquisition request to the main display device, where the content acquisition request includes identification information of the visible virtual extended screen;
  • the content receiving unit is configured to receive extended display content corresponding to the visible virtual extended screen sent by the main display device.
  • the content acquisition request further includes position information of a visible area of the visible virtual expansion screen, and the visible area of the visible virtual expansion screen refers to the visible virtual expansion screen being displayed on the see-through near-eye display device The display area within the FOV;
  • the content receiving unit is configured to receive extended display content corresponding to the visible area of the visible virtual extended screen sent by the main display device.
  • the device further includes:
  • the content extraction module is configured to extract the extended display content corresponding to the visible area of the visible virtual extended screen from the extended display content corresponding to the visible virtual extended screen according to the position information of the visible area of the visible virtual extended screen
  • the visible area of the visible virtual extended screen refers to the display area of the visible virtual extended screen within the FOV of the see-through near-eye display device
  • the content sending module is configured to send extended display content corresponding to the visible area of the visible virtual extended screen to the see-through near-eye display device.
  • the device further includes:
  • the parameter acquisition module is configured to acquire a pose adjustment parameter corresponding to the i-th virtual expansion screen among the n virtual expansion screens, where i is a positive integer less than or equal to n;
  • the pose determination module is configured to determine the adjusted pose of the i-th virtual expansion screen according to the pose information of the i-th virtual expansion screen and the posture adjustment parameters of the i-th virtual expansion screen information;
  • a posture sending module configured to send the adjusted posture information of the i-th virtual extended screen to the fluoroscopic near-eye display device, so that the fluoroscopic near-eye display device adjusts the i-th virtual extended screen Posture.
  • the pose acquisition module includes:
  • a data receiving unit configured to receive sensor data collected by the fluoroscopic near-eye display device through a pose sensor
  • the pose determination unit is configured to determine the pose information of the see-through near-eye display device according to the sensor data.
  • the posture information of the see-through near-eye display device includes: distance offset information of the current position of the see-through near-eye display device relative to a reference position, and the current orientation of the see-through near-eye display device relative to Information about the angle offset of the reference orientation.
  • an extended content display system including: a near-eye displayable device, a computing component, and a main display device;
  • the see-through near-eye display device is configured to create n virtual expansion screens, where n is a positive integer;
  • the computing component is configured to determine the n virtual extensions based on the posture information of the see-through near-eye display device, the FOV of the see-through near-eye display device, and the posture information of the n virtual expansion screens A visible virtual expansion screen in the screen; wherein, the visible virtual expansion screen refers to a virtual expansion screen within the FOV of the see-through near-eye display device;
  • the computing component is further configured to obtain the extended display content corresponding to the visible virtual extended screen from the main display device;
  • the computing component is further configured to send the extended display content to the see-through near-eye display device;
  • the see-through near-eye display device is configured to display the extended display content in the visible virtual extended screen.
  • an extended content display device including:
  • a memory for storing executable instructions of the processor
  • the processor is configured to:
  • the visible virtual extended screen refers to a virtual extended screen within the FOV of the see-through near-eye display device
  • a non-transitory computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the steps of the method according to the first aspect.
  • the number of virtual expansion screens is not subject to the number of video output interfaces provided by the main display device Restrictions, more expansibility, and more convenient than physical expansion screen.
  • Fig. 1 is a schematic diagram of an implementation environment according to an exemplary embodiment
  • Fig. 2 is a flow chart showing a method for displaying extended content according to an exemplary embodiment
  • FIG. 3 exemplarily shows a schematic diagram of a screen with visible virtual expansion
  • Fig. 4 is a flow chart showing a method for displaying extended content according to another exemplary embodiment
  • Fig. 5 is a flow chart showing a method for displaying extended content according to another exemplary embodiment
  • Fig. 6 is a schematic diagram showing an extended content display process according to an exemplary embodiment
  • Fig. 7 is a block diagram of an extended content display device according to an exemplary embodiment
  • Fig. 8 is a block diagram showing a device for expanding content according to another exemplary embodiment
  • Fig. 9 is a block diagram of a device according to an exemplary embodiment.
  • Fig. 1 is a schematic diagram of an implementation environment according to an exemplary embodiment.
  • the implementation environment may include a near-eye display device 101, a computing component 102, and a main display device 103.
  • a see-through near-eye display device 101 refers to a wearable optical display device that can create a virtual image within the field of view of the user's eyes and allow the user to see a real scene.
  • the see-through near-eye display device 101 is an AR (Augumented Reality, augmented reality) display device or an MR (Mixed Reality, mixed reality) display device.
  • the see-through near-eye display device 101 may be a head-mounted display device such as AR glasses, AR helmets, MR glasses, MR helmets.
  • the see-through near-eye display device 101 can create a virtual expansion screen and display related content in the virtual expansion screen.
  • the computing component 102 refers to a device with data computing and processing capabilities.
  • the computing component 102 can perform corresponding processing work on the data transmitted from the see-through near-eye display device 101 and the main display device 103.
  • the computing component 102 is integrated into the see-through near-eye display device 101; or, the computing component 102 is integrated into the main display device 103; or, the computing component 102 is independent of the see-through near-eye display device 101 and the main display device 103 installation.
  • the main display device 103 refers to a display device that provides a main screen.
  • the main display device 103 may be an electronic device such as a PC (Personal Computer), mobile phone, tablet computer, or the like.
  • the main display device 103 is used to generate respective display contents of the main screen and the virtual expansion screen, and place the above-mentioned display contents on the response screen for display.
  • a communication connection can be established between the see-through near-eye display device 101 and the main display device 103 in a wired or wireless manner .
  • a communication connection can be established between the computing component 102 and the see-through near-eye display device 101 by wired or wireless means, and the computing component 102 and The main display device 103 may also establish a communication connection in a wired or wireless manner.
  • the above-mentioned wired mode may be a wired connection based on a video output interface, such as an HDMI (High Definition Multimedia Interface), VGA (Video Graphics Array) interface, etc.
  • the above wireless mode may be a connection established based on a wireless network, such as a wireless local area network.
  • Fig. 2 is a flow chart showing a method for displaying extended content according to an exemplary embodiment. This method can be applied to the implementation environment shown in FIG. 1. The method may include the following steps.
  • n virtual expansion screens can be created through the near-eye display device, where n is a positive integer.
  • the virtual expansion screen refers to a non-physical expansion screen displayed by a see-through near-eye display device.
  • the user can see the virtual extended screen only by using a see-through near-eye display device. For example, after wearing the AR glasses, the user can see the virtual extended screen displayed by the AR glasses.
  • the number of virtual expansion screens created by the see-through near-eye display device is not limited.
  • the see-through near-eye display device can create one virtual expansion screen or multiple virtual expansion screens.
  • the see-through near-eye display device can create a virtual expansion screen according to user needs. For example, after receiving the creation instruction, the see-through near-eye display device creates n virtual expansion screens according to the creation instruction.
  • the creation instruction may be triggered by the user and sent from the main display device to the see-through near-eye display device.
  • the number of virtual expansion screens created by the see-through near-eye display device can be specified by the user and carried in the creation instruction, or it can be a preset default value.
  • the initial posture of the virtual expansion screen can be specified by the user and carried in the creation instruction, or can be determined by the computing component and sent to the see-through near-eye display device, or the preset default initial posture.
  • the computing component determines the initial posture information of the virtual extended screen, and sends the initial posture information to the fluoroscopic near-eye display device, and the fluoroscopic near-eye display device creates the virtual extended screen according to the initial posture information.
  • the computing component may determine the initial posture information of the virtual expansion screen according to the user's historical habits or pre-configured information.
  • the posture of the virtual expansion screen is fixed and not adjustable.
  • the posture of the virtual expansion screen is non-fixed and adjustable, for example, the position of the virtual expansion screen is movable, and the orientation of the virtual expansion screen is adjustable, so that the virtual expansion screen is more flexible Sex, to better meet user needs.
  • the computing component determines the visible virtual expansion screens among the n virtual expansion screens based on the posture information of the fluoroscopic near-eye display device, the FOV of the fluoroscopic near-eye display device and the posture information of the n virtual expansion screens.
  • the visible virtual extended screen refers to the virtual extended screen within the FOV that can see through the near-eye display device. That is, the visible virtual extended screen refers to the virtual extended screen that the user can see. Due to the limited FOV of the see-through near-eye display device, among the virtual expansion screens created by the see-through near-eye display device, some may be within the above FOV, and some may not be within the above FOV. Users can view different virtual expansion screens by adjusting the position and orientation of the near-eye display device.
  • the virtual expansion screen when all or part of a virtual expansion screen is within the FOV of the see-through near-eye display device, the virtual expansion screen is a visible virtual expansion screen; when all areas of a certain virtual expansion screen are not in the near-eye display When the device is in the FOV, the virtual expansion screen is not a visible virtual expansion screen.
  • the posture information of the see-through near-eye display device is used to indicate the position and posture of the see-through near-eye display device in space.
  • the posture information of the see-through near-eye display device may be represented by relative posture information of the see-through near-eye display device with respect to the reference posture information, or by the absolute posture information of the see-through near-eye display device in space.
  • the above-mentioned reference posture information may be set in advance, for example, the posture information of the fluoroscopic near-eye display device at a certain moment is determined as the reference posture information.
  • the posture information of the see-through near-eye display device includes: distance offset information of the current position of the see-through near-eye display device relative to the reference position, and the angular offset of the current orientation of the see-through near-eye display device relative to the reference orientation information.
  • the posture information of the see-through near-eye display device can be represented by 6 parameters; wherein the distance offset information includes parameters in three dimensions, such as (x, y, z); the angle offset information also includes three Dimensional parameters, such as (pitch, yaw, roll).
  • the posture information of the see-through near-eye display device may be provided to the computing component by the see-through near-eye display device.
  • a fluoroscopic near-eye display device can obtain the above-mentioned posture information through a posture sensor.
  • the pose sensor can be a visual sensor or a motion sensor, including an acceleration sensor, a gyroscope, and so on.
  • the FOV of the see-through near-eye display device is the angle of view of the see-through near-eye display device, and the FOV of the see-through near-eye display device is determined by its own hardware performance.
  • the FOV of the see-through near-eye display device may be pre-stored in the computing component, or may be sent to the computing component by the see-through near-eye display device.
  • the posture information of the virtual extended screen is used to indicate the position and posture of the virtual extended screen.
  • the posture information of the virtual extended screen can be represented by the relative posture information of the virtual extended screen relative to the reference posture information.
  • the above-mentioned reference pose information can be set in advance, for example, the initial pose information of the virtual expansion screen is determined as the reference pose information.
  • the reference posture information of the virtual extended screen introduced here may be the same as or different from the reference posture information of the see-through near-eye display device described above, which is not limited in the embodiments of the present disclosure.
  • the posture information of the virtual extended screen includes: distance offset information of the current position of the virtual extended screen relative to the reference position, and angle offset information of the current orientation of the virtual extended screen relative to the reference orientation.
  • the reference position and reference orientation of the virtual expansion screen described here may be the same as or different from the reference position and reference orientation of the see-through near-eye display device described above, which is not limited in the embodiments of the present disclosure.
  • the posture information of the virtual extended screen can also be represented by 6 parameters; the distance offset information includes three-dimensional parameters, such as (x, y, z); angle The offset information also includes parameters in three dimensions, such as (pitch, yaw, roll).
  • the calculation component can determine the FOV of the near-eye display device based on the posture information of the near-eye display device and the FOV of the near-eye display device, and then combine the posture information of the n virtual expansion screens to expand the virtual FOV
  • the screen is determined to be a visible virtual extended screen.
  • FIG. 3 it exemplarily shows a schematic diagram of a visible virtual expansion screen.
  • the see-through near-eye display device 101 creates two virtual expansion screens, namely expansion screen 1 and expansion screen 2.
  • the expansion screen 1 is within the FOV that can see through the near-eye display device 101, so the expansion screen 1 is a visible virtual expansion screen.
  • step 203 the computing component obtains the extended display content corresponding to the visible virtual extended screen from the main display device.
  • the main display device is used to render and generate display content.
  • the main display device may be equipped with a graphics card, which is used for rendering and generating display content.
  • the display content generated by the main display device includes the main display content corresponding to the main screen and the extended display content corresponding to the visible virtual extended screen.
  • the main screen refers to the screen of the main display device, which is a physical screen, not a virtual screen.
  • the main display device stitches and renders the main display content and the extended display content to obtain the spliced display content, and then extracts the main display content and the extended display content from the spliced display content, respectively.
  • the main display device may render and generate the extended display content corresponding to each of the n virtual extended screens, and send the extended display content corresponding to the n virtual extended screens to the computing component, and the computing component selects the visible virtual The expanded display content corresponding to the expanded screen.
  • the computing component may send the identification information of the visible virtual extended screen to the main display device, and according to the identification information of the visible virtual extended screen, the main display device only renders and generates the visible virtual extended screen The corresponding extended display content, and then send the extended display content corresponding to the visible virtual extended screen to the computing component.
  • This method can save the processing overhead of the main display device.
  • the main display device may send extended display content to the computing component through the connection established with the computing component.
  • the main display device can send extended display content to the computing component through the video output interface.
  • step 204 the computing component sends the extended display content corresponding to the visible virtual extended screen to the see-through near-eye display device.
  • the computing component may send extended display content to the see-through near-eye display device through the connection established with the see-through near-eye display device.
  • the computing component may send the extended display content to the see-through near-eye display device through the video output interface.
  • step 205 the see-through near-eye display device displays the expanded display content in the visible virtual expanded screen.
  • the see-through near-eye display device After receiving the extended display content corresponding to the visible virtual extended screen, the see-through near-eye display device displays the extended display content in the visible virtual extended screen.
  • the near-eye display device that can see through the expanded display content received will be displayed in this visible virtual expansion screen; when the number of visible virtual expansion screens is multiple, the The perspective near-eye display device respectively displays the expanded display content corresponding to each visible virtual expansion screen in the corresponding visible virtual expansion screen.
  • the number of visible virtual expansion screens is two, and a near-eye display device can be used to display a document editing interface of an office software in one visible virtual expansion screen, and a chat session interface of a social application in another visible virtual expansion screen.
  • the main display device can also display the main display content on the main screen.
  • a virtual expansion screen is created through a see-through near-eye display device, and the expanded display content is displayed in the virtual expansion screen; compared with the physical expansion screen, the number of virtual expansion screen It will not be limited by the number of video output interfaces possessed by the main display device, and is more expandable and more convenient than the physical expansion screen.
  • Fig. 4 is a flow chart showing a method for displaying extended content according to another exemplary embodiment. This method can be applied to the implementation environment shown in FIG. 1. The method may include the following steps.
  • n virtual expansion screens can be created through the near-eye display device, where n is a positive integer.
  • step 402 the see-through near-eye display device collects sensor data through a pose sensor.
  • step 403 the see-through near-eye display device sends the aforementioned sensor data to the computing component.
  • step 404 the computing component determines posture information that can see through the near-eye display device based on the sensor data.
  • step 405 the computing component determines the visible virtual expansion screens among the n virtual expansion screens based on the posture information of the fluoroscopic near-eye display device, the FOV of the fluoroscopic near-eye display device, and the posture information of the n virtual expansion screens.
  • step 406 the computing component sends a content acquisition request to the main display device.
  • the content acquisition request includes identification information of the visible virtual extended screen and location information of the visible area of the visible virtual extended screen.
  • the content acquisition request is used to request to acquire extended display content.
  • the identification information of the visible virtual extended screen is used to uniquely indicate the visible virtual extended screen. Different virtual expansion screens have different identification information.
  • the content acquisition request includes identification information of the visible virtual extended screen, so that the main display device generates extended display content corresponding to the visible virtual extended screen according to the identification information of the visible virtual extended screen.
  • the content acquisition request further includes position information of the visible area of the visible virtual extended screen
  • the visible area of the visible virtual extended screen refers to the display area of the visible virtual extended screen in the FOV that can see through the near-eye display device.
  • the position information of the visible area is used to indicate the position of the visible area in the visible virtual extended screen.
  • the position information of the visible area may include the coordinates of a vertex of the visible area, and the length and width of the visible area.
  • the visible area of the visible virtual expansion screen is the entire screen area; when part of the screen area of the visible virtual expansion screen is in the FOV of the near-eye display device When the other part of the screen area is outside the FOV that can see through the near-eye display device, the visible area of the visible virtual expansion screen is a part of the screen area.
  • step 407 the main display device generates extended display content corresponding to the visible area of the visible virtual extended screen according to the position information of the visible area of the visible virtual extended screen.
  • the main display device only renders the extended display content corresponding to the visible area of the visible virtual extended screen, other non-visible virtual extended screens and the extended display content corresponding to the non-visible area of the visible virtual extended screen, the main display device No rendering is generated, thereby saving the processing overhead of the main display device.
  • step 408 the main display device sends the extended display content corresponding to the visible area of the visible virtual extended screen to the computing component.
  • step 409 the computing component sends the expanded display content corresponding to the visible area of the visible virtual expanded screen to the see-through near-eye display device.
  • step 410 the see-through near-eye display device displays the extended display content corresponding to the visible area in the visible virtual extended screen.
  • the main display device can also generate main display content corresponding to the main screen and display the main display content on the main screen.
  • the location information of the visible area of the visible virtual extended screen is sent to the main display device through the computing component, and the main display device can only render the visible area corresponding to the visible virtual extended screen.
  • the main display device does not have to be generated without rendering, thereby saving the processing overhead of the main display device.
  • Fig. 5 is a flow chart showing a method for displaying extended content according to another exemplary embodiment. This method can be applied to the implementation environment shown in FIG. 1. The method may include the following steps.
  • n virtual expansion screens can be created through the near-eye display device, where n is a positive integer.
  • step 502 the see-through near-eye display device collects sensor data through a pose sensor.
  • step 503 the see-through near-eye display device sends the aforementioned sensor data to the computing component.
  • step 504 the computing component determines posture information that can see through the near-eye display device based on the sensor data.
  • step 505 the computing component determines the visible virtual expansion screens among the n virtual expansion screens based on the posture information of the fluoroscopic near-eye display device, the FOV of the fluoroscopic near-eye display device and the posture information of the n virtual expansion screens.
  • step 506 the computing component sends a content acquisition request to the main display device, and the content acquisition request includes identification information of the visible virtual expansion screen.
  • step 507 the main display device generates extended display content corresponding to the visible virtual extended screen according to the content acquisition request.
  • step 508 the main display device sends the extended display content corresponding to the visible virtual extended screen to the computing component.
  • step 509 the computing component extracts the expanded display content corresponding to the visible area of the visible virtual expansion screen from the expanded display content corresponding to the visible virtual expansion screen according to the position information of the visible area of the visible virtual expansion screen.
  • the main display device sends the extended display content of the entire screen area of the visible virtual extended screen to the computing component, and the computing component extracts the extended display content corresponding to the visible area from the computing component.
  • the FOV of the see-through near-eye display device changes, so the visible area changes, so that the extended display content displayed needs to be adjusted accordingly.
  • the visible area changes the extended display content corresponding to the changed visible area has been rendered and sent to the computing component, so the response is more timely and smooth.
  • step 510 the computing component sends the extended display content corresponding to the visible area of the visible virtual extended screen to the see-through near-eye display device.
  • step 511 the see-through near-eye display device displays the extended display content corresponding to the visible area in the visible virtual extended screen.
  • the main display device can also generate main display content corresponding to the main screen and display the main display content on the main screen.
  • the extended display content corresponding to the visible virtual expansion screen is sent to the computing component through the main display device, and the computing component according to the location information of the visible area of the visible virtual expansion screen, from In the extended display content corresponding to the visible virtual extended screen, the extended display content corresponding to the visible area of the visible virtual extended screen is extracted, so that when the visible area changes, the response is more timely and smooth.
  • Fig. 6 is a schematic diagram showing an extended content display process according to an exemplary embodiment.
  • the see-through near-eye display device 101 creates a virtual expansion screen on the left and right sides of the main display device 103, namely expansion screen 1 and expansion screen 2.
  • the expansion screen 1 is within the FOV that can see through the near-eye display device 101, so the expansion screen 1 is a visible virtual expansion screen.
  • the graphics card rendering of the main display device 103 generates the extended display content corresponding to the extended screen 1, and the rendering generates the main display content corresponding to the main screen of the main display device 103.
  • the main display device 103 sends the extended display content corresponding to the extended screen 1 to the computing component 102, and displays the main display content on the main screen.
  • the computing component 102 sends the expanded display content corresponding to the expanded screen 1 to the see-through near-eye display device 101, and the see-through near-eye display device 101 displays the above-mentioned expanded display content on the expanded screen 1.
  • the expansion screen 2 is a visible virtual expansion screen.
  • the shaded area on the left is the visible area of the extended screen 2.
  • the graphics card of the main display device 103 renders and generates the extended display content corresponding to the extended screen 2, and renders and generates the main display content corresponding to the main screen.
  • the main display device 103 sends the extended display content corresponding to the extended screen 2 to the computing component 102, and displays the main display content on the main screen.
  • the computing component 102 extracts the extended display content corresponding to the visible area of the extended screen 2 from the extended display content corresponding to the extended screen 2 according to the position information of the visible area of the extended screen 2, and sends it to the fluoroscopic near-eye display device 101,
  • the see-through near-eye display device 101 displays the received expanded display content in the aforementioned visible area of the expanded screen 2.
  • the main display device 103 is a notebook computer
  • the see-through near-eye display device 101 is AR glasses.
  • Users wear AR glasses and use AR glasses to create a virtual expansion screen on the left and right sides of the laptop.
  • the desktop interface can be displayed on the main screen of the laptop, and the user can drag a document from the main screen to the virtual extended screen on the left, and drag the chat interface of the instant messaging application from the main screen to the virtual extended screen on the right.
  • the user can complete the dragging with a mouse, AR handle, or gesture.
  • a function for adjusting the posture of the virtual extended screen is also provided.
  • the adjustment process may include the following steps:
  • the calculation component obtains the pose adjustment parameters corresponding to the i-th virtual expansion screen among the n virtual expansion screens, i being a positive integer less than or equal to n.
  • the calculation component determines the adjusted posture information of the i-th virtual expansion screen according to the posture information of the i-th virtual expansion screen and the posture adjustment parameters of the i-th virtual expansion screen.
  • the computing component sends the adjusted posture information of the i-th virtual expansion screen to the see-through near-eye display device.
  • the perceptible near-eye display device can display the i-th virtual expansion screen according to the adjusted posture information of the i-th virtual expansion screen.
  • the user can perform gesture adjustment operations corresponding to the virtual expansion screen through input components such as gestures, AR handles, or mice.
  • the calculation component can obtain the posture adjustment parameters according to the above adjustment operation.
  • only the posture adjustment for the i-th virtual expansion screen is taken as an example for description and description, and the posture adjustment process of any virtual expansion screen may refer to the method flow provided in this embodiment.
  • the posture adjustment parameters include: position adjustment parameters and / or posture adjustment parameters.
  • the position adjustment parameters are used to adjust the position of the virtual expansion screen, and the attitude adjustment parameters are used to adjust the attitude of the virtual expansion screen.
  • the technical solutions of the present disclosure are described and described only from the perspective of interaction between the see-through near-eye display device, the computing component, and the main display device.
  • the steps related to the see-through near-eye display device in the above embodiments can be implemented separately as an extended content display method on the side of the see-through near-eye display device.
  • the steps of the display device can be individually implemented as an extended content display method on the side of the main display device.
  • Fig. 7 is a block diagram of an extended content display device according to an exemplary embodiment.
  • the device has a function to realize the above method example, and the function may be realized by hardware, or may be realized by hardware executing corresponding software.
  • the apparatus may include: a pose acquisition module 710, a visible screen determination module 720, a content acquisition module 730, and a content transmission module 740.
  • the posture acquisition module 710 is configured to acquire posture information of the augmented reality perspective near-eye display device and posture information of n virtual expansion screens created by the perspective near-eye display device, where n is a positive integer.
  • the visible screen determination module 720 is configured to determine the visible virtual expansion screens among the n virtual expansion screens based on the posture information of the fluoroscopic near-eye display device, the FOV of the fluoroscopic near-eye display device, and the posture information of the n virtual expansion screens ; Among them, the visible virtual expansion screen refers to the virtual expansion screen within the FOV that can see through the near-eye display device.
  • the content obtaining module 730 is configured to obtain extended display content corresponding to the visible virtual extended screen.
  • the content sending module 740 is configured to send the extended display content to the see-through near-eye display device, so that the see-through near-eye display device displays the extended display content in the visible virtual extended screen.
  • a virtual expansion screen is created through a see-through near-eye display device, and the expanded display content is displayed in the virtual expansion screen; compared with the physical expansion screen, the number of virtual expansion screens It will not be limited by the number of video output interfaces possessed by the main display device, and is more expandable and more convenient than the physical expansion screen.
  • the content acquisition module 730 includes a request sending unit 731 and a content receiving unit 732.
  • the request sending unit 731 is configured to send a content acquisition request to the main display device, where the content acquisition request includes identification information of the visible virtual expansion screen.
  • the content receiving unit 732 is configured to receive extended display content corresponding to the visible virtual extended screen sent by the main display device.
  • the above content acquisition request further includes location information of the visible area of the visible virtual extended screen.
  • the visible area of the visible virtual extended screen refers to the display area of the visible virtual extended screen within the FOV of the near-eye display device that can see through.
  • the content receiving unit 732 is configured to receive the extended display content corresponding to the visible area of the visible virtual extended screen sent by the main display device.
  • the above device further includes: a content extraction module 750.
  • the content extraction module 750 is configured to extract the expanded display content corresponding to the visible area of the visible virtual expansion screen from the expanded display content corresponding to the visible virtual expansion screen according to the position information of the visible area of the visible virtual expansion screen;
  • the visible area of the extended screen refers to the display area of the visible virtual extended screen within the FOV that can see through the near-eye display device.
  • the content sending module 740 is configured to send the extended display content corresponding to the visible area of the visible virtual extended screen to the see-through near-eye display device.
  • the above apparatus further includes: a parameter acquisition module 760, a pose determination module 770, and a pose transmission module 780.
  • the parameter acquisition module 760 is configured to acquire the pose adjustment parameters corresponding to the i-th virtual expansion screen among the n virtual expansion screens, i being a positive integer less than or equal to n.
  • the pose determination module 770 is configured to determine the adjusted pose information of the i-th virtual expansion screen according to the pose information of the i-th virtual expansion screen and the pose adjustment parameters of the i-th virtual expansion screen.
  • the posture sending module 780 is configured to send the adjusted posture information of the i-th virtual expansion screen to the fluoroscopic near-eye display device, so that the fluoroscopic near-eye display device adjusts the posture of the i-th virtual expansion screen.
  • the above-mentioned pose acquisition module 710 includes: a data receiving unit 711, a pose determination unit 712 and Pose acquisition unit 713.
  • the data receiving unit 711 is configured to receive sensor data collected by the near-eye display device through the posture sensor.
  • the posture determining unit 712 is configured to determine posture information that can see through the near-eye display device according to the sensor data.
  • the posture acquisition unit 713 is configured to acquire posture information of n virtual expansion screens created through the near-eye display device.
  • the posture information of the see-through near-eye display device includes: a distance deviation of the current position of the see-through near-eye display device relative to the reference position Shift information, and angle shift information of the current orientation of the near-eye display device that can be seen through relative to the reference orientation.
  • the device provided in the above embodiment realizes its function, it is only exemplified by the division of the above functional modules.
  • the above functions can be allocated by different functional modules according to actual needs. That is, the content structure of the device is divided into different functional modules to complete all or part of the functions described above.
  • An exemplary embodiment of the present disclosure also provides an extended content display system.
  • the above system may include a see-through near-eye display device, a computing component, and a main display device.
  • a near-eye display device that can be seen through is configured to create n virtual expansion screens, where n is a positive integer.
  • the computing component is configured to determine the visible virtual expansion screens among the n virtual expansion screens based on the posture information of the fluoroscopic near-eye display device, the FOV of the fluoroscopic near-eye display device, and the posture information of the n virtual expansion screens;
  • the visible virtual expansion screen refers to the virtual expansion screen within the FOV that can see through the near-eye display device.
  • the computing component is further configured to obtain the extended display content corresponding to the visible virtual extended screen from the main display device.
  • the computing component is also configured to send the expanded display content to the see-through near-eye display device.
  • the see-through near-eye display device is configured to display extended display content in a visible virtual extended screen.
  • An exemplary embodiment of the present disclosure also provides an extended content display device capable of implementing the extended content display method provided by the present disclosure.
  • the device includes a processor and a memory for storing executable instructions of the processor.
  • the processor is configured as:
  • the posture information of the see-through near-eye display device determines the visible virtual expansion screens among the n virtual expansion screens; where the visible virtual expansion screen refers to Virtual expansion screen in FOV that can see through the near-eye display device;
  • the extended display content is sent to the see-through near-eye display device, so that the see-through near-eye display device displays the extended display content in the visible virtual extended screen.
  • the processor is configured to:
  • the content acquisition request includes identification information of the visible virtual expansion screen
  • the content acquisition request further includes position information of the visible area of the visible virtual extended screen
  • the processor is also configured to:
  • the processor is also configured to:
  • the extended display content corresponding to the visible area of the visible virtual extended screen is sent to the see-through near-eye display device.
  • the processor is also configured to:
  • the posture information of the i-th virtual expansion screen and the posture adjustment parameters of the i-th virtual expansion screen determine the posture information of the i-th virtual expansion screen after adjustment;
  • the processor is configured to:
  • the posture information of the near-eye display device that can be seen through is determined.
  • the posture information of the see-through near-eye display device includes: distance offset information of the current position of the see-through near-eye display device relative to a reference position, and the current orientation of the see-through near-eye display device relative to Information about the angle offset of the reference orientation.
  • Fig. 9 is a block diagram of a device 900 according to an exemplary embodiment.
  • the apparatus 900 may be the see-through near-eye display device, computing component, or main display device described above.
  • the device 900 may include one or more of the following components: a processing component 902, a memory 904, a power supply component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and Communication component 916.
  • the processing component 902 generally controls the overall operations of the device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations.
  • the processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps in the above method.
  • the processing component 902 may include one or more modules to facilitate interaction between the processing component 902 and other components.
  • the processing component 902 may include a multimedia module to facilitate interaction between the multimedia component 908 and the processing component 902.
  • the memory 904 is configured to store various types of data to support operation at the device 900. Examples of these data include instructions for any application or method operating on the device 900, contact data, phone book data, messages, pictures, videos, and so on.
  • the memory 904 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable and removable Programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read only memory
  • EPROM erasable and removable Programmable read only memory
  • PROM programmable read only memory
  • ROM read only memory
  • magnetic memory flash memory
  • flash memory magnetic disk or optical disk.
  • the power supply component 906 provides power to various components of the device 900.
  • the power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 900.
  • the multimedia component 908 includes a screen between the device 900 and the user that provides an output interface.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user.
  • the touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundary of the touch or sliding action, but also detect the duration and pressure related to the touch or sliding operation.
  • the multimedia component 908 includes a front camera and / or a rear camera. When the device 900 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
  • the audio component 910 is configured to output and / or input audio signals.
  • the audio component 910 includes a microphone (MIC), and when the device 900 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal.
  • the received audio signal may be further stored in the memory 904 or transmitted via the communication component 916.
  • the audio component 910 further includes a speaker for outputting audio signals.
  • the I / O interface 912 provides an interface between the processing component 902 and a peripheral interface module.
  • the peripheral interface module may be a keyboard, a click wheel, or a button. These buttons may include, but are not limited to: home button, volume button, start button, and lock button.
  • the sensor assembly 914 includes one or more sensors for providing the device 900 with status assessments in various aspects.
  • the sensor component 914 can detect the on / off state of the device 900, and the relative positioning of the components, for example, the components are the display and keypad of the device 900, and the sensor component 914 can also detect the position change of the device 900 or a component of the device 900 The presence or absence of user contact with the device 900, the orientation or acceleration / deceleration of the device 900, and the temperature change of the device 900.
  • the sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact.
  • the sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications.
  • the sensor assembly 914 may also include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
  • the communication component 916 is configured to facilitate wired or wireless communication between the device 900 and other devices.
  • the device 900 can access a wireless network based on a communication standard, such as Wi-Fi, 2G or 3G, or a combination thereof.
  • the communication component 916 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communication component 916 further includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra wideband
  • Bluetooth Bluetooth
  • the apparatus 900 may be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are implemented to perform the above method.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA field programmable A gate array
  • controller microcontroller, microprocessor or other electronic components are implemented to perform the above method.
  • a non-transitory computer-readable storage medium including instructions is also provided, for example, a memory 904 including instructions, and the above instructions may be executed by the processor 920 of the device 900 to complete the above method.
  • the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, or the like.
  • a non-transitory computer-readable storage medium when instructions (or computer programs) in the storage medium are executed by a processor of the device 900, enable the device 900 to execute the above-mentioned extended content display method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本公开是关于一种扩展内容显示方法、装置、系统及存储介质，属于显示控制技术领域。所述方法包括：可透视近眼显示设备创建n个虚拟扩展屏；计算组件根据可透视近眼显示设备的位姿信息和FOV以及n个虚拟扩展屏的位姿信息，确定可见虚拟扩展屏；计算组件从主显示设备获取可见虚拟扩展屏对应的扩展显示内容，并将扩展显示内容发送给可透视近眼显示设备；可透视近眼显示设备在可见虚拟扩展屏中显示扩展显示内容。本公开通过可透视近眼显示设备创建虚拟扩展屏，将扩展显示内容在虚拟扩展屏中显示；相较于物理扩展屏，虚拟扩展屏不受主显示设备的输出接口数量的限制，扩展性和便捷性强。

Description

扩展内容显示方法、装置、系统及存储介质 技术领域
本公开涉及显示控制技术领域，特别涉及一种扩展内容显示方法、装置、系统及存储介质。
背景技术
多屏扩展技术可以实现将内容投放至多个不同的屏幕中进行展示，从而为用户提供更为丰富的交互体验。
在相关技术中，终端可以通过视频输出接口与物理扩展屏连接，从而将扩展显示内容投放至物理扩展屏中显示。终端可以通过多个不同的视频输出接口，分别与多个不同的物理扩展屏连接，实现多屏扩展功能。
发明内容
本公开实施例提供了一种扩展内容显示方法、装置、系统及存储介质，所述技术方案如下：
根据本公开实施例的第一方面，提供了一种扩展内容显示方法，所述方法包括：
获取可透视近眼显示设备的位姿信息以及所述可透视近眼显示设备所创建的n个虚拟扩展屏的位姿信息，所述n为正整数；
根据所述可透视近眼显示设备的位姿信息、所述可透视近眼显示设备的视场角FOV(Field of View，视场角)和所述n个虚拟扩展屏的位姿信息，确定所述n个虚拟扩展屏中的可见虚拟扩展屏；其中，所述可见虚拟扩展屏是指在所述可透视近眼显示设备的FOV内的虚拟扩展屏；
获取所述可见虚拟扩展屏对应的扩展显示内容；
将所述扩展显示内容发送给所述可透视近眼显示设备，以便所述可透视近眼显示设备在所述可见虚拟扩展屏中显示所述扩展显示内容。
可选地，所述获取所述可见虚拟扩展屏对应的扩展显示内容，包括：
向主显示设备发送内容获取请求，所述内容获取请求中包括所述可见虚拟扩展屏的标识信息；
接收所述主显示设备发送的所述可见虚拟扩展屏对应的扩展显示内容。
可选地，所述内容获取请求中还包括所述可见虚拟扩展屏的可见区域的位置信息，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
所述接收所述主显示设备发送的所述可见虚拟扩展屏对应的扩展显示内容，包括：
接收所述主显示设备发送的所述可见虚拟扩展屏的可见区域对应的扩展显示内容。
可选地，所述方法还包括：
根据所述可见虚拟扩展屏的可见区域的位置信息，从所述可见虚拟扩展屏对应的扩展显示内容中，提取所述可见虚拟扩展屏的可见区域对应的扩展显示内容；其中，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
所述将所述扩展显示内容发送给所述可透视近眼显示设备，包括：
将所述可见虚拟扩展屏的可见区域对应的扩展显示内容发送给所述可透视近眼显示设备。
可选地，所述方法还包括：
获取对应于所述n个虚拟扩展屏中的第i个虚拟扩展屏的位姿调整参数，所述i为小于等于n的正整数；
根据所述第i个虚拟扩展屏的位姿信息和所述第i个虚拟扩展屏的位姿调整参数，确定所述第i个虚拟扩展屏调整后的位姿信息；
向所述可透视近眼显示设备发送所述第i个虚拟扩展屏调整后的位姿信息，以便所述可透视近眼显示设备调整所述第i个虚拟扩展屏的位姿。
可选地，所述获取可透视近眼显示设备的位姿信息，包括：
接收所述可透视近眼显示设备通过位姿传感器采集的传感器数据；
根据所述传感器数据确定所述可透视近眼显示设备的位姿信息。
可选地，所述可透视近眼显示设备的位姿信息包括：所述可透视近眼显示设备的当前位置相对于基准位置的距离偏移信息，以及所述可透视近眼显示设备的当前朝向相对于基准朝向的角度偏移信息。
根据本公开实施例的第二方面，提供了一种扩展内容显示方法，应用于扩展 内容显示系统中，所述系统包括：可透视近眼显示设备、计算组件和主显示设备；所述方法包括：
所述可透视近眼显示设备创建n个虚拟扩展屏，所述n为正整数；
所述计算组件根据所述可透视近眼显示设备的位姿信息、所述可透视近眼显示设备的FOV和所述n个虚拟扩展屏的位姿信息，确定所述n个虚拟扩展屏中的可见虚拟扩展屏；其中，所述可见虚拟扩展屏是指在所述可透视近眼显示设备的FOV内的虚拟扩展屏；
所述计算组件从所述主显示设备获取所述可见虚拟扩展屏对应的扩展显示内容；
所述计算组件将所述扩展显示内容发送给所述可透视近眼显示设备；
所述可透视近眼显示设备在所述可见虚拟扩展屏中显示所述扩展显示内容。
可选地，所述计算组件从所述主显示设备获取所述可见虚拟扩展屏对应的扩展显示内容，包括：
所述计算组件向所述主显示设备发送内容获取请求，所述内容获取请求中包括所述可见虚拟扩展屏的标识信息；
所述主显示设备根据所述内容获取请求，生成所述可见虚拟扩展屏对应的扩展显示内容；
所述主显示设备向所述计算组件发送所述扩展显示内容。
可选地，所述内容获取请求中还包括所述可见虚拟扩展屏的可见区域的位置信息，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
所述主显示设备根据所述内容获取请求，生成所述可见虚拟扩展屏对应的扩展显示内容，包括：
所述主显示设备根据所述可见虚拟扩展屏的可见区域的位置信息，生成所述可见虚拟扩展屏的可见区域对应的扩展显示内容。
可选地，所述方法还包括:
所述计算组件根据所述可见虚拟扩展屏的可见区域的位置信息，从所述可见虚拟扩展屏对应的扩展显示内容中，提取所述可见虚拟扩展屏的可见区域对应的扩展显示内容；其中，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
所述计算组件将所述扩展显示内容发送给所述可透视近眼显示设备，包括：
所述计算组件将所述可见虚拟扩展屏的可见区域对应的扩展显示内容发送给所述可透视近眼显示设备。
可选地，所述方法还包括：
所述计算组件获取对应于所述n个虚拟扩展屏中的第i个虚拟扩展屏的位姿调整参数，所述i为小于等于n的正整数；
所述计算组件根据所述第i个虚拟扩展屏的位姿信息和所述第i个虚拟扩展屏的位姿调整参数，确定所述第i个虚拟扩展屏调整后的位姿信息；
所述计算组件向所述可透视近眼显示设备发送所述第i个虚拟扩展屏调整后的位姿信息；
所述可透视近眼显示设备根据所述第i个虚拟扩展屏调整后的位姿信息，展示所述第i个虚拟扩展屏。
可选地，所述方法还包括：
所述可透视近眼显示设备通过位姿传感器采集传感器数据，向所述计算组件发送所述传感器数据；
所述计算组件根据所述传感器数据，确定所述可透视近眼显示设备的位姿信息。
可选地，所述方法还包括：
所述主显示设备生成主屏幕对应的主显示内容，所述主屏幕是指所述主显示设备的屏幕；
所述主显示设备在所述主屏幕中显示所述主显示内容。
可选地，所述计算组件集成于所述可透视近眼显示设备中；
或者，
所述计算组件集成于所述主显示设备中；
或者，
所述计算组件是独立于所述可透视近眼显示设备和所述主显示设备的装置。
根据本公开实施例的第三方面，提供了一种扩展内容显示装置，所述装置包括：
位姿获取模块，被配置为获取可透视近眼显示设备的位姿信息以及所述可透视近眼显示设备所创建的n个虚拟扩展屏的位姿信息，所述n为正整数；
可见屏确定模块，被配置为根据所述可透视近眼显示设备的位姿信息、所述可透视近眼显示设备的FOV和所述n个虚拟扩展屏的位姿信息，确定所述n个虚拟扩展屏中的可见虚拟扩展屏；其中，所述可见虚拟扩展屏是指在所述可透视近眼显示设备的FOV内的虚拟扩展屏；
内容获取模块，被配置为获取所述可见虚拟扩展屏对应的扩展显示内容；
内容发送模块，被配置为将所述扩展显示内容发送给所述可透视近眼显示设备，以便所述可透视近眼显示设备在所述可见虚拟扩展屏中显示所述扩展显示内容。
可选地，所述内容获取模块，包括：
请求发送单元，被配置为向主显示设备发送内容获取请求，所述内容获取请求中包括所述可见虚拟扩展屏的标识信息；
内容接收单元，被配置为接收所述主显示设备发送的所述可见虚拟扩展屏对应的扩展显示内容。
可选地，所述内容获取请求中还包括所述可见虚拟扩展屏的可见区域的位置信息，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
所述内容接收单元，被配置为接收所述主显示设备发送的所述可见虚拟扩展屏的可见区域对应的扩展显示内容。
可选地，所述装置还包括:
内容提取模块，被配置为根据所述可见虚拟扩展屏的可见区域的位置信息，从所述可见虚拟扩展屏对应的扩展显示内容中，提取所述可见虚拟扩展屏的可见区域对应的扩展显示内容；其中，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
所述内容发送模块，被配置为将所述可见虚拟扩展屏的可见区域对应的扩展显示内容发送给所述可透视近眼显示设备。
可选地，所述装置还包括:
参数获取模块，被配置为获取对应于所述n个虚拟扩展屏中的第i个虚拟扩展屏的位姿调整参数，所述i为小于等于n的正整数；
位姿确定模块，被配置为根据所述第i个虚拟扩展屏的位姿信息和所述第i个虚拟扩展屏的位姿调整参数，确定所述第i个虚拟扩展屏调整后的位姿信息；
位姿发送模块，被配置为向所述可透视近眼显示设备发送所述第i个虚拟扩展屏调整后的位姿信息，以便所述可透视近眼显示设备调整所述第i个虚拟扩展屏的位姿。
可选地，所述位姿获取模块，包括：
数据接收单元，被配置为接收所述可透视近眼显示设备通过位姿传感器采集的传感器数据；
位姿确定单元，被配置为根据所述传感器数据确定所述可透视近眼显示设备的位姿信息。
可选地，所述可透视近眼显示设备的位姿信息包括：所述可透视近眼显示设备的当前位置相对于基准位置的距离偏移信息，以及所述可透视近眼显示设备的当前朝向相对于基准朝向的角度偏移信息。
根据本公开实施例的第四方面，提供了一种扩展内容显示系统，所述系统包括：可透视近眼显示设备、计算组件和主显示设备；
所述可透视近眼显示设备，被配置为创建n个虚拟扩展屏，所述n为正整数；
所述计算组件，被配置为根据所述可透视近眼显示设备的位姿信息、所述可透视近眼显示设备的FOV和所述n个虚拟扩展屏的位姿信息，确定所述n个虚拟扩展屏中的可见虚拟扩展屏；其中，所述可见虚拟扩展屏是指在所述可透视近眼显示设备的FOV内的虚拟扩展屏；
所述计算组件，还被配置为从所述主显示设备获取所述可见虚拟扩展屏对应的扩展显示内容；
所述计算组件，还被配置为将所述扩展显示内容发送给所述可透视近眼显示设备；
所述可透视近眼显示设备，被配置为在所述可见虚拟扩展屏中显示所述扩展显示内容。
根据本公开实施例的第五方面，提供了一种扩展内容显示装置，所述装置包括：
处理器；
用于存储所述处理器的可执行指令的存储器；
其中，所述处理器被配置为：
获取可透视近眼显示设备的位姿信息以及所述可透视近眼显示设备所创建 的n个虚拟扩展屏的位姿信息，所述n为正整数；
根据所述可透视近眼显示设备的位姿信息、所述可透视近眼显示设备的FOV和所述n个虚拟扩展屏的位姿信息，确定所述n个虚拟扩展屏中的可见虚拟扩展屏；其中，所述可见虚拟扩展屏是指在所述可透视近眼显示设备的FOV内的虚拟扩展屏；
获取所述可见虚拟扩展屏对应的扩展显示内容；
将所述扩展显示内容发送给所述可透视近眼显示设备，以便所述可透视近眼显示设备在所述可见虚拟扩展屏中显示所述扩展显示内容。
根据本公开实施例的第六方面，提供了一种非临时性计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如第一方面所述方法的步骤。
本公开实施例提供的技术方案可以包括以下有益效果：
通过可透视近眼显示设备创建虚拟扩展屏，将扩展显示内容在虚拟扩展屏中进行显示；相较于物理扩展屏，虚拟扩展屏的数量不会受制于主显示设备所具备的视频输出接口的数量限制，扩展性更强，且比物理扩展屏更具便捷性。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
图1是根据一示例性实施例示出的一种实施环境的示意图；
图2是根据一示例性实施例示出的一种扩展内容显示方法的流程图；
图3示例性示出了一种可见虚拟扩展的屏示意图；
图4是根据另一示例性实施例示出的一种扩展内容显示方法的流程图；
图5是根据另一示例性实施例示出的一种扩展内容显示方法的流程图；
图6是根据一示例性实施例示出的一种扩展内容显示过程的示意图；
图7是根据一示例性实施例示出的一种扩展内容显示装置的框图；
图8是根据另一示例性实施例示出的一种扩展内容装置的框图；
图9是根据一示例性实施例示出的一种装置的框图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。
图1是根据一示例性实施例示出的一种实施环境的示意图，该实施环境可以包括：可透视近眼显示设备101、计算组件102和主显示设备103。
可透视近眼显示(see-through near-eye display)设备101是指能够在用户眼睛的视野范围内创建虚拟图像，且能够让用户看到现实场景的佩戴式光学显示设备。可选地，可透视近眼显示设备101为AR(Augumented Reality，增强现实)显示设备或MR(Mixed Reality，混合现实)显示设备。例如，可透视近眼显示设备101可以是诸如AR眼镜、AR头盔、MR眼镜、MR头盔之类的头戴式显示设备。在本公开实施例中，可透视近眼显示设备101能够创建虚拟扩展屏，并在虚拟扩展屏中显示相关内容。
计算组件102是指具有数据计算和处理能力的装置。计算组件102可以对可透视近眼显示设备101和主显示设备103传来的数据执行相应的处理工作。可选地，计算组件102集成于可透视近眼显示设备101中；或者，计算组件102集成于主显示设备103中；又或者，计算组件102是独立于可透视近眼显示设备101和主显示设备103的装置。
主显示设备103是指提供主屏幕的显示设备。例如，主显示设备103可以是诸如PC(Personal Computer，个人计算机)、手机、平板电脑等电子设备。在本公开实施例中，主显示设备103用于生成主屏幕和虚拟扩展屏各自的显示内容，并将上述显示内容投放至响应的屏幕进行展示。
当计算组件102集成于可透视近眼显示设备101中，或者计算组件102集成于主显示设备103中时，可透视近眼显示设备101和主显示设备103之间可以通过有线或者无线的方式建立通信连接。当计算组件102是独立于可透视近眼显示 设备101和主显示设备103的装置时，计算组件102和可透视近眼显示设备101之间可以通过有线或者无线的方式建立通信连接，且计算组件102和主显示设备103之间也可以通过有线或者无线的方式建立通信连接。可选地，上述有线方式可以是基于视频输出接口的有线连接，如HDMI(High Definition Multimedia Interface，高清晰度多媒体接口)、VGA(Video Graphics Array，视频图形阵列)接口等。可选地，上述无线方式可以是基于无线网络建立的连接，如无线局域网等。
图2是根据一示例性实施例示出的一种扩展内容显示方法的流程图。该方法可应用于图1所示的实施环境中。该方法可以包括如下几个步骤。
在步骤201中，可透视近眼显示设备创建n个虚拟扩展屏，n为正整数。
虚拟扩展屏是指由可透视近眼显示设备展示的非物理扩展屏。用户只有使用可透视近眼显示设备才能看到虚拟扩展屏，例如用户戴上AR眼镜之后可以看到AR眼镜所展示的虚拟扩展屏。
在本公开实施例中，对于可透视近眼显示设备所创建的虚拟扩展屏的数量不作限定。可透视近眼显示设备可以创建1个虚拟扩展屏，也可以创建多个虚拟扩展屏。可透视近眼显示设备可以根据用户需求来创建虚拟扩展屏。例如，可透视近眼显示设备在接收到创建指令之后，根据创建指令创建n个虚拟扩展屏。其中，创建指令可以是由用户触发，并由主显示设备发送给可透视近眼显示设备。可透视近眼显示设备所创建的虚拟扩展屏的数量，可以由用户指定并携带在创建指令中，也可以是预先设定的默认值。
另外，虚拟扩展屏的初始位姿，可以由用户指定并携带在创建指令中，也可以由计算组件确定并发送给可透视近眼显示设备，或者是预先设定的默认初始位姿。在一个示例中，计算组件确定虚拟扩展屏的初始位姿信息，将该初始位姿信息发送给可透视近眼显示设备，可透视近眼显示设备根据该初始位姿信息创建虚拟扩展屏。可选地，计算组件可以根据用户历史习惯或者预配置信息，确定虚拟扩展屏的初始位姿信息。
在一个示例中，虚拟扩展屏的位姿是固定的、不可调整的。在另一个示例中，虚拟扩展屏的位姿是非固定的、可调整的，例如虚拟扩展屏的位置是可移动的，且虚拟扩展屏的朝向是可调整的，以使得虚拟扩展屏更具灵活性，更好地满足用 户需求。
在步骤202中，计算组件根据可透视近眼显示设备的位姿信息、可透视近眼显示设备的FOV和n个虚拟扩展屏的位姿信息，确定n个虚拟扩展屏中的可见虚拟扩展屏。
在本公开实施例中，可见虚拟扩展屏是指在可透视近眼显示设备的FOV内的虚拟扩展屏。也即，可见虚拟扩展屏是指用户能够看见的虚拟扩展屏。由于可透视近眼显示设备的FOV有限，可透视近眼显示设备创建的各个虚拟扩展屏中，可能有的在上述FOV内，而有的不在上述FOV内。用户可以通过调整可透视近眼显示设备的位置和朝向，来查看不同的虚拟扩展屏。另外，当某一虚拟扩展屏的全部或者部分区域在可透视近眼显示设备的FOV内时，该虚拟扩展屏即为可见虚拟扩展屏；当某一虚拟扩展屏的全部区域均不在可透视近眼显示设备的FOV内时，该虚拟扩展屏则不是可见虚拟扩展屏。
可透视近眼显示设备的位姿信息用于指示可透视近眼显示设备在空间中的位置和姿态。可透视近眼显示设备的位姿信息可以采用可透视近眼显示设备相对于基准位姿信息的相对位姿信息来表示，也可以采用可透视近眼显示设备在空间中的绝对位姿信息来表示。上述基准位姿信息可以预先设定，例如将可透视近眼显示设备在某一时刻的位姿信息确定为基准位姿信息。
可选地，可透视近眼显示设备的位姿信息包括：可透视近眼显示设备的当前位置相对于基准位置的距离偏移信息，以及可透视近眼显示设备的当前朝向相对于基准朝向的角度偏移信息。示例性地，可透视近眼显示设备的位姿信息可以采用6个参数来表示；其中距离偏移信息包括三个维度的参数，如(x,y,z)；角度偏移信息也包括三个维度的参数，如(pitch,yaw,roll)。可透视近眼显示设备的位姿信息可以由可透视近眼显示设备提供给计算组件。例如，可透视近眼显示设备通过位姿传感器来获取上述位姿信息。位姿传感器可以是视觉传感器，也可以是运动传感器，包括加速度传感器、陀螺仪等。
可透视近眼显示设备的FOV即为可透视近眼显示设备的视场角，可透视近眼显示设备的FOV由其自身的硬件性能决定。可透视近眼显示设备的FOV可以预存在计算组件中，也可以由可透视近眼显示设备发送给计算组件。
虚拟扩展屏的位姿信息用于指示虚拟扩展屏的位置和姿态。虚拟扩展屏的位姿信息可以采用虚拟扩展屏相对于基准位姿信息的相对位姿信息来表示。上述基 准位姿信息可以预先设定，例如将虚拟扩展屏的初始位姿信息确定为基准位姿信息。另外，此处所介绍的虚拟扩展屏的基准位姿信息，与上文介绍的可透视近眼显示设备的基准位姿信息，两者可以相同，也可以不同，本公开实施例对此不作限定。
可选地，虚拟扩展屏的位姿信息包括：虚拟扩展屏的当前位置相对于基准位置的距离偏移信息，以及虚拟扩展屏的当前朝向相对于基准朝向的角度偏移信息。另外，此处所介绍的虚拟扩展屏的基准位置和基准朝向，与上文介绍的可透视近眼显示设备的基准位置和基准朝向，可以相同，也可以不同，本公开实施例对此不作限定。与可透视近眼显示设备的位姿信息类似，虚拟扩展屏的位姿信息也可以采用6个参数来表示；其中距离偏移信息包括三个维度的参数，如(x,y,z)；角度偏移信息也包括三个维度的参数，如(pitch,yaw,roll)。
计算组件可以根据可透视近眼显示设备的位姿信息、可透视近眼显示设备的FOV，确定可透视近眼显示设备的FOV，然后结合n个虚拟扩展屏的位姿信息，将上述FOV内的虚拟扩展屏确定为可见虚拟扩展屏。
结合参考图3，其示例性示出了可见虚拟扩展屏的示意图。如图3所示，可透视近眼显示设备101创建了两个虚拟扩展屏，即扩展屏1和扩展屏2。其中，扩展屏1在可透视近眼显示设备101的FOV内，所以扩展屏1为可见虚拟扩展屏。
在步骤203中，计算组件从主显示设备获取可见虚拟扩展屏对应的扩展显示内容。
在本公开实施例中，主显示设备用于渲染生成显示内容。主显示设备可配备有显卡，该显卡用于渲染生成显示内容。可选地，主显示设备生成的显示内容，包括主屏幕对应的主显示内容，以及可见虚拟扩展屏对应的扩展显示内容。其中，主屏幕是指主显示设备的屏幕，其是物理屏幕，而非虚拟屏幕。可选地，主显示设备将主显示内容和扩展显示内容拼接渲染，得到拼接显示内容，然后从拼接显示内容中分别提取主显示内容和扩展显示内容。
在一个示例中，主显示设备可以渲染生成上述n个虚拟扩展屏各自对应的扩展显示内容，并将该n个虚拟扩展屏各自对应的扩展显示内容发送给计算组件，由计算组件从中选取可见虚拟扩展屏对应的扩展显示内容。
在另一个示例中，计算组件在确定可见虚拟扩展屏之后，可以将可见虚拟扩 展屏的标识信息发送给主显示设备，主显示设备根据可见虚拟扩展屏的标识信息，仅渲染生成可见虚拟扩展屏对应的扩展显示内容，然后将可见虚拟扩展屏对应的扩展显示内容发送给计算组件。这种方式可以节省主显示设备的处理开销。
另外，主显示设备可以通过与计算组件之间建立的连接，向计算组件发送扩展显示内容。例如，主显示设备可以通过视频输出接口，向计算组件发送扩展显示内容。
在步骤204中，计算组件将可见虚拟扩展屏对应的扩展显示内容发送给可透视近眼显示设备。
计算组件可以通过与可透视近眼显示设备之间建立的连接，向可透视近眼显示设备发送扩展显示内容。例如，计算组件可以通过视频输出接口，向可透视近眼显示设备发送扩展显示内容。
在步骤205中，可透视近眼显示设备在可见虚拟扩展屏中显示扩展显示内容。
可透视近眼显示设备接收到可见虚拟扩展屏对应的扩展显示内容之后，在可见虚拟扩展屏中显示上述扩展显示内容。
可选地，当可见虚拟扩展屏的数量为一个时，可透视近眼显示设备将接收到的扩展显示内容，在这个可见虚拟扩展屏中显示；当可见虚拟扩展屏的数量为多个时，可透视近眼显示设备分别将每一个可见虚拟扩展屏对应的扩展显示内容，在对应的可见虚拟扩展屏中显示。例如，可见虚拟扩展屏的数量为2个，可透视近眼显示设备在一个可见虚拟扩展屏中显示办公软件的文档编辑界面，在另一个可见虚拟扩展屏中显示社交应用的聊天会话界面。
另外，主显示设备还可以在主屏幕中显示主显示内容。
综上所述，本公开实施例提供的技术方案中，通过可透视近眼显示设备创建虚拟扩展屏，将扩展显示内容在虚拟扩展屏中进行显示；相较于物理扩展屏，虚拟扩展屏的数量不会受制于主显示设备所具备的视频输出接口的数量限制，扩展性更强，且比物理扩展屏更具便捷性。
图4是根据另一示例性实施例示出的一种扩展内容显示方法的流程图。该方法可应用于图1所示的实施环境中。该方法可以包括以下几个步骤。
在步骤401中，可透视近眼显示设备创建n个虚拟扩展屏，n为正整数。
在步骤402中，可透视近眼显示设备通过位姿传感器采集传感器数据。
在步骤403中，可透视近眼显示设备向计算组件发送上述传感器数据。
在步骤404中，计算组件根据传感器数据，确定可透视近眼显示设备的位姿信息。
在步骤405中，计算组件根据可透视近眼显示设备的位姿信息、可透视近眼显示设备的FOV和n个虚拟扩展屏的位姿信息，确定n个虚拟扩展屏中的可见虚拟扩展屏。
上述步骤401-405的介绍说明，可参见图2实施例，本实施例对此不再赘述。
在步骤406中，计算组件向主显示设备发送内容获取请求，内容获取请求中包括可见虚拟扩展屏的标识信息，以及可见虚拟扩展屏的可见区域的位置信息。
内容获取请求用于请求获取扩展显示内容。可见虚拟扩展屏的标识信息用于唯一指示该可见虚拟扩展屏。不同的虚拟扩展屏具有不同的标识信息。
可选地，内容获取请求中包括可见虚拟扩展屏的标识信息，以便主显示设备根据该可见虚拟扩展屏的标识信息，生成该可见虚拟扩展屏对应的扩展显示内容。
可选地，内容获取请求中还包括可见虚拟扩展屏的可见区域的位置信息，可见虚拟扩展屏的可见区域是指可见虚拟扩展屏在可透视近眼显示设备的FOV内的显示区域。可见区域的位置信息用于指示可见区域在可见虚拟扩展屏中的位置。例如，可见区域的位置信息可以包括可见区域的一个顶点的坐标，以及可见区域的长和宽。
当可见虚拟扩展屏的整个屏幕区域在可透视近眼显示设备的FOV内时，该可见虚拟扩展屏的可见区域是整个屏幕区域；当可见虚拟扩展屏的一部分屏幕区域在可透视近眼显示设备的FOV内，且另一部分屏幕区域在可透视近眼显示设备的FOV外时，该可见虚拟扩展屏的可见区域是部分屏幕区域。
在步骤407中，主显示设备根据可见虚拟扩展屏的可见区域的位置信息，生成可见虚拟扩展屏的可见区域对应的扩展显示内容。
在本实施例中，主显示设备仅渲染生成可见虚拟扩展屏的可见区域对应的扩展显示内容，其它非可见的虚拟扩展屏以及可见虚拟扩展屏的非可见区域对应的扩展显示内容，主显示设备并不渲染生成，从而节省主显示设备的处理开销。
在步骤408中，主显示设备向计算组件发送可见虚拟扩展屏的可见区域对应的扩展显示内容。
在步骤409中，计算组件将可见虚拟扩展屏的可见区域对应的扩展显示内容 发送给可透视近眼显示设备。
在步骤410中，可透视近眼显示设备在可见虚拟扩展屏中显示上述可见区域对应的扩展显示内容。
另外，主显示设备还可以生成主屏幕对应的主显示内容，并在主屏幕中显示主显示内容。
综上所述，本公开实施例提供的技术方案中，通过计算组件将可见虚拟扩展屏的可见区域的位置信息发送给主显示设备，主显示设备可以仅渲染生成可见虚拟扩展屏的可见区域对应的扩展显示内容，而其它非可见的虚拟扩展屏以及可见虚拟扩展屏的非可见区域对应的扩展显示内容，主显示设备不必不渲染生成，从而节省主显示设备的处理开销。
图5是根据另一示例性实施例示出的一种扩展内容显示方法的流程图。该方法可应用于图1所示的实施环境中。该方法可以包括以下几个步骤。
在步骤501中，可透视近眼显示设备创建n个虚拟扩展屏，n为正整数。
在步骤502中，可透视近眼显示设备通过位姿传感器采集传感器数据。
在步骤503中，可透视近眼显示设备向计算组件发送上述传感器数据。
在步骤504中，计算组件根据传感器数据，确定可透视近眼显示设备的位姿信息。
在步骤505中，计算组件根据可透视近眼显示设备的位姿信息、可透视近眼显示设备的FOV和n个虚拟扩展屏的位姿信息，确定n个虚拟扩展屏中的可见虚拟扩展屏。
上述步骤501-505的介绍说明，可参见图2实施例，本实施例对此不再赘述。
在步骤506中，计算组件向主显示设备发送内容获取请求，内容获取请求中包括可见虚拟扩展屏的标识信息。
在步骤507中，主显示设备根据内容获取请求，生成可见虚拟扩展屏对应的扩展显示内容。
在步骤508中，主显示设备向计算组件发送可见虚拟扩展屏对应的扩展显示内容。
在步骤509中，计算组件根据可见虚拟扩展屏的可见区域的位置信息，从可见虚拟扩展屏对应的扩展显示内容中，提取可见虚拟扩展屏的可见区域对应的扩 展显示内容。
在本实施例中，主显示设备将可见虚拟扩展屏的整个屏幕区域的扩展显示内容发送给计算组件，由计算组件从中提取可见区域对应的扩展显示内容。在可透视近眼显示设备的位姿发生变化时，由于可透视近眼显示设备的FOV发生变化，因此可见区域会发生变化，使得所展示的扩展显示内容需要相应调整。相较于图4实施例，本实施例提供的方案中，当可见区域发生变化时，由于变化后的可见区域对应的扩展显示内容已经渲染生成并发送给计算组件，因此响应更加及时流畅。
在步骤510中，计算组件将可见虚拟扩展屏的可见区域对应的扩展显示内容发送给可透视近眼显示设备。
在步骤511中，可透视近眼显示设备在可见虚拟扩展屏中显示上述可见区域对应的扩展显示内容。
另外，主显示设备还可以生成主屏幕对应的主显示内容，并在主屏幕中显示主显示内容。
综上所述，本公开实施例提供的技术方案中，通过主显示设备将可见虚拟扩展屏对应的扩展显示内容发送给计算组件，由计算组件根据可见虚拟扩展屏的可见区域的位置信息，从可见虚拟扩展屏对应的扩展显示内容中，提取可见虚拟扩展屏的可见区域对应的扩展显示内容，从而当可见区域发生变化时，响应更加及时流畅。
图6是根据一示例性实施例示出的一种扩展内容显示过程的示意图。如图6所示，可透视近眼显示设备101在主显示设备103左右两侧各创建了一个虚拟扩展屏，即扩展屏1和扩展屏2。在图6的上侧部分中，扩展屏1在可透视近眼显示设备101的FOV内，因此扩展屏1为可见虚拟扩展屏。主显示设备103的显卡渲染生成扩展屏1对应的扩展显示内容，以及渲染生成主显示设备103的主屏幕对应的主显示内容。主显示设备103将扩展屏1对应的扩展显示内容发送给计算组件102，并在主屏幕上显示主显示内容。计算组件102将扩展屏1对应的扩展显示内容发送给可透视近眼显示设备101，可透视近眼显示设备101在扩展屏1上显示上述扩展显示内容。
当可透视近眼显示设备101的位姿发生变化，如图6的下侧部分所示，扩展 屏2的部分区域在可透视近眼显示设备101的FOV内，因此扩展屏2为可见虚拟扩展屏，其左侧阴影区域为扩展屏2的可见区域。在一个示例中，主显示设备103的显卡渲染生成扩展屏2对应的扩展显示内容，以及渲染生成主屏幕对应的主显示内容。主显示设备103将扩展屏2对应的扩展显示内容发送给计算组件102，并在主屏幕上显示主显示内容。计算组件102根据扩展屏2的可见区域的位置信息，从扩展屏2对应的扩展显示内容中，提取扩展屏2的可见区域对应的扩展显示内容，并将其发送给可透视近眼显示设备101，可透视近眼显示设备101在扩展屏2的上述可见区域中显示接收到的扩展显示内容。
在一种可能的实现场景中，主显示设备103为笔记本电脑，可透视近眼显示设备101为AR眼镜。用户佩戴AR眼镜，使用AR眼镜在笔记本电脑左右两侧各创建一个虚拟扩展屏。笔记本电脑的主屏幕上可以显示桌面界面，用户可以将某一文档从主屏幕拖动到左侧的虚拟扩展屏，将即时通信应用的聊天界面从主屏幕拖动到右侧的虚拟扩展屏。例如，用户可以通过鼠标、AR手柄或手势动作完成上述拖动。当用户看向左侧的虚拟扩展屏时，左侧的虚拟扩展屏中显示相应的文档内容，用户可以编辑文档；当用户看向右侧的虚拟扩展屏时，右侧的虚拟扩展屏中显示相应的聊天界面，用户可以进行聊天会话。
在基于上述任一实施例提供的另一个可选实施例中，还提供了对虚拟扩展屏的位姿的调整功能。该调整过程可以包括如下几个步骤：
1、计算组件获取对应于n个虚拟扩展屏中的第i个虚拟扩展屏的位姿调整参数，i为小于等于n的正整数。
2、计算组件根据第i个虚拟扩展屏的位姿信息和第i个虚拟扩展屏的位姿调整参数，确定第i个虚拟扩展屏调整后的位姿信息。
3、计算组件向可透视近眼显示设备发送第i个虚拟扩展屏调整后的位姿信息。
4、可透视近眼显示设备根据第i个虚拟扩展屏调整后的位姿信息，展示第i个虚拟扩展屏。
用户可以通过手势、AR手柄或者鼠标等输入组件，执行对应于虚拟扩展屏的位姿的调整操作。相应地，计算组件可以根据上述调整操作，获取位姿调整参数。在本实施例中，仅以针对第i个虚拟扩展屏进行位姿调整为例进行介绍说明， 任意一个虚拟扩展屏的位姿调整过程均可参见本实施例提供的方法流程。
可选地，位姿调整参数包括：位置调整参数和/或姿态调整参数。位置调整参数用于对虚拟扩展屏的位置进行调整，姿态调整参数用于对虚拟扩展屏的姿态进行调整。
通过上述方式，实现了对虚拟扩展屏的位姿进行调整，使得虚拟扩展屏的显示更具灵活性，更好地满足用户需求。
需要说明的是，在上述方法实施例中，仅从可透视近眼显示设备、计算组件和主显示设备交互的角度，对本公开技术方案进行了介绍说明。上述实施例中有关可透视近眼显示设备的步骤可以单独实现成为可透视近眼显示设备一侧的扩展内容显示方法，有关计算组件的步骤可以单独实现成为计算组件一侧的扩展内容显示方法，有关主显示设备的步骤可以单独实现成为主显示设备一侧的扩展内容显示方法。
下述为本公开装置实施例，可以用于执行本公开方法实施例。对于本公开装置实施例中未披露的细节，请参照本公开方法实施例。
图7是根据一示例性实施例示出的一种扩展内容显示装置的框图。该装置具有实现上述方法示例的功能，所述功能可以由硬件实现，也可以由硬件执行相应的软件实现。该装置可以包括：位姿获取模块710、可见屏确定模块720、内容获取模块730和内容发送模块740。
位姿获取模块710，被配置为获取增强现实可透视近眼显示设备的位姿信息以及可透视近眼显示设备所创建的n个虚拟扩展屏的位姿信息，n为正整数。
可见屏确定模块720，被配置为根据可透视近眼显示设备的位姿信息、可透视近眼显示设备的FOV和n个虚拟扩展屏的位姿信息，确定n个虚拟扩展屏中的可见虚拟扩展屏；其中，可见虚拟扩展屏是指在可透视近眼显示设备的FOV内的虚拟扩展屏。
内容获取模块730，被配置为获取可见虚拟扩展屏对应的扩展显示内容。
内容发送模块740，被配置为将扩展显示内容发送给可透视近眼显示设备，以便可透视近眼显示设备在可见虚拟扩展屏中显示扩展显示内容。
综上所述，本公开实施例提供的技术方案中，通过可透视近眼显示设备创建 虚拟扩展屏，将扩展显示内容在虚拟扩展屏中进行显示；相较于物理扩展屏，虚拟扩展屏的数量不会受制于主显示设备所具备的视频输出接口的数量限制，扩展性更强，且比物理扩展屏更具便捷性。
在基于图7实施例提供的一个可选实施例中，如图8所示，上述内容获取模块730，包括：请求发送单元731和内容接收单元732。
请求发送单元731，被配置为向主显示设备发送内容获取请求，内容获取请求中包括可见虚拟扩展屏的标识信息。
内容接收单元732，被配置为接收主显示设备发送的可见虚拟扩展屏对应的扩展显示内容。
可选地，上述内容获取请求中还包括可见虚拟扩展屏的可见区域的位置信息，可见虚拟扩展屏的可见区域是指可见虚拟扩展屏在可透视近眼显示设备的FOV内的显示区域。
上述内容接收单元732，被配置为接收主显示设备发送的可见虚拟扩展屏的可见区域对应的扩展显示内容。
在基于图7实施例提供的另一个可选实施例中，如图8所示，上述装置还包括：内容提取模块750。
内容提取模块750，被配置为根据可见虚拟扩展屏的可见区域的位置信息，从可见虚拟扩展屏对应的扩展显示内容中，提取可见虚拟扩展屏的可见区域对应的扩展显示内容；其中，可见虚拟扩展屏的可见区域是指可见虚拟扩展屏在可透视近眼显示设备的FOV内的显示区域。
内容发送模块740，被配置为将可见虚拟扩展屏的可见区域对应的扩展显示内容发送给可透视近眼显示设备。
在基于图7实施例或者上述任一可选实施例提供的另一个可选实施例中，如图8所示，上述装置还包括：参数获取模块760、位姿确定模块770和位姿发送模块780。
参数获取模块760，被配置为获取对应于n个虚拟扩展屏中的第i个虚拟扩展屏的位姿调整参数，i为小于等于n的正整数。
位姿确定模块770，被配置为根据第i个虚拟扩展屏的位姿信息和第i个虚拟扩展屏的位姿调整参数，确定第i个虚拟扩展屏调整后的位姿信息。
位姿发送模块780，被配置为向可透视近眼显示设备发送第i个虚拟扩展屏 调整后的位姿信息，以便可透视近眼显示设备调整第i个虚拟扩展屏的位姿。
在基于图7实施例或者上述任一可选实施例提供的另一个可选实施例中，如图8所示，上述位姿获取模块710，包括：数据接收单元711、位姿确定单元712和位姿获取单元713。
数据接收单元711，被配置为接收可透视近眼显示设备通过位姿传感器采集的传感器数据。
位姿确定单元712，被配置为根据传感器数据确定可透视近眼显示设备的位姿信息。
位姿获取单元713，被配置为获取可透视近眼显示设备所创建的n个虚拟扩展屏的位姿信息。
在基于图7实施例或者上述任一可选实施例提供的另一个可选实施例中，可透视近眼显示设备的位姿信息包括：可透视近眼显示设备的当前位置相对于基准位置的距离偏移信息，以及可透视近眼显示设备的当前朝向相对于基准朝向的角度偏移信息。
需要说明的一点是，上述实施例提供的装置在实现其功能时，仅以上述各个功能模块的划分进行举例说明，实际应用中，可以根据实际需要而将上述功能分配由不同的功能模块完成，即将设备的内容结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。
本公开一示例性实施例还提供了一种扩展内容显示系统，上述系统可以包括：可透视近眼显示设备、计算组件和主显示设备。
可透视近眼显示设备，被配置为创建n个虚拟扩展屏，n为正整数。
计算组件，被配置为根据可透视近眼显示设备的位姿信息、可透视近眼显示设备的FOV和n个虚拟扩展屏的位姿信息，确定n个虚拟扩展屏中的可见虚拟扩展屏；其中，可见虚拟扩展屏是指在可透视近眼显示设备的FOV内的虚拟扩展屏。
计算组件，还被配置为从主显示设备获取可见虚拟扩展屏对应的扩展显示内 容。
计算组件，还被配置为将扩展显示内容发送给可透视近眼显示设备。
可透视近眼显示设备，被配置为在可见虚拟扩展屏中显示扩展显示内容。
本公开一示例性实施例还提供了一种扩展内容显示装置，能够实现本公开提供的扩展内容显示方法。该装置包括：处理器，以及用于存储处理器的可执行指令的存储器。其中，处理器被配置为：
获取可透视近眼显示设备的位姿信息以及可透视近眼显示设备所创建的n个虚拟扩展屏的位姿信息，n为正整数；
根据可透视近眼显示设备的位姿信息、可透视近眼显示设备的FOV和n个虚拟扩展屏的位姿信息，确定n个虚拟扩展屏中的可见虚拟扩展屏；其中，可见虚拟扩展屏是指在可透视近眼显示设备的FOV内的虚拟扩展屏；
获取可见虚拟扩展屏对应的扩展显示内容；
将扩展显示内容发送给可透视近眼显示设备，以便可透视近眼显示设备在可见虚拟扩展屏中显示扩展显示内容。
可选地，处理器被配置为：
向主显示设备发送内容获取请求，内容获取请求中包括可见虚拟扩展屏的标识信息；
接收主显示设备发送的可见虚拟扩展屏对应的扩展显示内容。
可选地，内容获取请求中还包括可见虚拟扩展屏的可见区域的位置信息；
相应地，处理器还被配置为：
接收主显示设备发送的可见虚拟扩展屏的可见区域对应的扩展显示内容。
可选地，处理器还被配置为：
根据可见虚拟扩展屏的可见区域的位置信息，从可见虚拟扩展屏对应的扩展显示内容中，提取可见虚拟扩展屏的可见区域对应的扩展显示内容；
将可见虚拟扩展屏的可见区域对应的扩展显示内容发送给可透视近眼显示设备。
可选地，处理器还被配置为：
获取对应于n个虚拟扩展屏中的第i个虚拟扩展屏的位姿调整参数，i为小于等于n的正整数；
根据第i个虚拟扩展屏的位姿信息和第i个虚拟扩展屏的位姿调整参数，确定第i个虚拟扩展屏调整后的位姿信息；
向可透视近眼显示设备发送第i个虚拟扩展屏调整后的位姿信息，以便可透视近眼显示设备调整第i个虚拟扩展屏的位姿。
可选地，处理器被配置为：
接收可透视近眼显示设备通过位姿传感器采集的传感器数据；
根据传感器数据确定可透视近眼显示设备的位姿信息。
可选地，所述可透视近眼显示设备的位姿信息包括：所述可透视近眼显示设备的当前位置相对于基准位置的距离偏移信息，以及所述可透视近眼显示设备的当前朝向相对于基准朝向的角度偏移信息。
图9是根据一示例性实施例示出的一种装置900的框图。例如，装置900可以是上文介绍的可透视近眼显示设备、计算组件或主显示设备。
参照图9，装置900可以包括以下一个或多个组件：处理组件902，存储器904，电源组件906，多媒体组件908，音频组件910，输入/输出(I/O)接口912，传感器组件914，以及通信组件916。
处理组件902通常控制装置900的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件902可以包括一个或多个处理器920来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件902可以包括一个或多个模块，便于处理组件902和其他组件之间的交互。例如，处理组件902可以包括多媒体模块，以方便多媒体组件908和处理组件902之间的交互。
存储器904被配置为存储各种类型的数据以支持在装置900的操作。这些数据的示例包括用于在装置900上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器904可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
电源组件906为装置900的各种组件提供电力。电源组件906可以包括电源管理系统，一个或多个电源，及其他与为装置900生成、管理和分配电力相关联 的组件。
多媒体组件908包括在所述装置900和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件908包括一个前置摄像头和/或后置摄像头。当装置900处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。
音频组件910被配置为输出和/或输入音频信号。例如，音频组件910包括一个麦克风(MIC)，当装置900处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器904或经由通信组件916发送。在一些实施例中，音频组件910还包括一个扬声器，用于输出音频信号。
I/O接口912为处理组件902和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件914包括一个或多个传感器，用于为装置900提供各个方面的状态评估。例如，传感器组件914可以检测到装置900的打开/关闭状态，组件的相对定位，例如所述组件为装置900的显示器和小键盘，传感器组件914还可以检测装置900或装置900一个组件的位置改变，用户与装置900接触的存在或不存在，装置900方位或加速/减速和装置900的温度变化。传感器组件914可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件914还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件914还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。
通信组件916被配置为便于装置900和其他设备之间有线或无线方式的通信。装置900可以接入基于通信标准的无线网络，如Wi-Fi，2G或3G，或它们的组合。在一个示例性实施例中，通信组件916经由广播信道接收来自外部广播 管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件916还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在示例性实施例中，装置900可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器904，上述指令可由装置900的处理器920执行以完成上述方法。例如，所述非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
一种非临时性计算机可读存储介质，当所述存储介质中的指令(或计算机程序)由装置900的处理器执行时，使得装置900能够执行上述扩展内容显示方法。
应当理解的是，在本文中提及的“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其它实施方案。本申请旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。

Claims (25)

  1. 一种扩展内容显示方法，其特征在于，所述方法包括：
    获取可透视近眼显示设备的位姿信息以及所述可透视近眼显示设备所创建的n个虚拟扩展屏的位姿信息，所述n为正整数；
    根据所述可透视近眼显示设备的位姿信息、所述可透视近眼显示设备的视场角FOV和所述n个虚拟扩展屏的位姿信息，确定所述n个虚拟扩展屏中的可见虚拟扩展屏；其中，所述可见虚拟扩展屏是指在所述可透视近眼显示设备的视场角FOV内的虚拟扩展屏；
    获取所述可见虚拟扩展屏对应的扩展显示内容；
    将所述扩展显示内容发送给所述可透视近眼显示设备，以便所述可透视近眼显示设备在所述可见虚拟扩展屏中显示所述扩展显示内容。
  2. 根据权利要求1所述的方法，其特征在于，所述获取所述可见虚拟扩展屏对应的扩展显示内容，包括：
    向主显示设备发送内容获取请求，所述内容获取请求中包括所述可见虚拟扩展屏的标识信息；
    接收所述主显示设备发送的所述可见虚拟扩展屏对应的扩展显示内容。
  3. 根据权利要求2所述的方法，其特征在于，所述内容获取请求中还包括所述可见虚拟扩展屏的可见区域的位置信息，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
    所述接收所述主显示设备发送的所述可见虚拟扩展屏对应的扩展显示内容，包括：
    接收所述主显示设备发送的所述可见虚拟扩展屏的可见区域对应的扩展显示内容。
  4. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    根据所述可见虚拟扩展屏的可见区域的位置信息，从所述可见虚拟扩展屏对应的扩展显示内容中，提取所述可见虚拟扩展屏的可见区域对应的扩展显示内容； 其中，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
    所述将所述扩展显示内容发送给所述可透视近眼显示设备，包括：
    将所述可见虚拟扩展屏的可见区域对应的扩展显示内容发送给所述可透视近眼显示设备。
  5. 根据权利要求1至4任一项所述的方法，其特征在于，所述方法还包括：
    获取对应于所述n个虚拟扩展屏中的第i个虚拟扩展屏的位姿调整参数，所述i为小于等于n的正整数；
    根据所述第i个虚拟扩展屏的位姿信息和所述第i个虚拟扩展屏的位姿调整参数，确定所述第i个虚拟扩展屏调整后的位姿信息；
    向所述可透视近眼显示设备发送所述第i个虚拟扩展屏调整后的位姿信息，以便所述可透视近眼显示设备调整所述第i个虚拟扩展屏的位姿。
  6. 根据权利要求1至4任一项所述的方法，其特征在于，所述获取可透视近眼显示设备的位姿信息，包括：
    接收所述可透视近眼显示设备通过位姿传感器采集的传感器数据；
    根据所述传感器数据确定所述可透视近眼显示设备的位姿信息。
  7. 根据权利要求1至4任一项所述的方法，其特征在于，所述可透视近眼显示设备位姿信息包括：所述可透视近眼显示设备的当前位置相对于基准位置的距离偏移信息，以及所述可透视近眼显示设备的当前朝向相对于基准朝向的角度偏移信息。
  8. 一种扩展内容显示方法，其特征在于，应用于扩展内容显示系统中，所述系统包括：增强现实可透视近眼显示设备、计算组件和主显示设备；所述方法包括：
    所述可透视近眼显示设备创建n个虚拟扩展屏，所述n为正整数；
    所述计算组件根据所述可透视近眼显示设备的位姿信息、所述可透视近眼显示设备的视场角FOV和所述n个虚拟扩展屏的位姿信息，确定所述n个虚拟扩 展屏中的可见虚拟扩展屏；其中，所述可见虚拟扩展屏是指在所述可透视近眼显示设备的FOV内的虚拟扩展屏；
    所述计算组件从所述主显示设备获取所述可见虚拟扩展屏对应的扩展显示内容；
    所述计算组件将所述扩展显示内容发送给所述可透视近眼显示设备；
    所述可透视近眼显示设备在所述可见虚拟扩展屏中显示所述扩展显示内容。
  9. 根据权利要求8所述的方法，其特征在于，所述计算组件从所述主显示设备获取所述可见虚拟扩展屏对应的扩展显示内容，包括：
    所述计算组件向所述主显示设备发送内容获取请求，所述内容获取请求中包括所述可见虚拟扩展屏的标识信息；
    所述主显示设备根据所述内容获取请求，生成所述可见虚拟扩展屏对应的扩展显示内容；
    所述主显示设备向所述计算组件发送所述扩展显示内容。
  10. 根据权利要求9所述的方法，其特征在于，所述内容获取请求中还包括所述可见虚拟扩展屏的可见区域的位置信息，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
    所述主显示设备根据所述内容获取请求，生成所述可见虚拟扩展屏对应的扩展显示内容，包括：
    所述主显示设备根据所述可见虚拟扩展屏的可见区域的位置信息，生成所述可见虚拟扩展屏的可见区域对应的扩展显示内容。
  11. 根据权利要求8所述的方法，其特征在于，所述方法还包括：
    所述计算组件根据所述可见虚拟扩展屏的可见区域的位置信息，从所述可见虚拟扩展屏对应的扩展显示内容中，提取所述可见虚拟扩展屏的可见区域对应的扩展显示内容；其中，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
    所述计算组件将所述扩展显示内容发送给所述可透视近眼显示设备，包括：
    所述计算组件将所述可见虚拟扩展屏的可见区域对应的扩展显示内容发送 给所述可透视近眼显示设备。
  12. 根据权利要求8至11任一项所述的方法，其特征在于，所述方法还包括：
    所述计算组件获取对应于所述n个虚拟扩展屏中的第i个虚拟扩展屏的位姿调整参数，所述i为小于等于n的正整数；
    所述计算组件根据所述第i个虚拟扩展屏的位姿信息和所述第i个虚拟扩展屏的位姿调整参数，确定所述第i个虚拟扩展屏调整后的位姿信息；
    所述计算组件向所述可透视近眼显示设备发送所述第i个虚拟扩展屏调整后的位姿信息；
    所述可透视近眼显示设备根据所述第i个虚拟扩展屏调整后的位姿信息，展示所述第i个虚拟扩展屏。
  13. 根据权利要求8至11任一项所述的方法，其特征在于，所述方法还包括：
    所述可透视近眼显示设备通过位姿传感器采集传感器数据，向所述计算组件发送所述传感器数据；
    所述计算组件根据所述传感器数据，确定所述可透视近眼显示设备的位姿信息。
  14. 根据权利要求8至11任一项所述的方法，其特征在于，所述方法还包括：
    所述主显示设备生成主屏幕对应的主显示内容，所述主屏幕是指所述主显示设备的屏幕；
    所述主显示设备在所述主屏幕中显示所述主显示内容。
  15. 根据权利要求8至11任一项所述的方法，其特征在于，
    所述计算组件集成于所述可透视近眼显示设备中；
    或者，
    所述计算组件集成于所述主显示设备中；
    或者，
    所述计算组件是独立于所述可透视近眼显示设备和所述主显示设备的装置。
  16. 一种扩展内容显示装置，其特征在于，所述装置包括：
    位姿获取模块，被配置为获取可透视近眼显示设备的位姿信息以及所述可透视近眼显示设备所创建的n个虚拟扩展屏的位姿信息，所述n为正整数；
    可见屏确定模块，被配置为根据所述可透视近眼显示设备的位姿信息、所述可透视近眼显示设备的视场角FOV和所述n个虚拟扩展屏的位姿信息，确定所述n个虚拟扩展屏中的可见虚拟扩展屏；其中，所述可见虚拟扩展屏是指在所述可透视近眼显示设备的FOV内的虚拟扩展屏；
    内容获取模块，被配置为获取所述可见虚拟扩展屏对应的扩展显示内容；
    内容发送模块，被配置为将所述扩展显示内容发送给所述可透视近眼显示设备，以便所述可透视近眼显示设备在所述可见虚拟扩展屏中显示所述扩展显示内容。
  17. 根据权利要求16所述的装置，其特征在于，所述内容获取模块，包括：
    请求发送单元，被配置为向主显示设备发送内容获取请求，所述内容获取请求中包括所述可见虚拟扩展屏的标识信息；
    内容接收单元，被配置为接收所述主显示设备发送的所述可见虚拟扩展屏对应的扩展显示内容。
  18. 根据权利要求17所述的装置，其特征在于，所述内容获取请求中还包括所述可见虚拟扩展屏的可见区域的位置信息，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
    所述内容接收单元，被配置为接收所述主显示设备发送的所述可见虚拟扩展屏的可见区域对应的扩展显示内容。
  19. 根据权利要求16所述的装置，其特征在于，所述装置还包括：
    内容提取模块，被配置为根据所述可见虚拟扩展屏的可见区域的位置信息，从所述可见虚拟扩展屏对应的扩展显示内容中，提取所述可见虚拟扩展屏的可见 区域对应的扩展显示内容；其中，所述可见虚拟扩展屏的可见区域是指所述可见虚拟扩展屏在所述可透视近眼显示设备的FOV内的显示区域；
    所述内容发送模块，被配置为将所述可见虚拟扩展屏的可见区域对应的扩展显示内容发送给所述可透视近眼显示设备。
  20. 根据权利要求16至19任一项所述的装置，其特征在于，所述装置还包括：
    参数获取模块，被配置为获取对应于所述n个虚拟扩展屏中的第i个虚拟扩展屏的位姿调整参数，所述i为小于等于n的正整数；
    位姿确定模块，被配置为根据所述第i个虚拟扩展屏的位姿信息和所述第i个虚拟扩展屏的位姿调整参数，确定所述第i个虚拟扩展屏调整后的位姿信息；
    位姿发送模块，被配置为向所述可透视近眼显示设备发送所述第i个虚拟扩展屏调整后的位姿信息，以便所述可透视近眼显示设备调整所述第i个虚拟扩展屏的位姿。
  21. 根据权利要求16至19任一项所述的装置，其特征在于，所述位姿获取模块，包括：
    数据接收单元，被配置为接收所述可透视近眼显示设备通过位姿传感器采集的传感器数据；
    位姿确定单元，被配置为根据所述传感器数据确定所述可透视近眼显示设备的位姿信息。
  22. 根据权利要求16至19任一项所述的装置，其特征在于，所述可透视近眼显示设备的位姿信息包括：所述可透视近眼显示设备的当前位置相对于基准位置的距离偏移信息，以及所述可透视近眼显示设备的当前朝向相对于基准朝向的角度偏移信息。
  23. 一种扩展内容显示系统，其特征在于，所述系统包括：可透视近眼显示设备、计算组件和主显示设备；
    所述可透视近眼显示设备，被配置为创建n个虚拟扩展屏，所述n为正整数；
    所述计算组件，被配置为根据所述可透视近眼显示设备的位姿信息、所述可透视近眼显示设备的视场角FOV和所述n个虚拟扩展屏的位姿信息，确定所述n个虚拟扩展屏中的可见虚拟扩展屏；其中，所述可见虚拟扩展屏是指在所述可透视近眼显示设备的FOV内的虚拟扩展屏；
    所述计算组件，还被配置为从所述主显示设备获取所述可见虚拟扩展屏对应的扩展显示内容；
    所述计算组件，还被配置为将所述扩展显示内容发送给所述可透视近眼显示设备；
    所述可透视近眼显示设备，被配置为在所述可见虚拟扩展屏中显示所述扩展显示内容。
  24. 一种扩展内容显示装置，其特征在于，所述装置包括：
    处理器；
    用于存储所述处理器的可执行指令的存储器；
    其中，所述处理器被配置为：
    获取可透视近眼显示设备的位姿信息以及所述可透视近眼显示设备所创建的n个虚拟扩展屏的位姿信息，所述n为正整数；
    根据所述可透视近眼显示设备的位姿信息、所述可透视近眼显示设备的视场角FOV和所述n个虚拟扩展屏的位姿信息，确定所述n个虚拟扩展屏中的可见虚拟扩展屏；其中，所述可见虚拟扩展屏是指在所述可透视近眼显示设备的FOV内的虚拟扩展屏；
    获取所述可见虚拟扩展屏对应的扩展显示内容；
    将所述扩展显示内容发送给所述可透视近眼显示设备，以便所述可透视近眼显示设备在所述可见虚拟扩展屏中显示所述扩展显示内容。
  25. 一种非临时性计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至7任一项所述方法的步骤。
PCT/CN2018/110138 2018-10-12 2018-10-12 扩展内容显示方法、装置、系统及存储介质 WO2020073334A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2018/110138 WO2020073334A1 (zh) 2018-10-12 2018-10-12 扩展内容显示方法、装置、系统及存储介质
CN201880001631.1A CN109496293B (zh) 2018-10-12 2018-10-12 扩展内容显示方法、装置、系统及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/110138 WO2020073334A1 (zh) 2018-10-12 2018-10-12 扩展内容显示方法、装置、系统及存储介质

Publications (1)

Publication Number Publication Date
WO2020073334A1 true WO2020073334A1 (zh) 2020-04-16

Family

ID=65713825

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/110138 WO2020073334A1 (zh) 2018-10-12 2018-10-12 扩展内容显示方法、装置、系统及存储介质

Country Status (2)

Country Link
CN (1) CN109496293B (zh)
WO (1) WO2020073334A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109831662B (zh) * 2019-03-22 2021-10-08 芋头科技（杭州）有限公司 Ar眼镜屏幕的实时画面投射方法、装置及控制器和介质
CN110069230A (zh) * 2019-04-24 2019-07-30 北京小米移动软件有限公司 扩展内容显示方法、装置及存储介质
CN111913674B (zh) * 2019-05-07 2024-07-26 广东虚拟现实科技有限公司 虚拟内容的显示方法、装置、系统、终端设备及存储介质
CN111913560B (zh) * 2019-05-07 2024-07-02 广东虚拟现实科技有限公司 虚拟内容的显示方法、装置、系统、终端设备及存储介质
CN111176520B (zh) * 2019-11-13 2021-07-16 联想(北京)有限公司 一种调整方法及装置
CN111708504A (zh) * 2020-06-16 2020-09-25 成都忆光年文化传播有限公司 一种扩展屏幕的显示方法
CN116107533A (zh) * 2023-02-14 2023-05-12 维沃移动通信有限公司 显示处理方法、装置和电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101059954A (zh) * 2006-04-20 2007-10-24 松下电器产业株式会社 显示装置以及显示方法
CN101726296A (zh) * 2009-12-22 2010-06-09 哈尔滨工业大学 空间机器人视觉测量、路径规划、gnc一体化仿真系统
CN102771121A (zh) * 2010-01-06 2012-11-07 佳能株式会社 照相机平台系统
US20130093646A1 (en) * 2011-10-18 2013-04-18 Reald Inc. Electronic display tiling apparatus and propagation based method thereof

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8538741B2 (en) * 2009-12-15 2013-09-17 Ati Technologies Ulc Apparatus and method for partitioning a display surface into a plurality of virtual display areas
KR102191867B1 (ko) * 2013-07-10 2020-12-16 엘지전자 주식회사 복수의 유저 인터페이스 포맷을 포함하는 헤드 마운티드 디스플레이 디바이스 및 그 제어 방법

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101059954A (zh) * 2006-04-20 2007-10-24 松下电器产业株式会社 显示装置以及显示方法
CN101726296A (zh) * 2009-12-22 2010-06-09 哈尔滨工业大学 空间机器人视觉测量、路径规划、gnc一体化仿真系统
CN102771121A (zh) * 2010-01-06 2012-11-07 佳能株式会社 照相机平台系统
US20130093646A1 (en) * 2011-10-18 2013-04-18 Reald Inc. Electronic display tiling apparatus and propagation based method thereof

Also Published As

Publication number Publication date
CN109496293B (zh) 2020-12-04
CN109496293A (zh) 2019-03-19

Similar Documents

Publication Publication Date Title
WO2020073334A1 (zh) 扩展内容显示方法、装置、系统及存储介质
EP3540571B1 (en) Method and device for editing virtual scene, and non-transitory computer-readable storage medium
US10832448B2 (en) Display control device, display control method, and program
US20220229524A1 (en) Methods for interacting with objects in an environment
US11231845B2 (en) Display adaptation method and apparatus for application, and storage medium
WO2022100712A1 (zh) 真实环境画面中虚拟道具的显示方法、系统及存储介质
CN107977083B (zh) 基于vr系统的操作执行方法及装置
US11029778B2 (en) Device and method for processing user input
CN111368114B (zh) 信息展示方法、装置、设备及存储介质
CN111610912B (zh) 应用显示方法、应用显示装置及存储介质
WO2022156703A1 (zh) 一种图像显示方法、装置及电子设备
CN110782532A (zh) 图像生成方法、生成装置、电子设备及存储介质
CN107797662B (zh) 视场角控制方法、装置及电子设备
US20230409192A1 (en) Device Interaction Method, Electronic Device, and Interaction System
KR20140102386A (ko) 디스플레이장치 및 그 제어방법
KR20140141419A (ko) 디스플레이장치 및 그 제어방법
WO2020226956A1 (en) Device, method, and graphical user interface for generating cgr objects
WO2023115777A1 (zh) 光标控制方法、装置、电子设备和存储介质
US20240070931A1 (en) Distributed Content Rendering
US11295505B2 (en) Animation generation using a target animation model and animation state parameters
GB2574780A (en) Electronic device and method for controlling same
CN107918514B (zh) 展示方法及装置、电子设备、计算机可读存储介质
US9619016B2 (en) Method and device for displaying wallpaper image on screen
CN106598217B (zh) 显示方法、显示装置和电子设备
US20220291890A1 (en) Method for interaction between devices and medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18936488

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18936488

Country of ref document: EP

Kind code of ref document: A1