CN114637400A - Visual content updating method, head-mounted display device assembly and computer readable medium - Google Patents

Visual content updating method, head-mounted display device assembly and computer readable medium Download PDF

Info

Publication number
CN114637400A
CN114637400A CN202210237065.2A CN202210237065A CN114637400A CN 114637400 A CN114637400 A CN 114637400A CN 202210237065 A CN202210237065 A CN 202210237065A CN 114637400 A CN114637400 A CN 114637400A
Authority
CN
China
Prior art keywords
visual content
head
mounted display
display device
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210237065.2A
Other languages
Chinese (zh)
Inventor
李治恒
张玉乐
王俊杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Companion Technology Co ltd
Original Assignee
Hangzhou Companion Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Companion Technology Co ltd filed Critical Hangzhou Companion Technology Co ltd
Priority to CN202210237065.2A priority Critical patent/CN114637400A/en
Publication of CN114637400A publication Critical patent/CN114637400A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K17/00—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
    • G06K17/0022—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations arrangements or provisious for transferring data to distant stations, e.g. from a sensing device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment

Abstract

Embodiments of the present disclosure disclose visual content update methods, head-mounted display device assemblies, and computer-readable media. One embodiment of the method comprises: in response to detecting that the connection state of the head mounted display device and the mobile device is connected, displaying an application interface in a display screen of the head mounted display device and displaying a touch panel interface in a touch screen of the mobile device; capturing visual content corresponding to an application interface displayed in the head-mounted display equipment to generate a visual content capturing image; storing the visual content capture image to the mobile device; in response to detecting a recognition login operation for a visual content capture image stored in the mobile device, the visual content in the display screen of the head-mounted display device is updated in accordance with the recognition login operation and the visual content capture image. The implementation method simplifies the flow of identifying the captured screen content and improves the user experience.

Description

Visual content updating method, head-mounted display device assembly and computer readable medium
Technical Field
Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a visual content updating method, a head-mounted display device assembly, and a computer-readable medium.
Background
Head-mounted display devices, such as AR (Augmented Reality) glasses or MR (Mixed Reality) glasses, may display visual content in a display screen for viewing by a user. When the head-mounted display device is connected with mobile devices such as a mobile phone, the mobile devices such as the mobile phone can be simulated into a touch pad to control the display content of the head-mounted display device, and at the moment, because the display interfaces of the mobile devices such as the mobile phone and the head-mounted display device are different, a user is difficult to complete the operation of two-dimensional code identification or the sharing operation in the head-mounted display device.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure propose a visual content updating method, a head-mounted display device assembly and a computer readable medium to solve the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide a visual content updating method applied to a head-mounted display device assembly including a head-mounted display device having a display screen and a mobile device having a touch screen, the method including: in response to detecting that the connection state of the head-mounted display device and the mobile device is connected, displaying an application interface in a display screen of the head-mounted display device and displaying a touch panel interface in a touch screen of the mobile device; in response to determining that a visual content capture function for the head-mounted display device is activated, capturing visual content corresponding to an application interface displayed in the head-mounted display device to generate a visual content capture image; storing the visual content capture image to the mobile device; in response to detecting an identification login operation for the visual content capture image stored in the mobile device, updating the visual content in the display screen of the head-mounted display device according to the identification login operation and the visual content capture image.
In a second aspect, some embodiments of the present disclosure provide a head mounted display device assembly comprising: a head-mounted display device having a display screen disposed thereon, the display screen configured to display an application interface; a mobile device having disposed thereon a touch-sensitive display screen configured to display a touch panel interface; one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors, cause the one or more processors to implement the method described in any of the implementations of the first aspect.
In a third aspect, some embodiments of the present disclosure provide a computer readable medium on which a computer program is stored, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect.
The above embodiments of the present disclosure have the following advantages: through the visual content updating method of some embodiments of the disclosure, the operation of two-dimensional code identification or the sharing operation can be completed in the head-mounted display device under the condition that the display interfaces of the mobile device such as a mobile phone and the head-mounted display device are different. Specifically, the reason why it is difficult for the user to complete the operation of two-dimensional code recognition or the sharing operation in the head-mounted display device is that: when the head-mounted display device is connected with mobile devices such as a mobile phone, the mobile devices such as the mobile phone can be simulated into a touch pad to control the display content of the head-mounted display device, and at the moment, the display interfaces of the mobile devices such as the mobile phone and the head-mounted display device are different. Based on this, the visual content updating method of some embodiments of the present disclosure stores the visual content capture image into the mobile device after generating the visual content capture image according to the visual content corresponding to the application interface displayed in the head-mounted display device. And then, directly updating the visual content in the display screen of the head-mounted display equipment according to the identification login operation and the visual content capture image. Therefore, under the condition that the display interfaces of mobile equipment such as a mobile phone and the like and head-mounted display equipment are different, the operation of two-dimensional code identification or the sharing operation can be completed in the head-mounted display equipment. The captured screen content recognition process is simplified, and user experience is improved.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.
FIG. 1 is a schematic diagram of head mounted display device components of some embodiments of the present disclosure;
FIG. 2 is a schematic diagram of one application scenario of the visual content update method of some embodiments of the present disclosure;
FIG. 3 is a flow diagram of some embodiments of a visual content update method according to the present disclosure;
FIG. 4 is a flow diagram of further embodiments of a visual content update method according to the present disclosure;
FIG. 5 is a schematic structural diagram of a head mounted display device assembly suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and the embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings. The embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 is a schematic diagram of head mounted display device components of some embodiments of the present disclosure.
As shown in fig. 1, the head mounted display device assembly 100 may include a head mounted display device 101 and a mobile device 102.
The head mounted display device 101 may include at least one display screen 1011. The display screen 1011 described above may be used to display visual content. Optionally, the head-mounted display device 101 may further include a camera 1012. In addition, the head mounted display device 101 includes a frame 1013 and a frame 1014. In some embodiments, the processing unit, memory, and battery of the head mounted display device 101 can be placed inside the frame 1013. In alternative implementations of some implementations, one or more of the processing unit, memory, and battery may also be integrated into another separate accessory (not shown) that is connected to the frame 1013 via a data cable.
The mobile device 102 may communicate with the head mounted display device 101 through a wireless connection or a wired connection.
It should be understood that the number of head mounted display devices and mobile devices in fig. 1 is merely illustrative. There may be any number of head mounted display devices and mobile devices, as desired for implementation.
Fig. 2 is a schematic diagram of one application scenario of the visual content updating method of some embodiments of the present disclosure.
In the application scenario of fig. 2, first, in response to detecting that the connection state of the head-mounted display device 201 and the mobile device 202 is connected, the computing device may display an application interface 203 in the display screen of the head-mounted display device 201 and a touch panel interface 204 in the touch screen of the mobile device 202. Then, in response to determining that the visual content capture function for the head-mounted display device 201 is activated, the computing device may capture the visual content corresponding to the application interface 203 displayed in the head-mounted display device to generate the visual content capture image 205. The computing device may then store the visual content capture image 205 to the mobile device 202. Finally, in response to detecting the identification login operation for the visual content capture image 205 stored in the mobile device 202, the computing device may update the visual content in the display screen of the head-mounted display device 201 according to the identification login operation and the visual content capture image 205.
The computing device may be hardware or software. When the computing device is hardware, it may be implemented as a distributed cluster of multiple servers or terminal devices, or as a single server or single terminal device, such as a head-mounted display device or a mobile device. When the computing device is embodied as software, it may be installed in the hardware devices enumerated above. It may be implemented, for example, as multiple software or software modules for providing distributed services, or as a single software or software module. And is not particularly limited herein.
It should be understood that the number of computing devices described above is merely illustrative. There may be any number of computing devices, as implementation needs dictate.
With continued reference to fig. 3, a flow 300 of some embodiments of a visual content update method according to the present disclosure is shown. The visual content updating method comprises the following steps:
step 301, in response to detecting that the connection state of the head-mounted display device and the mobile device is connected, displaying an application interface in a display screen of the head-mounted display device, and displaying a touch panel interface in a touch screen of the mobile device.
In some embodiments, the executing entity of the visual content updating method may display an application interface in a display screen of the head-mounted display device and a touch panel interface in a touch screen of the mobile device in response to detecting that the connection state of the head-mounted display device and the mobile device is connected. The head-mounted display device and the mobile device can be connected in a wired connection mode or a wireless connection mode. Such mobile devices include, but are not limited to: cell-phone, panel computer, intelligent bracelet, notebook computer etc.. The application interface displayed on the display screen of the head-mounted display device may be an interface of an application installed in the mobile device.
Thus, after the head-mounted display device is connected with the mobile device, the touch panel interface is displayed in the touch screen of the mobile device. Furthermore, a user wearing the head-mounted display device can conveniently control the display screen of the head-mounted display device to display an application interface through operation in the touch panel interface.
In response to determining that the visual content capture function for the head-mounted display device is activated, capturing visual content corresponding to the application interface displayed in the head-mounted display device to generate a visual content capture image, step 302.
In some embodiments, the execution subject may capture visual content corresponding to an application interface displayed in the head-mounted display device to generate a visual content capture image in response to determining that a visual content capture function for the head-mounted display device is activated. The head mounted display device may further include a visual content capture entity key. The above-described visual content capturing entity key may be provided on a frame of the head-mounted display device. It may be determined that the visual content capturing function of the head-mounted display device is activated when the visual content capturing entity key is pressed. Then, the pixel value of each pixel in the display screen of the head-mounted display device can be recorded, and a visual content capture image is obtained.
In some optional implementations of some embodiments, before the capturing of the visual content corresponding to the application interface displayed in the head-mounted display device to generate the visual content capture image in response to determining that the visual content capture function for the head-mounted display device is activated, the executing subject may further: in response to detecting the visual content capture voice command, activating the visual content capture function. The voice recognition method such as a template matching method, a random model method, a probabilistic grammar analysis method and the like can be used for carrying out voice recognition, so that the detection of the visual content capturing voice command is realized.
As an example, the visual content capture voice instruction described above may be a "glasses screenshot.
Optionally, the head-mounted display device further includes a camera. Before the above-mentioned in response to determining that the visual content capturing function for the head-mounted display device is activated, capturing the visual content corresponding to the application interface displayed in the head-mounted display device to generate the visual content capturing image, the executing body may further perform the following steps:
firstly, performing gesture recognition on image information acquired by the camera to obtain gesture recognition information. The image information collected by the camera may be gesture-recognized by using a CNN (Convolutional Neural Networks).
And secondly, in response to determining that the gesture recognition information represents visual content capture, activating the visual content capture function. The corresponding relationship between the gesture recognition information and the instruction may be preset. The gesture recognition information can be directly utilized to determine the instruction and meaning represented by the gesture recognition information, and then the corresponding instruction or function is executed.
Optionally, a visual content capturing key is displayed in a display screen of the head-mounted display device. Before the visual content capturing function for the head-mounted display device is determined to be activated and the corresponding visual content of the application interface displayed in the head-mounted display device is captured to generate a visual content capturing image, the executing body may further activate the visual content capturing function in response to detecting the selection operation of the visual content capturing key. The operation of selecting the visual content capture key may be a click operation, a swipe operation, or the like, which is applied to the display screen of the mobile device. The display screen of the head-mounted display device can also display control rays, and a user wearing the head-mounted display device can move the mobile device so as to control the movement of the control rays. And then, the selection of the visual content capture key displayed in the display screen of the head-mounted display equipment is realized through the control ray.
Step 303, store the visual content capture image to the mobile device.
In some embodiments, the execution principal may store the visual content capture image in an album in the mobile device.
And step 304, in response to detecting the identification login operation aiming at the visual content capture image stored in the mobile device, updating the visual content in the display screen of the head-mounted display device according to the identification login operation and the visual content capture image.
In some embodiments, the execution subject may update the visual content in the display screen of the head-mounted display device according to the recognition login operation and the visual content capture image in response to detecting the recognition login operation for the visual content capture image stored in the mobile device. The above-mentioned recognition login operation may be a voice instruction operation. Or may be a preset operation acting on the touch panel interface in the touch screen of the mobile device. The preset operation may be a double-click operation on the touch panel interface. The operation may be a sliding operation applied to the touch panel interface. In practice, the voice command operation and the preset operation may be set according to the actual application requirement, and are not limited herein. The visual content capture image may include images or information representing user information, such as a two-dimensional code and a fingerprint.
The executing body may perform information recognition processing on the visual content captured image stored in the mobile device after detecting the recognition login operation, to obtain user information included in the visual content captured image. Then, the recognized information is used to perform operations such as login and sharing. And displaying the operation result interface in the display screen of the head-mounted display equipment.
The above embodiments of the present disclosure have the following advantages: through the visual content updating method of some embodiments of the disclosure, the operation of two-dimensional code identification or the sharing operation can be completed in the head-mounted display device under the condition that the display interfaces of the mobile device such as a mobile phone and the head-mounted display device are different. Specifically, the reason why it is difficult for the user to complete the operation of two-dimensional code recognition or the sharing operation in the head-mounted display device is that: when the head-mounted display device is connected with mobile devices such as a mobile phone, the mobile devices such as the mobile phone can be simulated into a touch pad to control the display content of the head-mounted display device, and at the moment, the display interfaces of the mobile devices such as the mobile phone and the head-mounted display device are different. Based on this, the visual content updating method of some embodiments of the present disclosure stores the visual content capture image into the mobile device after generating the visual content capture image according to the visual content corresponding to the application interface displayed in the head-mounted display device. And then, directly updating the visual content in the display screen of the head-mounted display equipment according to the identification login operation and the visual content capture image. Therefore, under the condition that the display interfaces of mobile equipment such as a mobile phone and the like and head-mounted display equipment are different, the operation of two-dimensional code identification or the sharing operation can be completed in the head-mounted display equipment. The captured screen content recognition process is simplified, and user experience is improved.
With further reference to fig. 4, a flow 400 of further embodiments of a visual content update method is illustrated. The process 400 of the visual content updating method includes the following steps:
in response to detecting that the connection state of the head-mounted display device and the mobile device is connected, displaying an application interface in a display screen of the head-mounted display device and displaying a touch panel interface in a touch screen of the mobile device, step 401.
In some embodiments, the specific implementation manner and the technical effect of step 401 may refer to step 301 in those embodiments corresponding to fig. 3, which are not described herein again.
Step 402, in response to determining that a visual content capture function for the head-mounted display device is activated, determining the visual content rendered in real time in the application interface as the captured visual content, resulting in a visual content capture image.
In some embodiments, the execution subject may determine, as the captured visual content, the visual content rendered in real-time in the display application in response to determining that the visual content capture function for the head-mounted display device is activated, resulting in a visual content capture image.
As an example, the Render function may be utilized to obtain visual content rendered in real time in the display screen described above.
At step 403, the visual content capture image is stored to the mobile device.
In response to detecting the recognition login operation for the visual content capture image stored in the mobile device, the step 404 updates the visual content in the display screen of the head-mounted display device according to the recognition login operation and the visual content capture image.
In some embodiments, the specific implementation manner and technical effects of steps 403 and 404 can refer to steps 303 and 304 in those embodiments corresponding to fig. 3, which are not described herein again.
Step 405, in response to generating the visual content capture image, displaying a sharing and recognition frame in a display screen of the head-mounted display device, and generating a capture image operation instruction according to a selection operation of a user wearing the head-mounted display device on the sharing and recognition frame.
In some embodiments, the executing subject of the visual content updating method may display a sharing and recognition frame in a display screen of the head-mounted display device in response to generating the visual content capturing image, and generate a capturing image operation instruction according to a selection operation of a user wearing the head-mounted display device on the sharing and recognition frame. The captured image operation command may be an identification command or a sharing command. The sharing and identifying frame may include a sharing button and an identifying button. And if the user wearing the head-mounted display equipment selects a sharing button, generating a sharing instruction. And if the user wearing the head-mounted display equipment selects the identification button, generating an identification instruction.
And step 406, updating the visual content in the display screen of the head-mounted display device according to the visual content capture image and the capture image operation instruction.
In some embodiments, the executing body updates the visual content in the display screen of the head-mounted display device according to the visual content captured image and the captured image operation instruction, and may include the following steps:
in step 4061, in response to determining that the captured image operation instruction is a sharing instruction, a sharing page is presented in the display screen to share the visual content captured image.
The sharing page is used for displaying an identifier of at least one sharing participation application. The content of the sharing instruction and the identification instruction can be preset. Whether the captured image operation instruction is a sharing instruction can be determined according to the content of the captured image operation instruction.
In some optional implementation manners of some embodiments, in response to determining that the captured image operation instruction is a sharing instruction, the executing main body may display a sharing page in the display screen to share the visual content captured image, and include the following steps:
in a first step, at least one sharing participation application is determined.
Whether the application is a sharing participating application can be determined according to the configuration information of each application installed in the mobile device. And if the configuration information of the application comprises the information representing the receiving and sharing, the application is a sharing and participating application.
And secondly, generating a visual content updating instruction representing a display sharing page by utilizing the at least one sharing participation application. The content of the visual content updating instruction representing the display sharing page can be preset.
And thirdly, controlling the head-mounted display equipment to display the sharing page in the display screen by using the visual content updating instruction representing the display sharing page.
Therefore, the user wearing the head-mounted display device can select the sharing participation application displayed in the sharing page by displaying the sharing page. And then, sharing participation application selected by the visual content capture image sharing value is used.
In step 4062, in response to determining that the captured image operation instruction is the identification instruction and that the visual content captured image includes the two-dimensional code, a two-dimensional code skip page corresponding to the two-dimensional code is displayed in the display screen.
As an example, the two-dimensional code jump page may be a login page.
In some optional implementations of some embodiments, in response to determining that the captured image operation instruction is an identification instruction and that the visual content captured image includes a two-dimensional code, the executing body may display a two-dimensional code skip page corresponding to the two-dimensional code on the display screen, and may include:
and step one, identifying the two-dimension code in the visual content capture image to obtain two-dimension code identification information.
Optionally, the identifying the two-dimensional code in the visual content capture image by the executing body to obtain the two-dimensional code identification information may include the following steps:
the first substep is to pre-process the visual content captured image to obtain a binary image. Wherein, the pretreatment may include: graying processing, image filtering processing and binarization processing.
And a second substep, performing image searching pattern detection on the binary image to obtain a two-dimensional code pattern area.
And a third substep of performing projection transformation view correction processing and image finding pattern position correction processing on the two-dimensional code pattern area to obtain two-dimensional code identification information.
And secondly, determining the application associated with the two-dimension code as a target application according to the two-dimension code identification information. If the two-dimensional code identification information includes an application name or an application identifier, the application represented by the application name or the application identifier included in the two-dimensional code identification information may be determined as the application associated with the two-dimensional code and determined as the target application.
And thirdly, generating a visual content updating instruction for representing and displaying the two-dimension code skip page by using the two-dimension code identification information and the target application.
And fourthly, controlling the head-mounted display equipment to display the two-dimensional code skip page corresponding to the two-dimensional code in the display screen by using the visual content updating instruction for representing and displaying the two-dimensional code skip page.
As can be seen from fig. 4, the flow 400 of the visual content updating method in some embodiments corresponding to fig. 4 embodies automatic identification and sharing of the visual content capture image, compared to the description of some embodiments corresponding to fig. 3. Therefore, the scheme described in the embodiments can simplify the process of identifying and sharing the captured screen content, and further improve the user experience.
Referring now to FIG. 5, a schematic structural diagram of a head mounted display device assembly 500 suitable for use in implementing some embodiments of the present disclosure is shown. The head mounted display device assembly shown in fig. 5 is only one example and should not bring any limitations to the functionality and scope of use of the embodiments of the present disclosure.
As shown in fig. 5, the head mounted display device assembly 500 may include a processing device (e.g., central processing unit, graphics processor, etc.) 501 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage device 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the head mounted display apparatus assembly 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
Generally, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, and the like; and a communication device 509. The communication means 509 may allow the head mounted display device assembly 500 to communicate with other devices wirelessly or by wire to exchange data. While fig. 5 illustrates a head mounted display device assembly 500 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 5 may represent one device or may represent multiple devices as desired.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In some such embodiments, the computer program may be downloaded and installed from a network via the communication means 509, or installed from the storage means 508, or installed from the ROM 502. The computer program, when executed by the processing device 501, performs the above-described functions defined in the methods of some embodiments of the present disclosure.
It should be noted that the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: in response to detecting that the connection state of the head-mounted display device and the mobile device is connected, displaying an application interface in a display screen of the head-mounted display device and displaying a touch panel interface in a touch screen of the mobile device; in response to determining that a visual content capture function for the head-mounted display device is activated, capturing visual content corresponding to an application interface displayed in the head-mounted display device to generate a visual content capture image; storing the visual content capture image to the mobile device; in response to detecting an identification login operation for the visual content capture image stored in the mobile device, updating the visual content in the display screen of the head-mounted display device according to the identification login operation and the visual content capture image.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

Claims (12)

1. A visual content updating method is applied to a head-mounted display device assembly, the head-mounted display device assembly comprises a head-mounted display device with a display screen and a mobile device with a touch screen, and the visual content updating method comprises the following steps:
in response to detecting that the connection state of the head mounted display device and the mobile device is connected, displaying an application interface in a display screen of the head mounted display device and displaying a touch panel interface in a touch screen of the mobile device;
in response to determining that a visual content capture function for the head-mounted display device is activated, capturing visual content corresponding to an application interface displayed in the head-mounted display device to generate a visual content capture image;
storing the visual content capture image to the mobile device;
in response to detecting an identifying login operation for the visual content capture image stored in the mobile device, updating visual content in a display screen of the head-mounted display device according to the identifying login operation and the visual content capture image.
2. The method of claim 1, wherein the method further comprises:
displaying a sharing and identifying frame in a display screen of the head-mounted display device in response to the generation of the visual content capturing image, and generating a capturing image operation instruction according to the selection operation of a user wearing the head-mounted display device on the sharing and identifying frame, wherein the capturing image operation instruction is an identifying instruction or a sharing instruction;
and updating the visual content in the display screen of the head-mounted display equipment according to the visual content capture image and the capture image operation instruction.
3. The method of claim 1, wherein prior to the capturing visual content corresponding to the application interface displayed in the head-mounted display device to generate a visual content capture image in response to determining that the visual content capture function for the head-mounted display device is activated, the method further comprises:
activating the visual content capture function in response to detecting a visual content capture voice instruction.
4. The method of claim 1, wherein the head mounted display device further comprises a camera: and
before the capturing, in response to determining that the visual content capture function for the head-mounted display device is activated, visual content corresponding to an application interface displayed in the head-mounted display device to generate a visual content capture image, the method further includes:
performing gesture recognition on the image information acquired by the camera to obtain gesture recognition information;
activating the visual content capture function in response to determining that the gesture recognition information characterizes visual content capture.
5. The method of claim 1, wherein a visual content capture key is displayed in a display screen of the head mounted display device; and
before the capturing, in response to determining that the visual content capture function for the head-mounted display device is activated, visual content corresponding to an application interface displayed in the head-mounted display device to generate a visual content capture image, the method further comprises:
activating the visual content capture function in response to detecting a selection operation of the visual content capture button.
6. The method of claim 2, wherein the updating the visual content in the display screen of the head-mounted display device according to the visual content capture image and the capture image operation instruction comprises:
in response to determining that the captured image operation instruction is a sharing instruction, displaying a sharing page in the display screen to share the visual content captured image, wherein the sharing page is used for displaying an identifier of at least one sharing participation application;
in response to determining that the captured image operation instruction is an identification instruction and that the visual content captured image includes a two-dimensional code, a two-dimensional code skip page corresponding to the two-dimensional code is presented in the display screen.
7. The method of claim 1, wherein the capturing visual content corresponding to the application interface displayed in the head-mounted display device to generate a visual content capture image comprises:
and determining the visual content rendered in the application interface in real time as the captured visual content to obtain a visual content capturing image.
8. The method of claim 6, wherein the responsive to determining that the captured image manipulation instruction is an identification instruction and that a two-dimensional code is included in the visual content captured image, presenting a two-dimensional code jump page in the display screen corresponding to the two-dimensional code comprises:
identifying the two-dimension code in the visual content capture image to obtain two-dimension code identification information;
determining the application associated with the two-dimension code as a target application according to the two-dimension code identification information;
generating a visual content updating instruction representing and displaying a two-dimension code skip page by using the two-dimension code identification information and the target application;
and controlling the head-mounted display equipment to display the two-dimension code skip page corresponding to the two-dimension code in the display screen by using the visual content updating instruction for representing and displaying the two-dimension code skip page.
9. The method of claim 6, wherein the responsive to determining that the captured image manipulation instruction is a share instruction, presenting a share page in the display screen to share the visual content captured image comprises:
determining at least one sharing participant application;
generating a visual content updating instruction representing a display sharing page by utilizing the at least one sharing participation application;
and controlling the head-mounted display equipment to display the shared page in the display screen by using the visual content updating instruction of the representative display shared page.
10. The method of claim 8, wherein the identifying the two-dimensional code in the visual content capture image to obtain two-dimensional code identification information comprises:
preprocessing the visual content capture image to obtain a binary image, wherein the preprocessing comprises the following steps: graying, image filtering and binaryzation;
performing image searching graphic detection on the binary image to obtain a two-dimensional code graphic area;
and carrying out projection transformation view correction processing and image finding pattern position correction processing on the two-dimensional code pattern area to obtain two-dimensional code identification information.
11. A head-mounted display device assembly, comprising:
a head-mounted display device having a display screen disposed thereon, the display screen configured to display an application interface;
a mobile device having a touch-sensitive display screen disposed thereon, the touch-sensitive display screen configured to display a touch panel interface;
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-10.
12. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1-10.
CN202210237065.2A 2022-03-11 2022-03-11 Visual content updating method, head-mounted display device assembly and computer readable medium Pending CN114637400A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210237065.2A CN114637400A (en) 2022-03-11 2022-03-11 Visual content updating method, head-mounted display device assembly and computer readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210237065.2A CN114637400A (en) 2022-03-11 2022-03-11 Visual content updating method, head-mounted display device assembly and computer readable medium

Publications (1)

Publication Number Publication Date
CN114637400A true CN114637400A (en) 2022-06-17

Family

ID=81948298

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210237065.2A Pending CN114637400A (en) 2022-03-11 2022-03-11 Visual content updating method, head-mounted display device assembly and computer readable medium

Country Status (1)

Country Link
CN (1) CN114637400A (en)

Similar Documents

Publication Publication Date Title
CN110827378B (en) Virtual image generation method, device, terminal and storage medium
CN109993150B (en) Method and device for identifying age
CN110021052B (en) Method and apparatus for generating fundus image generation model
CN109981787B (en) Method and device for displaying information
CN110059623B (en) Method and apparatus for generating information
CN112015314B (en) Information display method and device, electronic equipment and medium
CN111459364B (en) Icon updating method and device and electronic equipment
CN111784712B (en) Image processing method, device, equipment and computer readable medium
US20240119082A1 (en) Method, apparatus, device, readable storage medium and product for media content processing
CN113392764A (en) Video processing method and device, electronic equipment and storage medium
CN114416259A (en) Method, device, equipment and storage medium for acquiring virtual resources
CN112102836B (en) Voice control screen display method and device, electronic equipment and medium
US11810336B2 (en) Object display method and apparatus, electronic device, and computer readable storage medium
CN113703704B (en) Interface display method, head-mounted display device, and computer-readable medium
CN111586295B (en) Image generation method and device and electronic equipment
CN114637400A (en) Visual content updating method, head-mounted display device assembly and computer readable medium
CN112418233A (en) Image processing method, image processing device, readable medium and electronic equipment
CN112817441A (en) Method and device for combining key and human eye identification
CN112308745A (en) Method and apparatus for generating information
CN110991312A (en) Method, apparatus, electronic device, and medium for generating detection information
CN112231023A (en) Information display method, device, equipment and storage medium
CN112269518B (en) Touch control method and device
CN112346630B (en) State determination method, device, equipment and computer readable medium
US20230376122A1 (en) Interface displaying method, apparatus, device and medium
CN112328340B (en) Model process display method, device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination