WO2017066992A1

WO2017066992A1 - Head-mounted display device, and prompt information display system and method

Info

Publication number: WO2017066992A1
Application number: PCT/CN2015/092688
Authority: WO
Inventors: 李金鑫
Original assignee: 深圳市柔宇科技有限公司
Priority date: 2015-10-23
Filing date: 2015-10-23
Publication date: 2017-04-27
Also published as: CN107209375A

Abstract

Provided are a head-mounted display device, and a prompt information display system and method. The method comprises: determining whether a display device is in a multimedia playing state; if the display device is in the multimedia playing state, receiving sound collected by a plurality of microphones disposed at different positions of the display device; determining whether the received sound satisfies a pre-set condition; if the received sound satisfies the pre-set condition, determining the position of a speaker relative to a wearer according to the order in which the plurality of microphones collect the sound; setting prompt information that needs to be displayed according to the position of the speaker relative to the wearer and setting a display position of the prompt information in a display screen; and displaying the prompt information at the display position. In the present invention, when a wearer is immersed in a played multimedia content, the wearer may be prompted, if necessary, that someone wants to communicate with the wearer, and the wearer can quickly determine the approximate position of a speaker according to the prompt information.

Description

Head mounted display device, prompt information display system and method

Technical field

The present invention relates to the field of display devices, and in particular, to a head mounted display device, a prompt information display system, and a method.

Background technique

A head mounted display device (HMD) refers to a display device that can be worn on a head. HMD generally uses "near-eye optical system" to display multimedia information such as graphic pictures on a display screen a few centimeters from the eyeball. The HMD type currently includes a transmissive HMD that can see external and HMD display content and an immersive HMD that only sees HMD display content. Immersive HMDs have many entertaining applications, but when the wearer is immersed in the multimedia content played by HMD, it is difficult for others to communicate with the wearer. In addition, when communicating with other people, it is usually more troublesome for the wearer to take off the HMD.

Summary of the invention

The invention provides a head-mounted display device, a prompt information display system and a method. When the wearer is immersed in the multimedia content played by the head-mounted display device, the wearer can be prompted if necessary, and the person wants to wear The person communicates, and the wearer can know the approximate location of the speaker according to the prompt information.

The present invention provides a head mounted display device including a display screen, a camera disposed on a front side of the head mounted display device, a processor, and a memory, the processor controlling the display screen to display an image captured by the camera . The head mounted display device further includes: a plurality of microphones disposed at different positions of the head mounted display device. The memory stores computer executable program code. The processor invoking the computer executable program code, configured to: determine whether the head mounted display device is in a multimedia playing state; and when the head mounted display device is in the multimedia playing state, receive the a sound collected by the plurality of microphones; determining whether the received sound meets a preset condition; and when the received sound satisfies the preset condition, collecting sounds according to the plurality of microphones Determining sequentially the position of the speaker relative to the wearer; setting the prompt information to be displayed according to the position of the speaker relative to the wearer and setting the display position of the prompt information in the display screen; and displaying the prompt information in the Show location.

Further, the head mounted display device further includes a sensing unit for sensing a movement of the wearer, the processor invoking the computer executable program code for a preset time period during which the prompt information is displayed, And determining, according to an output of the sensing unit, that the wearer performs a preset action, turning on the camera, and controlling the display screen to display an image captured by the camera.

The present invention further provides a prompt information display system, which is applied to a head mounted display device, comprising: a state determination module, configured to determine whether the head mounted display device is in a multimedia play state; and a sound receiving module, configured to When the head mounted display device is in the multimedia playing state, receiving sounds collected by a plurality of microphones disposed at different positions of the head mounted display device; and a preset condition determining module configured to determine the received sound Whether the preset condition is met; the position determining module is configured to determine, according to the sequence of the sounds collected by the plurality of microphones, the position of the speaker relative to the wearer when the received sound meets the preset condition; And for setting a prompt information to be displayed according to the position of the speaker relative to the wearer and setting a display position of the prompt information in the display screen; and a display control module, configured to display the prompt information on the display position.

The present invention also provides a prompt information display method, which is applied to a head mounted display device, the method comprising: determining whether the head mounted display device is in a multimedia play state; wherein the head mounted display device is in the multimedia Receiving a sound collected by a plurality of microphones disposed at different positions of the head mounted display device in a playback state; determining whether the received sound satisfies a preset condition; and satisfying the pre-satisfaction in the received sound When the condition is set, determining the position of the speaker relative to the wearer according to the sequence of the sounds collected by the plurality of microphones; setting the prompt information to be displayed according to the position of the speaker relative to the wearer and setting the prompt information on the display screen a display position in the display; and displaying the prompt information at the display position.

The present invention also provides a computer readable storage medium storing computer readable program code, the computer readable program code being operative in a head mounted display device for determining whether the head mounted display device is a multimedia playing state; receiving, when the head mounted display device is in the multimedia playing state, sounds collected by a plurality of microphones disposed at different positions of the head mounted display device; determining whether the received sound is Satisfying a preset condition; the received sound satisfies the When the condition is preset, the position of the speaker relative to the wearer is determined according to the sequence of the sounds collected by the plurality of microphones; and the prompt information that needs to be displayed and the setting prompt information are displayed on the display according to the position of the speaker relative to the wearer. a display position in the screen; and displaying the prompt information at the display position.

In the present invention, when the wearer is immersed in the played multimedia content, when the sounds collected by the plurality of microphones satisfy the preset condition, the prompt information is displayed, which prevents unnecessary prompts from causing interference to the wearer. Moreover, the position of the speaker relative to the wearer can be determined according to the sequence of the sounds collected by the plurality of microphones, and the prompt information is displayed according to the position of the speaker relative to the wearer, so that the wearer can know the approximate position of the speaker according to the prompt information.

Further, in the present invention, during the preset time period in which the prompt information is displayed, when the wearer performs the preset action, the camera is turned on, and the image captured by the camera is displayed, so that the wearer communicates with the outside world. In this way, the wearer is given a choice to prevent unnecessary interference when the wearer is immersed in the multimedia content.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings to be used in the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without paying any creative work.

1 is a schematic diagram of a basic structure of a head mounted display device according to an embodiment of the present invention;

2 is a perspective view of a head mounted display device in accordance with an embodiment of the present invention;

3 is a schematic diagram of a basic structure of a prompt information display system according to an embodiment of the present invention;

4 to 8 are schematic diagrams showing prompt information in an embodiment of the present invention;

FIG. 9 is a flowchart of a method for displaying prompt information according to an embodiment of the present invention;

FIG. 10 is a flowchart of a method for displaying prompt information according to another embodiment of the present invention.

detailed description

The technical solution in the embodiment of the present invention will be clarified in the following with reference to the accompanying drawings in the embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS It is apparent that the described embodiments are only a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Referring to FIG. 1 , a head mounted display device 100 according to an embodiment of the present invention includes a display screen 11 , a camera 12 disposed on a front side of the head mounted display device 100 , and a plurality of cameras disposed at different positions of the head mounted display device 100 . The microphone 13, the processor 14, and the memory 15. The processor 14 can display the image captured by the camera 12 in the display screen 11. In this embodiment, a plurality of microphones 13 are disposed in different orientations of the head mounted display device 100. For example, as shown in FIG. 2, four intervals are evenly disposed on a horizontal line on the front of the head mounted display device 100. And at least one is disposed behind the four microphones 13, such as the headband of the head mounted display device 100, and the microphone 13 is used to determine that the sound source is from the front compared with any of the four front microphones 13. Still behind. Thus, when the speaker speaks with the wearer on either side of the wearer, the voice of the speaker can be collected by different microphones 13 in succession.

Referring to FIG. 3, the prompt information display system 200 provided by the embodiment of the present invention runs in the head mounted display device 100, and may include a state determining module 201, a sound receiving module 202, a preset condition determining module 203, a position determining module 204, and a setting. Module 205 and display control module 206. The above various modules may be stored in the memory 15 and called by the processor 16 to perform the prompt information display function of the head mounted display device 100. Each of the above modules may also be a software program that is cured in the processor 16.

The state determining module 201 is configured to determine whether the head mounted display device 100 is in a multimedia playing state. When in the multimedia playing state, it means that the wearer is immersed in the played multimedia content, and if others want to communicate with the wearer, it takes a lot of effort to attract the attention of the wearer.

Optionally, the state determining module 201 can determine whether the head mounted display device 100 is in a multimedia playing state by determining whether the camera 12 is in an open state. When the camera 12 is in the on state, it generally indicates that the wearer is communicating with the outside world using the head mounted display device 100. When the camera 12 is not in the on state, it generally indicates that the multimedia display device 100 is in the multimedia playing state, and the wearer is immersed in the played multimedia content.

Optionally, the state determining module 201 can determine whether the head mounted display device 100 is played in full screen. The multimedia content determines whether the head mounted display device 100 is in a multimedia playing state. When the multimedia content is played in full screen, it means that the head mounted display device 100 is in the multimedia playing state, and the wearer is immersed in the played multimedia content.

If the head mounted display device 100 is playing multimedia but is not playing in full screen mode, it is necessary to further determine whether the head mounted display device 100 is in the multimedia playing state by determining whether the camera 12 is in the on state. When it is further determined that the camera 12 is not turned on, it can be determined that the head mounted display device 100 is in a multimedia playing state, and the wearer is immersed in the played multimedia content. If it is further determined that the camera 12 is turned on, it can be judged that the wearer is watching the multimedia and also viewing the image captured by the camera 12, that is, may communicate with the outside world. At this time, it can be judged that the head mounted display device 100 is not in the multimedia playing state.

The sound receiving module 202 is configured to receive the sounds collected by the plurality of microphones 13 when determining that the head mounted display device 100 is in the multimedia playing state.

The preset condition determining module 203 is configured to determine whether the received sound meets the preset condition.

Optionally, the preset condition determining module 203 determines that the received sound satisfies the preset condition when the received sound satisfies any one of the following multiple conditions. The plurality of conditions include: the intensity of the received sound is greater than a preset value, the received sound includes at least one preset content, and the timbre of the received sound matches a preset timbre.

Optionally, the preset condition determining module 203 determines that the received sound meets the preset condition when the received sound satisfies any one of the following multiple conditions. The plurality of conditions include: the intensity of the received sound is greater than a preset value, the received sound includes at least one preset content, and the timbre of the received sound matches a preset timbre.

If at least the received sound includes at least one preset content as the determination condition, the content of interest to the wearer may be set in the head mounted display device 100 in advance, such as the name of the wearer and the wearer's interest. Sports and so on. If at least the timbre of the received sound is matched with a preset timbre as a determination condition, at least one timbre may be set in the head mounted display device 100 in advance, such as the timbre of the wearer's friend, the wearer's The tone of the child and so on.

The location determining module 204 is configured to, according to the multiple microphones, when the received sound meets the preset condition The order in which the wind 13 collects the sound determines the position of the speaker relative to the wearer.

In the present embodiment, the order in which the plurality of microphones 13 collect the sound is determined by first detecting an effective sound signal input by the speaker, and recording the time at which each of the microphones 13 collects the sound signal after detecting the effective sound signal. Next, the order in which the respective microphones 13 collect the sound signals is calculated based on the time at which the respective microphones 13 collect the sound signals.

In this embodiment, determining the position of the speaker relative to the wearer according to the sequence of the sounds collected by the plurality of microphones 13 may specifically be: determining that the microphone 13 disposed in the left front of the head mounted display device 100 first collects the sound, and determines to speak. The person is located at the left front of the wearer; when it is judged that the microphone 13 disposed at the right front of the head mounted display device 100 first collects the sound, it is determined that the speaker is located at the right front of the wearer; and so on, when the speaker is located around the wearer The position of the speaker relative to the wearer can be determined according to the order in which the sounds are collected by the plurality of microphones 13.

The setting module 205 is configured to set prompt information that needs to be displayed according to the position of the speaker relative to the wearer and set a display position of the prompt information in the display screen 11.

In the present embodiment, since the position of the camera 12 is fixed and the lens angle of view (FOV) is also fixed, it is possible to know whether the speaker is in the FOV of the camera 12 after obtaining the position of the speaker relative to the wearer, that is, Know if the speaker is in the camera's shooting range. The setting information to be displayed according to the position of the speaker relative to the wearer and the display position of the setting prompt information in the display screen 11 may be specifically: the setting module 205 determines that the speaker is located in the shooting range of the camera 12 according to the position of the speaker relative to the wearer. During the setting, the prompt information to be displayed is set as the first prompt information, and the display position of the first prompt information in the display screen 11 is set as the first display position according to the position of the speaker relative to the wearer, and the speaker is relatively worn according to the speaker. When the position of the person determines that the speaker is not within the shooting range of the camera 12, the prompt information to be displayed is set as the first prompt information or the second prompt information different from the first prompt information, and according to the position of the speaker relative to the wearer. The display position of the first prompt information in the display screen 11 is set to be a second display position different from the first display position. The first prompt information and the second prompt information may be graphics, arrows, characters, and the like. The difference between the first prompt information and the second prompt information may include a difference in color, graphics, and the like.

Optionally, the needs can be set according to the speaker's position relative to the wearer and the setting of the judgment condition. The prompt message displayed. For example, when at least the preset content is matched as the determination condition, the prompt information that needs to be displayed may be set to include the preset content. Then, the display color of the presentation information and the like are determined based on the position of the speaker relative to the wearer. It is also possible to first determine the display color of the prompt information according to the position of the speaker relative to the wearer, and then set the specific content included in the displayed prompt information according to the judgment condition. When the displayed prompt information includes the preset content, the wearer can quickly judge whether it is necessary to communicate with the wearer according to the prompt information.

In another example, if at least the timbre is matched as the judgment condition, the prompt information to be displayed may be set to include the identity information of the speaker, such as the name of the speaker or the relationship between the speaker and the wearer, and the like. In this way, the wearer can determine the identity of the current speaker according to the displayed prompt information to quickly determine whether it is necessary to communicate with the speaker.

The display control module 206 is configured to display the prompt information at the display position.

In order to better illustrate the embodiments of the present invention, the display of the prompt information will be exemplified below with reference to FIGS. 4 to 8. As shown in FIG. 4 to FIG. 6, when it is determined that the person in front of the wearer is speaking, it is determined that the speaker is located within the shooting range of the camera 12, and is displayed at the first display position 111 (upper left) of the display screen 11. The first prompt information, the first prompt information may be a text (such as a caller in front of the left), an arrow (pointing to the position of the speaker), or a graphic. As shown in FIG. 7 to FIG. 8, when it is determined that the left rear of the wearer is speaking, it is determined that the speaker is not within the shooting range of the camera 12, and at the second display position 112 of the display screen 11 (upper left corner) The second prompt information is displayed. The first prompt information may be the same as the second prompt information. The second prompt information (graphic) in FIG. 7 is the same as the first prompt information (graphic) in FIG. 6. The first prompt information may also be different from the second prompt information. Similarly, the second prompt information in FIG. 8 is different from any one of the first prompt information in FIG. 4 to FIG. 6. The figure shows that the same graphic has different filling effects, but is not limited thereto. Is the same graphics different colors, different graphics and so on. In the present embodiment, in the case where the speaker is located within the shooting range of the camera 12 and not in the shooting range of the camera 12, the same prompt information may be displayed at different display positions, or may be set. Different prompt information is displayed at different display positions, so that the wearer can quickly determine the approximate position of the speaker according to the display of the prompt information, thereby judging whether it is necessary to turn the head or turn around to see the speaker. In addition, in the shooting range where the speaker is located in the camera 12 When it is within the circumference, the first display position 111 on which the presentation information is displayed may be changed according to the approximate position of the speaker determined based on the determined approximate position of the speaker. For example, in the above embodiment, when the approximate position of the speaker is in the left front, the first display position 111 is located at the upper left of the display screen 11; and when it is determined that the approximate position of the speaker is directly in front, the first display position 111 is located on the display screen. Just above the top of the screen, the middle position above the display. Therefore, the first display position 111 may be an area between the upper display corner of the display screen 11 and the second display position of the upper right corner.

In this embodiment, when the wearer is immersed in the played multimedia content, when the sounds collected by the plurality of microphones 13 satisfy the preset condition, the prompt information is displayed, which prevents unnecessary prompts from causing interference to the wearer. Moreover, in the embodiment, the position of the speaker relative to the wearer can be determined according to the sequence of the sounds collected by the plurality of microphones 13, and when the position of the relative wearer is different, the prompt information is displayed at different positions, so that the wearer can According to the prompt information, the approximate position of the speaker is quickly determined, thereby judging whether it is necessary to turn the head or turn around to see the speaker.

In the present embodiment, the head mounted display device 100 may further include a sensing unit 16 for sensing the movement of the wearer. The prompt information display system 200 further includes a preset action determining module 207, configured to determine, according to the output of the sensing unit 15, whether the wearer performs a preset action within a preset time period during which the prompt information is displayed.

Alternatively, the sensing unit 15 may be a gyroscope for sensing the head motion of the wearer. When the sensing unit 15 is a gyroscope, the preset action may be a head, a nod, a left swing, a right swing, or a shaking head.

Alternatively, the sensing unit 15 may be a camera for capturing an eye or a light sensor for sensing eye movements. Under these conditions, the preset action can be two consecutive eyes or two seconds of closing the eyes.

The display control module 206 is further configured to: when the wearer performs the preset action within the preset time period during which the prompt information is displayed, turn on the camera 12, control the display screen 11 to display the image captured by the camera 12, and stop displaying the prompt information.

The display control module 206 can control the display screen 11 to display the image captured by the camera 12 in full screen, or display the avatar captured by the camera 12 and the multimedia content being played by the head mounted display device 100 in a split screen display manner.

The display control module 206 is further configured to: the wearer does not execute within a preset time period during which the prompt information is displayed. When the preset action is reached, the prompt message is stopped when the preset time period is reached.

In this embodiment, during the preset time period in which the prompt information is displayed, when the wearer performs the preset action, the camera 12 is turned on, and the image captured by the camera 12 is displayed so that the wearer can communicate with the outside world. In this way, the wearer is given a choice to prevent unnecessary interference when the wearer is immersed in the multimedia content.

Referring to FIG. 9, an embodiment of the present invention provides a method for displaying a prompt information, which is applied to the above-described head mounted display device 100, and includes the following steps:

Step 301: Determine whether the head mounted display device 100 is in a multimedia playing state. If no. Proceed to step 301, and if yes, go to step 302.

Optionally, whether the head mounted display device 100 is in a multimedia playing state can be determined by determining whether the camera 12 is in an open state. When the camera 12 is in the on state, it generally indicates that the wearer is communicating with the outside world using the head mounted display device 100. When the camera 12 is not in the on state, it generally indicates that the multimedia display device 100 is in the multimedia playing state, and the wearer is immersed in the played multimedia content.

Alternatively, whether the head mounted display device 100 is in the multimedia playing state can be determined by determining whether the head mounted display device 100 plays the multimedia content in full screen. When the multimedia content is played in full screen, it means that the head mounted display device 100 is in the multimedia playing state, and the wearer is immersed in the played multimedia content.

If the head mounted display device 100 is playing the multimedia content but is not playing in the full screen mode, it is further determined whether the head mounted display device 100 is in the multimedia playing state by determining whether the camera 12 is in the on state. When it is further determined that the camera 12 is not turned on, it can be determined that the head mounted display device 100 is in a multimedia playing state, and the wearer is immersed in the played multimedia content. If it is further determined that the camera 12 is turned on, it can be determined that the wearer is watching the multimedia content and also viewing the image captured by the camera 12, that is, may communicate with the outside world. At this time, it can be judged that the head mounted display device 100 is not in the multimedia playing state.

Step 302, receiving sounds collected by the plurality of microphones 13.

Step 303: Determine whether the received sound meets a preset condition. If yes, go to step 304. Otherwise, return to step 301.

Alternatively, when the received sound satisfies any one of the following conditions, it is determined that the received sound satisfies the preset condition. The plurality of conditions include: the intensity of the received sound is greater than a preset value, the received sound includes at least one preset content, and the timbre of the received sound matches a preset timbre.

Alternatively, when the received sound satisfies any one of the following multiple conditions, it is determined that the received sound satisfies a preset condition. The plurality of conditions include: the intensity of the received sound is greater than a preset value, the received sound includes at least one preset content, and the timbre of the received sound matches a preset timbre.

Step 304: Determine the position of the speaker relative to the wearer according to the order in which the plurality of microphones 13 collect the sound.

In this embodiment, determining the position of the speaker relative to the wearer according to the sequence of the sounds collected by the plurality of microphones 13 may specifically be: determining that the microphone 13 disposed in the left front of the head mounted display device 100 first collects the sound, and determines to speak. The person is located at the left front of the wearer; it is judged that the microphone 13 disposed at the right front of the head mounted display device 100 first collects the sound, and determines that the speaker is located at the right front of the wearer; and so on, when the speaker is located around the wearer, The position of the speaker relative to the wearer can be determined according to the order in which the sounds are collected by the plurality of microphones 13.

Step 305, setting prompt information and setting according to the position of the speaker relative to the wearer. The display position of the prompt information in the display screen 11.

In this embodiment, the prompt information that needs to be displayed according to the position of the speaker relative to the wearer and the display position of the setting prompt information in the display screen 11 may be specifically determined according to the position of the speaker relative to the wearer that the speaker is located at the camera 12. When the shooting range is within the range, the prompt information to be displayed is set as the first prompt information, and the display position of the first prompt information in the display screen 11 is set as the first display position according to the position of the speaker relative to the wearer, and according to the speaker When the position of the wearer is determined not to be within the shooting range of the camera 12, the prompt information to be displayed is set as the first prompt information or the second prompt information different from the first prompt information, and according to the speaker relative wearer The position setting of the second prompt information in the display screen 11 is a second display position different from the first display position. The first prompt information and the second prompt information may be graphics, arrows, characters, and the like. The difference between the first prompt information and the second prompt information may include a difference in color, graphics, and the like. In the present embodiment, in the case where the speaker is located within the shooting range of the camera 12 and not in the shooting range of the camera 12, the same prompt information may be displayed at different display positions, or may be set. Different prompt information is displayed at different display positions, so that the wearer can quickly determine the approximate position of the speaker according to the display of the prompt information, thereby judging whether it is necessary to turn the head or turn around to see the speaker.

Alternatively, the prompt information to be displayed may be set in common according to the position of the speaker relative to the wearer and the setting of the judgment condition. For example, when at least the preset content is matched as the determination condition, the prompt information that needs to be displayed may be set to include the preset content. Then, the display color of the presentation information and the like are determined based on the position of the speaker relative to the wearer. It is also possible to first determine the display color of the prompt information according to the position of the speaker relative to the wearer, and then set the specific content included in the displayed prompt information according to the judgment condition. When the displayed prompt information includes the preset content, the wearer can quickly judge whether it is necessary to communicate with the wearer according to the prompt information.

In step 306, the prompt information is displayed at the display position.

In this embodiment, when the wearer is immersed in the played multimedia content, when the sounds collected by the plurality of microphones 13 satisfy the preset condition, the prompt information is displayed, which prevents unnecessary prompts from causing interference to the wearer. Moreover, in the present embodiment, the position of the speaker relative to the wearer can be determined according to the order in which the plurality of microphones 13 collect the sound, and when the position relative to the wearer is different, different prompt information is displayed, so that the wearer can The prompt information quickly determines the approximate position of the speaker, thereby judging whether it is necessary to turn around or turn around to see the speaker.

Referring to FIG. 10, another embodiment of the present invention provides a method for displaying a prompt information, which is applied to the above-described head mounted display device 100, and includes the following steps:

Step 401: Determine whether the head mounted display device 100 is in a multimedia playing state. If no. Proceed to step 401, and if yes, perform step 402. Step 401 is similar to step 301, and details are not described herein again.

Step 402, receiving sounds collected by the plurality of microphones 13.

Step 403: Determine whether the received sound meets a preset condition. If yes, go to step 404, otherwise go back to step 401. Step 403 is similar to step 303, and details are not described herein again.

Step 404, determining the position of the speaker relative to the wearer according to the order in which the plurality of microphones 13 collect the sound. Step 404 is similar to step 304, and details are not described herein again.

Step 405, setting the prompt information to be displayed according to the position of the speaker relative to the wearer and setting the display position of the prompt information in the display screen 11. Step 405 is similar to step 305, and details are not described herein again.

In step 406, the prompt information is displayed at the display position.

Step 407: Determine whether the wearer performs a preset action within a preset time period during which the prompt information is displayed. If yes, go to step 408, otherwise go to step 409.

Specifically, it is determined according to the output of the sensing unit 15 whether the wearer performs a preset action. Alternatively, the sensing unit 15 may be a gyroscope for sensing the head motion of the wearer. When the sensing unit 15 is a gyroscope, the preset action may be a head, a nod, a left swing, a right swing, or a shaking head. Alternatively, the sensing unit 15 may be a camera for capturing an eye or a light sensor for sensing eye movements. Under such conditions, The preset action can be two consecutive eyes or two seconds of closing the eyes.

In step 408, the camera 12 is turned on, the display screen 11 is displayed to display the image captured by the camera 12, and the prompt information is stopped, and the process ends.

Specifically, the display screen 11 can be controlled to display the image captured by the camera 12 in full screen, or display the avatar captured by the camera 12 and the multimedia content being played by the head mounted display device 100 in a split screen display manner.

Step 409, after the preset time period elapses, the display of the prompt information is stopped, and the process returns to step 401.

The above is a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It is the scope of protection of the present invention.

Claims

A head mounted display device includes a display screen, a camera disposed on a front side of the head mounted display device, a processor, and a memory, the processor controlling the display screen to display an image captured by the camera, characterized by The head-mounted display device further includes:

a plurality of microphones disposed at different positions of the head mounted display device;

The memory stores computer executable program code;

The processor invokes the computer executable program code for:

Determining whether the head mounted display device is in a multimedia playing state;

Receiving a sound collected by the plurality of microphones when the head mounted display device is in the multimedia playing state;

Determining whether the received sound meets a preset condition;

And determining, when the received sound meets the preset condition, a position of the speaker relative to the wearer according to a sequence of the sounds collected by the plurality of microphones;

Setting the prompt information to be displayed according to the position of the speaker relative to the wearer and setting the display position of the prompt information in the display screen;

The prompt information is displayed at the display position.
A head mounted display device according to claim 1, wherein said processor calls said computer executable program code to determine when said received sound satisfies at least one of a plurality of conditions The received sound satisfies the preset condition, wherein the plurality of conditions include: the intensity of the received sound is greater than a preset value, the received sound includes a preset content, and the received The sound of the sound matches the preset sound.
A head mounted display device according to claim 1 or 2, wherein said processor calls said computer executable program code to determine said speaker is located according to said speaker's position relative to the wearer When the camera is within the shooting range, the prompt information to be displayed is set as the first prompt information, and the display position of the first prompt information in the display screen is set according to the position of the speaker relative to the wearer. When the first display position is determined, and when the speaker is determined to be within the shooting range of the camera according to the position of the speaker relative to the wearer, setting the prompt information to be displayed as the first prompt information or Different from the second prompt information of the first prompt information, And setting a display position of the second prompt information in the display screen to be a second display position different from the first display position according to the position of the speaker relative to the wearer.
A head mounted display device according to claim 1 or 2, wherein said head mounted display device further comprises a sensing unit for sensing the movement of the wearer, said processor invoking said computer executable program code And in the preset time period for displaying the prompt information, when determining, according to the output of the sensing unit, that the wearer performs a preset action, turning on the camera, and controlling the display screen to display the camera Captured image.
A prompt information display system is applied to a head-mounted display device, which comprises:

a state determining module, configured to determine whether the head mounted display device is in a multimedia playing state;

a sound receiving module, configured to receive sounds collected by a plurality of microphones disposed at different positions of the head mounted display device when the head mounted display device is in the multimedia playing state;

a preset condition determining module, configured to determine whether the received sound meets a preset condition;

a position determining module, configured to determine, according to a sequence of the sounds collected by the plurality of microphones, a position of the speaker relative to the wearer when the received sound meets the preset condition;

a setting module, configured to set, according to the position of the speaker relative to the wearer, prompt information to be displayed and a display position of the prompt information in the display screen;

And a display control module, configured to display the prompt information at the display position.
The prompt information display system according to claim 5, wherein the preset condition determination module determines that the received sound satisfies the sound when the received sound satisfies at least one of the following plurality of conditions The preset condition, wherein the plurality of conditions include: the intensity of the received sound is greater than a preset value, the received sound includes a preset content, and a timbre and a preset timbre of the received sound match.
The prompt information display system according to claim 5 or 6, wherein the setting module determines that the speaker is located within the shooting range of the camera according to the position of the speaker relative to the wearer The prompt information to be displayed is the first prompt information, and the display position of the first prompt information in the display screen is set as the first display position according to the position of the speaker relative to the wearer, and according to the speaking When the position of the person relative to the wearer determines that the speaker is not located within the shooting range of the camera, setting the prompt information to be displayed as the first prompt information or the second different from the first prompt information Prompt information and according to the speaker is relatively worn The position of the second prompt information in the display screen is a second display position different from the first display position.
The prompt information display system according to claim 5 or 6, wherein the prompt information display system further comprises a preset action determining module, configured to: according to the preset time period during which the prompt information is displayed, according to the The output of the sensing unit of the head mounted display device determines whether the wearer performs a preset action; the display control module is configured to: when the wearer performs a preset action, turn on the camera, and control the display The screen displays an image captured by a camera of the head mounted display device.
A prompt information display method is applied to a head mounted display device, and the method includes:

Determining whether the head mounted display device is in a multimedia playing state;

Receiving sounds collected by a plurality of microphones disposed at different positions of the head mounted display device when the head mounted display device is in the multimedia playing state;

Determining whether the received sound meets a preset condition;

And determining, when the received sound meets the preset condition, a position of the speaker relative to the wearer according to a sequence of the sounds collected by the plurality of microphones;

Setting the prompt information to be displayed according to the position of the speaker relative to the wearer and setting the display position of the prompt information in the display screen;

The prompt information is displayed at the display position.
The prompt information display method according to claim 9, wherein the determining whether the received sound meets a preset condition is specifically:

Determining whether the received sound satisfies at least one of the following conditions, wherein the plurality of conditions include: the intensity of the received sound is greater than a preset value, and the received sound includes a preset content And the timbre of the received sound matches the preset timbre;

When the received sound satisfies at least one of the following plurality of conditions, it is determined that the received sound satisfies the preset condition.
The prompt information display method according to claim 9 or 10, wherein the prompt information that needs to be displayed according to the position of the speaker relative to the wearer and the display position of the setting prompt information in the display screen are specific for:

When it is determined that the speaker is located within the shooting range of the camera according to the position of the speaker relative to the wearer, setting the prompt information that needs to be displayed as the first prompt information, and according to the Setting a position of the first prompt information in the display screen to a first display position of the speaker relative to the wearer; and

Setting the prompt information to be displayed as the first prompt information or different from the first when determining that the speaker is not located within the shooting range of the camera according to the position of the speaker relative to the wearer a second prompt information of the prompt information, and setting a display position of the second prompt information in the display screen to be a second display position different from the first display position according to the position of the speaker relative to the wearer.
The method for displaying a prompt information according to claim 9 or 10, wherein the method further comprises:

Determining, according to an output of the sensing unit of the head mounted display device, whether the wearer performs a preset action during a preset time period in which the prompt information is displayed;

When it is determined that the wearer performs a preset action, the camera is turned on, and the display screen is controlled to display an image captured by the camera.
A computer readable storage medium storing computer readable program code for execution in a head mounted display device for:

Determining whether the head mounted display device is in a multimedia playing state;

Receiving sounds collected by a plurality of microphones disposed at different positions of the head mounted display device when the head mounted display device is in the multimedia playing state;

Determining whether the received sound meets a preset condition;

And determining, when the received sound meets the preset condition, a position of the speaker relative to the wearer according to a sequence of the sounds collected by the plurality of microphones;

Setting the prompt information to be displayed according to the position of the speaker relative to the wearer and setting the display position of the prompt information in the display screen;

The prompt information is displayed at the display position.
The computer readable storage medium according to claim 13, wherein the determining whether the received sound meets a preset condition is specifically:

Determining whether the received sound satisfies at least one of the following conditions, wherein the plurality of conditions include: the intensity of the received sound is greater than a preset value, and the received sound includes a preset content And the timbre of the received sound matches the preset timbre;

When the received sound satisfies at least one of the following plurality of conditions, it is determined that the received sound satisfies the preset condition.
A computer readable storage medium according to claim 13 or 14, wherein said prompting information to be displayed and setting display position of said setting prompt information in said display screen are set according to said speaker's position relative to the wearer Specifically:

When it is determined that the speaker is located within the shooting range of the camera according to the position of the speaker relative to the wearer, setting the prompt information that needs to be displayed as the first prompt information, and according to the speaker relative to the wearer Position setting a display position of the first prompt information in the display screen as a first display position;

Setting the prompt information to be displayed as the first prompt information or different from the first when determining that the speaker is not located within the shooting range of the camera according to the position of the speaker relative to the wearer a second prompt information of the prompt information, and setting a display position of the second prompt information in the display screen to be a second display position different from the first display position according to the position of the speaker relative to the wearer.
The computer readable storage medium according to claim 13 or 14, wherein the computer readable program code is run in the head mounted device, and is further configured to:

Determining, according to an output of the sensing unit of the head mounted display device, whether the wearer performs a preset action during a preset time period in which the prompt information is displayed;

When it is determined that the wearer performs a preset action, the camera is turned on, and the display screen is controlled to display an image captured by the camera.