CN114416237A

CN114416237A - Display state switching method, device and system, electronic equipment and storage medium

Info

Publication number: CN114416237A
Application number: CN202111630842.1A
Authority: CN
Inventors: 池卓哲
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2021-12-28
Filing date: 2021-12-28
Publication date: 2022-04-29
Also published as: WO2023124972A1

Abstract

The embodiment of the disclosure relates to a display state switching method, a display state switching device, a display state switching system, electronic equipment and a storage medium, and relates to the technical field of augmented reality and virtual reality, wherein the display state switching method comprises the following steps: acquiring a display state of display equipment, wherein the display state is an augmented reality state containing virtual information and reality information; the virtual information comprises virtual audio and a virtual picture, and the reality information comprises real environment audio and a real picture; responding to interactive operation acted on audio equipment or display equipment, and switching the mode information of the audio equipment to determine target mode information; and adjusting the real environment audio in the reality information according to the target mode information, and adjusting the reality picture according to the target mode information so as to switch the augmented reality state and the virtual reality state. The technical scheme of the disclosure can improve the accuracy and comprehensiveness of mode switching.

Description

Display state switching method, device and system, electronic equipment and storage medium

The present disclosure relates to the field of augmented reality and virtual reality technologies, and in particular, to a display state switching method, a display state switching apparatus, a display state switching system, an electronic device, and a computer-readable storage medium.

Background

The virtual reality and the augmented reality can bring a strong sense of immersion to a user, so that the user can feel the digital world more truly, and infinite possibility is provided for the development of the digital world in the future. Virtual reality and augmented reality can only provide information at the visual level.

In the related art, some devices can provide real information on the premise of providing virtual information, but can only realize switching of screens. In the above manner, only partial switching of states can be performed, which has certain limitations, and thus, switching of different states cannot be accurately completed. Moreover, the operation steps are more, and the convenience is poorer.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Disclosure of Invention

The present disclosure is directed to a display state switching method and apparatus, an electronic device, and a storage medium, which overcome, at least to a certain extent, the problem that different states cannot be accurately switched due to limitations and defects of related technologies.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.

According to an aspect of the present disclosure, there is provided a display state switching method, including: acquiring a display state of display equipment, wherein the display state is an augmented reality state containing virtual information and reality information; the virtual information comprises virtual audio and a virtual picture, and the reality information comprises real environment audio and a real picture; responding to interactive operation acted on audio equipment or display equipment, and switching the mode information of the audio equipment to determine target mode information; and adjusting the real environment audio in the reality information according to the target mode information, and adjusting the reality picture according to the target mode information so as to switch the augmented reality state and the virtual reality state.

According to an aspect of the present disclosure, there is provided a display state switching apparatus including: the display device comprises a state acquisition module, a display module and a display module, wherein the state acquisition module is used for acquiring a display state of the display device, and the display state is an augmented reality state containing virtual information and reality information; the virtual information comprises virtual audio and a virtual picture, and the reality information comprises real environment audio and a real picture; the mode switching module is used for responding to the interactive operation acted on the audio equipment or the display equipment and switching the mode information of the audio equipment to determine target mode information; and the audio picture adjusting module is used for adjusting the real environment audio in the reality information according to the target mode information and adjusting the reality picture according to the target mode information so as to switch the augmented reality state and the virtual reality state.

According to an aspect of the present disclosure, there is provided a display state switching system including: the audio equipment is used for responding to the interactive operation to determine target mode information and playing the real environment audio in the real information adjusted according to the target mode information; a communication device for connecting the audio device and the display device; and the display equipment is connected with the audio equipment and used for responding to interactive operation to determine target mode information and displaying a real picture in the real information adjusted according to the target mode information so as to switch an augmented reality state and a virtual reality state.

According to an aspect of the present disclosure, there is provided an electronic device including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any one of the above display state switching methods via execution of the executable instructions.

According to an aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the display state switching method of any one of the above.

In the display state switching method, the display state switching apparatus, the display state switching system, the electronic device, and the computer-readable storage medium provided in the embodiments of the present disclosure, on one hand, the attribute information of the audio and the real image in the real environment is adjusted according to the target mode information switched by the audio device or the interactive mode on the display device, so that the limitation that two fixed mode switching can be performed only according to one dimension of the image in the related art is avoided, the switching of multiple modes between the augmented reality state and the virtual reality state is accurately achieved from the two dimensions of the audio and the video, the complete switching from the augmented reality state to the virtual reality state is achieved, and the comprehensiveness and the application range are increased. On the other hand, the real environment audio and the real picture are adjusted according to the target mode information, so that the user can adjust the real audio and the real picture according to actual requirements, mode switching is achieved, convenience and diversity of user operation are improved, universality is improved, and mode switching efficiency is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty.

Fig. 1 shows a schematic diagram of a display state switching method or a system architecture of a display state switching method to which an embodiment of the present disclosure may be applied.

Fig. 2 shows a schematic diagram of a related art device.

Fig. 3 schematically illustrates a schematic diagram of a display state switching method in an embodiment of the present disclosure.

Fig. 4 schematically shows a schematic diagram of an audio device in an embodiment of the disclosure.

Fig. 5 schematically illustrates a flow chart of adjusting real-world audio in an embodiment of the present disclosure.

Fig. 6 schematically illustrates a flow chart of adjusting a real picture in an embodiment of the present disclosure.

Fig. 7 schematically illustrates a comparative schematic diagram of state switching in an embodiment of the present disclosure.

Fig. 8 schematically illustrates an interface schematic of a pass-through mode in an embodiment of the present disclosure.

Fig. 9 schematically illustrates an interface diagram of a weak noise reduction mode in an embodiment of the present disclosure.

Fig. 10 schematically illustrates an interface diagram of a strong noise reduction mode in an embodiment of the present disclosure.

Fig. 11 schematically illustrates a block diagram of a display state switching apparatus in an embodiment of the present disclosure.

Fig. 12 schematically illustrates a block diagram of a display state switching system in an embodiment of the present disclosure.

FIG. 13 shows a schematic structural diagram of an electronic device suitable for use in implementing embodiments of the present disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The embodiment of the disclosure provides a display state switching method, which can be applied to an application scene for switching the mode of a display device. Fig. 1 is a schematic diagram illustrating a display state switching method and apparatus or a system architecture of a display state switching method and apparatus to which an embodiment of the present disclosure may be applied.

As shown in fig. 1, the system architecture 100 may include a display device 101, a network 102, and an audio device 103. The display device can be a terminal with a display screen, such as a smart phone, a computer, a tablet computer and the like. The display device is used for responding to the mode information of the interactive operation switching display device. Network 102 is used to provide a medium for a communication link between display device 101 and audio device 103. Network 102 may include various connection types, such as wired communication links, wireless communication links, and so forth. In the disclosed embodiment, the network 102 between the display device 101 and the audio device 103 may be a wired communication link, for example, a communication link may be provided through a serial connection line; or may be a wireless communication link provided through a wireless network. The audio device 103 may be a device for playing audio and triggering mode switching, for example, a headset.

In the embodiment of the disclosure, the display state of the display device is acquired. The display state may be an augmented reality state containing virtual information and reality information. Further, when the audio device or the display device receives the interactive operation, the target mode information is determined and the real environment audio in the real information is adjusted. Meanwhile, the real picture is adjusted according to the target mode information, so that mutual switching between the augmented reality state and the virtual reality state is realized.

The device 1 shown in fig. 2, panel a, is VR glasses that can provide a visual experience that the user is nearly fully immersed. Wherein the outside picture that a plurality of cameras of accessible VR outside shot maps to inside display device after the synthesis in, and the user can see external environment, superposes some virtual information even, forms an AR effect. The device 2 shown in fig. 2B has an augmented reality function, and the AR/VR mode of the virtual object can be switched by the top tag in the touch screen.

In order to solve the problems in the related art, an embodiment of the present disclosure provides a display state switching method. Next, a display state switching method in the embodiment of the present disclosure is described in detail with reference to fig. 3.

In step S310, a display state of a display device is obtained, where the display state is an augmented reality state including virtual information and reality information; the virtual information includes virtual audio and a virtual picture, and the reality information includes real environment audio and a real picture.

In the embodiments of the present disclosure, the display device may be any type of electronic device, for example, a mobile phone, a tablet computer, or a wearable device. The wearable device may be, for example, VR/AR glasses or the like. The display device is composed of a display screen, an environment understanding system and an interaction system. The display screen is arranged on one side close to the user and used for projecting the digital information calculated in the display equipment to the eyes of the user in real time. Based on this, the types of display screens include, but are not limited to, cell phone screens, flat screens, display devices where VR/AR glasses project towards the user's glasses.

And the environment understanding system is arranged on the side far away from the user, namely the side close to the external environment. The environment understanding system can be a plurality of cameras or 3D scanning sensors, and the environment understanding system can capture real environment information according to actual needs, and superimpose virtual information on the photographed real information based on SLAM (Simultaneous localization and mapping) technology to generate a display state of the display device.

The display device is provided with an interactive system outside the visual information, and the interactive system is used for executing interactive operation. The type of interaction may be determined in conjunction with the type of display device, and the type of display device may be different, as may the type of interaction. Based on this, the type of the interactive operation can be various types, thus increasing the diversity of the operation types.

The display state refers to a current display state of the display device, which may be an augmented reality state. Specifically, the environment understanding system captures reality information including real-environment audio and a real screen, and superimposes virtual information including virtual audio and a virtual screen on the reality information based on the SLAM technique to generate an augmented reality state. Thus, the display information also includes virtual information and reality information, and both the virtual information and the reality information include two dimensions of audio and picture.

With continued reference to fig. 3, in step S320, in response to an interactive operation acting on an audio device or a display device, mode information of the audio device is switched to determine target mode information.

In the embodiment of the present disclosure, the audio device may be a headphone, and specifically may be a noise reduction headphone, for example, the noise reduction headphone shown in fig. 4 may be an active noise reduction headphone. The active noise reduction earphone may include an in-ear noise reduction earphone or a head noise reduction earphone, which is not particularly limited herein. Active noise reduction headphones can achieve active noise reduction. Active noise reduction refers to the emission of diametrically opposed sound waves according to a given noise, which cancel each other when they meet, thereby isolating the noise. In the embodiment of the disclosure, the active noise reduction earphone collects an external noise signal through the microphone, and then sends a sound wave opposite to the noise in real time through the earphone, so as to offset the noise entering the ear.

The audio device may be used to play virtual audio and real-world audio, in addition to receiving interactive operations. The mode information of the audio device refers to its noise reduction mode. Referring to fig. 4, the noise reduction mode may include four modes of a strong noise reduction mode, a weak noise reduction mode, a pass-through mode, and off. The strong noise reduction mode refers to the mode that the reverse sound wave is highest in grade so as to obstruct the influence of external environment sound to the maximum extent and guarantee the maximum immersion. The weak noise reduction mode refers to a mixed mode, wherein the level of reverse sound waves in the weak noise reduction mode is weakened, and real audio of the external environment can be fuzzily heard when virtual audio can be heard clearly. Reverse sound wave not only closes in penetrating mode, moreover through mutually supporting of inside and outside microphone, can eliminate the passive noise reduction effect that the silica gel earplug brought even to the user is more audio information of collecting the surrounding environment directly perceived. In the transparent mode, the sound of the external environment can be clearly heard even if the noise reduction earphone is worn.

Mode switching of the display device can be achieved based on interactive operations on the audio device. For the audio device being a noise reducing headphone, the interaction may be determined according to the type of keys included in the noise reducing headphone. The key types in the audio equipment comprise a touch sensing area, a physical key and a physical knob. Thus, the interaction may include, but is not limited to, a first audio control operation on the touch sensitive area, a second audio control operation on the physical button, and a third audio control operation on the physical knob.

Upon receiving the interactive operation, mode information of the audio device corresponding to the interactive operation may be determined to complete the mode switching. In particular, the mode switching interaction may be accomplished in response to a first audio control operation on the touch sensitive area. The first audio control operation may be, for example, a single click, a double click, a continuous single click (triple click), a long press, or a swipe up and down. The correspondence between the first audio control operation and the mode may be: the single click is correspondingly started, the double click is in a strong noise reduction mode, the continuous single click is in a weak noise reduction mode, and the long click or the up-and-down sliding is in a transparent mode. The mode switching interaction may also be accomplished in response to a second audio control operation on the physical key. The number of physical keys may be one or more. When the number of the physical keys is plural, each mode corresponds to one physical key, and the second audio control operation may be a pressing operation on each physical key. When the number of the physical keys is one, mode switching can be completed by clicking, double clicking, continuously clicking and long pressing the physical keys. Namely, different mode switching is realized for different operation modes of the same physical key. For example, a single click is correspondingly turned on, a double click is in a strong noise reduction mode, a continuous single click is in a weak noise reduction mode, and a long press is in a transparent mode. Mode switching interaction may also be effected in response to a third audio control operation applied to the physical knob. The third audio control operation may be a clockwise rotation or a counterclockwise rotation of the physical knob. For example, clockwise rotation is used for the strong noise reduction mode and the weak noise reduction mode, and counterclockwise rotation is used for the pass-through mode. Besides, the interactive mode and the switching sequence of the switching mode may also be determined according to actual needs and personal preferences, which are not limited herein.

The display device may be any type of electronic device with a screen, such as a cell phone, tablet, VR/AR glasses. The display device is used for displaying picture information and receiving interactive operation. The picture information may include a virtual picture and a real picture. The interactive operation is used to switch the mode information, and the interactive operation may be specifically determined according to the type of the display device. Since the types of the display device may include a touch device, a wearable device, and may further include a peripheral device, the interactive operation may include, but is not limited to, a first screen control operation on a display screen of the touch device, a second screen control operation on a display screen of the wearable device, a third screen control operation on the peripheral device, and a fourth screen control operation on the peripheral device.

The first screen control operation may be a touch operation on a screen control in a display screen of the touch device. The picture control can be a UI control located at any position of the display screen for performing mode switching. The touch operation may include, but is not limited to, a click operation or a press operation, and the like. The second picture control operation may be a gesture operation on a picture control in a display screen of the wearable device. The gesture operation may be a 3D gesture, such as waving or clapping a hand, and the like. Mode switching interaction may be accomplished when the gesture operation is captured and understood by the environment understanding system. The second picture control operation is applicable to wearable devices (e.g., VR/AR glasses). The third screen control operation may be a pressing operation of a physical key in the peripheral device. The peripheral equipment is connected with the display equipment, and the picture control on the display screen of the display equipment is controlled through a physical key of the peripheral equipment. Specifically, the peripheral device may be an operation handle, and the type of the operation handle includes a hand-held handle and a wearable handle. The operating handle can output a ray pointer, and when the ray pointer is located on the picture control on the display screen, a user completes mode switching interaction by pressing a physical key on the operating handle connected with the display device, so that the method is suitable for wearable devices (VR/AR). The fourth screen control operation may be a pressing operation of a physical key in the peripheral device in conjunction with the user's sight. The peripheral equipment is connected with the display equipment, and the picture control on the display screen of the display equipment is controlled through a physical key of the peripheral equipment. Specifically, the display device worn on the head of the user is provided with an eye movement tracking device on one side close to the user, the eye movement tracking device positions the accurate sight direction of the user by capturing infrared reflection of pupils of the user, and a pointer which is in the same direction as the sight direction of the user can be output by capturing eyeball tracking of the user. When the pointer is located on the corresponding picture control on the display screen, if the fact that the user presses a physical key on an operation handle connected with the display equipment is detected, mode switching interaction is completed, and the method is suitable for wearable equipment (VR/AR).

In the embodiment of the disclosure, by acting on the interactive operation of the audio device and the display device, the display state of the display device can be switched to obtain the target mode information. The interactive operation of the display equipment and the audio equipment can be mutually supplemented, the limitation that the mode can be switched only through one mode is avoided, and the mode can be switched according to the interactive operation on other types of equipment when the display equipment or the audio equipment is inconvenient to operate, so that the convenience of mode switching is improved. Moreover, compared with the related art, the target mode information can include a plurality of different modes, not only a virtual reality state and an augmented reality state, but also a weak noise reduction mode, so that multi-mode switching can be realized, diversity and operation comprehensiveness are improved, and the application range is enlarged.

Continuing to refer to fig. 3, in step S330, the real-environment audio in the reality information is adjusted according to the target mode information, and the real picture is adjusted according to the target mode information to switch between the augmented reality state and the virtual reality state.

In the embodiment of the present disclosure, since the real information corresponding to each mode information is different, when the mode information of the display device is switched to determine the target mode information in response to the interactive operation acting on the audio device or the display device, the real picture and the real environment audio in the real information need to be adjusted simultaneously according to the target mode information. When the real environment audio is adjusted, the volume information of the real environment audio can be adjusted. When the real picture is adjusted, the display brightness and the definition of the real picture can be adjusted at the same time.

Fig. 5 schematically shows a flow chart for adjusting real-world audio, and referring to fig. 5, mainly includes the following steps:

in step S510, if the degree information is the first degree, adjusting the volume information to a maximum volume value;

in step S520, if the level information is the second level, the volume information is reduced to an intermediate volume value;

in step S530, if the level information is the third level, the volume information is adjusted to the minimum volume value.

In the embodiment of the present disclosure, after the target mode information is determined according to the interactive operation, the real environment audio in the reality information may be adjusted according to the degree information of the target mode information. The degree information refers to a proportion of the real information in the target mode information, and the first degree is greater than the second degree and greater than the third degree. The volume information is positively correlated with the level information. When the target mode information is the transparent mode, the degree information is the first degree, that is, the proportion occupied by the real information is the maximum. When the target mode information is the weak noise reduction mode, the degree information is a second degree, namely, the proportion occupied by the reality information is reduced. When the target mode information is the strong noise reduction mode, the degree information is a third degree, that is, the proportion occupied by the real information is the minimum. For real-environment audio, volume information of the real-environment audio may be adjusted.

Based on this, when the target mode information is the transparent mode, the volume information of the real environment audio can be increased to the maximum volume value because the proportion occupied by the real information is the maximum. When the target mode information is in the weak noise reduction mode, the volume information of the audio in the real environment can be reduced to the intermediate volume value because the occupied proportion of the real information is smaller than the maximum value. The intermediate volume value may be greater than the minimum volume value and less than the maximum volume value, and the intermediate volume value may be any value between 25% and 75% of the maximum volume value, such as 50%. When the target mode information is the strong noise reduction mode, the volume information of the real environment audio can be reduced to the minimum volume value, which can be 0, because the proportion occupied by the real information is minimum. In the strong noise reduction mode, only the virtual audio in the virtual information can be heard, and the real environment audio cannot be heard.

Fig. 6 schematically shows a flow chart for adjusting a real picture, and referring to fig. 6, the method mainly includes the following steps:

in step S610, if the degree information is a first degree, determining the display brightness as a maximum brightness value, and determining the definition as a maximum definition;

in step S620, if the degree information is a second degree, reducing the display brightness to an intermediate brightness value, and reducing the sharpness to obtain an intermediate sharpness;

in step S630, if the degree information is a third degree, the display brightness is reduced to a minimum brightness value, and the definition is reduced to a minimum definition.

In the embodiment of the present disclosure, the display brightness and the definition are positively correlated to the degree information. When the target mode information is in a transparent mode, the real information and the virtual information can be displayed simultaneously, and the occupation proportion of the real information can be larger than or equal to that of the virtual information. At this time, the ratio occupied by the real information is the largest, so that the display brightness of the real picture can be increased to the maximum brightness value, and the definition can be adjusted to the maximum definition. When the target mode information is the weak noise reduction mode, the display brightness of the audio in the real environment can be reduced to the middle brightness value and the definition can be reduced to the middle definition due to the fact that the occupied proportion of the real information is reduced. The intermediate brightness value may be greater than the minimum brightness value and less than the maximum brightness value, and the intermediate definition is greater than the minimum definition and less than the maximum definition. The intermediate brightness value may be any value between 25% and 75% of the maximum brightness value, for example 50%. The intermediate sharpness may be any value between 25% and 75% of the maximum sharpness. And, the adjustment degree of the display brightness and the definition may be the same or different. In this mode, a virtual screen and a real screen can be simultaneously displayed. When the target mode information is the strong noise reduction mode, the display brightness of the real picture can be reduced to the minimum brightness value and the definition can be reduced to the minimum definition due to the fact that the occupied proportion of the real information is the minimum, and the minimum brightness value and the minimum definition can be 0. In the strong noise reduction mode, only the virtual picture is displayed, and the real picture is not displayed. Since only the virtual information may be displayed without displaying the real information in the strong noise reduction mode, the display device is in a virtual reality state. It should be noted that, in the whole process of switching the mode information, the states of the virtual audio and the virtual picture included in the virtual information are kept unchanged, the volume information of the virtual audio may be a maximum volume value, the display brightness of the virtual picture may be a maximum brightness value, and the definition of the virtual picture may be a maximum definition.

The method and the device realize the conversion between the augmented reality state and the virtual reality state from two dimensions of audio and video by adjusting the attribute information of the audio and the real picture of the real environment according to the target mode information switched by the audio equipment. The limitation that different modes can be converted according to one dimension of a picture in the related technology is avoided, mode switching between different modes can be accurately realized from multiple dimensions, complete switching from an augmented reality state to a virtual reality state is realized, and comprehensiveness and an application range are increased. In addition, the audio and the real picture of the real environment are adjusted through interactive operation acting on the audio equipment or the display equipment, so that the accuracy of the adjustment of the audio and the picture can be improved, and the convenience and the comprehensiveness of user operation are also improved. In addition, the real audio and the real picture can be adjusted simultaneously, so that the problem of audio and video asynchronism is avoided, the synchronous switching of the audio and the picture can be realized, and the display effect is improved.

In the embodiment of the present disclosure, if the display device is in communication connection with the audio device through a bluetooth communication protocol in the communication device, and the camera on the display device has started the recording function, so as to transmit the real picture taken by the display device to the display screen. Meanwhile, the environment understanding system of the display device performs real-time three-dimensional modeling on the real environment through the SLAM technology, and superimposes the virtual information on the real picture so as to determine the display state of the display device as the augmented reality state. Based on this, a real picture in an augmented reality state and real environment audio can be displayed. In the whole switching process, the virtual audio and the virtual picture corresponding to the virtual information are kept unchanged, for example, the virtual audio may be a maximum volume value, and the virtual picture may keep a maximum brightness value and a maximum definition unchanged.

A comparison diagram of state switching is schematically shown in fig. 7, and referring to fig. 7, in terms of modes, the device 1 and the device 2 only provide two states of AR and VR, and a weak noise reduction mode is added to the embodiment of the present disclosure, in which the user can focus main attention on virtual information while feeling necessary real information. In the aspect of sound and picture synchronization, the device 1 and the device 2 only provide AR/VR mode switching of pictures, the dimension of sound is added, and meanwhile, the pictures and the sound are in one-to-one correspondence in different noise reduction modes, so that a multi-mode switching effect is provided for a user. In the aspect of an interactive mode, the device 1 only provides switching of VR glasses in cooperation with a pointing handle or a 3D gesture, the device 2 only provides switching of a screen touch mode, and linkage of pictures and sound is not considered during switching of the VR glasses and the 3D gesture. The disclosed embodiment provides a plurality of switching interactive modes: the switching interactive mode who contains screen touch-control, 3D gesture capture, directional handle, eye movement tracking + handle among the display device interactive system to and the switching interactive mode of touch-sensitive, button, knob on the earphone of falling the noise voluntarily, increased the variety of operation, improved the variety and the comprehensive of switching mode, increased the classification of mode information, can realize the complete switch-over of a plurality of modes.

And if the user needs to pay attention to the reality information, the target mode information is a transparent mode so as to reinforce the reality information. Referring to fig. 8, a user may switch to pass-through mode 801 through the interactive system of the display device or the interactive system of active noise reduction headphones 802. In the pass-through mode, the display device generates virtual information (e.g., a virtual puppy). The virtual information includes virtual audio 803 (sound of a puppy) and a virtual screen 804 (image of a puppy). And virtual information may be superimposed over the captured reality information, including real-environment audio 805 (real-environment sounds recorded in real-time) and a real-image 806 (e.g., a table, a couch, etc., therein). The display brightness and the definition of the real-world audio and the real-world picture are maximum, for example, 100%.

If the user needs to focus on the virtual information and focus on the real information, the target mode information is a weak noise reduction mode so as to weaken the real information. When the target mode information is the weak noise reduction mode, the augmented reality state is still maintained, but the weakened reality information is needed. Referring to fig. 9, a user may switch to a weak noise reduction mode 901 through an interactive system of a display device or an interactive system of an active noise reduction headphone 902. In the weak noise reduction mode, virtual audio 903 (sound of a puppy) and a virtual picture 904 (image of the puppy) may be superimposed on the captured real information, including real environment audio 905 (real environment sound recorded in real time) and a real picture 906 (e.g., a table, a sofa, etc. therein). In this case, the display brightness and the definition of the real-world audio and the real-world image are reduced, for example, 25% to 75%.

And if the user does not need to pay attention to the real information but only needs to pay attention to the virtual information, the target mode information is in a strong noise reduction mode so as to completely shield the real information. When the target mode information is in the strong noise reduction mode, the target mode information is in a virtual reality state, namely, the real information is not displayed. Referring to fig. 10, a user may switch to a strong noise reduction mode 1001 through an interactive system of a display device or an interactive system of an active noise reduction headphone 1002. In the strong noise reduction mode, virtual audio 1003 (sound of a puppy) and a virtual screen 1004 (image of the puppy) may be superimposed on the captured real information, including real-environment audio 1005 (real-environment sound recorded in real time) and a real screen 1006 (e.g., a table, a couch, etc., therein). The display brightness and the definition of the real environment audio and the real picture are both reduced to 0, and only the virtual information is displayed.

According to the technical scheme provided by the embodiment of the disclosure, the corresponding relation between the picture and the sound in different noise reduction states is considered, and the function of switching the AR/VR mode by fusing the visual auditory modes is realized. By matching with a multi-mode sound-picture transparent mode, the picture and sound information in the real environment can be enhanced, and even if a user wears a thick headset which can show the head and block the outside sound, the picture and the sound in the real environment can be seen clearly, so that the immersion feeling is improved. By matching with a multi-mode sound and picture weak noise reduction mode, a user can clearly see and hear the virtually generated picture and sound and weaken the picture and sound in a real environment, and the user can experience virtual information more in a focused manner while ensuring safe use. The mode of making an uproar falls by force in the sound picture of cooperation multimode, shields picture and sound in the real environment comprehensively, and the user can experience more and immerse, more focused VR effect. The synchronous switching of the picture and the sound is realized, and a convenient and comprehensive switching interaction mode is provided for switching the noise reduction modes of different levels.

In an embodiment of the present disclosure, a display state switching apparatus is further provided, as shown in fig. 11, the display state switching apparatus 1100 specifically includes:

a state obtaining module 1101, configured to obtain a display state of a display device, where the display state is an augmented reality state including virtual information and reality information; the virtual information comprises virtual audio and a virtual picture, and the reality information comprises real environment audio and a real picture;

a mode switching module 1102, configured to switch mode information of an audio device to determine target mode information in response to an interactive operation performed on the audio device or a display device;

an audio frame adjusting module 1103, configured to adjust the real environment audio in the reality information according to the target mode information, and adjust the reality frame according to the target mode information, so as to switch between the augmented reality state and the virtual reality state.

In an exemplary embodiment of the present disclosure, the adjusting the real-world environment audio in the reality information according to the target mode information includes: and the volume adjusting module is used for adjusting the volume information of the real environment audio according to the degree information corresponding to the target mode information.

In an exemplary embodiment of the present disclosure, the volume adjustment module includes: the first volume adjusting module is used for adjusting the volume information to be a maximum volume value if the degree information is a first degree; the second volume adjusting module is used for reducing the volume information to an intermediate volume value if the degree information is a second degree; and the third volume adjusting module is used for reducing the volume information to the minimum volume value if the degree information is the third degree.

In an exemplary embodiment of the present disclosure, the screen adjusting module includes: and the display information adjusting module is used for adjusting the display brightness and the definition of the real picture according to the degree information of the target mode information.

In an exemplary embodiment of the present disclosure, the display information adjusting module includes: the first display adjusting module is used for determining the display brightness as a maximum brightness value and determining the definition as a maximum definition if the degree information is a first degree; the second display adjusting module is used for reducing the display brightness to an intermediate brightness value and reducing the definition to obtain intermediate definition if the degree information is a second degree; and the third display adjusting module is used for reducing the display brightness to a minimum brightness value and reducing the definition to a minimum definition if the degree information is a third degree.

In an exemplary embodiment of the present disclosure, the apparatus further includes: and the synchronous adjustment module is used for synchronously adjusting the real environment audio and the real picture and keeping the virtual audio and the virtual picture contained in the virtual information unchanged.

It should be noted that the specific details of each module in the display state switching device have been described in detail in the corresponding display state switching method, and therefore are not described herein again.

In an embodiment of the present disclosure, a display state switching system is further provided, and as shown in fig. 12, the display state switching system 1200 specifically includes:

the audio device 1201 is configured to determine mode information in response to an interactive operation, and adjust a real environment audio included in the display state in the real information according to the mode information;

a communication device 1202 for connecting the audio device and the display device;

and the display device 1203 is connected with the audio device and is used for displaying an augmented reality state containing virtual information and reality information, responding to interactive operation, and adjusting a real picture in the reality information according to mode information so as to switch the augmented reality state and the virtual reality state.

In addition, the real-state switching system further includes: the peripheral device 1204 is connected to the display device through the communication device 1202, and is configured to receive the interoperation determination target mode information.

FIG. 13 shows a schematic diagram of an electronic device suitable for use in implementing exemplary embodiments of the present disclosure. The terminal of the present disclosure may be configured in the form of an electronic device as shown in fig. 13, however, it should be noted that the electronic device shown in fig. 13 is only one example, and should not bring any limitation to the functions and the use range of the embodiment of the present disclosure.

The electronic device of the present disclosure includes at least a processor and a memory for storing one or more programs, which when executed by the processor, cause the processor to implement the method of the exemplary embodiments of the present disclosure.

Specifically, as shown in fig. 13, the electronic device 1300 may include: processor 1310, internal memory 1321, external memory interface 1322, Universal Serial Bus (USB) interface 1330, charging management Module 1340, power management Module 1341, battery 1342, antenna 1, antenna 2, mobile communication Module 1350, wireless communication Module 1360, audio Module 1370, speaker 1371, receiver 1372, microphone 1373, headphone interface 1374, sensor Module 1380, display 1390, camera Module 1391, indicator 1392, motor 1393, button 1394, and Subscriber Identity Module (SIM) card interface 1395, etc. The sensor module 1380 may include a depth sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

It is to be understood that the illustrated structure of the embodiments of the present application does not limit the electronic device 1300 specifically. In other embodiments of the present application, the electronic device 1300 may include more or fewer components than illustrated, or combine certain components, or split certain components, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 1310 may include one or more processing units, such as: the processor 1310 may include an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a video codec, a digital signal processor, a baseband processor, and/or a Neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors. Additionally, a memory may be provided in processor 1310 for storing instructions and data. The display state switching method in the present exemplary embodiment may be performed by an application processor, a graphic processor, or an image signal processor, and may be performed by an NPU when the method involves neural network-related processing.

Internal memory 1321 may be used to store computer-executable program code, including instructions. The internal memory 1321 may include a program storage area and a data storage area. The external memory interface 1322 may be used for connecting an external memory card, such as a Micro SD card, to extend the memory capability of the electronic device 1300.

The communication function of the mobile terminal 1300 may be implemented by a mobile communication module, an antenna 1, a wireless communication module, an antenna 2, a modem processor, a baseband processor, and the like. The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. The mobile communication module may provide a mobile communication solution of 2G, 3G, 4G, 5G, etc. applied to the mobile terminal 1400. The wireless communication module may provide wireless communication solutions such as wireless lan, bluetooth, near field communication, etc. applied to the mobile terminal 200.

The display screen is used for realizing display functions, such as displaying user interfaces, images, videos and the like. The camera module is used for realizing shooting functions, such as shooting images, videos and the like. The audio module is used for realizing audio functions, such as audio playing, voice acquisition and the like. The power module is used for realizing power management functions, such as charging a battery, supplying power to equipment, monitoring the state of the battery and the like.

The present application also provides a computer-readable storage medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device.

A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable storage medium may transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The computer-readable storage medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method as described in the embodiments below.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

Furthermore, the above-described figures are merely schematic illustrations of processes included in methods according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims. It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.

Claims

1. A display state switching method is characterized by comprising the following steps:

acquiring a display state of display equipment, wherein the display state is an augmented reality state containing virtual information and reality information; the virtual information comprises virtual audio and a virtual picture, and the reality information comprises real environment audio and a real picture;

responding to interactive operation acted on audio equipment or display equipment, and switching the mode information of the audio equipment to determine target mode information;

and adjusting the real environment audio in the reality information according to the target mode information, and adjusting the reality picture according to the target mode information so as to switch the augmented reality state and the virtual reality state.

2. The method according to claim 1, wherein the adjusting the real-world environment audio in the real information according to the target mode information comprises:

and adjusting the volume information of the real environment audio according to the degree information corresponding to the target mode information.

3. The method according to claim 2, wherein the adjusting the volume information of the real-environment audio according to the degree information corresponding to the target mode information includes:

if the degree information is the first degree, adjusting the volume information to be the maximum volume value;

if the degree information is a second degree, reducing the volume information to an intermediate volume value;

and if the degree information is a third degree, reducing the volume information to a minimum volume value.

4. The method according to claim 1, wherein the adjusting the real screen according to the target mode information includes:

and adjusting the display brightness and the definition of the real picture according to the degree information of the target mode information.

5. The method according to claim 4, wherein the adjusting the display brightness and the sharpness of the real picture according to the degree information of the mode information includes:

if the degree information is a first degree, determining the display brightness as a maximum brightness value, and determining the definition as a maximum definition;

if the degree information is a second degree, reducing the display brightness to an intermediate brightness value, and reducing the definition to obtain an intermediate definition;

and if the degree information is a third degree, reducing the display brightness to a minimum brightness value and reducing the definition to a minimum definition.

6. The display state switching method according to claim 1, further comprising:

and synchronously adjusting the real environment audio and the real picture, and keeping the virtual audio and the virtual picture contained in the virtual information unchanged.

7. A display state switching apparatus, comprising:

the display device comprises a state acquisition module, a display module and a display module, wherein the state acquisition module is used for acquiring a display state of the display device, and the display state is an augmented reality state containing virtual information and reality information; the virtual information comprises virtual audio and a virtual picture, and the reality information comprises real environment audio and a real picture;

the mode switching module is used for responding to the interactive operation acted on the audio equipment or the display equipment and switching the mode information of the audio equipment to determine target mode information;

and the audio picture adjusting module is used for adjusting the real environment audio in the reality information according to the target mode information and adjusting the reality picture according to the target mode information so as to switch the augmented reality state and the virtual reality state.

8. A display state switching system, comprising:

the audio equipment is used for responding to the interactive operation to determine target mode information and playing the real environment audio in the real information adjusted according to the target mode information;

a communication device for connecting the audio device and the display device;

and the display equipment is connected with the audio equipment and used for responding to interactive operation to determine target mode information and displaying a real picture in the real information adjusted according to the target mode information so as to switch an augmented reality state and a virtual reality state.

9. An electronic device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the display state switching method of any one of claims 1-6 via execution of the executable instructions.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the display state switching method according to any one of claims 1 to 6.