WO2021143574A1

WO2021143574A1 - Augmented reality glasses, augmented reality glasses-based ktv implementation method and medium

Info

Publication number: WO2021143574A1
Application number: PCT/CN2021/070281
Authority: WO
Inventors: 劳逸
Original assignee: Oppo广东移动通信有限公司
Priority date: 2020-01-16
Filing date: 2021-01-05
Publication date: 2021-07-22
Also published as: CN111273775A

Abstract

Augmented reality glasses, an augmented reality glasses-based KTV implementation method, and a storage medium. The augmented reality glasses comprise: an image capturing unit used for capturing a current scene image; a storage unit used for storing executable instructions; a processing unit used for executing the executable instructions so as to render a virtual image in the scene image according to a target video; a display unit used for displaying the virtual image; and an audio unit used for playing back a target audio, and receiving and playing back other audio, wherein the target video and the target audio are a video and audio of the same song. The augmented reality glasses implement a KTV function. The implementation process is very convenient, and the device costs are low. Moreover, the glasses may bring users a strong sense of interaction and immersion.

Description

Augmented reality glasses, KTV realization method and medium based on augmented reality glasses

This application claims the priority of the Chinese patent application filed on January 16, 2020, with the application number 202010057956.0, titled "Augmented Reality Glasses, KTV Implementation Method and Medium Based on Augmented Reality Glasses", all of the Chinese patent applications The content is incorporated herein by reference.

Technical field

The present disclosure relates to the technical field of virtual reality and augmented reality, and in particular to an augmented reality glasses, a KTV implementation method based on the augmented reality glasses, and a computer-readable storage medium.

Background technique

KTV is a kind of place that provides people with karaoke audio-visual equipment and sight-singing space. Going to KTV "K song" has become a popular way of leisure and entertainment.

Current KTV implementation methods are relatively limited, requiring people to go to KTV places, which is very inconvenient.

Summary of the invention

The present disclosure provides an augmented reality glasses, a KTV implementation method based on the augmented reality glasses, and a computer-readable storage medium, thereby overcoming the problem of the relatively limited implementation of KTV at least to a certain extent.

According to a first aspect of the present disclosure, there is provided an augmented reality glasses, including: a camera unit for shooting current scene images; a storage unit for storing executable instructions; a processing unit for executing the executable instructions , To render a virtual image in the scene image according to the target video; a display unit for displaying the virtual image; an audio unit for playing the target audio and receiving and playing other audio; wherein the target video And the target audio are the video and audio of the same song.

According to a second aspect of the present disclosure, there is provided a KTV implementation method based on augmented reality glasses, including: acquiring a current scene image; rendering a virtual image in the scene image according to a target video; and synchronously playing the virtual image and The target audio, the target video and the target audio are the video and audio of the same song; when the audio input by the user is received, the input audio is played.

According to a third aspect of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, realizes the above-mentioned KTV implementation method based on augmented reality glasses.

The technical solution of the present disclosure has the following beneficial effects:

According to the above-mentioned augmented reality glasses, the KTV implementation method based on augmented reality glasses, and the computer-readable storage medium, on the one hand, the KTV function is realized through the augmented reality glasses, so that users can wear augmented reality glasses for singing entertainment, and the realization process is very convenient, and Without the assistance of other equipment, the implementation cost is low and the space occupied is small. On the other hand, augmented reality glasses render and display virtual images based on real scene images, so that users are immersed in an audio-visual environment that combines virtual and real, with a strong sense of interaction and immersiveness, and a better user experience.

Description of the drawings

FIG. 1 shows a schematic diagram of the structure of augmented reality glasses in this exemplary embodiment;

FIG. 2 shows a structural diagram of augmented reality glasses in this exemplary embodiment;

Fig. 3 shows a flowchart of a KTV implementation method based on augmented reality glasses in this exemplary embodiment;

FIG. 4 shows a flowchart of rendering a virtual image in this exemplary embodiment;

Fig. 5 shows a schematic diagram of a computer-readable storage medium in this exemplary embodiment.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms, and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed To those skilled in the art. The described features, structures or characteristics can be combined in one or more embodiments in any suitable way. In the following description, many specific details are provided to give a sufficient understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be used. In other cases, the well-known technical solutions are not shown or described in detail in order to avoid overwhelming the crowd and obscure all aspects of the present disclosure.

In addition, the drawings are only schematic illustrations of the present disclosure, and are not necessarily drawn to scale. The same reference numerals in the figures denote the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in the form of software, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and/or processor devices and/or microcontroller devices.

With the development of electronic technology and the Internet, in order to make it easier for people to sing outside of KTV stores, KTV mobile cabinets and home KTV systems have appeared, and karaoke APPs (Applications) have also appeared on terminal platforms such as mobile phones. The way people sing. However, the equipment cost of KTV mobile cabinets and home KTV systems is relatively high, and the equipment occupies a large area, which is inconvenient to use; using K song APP on mobile phones and other terminals to sing, the interactivity and audiovisual experience are poor.

In view of the foregoing problems, exemplary embodiments of the present disclosure provide an augmented reality (Augmented Reality, AR) glasses. The following takes the augmented reality glasses 100 in FIG. 1 and FIG. 2 as an example to illustrate the internal unit structure thereof. Those skilled in the art should understand that the augmented reality glasses 100 may include more or fewer components than shown, or combine certain components, or disassemble certain components, or arrange different components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware. The interface connection relationship between the components is only shown schematically, and does not constitute a structural limitation on the augmented reality glasses 100. In other embodiments, the augmented reality glasses 100 may also adopt a different interface connection mode from that in FIG. 1, or a combination of multiple interface connection modes.

As shown in FIG. 1, the augmented reality glasses 100 may specifically include a camera unit 110, a storage unit 120, a processing unit 130, a display unit 140 and an audio unit 150.

The imaging unit 110 may be composed of components such as a lens and a photosensitive element. As shown in FIG. 2, it can be located in the middle of the two lenses. When the user wears the augmented reality glasses 100, the camera unit 110 faces directly in front of the user, and can capture a still image or video in front.

In this exemplary embodiment, when the KTV function of the augmented reality glasses 100 is activated, the camera unit 110 is used to take pictures of the current scene. For example, if the user is in a room, the camera unit 110 can take pictures of the scene in front of the user, including the room. Inside walls, floors, tables and chairs and other objects.

In an alternative implementation, as shown in FIG. 1, the camera unit 110 may include a depth camera 1101, for example, a TOF (Time Of Flight) camera, a binocular camera, etc., which can detect every scene in the image. Depth information of each part or each object (that is, the axial distance from the augmented reality glasses 100).

The storage unit 120 is used to store executable instructions, such as operating system code, program code, and data generated during the running of the program, such as image data taken by the camera unit 110, user data in the APP, and so on. Referring to FIG. 2, the storage unit 120 can be arranged in the mirror body between the two lenses, or can be arranged in other positions. The storage unit 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (Universal Flash Storage, UFS), and the like.

The processing unit 130 may include a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), a modem processor, and an image signal processor (Image Signal Processor). Processor, ISP), controller, video codec, digital signal processor (Digital Signal Processor, DSP), baseband processor, and/or Neural-Network Processing Unit (NPU), etc. Among them, different processors can be used as independent units or integrated into one processing unit. Referring to FIG. 2, the processing unit 130 may be arranged in the lens body between the two lenses, or may be arranged in other positions. The processing unit 130 may execute executable instructions on the storage unit 120 to execute corresponding program commands.

In this exemplary embodiment, when the KTV function of the augmented reality glasses 100 is activated, the processing unit 130 may obtain a scene image from the camera unit 110, and render a virtual image in the scene image according to the target video. Among them, the target video is the video of the currently playing song, usually a song MV (Music Video); the currently playing song can be selected by the user. For example, when the augmented reality glasses 100 runs the KTV APP, the song selection interface is displayed, and the user can click Stop, click and other operations to order a song, or it can be automatically selected by the system, such as random play mode or play according to a certain preset playlist. After determining the currently playing song, the processing unit 130 may obtain the corresponding target video, and render a virtual image in the scene image according to the video. The virtual image may be a combination of a video and a real scene image, for example, a picture in the video is embedded in the scene image for display.

In an optional implementation manner, the processing unit 130 may be used to: identify a plane in the scene image; render a virtual display screen on the plane, and display the target video in the virtual display screen. For example, if the wall plane is recognized in the scene image, a virtual display screen can be rendered on the wall plane to simulate the display screen in the KTV room. The present disclosure does not limit its style, size, etc.; then it is displayed in the virtual display screen The MV of the song currently being played makes the user feel like being in a KTV room with a strong sense of immersion.

It should be noted that when the scene image is a plane image, the processing unit 130 may perform plane recognition based on the color information, shape information, and texture information in the scene image, for example, recognize connected areas with consistent colors and consistent textures as a plane. When the camera unit 110 includes a depth camera 1101, the collected scene image can carry depth information, and the processing unit 130 can perform more accurate plane recognition based on the depth information, such as identifying connected areas with consistent colors, consistent textures, and continuously changing depths. Recognized as a plane.

Further, an image recognition unit 1301 can be provided in the processing unit 130 to recognize planes in the scene image. For example, the image recognition unit 1301 can run a Convolutional Neural Network (CNN) to process the scene image and output the plane recognition result. .

An image rendering unit 1302 may also be provided in the processing unit 130, specifically for rendering virtual images.

In an optional implementation manner, in addition to rendering the virtual playing video, other virtual elements can also be rendered, including but not limited to: rendering virtual lights in the scene image, simulating the lighting effects of the KTV room, and according to Switching for different song styles; rendering a virtual microphone or singing stage; rendering an animation of virtual characters dancing in front.

The display unit 140 can display images, videos, and the like. As shown in FIG. 2, the display unit 140 is generally set in the form of a lens. The user sees the real scene through the lens, and the processing unit 130 transmits the virtual image to the display unit 140 for display, so that the user can see the real and virtual superimposed images. Effect. Therefore, the display unit 140 needs to have a “see-through” function, which not only needs to see the real external world, but also sees virtual information, so as to realize the integration and “enhancement” of reality and virtuality. In an alternative embodiment, as shown in FIG. 1, the display unit 140 may include a micro display (Display) 1401 and a lens (Lens) 1402. The micro display screen 1401 is used to provide display content, and can be a self-luminous active device, such as a light-emitting diode panel, or a liquid crystal display with external light source illumination, etc.; the lens 1402 is used to make the human eye see the real scene, so as to The image and the virtual image are superimposed.

In this exemplary embodiment, after the processing unit 130 renders the virtual image, it is transmitted to the display unit 140 for display. In this way, the user can see the real scene and the virtual image in front at the same time through the augmented reality glasses 100.

The audio unit 150 can convert a digital audio signal into an analog audio signal for output, can also convert an analog audio input into a digital audio signal, and can also be used to encode and decode audio signals. In some embodiments, the audio unit 150 may be provided in the processing unit 130, or part of the functional modules of the audio unit 150 may be provided in the processing unit 130.

In this exemplary embodiment, the audio unit 150 is used to play target audio, and to receive and play other audio. The target video and target audio are the video and audio of the same song; other audio refers to the externally input audio besides the target audio. When the display unit 140 plays the virtual image, the audio unit 150 plays the target audio synchronously, so that the user can start singing according to the audio and video. The audio unit 150 may include a microphone 1501, also called "microphone", "microphone", etc., for receiving externally input audio, such as the user's singing voice or the audio of voice control instructions. For the user's singing voice, it can be converted to It is played out synchronously. The audio unit 150 may also include earphones, such as a bone conduction earphone 1502, which can realize high-quality audio playback, such as Dolby sound effects.

The present disclosure does not limit the number and positions of the microphone 1501 and the bone conduction earphone 1502. As shown in FIG. 2, in an optional implementation manner, the microphone 1501 may be located below the front end of the temple part and present an array distribution to better receive the user's voice. The bone conduction earphone 1502 can be located at the middle and rear ends of the temples, close to the position of the ear, which is conducive to achieving high-quality sound conduction.

In an optional implementation manner, when the user sings, the voice input by the user is received through the microphone 1501, and the processing unit 130 may optimize the voice and control the audio unit 150 to play the optimized voice. Optimization processing can include any one or more of the following: denoising (such as removing environmental noise), sound modification (such as appropriate beautification of pitch, vibrato, etc.), volume adjustment (adaptively adjusting the volume of the voice according to the volume of the audio playback) To prevent the voice volume from being too high or too low). Allow users to hear the optimized singing voice, and the experience is better.

In the process of singing, the user may need to perform related control of the song, such as cutting songs, ordering songs, pausing playback, continuing playback, increasing the volume, reducing the volume, etc. In order to facilitate user operations, the following provides two ways to implement user control:

(1) The camera unit 110 can also be used to take a user's gesture image, which can be a static image of one frame or a dynamic image of multiple consecutive frames. The processing unit 130 may also recognize the gesture control instruction corresponding to the gesture image, and execute the gesture control instruction. In this way, the user can perform gesture control during the singing process, such as sliding the finger up and down, swinging the finger left and right, etc. The camera unit 110 takes the gesture image and sends it to the processing unit 130. The processing unit 130 recognizes the gesture image, for example, through the image recognition unit. 1301 runs the convolutional neural network to obtain the gesture recognition result, and then determines the gesture control instruction according to the preset correspondence between the gesture and the control instruction, and executes the gesture control instruction.

(2) The user can also input voice control instructions through the microphone 1501 of the audio unit 150, for example, the user speaks "cut song" into the microphone 1501. Then the processing unit 130 recognizes the voice control instruction input by the user, and executes the voice control instruction if it recognizes the "cut song" instruction. In an optional implementation manner, a voice recognition unit 1303 may be provided in the processing unit 130, and a machine learning model of voice recognition is run to process the user's voice and output the recognition result. Generally, a specific evoked voice can be set. When the user speaks the evoked voice, the voice control function is awakened, and at other times, the voice input by the user is played as a singing voice.

Whether it is a gesture control instruction or a voice control instruction, operations such as switching songs, ordering songs, pausing playback, continuing playback, increasing volume, and decreasing volume can be implemented, which are not limited in the present disclosure.

As shown in FIG. 1, the augmented reality glasses 100 may also include a communication unit 160, which may provide wireless local area networks (Wireless Local Area Networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) networks), Bluetooth (Bluetooth) , BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared Technology (Infrared, IR) and other wireless communication solutions The solution is to connect the augmented reality glasses 100 to the Internet or to form a connection with other devices. Therefore, the communication unit 160 can be used to obtain the target video and target audio. For example, the communication unit 160 is connected to the Internet, and when the user orders a song, the corresponding video and audio are searched and downloaded from the Internet, or the communication unit 160 is connected to Wi-Fi. The song library is automatically updated, and the video and audio are stored in the storage unit 120 in advance. When the user sings, the video and audio files can be directly read from the storage unit 120.

In an alternative embodiment, the communication unit 160 can be used to access a wireless network, and if other devices also access the wireless network, the augmented reality glasses 100 can play the target audio synchronously with other devices; when the augmented reality glasses When any device in the 100 and other devices receives the voice input by the user, it can be synchronized to all devices in the wireless network to realize simultaneous KTV for multiple people. Further, if other devices connected to the wireless network are also augmented reality glasses, each augmented reality glasses can also display virtual images synchronously, so as to give the user a strong sense of trial and listening interaction.

In an optional implementation manner, the augmented reality glasses 100 may further include a sensor unit 170, which is composed of different types of sensors and is used to implement different functions. For example, the 6DOF (Degree Of Freedom) sensor 1701 can detect the posture information of the augmented reality glasses 100; the pressure sensor 1702 is used to sense pressure signals and can convert the pressure signals into electrical signals; the fingerprint sensor 1703 is used to detect the user's fingerprints Data to achieve user identity verification and other functions; the air pressure sensor 1704 is used to measure air pressure, which can calculate the altitude and assist in positioning and navigation; and so on.

In an optional implementation manner, the augmented reality glasses 100 may further include a USB (Universal Serial Bus, Universal Serial Bus) interface 180, which complies with the USB standard specification, and may specifically be a MiniUSB interface, a MicroUSB interface, a USBTypeC interface, and the like. The USB interface 180 can be used to connect a charger to charge the augmented reality glasses 100, can also connect to a headset, play audio through the headset, and can also be used to connect to other electronic devices, such as a computer, a peripheral device, and the like.

In an optional implementation manner, the augmented reality glasses 100 may further include a charging management unit 190, configured to receive a charging input from the charger to charge the battery 1901. Among them, the charger can be a wireless charger or a wired charger. In some wired charging implementations, the charging management unit 190 may receive the charging input of the wired charger through the USB interface 180. In some implementations of wireless charging, the charging management unit 190 may receive a wireless charging input through the wireless charging coil of the augmented reality glasses 100. While charging the battery 1901, the charging management unit 190 can also supply power to the device.

Exemplary embodiments of the present disclosure also provide a KTV implementation method based on augmented reality glasses. Referring to FIG. 3, the method may include the following steps S310 to S340:

Step S310, acquiring the current scene image;

In step S320, a virtual image is rendered in the scene image according to the target video;

In step S330, the virtual image and the target audio are played synchronously;

Step S340: When the input audio is received, the audio is played.

Among them, the current scene image can be captured by the camera unit of the augmented reality glasses. When rendering the virtual image according to the target video of the currently playing song, a time stamp can be added to the virtual image so that the time stamp corresponds to the frame time stamp of the video. The target audio and target video are the audio and video of the same song. The target audio itself also has a timestamp, so the virtual image and the target audio can be played synchronously. The two start playing at the same time and keep the timestamps synchronized to ensure that the virtual image seen by the user is synchronized with the music heard. When the user sings, the augmented reality glasses can receive the input audio and play the audio, so that the user's singing voice is superimposed with the target audio to achieve the effect of K song.

It should be noted that the rendering of the virtual image and the playing of the virtual image can be a simultaneous process, that is, the process is played while rendering. Playing the target audio and playing the user's voice are two separate processes. The target audio needs to be played according to the timestamp, while the voice is generally played instantly, that is, it is processed and played immediately after being received.

In an optional implementation manner, referring to FIG. 4, step S320 may include the following steps S401 and S402:

Step S401, identifying a plane in the scene image;

In step S402, the virtual display screen is rendered on the above-mentioned plane, and the target video is displayed in the virtual display screen.

For example, if the wall plane is recognized in the scene image, a virtual display screen can be rendered on the wall plane to simulate the display screen in the KTV room. The present disclosure does not limit its style, size, etc.; then it is displayed in the virtual display screen The target video makes the user feel as if they are in a KTV room and has a strong sense of immersion.

When the scene image is a plane image, plane recognition can be performed based on the color information, shape information, and texture information in the scene image, for example, a connected area with the same color and texture is recognized as a plane. When the scene image is a depth image, more accurate plane recognition can be performed based on the depth information, for example, a connected area with consistent color, consistent texture, and continuously changing depth can be recognized as a plane.

Further, step S320 may also include: rendering virtual lights in the scene image. For example, the lighting effect of a KTV room can be simulated, and the lighting style can be adjusted according to the style of the song. In addition, it is also possible to render a virtual microphone or a singing stage, and render an animation of a virtual character dancing in front, which is not limited in the present disclosure.

In an optional implementation manner, the KTV implementation method may further include the following steps:

Obtain the user's gesture image;

The gesture control instruction corresponding to the gesture image is recognized, and the gesture control instruction is executed.

When it is recognized that the voice input by the user is a voice control instruction, the voice control instruction is executed.

Wherein, the aforementioned gesture control instructions and voice control instructions may include any one or more of the following instructions: cut songs, order songs, pause playing, continue playing, increase volume, and decrease volume.

In an optional implementation manner, step S340 may include: receiving the voice input by the user, performing optimization processing on it, and playing it. Optimization processing can include: noise removal, sound modification, volume adjustment, etc.

In an optional implementation manner, the target video and target audio may be searched on the Internet, or the target video and target audio may be searched in a local pre-stored file library.

In an optional implementation manner, when the augmented reality glasses are connected to the wireless network, they can play the target audio synchronously with other devices connected to the wireless network; when any of the augmented reality glasses and other devices receive user input The voice can be synchronized to all devices in the wireless network to achieve simultaneous KTV for multiple people. Further, if other devices connected to the wireless network are also augmented reality glasses, each augmented reality glasses can also display virtual images simultaneously.

In summary, in this exemplary embodiment, based on the above-mentioned augmented reality glasses and KTV implementation method, on the one hand, the KTV function is realized through the augmented reality glasses, so that the user can wear the augmented reality glasses for singing entertainment, and the implementation process is very convenient. And without the assistance of other equipment, the implementation cost is low and the space occupied is small. On the other hand, augmented reality glasses render and display virtual images based on real scene images, so that users are immersed in an audio-visual environment that combines virtual and real, with a strong sense of interaction and immersiveness, and a better user experience.

Exemplary embodiments of the present disclosure also provide a computer-readable storage medium, which can be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to make the terminal device Perform the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

As shown in FIG. 5, a program product 500 for implementing the above method according to an exemplary embodiment of the present disclosure is described. It may adopt a portable compact disk read-only memory (CD-ROM) and include program codes, and may be used in a terminal Running on equipment, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device, or device.

The program product can adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Type programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

The computer-readable signal medium may include a data signal propagated in baseband or as a part of a carrier wave, and readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than a readable storage medium, and the readable medium may send, propagate, or transmit a program for use by or in combination with the instruction execution system, apparatus, or device.

The program code contained on the readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

The program code for performing the operations of the present disclosure can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural programming. Language-such as "C" language or similar programming language. The program code can be executed entirely on the user's computing device, partly on the user's device, executed as an independent software package, partly on the user's computing device and partly executed on the remote computing device, or entirely on the remote computing device or server Executed on. In the case of a remote computing device, the remote computing device can be connected to a user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (for example, using Internet service providers). Business to connect via the Internet).

Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described here can be implemented by software, or can be implemented by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or on the network , Including several instructions to make a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the exemplary embodiment of the present disclosure.

Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method, or a program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "Circuit", "Module" or "System".

In addition, the above-mentioned drawings are merely schematic illustrations of the processing included in the method according to the exemplary embodiment of the present disclosure, and are not intended for limitation. It is easy to understand that the processing shown in the above drawings does not indicate or limit the time sequence of these processings. In addition, it is easy to understand that these processes can be executed synchronously or asynchronously in multiple modules, for example.

It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of a module or unit described above can be further divided into multiple modules or units to be embodied.

Those skilled in the art will easily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptive changes of the present disclosure. These variations, uses, or adaptive changes follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field that are not disclosed in the present disclosure. . The description and the embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

It should be understood that the present disclosure is not limited to the precise structure that has been described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is limited only by the appended claims.

Claims

An augmented reality glasses, characterized by comprising:

Camera unit, used to shoot the current scene image;

The storage unit is used to store executable instructions;

A processing unit, configured to execute the executable instruction to render a virtual image in the scene image according to the target video;

A display unit for displaying the virtual image;

Audio unit, used to play target audio, and receive and play other audio;

Wherein, the target video and the target audio are the video and audio of the same song.
The augmented reality glasses according to claim 1, wherein the camera unit is also used to take a user's gesture image;

The processing unit is further configured to recognize a gesture control instruction corresponding to the gesture image, and execute the gesture control instruction.
The augmented reality glasses according to claim 1, wherein the processing unit is further configured to recognize a voice control instruction input by a user, and execute the voice control instruction.
The augmented reality glasses according to claim 2 or 3, wherein the gesture control instruction or voice control instruction includes any one or more of the following:

Cut song, order song, pause playing, continue playing, increase volume, decrease volume.
The augmented reality glasses according to any one of claims 1 to 4, wherein the augmented reality glasses further comprise:

The communication unit is used to obtain the target video and the target audio.
The method according to claim 5, wherein the communication unit is further configured to access a wireless network, so that the augmented reality glasses and other devices connected to the wireless network can play the target audio synchronously.
The augmented reality glasses according to any one of claims 1 to 4, wherein the processing unit is further configured to optimize the voice input by the user, and control the audio unit to play the optimized voice.
The method according to claim 7, wherein the optimization processing includes any one or more of the following: noise removal, sound modification, and volume adjustment.
The augmented reality glasses according to any one of claims 1 to 4, wherein the audio unit comprises a bone conduction headset, and the bone conduction headset is provided on a temple of the augmented reality glasses.
The augmented reality glasses according to any one of claims 1 to 4, wherein the camera unit comprises a depth camera.
A KTV implementation method based on augmented reality glasses, characterized in that it includes:

Get the current scene image;

According to the target video, render a virtual image in the scene image;

Synchronously playing the virtual image and target audio, where the target video and the target audio are the video and audio of the same song;

When the input audio is received, the input audio is played.
The method according to claim 11, wherein the rendering a virtual image in the scene image according to the target video comprises:

Identifying a plane in the scene image;

A virtual display screen is rendered on the plane, and the target video is displayed in the virtual display screen.
The method of claim 12, wherein the identifying a plane in the scene image comprises:

When the scene image is a plane image, the connected areas with the same color and the same texture in the scene image are identified as one plane.
The method of claim 12, wherein the identifying a plane in the scene image comprises:

When the scene image is a depth image, a connected area in the scene image that has the same color, the same texture, and continuously changes in depth is identified as one of the planes.
The method according to claim 12, wherein said rendering a virtual image in said scene image according to the target video, further comprising:

Render virtual lights in the scene image.
The method according to claim 11, wherein the method further comprises:

Obtain the user's gesture image;

The gesture control instruction corresponding to the gesture image is recognized, and the gesture control instruction is executed.
The method according to claim 11, wherein the method further comprises:

When it is recognized that the voice input by the user is a voice control instruction, the voice control instruction is executed.
The method according to claim 16 or 17, wherein the gesture control instruction or voice control instruction includes any one or more of the following:

Cut song, order song, pause playing, continue playing, increase volume, decrease volume.
The method according to claim 11, wherein when receiving the input audio, playing the input audio comprises:

Receiving the voice input by the user, and playing the voice after optimization processing;

The optimization processing includes any one or more of the following:

Noise removal, sound modification, volume adjustment.
A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 11 to 19 when the computer program is executed by a processor.