CN111261133A

CN111261133A - Singing processing method and device, electronic equipment and storage medium

Info

Publication number: CN111261133A
Application number: CN202010043120.5A
Authority: CN
Inventors: 袁瑀
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-01-15
Filing date: 2020-01-15
Publication date: 2020-06-09

Abstract

The invention provides a singing processing method, a singing processing device, electronic equipment and a storage medium; the method comprises the following steps: an applet in a first client receives accompaniment content for singing and presents the accompaniment content; acquiring singing content of the synchronous performance corresponding to the accompaniment content in real time; and sending the singing content and the corresponding accompaniment content acquired in real time to a second client. By the method and the device, real-time and convenient singing interaction among different users can be realized.

Description

Singing processing method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of internet technologies, and in particular, to a singing processing method and apparatus, an electronic device, and a storage medium.

Background

The social networking breaks through the limit of industrial social interaction, networks the relationship between people, and shows the popularization of social networking with various social networking software.

The network social contact form is changed from the initial network chat into various rich forms, the network singing (also called network Karaoke) is an important network social contact form at present, users participating in singing entertainment are not limited by regions any more, and the user experience is greatly improved.

At present, users participating in network singing need to install a special singing client on a held terminal device to use corresponding functions, however, the requirements of the users on singing entertainment cannot be met conveniently due to various reasons (such as saving of storage space of the terminal device and high operation cost); although there are applets for singing, one user can only share the applets with other users after singing, and the delay is large.

The related art has no effective scheme for solving the delay of network singing and the high operation cost.

Disclosure of Invention

The embodiment of the invention provides a singing processing method and device, electronic equipment and a storage medium, which can realize real-time and convenient singing interaction among different users.

The technical scheme of the embodiment of the invention is realized as follows:

the embodiment of the invention provides a singing processing method, which comprises the following steps:

an applet in a first client receives accompaniment content for singing and presents the accompaniment content;

acquiring singing content corresponding to the synchronous performance of the accompaniment content in real time;

and sending the singing content and the corresponding accompaniment content acquired in real time to a second client.

An embodiment of the present invention provides a singing processing apparatus, including:

the system comprises a receiving module, a singing module and a singing module, wherein the receiving module is used for receiving accompaniment content for singing by an applet in a first client and presenting the accompaniment content;

the acquisition module is used for acquiring singing content corresponding to the synchronous performance of the accompaniment content in real time;

and the sending module is used for sending the singing content acquired in real time and the corresponding accompaniment content to a second client.

In the foregoing solution, the singing processing apparatus further includes: a request module for responding to the song search operation received in the applet and sending a singing request of the searched song; alternatively, a hit song list is presented; in response to a song selection operation for the hit song list, transmitting a singing request for the selected hit song; wherein the singing request is used for requesting accompaniment content.

In the above scheme, the receiving module is further configured to receive the accompaniment content sent by the server; and the server responds to the singing request, calls a stream pushing component, converts the accompaniment content into a form of stream data and then sends the stream data.

In the above scheme, the receiving module is further configured to play the accompaniment content in the applet, and present the synchronized lyrics in a page of the applet corresponding to the playing progress of the accompaniment content.

In the above scheme, the receiving module is further configured to start an asynchronous timer to time when the playing of the accompaniment content starts; and determining a time stamp for displaying the lyrics when the asynchronous timer counts a calibrated time period threshold value, and calibrating the time stamp for displaying the lyrics so as to display the lyrics synchronous with the playing progress of the accompaniment content in the page of the small program.

In the above scheme, the obtaining module is further configured to obtain a singing video or a singing audio of the performer performing synchronously corresponding to the accompaniment content in real time; wherein, the obtaining of the singing video or audio of the performer corresponding to the accompaniment content synchronous performance in real time includes: the small program in the first client calls a camera assembly and a microphone assembly in a basic library of the first client, and collects the performance of the performer in real time to form a singing video; or the small program in the first client calls a microphone component in a basic library of the first client to collect the performance of the performer in real time to form singing audio.

In the foregoing solution, the singing processing apparatus further includes: and the login module is used for logging in the live broadcast room by the applet in the first client based on the user account of the first client.

In the above scheme, the sending module is further configured to send the singing content and the corresponding accompaniment content acquired in real time to a second client associated with a viewer account participating in the live broadcast room, so that the second client presents the singing content and the corresponding accompaniment content synchronized with the performance.

In the above scheme, the sending module is further configured to invoke, by the applet in the first client, a stream pushing component in a base library of the first client, convert the singing content and the accompaniment content into at least two streaming media segments, and send the at least two streaming media segments to the second client; wherein each of the streaming media segments can be independently decoded and played in the second client.

In the foregoing solution, the singing processing apparatus further includes: the invitation module is used for presenting a social account number with a social relationship with the user account number of the first client; in response to an invitation operation aiming at the social account, sending an invitation message to the invited social account so that the invited social account is added into the live room through the invitation message to become the audience account; wherein the type of social relationship comprises at least one of: a friend; subscribing; paying attention to; and (5) commenting.

In the above scheme, the sending module is further configured to respond to a social account selection operation; sending the singing content and the accompanying content to a second client side associated with the selected social account; or sending the singing content and the accompanying content to an applet in a second client associated with the selected social account; wherein the social account is an account having a social relationship with a user account of the first client; the type of social relationship includes at least one of: a friend; subscribing; paying attention to; and (5) commenting.

In the foregoing solution, the singing processing apparatus further includes: the running module is used for responding to the applet searching operation by the first client, downloading the applet in the first client and running the downloaded applet; or the first client acquires an applet sharing link; and responding to the triggering operation aiming at the applet sharing link, downloading the applet in the first client, and running the downloaded applet.

In the foregoing solution, the singing processing apparatus further includes: a cleaning module for determining an available cache of the applet; when the available cache in the small program is lower than the capacity threshold value, the cache space in the small program is cleared, so that the available cache of the small program is not lower than the capacity threshold value.

In the above scheme, the cleaning module is further configured to clean buffered accompaniment content in a buffer space of the applet, so that the applet can receive the accompaniment content for singing and store the accompaniment content in the buffer space of the applet.

In the foregoing solution, the singing processing apparatus further includes: the decryption module is used for decrypting the accompaniment content when the accompaniment content received by the applet is the encrypted content and storing the decrypted accompaniment content into the cache space of the applet; and when the applet does not support the format of the decrypted accompaniment content, converting the format of the decrypted accompaniment content into a format supported by the applet.

In the foregoing solution, the singing processing apparatus further includes: the interactive module is used for receiving interactive contents aiming at the singing contents acquired by the second client; presenting the interactive content in real time in an applet of the first client; wherein the type of the interactive content at least comprises one of the following: commenting; and E, electronic red package.

An embodiment of the invention provides an electronic device, which includes

A memory for storing executable instructions;

and the processor is used for realizing the singing processing method provided by the embodiment of the invention when the executable instructions stored in the memory are executed.

The embodiment of the invention provides a storage medium, which stores executable instructions and is used for causing a processor to execute so as to realize the singing processing method provided by the embodiment of the invention.

The embodiment of the invention has the following beneficial effects:

the accompaniment content for singing is presented to the performer through the small program of the first client, so that the performer can realize network singing interaction without a special client, and the operation difficulty is obviously reduced; through sending the singing content of the performer to the second client side in real time, the audience of the second client side can watch the singing of the performer in real time, real-time network singing interaction is achieved, user experience is good, and therefore propagation of social relations is benefited.

Drawings

Fig. 1 is a schematic structural diagram of a singing processing system according to an embodiment of the present invention;

fig. 2A is a schematic structural diagram of an electronic device 500 according to an embodiment of the present invention;

fig. 2B is a schematic structural diagram of a singing processing apparatus according to an embodiment of the present invention;

FIG. 2C is a block diagram of an applet according to an embodiment of the present invention;

fig. 3A is a schematic flowchart of a singing processing method according to an embodiment of the present invention;

fig. 3B is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention;

fig. 3C is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention;

fig. 3D is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention;

fig. 4A is a schematic flowchart of a singing processing method according to an embodiment of the present invention;

fig. 4B is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention;

fig. 5A is a schematic flowchart of a singing processing method according to an embodiment of the present invention;

fig. 5B is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention;

fig. 5C is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention;

fig. 6 is a schematic diagram of a local decryption library provided by an embodiment of the present invention;

FIG. 7A is a schematic diagram of lyric calibration provided by the related art;

FIG. 7B is a schematic diagram of lyric calibration provided by an embodiment of the present invention;

fig. 8 is a schematic flow chart of a singing method according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail with reference to the accompanying drawings, the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

In the description that follows, references to the terms "first", "second", and the like, are intended only to distinguish similar objects and not to indicate a particular ordering for the objects, it being understood that "first", "second", and the like may be interchanged under certain circumstances or sequences of events to enable embodiments of the invention described herein to be practiced in other than the order illustrated or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.

Before further detailed description of the embodiments of the present invention, terms and expressions mentioned in the embodiments of the present invention are explained, and the terms and expressions mentioned in the embodiments of the present invention are applied to the following explanations.

1) A client, such as a local client (Native App), is software that a user needs to download and install on a user terminal (e.g., a smartphone) in order to use.

2) A Mini Program (also called a Web Program) is a Program developed based on a front-end-oriented language (e.g., JavaScript) and implementing a service in a hypertext markup language (HTML) page, and software downloaded by a client (e.g., a browser or any client embedded in a browser core) via a network (e.g., the internet) and interpreted and executed in a browser environment of the client saves steps to be installed in the client. For example, an applet for implementing a singing service may be downloaded and run in a social network client.

3) In response to the condition or state on which the performed operation depends, one or more of the performed operations may be in real-time or may have a set delay when the dependent condition or state is satisfied; there is no restriction on the order of execution of the operations performed unless otherwise specified.

4) A Component (Component), which is a functional module of a view of an applet, also called the front-end Component, buttons, titles, tables, sidebars, content, and footers in a page, includes modular code to facilitate reuse among different pages of the applet.

5) The Live-push is a component used by an applet to transmit sound or video to a network, and is located in an applet base library of a client.

6) And the basic library is positioned in the client and encapsulates the operating system of the user terminal and various capabilities of the client, such as media playing, plug flow and the like, and provides calls to the small program in a component/interface mode.

7) The Template (Template), the structure and the style of the data to be rendered in the applet are used for rendering in the rendering environment after filling different data, so that the multiplexing of the structure and the style can be realized.

For example, in a template of the applet page, a structure is described in an extensible markup language (XML) file, a Style is described in a Cascading Style Sheets (CSSs) file describing a component Style, the layout and the Style of the applet page may be multiplexed, and the template of the component may be used for multiplexing the layout and the Style of the component.

8) Background (or back-end), as opposed to the front-end, where written code is basically provided for invocation by the front-end, the back-end is what does not need to process a User Interface (UI), e.g., a logical layer.

9) And the plug flow refers to a process of transmitting the content packaged in the acquisition stage to a server. I.e. the process of transmitting the video signal of the scene to the network.

In the related art, a user can only sing by downloading a singing type client, but due to the complexity of the operation process of installing the client, the operation cost is high for the user, and if singing content of the user needs to be shared by friends, the friends also need to install the same client, so that the sharing threshold is high. In addition, the client side of the singing class only supports asynchronous singing (that is, the user can only record the singing song and then share the singing song with the friends), and the user cannot invite the friends to listen and interact with the friends while singing.

In view of the above problems, embodiments of the present invention provide a singing processing method, apparatus, electronic device and storage medium, which can conveniently meet the demand of a user for singing and entertainment. An exemplary application of the singing processing method provided by the embodiment of the present invention is described below, and the singing processing method provided by the embodiment of the present invention may be applied to various electronic devices, for example, various user terminals (hereinafter also referred to as terminals) such as smart phones, tablet computers, in-vehicle terminals, and smart wearable devices.

An exemplary application system architecture of a terminal for implementing the singing processing method provided by the embodiment of the present invention is described below by taking an electronic device as an example, referring to fig. 1, fig. 1 is a schematic architecture diagram of the singing processing system provided by the embodiment of the present invention, and shows a performance-side terminal 100, a viewer-side terminal 200, a client server (i.e., a background server of a client) 300, and a singing server (i.e., a server for implementing a singing function, i.e., a background server of an applet, or an applet server) 400, which are described below.

The performance side terminal 100 is used to run a first client 110, and the first client 110 may be various Applications (APP) with applet downloading function, such as a social APP (typically, e.g. WeChat) or a video APP. The first client 110 may download the applet 111 for singing through the client server 300.

The audience terminal 200 is configured to run a second client 210, and the second client 210 is an application having a function of receiving the singing content and the accompaniment content transmitted by the first client 110 and presenting the singing content and the accompaniment content synchronized with the performance. The second client 210 is not specifically a client but a type of client, that is, a type of client used by the viewer, and the number of the second clients may be one or more.

The client server 300 is used for receiving a request of a client and submitting data to the client so as to support the native functions of the client, and is also used for providing various applets to the client for downloading by the client.

The singing server 400 is configured to provide accompaniment content to the applet 111 in the first client 110, and is further configured to synchronously transmit the singing content synchronized with the performance and the corresponding accompaniment content to the second client 210.

It should be noted that the first client 110 is a client associated with the user account of the performer, and the second client 210 is a client associated with the user account of the audience. The client server 300 and the singing server 400 may be different servers or the same server.

With reference to fig. 1, in an application scenario of the singing processing method provided in the embodiment of the present invention, first, an performer logs in the client server 300 based on an account in the first client 110, requests the client server 300 to download the applet 111, and runs the applet 111 in the first client 110 after the download is completed; then, the performer performs a song requesting operation through the applet 111 to receive the accompaniment content for singing sent by the singing server 400, and presents the accompaniment content in the applet 111; thirdly, the performer sings according to the accompaniment content presented in the applet 111, the applet 111 collects the singing content of the performer in real time and sends the singing content of the performer to the singing server 400; finally, the singing server 400 synchronously sends the singing content synchronized with the performance and the corresponding accompaniment content to the second client 210, so that the second client 210 presents the singing content synchronized with the performance and the corresponding accompaniment content.

Next, a structure of an electronic device for implementing a singing processing method according to an embodiment of the present invention is described, and as described above, the electronic device according to an embodiment of the present invention may be the performance-side terminal 100 shown in fig. 1. Referring to fig. 2A, fig. 2A is a schematic structural diagram of an electronic device 500 according to an embodiment of the present invention, where the electronic device 500 shown in fig. 2A includes: a hardware layer 510, an operating system layer 520, and an application layer 530.

In some embodiments, the hardware layer 510 includes a processor 511, a user interface 512, a memory 513, and a network interface 514.

The Processor 511 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like, wherein the general purpose Processor may be a microprocessor or any conventional Processor, or the like.

The user interface 512 includes one or more output devices, including one or more speakers and/or one or more visual display screens, that enable the presentation of media content. The user interface 512 also includes one or more input devices, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

The memory 513 includes volatile memory or non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile Memory may be a Read Only Memory (ROM), and the volatile Memory may be a Random Access Memory (RAM). The memory 513 described in embodiments of the invention is intended to comprise any suitable type of memory. The memory 513 may optionally include one or more storage devices physically located remote from the processor 511. The memory 513 is used to store various types of data to support the operation of the electronic device 500. Examples of such data include: any program for operating on the electronic device 500, such as an operating system layer 520 and an application layer 530.

The network interface 514 includes bluetooth, wireless-compliant authentication (WiFi), Universal Serial Bus (USB), and the like.

In some embodiments, the operating system layer 520, including system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer 521, the driver layer 522, the graphics library layer 523, and the core library layer 524, is used to implement various underlying services and to process hardware-based tasks

In some embodiments, the application layer 530 may contain various applications built into the operating system, such as desktop starters and system setup programs; the method can also comprise application programs of a third party, such as a Media Player (Media Player), a Browser (Browser), a social/instant messaging application program and other application programs which need to display a view and receive various touch events and respond through the view, and are used for realizing various services, and the program for realizing the method of the embodiment of the invention can be contained in the application program; the first client described herein may be any one of an application native to the system and an application of a third party, unless otherwise specified.

The singing processing device 531 in the application layer 530 may be software in the form of programs, plug-ins, and the like, and specifically includes the following software modules: a receiving module 5311, an obtaining module 5312, and a sending module 5313. These modules may be logical functional modules and thus may be arbitrarily combined or further divided according to the functions implemented. The functions of the respective modules will be explained below.

The method described in the embodiment of the present invention may be applied to the processor 511, or implemented by the processor 511. The method described in the embodiments of the present invention may be implemented by an integrated logic circuit of hardware in the processor 511 or an instruction in the form of software. The method described in the embodiments of the present invention may be directly implemented as a hardware decoding processor, or implemented by using a combination of hardware and software modules in the decoding processor, where the software module may be located in a storage medium located in the memory 513, and the processor 511 reads an executable instruction in the memory 513, and implements the method described in the embodiments of the present invention by combining with the hardware.

In some embodiments, referring to fig. 2B, fig. 2B is a schematic structural diagram of a singing processing apparatus according to an embodiment of the present invention, and as a specific implementation manner of three modules in the singing processing apparatus 531, the three modules may be implemented by a first client, and an applet and a basic library running in the first client, which will be described in detail.

The development framework for deploying the applet in the development device (e.g., a user terminal such as a computer, a server) by the developer of the applet 111 completes the development of the applet, and the applet 111 can provide a service for singing. The development framework is provided with an applet building tool for encapsulating code in the applet project into one or more JavaScript files capable of running in the browser environment of the first client 110 and uploading to the client server 300.

Various types of clients, for example, the first client 110, may be operated in the performance-side terminal 100 held by the performer. Various types of applets may be provided in the first client 110 for the user to select for use, for example, a singing applet, so that the user can flexibly use various functions only through the first client 110, thereby eliminating the need to install a dedicated client, for example, a singing client.

The first client 110 includes a base library, which encapsulates the operating system of the performance-side terminal 100 and various capabilities of the first client 110 (e.g., media playing, streaming, etc.), and provides calls to the applet in the form of components/interfaces. For example, the applet 111 first receives accompaniment content for singing by calling a plug flow component in the base library and presents the accompaniment content in the applet 111, then acquires the singing content and corresponding accompaniment content in real time by calling a microphone component and/or a camera component in the base library, and finally sends the accompaniment content to the second client by calling the plug flow component in the base library.

For example, once the user confirms the downloading of the applet 111 in the first client 110, the first client 110 will request the client server 300 for a JavaScript file of the downloading applet 111 through the network, and interpret and execute the JavaScript file in the browser environment of the client, thereby running an instance of the applet 111.

An instance of the applet 111 first initializes a page, and may request page data from the client server 300 and render in the page. The page is composed of a series of components, and when the user operates the components, the operating system of the performance-side terminal 100 will deliver corresponding events to the instances of the applet 111, and the components will re-render according to their own logic, for example, when the component is a button in the page of the applet 111, the rendering can be reproduced to display the dynamic effect of the button being pressed and the corresponding content. When an instance of applet 111 receives data for a page updated by client server 300, the component will also re-render according to the data to update the page.

Referring to fig. 2C, fig. 2C is a schematic diagram of a framework structure of an applet provided in an embodiment of the present invention, and a composition framework of the applet 111 is described below with reference to fig. 2C, where the composition framework includes: scene values, logical layers, and view layers, as will be described in detail.

The scene value is used to describe the path of the user into the applet. The developer may obtain the scene values in the event of the lifecycle functions of the client. The client triggers the events of the life cycle function for many times, and the obtained scene values may be different.

The logic layer uses the JavaScript engine to provide the applet with the runtime environment of the developer JavaScript code and the specific functionality of the applet. And the logic layer processes the data and then sends the processed data to the view layer, and meanwhile receives event feedback of the view layer. All code written by the developer is finally packaged into a JavaScript file and runs when the applet is started until the applet is destroyed. The logic layer comprises components of relevant implementation of the basic capability of the applet, specifically: a network component (for network communication with a specified domain name), a storage component (for storing cached files), a file system component (for managing cached files), and a canvas component (for rendering display pages), among others.

When the view layer receives the initial data and the updated data, the view layer needs to be rendered. During the life cycle of a page, the viewing layer receives one piece of initial data and multiple pieces of updated data. The initial rendering needs to be performed when the initial data is received, and the re-rendering needs to be performed each time the updated data is received. The view layer is written by a tag language and a style language and is displayed by the component. And the view layer reflects the data of the logic layer into a view and simultaneously sends the event of the view layer to the logic layer. The label language is used for describing the structure of the page; the structure of the page can be constructed by combining the script language and the tag language; the style language is used to describe the style of a page.

The applet can also implement the extension of the capability by calling components in a base library of the client, where the base library includes various types of components, for example, an audio (audio) component (for playing audio), a camera (c _ aura) component (for providing a function of scanning a two-dimensional code), a picture (image) component (for displaying a picture), a live-play component (for real-time audio recording), a live-play component (for real-time audio playing), a video (video) component (for playing video), a camera component (for capturing an image), and a microphone component (for capturing audio).

The following describes an embodiment of the present invention by taking an example in which the first client 110 and the second client 210 in fig. 1 jointly implement a singing processing method provided by the embodiment of the present invention. Referring to fig. 3A, fig. 3A is a schematic flowchart of a singing processing method according to an embodiment of the present invention, and will be described with reference to the steps shown in fig. 3A.

In step S101, the first client requests the client server to download the applet, so as to receive the applet sent by the client server, and run the applet in the first client.

Here, the first client is a client associated with the user account of the performer.

In some embodiments, the first client requests the client server to download the applet according to the information input in the applet search operation in response to the applet search operation received in the applet search field, so as to receive the applet sent by the client server, and run the applet in the first client.

For example, referring to fig. 3B, fig. 3B is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention. In fig. 3B, the user may trigger the displaying of the applet search field 301 in the first client, and by typing a related character (e.g., "song room") in the applet search field 301, the applet 302 corresponding to the typed related character may be displayed in the first client. When the user triggers the applet 302, the first client automatically downloads the applet 302 or prompts the user whether to download the applet 302, downloads the applet 302 after the user agrees, and jumps to the applet 302 after the download is completed.

In some embodiments, the first client obtains an applet sharing link (or applet download link); and responding to the triggering operation aiming at the applet sharing link (or the applet downloading link), requesting the client server to download the applet so as to receive the applet sent by the client server, and running the applet in the first client.

For example, referring to fig. 3C, fig. 3C is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention. In fig. 3C, the user may share the applet sharing link 303 with other users, so that the other users can download the applet in the first client by clicking the applet sharing link 303, and jump to the applet after the download is completed.

In some embodiments, the user may further request the client server to download the applet through an applet store in the first client, so as to receive the applet sent by the client server, and run the applet in the first client.

In some embodiments, when the first client has previously downloaded the applet, the applet may be run directly in the first client without downloading the applet.

The embodiment of the invention can realize the purpose of singing without downloading a client special for singing again by providing various simple channels, thereby conveniently meeting the requirements of singing and entertainment of users.

In step S102, the applet in the first client sends a singing request to the singing server, and receives the accompaniment content for singing sent by the singing server.

Here, the type of singing may mimic song singing, with the accompaniment content being background music in the mimicked song; or an impromptu singing, and the accompanying content is any music with a rhythm. The accompanying content may be audio or video of the accompanying, for example, the accompanying content may be background video sent to the applet by the singing server. The singing request is used for requesting the singing server to send corresponding accompaniment contents.

In some embodiments, the accompaniment content may be obtained through a singing server or may be obtained from a local cache of the user terminal. For example, before acquiring the accompaniment content from the singing server, the applet in the first client preferentially queries the accompaniment content from the local cache, and when the accompaniment content for singing is contained in the local cache, the applet can directly acquire the corresponding accompaniment content from the local cache without requesting the singing server to acquire the corresponding accompaniment content.

In some embodiments, the performer may log in to the singing server using a user account (or called account number) independent of the first client to obtain accompaniment content; the user account independent of the first client side is not registered, the user account of the first client side is directly used for logging in the singing server to obtain the accompaniment content, and therefore singing can be carried out quickly.

In some embodiments, the applet in the first client sends a singing request of the searched song to the singing server in response to the song searching operation received in the applet, and receives accompaniment content sent by the singing server for singing.

For example, referring to fig. 3D, fig. 3D is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention. In fig. 3D, after the first client runs the applet, the song requesting button 304 is displayed in the display interface of the applet, when the song requesting button 304 is triggered by the user, the song requesting interface is skipped to, the song searching bar 305 is displayed in the song requesting interface, and the song searching bar 305 is searched for a song, so that the singing server sends a singing request of the searched song to the singing server, so that the singing server sends corresponding accompaniment content.

In some embodiments, the applet in the first client presents a popular song list; and responding to a song selection operation aiming at the popular song list, sending a singing request of the selected popular song to the singing server, and receiving the accompaniment content sent by the singing server for singing.

For example, in fig. 3D, after the first client runs the applet, the song requesting button 304 is displayed in the display interface of the applet, when the song requesting button 304 is triggered by the user, the user jumps to the song requesting interface, the hit song list 306 is displayed in the song requesting interface, and the singing request of the selected song is sent to the singing server through the selection operation on the hit song list 306, so that the singing server sends the corresponding accompaniment content.

Here, the accompaniment content is transmitted by the singing server in response to the singing request, calling the plug flow component, converting the accompaniment content into a form of streaming data.

According to the embodiment of the invention, the popular song list is displayed to the user, so that the user can obtain the popular songs more intuitively, and the user is prevented from searching for the songs, thereby facilitating the user to request the songs, and further improving the user experience.

In some embodiments, the applet in the first client, after receiving the accompaniment content for singing, comprises: when the accompaniment content received by the small program is the encrypted content, decrypting the accompaniment content and storing the decrypted accompaniment content into a cache space of the small program; and when the small program does not support the format of the decrypted accompaniment content, converting the format of the decrypted accompaniment content into a format supported by the small program.

As an example, the accompaniment music library provided by the singing server to the applet is an encrypted music library, and the applet cannot be directly played online through a Uniform Resource Locator (URL). Therefore, when a user requests a song, the applet must download an audio file in advance and perform decryption processing, the file format of the decrypted accompaniment content is m4a, and some clients with low version levels do not support direct playback of the accompaniment content, and therefore compatibility processing is required. That is, for a client with a high version, the decrypted accompaniment content in the m4a file format can be directly played, but for a client with a low version, the applet in the client needs to convert the m4a file format into the mp3 file format in advance and then play the file.

In some embodiments, the applet in the first client, prior to receiving the accompaniment content for singing, comprises: an applet in a first client determines an available cache of the applet; when the available cache in the applet is below the capacity threshold, the cache space in the applet is cleared so that the available cache of the applet is not below the capacity threshold.

Here, the capacity threshold may be an average of memory space occupied by the received accompaniment content; or the maximum value of the storage space occupied by the accompaniment contents in the song library of the singing server.

In some embodiments, the applet in the first client, prior to receiving the accompaniment content for singing, comprises: the small program in the first client clears the buffered accompaniment content in the buffer space of the small program, so that the small program can receive the accompaniment content for singing and store the accompaniment content in the buffer space of the small program.

Here, the principle of cleaning up the cache space of the applet is: the deletion is performed in the order of first-in first-out of the buffer space (i.e., the content stored for the longest time is deleted first), or in the order of the size of the buffer content in the buffer space (i.e., the content occupying the larger buffer space is deleted first).

For example, based on the consideration of the security of the accompaniment content, the accompaniment music library provided by the singing server to the applet is an encrypted music library, and the applet cannot be directly played online through the uniform resource locator. Therefore, when a user requests a song, the applet must download an audio file in advance and perform a decryption process. Since the decrypted data is stored in the memory of the applet, the memory limit of the applet is easily exceeded, resulting in the applet frequently prompting the user for "song download failure" errors. In order to solve the problem, after the user finishes ordering songs each time, the small program can automatically delete the cache file of the last song ordering, and the deletion request can be automatically retried, so that the user can not generate the error of 'song downloading failure' because the cache exceeds the limit of the available cache of the small program.

The embodiment of the invention overcomes the defects of small inherent storage space, poor performance and the like of the small program, and ensures that a user can smoothly complete actions of ordering and singing.

In step S103, the applet in the first client presents the accompaniment content.

In some embodiments, the accompaniment content is played in an applet in the first client, and the synchronized lyrics are presented in a page of the applet corresponding to the progress of the accompaniment content.

As an example, when the playing of the accompaniment content starts, an applet in the first client starts an asynchronous timer to time; and determining a time stamp for displaying the lyrics when the asynchronous timer counts a calibrated time period threshold value, and calibrating the time stamp for displaying the lyrics so as to display the lyrics synchronous with the playing progress of the played accompaniment content in the page of the small program.

Here, the time stamp of displaying the lyrics may be calibrated according to the time stamp of playing the accompaniment contents or the time counted by the asynchronous timer.

For example, when the playing of the accompaniment content starts, the applet in the first client starts an asynchronous timer to time; and when the asynchronous timer counts a threshold value of a calibration time period, determining a time stamp for displaying the lyrics and a time stamp for playing the accompaniment content, and calibrating the time stamp for displaying the lyrics according to the time stamp for playing the accompaniment content so as to display the lyrics synchronous with the playing progress of the accompaniment content in a page of the small program.

For example, when the playing of the accompaniment content starts, the applet in the first client starts an asynchronous timer to time; and determining a time stamp for displaying the lyrics when the asynchronous timer counts a threshold value of a calibration time period, and calibrating the time stamp for displaying the lyrics according to the time counted by the asynchronous timer so as to display the lyrics synchronous with the playing progress of the played accompaniment content in the page of the small program.

In the related art, in order to synchronize lyrics with accompaniment in the process of singing a song by a user, frequent call-back is required, which causes a Central Processing Unit (CPU) to occupy too high, and directly shows a phenomenon of voice jamming or no sound hearing at the user side. Therefore, the embodiment of the invention abandons the song progress callback function of a live-push component, and establishes an asynchronous timer in the background of the applet. When the playing of the accompaniment content is started, the asynchronous timer starts to time, the time progress of the current position is calculated at intervals of a calibration time period threshold (for example, 500ms), and a time stamp for displaying the lyrics and a time stamp for playing the accompaniment content are calibrated, so that the song accompaniment and the lyrics are synchronized. Therefore, the user can not only hear the accompaniment, but also see the lyrics synchronized with the accompaniment, and sing the song through the microphone.

In step S104, the applet in the first client acquires, in real time, singing content performed by the performer corresponding to the accompaniment content.

In some embodiments, the applet in the first client obtains in real time a singing video or audio of the synchronized performance of the performer's corresponding accompaniment content.

Here, the type of content performed by the performer may be in the form of a recording (i.e., only singing audio is captured); or in video form (i.e., capturing both the singing audio and the singing image).

As one example, an applet in the first client invokes a camera component and a microphone component in the first client's base library to capture the performer's performance in real-time to form a video of singing.

As another example, an applet in the first client invokes a microphone component in the base library of the first client to capture the performer's performance in real-time to form the singing audio.

In some embodiments, when the playing of the accompaniment content is finished, the applet in the first client stops capturing the performer's singing content and displays the performer's applet in the first client in a next (e.g., no longer in singing status, but merely as a spectator viewing the performance) state.

The embodiment of the invention can collect the audio frequency and the video of the performer performance, meets the singing requirement of the performer, and can show the voice and the action of the audience through the small program, thereby improving the user experience of both the performer and the audience. In addition, the embodiment of the invention can enable the audience to watch the performance of the performer in real time by acquiring the singing video or the singing audio of the performer corresponding to the accompaniment content synchronous performance in real time, and is different from the prior art in which the performer records the performance and then sends the recorded performance to the audience, thereby reducing the waiting time of the audience.

In step S105, the applet in the first client sends the singing content and the corresponding accompaniment content acquired in real time to the second client through the singing server.

In some embodiments, an applet in the first client invokes a stream pushing component (i.e., the audio/video uplink component) in a base library of the first client, converts the singing content and the accompaniment content into at least two streaming media fragments, and sends the at least two streaming media fragments to the second client; wherein each streaming media segment can be decoded and played independently in the second client.

In some embodiments, an applet in a first client invokes a plug-flow component in a base library of the first client, converts the singing content and the accompanying content into at least two streaming media segments, and sends the at least two streaming media segments to an applet in a second client; and the small program in the second client decodes and plays each streaming media fragment by calling the media component in the base library of the second client.

Here, the parameters (including the resolution/bitrate) of the streaming media segment are determined according to the decoding capability (i.e. software decoding capability) of the second client, the hardware decoding capability, and the network condition of the second client, for example, when the decoding capability of the second client is stronger, the resolution/bitrate of the streaming media segment is higher; when the hardware decoding capability of the terminal operating the second client is stronger, the resolution/code rate of the streaming media fragment is higher; the higher the resolution/bitrate of the streaming media segment is, the better the network environment the second client is in (i.e. the shorter the time it takes to request and return data when the second client uploads and downloads data).

As an example, the applet in the first client may send only the singing content acquired in real time to the singing server, and then the singing server packages the accompaniment content requested to be acquired by the applet and the received singing content together to send to the second client or the applet of the second client.

As another example, the applet in the first client may send the singing content and the accompaniment content acquired in real time to the singing server together, and send the received singing content and accompaniment content to the second client or the applet of the second client via the singing server.

The embodiment of the invention also supports the second client to present the singing content and the corresponding accompaniment content which are synchronous with the performance, so that the second client can present the singing content and the accompaniment content without downloading small programs.

In step S106, the second client presents the singing content synchronized with the performance and the corresponding accompaniment content.

In some embodiments, the second client may present the singing content and the corresponding accompaniment content synchronized with the performance by itself, or the applet in the second client may present the singing content and the corresponding accompaniment content synchronized with the performance by itself.

Here, when the singing content synchronized with the performance and the corresponding accompaniment content are presented by the applet itself in the second client, the process of running the applet by the second client is the same as the process of running the applet by the first client, and will not be described herein again.

As an example, when the singing content synchronized with the performance and the corresponding accompaniment content are presented by the second client itself, the specific implementation process of step S105 and step S106 is: responding to a social account selection operation by an applet in the first client; and sending the singing content and the accompaniment content to a second client terminal associated with the selected social account so as to enable the second client terminal to present the singing content and the corresponding accompaniment content which are synchronous with the performance.

Here, the social account is an account having a social relationship with a user account of the first client; the type of social relationship includes at least one of: a friend; subscribing; paying attention to; and (5) commenting.

As an example, when the singing content synchronized with the performance and the corresponding accompaniment content are presented by the applet itself in the second client, the specific implementation procedures of step S105 and step S106 are: responding to a social account selection operation by an applet in the first client; and sending the singing content and the accompaniment content to the small program in the second client terminal associated with the selected social account, so that the small program in the second client terminal presents the singing content and the corresponding accompaniment content synchronized with the performance.

For example, when at least one friend account exists in the user accounts of the applets logged in the first client, the user can select the friend account in the applets to send the singing content and the accompaniment content to the second client associated with the friend account or the applets in the second client, so that the friends can watch the performances.

According to the embodiment of the invention, the small program in the first client can send the singing content and the accompaniment content to any user without establishing a live broadcast room, so that the sharing of the user is facilitated, and the audience can watch the singing of the performer without entering the live broadcast room, so that the user experience is improved.

The embodiment of the invention presents the accompaniment content for singing to the performer through the small program of the first client, so that the performer can realize the purpose of singing without downloading a client special for singing again, thereby conveniently meeting the requirement of singing and entertainment of a user. In addition, the singing content of the performer is sent to the second client side, so that the second client side presents the singing content and the corresponding accompaniment content which are synchronous with the performance, the audience can watch the singing of the performer in real time, the entertainment of singing is improved, and the propagation of social relations is facilitated.

Referring to fig. 4A, fig. 4A is a schematic flowchart of a singing processing method according to an embodiment of the present invention, based on fig. 3A, step S107 may be included before step S102, and step S105 may be replaced with step S108.

In step S107, the applet in the first client logs into the live room based on the user account of the first client.

Here, the live broadcasting room is a synchronization and sharing mode of the singing server for singing songs in real time, and the singing content and the accompaniment content of any user account in the live broadcasting room are pushed to the singing server and are synchronized to the second client end associated with the audience account in real time by the singing server.

In some embodiments, the performer may log into the live room using a user account separate from the first client; or the user account independent of the first client side is not registered, and the user account of the first client side can be directly used for logging in the live broadcast room.

In step S108, the applet in the first client sends the singing content and the corresponding accompaniment content acquired in real time to the second client associated with the viewer account participating in the live broadcast room.

In some embodiments, step S108 may also be: and the small program in the first client sends the singing content and the corresponding accompaniment content which are acquired in real time to the small program in the second client which is associated with the audience account number added into the live broadcast room.

Here, the process of sending the singing content and the accompaniment content by the applet in the first client to the second client or the applet in the second client associated with the viewer account number added to the live broadcast room is the same as step S105, and will not be described again here.

In some embodiments, the manner of joining the live room specifically includes: presenting a social account having a social relationship with a user account of a first client in an applet in the first client; in response to an invitation operation aiming at the social account, sending an invitation message to the invited social account so that the invited social account becomes a viewer account by joining a live room through the invitation message; wherein the type of social relationship comprises at least one of: a friend; subscribing; paying attention to; and (5) commenting.

Here, the type of the invitation information includes at least one of: a graphic code (e.g., a one-dimensional code or a two-dimensional code); and (5) network linking. When the type of invitation information is a graphical code, the invited user may become a viewer by scanning the graphical code to join the live room. When the type of invitation information is a web link, the invited user may become a viewer by clicking on the web link to join the live room.

For example, referring to fig. 4B, fig. 4B is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention. In fig. 4B, the performer may send an invitation message 401 to the invited friend through a session so that the invited friend becomes a spectator by clicking on the invitation message 401 to join the live room.

The embodiment of the invention simultaneously has the advantages of strong interactivity of singing in real time at the client, convenience and easy propagation of the small program, and enables a user to invite friends to sing at any time and any place.

Referring to fig. 5A, fig. 5A is a schematic flowchart of a singing processing method according to an embodiment of the present invention, and based on fig. 3A, step S109 and step S110 may be included after step S106.

In step S109, the applet in the first client receives the interactive content for the singing content acquired by the second client.

Here, the type of the interactive content includes at least one of: commenting; and E, electronic red package.

In some embodiments, the second client (or an applet in the second client) presents a comment interface in the second client (or an applet in the second client) in response to the received comment operation; and responding to the comment operation aiming at the comment interface, acquiring the comment aiming at the singing content, and sending the comment to the small program in the first client.

For example, referring to fig. 5B, fig. 5B is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention. In fig. 5B, when the viewer needs to comment on the performance content, a comment interface may be displayed in the applet of the second client by performing a comment operation in the applet of the second client, and a comment bar 501 is included in the comment interface, and the viewer may type comment content in the comment bar 501 or select comment content (for example, comment text or an emoticon) in the comment bar 501, and finally send the input comment content or the selected comment content to the applet in the first client.

In some embodiments, the second client (or an applet in the second client) presents a reward interface in the second client (or an applet in the second client) in response to the received reward operation; in response to a reward operation for the reward interface, reward information (electronic red envelope or gift) for the singing content is acquired and sent to the applet in the first client. Wherein the reward operation may be sending an electronic red packet.

For example, referring to fig. 5C, fig. 5C is a schematic view of an application scenario of the singing processing method according to the embodiment of the present invention. In fig. 5C, when the viewer desires to enjoy the show content, the viewer can display a enjoyment interface in the applet of the second client by performing a enjoyment operation in the applet of the second client, and the appreciation column 502 is included in the comment interface, and the viewer can select the enjoyment content in the enjoyment column 502 and finally transmit the selected enjoyment content to the applet in the first client. Wherein, the reward content can be an electronic red packet.

In step S110, the interactive content is presented in real time in the applet of the first client.

In some embodiments, a special effect associated with the interactive content is presented in real-time in the applet of the first client; wherein the type of special effect comprises at least one of: visual feedback; auditory feedback; tactile feedback.

Here, the visual feedback may be an animation effect representing the received interactive content; the auditory feedback may be an action sound characterizing the receipt of the interactive content; the haptic feedback can be a haptic vibration or the like that characterizes the received interactive content, and the special effects can attract the attention of the user to the interactive content without influencing the performance of the user.

For example, when the type of the interactive content is a comment, comment text or an emoticon may be presented in an animation form (for example, scrolling play or barrage play) in the applet of the first client, the comment text may also be broadcast in a voice form in the applet of the first client, or a terminal running the first client may be made to vibrate to prompt a performer to receive the comment.

For example, when the type of the interactive content is an electronic red packet, an animation corresponding to the electronic red packet may be presented in the applet of the first client, the amount of the electronic red packet may be broadcast in the applet of the first client in a voice manner, or a terminal operating the first client may be vibrated to prompt the performer to receive the electronic red packet.

The embodiment of the invention can enable the audience to smoothly interact with the performer in real time, and improve the experience of both the performer and the audience.

Continuing with the exemplary structure of the singing processing apparatus 555 implemented as a software module provided by the embodiment of the present invention, in some embodiments, as shown in fig. 2A, the software module in the singing processing apparatus 531 may include: a receiving module 5311, an obtaining module 5312, and a sending module 5313.

A receiving module 5311, configured to receive, by an applet in the first client, accompaniment content for singing and present the accompaniment content;

the obtaining module 5312 is configured to obtain singing content of the synchronized performance corresponding to the accompaniment content in real time;

the sending module 5313 is configured to send the singing content and the corresponding accompaniment content acquired in real time to the second client, so that the second client presents the singing content and the corresponding accompaniment content synchronized with the performance.

In some embodiments, the singing processing device 531 further comprises: the request module is used for responding to song searching operation received in the applet and sending a singing request of a searched song to the singing server; alternatively, a hit song list is presented; responding to a song selection operation aiming at the hit song list, and sending a singing request of the selected hit song to a singing server; the singing request is used for requesting the singing server to send corresponding accompaniment contents.

In some embodiments, the receiving module 5311 is further configured to receive accompaniment content sent by the singing server; the accompaniment content is sent by the singing server responding to the singing request and calling the stream pushing component to convert the accompaniment content into a form of stream data.

In some embodiments, the receiving module 5311 is further configured to play the accompaniment content in the applet, and present the synchronized lyrics in a page of the applet corresponding to the playing progress of the accompaniment content.

In some embodiments, the receiving module 5311 is further configured to start an asynchronous timer for timing when the playing of the accompaniment content starts; and determining a time stamp for displaying the lyrics when the asynchronous timer counts a calibrated time period threshold value, and calibrating the time stamp for displaying the lyrics so as to display the lyrics synchronous with the playing progress of the played accompaniment content in the page of the small program.

In some embodiments, the obtaining module 5312 is further configured to obtain, in real time, a singing video or a singing audio of the performer corresponding to the accompaniment content synchronous performance; wherein, obtain the singing video or audio frequency that the performer corresponds the synchronous performance of accompaniment content in real time, include: a small program in the first client calls a camera assembly and a microphone assembly in a basic library of the first client, and collects the performance of a performer in real time to form a singing video; or the small program in the first client calls a microphone component in a basic library of the first client, and the performer performance is collected in real time to form singing audio.

In some embodiments, the singing processing device 531 further comprises: and the login module is used for logging in the live broadcast room by the applet in the first client based on the user account of the first client.

In some embodiments, the sending module 5313 is further configured to send the singing content and the corresponding accompaniment content acquired in real time to a second client associated with the viewer account participating in the live broadcast room, so that the second client presents the singing content and the corresponding accompaniment content synchronized with the performance.

In some embodiments, the sending module 5313 is further configured to invoke a stream pushing component in a base library of the first client by an applet in the first client, convert the singing content and the accompaniment content into at least two streaming media fragments, and send the at least two streaming media fragments to the second client; wherein each streaming media segment can be decoded and played independently in the second client.

In some embodiments, the singing processing device 531 further comprises: the invitation module is used for presenting a social account number with a social relationship with a user account number of the first client; in response to an invitation operation aiming at the social account, sending an invitation message to the invited social account so that the invited social account becomes a viewer account by joining a live room through the invitation message; wherein the type of social relationship comprises at least one of: a friend; subscribing; paying attention to; and (5) commenting.

In some embodiments, the sending module 5313 is further configured to respond to a social account selection operation; sending the singing content and the accompaniment content to a second client side associated with the selected social account; or sending the singing content and the accompaniment content to the small program in the second client side associated with the selected social account; the social account number is an account number which has a social relationship with a user account number of the first client; the type of social relationship includes at least one of: a friend; subscribing; paying attention to; and (5) commenting.

In some embodiments, the singing processing device 531 further comprises: the operation module is used for responding to the applet searching operation by the first client, downloading the applet in the first client and operating the downloaded applet; or the first client acquires the applet sharing link; and responding to the triggering operation aiming at the applet sharing link, downloading the applet in the first client, and running the downloaded applet.

In some embodiments, the singing processing device 531 further comprises: the cleaning module is used for determining the available cache of the applet; when the available cache in the applet is below the capacity threshold, the cache space in the applet is cleared so that the available cache of the applet is not below the capacity threshold.

In some embodiments, the cleaning module is further configured to clean the buffered accompaniment content in the buffer space of the applet, so that the applet can receive the accompaniment content for singing and store the accompaniment content in the buffer space of the applet.

In some embodiments, the singing processing device 531 further comprises: the decryption module is used for decrypting the accompaniment content when the accompaniment content received by the applet is the encrypted content and storing the decrypted accompaniment content into a cache space of the applet; and when the small program does not support the format of the decrypted accompaniment content, converting the format of the decrypted accompaniment content into a format supported by the small program.

In some embodiments, the singing processing device 531 further comprises: the interactive module is used for receiving interactive contents aiming at the singing contents acquired by the second client; presenting interactive content in real time in an applet of a first client; wherein the type of the interactive content at least comprises one of the following types: commenting; and E, electronic red package.

Embodiments of the present invention provide a storage medium storing executable instructions, which when executed by a processor, will cause the processor to execute a singing processing method provided by embodiments of the present invention, for example, as shown in fig. 3A, 4A, 5A, and so on.

In some embodiments, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EE PROM, flash, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

In some embodiments, executable instructions may be written in any form of programming language (including compiled or interpreted languages), in the form of programs, software modules, scripts or code, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

By way of example, executable instructions may correspond, but do not necessarily have to correspond, to files in a file system, may be stored in a portion of a file that holds other programs or data, e.g., in one or more scripts in a HyperText markup Language (H TML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).

By way of example, executable instructions may be deployed to be executed on one computing device or on multiple computing devices at one site or distributed across multiple sites and interconnected by a communication network.

In the following, an exemplary application of singing processing in an actual application scenario will be described by taking the example that the first client and the second client are social APPs.

The singing method provided by the embodiment of the invention can solve the following technical problems.

1) Problem of memory management of local decryption library

Based on the consideration of content security, the accompaniment music library provided by the music server (i.e. the singing server) to the client is an encrypted music library and cannot be directly played online through a URL. Therefore, when the user requests a song, the client must download the audio file (i.e., the accompaniment content) and perform the decryption process in advance. Since the decrypted data is stored in the memory of the applet (i.e. the above-mentioned cache space), the memory limit of the applet (e.g. 12M) is easily exceeded, so that the applet may frequently prompt the user with an error of "song download failure". In order to solve the problem, referring to fig. 6, fig. 6 is a schematic diagram illustrating a principle of local decryption of a song library according to an embodiment of the present invention, in fig. 6, after a user finishes song selection, an applet will automatically delete a cached file of a last song selection, and the deletion request will be automatically retried, so as to ensure that the user will not have an error of "failed song download" due to the fact that the cache exceeds the memory limit of the applet.

In addition, the file format after the applet directly decrypts is m4a, and the base library in the low-version client does not support direct playing, so that the low-version client also needs to be subjected to compatible processing. That is, for a client with a high version, the decrypted m4a file may be directly played, but for a client with a low version, the background (or backend) needs to convert the m4a format into the mp3 format.

2) Local asynchronous song timer solves the problem of over-high CPU occupancy

In the related art, referring to fig. 7A, fig. 7A is a schematic diagram illustrating a principle of lyric calibration provided in the related art, in fig. 7A, an audio-video uplink (live-push) component is used when calibrating lyrics, and in order to synchronize the lyrics with an accompaniment during a singing process of a user, frequent call-back is required, which may cause an over-high CPU occupancy and directly appear as a phenomenon of a sound stuck or inaudible sound at the user side.

In order to solve the above problems, the embodiment of the present invention abandons the song progress callback function of the audio/video uplink component, and establishes an asynchronous timer in the background of the applet. Referring to fig. 7B, fig. 7B is a schematic diagram illustrating the principle of lyric calibration according to an embodiment of the present invention, in fig. 7B, when the playing of the accompaniment content starts, a front-end development component (event) controls the start of an asynchronous timer, the asynchronous timer starts to count time after starting, the time progress of the current position is calculated every 500ms, and the timestamp of the displayed lyrics and the timestamp of the played accompaniment content are calibrated, so that the song accompaniment and the lyrics are synchronized. The performer is automatically kicked off the table when he stays (using the microphone) on the microphone for a time close to the duration of the song.

As can be seen from the above, a specific implementation process of the singing method provided by the embodiment of the present invention is shown in fig. 8, where fig. 8 is a schematic flow chart of the singing method provided by the embodiment of the present invention, and in fig. 8, when a singer (i.e., the performer) starts to request a song, a background downloads an audio file from a music library provided by a music server, then performs local decryption, and then pushes a stream to the singer through a live-player component, so that the singer can hear an accompaniment. When the stream pushing is started, a local asynchronous timer of the small program starts to time, aligns with the time stamp of the lyrics when each time is counted for 500ms, and displays the lyrics with the aligned time stamps in a display page of the small program so as to ensure that the lyrics are synchronous with the accompaniment. At this time, the singer can hear the accompaniment and also can see the lyrics synchronized with the accompaniment, and can sing the song through the microphone. When a singer sings a song through a microphone, the live-push component pushes the singing audio (namely the singing content) to the server, and then the server pushes the audio to the audience in real time, so that the audience can hear the song sung by human voice in real time until the whole song is finished.

In summary, the embodiments of the present invention have the following beneficial effects:

(1) meanwhile, the method is compatible with the advantages of strong interactivity of singing in real time at the client, convenience of the small program and easiness in propagation. For the middle-aged and the elderly users, the complicated process of downloading and installing the client is not needed any more, and meanwhile, friends can be invited to sing together anytime and anywhere.

(2) The defects of small inherent storage space, poor performance and the like of small programs are overcome, the user can smoothly complete actions of ordering and singing, and meanwhile, audiences can smoothly interact with the small programs in real time (for example, public screen information is sent, gifts are given and the like).

The above description is only an example of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present invention are included in the protection scope of the present invention.

Claims

1. A method of singing processing, the method comprising:

2. The method of claim 1, wherein before the applet in the first client receives the accompaniment content for singing, the method further comprises:

transmitting a singing request of a searched song in response to a song search operation received in the applet;

or,

presenting a popular song list;

in response to a song selection operation for the hit song list, transmitting a singing request for the selected hit song;

wherein the singing request is used for requesting the accompaniment content.

3. The method of claim 2, wherein the applet in the first client receives accompaniment content for singing, comprising:

receiving the accompaniment content sent by a server by an applet in the first client;

and the server responds to the singing request, calls a stream pushing component, converts the accompaniment content into a form of stream data and then sends the stream data.

4. The method of claim 1, wherein said presenting the accompaniment content comprises:

the small program in the first client plays the accompaniment content and

displaying synchronous lyrics in a page of the applet corresponding to the playing progress of the accompaniment content;

wherein, the displaying synchronous lyrics in the page of the applet corresponding to the playing progress of the accompaniment content comprises:

when the playing of the accompaniment content is started, starting an asynchronous timer to time;

determining a timestamp for displaying the lyrics each time the asynchronous timer times a calibrated time period threshold, and calibrating the timestamp for displaying the lyrics such that

And displaying lyrics synchronous with the playing progress of the accompaniment content in the page of the small program.

5. The method of claim 1, wherein said obtaining singing content corresponding to the synchronized performance of the accompanying content in real time comprises:

acquiring a singing video or a singing audio of the performer corresponding to the accompaniment content synchronous performance in real time;

wherein, the obtaining of the singing video or audio of the performer corresponding to the accompaniment content synchronous performance in real time includes:

an applet in the first client calls a camera assembly and a microphone assembly in a base library of the first client to collect the performance of the performer in real time to form a singing video;

or,

an applet in the first client invokes a microphone component in a base library of the first client to capture the performer's performance in real-time to form singing audio.

6. The method of claim 1,

before the applet in the first client receives the accompaniment content for singing, the method further comprises:

an applet in the first client logs in a live broadcast room based on a user account of the first client;

the sending the singing content and the corresponding accompaniment content acquired in real time to a second client comprises:

and sending the singing content and the corresponding accompaniment content acquired in real time to a second client terminal associated with the audience account number added into the live broadcast room.

7. The method of claim 6, wherein sending the real-time captured singing content and corresponding accompanying content to a second client associated with a viewer account joining the live broadcast room comprises:

the small program in the first client calls a plug-flow component in a basic library of the first client, converts the singing content and the accompanying content into at least two streaming media fragments, and converts the two streaming media fragments into a single stream and a single stream

Sending the at least two streaming media fragments to the second client;

wherein each of the streaming media segments can be independently decoded and played in the second client.

8. The method of claim 6, wherein prior to said sending said real-time captured singing content and corresponding accompanying content to a second client associated with a viewer account joining said live broadcast room, said method further comprises:

presenting a social account having a social relationship with a user account of the first client;

in response to the invitation operation aiming at the social account, sending an invitation message to the invited social account so as to enable the invited social account to be invited

The invited social account becomes the audience account by joining the live room through the invitation message;

wherein the type of social relationship comprises at least one of: a friend; subscribing; paying attention to; and (5) commenting.

9. The method of claim 1, wherein said sending the real-time captured singing content and the corresponding accompanying content to a second client comprises:

responding to a social account selection operation;

sending the singing content and the accompanying content to a second client side associated with the selected social account; or,

sending the singing content and the accompanying content to an applet in a second client associated with the selected social account;

wherein the social account is an account having a social relationship with a user account of the first client;

the type of social relationship includes at least one of: a friend; subscribing; paying attention to; and (5) commenting.

10. The method of any of claims 1 to 9, wherein before the applet in the first client receives accompaniment content for singing, the method further comprises:

detecting an available cache of the applet; when the available cache in the small program is lower than a capacity threshold value, cleaning the cache space in the small program so that the available cache of the small program is not lower than the capacity threshold value;

or,

clearing the cached accompaniment content in the cache space of the small program so that the small program can receive the accompaniment content for singing and store the accompaniment content in the cache space of the small program.

11. The method of any of claims 1 to 9, wherein after the applet in the first client receives accompaniment content for singing, the method further comprises:

when the accompaniment content received by the small program is the encrypted content, decrypting the accompaniment content and storing the decrypted accompaniment content into a cache space of the small program;

and when the applet does not support the format of the decrypted accompaniment content, converting the format of the decrypted accompaniment content into a format supported by the applet.

12. The method according to any one of claims 1 to 9, wherein after said transmitting said real-time captured singing content and corresponding accompanying content to a second client, the method further comprises:

receiving interactive content aiming at the singing content acquired by the second client;

presenting the interactive content in real time in an applet of the first client;

wherein the type of the interactive content at least comprises one of the following: commenting; and E, electronic red package.

13. A singing processing apparatus, comprising:

14. An electronic device, comprising:

a memory for storing executable instructions;

a processor for implementing the singing processing method of any one of claims 1 to 12 when executing executable instructions stored in the memory.

15. A storage medium having stored thereon executable instructions for causing a processor to execute the method of singing processing of any one of claims 1 to 12.