CN111343509A

CN111343509A - Action control method of virtual image and display equipment

Info

Publication number: CN111343509A
Application number: CN202010095812.4A
Authority: CN
Inventors: 王光强
Original assignee: Qingdao Hisense Media Network Technology Co Ltd
Current assignee: Qingdao Hisense Media Network Technology Co Ltd; Juhaokan Technology Co Ltd
Priority date: 2020-02-17
Filing date: 2020-02-17
Publication date: 2020-06-26

Abstract

The application discloses a method for controlling the action of an avatar and a display device, wherein when a song file is played, the avatar is displayed and a lyric text obtained by analyzing the song file is analyzed; detecting a label on a time axis of the song file, wherein the time axis is used for controlling the display of the lyric text; responding to the detected label, and acquiring an action instruction corresponding to the label; and controlling the action of the virtual image according to the action instruction. Therefore, when the intelligent television is used for K song, corresponding actions can be automatically matched when the intelligent television plays the corresponding lyric position according to different scenes corresponding to the lyric time axis corresponding to the song file, and the actions are displayed by the virtual image, so that participants have the sense of scene substitution, the interaction among users is increased, and the user experience is improved.

Description

Action control method of virtual image and display equipment

Technical Field

The application relates to the technical field of television software, in particular to a method for controlling the action of an avatar and display equipment.

Background

With the continuous development of communication technology, terminal devices such as computers, smart phones and display devices have become more and more popular. Among them, for the functions of the display device, different kinds of application programs can be installed in the display device generally due to the higher and higher demand of the user for the application experience. In order to provide a family entertainment scene, the display device can be adapted to various media resource scenes, for example, a karaoke APP can be installed on the display device to play song files, so that people can experience the fun of the karaoke without going out.

In order to realize deep penetration of the display device into home entertainment scenes, people can enjoy entertainment through a large screen. When the common display equipment at the present stage realizes the application of a media asset scene, for example, a large-screen karaoke APP, the application has a single function and an independent playing method, only one user can carry out karaoke, multiple users cannot sing together, and interaction among the users is lacked. Therefore, providing a method capable of realizing interaction between users under a certain medium resource scene becomes a technical problem to be solved urgently in the field.

Disclosure of Invention

The application provides a method for controlling the action of an avatar and display equipment, which are used for solving the problems of less interaction and poor experience among users in the existing media asset scene.

In a first aspect, the present application provides a method for controlling an action of an avatar, comprising the steps of:

when playing a song file, displaying an avatar and a lyric text obtained by analyzing the song file;

detecting a tag on a timeline of the song file, wherein the timeline is used for controlling display of the lyric text;

responding to the detected label, and acquiring an action instruction corresponding to the label;

and controlling the action of the virtual image according to the action instruction.

Further, before the displaying the avatar and the lyric text from parsing the song file, the method further comprises:

receiving the selection of a user on a song control, and loading a song file corresponding to the song control;

and loading a preset virtual image model and a preset background image file, and analyzing the song file to obtain the lyric text.

Further, the displaying the avatar and the lyric text obtained by parsing the song file includes:

determining an avatar based on the preset avatar model, determining a background image based on the preset background image file, and displaying the background image and the avatar on a video layer;

and displaying the lyric text on a suspension layer according to the lyric text and the time axis, wherein the suspension layer is positioned above the video layer.

Further, the obtaining, in response to detecting the tag, an action instruction corresponding to the tag includes:

responding to the detected label, accessing an action instruction library corresponding to the song style identification according to the label and the song style identification, wherein the song style identification is preset in the song file, and different action instruction libraries correspond to different song style identifications;

and receiving an action instruction corresponding to the label in the action instruction library.

responding to the detection of the label, accessing an action instruction library corresponding to the rhythm identification according to the label and the rhythm identification, wherein the rhythm identification is preset in the song file, and different action instruction libraries correspond to different rhythm identifications;

Further, still include:

in response to not detecting the tag, acquiring a default action instruction;

and controlling the action of the virtual image according to the default action instruction.

In a second aspect, the present application provides a display device comprising:

a display configured to display the action of the avatar and the lyric text;

a speaker configured to output a sound of the song file;

a controller configured to display an avatar and a lyric text resulting from parsing a song file while the song file is played;

Further, the controller is further configured to:

in response to not detecting the tag, acquiring a default action instruction;

In a third aspect, the present application further provides a computer storage medium, which may store a program that, when executed, may implement some or all of the steps in the embodiments of the motion control method for an avatar provided in the present application.

According to the technical scheme, the action control method and the display device of the virtual image provided by the embodiment of the invention have the advantages that when the song file is played, the virtual image is displayed and the lyric text obtained by analyzing the song file is analyzed; detecting a label on a time axis of the song file, wherein the time axis is used for controlling the display of the lyric text; responding to the detected label, and acquiring an action instruction corresponding to the label; and controlling the action of the virtual image according to the action instruction. Therefore, when the intelligent television is used for K song, corresponding actions can be automatically matched when the intelligent television plays the corresponding lyric position according to different scenes corresponding to the lyric time axis corresponding to the song file, and the actions are displayed by the virtual image, so that participants have the sense of scene substitution, the interaction among users is increased, and the user experience is improved.

Drawings

In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without any creative effort.

Fig. 1 is a schematic diagram illustrating an operation scenario between a display device and a control apparatus according to an embodiment;

fig. 2 is a block diagram exemplarily showing a hardware configuration of a display device 200 according to an embodiment;

fig. 3 is a block diagram exemplarily showing a hardware configuration of the control apparatus 100 according to the embodiment;

fig. 4 is a diagram exemplarily showing a functional configuration of the display device 200 according to the embodiment;

fig. 5a schematically shows a software configuration in the display device 200 according to an embodiment;

fig. 5b schematically shows a configuration of an application in the display device 200 according to an embodiment;

fig. 6 is a flowchart illustrating a motion control method of an avatar according to an embodiment;

fig. 7 is a data flow diagram exemplarily showing a motion control method of an avatar according to an embodiment;

fig. 8 is a flow chart illustrating a display method according to an embodiment;

FIG. 9 is a flow diagram illustrating a method of fetching action instructions according to an embodiment;

FIG. 10 is a flow diagram illustrating another method of fetching action instructions according to an embodiment;

fig. 11 is another flowchart illustrating a motion control method of an avatar according to an embodiment;

fig. 12 is a block diagram exemplarily showing a structure of a display device according to the embodiment.

Detailed Description

To make the objects, technical solutions and advantages of the exemplary embodiments of the present application clearer, the technical solutions in the exemplary embodiments of the present application will be clearly and completely described below with reference to the drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, but not all the embodiments.

All other embodiments, which can be derived by a person skilled in the art from the exemplary embodiments shown in the present application without inventive effort, shall fall within the scope of protection of the present application. Moreover, while the disclosure herein has been presented in terms of exemplary one or more examples, it is to be understood that each aspect of the disclosure can be utilized independently and separately from other aspects of the disclosure to provide a complete disclosure.

It should be understood that the terms "first," "second," "third," and the like in the description and in the claims of the present application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used are interchangeable under appropriate circumstances and can be implemented in sequences other than those illustrated or otherwise described herein with respect to the embodiments of the application, for example.

Furthermore, the terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or device that comprises a list of elements is not necessarily limited to those elements explicitly listed, but may include other elements not expressly listed or inherent to such product or device.

The term "module," as used herein, refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and/or software code that is capable of performing the functionality associated with that element.

The term "remote control" as used in this application refers to a component of an electronic device (such as the display device disclosed in this application) that is typically wirelessly controllable over a relatively short range of distances. Typically using infrared and/or Radio Frequency (RF) signals and/or bluetooth to connect with the electronic device, and may also include WiFi, wireless USB, bluetooth, motion sensor, etc. For example: the hand-held touch remote controller replaces most of the physical built-in hard keys in the common remote control device with the user interface in the touch screen.

The term "gesture" as used in this application refers to a user's behavior through a change in hand shape or an action such as hand motion to convey a desired idea, action, purpose, or result.

Fig. 1 is a schematic diagram illustrating an operation scenario between a display device and a control apparatus according to an embodiment. As shown in fig. 1, a user may operate the display device 200 through the mobile terminal 300 and the control apparatus 100.

The control device 100 may control the display device 200 in a wireless or other wired manner by using a remote controller, including infrared protocol communication, bluetooth protocol communication, other short-distance communication manners, and the like. The user may input a user command through a key on a remote controller, voice input, control panel input, etc. to control the display apparatus 200. Such as: the user can input a corresponding control command through a volume up/down key, a channel control key, up/down/left/right moving keys, a voice input key, a menu key, a power on/off key, etc. on the remote controller, to implement the function of controlling the display device 200.

In some embodiments, mobile terminals, tablets, computers, laptops, and other smart devices may also be used to control the display device 200. For example, the display device 200 is controlled using an application program running on the smart device. The application, through configuration, may provide the user with various controls in an intuitive User Interface (UI) on a screen associated with the smart device.

For example, the mobile terminal 300 may install a software application with the display device 200, implement connection communication through a network communication protocol, and implement the purpose of one-to-one control operation and data communication. Such as: the mobile terminal 300 and the display device 200 can establish a control instruction protocol, synchronize a remote control keyboard to the mobile terminal 300, and control the display device 200 by controlling a user interface on the mobile terminal 300. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200, so as to realize the synchronous display function.

As also shown in fig. 1, the display apparatus 200 also performs data communication with the server 400 through various communication means. The display device 200 may be allowed to be communicatively connected through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display apparatus 200. Illustratively, the display device 200 receives software program updates, or accesses a remotely stored digital media library, by sending and receiving information, as well as Electronic Program Guide (EPG) interactions. The servers 400 may be a group or groups of servers, and may be one or more types of servers. Other web service contents such as video on demand and advertisement services are provided through the server 400.

The display device 200 may be a liquid crystal display, an OLED display, a projection display device. The particular display device type, size, resolution, etc. are not limiting, and those skilled in the art will appreciate that the display device 200 may be modified in performance and configuration as desired.

The display apparatus 200 may additionally provide an intelligent network tv function that provides a computer support function in addition to the broadcast receiving tv function. Examples include a web tv, a smart tv, an Internet Protocol Tv (IPTV), and the like.

A hardware configuration block diagram of a display device 200 according to an exemplary embodiment is exemplarily shown in fig. 2. As shown in fig. 2, the display device 200 includes a controller 210, a tuning demodulator 220, a communication interface 230, a detector 240, an input/output interface 250, a video processor 260-1, an audio processor 60-2, a display 280, an audio output 270, a memory 290, a power supply, and an infrared receiver.

A display 280 for receiving the image signal from the video processor 260-1 and displaying the video content and image and components of the menu manipulation interface. The display 280 includes a display screen assembly for presenting a picture, and a driving assembly for driving the display of an image. The video content may be displayed from broadcast television content, or may be broadcast signals that may be received via a wired or wireless communication protocol. Alternatively, various image contents received from the network communication protocol and sent from the network server side can be displayed.

Meanwhile, the display 280 simultaneously displays a user manipulation UI interface generated in the display apparatus 200 and used to control the display apparatus 200.

And, a driving component for driving the display according to the type of the display 280. Alternatively, in case the display 280 is a projection display, it may also comprise a projection device and a projection screen.

The communication interface 230 is a component for communicating with an external device or an external server according to various communication protocol types. For example: the communication interface 230 may be a Wifi chip 231, a bluetooth communication protocol chip 232, a wired ethernet communication protocol chip 233, or other network communication protocol chips or near field communication protocol chips, and an infrared receiver (not shown).

The display apparatus 200 may establish control signal and data signal transmission and reception with an external control apparatus or a content providing apparatus through the communication interface 230. And an infrared receiver, an interface device for receiving an infrared control signal for controlling the apparatus 100 (e.g., an infrared remote controller, etc.).

The detector 240 is a signal used by the display device 200 to collect an external environment or interact with the outside. The detector 240 includes a light receiver 242, a sensor for collecting the intensity of ambient light, and parameters such as parameter changes can be adaptively displayed by collecting the ambient light.

The image acquisition device 241, such as a camera and a camera, may be used to acquire an external environment scene, acquire attributes of a user or interact gestures with the user, adaptively change display parameters, and recognize gestures of the user, so as to implement an interaction function with the user.

In some other exemplary embodiments, the detector 240, a temperature sensor, etc. may be provided, for example, by sensing the ambient temperature, and the display device 200 may adaptively adjust the display color temperature of the image. For example, the display apparatus 200 may be adjusted to display a cool tone when the temperature is in a high environment, or the display apparatus 200 may be adjusted to display a warm tone when the temperature is in a low environment.

In other exemplary embodiments, the detector 240, and a sound collector, such as a microphone, may be used to receive a user's voice, a voice signal including a control instruction from the user to control the display device 200, or collect an ambient sound for identifying an ambient scene type, and the display device 200 may adapt to the ambient noise.

The input/output interface 250 controls data transmission between the display device 200 of the controller 210 and other external devices. Such as receiving video and audio signals or command instructions from an external device.

Input/output interface 250 may include, but is not limited to, the following: any one or more of high definition multimedia interface HDMI interface 251, analog or data high definition component input interface 253, composite video input interface 252, USB input interface 254, RGB ports (not shown in the figures), etc.

In some other exemplary embodiments, the input/output interface 250 may also form a composite input/output interface with the above-mentioned plurality of interfaces.

The tuning demodulator 220 receives the broadcast television signals in a wired or wireless receiving manner, may perform modulation and demodulation processing such as amplification, frequency mixing, resonance, and the like, and demodulates the television audio and video signals carried in the television channel frequency selected by the user and the EPG data signals from a plurality of wireless or wired broadcast television signals.

The tuner demodulator 220 is responsive to the user-selected television signal frequency and the television signal carried by the frequency, as selected by the user and controlled by the controller 210.

The tuner-demodulator 220 may receive signals in various ways according to the broadcasting system of the television signal, such as: terrestrial broadcast, cable broadcast, satellite broadcast, or internet broadcast signals, etc.; and according to different modulation types, the modulation mode can be digital modulation or analog modulation. Depending on the type of television signal received, both analog and digital signals are possible.

In other exemplary embodiments, the tuner/demodulator 220 may be in an external device, such as an external set-top box. In this way, the set-top box outputs television audio/video signals after modulation and demodulation, and the television audio/video signals are input into the display device 200 through the input/output interface 250.

The video processor 260-1 is configured to receive an external video signal, and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, and the like according to a standard codec protocol of the input signal, so as to obtain a signal that can be displayed or played on the direct display device 200.

Illustratively, the video processor 260-1 includes a demultiplexing module, a video decoding module, an image synthesizing module, a frame rate conversion module, a display formatting module, and the like.

The demultiplexing module is used for demultiplexing the input audio and video data stream, and if the input MPEG-2 is input, the demultiplexing module demultiplexes the input audio and video data stream into a video signal and an audio signal.

And the video decoding module is used for processing the video signal after demultiplexing, including decoding, scaling and the like.

And the image synthesis module is used for carrying out superposition mixing processing on the GUI signal input by the user or generated by the user and the video image after the zooming processing by the graphic generator so as to generate an image signal for display.

The frame rate conversion module is configured to convert an input video frame rate, such as a 60Hz frame rate into a 120Hz frame rate or a 240Hz frame rate, and the normal format is implemented in, for example, an interpolation frame mode.

The display format module is used for converting the received video output signal after the frame rate conversion, and changing the signal to conform to the signal of the display format, such as outputting an RGB data signal.

The audio processor 260-2 is configured to receive an external audio signal, decompress and decode the received audio signal according to a standard codec protocol of the input signal, and perform noise reduction, digital-to-analog conversion, amplification processing, and the like to obtain an audio signal that can be played in the speaker.

In other exemplary embodiments, video processor 260-1 may comprise one or more chips. The audio processor 260-2 may also comprise one or more chips.

And, in other exemplary embodiments, the video processor 260-1 and the audio processor 260-2 may be separate chips or may be integrated together with the controller 210 in one or more chips.

An audio output 272, which receives the sound signal output from the audio processor 260-2 under the control of the controller 210, such as: the speaker 272, and the external sound output terminal 274 that can be output to the generation device of the external device, in addition to the speaker 272 carried by the display device 200 itself, such as: an external sound interface or an earphone interface and the like.

The power supply provides power supply support for the display device 200 from the power input from the external power source under the control of the controller 210. The power supply may include a built-in power supply circuit installed inside the display device 200, or may be a power supply interface installed outside the display device 200 to provide an external power supply in the display device 200.

A user input interface for receiving an input signal of a user and then transmitting the received user input signal to the controller 210. The user input signal may be a remote controller signal received through an infrared receiver, and various user control signals may be received through the network communication module.

For example, the user inputs a user command through the remote controller 100 or the mobile terminal 300, the user input interface responds to the user input through the controller 210 according to the user input, and the display device 200 responds to the user input.

In some embodiments, a user may enter a user command on a Graphical User Interface (GUI) displayed on the display 280, and the user input interface receives the user input command through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor.

The controller 210 controls the operation of the display apparatus 200 and responds to the user's operation through various software control programs stored in the memory 290.

As shown in fig. 2, the controller 210 includes a RAM213 and a ROM214, and a graphic processor 216, a CPU processor 212, a communication interface 218, such as: a first interface 218-1 through an nth interface 218-n, and a communication bus. The RAM213 and the ROM214, the graphic processor 216, the CPU processor 212, and the communication interface 218 are connected via a bus.

A ROM213 for storing instructions for various system boots. If the display apparatus 200 starts power-on upon receipt of the power-on signal, the CPU processor 212 executes a system boot instruction in the ROM, copies the operating system stored in the memory 290 to the RAM213, and starts running the boot operating system. After the start of the operating system is completed, the CPU processor 212 copies the various application programs in the memory 290 to the RAM213, and then starts running and starting the various application programs.

A graphics processor 216 for generating various graphics objects, such as: icons, operation menus, user input instruction display graphics, and the like. The display device comprises an arithmetic unit which carries out operation by receiving various interactive instructions input by a user and displays various objects according to display attributes. And a renderer for generating various objects based on the operator and displaying the rendered result on the display 280.

A CPU processor 212 for executing operating system and application program instructions stored in memory 290. And executing various application programs, data and contents according to various interactive instructions received from the outside so as to finally display and play various audio and video contents.

In some exemplary embodiments, the CPU processor 212 may include a plurality of processors. The plurality of processors may include one main processor and a plurality of or one sub-processor. A main processor for performing some operations of the display apparatus 200 in a pre-power-up mode and/or operations of displaying a screen in a normal mode. A plurality of or one sub-processor for one operation in a standby mode or the like.

The controller 210 may control the overall operation of the display apparatus 100. For example: in response to receiving a user command for selecting a UI object to be displayed on the display 280, the controller 210 may perform an operation related to the object selected by the user command.

Wherein the object may be any one of selectable objects, such as a hyperlink or an icon. Operations related to the selected object, such as: displaying an operation connected to a hyperlink page, document, image, or the like, or performing an operation of a program corresponding to the icon. The user command for selecting the UI object may be a command input through various input means (e.g., a mouse, a keyboard, a touch pad, etc.) connected to the display apparatus 200 or a voice command corresponding to a voice spoken by the user.

The memory 290 includes a memory for storing various software modules for driving the display device 200. Such as: various software modules stored in memory 290, including: the system comprises a basic module, a detection module, a communication module, a display control module, a browser module, various service modules and the like.

Wherein the basic module is a bottom layer software module for signal communication among the various hardware in the postpartum care display device 200 and for sending processing and control signals to the upper layer module. The detection module is used for collecting various information from various sensors or user input interfaces, and the management module is used for performing digital-to-analog conversion and analysis management.

For example: the voice recognition module comprises a voice analysis module and a voice instruction database module. The display control module is a module for controlling the display 280 to display image content, and may be used to play information such as multimedia image content and UI interface. And the communication module is used for carrying out control and data communication with external equipment. And the browser module is used for executing a module for data communication between browsing servers. And the service module is used for providing various services and modules including various application programs.

Meanwhile, the memory 290 is also used to store visual effect maps and the like for receiving external data and user data, images of respective items in various user interfaces, and a focus object.

A block diagram of the configuration of the control apparatus 100 according to an exemplary embodiment is exemplarily shown in fig. 3. As shown in fig. 3, the control apparatus 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory 190, and a power supply 180.

The control device 100 is configured to control the display device 200 and may receive an input operation instruction of a user and convert the operation instruction into an instruction recognizable and responsive by the display device 200, serving as an interaction intermediary between the user and the display device 200. Such as: the user responds to the channel up and down operation by operating the channel up and down keys on the control device 100.

In some embodiments, the control device 100 may be a smart device. Such as: the control apparatus 100 may install various applications that control the display apparatus 200 according to user demands.

In some embodiments, as shown in fig. 1, a mobile terminal 300 or other intelligent electronic device may function similar to the control device 100 after installing an application that manipulates the display device 200. Such as: the user may implement the functions of controlling the physical keys of the device 100 by installing applications, various function keys or virtual buttons of a graphical user interface available on the mobile terminal 300 or other intelligent electronic device.

The controller 110 includes a processor 112 and RAM113 and ROM114, a communication interface 218, and a communication bus. The controller 110 is used to control the operation of the control device 100, as well as the internal components for communication and coordination and external and internal data processing functions.

The communication interface 130 enables communication of control signals and data signals with the display apparatus 200 under the control of the controller 110. Such as: the received user input signal is transmitted to the display apparatus 200. The communication interface 130 may include at least one of a WiFi chip, a bluetooth module, an NFC module, and other near field communication modules.

A user input/output interface 140, wherein the input interface includes at least one of a microphone 141, a touch pad 142, a sensor 143, keys 144, and other input interfaces. Such as: the user can realize a user instruction input function through actions such as voice, touch, gesture, pressing, and the like, and the input interface converts the received analog signal into a digital signal and converts the digital signal into a corresponding instruction signal, and sends the instruction signal to the display device 200.

The output interface includes an interface that transmits the received user instruction to the display apparatus 200. In some embodiments, the interface may be an infrared interface or a radio frequency interface. Such as: when the infrared signal interface is used, the user input instruction needs to be converted into an infrared control signal according to an infrared control protocol, and the infrared control signal is sent to the display device 200 through the infrared sending module. The following steps are repeated: when the rf signal interface is used, a user input command needs to be converted into a digital signal, and then the digital signal is modulated according to the rf control signal modulation protocol and then transmitted to the display device 200 through the rf transmitting terminal.

In some embodiments, the control device 100 includes at least one of a communication interface 130 and an output interface. The control device 100 is provided with a communication interface 130, such as: the WiFi, bluetooth, NFC, etc. modules may transmit the user input command to the display device 200 through the WiFi protocol, or the bluetooth protocol, or the NFC protocol code.

A memory 190 for storing various operation programs, data and applications for driving and controlling the control apparatus 200 under the control of the controller 110. The memory 190 may store various control signal commands input by a user.

And a power supply 180 for providing operational power support to the various elements of the control device 100 under the control of the controller 110. A battery and associated control circuitry.

Fig. 4 is a diagram schematically illustrating a functional configuration of the display device 200 according to an exemplary embodiment. As shown in fig. 4, the memory 290 is used to store an operating system, an application program, contents, user data, and the like, and performs system operations for driving the display device 200 and various operations in response to a user under the control of the controller 210. The memory 290 may include volatile and/or nonvolatile memory.

The memory 290 is specifically configured to store an operating program for driving the controller 210 in the display device 200, and to store various application programs installed in the display device 200, various application programs downloaded by a user from an external device, various graphical user interfaces related to the applications, various objects related to the graphical user interfaces, user data information, and internal data of various supported applications. The memory 290 is used to store system software such as an OS kernel, middleware, and applications, and to store input video data and audio data, and other user data.

The memory 290 is specifically used for storing drivers and related data such as the audio/video processors 260-1 and 260-2, the display 280, the communication interface 230, the tuning demodulator 220, the input/output interface of the detector 240, and the like.

In some embodiments, memory 290 may store software and/or programs, software programs for representing an Operating System (OS) including, for example: a kernel, middleware, an Application Programming Interface (API), and/or an application program. For example, the kernel may control or manage system resources, or functions implemented by other programs (e.g., the middleware, APIs, or applications), and the kernel may provide interfaces to allow the middleware and APIs, or applications, to access the controller to implement controlling or managing system resources.

The memory 290, for example, includes a broadcast receiving module 2901, a channel control module 2902, a volume control module 2903, an image control module 2904, a display control module 2905, an audio control module 2906, an external instruction recognition module 2907, a communication control module 2908, a light receiving module 2909, a power control module 2910, an operating system 2911, and other applications 2912, a browser module, and the like. The controller 210 performs functions such as: a broadcast television signal reception demodulation function, a television channel selection control function, a volume selection control function, an image control function, a display control function, an audio control function, an external instruction recognition function, a communication control function, an optical signal reception function, an electric power control function, a software control platform supporting various functions, a browser function, and the like.

A block diagram of a configuration of a software system in a display device 200 according to an exemplary embodiment is exemplarily shown in fig. 5 a.

As shown in fig. 5a, an operating system 2911, including executing operating software for handling various basic system services and for performing hardware related tasks, acts as an intermediary for data processing performed between application programs and hardware components. In some embodiments, portions of the operating system kernel may contain a series of software to manage the display device hardware resources and provide services to other programs or software code.

In other embodiments, portions of the operating system kernel may include one or more device drivers, which may be a set of software code in the operating system that assists in operating or controlling the devices or hardware associated with the display device. The drivers may contain code that operates the video, audio, and/or other multimedia components. Examples include a display screen, a camera, Flash, WiFi, and audio drivers.

The accessibility module 2911-1 is configured to modify or access the application program to achieve accessibility and operability of the application program for displaying content.

A communication module 2911-2 for connection to other peripherals via associated communication interfaces and a communication network.

The user interface module 2911-3 is configured to provide an object for displaying a user interface, so that each application program can access the object, and user operability can be achieved.

Control applications 2911-4 for controllable process management, including runtime applications and the like.

The event transmission system 2914, which may be implemented within the operating system 2911 or within the application program 2912, in some embodiments, on the one hand, within the operating system 2911 and on the other hand, within the application program 2912, is configured to listen for various user input events, and to refer to handlers that perform one or more predefined operations in response to the identification of various types of events or sub-events, depending on the various events.

The event monitoring module 2914-1 is configured to monitor an event or a sub-event input by the user input interface.

The event identification module 2914-1 is configured to input definitions of various types of events for various user input interfaces, identify various events or sub-events, and transmit the same to a process for executing one or more corresponding sets of processes.

The event or sub-event refers to an input detected by one or more sensors in the display device 200 and an input of an external control device (e.g., the control device 100). Such as: the method comprises the following steps of inputting various sub-events through voice, inputting gestures through gesture recognition, inputting sub-events through remote control key commands of the control equipment and the like. Illustratively, the one or more sub-events in the remote control include a variety of forms including, but not limited to, one or a combination of key presses up/down/left/right/, ok keys, key presses, and the like. And non-physical key operations such as move, hold, release, etc.

The interface layout manager 2913, directly or indirectly receiving the input events or sub-events from the event transmission system 2914, monitors the input events or sub-events, and updates the layout of the user interface, including but not limited to the position of each control or sub-control in the interface, and the size, position, and level of the container, and other various execution operations related to the layout of the interface.

As shown in fig. 5b, the application layer 2912 contains various applications that may also be executed at the display device 200. The application may include, but is not limited to, one or more applications such as: live television applications, video-on-demand applications, media center applications, application centers, gaming applications, and the like.

The live television application program can provide live television through different signal sources. For example, a live television application may provide television signals using input from cable television, radio broadcasts, satellite services, or other types of live television services. And, the live television application may display video of the live television signal on the display device 200.

A video-on-demand application may provide video from different storage sources. Unlike live television applications, video on demand provides a video display from some storage source. For example, the video on demand may come from a server side of the cloud storage, from a local hard disk storage containing stored video programs.

The media center application program can provide various applications for playing multimedia contents. For example, a media center, which may be other than live television or video on demand, may provide services that a user may access to various images or audio through a media center application.

The application program center can provide and store various application programs. The application may be a game, an application, or some other application associated with a computer system or other device that may be run on the smart television. The application center may obtain these applications from different sources, store them in local storage, and then be operable on the display device 200.

When the display device presents social functions, for example, when playing song K, at this time, the application program may be song K APP. The user utilizes the display device to carry out the K song, the audio and video file of the song selected when the K song is displayed in the user interface of the display device usually, and the user sings along with the displayed audio and video file. However, in the existing song-playing process, only the audio/video file and the lyric text can be displayed in the display equipment, so that the user is monotonous in playing the song-playing and poor in experience.

Fig. 6 is a flowchart illustrating a motion control method of an avatar according to an embodiment; fig. 7 is a data flow diagram exemplarily showing a motion control method of an avatar according to an embodiment. The action control method of the virtual image provided by the embodiment of the invention can automatically match scene actions according to different scenes when a smart television is used for singing K, so that the action of the virtual image is realized, and participants have the sense of scene substitution. The method is applied to a display device, such as a smart television, in which a 3D avatar karaoke APP capable of performing an avatar is installed, and specifically, referring to fig. 6 and 7, the method for controlling the action of the avatar includes the following steps:

and S1, displaying the virtual image and the lyric text obtained by analyzing the song file when playing the song file.

The intelligent television starts to play song files, the song files can be songs in an MV form, and actions with virtual images can follow different songs, different lyrics and the like to present different postures. When the song file is played, the controller in the display device can select the corresponding song file and the virtual image corresponding to the song file according to the triggering of the song control so as to display the song file and the virtual image in the user interface of the display device.

Specifically, before displaying the avatar and the lyric text obtained by parsing the song file, the method further includes:

and S01, receiving the selection of the user on the song control, and loading the song file corresponding to the song control.

And S02, loading the preset virtual image model and the preset background image file, and analyzing the song file to obtain the lyric text.

The user triggers the song control through the remote controller to select favorite songs, and after selection, the controller loads and clicks song files selected by the song control. Meanwhile, the controller loads an avatar model and a preset background image file, different types of avatars are stored in the avatar model, and the avatars are used for presenting actions; the background image file is used for presenting a background image of the song file, such as a stage style and an effect presented in a user interface in the process of singing K.

The song file comprises an audio file, a video file and a lyric text, wherein the audio file is used for inputting sound through a loudspeaker of the display device, the video file is presented in the user interface and combined with the background image, and the lyric text is used for being presented in the user interface and used as lyric prompt when a user K songs.

A flow chart of a display method according to an embodiment is exemplarily shown in fig. 8. After loading the avatar model and the background image file, the controller of the display device needs to select the avatar and the background image corresponding to the song file selected by the user for display. For this purpose, in this embodiment, referring to fig. 8, the method for displaying the avatar and the lyric text obtained by parsing the song file includes:

and S11, determining the virtual image based on the preset virtual image model, determining a background image based on the preset background image file, and displaying the background image and the virtual image in the video layer.

And S12, displaying the lyric text on a suspension layer according to the lyric text and the time axis, wherein the suspension layer is positioned above the video layer.

The user can select one of the preset virtual image models as the virtual image of the current K song, select one of the preset background image files as the background image of the current K song, establish a connection with the song file selected by the user and display the connection in the user interface.

When displaying, the background image and the virtual image are positioned on the same layer, and a video layer is generated for displaying. And displaying the lyric text and the time axis in a form of a suspension layer, wherein the suspension layer is positioned above the video layer.

When playing song files, the user interface of the display device displays virtual images, background images, lyric texts and the like, and at the moment, the virtual images have no action and are default images, namely static images and initial images preset by the system.

S2, detecting a label on a time axis of the song file, wherein the time axis is used for controlling display of the lyric text.

In order to match the action of the virtual image in the process of playing the K song with the playing scene of the song, a series of action libraries can be arranged for each song file, and the action libraries can determine the richness of the action of the virtual image.

For example, a call-calling action may be set before the song singing begins, a right hand holding the microphone close to the mouth is set during the song singing, a left hand is set to act, an action of interacting with the user may be set while waiting for the middle section and the lower section, an action of rotating and the like may be set at the song singing climax, and an action of thank exis and the like may be set at the song singing end.

In order to automatically match actions when a song file is played to a certain progress, the method provided by this embodiment may add corresponding tags to the lyric text of the song file according to time nodes in advance, that is, add tags on the time axis, so that the tags can be matched with the actions in the action library.

The label is that according to a preset time node, a zone bit used for identifying an action instruction is added on the lyrics corresponding to the song media assets. The label can be added to the time axis of each song file according to the preference of a user or a system preset mode, and a flag bit for identifying an action is added to a preset time node of the time axis to serve as the label of the current song file. For example, when a song file is played for 50s, a first tag is added, corresponding to the action "call"; when the player plays the game for 60s, the second label is added, and the corresponding action of 'arm shaking' is performed.

In order to accurately display the actions and match the actions with the song files, the method provided by the embodiment detects whether the tags exist in the lyric playing progress in real time in the process of playing the song files, namely whether the flag bits for identifying the actions exist on the lyric time axis.

And S3, responding to the detected label, and acquiring an action command corresponding to the label.

And if a label exists at a certain time node corresponding to the lyric time axis in the playing process of the song file, the controller responds to the label and acquires an action instruction corresponding to the label. The action command comprises an animation identification and a music style, wherein the animation identification is used for representing animation for realizing the action, and the music style is used for representing the style of the currently played song file, such as a fast-rhythm song, a slow-rhythm song, a sad song, a happy song and the like.

The action control method of the virtual image provided by the embodiment can call different animations according to different music styles to realize the action of the virtual image, so that a user can enter the song atmosphere according to the action of the virtual image presented in the user interface when the user plays a song, the user can be immersed in the action, and the experience is better.

A flow chart of a method of fetching action instructions according to an embodiment is illustrated in fig. 9. In one possible embodiment, referring to fig. 9, in response to detecting a tag, an action instruction corresponding to the tag is obtained according to the following steps, including:

s311, responding to the detected label, accessing an action instruction library corresponding to the song style identification according to the label and the song style identification, wherein the song style identification is preset in the song file, and different action instruction libraries correspond to different song style identifications.

And S312, receiving the action command corresponding to the label in the action command library.

According to the song file, the style of the song can be determined, and then the song style identification corresponding to the song file is obtained. And if the label is detected to exist on a certain time node of the time axis in the playing process of the song file, matching the label and the song style identification with the corresponding action instruction library to determine the action instruction corresponding to the label. The action command is used for controlling the action of the virtual image, namely realizing the corresponding animation so that the virtual image executes the corresponding action.

The tags and the action instructions are in a mapping relation, one tag corresponds to one action instruction, but the same action instruction may correspond to a plurality of tags according to different action types, namely under different music style scenes. For example, the action command "wave hand" is labeled "huishou", and in this case, the labels correspond to the action commands one by one. If the action frequency is usually large in a cheerful music scene, the label is "huishou 1" at this time; if the action frequency is usually small in a sad music scene, at this time, the label is "huishou 2", therefore, the same action command "wave hand", corresponds to multiple labels in different music scenes, and each label corresponds to an action command, at this time, the two action commands may be the same action command. Thus, upon detection of a tag, the corresponding action instruction may be determined.

Different types of action instructions are stored in an action instruction library preset in the display equipment so as to accord with music scenes with different music styles and different melodies (rhythms). The preset action instruction library can be a complete action library and comprises action instructions according with different music styles and different melody music scenes, namely all types of action instructions; the preset action instruction library may also be composed of a plurality of sub-action instruction libraries, each sub-action instruction library corresponds to one action type, for example, one music style corresponds to one action instruction library, and one melody corresponds to one action instruction library.

After the label is detected, the controller calls an action control module in the display device, and the action control module matches the label with an action instruction library. When the action instruction is matched with a preset action instruction library for determining the action instruction, a target action instruction library is determined according to the song style identification, and the action instruction matched with the song style identification is stored in the target action instruction library. After the target action instruction library is determined, the detected tags are matched with all action instructions in the target action instruction library, and the action instructions corresponding to the tags can be determined in the target action instruction library in view of the mapping relation between the tags and the action instructions.

For example, if the song style is identified as the cheerful music style, selecting an action instruction library of the cheerful music style from a preset action instruction library as a target action instruction library; and if the label is 'bengtiao', matching with the target action instruction library to determine that the action instruction 'bounces' in accordance with the cheerful music style.

A flow chart of another method of fetching action instructions according to an embodiment is illustrated in fig. 10. In another possible embodiment, referring to fig. 10, the step of obtaining an action instruction corresponding to the tag in response to detecting the tag includes:

s321, responding to the detected label, accessing an action instruction library corresponding to the rhythm identification according to the label and the rhythm identification, wherein the rhythm identification is preset in the song file, and different action instruction libraries correspond to different rhythm identifications.

And S322, receiving the action command corresponding to the label in the action command library.

The song files may also include songs of different tempos, such as fast-paced songs and slow-paced songs. At this time, the action instruction library preset in the display device comprises action instructions of music scenes with different rhythms, and one rhythm corresponds to one action instruction.

When the action instruction is matched with a preset action instruction library for determining the action instruction, a target action instruction library is determined according to the rhythm identification, and the action instruction matched with the rhythm identification is stored in the target action instruction library. After the target action instruction library is determined, the detected tags are matched with all action instructions in the target action instruction library, and the action instructions corresponding to the tags can be determined in the target action instruction library in view of the mapping relation between the tags and the action instructions.

For example, if the tempo is a fast tempo music style, selecting an action instruction library of the fast tempo music style from a preset action instruction library as a target action instruction library; and if the label is 'baishou', matching with a target action instruction library to determine that the action instruction 'hand waving' in the fast-rhythm music style is met.

Therefore, the method provided by the embodiment can select the correspondingly adapted action instruction library according to different music styles and different rhythms, uniquely determines the action instruction according to the label on the time axis in the process of detecting the playing of the song file, further realizes the action of the virtual image, and combines the animation of the virtual image with the song file.

And S4, controlling the action of the virtual image according to the action command.

The action instruction carries animation realization, after the action instruction corresponding to the label at a certain time point on a time axis in the playing process of the song file is determined, the action instruction is sent to the action control module by the controller, the action is triggered by the action control module to be displayed, and namely the action instruction controls the virtual image to execute corresponding action. When the current label is matched with a certain action instruction in the action instruction library, the action instruction is triggered by the action control module to perform action, and the action is represented by the virtual image and displayed on the user interface of the display device.

At this time, the video file, the background image, the lyric text and the avatar of the song file are simultaneously displayed in the user interface, and the avatar at this time can present different action animations. The virtual image presenting the action animation is combined with the background image, and different animations are presented by the virtual image along with the playing of the song file so as to enrich the user interface.

However, if a corresponding target action instruction library is not selected from the preset action instruction library based on different song style identifications and different rhythm identifications, that is, an action instruction corresponding to the tag cannot be received, which indicates that the current tag is an invalid action, the current operation is discarded, that is, the matching process with the preset action instruction library is ended, and the song file is continuously played and the tag detection of the next time node is continuously performed.

Another flowchart of a motion control method of an avatar according to an embodiment is exemplarily shown in fig. 11. If the tag is not detected on the time axis during the playing of the song file, at this time, in order to ensure the action of the avatar, the method provided in this embodiment, referring to fig. 11, further includes:

and S61, responding to the undetected label, and acquiring a default action instruction.

And S62, controlling the action of the virtual image according to the default action command.

And if the label is not detected on a certain time node on the lyric time axis corresponding to the song file, continuing to play the song file. At this time, the label is not detected, and the virtual image cannot be controlled to present the corresponding action animation according to the matched action instruction, and in order to ensure the richness of the user interface, the preset default action instruction can be obtained at this time.

When the label is not detected, the controller of the display device calls a prestored default action instruction, and the default action instruction can control the virtual image to execute default actions, such as in-situ shaking, in-situ arm swinging and the like. Therefore, when the label is not detected, the virtual image can be controlled to present default action instead of standing still in the process of playing the song file, and further the visual effect of the user interface can be ensured.

The method provided by the embodiment can be used for researching and developing the chorus function in a social system based on the social television, and aims to deeply establish the relationship between users, enhance the viscosity and activity of the users, thereby enriching the playing method of the karaoke platform of the television, enabling the users to want to sing with who at home, bringing the relationship between the users and the television closer, enabling the social network to be more comfortable, enabling the television to have higher temperature, breaking the spatial dimension limitation, enabling the users to want to sing with which, and playing the big screen karaoke social scene.

According to the technical scheme, the virtual image action control method provided by the embodiment of the invention has the advantages that when the song file is played, the virtual image is displayed and the lyric text obtained by analyzing the song file is analyzed; detecting a label on a time axis of the song file, wherein the time axis is used for controlling the display of the lyric text; responding to the detected label, and acquiring an action instruction corresponding to the label; and controlling the action of the virtual image according to the action instruction. Therefore, when the intelligent television is used for K song, corresponding actions can be automatically matched when the intelligent television plays the corresponding lyric position according to different scenes corresponding to the lyric time axis corresponding to the song file, and the actions are displayed by the virtual image, so that participants have the sense of scene substitution, the interaction among users is increased, and the user experience is improved.

Fig. 12 is a block diagram exemplarily showing a structure of a display device according to the embodiment. Referring to fig. 12, the present application provides a display apparatus for performing the relevant steps of the motion control method of the avatar shown in fig. 6, the display apparatus including: a display 20 configured to display the action of the avatar and the lyric text; a speaker 30 configured to output a sound of the song file; a controller 10 configured to display an avatar and a lyric text resulting from parsing a song file while the song file is played; detecting a tag on a timeline of the song file, wherein the timeline is used for controlling display of the lyric text; responding to the detected label, and acquiring an action instruction corresponding to the label; and controlling the action of the virtual image according to the action instruction.

Further, the controller 10 is further configured to: receiving the selection of a user on a song control, and loading a song file corresponding to the song control; and loading a preset virtual image model and a preset background image file, and analyzing the song file to obtain the lyric text.

Further, the controller 10 is further configured to: determining an avatar based on the preset avatar model, determining a background image based on the preset background image file, and displaying the background image and the avatar on a video layer; and displaying the lyric text on a suspension layer according to the lyric text and the time axis, wherein the suspension layer is positioned above the video layer.

Further, the controller 10 is further configured to: responding to the detected label, accessing an action instruction library corresponding to the song style identification according to the label and the song style identification, wherein the song style identification is preset in the song file, and different action instruction libraries correspond to different song style identifications; and receiving an action instruction corresponding to the label in the action instruction library.

Further, the controller 10 is further configured to: responding to the detection of the label, accessing an action instruction library corresponding to the rhythm identification according to the label and the rhythm identification, wherein the rhythm identification is preset in the song file, and different action instruction libraries correspond to different rhythm identifications; and receiving an action instruction corresponding to the label in the action instruction library.

Further, the controller 10 is further configured to: in response to not detecting the tag, acquiring a default action instruction; and controlling the action of the virtual image according to the default action instruction.

In a specific implementation, the present invention further provides a computer storage medium, where the computer storage medium may store a program, and the program may include some or all of the steps in each embodiment of the method for controlling the motion of the avatar provided by the present invention when executed. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a Random Access Memory (RAM).

Those skilled in the art will readily appreciate that the techniques of the embodiments of the present invention may be implemented as software plus a required general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.

The same and similar parts in the various embodiments in this specification may be referred to each other. In particular, for the embodiment of the display device, since it is substantially similar to the embodiment of the method, the description is simple, and for the relevant points, refer to the description in the embodiment of the method.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for controlling the movement of an avatar, comprising the steps of:

2. The method of claim 1, wherein prior to said displaying the avatar and parsing lyric text from the song file, the method further comprises:

3. The method of claim 2, wherein displaying the avatar and parsing the lyrics text from the song file comprises:

4. The method of claim 1, wherein the retrieving, in response to detecting the tag, an action instruction corresponding to the tag comprises:

5. The method of claim 1, wherein the retrieving, in response to detecting the tag, an action instruction corresponding to the tag comprises:

6. The method of claim 1, further comprising:

in response to not detecting the tag, acquiring a default action instruction;

7. A display device, comprising:

a display configured to display the action of the avatar and the lyric text;

a speaker configured to output a sound of the song file;

8. The display device of claim 7, wherein the controller is further configured to:

9. The display device of claim 8, wherein the controller is further configured to:

10. The display device of claim 7, wherein the controller is further configured to:

11. The display device of claim 7, wherein the controller is further configured to:

12. The display device of claim 7, wherein the controller is further configured to:

in response to not detecting the tag, acquiring a default action instruction;