WO2021031809A1 - 一种界面显示方法及显示设备 - Google Patents

一种界面显示方法及显示设备 Download PDF

Info

Publication number
WO2021031809A1
WO2021031809A1 PCT/CN2020/105277 CN2020105277W WO2021031809A1 WO 2021031809 A1 WO2021031809 A1 WO 2021031809A1 CN 2020105277 W CN2020105277 W CN 2020105277W WO 2021031809 A1 WO2021031809 A1 WO 2021031809A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
interface
display
demonstration
local image
Prior art date
Application number
PCT/CN2020/105277
Other languages
English (en)
French (fr)
Inventor
李园园
陈胤旸
谢尧
张凡文
Original Assignee
聚好看科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 聚好看科技股份有限公司 filed Critical 聚好看科技股份有限公司
Publication of WO2021031809A1 publication Critical patent/WO2021031809A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44213—Monitoring of end-user related data
    • H04N21/44218—Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4781—Games
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting

Definitions

  • the embodiments of the present application relate to display technology. More specifically, it relates to an interface display method and display device.
  • the display device can provide users with playback screens such as audio, video, and pictures, it has received widespread attention from users. Users can accomplish some things with the help of display devices. For example, some users play a coach’s instructional video through a display device and follow the instructional video to complete corresponding actions.
  • Some users play a coach’s instructional video through a display device and follow the instructional video to complete corresponding actions.
  • users' demand for the functions of display devices is increasing day by day.
  • the user wants to watch the coach teaching video frame he can see the video frame of the user's own action. Through the comparison of the video frame of his own action and the coach teaching video frame, he finds the insufficiency of his own action, and performs his own action in time correct.
  • this application provides an interface display method and display device.
  • the embodiment of the present application shows a display device, including:
  • a display screen for displaying a first display interface and the second display interface the first display interface includes a playback control for controlling the playback of a demonstration video
  • the second display interface includes a display for playing the demonstration video The first play window of and the second play window used to play the local image collected by the camera;
  • Camera used to collect local images
  • a controller configured to:
  • the second interface is displayed, the demonstration video is played in the first play window, and the local image is played in the second video window.
  • the embodiment of the present application shows a display device, including:
  • a display screen for displaying a first display interface and the second display interface the first display interface includes a playback control for controlling the playback of a demonstration video
  • the second display interface includes a display for playing the demonstration video The first play window of and the second play window used to play the local image collected by the camera;
  • Camera used to collect local video
  • a controller configured to:
  • the current video frame or the neighboring video frame and the demonstration video frame whose marked difference degree of the human body movement is lower than the difference threshold are buffered for display of the exercise evaluation interface.
  • the embodiment of the present application shows a display device, including:
  • a display screen for displaying a first display interface and the second display interface the first display interface includes a playback control for controlling the playback of a demonstration video
  • the second display interface includes a display for playing the demonstration video The first play window of and the second play window used to play the local image collected by the camera;
  • Camera used to collect local images
  • a controller configured to:
  • the second interface is displayed, the demonstration video is played in the first play window, and the local image with joint points marked is played in the second video window.
  • the first joint point is marked as the first color
  • the second joint point is marked as the second color
  • the degree of movement difference of is greater than the degree of movement difference between the body part where the second joint point is located and the corresponding body part in the demonstration video.
  • the embodiment of the present application shows an interface display method, including:
  • the second interface is displayed, the demonstration video is played in the first play window in the second interface, and the local image is played in the second video window in the second interface .
  • the embodiment of the present application shows an interface display method, and the method includes:
  • an exercise evaluation interface is displayed, wherein the exercise evaluation interface displays the current video frame or the neighboring video frame with a lower degree of difference in human body action after the annotation, and the demonstration video frame.
  • the embodiment of the present application shows an interface display method, and the method includes:
  • the second interface is displayed, the demonstration video is played in the first play window in the second interface, and the joint points are marked in the second video window in the second interface
  • the first joint point is marked as a first color
  • the second joint point is marked as a second color
  • the first joint point is located
  • the motion difference between the body part and the corresponding body part in the demonstration video is greater than the motion difference between the body part where the second joint point is located and the corresponding body part in the demonstration video.
  • Fig. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control device according to an embodiment
  • FIG. 2 exemplarily shows a block diagram of the hardware configuration of the control device 100 according to the embodiment
  • FIG. 3 exemplarily shows a block diagram of the hardware configuration of the display device 200 according to the embodiment
  • FIG. 4 exemplarily shows a block diagram of the hardware architecture of the display device 200 according to FIG. 3;
  • FIG. 5 exemplarily shows a schematic diagram of the functional configuration of the display device 200 according to the embodiment
  • Fig. 6a exemplarily shows a schematic diagram of software configuration in the display device 200 according to the embodiment
  • FIG. 6b exemplarily shows a configuration diagram of an application program in the display device 200 according to the embodiment
  • FIG. 7 exemplarily shows a schematic diagram of the user interface in the display device 200 according to the embodiment.
  • Fig. 8-1 is a schematic diagram showing a first interface according to some embodiments.
  • Fig. 8-2 is a schematic diagram showing a first interface according to some embodiments.
  • Figure 9-1 is a schematic diagram showing a prompt interface according to some embodiments.
  • Figure 9-2 is a schematic diagram showing a prompt interface according to some embodiments.
  • FIG. 10 is a schematic diagram showing a second display interface according to some embodiments.
  • FIG. 11 is a local image with 13 joint positions annotated according to some embodiments.
  • Figure 12 shows a local image with joint annotations according to some embodiments
  • FIG. 13 is a color-labeled local image according to some embodiments.
  • Figure 14-1 is an exercise evaluation interface according to some embodiments.
  • Figure 14-2 is a schematic diagram showing a second display interface according to some embodiments.
  • various external device interfaces are usually provided on the display device to facilitate the connection of different peripheral devices or cables to realize corresponding functions.
  • a high-definition camera is connected to the interface of the display device, if the hardware system of the display device does not have the hardware interface of the high-pixel camera that receives the source code, then the data received by the camera cannot be presented to the display of the display device. On the screen.
  • the hardware system of traditional display devices only supports one hard decoding resource, and usually only supports 4K resolution video decoding. Therefore, when you want to realize the video chat while watching Internet TV, in order not to reduce
  • the definition of the network video screen requires the use of hard decoding resources (usually the GPU in the hardware system) to decode the network video.
  • the general-purpose processor such as CPU
  • the video chat screen is processed by soft decoding.
  • this application discloses a dual hardware system architecture to realize multiple channels of video chat data (at least one local video).
  • module used in some embodiments of this application can refer to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or/and software code that can execute related to the component Function.
  • remote control used in some embodiments of this application refers to a component of an electronic device (such as the display device disclosed in this application), which can generally control the electronic device wirelessly within a short distance.
  • This component can generally use infrared and/or radio frequency (RF) signals and/or Bluetooth to connect to electronic devices, and can also include functional modules such as WiFi, wireless USB, Bluetooth, and motion sensors.
  • RF radio frequency
  • a handheld touch remote control uses a user interface in a touch screen to replace most of the physical built-in hard keys in general remote control devices.
  • gesture used in some embodiments of the present application refers to a user's behavior that expresses expected ideas, actions, goals, and/or results through an action such as a change of hand shape or hand movement.
  • the term "hardware system” used in some embodiments of this application may refer to a computing device composed of mechanical, optical, electrical, and magnetic devices such as an integrated circuit (IC) and a printed circuit board (PCB). , Control, storage, input and output functions.
  • IC integrated circuit
  • PCB printed circuit board
  • Control storage, input and output functions.
  • the hardware system is also usually referred to as a motherboard or a chip.
  • operating system used in some embodiments of the present application may refer to a computer system presented to the user after the processor reads the code in the memory, such as "Android OS”, “Mac OS”, “Windows OS” and so on.
  • Fig. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. As shown in FIG. 1, the user can operate the display device 200 by controlling the device 100.
  • the control device 100 may be a remote controller 100A, which can communicate with the display device 200 through infrared protocol communication, Bluetooth protocol communication, ZigBee protocol communication or other short-distance communication methods for wireless or other short-distance communication.
  • the display device 200 is controlled in a wired manner.
  • the user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, etc.
  • the user can control the display device 200 by inputting corresponding control commands through the volume plus and minus keys, channel control keys, up/down/left/right movement keys, voice input keys, menu keys, and switch buttons on the remote control. Function.
  • the control device 100 may also be a smart device, such as a mobile terminal 100B, a tablet computer, a computer, a notebook computer, etc., which may be connected to a local area network (LAN), a wide area network (WAN, Wide Area Network), and a wireless local area network ((WLAN) , Wireless Local Area Network) or other networks communicate with the display device 200, and realize the control of the display device 200 through an application program corresponding to the display device 200.
  • LAN local area network
  • WAN wide area network
  • WLAN wireless local area network
  • Wireless Local Area Network Wireless Local Area Network
  • both the mobile terminal 100B and the display device 200 can be installed with software applications, so that the connection and communication between the two can be realized through a network communication protocol, thereby realizing one-to-one control operation and data communication.
  • the mobile terminal 100B can establish a control command protocol with the display device 200, synchronize the remote control keyboard to the mobile terminal 100B, and control the display device 200 by controlling the user interface of the mobile terminal 100B; or the mobile terminal 100B
  • the audio and video content displayed on the screen is transmitted to the display device 200 to realize the synchronous display function.
  • the display device 200 can also communicate with the server 300 through multiple communication methods.
  • the display device 200 may be allowed to communicate with the server 300 via a local area network, a wireless local area network, or other networks.
  • the server 300 may provide various contents and interactions to the display device 200.
  • the display device 200 transmits and receives information, interacts with an Electronic Program Guide (EPG, Electronic Program Guide), receives software program updates, or accesses a remotely stored digital media library.
  • EPG Electronic Program Guide
  • the server 300 may be a group or multiple groups, and may be one or more types of servers.
  • the server 300 provides other network service content such as video on demand and advertising services.
  • the display device 200 may be a liquid crystal display, an OLED (Organic Light Emitting Diode) display, or a projection display device; on the other hand, the display device may be a smart TV or a display system composed of a display and a set-top box.
  • OLED Organic Light Emitting Diode
  • the display device 200 may make some changes in performance and configuration as required.
  • the display device 200 may additionally provide a smart network TV function that provides a computer support function. Examples include Internet TV, Smart TV, Internet Protocol TV (IPTV) and so on. In some embodiments, the display device may not have a broadcast receiving function.
  • a smart network TV function that provides a computer support function. Examples include Internet TV, Smart TV, Internet Protocol TV (IPTV) and so on.
  • IPTV Internet Protocol TV
  • the display device may not have a broadcast receiving function.
  • the display device may be connected or provided with a camera, which is used to present the picture captured by the camera on the display interface of the display device or other display devices to realize interactive chats between users.
  • the picture captured by the camera can be displayed on the display device in full screen, half screen, or in any optional area.
  • the camera is connected to the monitor rear shell through a connecting plate, and is fixedly installed on the upper middle of the monitor rear shell.
  • a connecting plate As an installable method, it can be fixedly installed at any position of the monitor rear shell to ensure its It is sufficient that the image capture area is not blocked by the rear shell, for example, the image capture area and the display device have the same orientation.
  • the camera can be connected to the display rear shell through a connecting plate or other conceivable connectors.
  • the connector is equipped with a lifting motor.
  • the user wants to use the camera or has an application to use the camera
  • it can be raised above the display.
  • the camera is not needed, it can be embedded behind the back shell to protect the camera from damage.
  • the camera used in this application may have 16 million pixels to achieve the purpose of ultra-high-definition display. In actual use, a camera with higher or lower than 16 million pixels can also be used.
  • the content displayed in different application scenarios of the display device can be merged in many different ways, so as to achieve functions that cannot be achieved by traditional display devices.
  • the user can video chat with at least one other user while watching a video program.
  • the presentation of the video program can be used as the background picture, and the video chat window is displayed on the background picture. Visually, you can call this function "watch and chat".
  • At least one video chat is performed across terminals.
  • the user can video chat with at least one other user while entering the education application for learning. For example, students can realize remote interaction with teachers while learning content in educational applications. Visually, you can call this function "learning and chatting”.
  • a video chat is conducted with players entering the game.
  • players For example, when a player enters a game application to participate in a game, it can realize remote interaction with other players. Visually, you can call this function "watch and play”.
  • the game scene and the video image are merged, and the portrait in the video image is cut out and displayed on the game image, thereby improving user experience.
  • somatosensory games such as ball games, boxing games, running games, dancing games, etc.
  • human body postures and movements are acquired through a camera, limb detection and tracking, and key point data detection of human bones.
  • Animations are integrated in the game to realize games such as sports, dance and other scenes.
  • the user can interact with at least one other user in video and voice in the K song application. Visually, you can call this function "watch and sing". Preferably, when at least one user enters the application in the chat scene, multiple users can jointly complete the recording of a song.
  • the user can turn on the camera locally to obtain pictures and videos, which is vivid, and this function can be called "mirror”.
  • more functions may be added or the above functions may be reduced. This application does not specifically limit the function of the display device.
  • Fig. 2 exemplarily shows a configuration block diagram of the control device 100 according to an exemplary embodiment.
  • the control device 100 includes a controller 110, a communicator 130, a user input/output interface 140, a memory 190, and a power supply 180.
  • the control device 100 is configured to control the display device 200, and can receive user input operation instructions, and convert the operation instructions into instructions that can be recognized and responded to by the display device 200, and serve as an interactive intermediary between the user and the display device 200 effect.
  • the user operates the channel addition and subtraction keys on the control device 100, and the display device 200 responds to the channel addition and subtraction operations.
  • control device 100 may be a smart device.
  • control device 100 can install various applications for controlling the display device 200 according to user requirements.
  • the mobile terminal 100B or other smart electronic devices can perform similar functions to the control device 100 after installing an application for controlling the display device 200.
  • the user can install various function keys or virtual buttons of the graphical user interface that can be provided on the mobile terminal 100B or other smart electronic devices by installing applications to realize the function of the physical keys of the control device 100.
  • the controller 110 includes a processor 112, RAM 113 and ROM 114, a communication interface, and a communication bus.
  • the controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components, and external and internal data processing functions.
  • the communicator 130 realizes communication of control signals and data signals with the display device 200 under the control of the controller 110. For example, the received user input signal is sent to the display device 200.
  • the communicator 130 may include at least one of communication modules such as a WIFI module 131, a Bluetooth module 132, and an NFC module 133.
  • the user input/output interface 140 wherein the input interface includes at least one of input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a button 144.
  • input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a button 144.
  • the user can implement the user instruction input function through voice, touch, gesture, pressing and other actions.
  • the input interface converts the received analog signal into a digital signal and the digital signal into a corresponding instruction signal, which is sent to the display device 200.
  • the output interface includes an interface for sending the received user instruction to the display device 200.
  • it may be an infrared interface or a radio frequency interface.
  • the user input instruction needs to be converted into an infrared control signal according to the infrared control protocol, and sent to the display device 200 via the infrared sending module.
  • a radio frequency signal interface a user input instruction needs to be converted into a digital signal, and then modulated according to the radio frequency control signal modulation protocol, and then sent to the display device 200 by the radio frequency transmitting terminal.
  • control device 100 includes at least one of a communicator 130 and an output interface.
  • the control device 100 is configured with a communicator 130, such as: WIFI, Bluetooth, NFC and other modules, which can encode user input instructions through the WIFI protocol, or Bluetooth protocol, or NFC protocol, and send to the display device 200.
  • a communicator 130 such as: WIFI, Bluetooth, NFC and other modules, which can encode user input instructions through the WIFI protocol, or Bluetooth protocol, or NFC protocol, and send to the display device 200.
  • the memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller 110.
  • the memory 190 can store various control signal instructions input by the user.
  • the power supply 180 is used to provide operating power support for each element of the control device 100 under the control of the controller 110. Can battery and related control circuit.
  • FIG. 3 exemplarily shows a hardware configuration block diagram of a hardware system in the display device 200 according to an exemplary embodiment.
  • the mechanism relationship of the hardware system can be shown in Figure 3.
  • one hardware system in the dual hardware system architecture is referred to as the first hardware system or A system, A chip, and the other hardware system is referred to as the second hardware system or N system, N chip.
  • the A chip includes the controller and various interfaces of the A chip
  • the N chip includes the controller and various interfaces of the N chip.
  • a relatively independent operating system can be installed in the A chip and the N chip.
  • the operating system of the A chip and the operating system of the N chip can communicate with each other through a communication protocol. Exemplary: the framework layer of the A chip's operating system and the N chip's
  • the framework layer of the operating system can communicate for the transmission of commands and data, so that there are two independent but interrelated subsystems in the display device 200.
  • the A chip and the N chip can realize connection, communication and power supply through multiple different types of interfaces.
  • the interface type of the interface between the A chip and the N chip may include at least one of general-purpose input/output (GPIO), USB interface, HDMI interface, UART interface, and the like.
  • GPIO general-purpose input/output
  • USB interface USB interface
  • HDMI interface HDMI interface
  • UART interface UART interface
  • One or more of these interfaces can be used between the A chip and the N chip for communication or power transmission.
  • the N chip can be powered by an external power source
  • the A chip can be powered by the N chip instead of the external power source.
  • the A chip may also include interfaces for connecting other devices or components, such as the MIPI interface for connecting to a camera (Camera) shown in FIG. 3, a Bluetooth interface, etc.
  • the N chip can also include a VBY interface for connecting to the display screen TCON (Timer Control Register), which is used to connect a power amplifier (Amplifier, AMP) and a speaker (Speaker). ) I2S interface; and at least one of IR/Key interface, USB interface, Wifi interface, Bluetooth interface, HDMI interface, Tuner interface, etc.
  • TCON Timer Control Register
  • AMP power amplifier
  • Speaker speaker
  • I2S interface I2S interface
  • I2S interface I2S interface
  • IR/Key interface at least one of IR/Key interface, USB interface, Wifi interface, Bluetooth interface, HDMI interface, Tuner interface, etc.
  • FIG. 4 is only an exemplary description of the dual hardware system architecture of the present application, and does not represent a limitation to the present application. In practical applications, both hardware systems can contain more or less hardware or interfaces as required.
  • FIG. 4 exemplarily shows a hardware architecture block diagram of the display device 200 according to FIG. 3.
  • the hardware system of the display device 200 may include an A chip and an N chip, and modules connected to the A chip or the N chip through various interfaces.
  • the N chip may include a tuner and demodulator 220, a communicator 230, an external device interface 250, a controller 210, a memory 290, a user input interface, a video processor 260-1, an audio processor 260-2, a display 280, and an audio output interface 270. At least one of the power supplies. In other embodiments, the N chip may also include more or fewer modules.
  • the tuner and demodulator 220 is used to perform modulation and demodulation processing such as amplifying, mixing, and resonating broadcast television signals received through wired or wireless methods, thereby demodulating the user’s information from multiple wireless or cable broadcast television signals. Select the audio and video signals carried in the frequency of the TV channel, and additional information (such as EPG data signals).
  • the signal path of the tuner and demodulator 220 can be many kinds, such as: terrestrial broadcasting, cable broadcasting, satellite broadcasting or Internet broadcasting, etc.; and according to different modulation types, the signal adjustment method can be digitally modulated The method may also be an analog modulation method; and according to different types of received television signals, the tuner demodulator 220 may demodulate analog signals and/or digital signals.
  • the tuner and demodulator 220 is also used to respond to the TV channel frequency selected by the user and the TV signal carried by the frequency according to the user's selection and control by the controller 210.
  • the tuner demodulator 220 may also be in an external device, such as an external set-top box.
  • the set-top box outputs TV audio and video signals through modulation and demodulation, and inputs them to the display device 200 through the external device interface 250.
  • the communicator 230 is a component for communicating with external devices or external servers according to various communication protocol types.
  • the communicator 230 may include a WIFI module 231, a Bluetooth module 232, a wired Ethernet module 233, and an infrared communication protocol module and other network communication protocol modules or near field communication protocol modules.
  • the display device 200 may establish a control signal and a data signal connection with an external control device or content providing device through the communicator 230.
  • the communicator may receive the control signal of the remote controller 100 according to the control of the controller.
  • the external device interface 250 is a component that provides data transmission between the N chip controller 210 and the A chip and other external devices.
  • the external device interface can be connected to external devices such as set-top boxes, game devices, notebook computers, etc. in a wired/wireless manner, and can receive external devices such as video signals (such as moving images), audio signals (such as music), and additional information (such as EPG). ) And other data.
  • the external device interface 250 may include: a high-definition multimedia interface (HDMI) terminal 251, a composite video blanking synchronization (CVBS) terminal 252, an analog or digital component terminal 253, a universal serial bus (USB) terminal 254, red, green, and blue ( RGB) terminal (not shown in the figure) and any one or more.
  • HDMI high-definition multimedia interface
  • CVBS composite video blanking synchronization
  • USB universal serial bus
  • RGB red, green, and blue
  • the controller 210 controls the work of the display device 200 and responds to user operations by running various software control programs (such as an operating system and/or various application programs) stored on the memory 290.
  • various software control programs such as an operating system and/or various application programs
  • the controller 210 includes a read-only memory RAM 214, a random access memory ROM 213, a graphics processor 216, a CPU processor 212, a communication interface 218, and a communication bus.
  • RAM213 and ROM214, graphics processor 216, CPU processor 212, and communication interface 218 are connected by a bus.
  • the graphics processor 216 is used to generate various graphics objects, such as icons, operation menus, and user input instructions to display graphics. Including an arithmetic unit, which performs operations by receiving various interactive commands input by the user, and displays various objects according to display attributes. As well as including a renderer, various objects obtained based on the arithmetic unit are generated, and the rendering result is displayed on the display 280.
  • the CPU processor 212 is configured to execute operating system and application program instructions stored in the memory 290. And according to receiving various interactive instructions input from the outside, to execute various applications, data and content, so as to finally display and play various audio and video content.
  • the CPU processor 212 may include multiple processors.
  • the multiple processors may include one main processor and multiple or one sub-processors.
  • the main processor is used to perform some operations of the display device 200 in the pre-power-on mode, and/or to display images in the normal mode.
  • the communication interface may include the first interface 218-1 to the nth interface 218-n. These interfaces may be network interfaces connected to external devices via a network.
  • the controller 210 may control the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 280, the controller 210 may perform an operation related to the object selected by the user command.
  • the object may be any one of the selectable objects, such as a hyperlink or an icon.
  • Operations related to the selected object for example: display operations connected to hyperlink pages, documents, images, etc., or perform operations corresponding to the icon.
  • the user command for selecting the UI object may be a command input through various input devices (for example, a mouse, a keyboard, a touch pad, etc.) connected to the display device 200 or a voice command corresponding to the voice spoken by the user.
  • the memory 290 includes storing various software modules for driving and controlling the display device 200.
  • various software modules stored in the memory 290 include: a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.
  • the basic module is the underlying software module used for signal communication between various hardware in the display device 200 and sending processing and control signals to the upper module.
  • the detection module is a management module used to collect various information from various sensors or user input interfaces, and perform digital-to-analog conversion and analysis management.
  • the voice recognition module includes a voice analysis module and a voice command database module.
  • the display control module is a module for controlling the display 280 to display image content, and can be used to play information such as multimedia image content and UI interfaces.
  • the communication module is a module used for control and data communication with external devices.
  • the browser module is a module used to perform data communication between browsing servers.
  • the service module is a module used to provide various services and various applications.
  • the memory 290 is also used to store and receive external data and user data, images of various items in various user interfaces, and visual effect diagrams of focus objects.
  • the user input interface is used to send the user's input signal to the controller 210, or to transmit the signal output from the controller to the user.
  • the control device (such as a mobile terminal or a remote control) may send input signals input by the user, such as a power switch signal, a channel selection signal, and a volume adjustment signal, to the user input interface, and then the user input interface forwards the input signal to the controller;
  • the control device may receive output signals such as audio, video, or data output from the user input interface processed by the controller, and display the received output signal or output the received output signal as audio or vibration.
  • the user may input a user command through a graphical user interface (GUI) displayed on the display 280, and the user input interface receives the user input command through the graphical user interface (GUI).
  • GUI graphical user interface
  • the user can input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.
  • the video processor 260-1 is used to receive video signals, and perform video data processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard codec protocol of the input signal.
  • the video signal displayed or played directly on the display 280.
  • the video processor 260-1 includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like.
  • the demultiplexing module is used to demultiplex the input audio and video data stream. For example, if MPEG-2 is input, the demultiplexing module will demultiplex into a video signal and an audio signal.
  • the video decoding module is used to process the demultiplexed video signal, including decoding and scaling.
  • An image synthesis module such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator with the zoomed video image according to user input or itself, to generate an image signal for display.
  • Frame rate conversion module used to convert the frame rate of the input video, such as converting the frame rate of the input 24Hz, 25Hz, 30Hz, 60Hz video to the frame rate of 60Hz, 120Hz or 240Hz, where the input frame rate can be compared with the source
  • the video stream is related, and the output frame rate can be related to the update rate of the display.
  • the input has a usual format, such as frame insertion.
  • the display formatting module is used to change the signal output by the frame rate conversion module into a signal that conforms to a display format such as a display, such as format conversion of the signal output by the frame rate conversion module to output RGB data signals.
  • the video processor and the graphics processor can be integrated in the chip in a unified manner.
  • the functional modules of the graphics processor and the video processor can be configured as required.
  • the display 280 is used to receive the image signal input from the video processor 260-1, display video content and images, and a menu control interface.
  • the display 280 includes a display component for presenting a picture and a driving component for driving image display.
  • the displayed video content can be from the video in the broadcast signal received by the tuner and demodulator 220, or from the video content input by the communicator or the external device interface.
  • the display 220 simultaneously displays a user manipulation interface UI generated in the display device 200 and used to control the display device 200.
  • the display 280 it also includes a driving component for driving the display.
  • the display 280 is a projection display, it may also include a projection device and a projection screen.
  • the audio processor 260-2 is used to receive audio signals, and perform decompression and decoding according to the standard codec protocol of the input signal, as well as audio data processing such as noise reduction, digital-to-analog conversion, and amplification processing, and the result can be in the speaker 272 The audio signal to be played.
  • the audio output interface 270 is used to receive the audio signal output by the audio processor 260-2 under the control of the controller 210.
  • the audio output interface may include a speaker 272 or output to an external audio output terminal 274 of a generator of an external device, such as : External audio terminal or headphone output terminal, etc.
  • the video processor 260-1 may include one or more chips.
  • the audio processor 260-2 may also include one or more chips.
  • the video processor 260-1 and the audio processor 260-2 may be separate chips, or they may be integrated with the controller 210 in one or more chips.
  • the power supply is used to provide power supply support for the display device 200 with power input from an external power supply under the control of the controller 210.
  • the power supply may include a built-in power supply circuit installed inside the display device 200, or may be a power supply installed outside the display device 200, such as a power interface that provides an external power supply in the display device 200.
  • the A chip may include a controller 310, a communicator 330, a detector 340, and a memory 390. In some embodiments, it may also include a user input interface, a video processor, an audio processor, a display, and an audio output interface. In some embodiments, there may also be a power supply that independently powers the A chip.
  • the communicator 330 is a component for communicating with external devices or external servers according to various communication protocol types.
  • the communicator 330 may include a WIFI module 331, a Bluetooth communication protocol module 332, a wired Ethernet communication protocol module 333, and an infrared communication protocol module and other network communication protocol modules or near field communication protocol modules.
  • the communicator 330 of the A chip and the communicator 230 of the N chip also interact with each other.
  • the WiFi module 231 in the N chip hardware system is used to connect to an external network and generate network communication with an external server and the like.
  • the WiFi module 331 in the A chip is used to connect to the WiFi module 231 in the N chip hardware system without direct connection with an external network or the like, and the A chip connects to the external network through the N chip. Therefore, for the user, a display device as in the above embodiment can externally display a WiFi account.
  • the detector 340 is a component used by the chip of the display device A to collect signals from the external environment or interact with the outside.
  • the detector 340 may include a light receiver 342, a sensor used to collect the intensity of ambient light, which can adaptively display parameter changes by collecting ambient light, etc.; it may also include an image collector 341, such as a camera, a camera, etc., which can be used to collect external Environmental scenes, as well as gestures used to collect user attributes or interact with users, can adaptively change display parameters, and can also recognize user gestures to achieve the function of interaction with users.
  • the external device interface provides components for data transmission between the controller 310 and the N chip or other external devices.
  • the external device interface can be connected to external devices such as set-top boxes, game devices, notebook computers, etc., in a wired/wireless manner.
  • the controller 310 controls the work of the display device 200 and responds to user operations by running various software control programs (such as installed third-party applications, etc.) stored on the memory 390 and interacting with the N chip.
  • various software control programs such as installed third-party applications, etc.
  • the controller 310 includes a read-only memory ROM 313, a random access memory RAM 314, a graphics processor 316, a CPU processor 312, a communication interface 318, and a communication bus.
  • the ROM 313 and the RAM 314, the graphics processor 316, the CPU processor 312, and the communication interface 318 are connected by a bus.
  • the CPU processor 312 runs the system startup instruction in the ROM, and copies the temporary data of the operating system stored in the memory 390 to the RAM 314 to run or start the operating system. After the operating system is started, the CPU processor 312 copies the temporary data of the various application programs in the memory 390 to the RAM 314, and then runs or starts the various application programs.
  • the CPU processor 312 is used to execute operating system and application instructions stored in the memory 390, communicate with the N chip, transmit and interact with signals, data, instructions, etc., and execute according to various interactive instructions received from external inputs Various applications, data and content, in order to finally display and play various audio and video content.
  • the communication interface may include the first interface 318-1 to the nth interface 318-n. These interfaces may be network interfaces connected to external devices via a network, or network interfaces connected to the N chip via a network.
  • the controller 310 may control the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 280, the controller 210 may perform an operation related to the object selected by the user command.
  • the graphics processor 316 is used to generate various graphics objects, such as icons, operation menus, and user input instructions to display graphics. Including an arithmetic unit, which performs operations by receiving various interactive commands input by the user, and displays various objects according to display attributes. As well as including a renderer, various objects obtained based on the arithmetic unit are generated, and the rendering result is displayed on the display 280.
  • Both the graphics processor 316 of the A chip and the graphics processor 216 of the N chip can generate various graphics objects. Differentily, if application 1 is installed on the A chip and application 2 is installed on the N chip, when the user is in the interface of the application 1 and the user inputs instructions in the application 1, the A chip graphics processor 316 generates a graphic object. When the user is on the interface of Application 2 and performs the user-input instructions in Application 2, the graphics processor 216 of the N chip generates the graphics object.
  • Fig. 5 exemplarily shows a schematic diagram of a functional configuration of a display device according to an exemplary embodiment.
  • the memory 390 of the A chip and the memory 290 of the N chip are respectively used to store the operating system, application programs, content and user data, etc., under the control of the controller 310 of the A chip and the controller 210 of the N chip. Perform system operations that drive the display device 200 and respond to various operations of the user.
  • the memory 390 of the A chip and the memory 290 of the N chip may include volatile and/or nonvolatile memory.
  • the memory 290 is specifically used to store the operating program that drives the controller 210 in the display device 200, and store various application programs built in the display device 200, and various application programs downloaded by the user from an external device, and application programs.
  • the memory 290 is used to store system software such as an operating system (OS) kernel, middleware, and applications, and to store input video data and audio data, and other user data.
  • OS operating system
  • the memory 290 is specifically used to store driver programs and related data such as the video processor 260-1 and the audio processor 260-2, the display 280, the communication interface 230, the tuner and demodulator 220, and the input/output interface.
  • the memory 290 may store software and/or programs.
  • the software programs used to represent an operating system (OS) include, for example, a kernel, middleware, application programming interface (API), and/or application programs.
  • OS operating system
  • the kernel may control or manage system resources, or functions implemented by other programs (such as the middleware, API, or application program), and the kernel may provide interfaces to allow middleware and APIs, or applications to access the controller , To achieve control or management of system resources.
  • the memory 290 includes a broadcast receiving module 2901, a channel control module 2902, a volume control module 2903, an image control module 2904, a display control module 2905, an audio control module 2906, an external command recognition module 2907, a communication control module 2908, and an optical receiver Module, power control module 2910, operating system 2911, and other application programs 2912, browser module, etc.
  • the controller 210 executes various software programs in the memory 290, such as: broadcast and television signal reception and demodulation function, TV channel selection control function, volume selection control function, image control function, display control function, audio control function, external command Various functions such as identification function, communication control function, optical signal receiving function, power control function, software control platform supporting various functions, and browser function.
  • the memory 390 includes storing various software modules for driving and controlling the display device 200.
  • various software modules stored in the memory 390 include: a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules. Since the functions of the memory 390 and the memory 290 are relatively similar, please refer to the memory 290 for related parts, and will not be repeated here.
  • the memory 390 includes an image control module 3904, an audio control module 2906, an external command recognition module 3907, a communication control module 3908, an optical receiving module 3909, an operating system 3911, and other application programs 3912, a browser module, and so on.
  • the controller 210 executes various software programs in the memory 290, such as: image control function, display control function, audio control function, external command recognition function, communication control function, light signal receiving function, power control function, support for various Functional software control platform, and various functions such as browser functions.
  • the external command recognition module 2907 of the N chip and the external command recognition module 3907 of the A chip can recognize different commands.
  • the external command recognition module 3907 of the A chip may include a graphic recognition module 2907-1.
  • the graphic recognition module 3907-1 stores a graphic database, and the camera receives external commands. In order to control the display device, the corresponding relationship is made with the instructions in the graphics database.
  • the voice receiving device and the remote controller are connected to the N chip, the external command recognition module 2907 of the N chip may include a voice recognition module 2907-2.
  • the graphics recognition module 2907-2 stores a voice database, and the voice receiving device, etc.
  • the external voice commands or time correspond to the commands in the voice database to control the display device.
  • a control device 100 such as a remote controller is connected to the N chip, and the key command recognition module interacts with the control device 100.
  • Fig. 6a exemplarily shows a configuration block diagram of the software system in the display device 200 according to an exemplary embodiment.
  • the operating system 2911 includes operating software for processing various basic system services and for implementing hardware-related tasks.
  • part of the operating system kernel may include a series of software to manage the hardware resources of the display device and provide services for other programs or software codes.
  • part of the operating system kernel may include one or more device drivers, and the device drivers may be a set of software codes in the operating system to help operate or control devices or hardware associated with the display device.
  • the drive may contain code to manipulate video, audio, and/or other multimedia components. Examples include displays, cameras, Flash, WiFi, and audio drivers.
  • the accessibility module 2911-1 is used to modify or access the application program, so as to realize the accessibility of the application program and the operability of its display content.
  • the communication module 2911-2 is used to connect to other peripherals via related communication interfaces and communication networks.
  • the user interface module 2911-3 is used to provide objects that display the user interface for access by various applications, and can realize user operability.
  • the control application 2911-4 is used to control process management, including runtime applications.
  • the event transmission system 2914 can be implemented in the operating system 2911 or in the application 2912. In some embodiments, it is implemented in the operating system 2911 on the one hand, and implemented in the application program 2912 at the same time, for monitoring various user input events, and responding to the recognition results of various events or sub-events according to various events. And implement one or more sets of pre-defined operation procedures.
  • the event monitoring module 2914-1 is used to monitor input events or sub-events of the user input interface.
  • Event recognition module 2914-2 used to input the definition of various events to various user input interfaces, recognize various events or sub-events, and transmit them to the processing to execute the corresponding one or more groups of processing programs .
  • the event or sub-event refers to the input detected by one or more sensors in the display device 200 and the input of an external control device (such as the control device 100).
  • an external control device such as the control device 100.
  • various sub-events of voice input, gesture input sub-events of gesture recognition, and sub-events of remote control button command input of control devices include multiple forms, including but not limited to one or a combination of pressing up/down/left/right/, confirming keys, and pressing keys.
  • non-physical buttons such as moving, pressing, and releasing.
  • the interface layout management module 2913 which directly or indirectly receives various user input events or sub-events monitored by the event transmission system 2914, is used to update the layout of the user interface, including but not limited to the position of each control or sub-control in the interface, and the container
  • the size, position, level, etc. of the interface are related to various execution operations.
  • the application layer of the display device includes various applications that can be executed on the display device 200.
  • the application layer 2912 of the N chip may include, but is not limited to, one or more applications, such as video-on-demand applications, application centers, and game applications.
  • the application layer 3912 of the A chip may include, but is not limited to, one or more applications, such as a live TV application, a media center application, and so on. It should be noted that the application programs contained on the A chip and the N chip are determined according to the operating system and other designs. This application does not need to specifically limit and divide the application programs contained on the A chip and the N chip.
  • Live TV applications can provide live TV through different sources.
  • a live TV application may use input from cable TV, wireless broadcasting, satellite services, or other types of live TV services to provide TV signals.
  • the live TV application can display the video of the live TV signal on the display device 200.
  • Video-on-demand applications can provide videos from different storage sources. Unlike live TV applications, VOD provides video display from certain storage sources. For example, the video on demand can come from the server side of cloud storage, and from the local hard disk storage that contains the stored video programs.
  • Media center applications can provide various multimedia content playback applications.
  • the media center can provide services that are different from live TV or video on demand, and users can access various images or audio through the media center application.
  • Application center can provide storage of various applications.
  • the application program may be a game, an application program, or some other application program that is related to a computer system or other device but can be run on a display device.
  • the application center can obtain these applications from different sources, store them in the local storage, and then run on the display device 200.
  • the operating system includes a kernel, a database, a framework layer, and an application layer, and the above-mentioned applications are all applications in the application layer.
  • FIG. 7 exemplarily shows a schematic diagram of a user interface in the display device 200 according to an exemplary embodiment.
  • the user interface includes multiple view display areas, for example, a first view display area 201 and a play screen 202, where the play screen includes layout of one or more different items.
  • the user interface also includes a selector indicating that the item is selected, and the position of the selector can be moved through user input to change the selection of different items.
  • multiple view display areas can present display screens of different levels.
  • the first view display area can present video chat item content
  • the second view display area can present application layer item content (eg, webpage video, VOD display, application program screen, etc.).
  • the presentation of different view display areas has priority differences, and the display priority of the view display areas is different between view display areas with different priorities.
  • the priority of the system layer is higher than that of the application layer.
  • the screen display in the view display area of the system layer is not blocked; and the application layer is enabled according to the user's choice
  • the size and position of the view display area change the size and position of the view display area of the system layer will not be affected.
  • the same level of display screen can also be presented.
  • the selector can switch between the display area of the first view and the display area of the second view, and when the size and position of the display area of the first view change, the second view The size and position of the display area can be changed at any time.
  • both the A chip and the N chip can be independently installed with Android and various APPs, so that each chip can realize a certain function, and the A chip and the N chip can cooperate to realize a certain function.
  • the controller of the display device may be located in only one physical chip, and may include two chips as in the above embodiments, and the controllers in the A chip and the N chip together serve as the control module.
  • the first aspect of some embodiments of the present application shows a display device, including:
  • a display screen for displaying a first display interface and the second display interface the first display interface includes a playback control for controlling the playback of a demonstration video
  • the second display interface includes a display for playing the demonstration video The first play window of and the second play window used to play the local image collected by the camera;
  • the first interface of the display device in the fitness environment is shown in Figure 8-1.
  • 8-1 is a schematic diagram of the first interface 200A, where the first interface 200A can display multiple demonstration videos in a scrolling manner, so that the user can select a target demonstration video among the multiple demonstration videos.
  • the display window is used to display the demonstration video selected by the user.
  • the controller may obtain and load the corresponding demonstration video source from the server based on API (Application Programming Interface, application program interface).
  • Figure 8-2 is a schematic diagram of the first interface of a fitness video according to some embodiments.
  • the first interface page of the fitness video can be called a detail interface, where the first interface can display more information in a scrolling manner.
  • a coaching video so that the user can select a target demonstration video from multiple demonstration videos. For example: squat high leg lift, backward lunge step, four-point rear kick... The user selects a target demonstration video among multiple coaching videos.
  • the play window is used to play the default training video or the last training video in the play history.
  • an introduction display control On the right side of the play window is set an introduction display control, "Start training" control (ie play control ), at least one of the "favorite" controls.
  • the interface also includes a training list control. In the training list control, display controls for multiple training videos are displayed.
  • the demonstration video may also be obtained after verification. Specifically, it is shown that the demonstration video is downloaded and stored in advance; and then the mapping relationship between the demonstration video and the check code is established.
  • a check code is generated based on the demonstration video selected by the user.
  • the controller may obtain the demonstration video corresponding to the check code in the stored demonstration videos based on the check code. Since the demonstration video is pre-stored, the controller can directly retrieve the demonstration video corresponding to the check code after obtaining the check code. Retrieving the demonstration video in the above manner can avoid the stuttering problem caused by factors such as the network, and the previous acquisition can be downloaded in advance to improve the smoothness of the demonstration video.
  • the camera is used to collect local images or local videos; when it is not turned on, the camera is in a hidden position to keep the edge of the display device smooth. After it is turned on, the camera rises and protrudes above the edge of the display device to avoid Obscure the display screen to obtain image data.
  • the camera in response to the user's selection of the start training control, the camera is raised and started to obtain image data.
  • the camera is always on, and the local video is collected in real time, and the collected local video is sent to the controller To display the user's actions on the follow-up interface. In this way, users can watch their actions in real time and demonstrate the actions of the video.
  • the camera in response to the user's selection of the start training control, the camera is raised but in a standby state. Whenever the demonstration video is played to a preset time point, a local image is collected and the collected local image is sent to the controller . This reduces the pressure on the processor and maintains the local image on the display until the next point in time.
  • a controller the controller is configured to: receive an input confirmation operation on the playback control, start a camera, and load the video data of the demonstration video;
  • the confirmation operation may be a selection of the start training control.
  • the controller is further configured to control the display to display a prompt interface 200C (and a guide interface) for instructing the user to enter a predetermined area.
  • a prompt interface 200C and a guide interface
  • FIG. 9-1 is a schematic diagram showing a prompt interface according to some embodiments, and the user adjusts his position according to the prompt interface.
  • the controller controls the display to display the second interface. This is because the camera's collection area has edges. In order to better collect local data, the camera acquires the current image and creates a new floating layer above the layer displaying the current image during the display process.
  • the character in the prompt interface 200C is located on the left side of the box area 200C1, and the user is correspondingly prompted to move to the right. If the character in the display screen is located on the right side of the box area, the user is correspondingly prompted to move to the left so that the user enters the predetermined area, which can be captured by the camera. The embodiment of the present application shows that the user is instructed to enter the predetermined area through the foregoing method.
  • the prompt interface is also used to display prompt information.
  • Figure 9-2 is a schematic diagram of a prompt interface according to some embodiments.
  • the prompt message is "Please face Screen. Reminders such as "Keep your body independent”.
  • the message that prompts the user to move can be a text displayed on the floating layer, a voice reminder, or an indication mark pointing to the best position frame.
  • the controller may also directly display the second interface in response to the confirmation operation, and play the demonstration video in the first play window, and play the demo video in the second video window.
  • the local image The user can adjust his position according to the image displayed in the second video window in the second interface.
  • the controller may also determine the number of appearances of the location guidance interface in response to the confirmation operation, display the guidance interface when the display times of the guidance interface does not meet the preset value, and directly display when the preset value is satisfied
  • the second interface and the demonstration video is played in the first play window, and the local image is played in the second video window. The user can adjust his position according to the image displayed in the second video window in the second interface.
  • FIG. 10 is a schematic diagram of a second display interface 200B according to some embodiments, wherein the second display interface 200B includes a first play window 200B1 for playing the exemplary video and Play the second play window 200B2 of the local image collected by the camera.
  • the demonstration video is played in the first play window, no joint points are shown in the demonstration video, and the second play serial port to play the local image data includes the controller obtaining the joints of the accompanying local image data according to the local image data.
  • the position of the point is displayed in the second playback window after superimposing the local image data and the joint point mark according to the position of the joint point.
  • superimposing the local image data and the joint point markers can add the joint point markers to the local image according to the positions of the joint points in the local image data, and then output in a layer to display the superimposed joint points.
  • image You can also display the local image obtained by the camera in a layer, add a floating layer above the layer, and add articulated surface markers to the floating layer according to the position of the joint points, and display the two layers after being superimposed.
  • the second playback window directly plays the local video collected by the camera.
  • the embodiment of the present application shows a display device, which includes a display, a camera, and a controller.
  • the controller is configured to, in response to the user's selection of the start training control in the display interface, acquire a demonstration video, raise and turn on the camera, which is used to collect local graphics; control the first playback window of the display to play the demonstration video,
  • the second playing window of the display displays the local image.
  • the technical solution shown in the embodiment of this application shows the demonstration video through the first play window, and the second play window displays the local pictures.
  • the user can adjust his/her own performance in time by comparing the contents displayed in the two windows. Action to improve user experience.
  • the camera is used to collect local video, which is a collection of continuous local images.
  • the comparison process if the comparison is performed for each frame of image, the data processing capacity of the controller is relatively large.
  • the controller can compare the local image with the demonstration video frame to generate the comparison result. After the user exercises, he can watch the comparison result through the display interface, thereby helping the user to better understand Own movement defects, so that the user can overcome movement defects in the subsequent fitness process.
  • the exemplary video frame is a graphic corresponding to the local image in the exemplary video.
  • the controller can control the camera to collect a local image when the demonstration video is played to a preset time point; and then compare the local image collected by the camera with the pre-stored demonstration video frame to obtain the comparison result.
  • the controller when the demonstration video is played to a preset time point, the controller will control the camera to collect a local image.
  • the preset time point may be the appearance of the first image in the demonstration video as a starting point, and each interval T is a preset time point until the last frame of image appears in the demonstration video.
  • the preset time point may also be generated based on the content of the demonstration video, and each action node in the content of the demonstration video serves as a preset time point.
  • the starting point of the first image is 3s
  • the T time is 10S
  • the length of the demonstration video is 53S.
  • the corresponding preset time points are: 3S, 13S, 23S, 33S, 43S and 53S.
  • the controller will control the camera to collect a local image when the video is released to 3S, 13S, 23S, 33S, 43S and 53S.
  • tags will be added according to the preset time node, and local images will be collected when the tag starts at the playback time.
  • the camera is always on, recording local video in real time, and sending the local video to the controller.
  • the controller may extract the corresponding local image in the collected local video at a preset time point; then compare the extracted local image with the pre-stored demonstration video frame to obtain the comparison result.
  • the specific implementation process when the demonstration video is played to a preset time point, the controller extracts one or more local images from the local video collected by the camera.
  • the preset time point may be the appearance of the first image in the demonstration video as the starting point, and every interval T (ie, the time when the demonstration action occurs) is a preset time point until the last frame of image appears in the demonstration video.
  • the preset time point may also be generated based on the content of the demonstration video or pre-marked, and each action node in the content of the demonstration video serves as a preset time point.
  • the starting point of the first image is 3S
  • the preset time points are: 3S, 16S, 23S, 45S, and 53S.
  • the controller will release the video to 3S, 16S, 23S, 45S, and 53S.
  • the appearance time of the demonstration action is arbitrary, and the acquisition of the image to be compared is triggered according to the tag or time point identifying the demonstration action.
  • the user imitates the coaching action to produce the corresponding action.
  • the technical solution shown in the embodiment of the present application shows a "delayed image acquisition method".
  • the technical solutions shown in the embodiments of this application have used a large amount of experimental data statistics.
  • the response time of the user during the period from the time the user receives the demonstration action to the corresponding action is 1S
  • the corresponding technical solution shown in the embodiment of the present application configures the preset response time to 1S.
  • the preset time points are: 3S, 13S, 23S, 33S, 43S, and 53S
  • the corresponding delayed collection time points are: 4S, 14S, 24S, 24S, 44S, and 54S.
  • the image frame appears for the first time in the demonstration video as the starting point. After the starting point, the 14S, 14S, 24S, 24S, 44S and 54S controllers control the cameras to collect local images respectively.
  • the controller compares the local image with the demonstration video frame to generate a comparison result; wherein the demonstration video frame is an image corresponding to the local image in the demonstration video, or a standard image corresponding to the preset time point in the demonstration video frame.
  • the image in the demonstration video may be an image with a logo.
  • the mark may be a time mark, but is not limited to a time mark.
  • the correspondence between the local image and the exemplary video frame may be determined based on the flag. For example, when the demonstration video is played to 4S, the controller controls the camera to collect the local view, the time mark corresponding to the local video is 3s, and the local video is compared with the target video whose time mark is 3S.
  • the joint point data of the demonstration video frame is stored in the demonstration video, and the joint point data is preset in advance. Since there is no demand for other image frames except the demonstration video frame, the joint points may not be preset data.
  • the preset reaction time is configured to be 1S.
  • 1S is a statistical data.
  • the user's response time is 1S, but 1S is not suitable for all users.
  • the preset response time can be set according to requirements.
  • Action comparison through image comparison will cause a greater processing burden.
  • the technical solution shown in this embodiment of the application can only compare the local image with some "key parts" in the demonstration video.
  • the specific realization process that is, the comparison of actions is completed by the comparison of joint points.
  • the controller is further configured to: identify the joint points in the local image; compare the joint points in the local image with the joint points in the demonstration video.
  • the controller is configured to, in response to the user selecting the start training control in the first display interface, the controller controls the camera to start, so that the camera collects local images.
  • the camera transmits the captured local image to the controller.
  • the controller recognizes the joint points in the local image; in some embodiments, the controller recognizes the joint points in the local image according to a preset model, and the joint points are the points corresponding to the human joints and the corresponding points of the human head ,
  • the human body includes 13 joint positions.
  • the controller marks 13 important bone joints in the whole body. Among them, the local image with 13 joint positions can be seen in Figure 11.
  • the 13 joint positions are: left wrist, left elbow, left shoulder, thoracic cavity, waist, left knee, left ankle, head, right wrist Department, right elbow, right shoulder, right knee, right ankle.
  • the human body is sometimes partially missing. At this time, only the human body part in the image can be identified.
  • the controller is also used to compare the joint points in the local image with the joint points in the demonstration video/demonstration video frame, and determine the degree of difference between the human motion in the local image and the human motion in the demonstration video;
  • the recognized joint points are marked in the image, and different colors are used to mark joint points with different degrees of motion difference.
  • the comparison method may be to compare the position of the joint points of the human body in the local image with the relative position of the joint points of the human body in the demonstration video.
  • the comparison result is obtained based on the difference in relative position. Different comparison results are marked with different colors.
  • the left wrist of the human body in the local image differs from the position of the left wrist of the human body in the demonstration video by 10 standard values, and the left wrist joint points can be marked in red.
  • the position of the right wrist of the human body in the local image and the right wrist of the human body in the demonstration video differs by 1 standard value, and the joint points of the right wrist can be marked in green.
  • the comparison method can calculate the matching degree of two joint positions, and generate corresponding results according to the matching degree. Or according to the relative position relationship between the own joint points, the matching degree of the action is determined.
  • the recognition and matching of joint points can also be replaced by other achievable means in related technologies.
  • the demonstration joint positions are marked in the demonstration video in advance, and the demonstration joint positions are stored together with the demonstration video in a local data list.
  • the marking process of the exemplary joint position is similar to the marking process shown in some of the foregoing embodiments, and will not be described in detail here.
  • the controller compares the first angle in the local image with the corresponding standard angle to generate a comparison result.
  • the first angle is the angle between the connection between each joint position and the adjacent joint position and the trunk connection in the local image
  • the standard angle is the angle between the connection between each joint position and the adjacent joint position and the trunk connection in the demonstration video The angle between.
  • the corresponding relationship between the first angle and the standard angle can be generated based on the time stamp.
  • the local image acquisition time is 10S
  • the standard angle corresponding to the first angle of the left ankle is the line between the left ankle and the adjacent joint position and the torso line in the image that appears in the 10s in the demonstration video Included angle.
  • FIG. 12 is a local image with joint annotations according to some embodiments.
  • the joint position adjacent to the left wrist 1A is the left elbow 1B, and the corresponding angle between the connecting line of the left wrist 1A and the left elbow 1B and the trunk is called the first angle 1a.
  • the first angles corresponding to the left elbow, left shoulder, left knee, left ankle, head, right wrist, right elbow, right shoulder, right knee, and right ankle can be calculated respectively.
  • the generation method of the standard angle can refer to the generation method of the first angle, which will not be repeated here.
  • the controller calculates the degree of matching between the first angle and the corresponding standard angle; the degree of matching can be used to evaluate the degree of completion of the user's action.
  • the technical party shown in the embodiment of the present application can calculate the difference between the position of each joint position and the standard position, thereby helping the user to complete each part and improving the user experience.
  • the controller calculates the matching degree between the first angle and the corresponding standard angle, and marks the corresponding joint points according to the matching area. s color.
  • the matching degree may be represented by an angular deviation, and the matching result is based on a preset standard deviation value.
  • the corresponding joint position For the angle deviation greater than 15 degrees, the corresponding joint position can be marked as red; for the deviation of 10 degrees to 15 degrees, the corresponding joint position can be marked as yellow. For degree deviations below 10 degrees, the corresponding joint position can be marked in green.
  • the first angle of the left wrist joint in the local image collected by 10S should be 20 degrees different from the standard angle corresponding to the 10s in the demonstration video.
  • the corresponding left wrist joint can be marked in red; the local image collected by 10S should be the left ankle
  • the difference between the first angle of the joint and the standard angle corresponding to 10s in the demonstration video is 12 degrees, and the corresponding left wrist joint can be marked as yellow; the first angle of the head in the local image collected by 10S corresponds to the 10s in the demonstration video
  • the standard angle difference of 6 degrees, the corresponding left wrist joint can be marked in green.
  • the marked local image can be seen in Figure 13.
  • the technical solutions shown in the embodiments of the present application show a "range" comparison method. That is, when the demonstration video is played to the demonstration video frame, the display device obtains multiple image frames adjacent to the time point from the local video.
  • the controller when the demonstration video is played to the preset time point, the controller is in the local Multiple image frames adjacent to the time point are selected from the video as the first image set, the first image set includes at least a first local image and a second local image, where the first local image is a local image corresponding to a preset time point , The second local image is a local image corresponding to a near preset time point.
  • the controller calculates the degree of matching between the local image in the first image set and the exemplary video frame, and uses the comparison result of the local image with the highest degree of matching as the comparison result at the time point, and compares it with the exemplary video frame.
  • the local image with the best matching degree is taken as the local image corresponding to the time point.
  • the controller calculates the degree of matching between the first local image and the demonstration video frame (also referred to as the degree of difference in human motion).
  • the controller In the first image set, the image with the highest degree of matching with the exemplary video frame is selected as the replacement image, and the replacement image is marked according to the comparison result of the replacement image and the exemplary video frame.
  • the matching degree is 20%, and the preset matching degree (preset threshold) is 25%.
  • the controller determines the first image set of the target data set; the first image set is the local images contained in the target data set in the period of 1S-13S.
  • the calculation result is that the matching degree between the data corresponding to 8S and the standard angle of the wrist joint in the 10S demonstration video frame is 80%, 80% is the highest matching degree.
  • the comparison result of the wrist joint corresponding to 10S is adjusted to 80%, and the wrist joint is marked with the color of 80% matching, and the controller caches the marked local video.
  • the controller may control the display to display an exercise evaluation interface, the exercise evaluation interface being used to display the annotated local pictures.
  • the exercise evaluation interface can display the user's rating level, user actions and standardized actions at the same time.
  • the scoring level can be generated based on the matching degree between the local image and the demonstration video frame.
  • the exercise evaluation interface may display multiple corresponding user actions and standardized actions in a scrolling manner.
  • the order of display can be: display in order from low to high score.
  • the display form of the exercise evaluation interface can be seen in Figure 14-1.
  • the exercise evaluation interface is equipped with two display windows, one displays the local image corresponding to the time point in the user action, and the other displays the demonstration corresponding to the standard action. Video frame.
  • the'joint point comparison process' can be placed on the server side for execution.
  • the controller before the local image is played in the second video window, the controller is further configured to: identify the joint points in the local image; and send the joint points in the local image to the server ,
  • the server enables the server to compare the joint points in the local image with the joint points of the demonstration video frame in the demonstration video, determine the degree of difference between the human motion in the local image and the demonstration video frame in the demonstration video, and Generate feedback information to the display device.
  • the joint point recognition unit in the display device recognizes and marks the joint points of all images collected by the camera, and displays the joint points in the second playback window.
  • the display device uploads the joint point data of the local image collected at this moment and/or the joint point data of the local image collected at an adjacent time to the server to determine the matching degree.
  • the comparison method of the human body motion in the local image and the human body motion difference degree in the demonstration video can refer to the above-mentioned embodiment, which will not be repeated here.
  • the controller is further configured to receive a feedback message sent by the server, and mark the identified joint points in the local image according to the feedback message, wherein different colors mark joint points with different degrees of motion difference.
  • the technical solutions shown in the embodiments of the present application use different colors to mark the completion of each joint position. Different colors are used to distinguish the completion of each joint of the user, and different colors play a striking role. It can be seen that the solution shown in the embodiment of the present application is used to further help the user to understand the completion of various parts of the action.
  • a training progress control is also provided above the second play window to show the degree of completion of the user action.
  • the controller detects that the user action When the matching degree with the demonstration action frame is higher than the preset value, the completion degree value displayed in the control training progress control is increased. When it is detected that the matching degree between the user action and the demonstration action frame is lower than the preset value, the completion degree value displayed in the control training progress control remains unchanged.
  • the server may process the local image corresponding to the preset time point.
  • the specific implementation process may be: the controller sends the joint points in the local image to
  • the server is specifically: the controller, when the playing time of the demonstration video reaches a preset time point, buffers local images collected within a predetermined time period before and after the preset time point. Identify the joint point data of the cached local image and send the identified joint point data to the server.
  • the caching process of the local image can be referred to the above implementation and will not be repeated here.
  • the solution shown in this embodiment sends the joint points of the local video to the server.
  • the controller may be further configured to send the identified joint point data to the server, and at the same time send a preset time point to a server, so that the server can be configured according to the The preset time point determines the image frame (that is, the target image) of the exemplary video for comparison.
  • the controller marks the local image and caches the marked picture of the local image and the demonstration video frame corresponding to the preset time point when the degree of difference in human motion is greater than a preset threshold. So that when the demonstration video is over, the local video with a large difference in action can be retrieved.
  • the controller controls the display to display the exercise evaluation interface after the playback ends, and displays the cached and annotated local image pictures and the demonstration video corresponding to the preset time point on the exercise evaluation interface frame.
  • the demonstration video frames at the preset time point and the corresponding local images are sorted according to the matching degree (or score), and after the demonstration video is played, the preset number with low matching degree (or score) is selected
  • the demonstration video frame and the corresponding local image at the time point are displayed.
  • the demonstration video frames at 5 time points and the corresponding local images are cached according to the matching degree, and after the demonstration video is played, the demonstration video frames at 3 time points with a low matching degree (or score) and The corresponding local image is displayed.
  • the embodiment of this application also shows a display device, including:
  • a display screen for displaying a first display interface and the second display interface the first display interface includes a playback control for controlling the playback of a demonstration video
  • the second display interface includes a display for playing the demonstration video The first play window of and the second play window used to play the local image collected by the camera;
  • Camera used to collect local images
  • a controller configured to:
  • the current video frame or the neighboring video frame and the demonstration video frame whose marked difference degree of the human body movement is lower than the difference threshold are buffered for display of the exercise evaluation interface.
  • the embodiment of the present application also shows a display device, including:
  • a display screen for displaying a first display interface and the second display interface the first display interface includes a playback control for controlling the playback of a demonstration video
  • the second display interface includes a display for playing the demonstration video The first play window of and the second play window used to play the local image collected by the camera;
  • Camera used to collect local images
  • a controller configured to:
  • the second interface is displayed, the demonstration video is played in the first play window, and the local image with joint points marked is played in the second video window.
  • the first joint point is marked as the first color
  • the second joint point is marked as the second color
  • the degree of movement difference of is greater than the degree of movement difference between the body part where the second joint point is located and the corresponding body part in the demonstration video.
  • the embodiment of the present application also shows an interface display method, including:
  • the second interface is displayed, the demonstration video is played in the first play window in the second interface, and the local image is played in the second video window in the second interface .
  • the embodiment of the present application also shows an interface display method, including:
  • the current video frame of the collected local video and the neighboring video frames ( The video frame may also be called an image in the solution shown in the embodiment of the present application);
  • an exercise evaluation interface is displayed, wherein the exercise evaluation interface displays the current video frame or the neighboring video frame with a lower degree of difference in human body action after the annotation, and the demonstration video frame.
  • the embodiment of the present application also shows an interface display method, including:
  • the second interface is displayed, the demonstration video is played in the first play window in the second interface, and the joint points are marked in the second video window in the second interface
  • the first joint point is marked as a first color
  • the second joint point is marked as a second color
  • the first joint point is located
  • the motion difference between the body part and the corresponding body part in the demonstration video is greater than the motion difference between the body part where the second joint point is located and the corresponding body part in the demonstration video.
  • the method provided in this embodiment may be applicable to the dual-chip display device provided in the foregoing embodiments, and may also be applicable to non-dual-chip display devices

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Controls And Circuits For Display Device (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本申请实施例示出一种界面显示方法及显示设备，所述显示设备包括显示器、摄像头及控制器。控制器被配置为响应于用户对显示界面中开始训练控件的选中，获取示范视频，并开启摄像头，所述摄像头用于采集本地图形；控制显示器的第一播放窗口播放所述示范视频，显示器的第二播放窗口展示所述本地图像。可见，本申请实施例示出的技术方案，通过第一播放窗口展示示范视频，第二播放窗口展示本地图片，用户在练习的过程中可以通过两个窗口内展示内容的比对及时的调整自身的动作，提高用户的体验感。

Description

一种界面显示方法及显示设备
相关申请的交叉引用
本申请要求于2019年08月18日提交中国专利局、申请号为201910761455.8、申请名称为“一种健身界面显示方法及显示设备”的中国专利申请以及2020年05月09日提交中国专利局、申请号为202010386547.5、申请名称为“一种界面显示方法及显示设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及显示技术。更具体地讲，涉及一种界面显示方法及显示设备。
背景技术
当前，由于显示设备可以为用户提供诸如音频、视频、图片等的播放画面，受到用户的广泛关注。用户可以借助显示设备，完成一些事情。例如，一些用户通过显示设备播放教练的教学视频，并跟随教学视频完成相应的动作。随着大数据与人工智能的发展，用户对显示设备的功能需求与日俱增。用户想要观看教练教学视频帧的同时，看到用户自身的动作的视频帧，通过自身的动作的视频帧，教练教学视频帧的比对，发现自身动作的不足，并及时对自身的动作进行更正。
但是，当前的显示设备，无法满足用户的上述要求。
因此，迫切需要一种能在呈现教练教学视频帧的同时，还能呈现用户自身动作视频的显示设备，给用户以良好的用户体验。
发明内容
基于上述技术问题，本申请提供一种界面显示方法及显示设备。
本申请实施例示出一种显示设备，包括：
显示屏，用于显示第一显示界面和所述第二显示界面，所述第一显示界面包括用于控制播放示范视频的播放控件，所述第二显示界面中包含用于播放所述示范视频的第一播放窗口和用于播放摄像头采集到的本地图像的第二播放窗口；
摄像头，用于采集本地图像；
控制器，所述控制器被配置为：
接收输入的对所述播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
响应于所述确认操作，展示所述第二界面，并在所述第一播放窗口播放所述示范视频，在所述第二视频窗口播放所述本地图像。
本申请实施例示出一种显示设备，包括：
显示屏，用于显示第一显示界面和所述第二显示界面，所述第一显示界面包括用于控制播放示范视频的播放控件，所述第二显示界面中包含用于播放所述示范视频的第一播放窗口和用于播放摄像头采集到的本地图像的第二播放窗口；
摄像头，用于采集本地视频；
控制器，所述控制器被配置为：
接收输入的对所述播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
响应于所述确认操作，展示所述第二界面；
在示范视频播放过程中，检测到表征所述示范视频的播放时间达到预设时间点的标签时，截取采集到的本地视频的当前视频帧及时间上邻近所述当前视频帧的邻近视频帧；
识别所述当前视频帧的关节点与所述邻近视频帧的关节点；
比对所述当前视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频帧的关节点；比对所述邻近视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频帧的关节点；
根据比对结果标注所述前视频帧或邻近视频帧的人体动作差异程度；
缓存标注后的人体动作差异程度低于差异阈值的所述当前视频帧或所述邻近视频帧，以及所述示范视频帧，以用于练习评价界面的展示。
本申请实施例示出一种显示设备，包括：
显示屏，用于显示第一显示界面和所述第二显示界面，所述第一显示界面包括用于控制播放示范视频的播放控件，所述第二显示界面中包含用于播放所述示范视频的第一播放窗口和用于播放摄像头采集到的本地图像的第二播放窗口；
摄像头，用于采集本地图像；
控制器，所述控制器被配置为：
接收输入的对所述播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
响应于所述确认操作，展示所述第二界面，并在所述第一播放窗口播放所述示范视频，在所述第二视频窗口播放进行关节点标注后的所述本地图像，其中在所述关节点标注后的所述本地图像中，第一关节点被标注为第一颜色，第二关节点被标注为第二颜色，第一关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度大于第二关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度。
本申请实施例示出一种界面显示方法，包括：
在显示第一界面时，接收输入的对所述第一界面中的播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
响应于所述确认操作，显示所述第二界面，并在所述第二界面中的第一播放窗口播放所述示范视频，在所述第二界面中的第二视频窗口播放所述本地图像。
本申请实施例示出一种界面显示方法，所述方法包括：
在显示第一界面时，接收输入的对所述播放控件的确认操作，启动摄像头以使得所述摄像头采集本地视频，并加载所述示范视频的视频数据；
响应于所述确认操作，展示所述第二界面；
在示范视频播放过程中，检测到表征所述示范视频的播放时间达到预设时间点的标签时，截取采集到的本地视频的当前视频帧及时间上邻近所述当前视频帧的邻近视频帧；
识别所述当前视频帧的关节点与所述邻近视频帧的关节点；
比对所述当前视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频 帧的关节点；比对所述邻近视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频帧的关节点；
根据比对结果标注所述前视频帧或邻近视频帧的人体动作差异程度；
缓存标注后的人体动作差异程度低于差异阈值的所述当前视频帧或所述邻近视频帧，以及所述示范视频帧；
响应于所述示范视频播放结束，显示练习评价界面，其中所述练习评价界面显示所述标注后的人体动作差异程度较低的所述当前视频帧或所述邻近视频帧，以及所述示范视频帧。
本申请实施例示出一种界面显示方法，所述方法包括：
在显示第一界面时，接收输入的对所述第一界面中的播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
响应于所述确认操作，展示所述第二界面，并在所述第二界面中的第一播放窗口播放所述示范视频，在所述第二界面中的第二视频窗口播放进行关节点标注后的所述本地图像，其中在所述关节点标注后的所述本地图像中，第一关节点被标注为第一颜色，第二关节点被标注为第二颜色，第一关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度大于第二关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度。
附图说明
为了更清楚地说明本申请实施例或相关技术中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1中示例性示出了根据实施例中显示设备与控制装置之间操作场景的示意图；
图2中示例性示出了根据实施例中控制装置100的硬件配置框图；
图3中示例性示出了根据实施例中显示设备200的硬件配置框图；
图4中示例性示出了根据图3显示设备200的硬件架构框图；
图5中示例性示出了根据实施例中显示设备200的功能配置示意图；
图6a中示例性示出了根据实施例中显示设备200中软件配置示意图；
图6b中示例性示出了根据实施例中显示设备200中应用程序的配置示意图；
图7中示例性示出了根据实施例中显示设备200中用户界面的示意图；
图8-1为根据一些实施例示出的第一界面的示意图；
图8-2为根据一些实施例示出的第一界面的示意图；
图9-1为根据一些实施例示出的提示界面的示意图；
图9-2为根据一些实施例示出的提示界面的示意图；
图10为根据一些实施例示出的第二显示界面的示意图；
图11为根据一些实施例示出的标注了13个关节位的本地图像；
图12为根据一些实施例示出的打了关节批注的本地图像；
图13为根据一些实施例示出的颜色标注后的本地图像；
图14-1为根据一些实施例示出的练习评价界面；
图14-2为根据一些实施例示出的第二显示界面的示意图。
具体实施方式
为使本申请示例性实施例的目的、技术方案和优点更加清楚，下面将结合本申请示例性实施例中的附图，对本申请示例性实施例中的技术方案进行清楚、完整地描述，显然，所描述的示例性实施例仅是本申请一部分实施例，而不是全部的实施例。
为便于用户使用，显示设备上通常会设置各种外部装置接口，以便于连接不同的外设设备或线缆以实现相应的功能。而在显示设备的接口上连接有高清晰度的摄像头时，如果显示设备的硬件系统没有接收源码的高像素摄像头的硬件接口，那么就会导致无法将摄像头接收到的数据呈现到显示设备的显示屏上。
并且，受制于硬件结构，传统显示设备的硬件系统仅支持一路硬解码资源，且通常最大仅能支持4K分辨率的视频解码，因此当要实现边观看网络电视边进行视频聊天时，为了不降低网络视频画面清晰度，就需要使用硬解码资源(通常是硬件系统中的GPU)对网络视频进行解码，而在此情况下，只能采取由硬件系统中的通用处理器(例如CPU)对视频进行软解码的方式处理视频聊天画面。
采用软解码处理视频聊天画面，会大大增加CPU的数据处理负担，当CPU的数据处理负担过重时，可能会出现画面卡顿或者不流畅的问题。进一步的，受制于CPU的数据处理能力，当采用CPU软解码处理视频聊天画面时，通常无法实现多路视频通话，当用户想要再同一聊天场景同时与多个其他用户进行视频聊天时，会出现接入受阻的情况。
基于上述各方面的考虑，为克服上述缺陷，本申请公开了一种双硬件系统架构，以实现多路视频聊天数据(至少一路本地视频)。
下面首先结合附图对本申请所涉及的概念进行说明。在此需要指出的是，以下对各个概念的说明，仅为了使本申请的内容更加容易理解，并不表示对本申请保护范围的限定。
本申请一些实施例中使用的术语“模块”，可以是指任何已知或后来开发的硬件、软件、固件、人工智能、模糊逻辑或硬件或/和软件代码的组合，能够执行与该元件相关的功能。
本申请一些实施例中使用的术语“遥控器”，是指电子设备(如本申请中公开的显示设备)的一个组件，该组件通常可在较短的距离范围内无线控制电子设备。该组件一般可以使用红外线和/或射频(RF)信号和/或蓝牙与电子设备连接，也可以包括WiFi、无线USB、蓝牙、动作传感器等功能模块。例如：手持式触摸遥控器，是以触摸屏中用户界面取代一般遥控装置中的大部分物理内置硬键。
本申请一些实施例中使用的术语“手势”，是指用户通过一种手型的变化或手部运动等动作，用于表达预期想法、动作、目的/或结果的用户行为。
本申请一些实施例中使用的术语“硬件系统”，可以是指由集成电路(Integrated Circuit，IC)、印刷电路板(Printed circuit board，PCB)等机械、光、电、磁器件构成的具有计算、控制、存储、输入和输出功能的实体部件。在本申请各个实施例中，硬件系统通常也会被称为主板(motherboard)或芯片。
本申请一些实施例中使用的术语“操作系统”，可以是指处理器读取存储器中的代码后呈现给用户的计算机系统，例如“Android OS”，“Mac OS”“Windows OS”等。
图1中示例性示出了根据实施例中显示设备与控制装置之间操作场景的示意图。如图1所示，用户可通过控制装置100来操作显示设备200。
其中，控制装置100可以是遥控器100A，其可与显示设备200之间通过红外协议通信、 蓝牙协议通信、紫蜂(ZigBee)协议通信或其他短距离通信方式进行通信，用于通过无线或其他有线方式来控制显示设备200。用户可以通过遥控器上按键、语音输入、控制面板输入等输入用户指令，来控制显示设备200。如：用户可以通过遥控器上音量加减键、频道控制键、上/下/左/右的移动按键、语音输入按键、菜单键、开关机按键等输入相应控制指令，来实现控制显示设备200的功能。
控制装置100也可以是智能设备，如移动终端100B、平板电脑、计算机、笔记本电脑等，其可以通过本地网(LAN，Local Area Network)、广域网(WAN，Wide Area Network)、无线局域网((WLAN，Wireless Local Area Network)或其他网络与显示设备200之间通信，并通过与显示设备200相应的应用程序实现对显示设备200的控制。例如，使用在智能设备上运行的应用程序控制显示设备200。该应用程序可以在与智能设备关联的屏幕上通过直观的用户界面(UI，User Interface)为用户提供各种控制。
示例的，移动终端100B与显示设备200均可安装软件应用，从而可通过网络通信协议实现二者之间的连接通信，进而实现一对一控制操作的和数据通信的目的。如：可以使移动终端100B与显示设备200建立控制指令协议，将遥控控制键盘同步到移动终端100B上，通过控制移动终端100B上用户界面，实现控制显示设备200的功能；也可以将移动终端100B上显示的音视频内容传输到显示设备200上，实现同步显示功能。
如图1所示，显示设备200还可与服务器300通过多种通信方式进行数据通信。在本申请各个实施例中，可允许显示设备200通过局域网、无线局域网或其他网络与服务器300进行通信连接。服务器300可以向显示设备200提供各种内容和互动。
示例的，显示设备200通过发送和接收信息，以及电子节目指南(EPG，Electronic Program Guide)互动，接收软件程序更新，或访问远程储存的数字媒体库。服务器300可以是一组，也可以是多组，可以是一类或多类服务器。通过服务器300提供视频点播和广告服务等其他网络服务内容。
显示设备200，一方面讲，可以是液晶显示器、OLED(Organic Light Emitting Diode)显示器、投影显示设备；另一方面讲，显示设备被可以是智能电视或显示器和机顶盒组成的显示系统。具体显示设备类型，尺寸大小和分辨率等不作限定，本领技术人员可以理解的是，显示设备200可以根据需要做性能和配置上的一些改变。
显示设备200除了提供广播接收电视功能之外，还可以附加提供计算机支持功能的智能网络电视功能。示例的包括，网络电视、智能电视、互联网协议电视(IPTV)等。在一些实施例中，显示设备可以不具备广播接收电视功能。
如图1所述，显示设备上可以连接或设置有摄像头，用于将摄像头拍摄到的画面呈现在本显示设备或其他显示设备的显示界面上，以实现用户之间的交互聊天。具体的，摄像头拍摄到的画面可在显示设备上全屏显示、半屏显示、或者显示任意可选区域。
作为一种可选的连接方式，摄像头通过连接板与显示器后壳连接，固定安装在显示器后壳的上侧中部，作为可安装的方式，可以固定安装在显示器后壳的任意位置，能保证其图像采集区域不被后壳遮挡即可，例如，图像采集区域与显示设备的显示朝向相同。
作为另一种可选的连接方式，摄像头通过连接板或者其他可想到的连接器可升降的与显示后壳连接，连接器上安装有升降马达，当用户要使用摄像头或者有应用程序要使用摄像头时，再升出显示器之上，当不需要使用摄像头时，其可内嵌到后壳之后，以达到保护摄像头免受损坏。
作为一种实施例，本申请所采用的摄像头可以为1600万像素，以达到超高清显示目的。在实际使用中，也可采用比1600万像素更高或更低的摄像头。
当显示设备上安装有摄像头以后，显示设备不同应用场景所显示的内容可得到多种不同方式的融合，从而达到传统显示设备无法实现的功能。
在一些实施例中，用户可以在边观看视频节目的同时，与至少一位其他用户进行视频聊天。视频节目的呈现可作为背景画面，视频聊天的窗口显示在背景画面之上。形象的，可以称该功能为“边看边聊”。
在一些实施例中，在“边看边聊”的场景中，在观看直播视频或网络视频的同时，跨终端的进行至少一路的视频聊天。
在一些实施例中，用户可以在边进入教育应用学习的同时，与至少一位其他用户进行视频聊天。例如，学生在学习教育应用程序中内容的同时，可实现与老师的远程互动。形象的，可以称该功能为“边学边聊”。
在一些实施例中，用户在玩纸牌游戏时，与进入游戏的玩家进行视频聊天。例如，玩家在进入游戏应用参与游戏时，可实现与其他玩家的远程互动。形象的，可以称该功能为“边看边玩”。
在一些实施例中，游戏场景与视频画面进行融合，将视频画面中人像进行抠图，显示在游戏画面中，提升用户体验。
在一些实施例中，在体感类游戏中(如打球类、拳击类、跑步类、跳舞类等)，通过摄像头获取人体姿势和动作，肢体检测和追踪、人体骨骼关键点数据的检测，再与游戏中动画进行融合，实现如体育、舞蹈等场景的游戏。
在一些实施例中，用户可以在K歌应用中，与至少一位其他用户进行视频和语音的交互。形象的，可以称该功能为“边看边唱”。优选的，当至少一位用户在聊天场景进入该应用时，可多个用户共同完成一首歌的录制。
在一些实施例中，用户可在本地打开摄像头获取图片和视频，形象的，可以称该功能为“照镜子”。
在一些实施例中，还可以再增加更多功能或减少上述功能。本申请对该显示设备的功能不作具体限定。
图2中示例性示出了根据示例性实施例中控制装置100的配置框图。如图3所示，控制装置100包括控制器110、通信器130、用户输入/输出接口140、存储器190、供电电源180。
控制装置100被配置为可控制所述显示设备200，以及可接收用户的输入操作指令，且将操作指令转换为显示设备200可识别和响应的指令，起到用户与显示设备200之间交互中介作用。如：用户通过操作控制装置100上频道加减键，显示设备200响应频道加减的操作。
在一些实施例中，控制装置100可是一种智能设备。如：控制装置100可根据用户需求安装控制显示设备200的各种应用。
在一些实施例中，如图1所示，移动终端100B或其他智能电子设备，可在安装操控显示设备200的应用之后，起到控制装置100类似功能。如：用户可以通过安装应用，在移动终端100B或其他智能电子设备上可提供的图形用户界面的各种功能键或虚拟按钮，以实现控制装置100实体按键的功能。
控制器110包括处理器112、RAM113和ROM114、通信接口以及通信总线。控制器110用于控制控制装置100的运行和操作，以及内部各部件之间通信协作以及外部和内部的数据处理功能。
通信器130在控制器110的控制下，实现与显示设备200之间控制信号和数据信号的通信。如：将接收到的用户输入信号发送至显示设备200上。通信器130可包括WIFI模块131、蓝牙模块132、NFC模块133等通信模块中至少一种。
用户输入/输出接口140，其中，输入接口包括麦克风141、触摸板142、传感器143、按键144等输入接口中至少一者。如：用户可以通过语音、触摸、手势、按压等动作实现用户指令输入功能，输入接口通过将接收的模拟信号转换为数字信号，以及数字信号转换为相应指令信号，发送至显示设备200。
输出接口包括将接收的用户指令发送至显示设备200的接口。在一些实施例中，可以是红外接口，也可以是射频接口。如：红外信号接口时，需要将用户输入指令按照红外控制协议转化为红外控制信号，经红外发送模块进行发送至显示设备200。再如：射频信号接口时，需将用户输入指令转化为数字信号，然后按照射频控制信号调制协议进行调制后，由射频发送端子发送至显示设备200。
在一些实施例中，控制装置100包括通信器130和输出接口中至少一者。控制装置100中配置通信器130，如：WIFI、蓝牙、NFC等模块，可将用户输入指令通过WIFI协议、或蓝牙协议、或NFC协议编码，发送至显示设备200.
存储器190，用于在控制器110的控制下存储驱动和控制控制装置100的各种运行程序、数据和应用。存储器190，可以存储用户输入的各类控制信号指令。
供电电源180，用于在控制器110的控制下为控制装置100各元件提供运行电力支持。可以电池及相关控制电路。
图3中示例性示出了根据示例性实施例中显示设备200中硬件系统的硬件配置框图。
在采用双硬件系统架构时，硬件系统的机构关系可以图3所示。为便于表述以下将双硬件系统架构中的一个硬件系统称为第一硬件系统或A系统、A芯片，并将另一个硬件系统称为第二硬件系统或N系统、N芯片。A芯片包含A芯片的控制器及各类接口，N芯片则包含N芯片的控制器及各类接口。A芯片及N芯片中可以各自安装有相对独立的操作系统，A芯片的操作系统和N芯片的操作系统可以通过通信协议相互通信，示例性的：A芯片的操作系统的framework层和N芯片的操作系统的framework层可以进行通信进行命令和数据的传输，从而使显示设备200中存在两个在独立但又存在相互关联的子系统。
如图3所示，A芯片与N芯片之间可以通过多个不同类型的接口实现连接、通信及供电。A芯片与N芯片之间接口的接口类型可以包括通用输入输出接口(General-purpose input/output，GPIO)、USB接口、HDMI接口、UART接口等中的至少一个。A芯片与N芯片之间可以使用这些接口中的一个或多个进行通信或电力传输。例如图3所示，在双硬件系统架构下，可以由外接的电源(power)为N芯片供电，而A芯片则可以不由外接电源，而由N芯片供电。
除用于与N芯片进行连接的接口之外，A芯片还可以包含用于连接其他设备或组件的接口，例如图3中所示的用于连接摄像头(Camera)的MIPI接口，蓝牙接口等。
类似的，除用于与N芯片进行连接的接口之外，N芯片还可以包含用于连接显示屏TCON(Timer Control Register)的VBY接口，用于连接功率放大器(Amplifier，AMP) 及扬声器(Speaker)的i2S接口；以及IR/Key接口，USB接口，Wifi接口，蓝牙接口，HDMI接口，Tuner接口等中的至少一个。
下面结合图4对本申请双硬件系统架构进行进一步的说明。需要说明的是图4仅仅是对本申请双硬件系统架构的一个示例性说明，并不表示对本申请的限定。在实际应用中，两个硬件系统均可根据需要包含更多或更少的硬件或接口。
图4中示例性示出了根据图3显示设备200的硬件架构框图。如图4所示，显示设备200的硬件系统可以包括A芯片和N芯片，以及通过各类接口与A芯片或N芯片相连接的模块。
N芯片可以包括调谐解调器220、通信器230、外部装置接口250、控制器210、存储器290、用户输入接口、视频处理器260-1、音频处理器260-2、显示器280、音频输出接口270、供电电源中的至少一个。在其他实施例中N芯片也可以包括更多或更少的模块。
其中，调谐解调器220，用于对通过有线或无线方式接收广播电视信号，进行放大、混频和谐振等调制解调处理，从而从多个无线或有线广播电视信号中解调出用户所选择电视频道的频率中所携带的音视频信号，以及附加信息(例如EPG数据信号)。根据电视信号广播制式不同，调谐解调器220的信号途径可以有很多种，诸如：地面广播、有线广播、卫星广播或互联网广播等；以及根据调制类型不同，所述信号的调整方式可以数字调制方式，也可以模拟调制方式；以及根据接收电视信号种类不同，调谐解调器220可以解调模拟信号和/或数字信号。
调谐解调器220，还用于根据用户选择，以及由控制器210控制，响应用户选择的电视频道频率以及该频率所携带的电视信号。
在其他一些示例性实施例中，调谐解调器220也可在外置设备中，如外置机顶盒等。这样，机顶盒通过调制解调后输出电视音视频信号，经过外置装置接口250输入至显示设备200中。
通信器230是用于根据各种通信协议类型与外部设备或外部服务器进行通信的组件。例如：通信器230可以包括WIFI模块231，蓝牙模块232，有线以太网模块233，及红外通信协议模块等其他网络通信协议模块或近场通信协议模块。
显示设备200可以通过通信器230与外部控制设备或内容提供设备之间建立控制信号和数据信号的连接。例如，通信器可根据控制器的控制接收遥控器100的控制信号。
外部装置接口250，是提供N芯片控制器210和A芯片及外部其他设备间数据传输的组件。外部装置接口可按照有线/无线方式与诸如机顶盒、游戏装置、笔记本电脑等的外部设备连接，可接收外部设备的诸如视频信号(例如运动图像)、音频信号(例如音乐)、附加信息(例如EPG)等数据。
其中，外部装置接口250可以包括：高清多媒体接口(HDMI)端子251、复合视频消隐同步(CVBS)端子252、模拟或数字分量端子253、通用串行总线(USB)端子254、红绿蓝(RGB)端子(图中未示出)等任一个或多个。本申请不对外部装置接口的数量和类型进行限制。
控制器210，通过运行存储在存储器290上的各种软件控制程序(如操作系统和/或各种应用程序)，来控制显示设备200的工作和响应用户的操作。
如图4所示，控制器210包括只读存储器RAM214、随机存取存储器ROM213、图形处理器216、CPU处理器212、通信接口218、以及通信总线。其中，RAM213和ROM214 以及图形处理器216、CPU处理器212、通信接口218通过总线相连接。
ROM213，用于存储各种系统启动的指令。如在收到开机信号时，显示设备200电源开始启动，CPU处理器212运行ROM中系统启动指令，将存储在存储器290的操作系统产生的临时数据拷贝至RAM214中，以运行或启动操作系统。当操作系统启动完成后，CPU处理器212将存储器290中各种应用程序产生的临时数据拷贝至RAM214中,然后，以运行或启动各种应用程序。
图形处理器216，用于产生各种图形对象，如：图标、操作菜单、以及用户输入指令显示图形等。包括运算器，通过接收用户输入各种交互指令进行运算，根据显示属性显示各种对象。以及包括渲染器，产生基于运算器得到的各种对象，进行渲染的结果显示在显示器280上。
CPU处理器212，用于执行存储在存储器290中操作系统和应用程序指令。以及根据接收外部输入的各种交互指令，来执行各种应用程序、数据和内容，以便最终显示和播放各种音视频内容。
在一些示例性实施例中，CPU处理器212，可以包括多个处理器。所述多个处理器中可包括一个主处理器以及多个或一个子处理器。主处理器，用于在预加电模式中执行显示设备200一些操作，和/或在正常模式下显示画面的操作。多个或一个子处理器，用于执行在待机模式等状态下的一种操作。
通信接口，可包括第一接口218-1到第n接口218-n。这些接口可以是经由网络被连接到外部设备的网络接口。
控制器210可以控制显示设备200的整体操作。例如：响应于接收到用于选择在显示器280上显示UI对象的用户命令，控制器210便可以执行与由用户命令选择的对象有关的操作。
其中，所述对象可以是可选对象中的任何一个，例如超链接或图标。与所选择的对象有关操作，例如：显示连接到超链接页面、文档、图像等操作，或者执行与图标相对应程序的操作。用于选择UI对象用户命令，可以是通过连接到显示设备200的各种输入装置(例如，鼠标、键盘、触摸板等)输入命令或者与由用户说出语音相对应的语音命令。
存储器290，包括存储用于驱动和控制显示设备200的各种软件模块。如：存储器290中存储的各种软件模块，包括：基础模块、检测模块、通信模块、显示控制模块、浏览器模块、和各种服务模块等。
其中，基础模块是用于显示设备200中各个硬件之间信号通信、并向上层模块发送处理和控制信号的底层软件模块。检测模块是用于从各种传感器或用户输入接口中收集各种信息，并进行数模转换以及分析管理的管理模块。
例如：语音识别模块中包括语音解析模块和语音指令数据库模块。显示控制模块是用于控制显示器280进行显示图像内容的模块，可以用于播放多媒体图像内容和UI界面等信息。通信模块，是用于与外部设备之间进行控制和数据通信的模块。浏览器模块，是用于执行浏览服务器之间数据通信的模块。服务模块，是用于提供各种服务以及各类应用程序在内的模块。
同时，存储器290还用于存储接收外部数据和用户数据、各种用户界面中各个项目的图像以及焦点对象的视觉效果图等。
用户输入接口，用于将用户的输入信号发送给控制器210，或者，将从控制器输出的 信号传送给用户。示例性的，控制装置(例如移动终端或遥控器)可将用户输入的诸如电源开关信号、频道选择信号、音量调节信号等输入信号发送至用户输入接口，再由用户输入接口转送至控制器；或者，控制装置可接收经控制器处理从用户输入接口输出的音频、视频或数据等输出信号，并且显示接收的输出信号或将接收的输出信号输出为音频或振动形式。
在一些实施例中，用户可在显示器280上显示的图形用户界面(GUI)输入用户命令，则用户输入接口通过图形用户界面(GUI)接收用户输入命令。或者，用户可通过输入特定的声音或手势进行输入用户命令，则用户输入接口通过传感器识别出声音或手势，来接收用户输入命令。
视频处理器260-1，用于接收视频信号，根据输入信号的标准编解码协议，进行解压缩、解码、缩放、降噪、帧率转换、分辨率转换、图像合成等视频数据处理，可得到直接在显示器280上显示或播放的视频信号。
示例的，视频处理器260-1，包括解复用模块、视频解码模块、图像合成模块、帧率转换模块、显示格式化模块等。
其中，解复用模块，用于对输入音视频数据流进行解复用处理，如输入MPEG-2,则解复用模块进行解复用成视频信号和音频信号等。
视频解码模块，用于对解复用后的视频信号进行处理，包括解码和缩放处理等。
图像合成模块，如图像合成器，其用于将图形生成器根据用户输入或自身生成的GUI信号，与缩放处理后视频图像进行叠加混合处理，以生成可供显示的图像信号。
帧率转换模块，用于对输入视频的帧率进行转换，如将输入的24Hz、25Hz、30Hz、60Hz视频的帧率转换为60Hz、120Hz或240Hz的帧率，其中，输入帧率可以与源视频流有关，输出帧率可以与显示器的更新率有关。输入有通常的格式采用如插帧方式实现。
显示格式化模块，用于将帧率转换模块输出的信号，改变为符合诸如显示器显示格式的信号，如将帧率转换模块输出的信号进行格式转换以输出RGB数据信号。
在一些实施例中视频处理器和图形处理器可以统一的集成在芯片内，在一些实施例中，图形处理器和视频处理器的功能模块可以按照需求设置。
显示器280，用于接收源自视频处理器260-1输入的图像信号，进行显示视频内容和图像以及菜单操控界面。显示器280包括用于呈现画面的显示器组件以及驱动图像显示的驱动组件。显示视频内容，可以来自调谐解调器220接收的广播信号中的视频，也可以来自通信器或外部设备接口输入的视频内容。显示器220，同时显示显示设备200中产生且用于控制显示设备200的用户操控界面UI。
以及，根据显示器280类型不同，还包括用于驱动显示的驱动组件。或者，倘若显示器280为一种投影显示器，还可以包括一种投影装置和投影屏幕。
音频处理器260-2，用于接收音频信号，根据输入信号的标准编解码协议，进行解压缩和解码，以及降噪、数模转换、和放大处理等音频数据处理，得到可以在扬声器272中播放的音频信号。
音频输出接口270，用于在控制器210的控制下接收音频处理器260-2输出的音频信号，音频输出接口可包括扬声器272，或输出至外接设备的发生装置的外接音响输出端子274，如：外接音响端子或耳机输出端子等。
在其他一些示例性实施例中，视频处理器260-1可以包括一个或多个芯片组成。音频 处理器260-2，也可以包括一个或多个芯片组成。
以及，在其他一些示例性实施例中，视频处理器260-1和音频处理器260-2，可以为单独的芯片，也可以与控制器210一起集成在一个或多个芯片中。
供电电源，用于在控制器210控制下，将外部电源输入的电力为显示设备200提供电源供电支持。供电电源可以包括安装显示设备200内部的内置电源电路，也可以是安装在显示设备200外部的电源，如在显示设备200中提供外接电源的电源接口。
与N芯片相类似，如图4所示，A芯片可以包括控制器310、通信器330、检测器340、存储器390。在某些实施例中还可以包括用户输入接口、视频处理器、音频处理器、显示器、音频输出接口。在某些实施例中，也可以存在独立为A芯片供电的供电电源。
通信器330是用于根据各种通信协议类型与外部设备或外部服务器进行通信的组件。例如：通信器330可以包括WIFI模块331，蓝牙通信协议模块332，有线以太网通信协议模块333，及红外通信协议模块等其他网络通信协议模块或近场通信协议模块。
A芯片的通信器330和N芯片的通信器230也有相互交互。例如，N芯片硬件系统内的WiFi模块231用于连接外部网络，与外部服务器等产生网络通信。A芯片内的WiFi模块331用于连接至N芯片硬件系统内的WiFi模块231，而不与外界网络等产生直接连接，A芯片通过N芯片连接外部网络。因此，对于用户而言，一个如上述实施例中的显示设备至对外显示一个WiFi账号。
检测器340，是显示设备A芯片用于采集外部环境或与外部交互的信号的组件。检测器340可以包括光接收器342，用于采集环境光线强度的传感器，可以通过采集环境光来自适应显示参数变化等；还可以包括图像采集器341，如相机、摄像头等，可以用于采集外部环境场景，以及用于采集用户的属性或与用户交互手势，可以自适应变化显示参数，也可以识别用户手势，以实现与用户之间互动的功能。
外部装置接口，提供控制器310与N芯片或外部其他设备间数据传输的组件。外部装置接口可按照有线/无线方式与诸如机顶盒、游戏装置、笔记本电脑等的外部设备连接。
控制器310，通过运行存储在存储器390上的各种软件控制程序(如用安装的第三方应用等)，以及与N芯片的交互，来控制显示设备200的工作和响应用户的操作。
如图4所示，控制器310包括只读存储器ROM313、随机存取存储器RAM314、图形处理器316、CPU处理器312、通信接口318、以及通信总线。其中，ROM313和RAM314以及图形处理器316、CPU处理器312、通信接口318通过总线相连接。
ROM313，用于存储各种系统启动的指令。CPU处理器312运行ROM中系统启动指令，将存储在存储器390的操作系统的临时数据拷贝至RAM314中，以运行或启动操作系统。当操作系统启动完成后，CPU处理器312将存储器390中各种应用程序的临时数据拷贝至RAM314中,然后，以运行或启动各种应用程序。
CPU处理器312，用于执行存储在存储器390中操作系统和应用程序指令,和与N芯片进行通信、信号、数据、指令等传输与交互，以及根据接收外部输入的各种交互指令，来执行各种应用程序、数据和内容，以便最终显示和播放各种音视频内容。
通信接口，可包括第一接口318-1到第n接口318-n。这些接口可以是经由网络被连接到外部设备的网络接口，也可以是经由网络被连接到N芯片的网络接口。
控制器310可以控制显示设备200的整体操作。例如：响应于接收到用于选择在显示器280上显示UI对象的用户命令，控制器210便可以执行与由用户命令选择的对象有 关的操作。
图形处理器316，用于产生各种图形对象，如：图标、操作菜单、以及用户输入指令显示图形等。包括运算器，通过接收用户输入各种交互指令进行运算，根据显示属性显示各种对象。以及包括渲染器，产生基于运算器得到的各种对象，进行渲染的结果显示在显示器280上。
A芯片的图形处理器316与N芯片的图形处理器216均能产生各种图形对象。区别性的，若应用1安装于A芯片，应用2安装在N芯片，当用户在应用1的界面，且在应用1内进行用户输入的指令时，由A芯片图形处理器316产生图形对象。当用户在应用2的界面，且在应用2内进行用户输入的指令时，由N芯片的图形处理器216产生图形对象。
图5中示例性示出了根据示例性实施例中显示设备的功能配置示意图。
如图5所示，A芯片的存储器390和N芯片的存储器290分别用于存储操作系统、应用程序、内容和用户数据等，在A芯片的控制器310和N芯片的控制器210的控制下执行驱动显示设备200的系统运行以及响应用户的各种操作。A芯片的存储器390和N芯片的存储器290可以包括易失性和/或非易失性存储器。
对于N芯片，存储器290，具体用于存储驱动显示设备200中控制器210的运行程序，以及存储显示设备200内置各种应用程序，以及用户从外部设备下载的各种应用程序、以及与应用程序相关的各种图形用户界面，以及与图形用户界面相关的各种对象，用户数据信息，以及各种支持应用程序的内部数据。存储器290用于存储操作系统(OS)内核、中间件和应用等系统软件，以及存储输入的视频数据和音频数据、及其他用户数据。
存储器290，具体用于存储视频处理器260-1和音频处理器260-2、显示器280、通信接口230、调谐解调器220、输入/输出接口等驱动程序和相关数据。
在一些实施例中，存储器290可以存储软件和/或程序，用于表示操作系统(OS)的软件程序包括，例如：内核、中间件、应用编程接口(API)和/或应用程序。示例性的，内核可控制或管理系统资源，或其它程序所实施的功能(如所述中间件、API或应用程序)，以及内核可以提供接口，以允许中间件和API，或应用访问控制器，以实现控制或管理系统资源。
示例的，存储器290，包括广播接收模块2901、频道控制模块2902、音量控制模块2903、图像控制模块2904、显示控制模块2905、音频控制模块2906、外部指令识别模块2907、通信控制模块2908、光接收模块、电力控制模块2910、操作系统2911、以及其他应用程序2912、浏览器模块等等。控制器210通过运行存储器290中各种软件程序，来执行诸如：广播电视信号接收解调功能、电视频道选择控制功能、音量选择控制功能、图像控制功能、显示控制功能、音频控制功能、外部指令识别功能、通信控制功能、光信号接收功能、电力控制功能、支持各种功能的软件操控平台、以及浏览器功能等各类功能。
存储器390，包括存储用于驱动和控制显示设备200的各种软件模块。如：存储器390中存储的各种软件模块，包括：基础模块、检测模块、通信模块、显示控制模块、浏览器模块、和各种服务模块等。由于存储器390与存储器290的功能比较相似，相关之处参见存储器290即可，在此就不再赘述。
示例的，存储器390，包括图像控制模块3904、音频控制模块2906、外部指令识别模块3907、通信控制模块3908、光接收模块3909、操作系统3911、以及其他应用程序3912、浏览器模块等等。控制器210通过运行存储器290中各种软件程序，来执行诸如：图像控 制功能、显示控制功能、音频控制功能、外部指令识别功能、通信控制功能、光信号接收功能、电力控制功能、支持各种功能的软件操控平台、以及浏览器功能等各类功能。
区别性的，N芯片的外部指令识别模块2907和A芯片的外部指令识别模块3907可识别不同的指令。
示例性的，由于摄像头等图像接收设备与A芯片连接，因此，A芯片的外部指令识别模块3907可包括图形识别模块2907-1，图形识别模块3907-1内存储有图形数据库，摄像头接收到外界的图形指令时，与图形数据库中的指令进行对应关系，以对显示设备作出指令控制。而由于语音接收设备以及遥控器与N芯片连接，因此，N芯片的外部指令识别模块2907可包括语音识别模块2907-2，图形识别模块2907-2内存储有语音数据库，语音接收设备等接收到外界的语音指令或时，与语音数据库中的指令进行对应关系，以对显示设备作出指令控制。同样的，遥控器等控制装置100与N芯片连接，由按键指令识别模块与控制装置100进行指令交互。
图6a中示例性示出了根据示例性实施例中显示设备200中软件系统的配置框图。
对N芯片，如图6a中所示，操作系统2911，包括用于处理各种基础系统服务和用于实施硬件相关任务的执行操作软件。
一些实施例中，部分操作系统内核可以包含一系列软件，用以管理显示设备硬件资源，并为其他程序或软件代码提供服务。
其他一些实施例中，部分操作系统内核可包含一个或多个设备驱动器，设备驱动器可以是操作系统中的一组软件代码，帮助操作或控制显示设备关联的设备或硬件。驱动器可以包含操作视频、音频和/或其他多媒体组件的代码。示例的，包括显示器、摄像头、Flash、WiFi和音频驱动器。
其中，可访问性模块2911-1，用于修改或访问应用程序，以实现应用程序的可访问性和对其显示内容的可操作性。
通信模块2911-2，用于经由相关通信接口和通信网络与其他外设的连接。
用户界面模块2911-3，用于提供显示用户界面的对象，以供各应用程序访问，可实现用户可操作性。
控制应用程序2911-4，用于控制进程管理，包括运行时间应用程序等。
事件传输系统2914，可在操作系统2911内或应用程序2912中实现。一些实施例中，一方面在在操作系统2911内实现，同时在应用程序2912中实现，用于监听各种用户输入事件，将根据各种事件指代响应各类事件或子事件的识别结果，而实施一组或多组预定义的操作的处理程序。
其中，事件监听模块2914-1，用于监听用户输入接口输入事件或子事件。
事件识别模块2914-2，用于对各种用户输入接口输入各类事件的定义，识别出各种事件或子事件，且将其传输给处理用以执行其相应一组或多组的处理程序。
其中，事件或子事件，是指显示设备200中一个或多个传感器检测的输入，以及外界控制设备(如控制装置100等)的输入。如：语音输入各种子事件，手势识别的手势输入子事件，以及控制装置的遥控按键指令输入的子事件等。示例的，遥控器中一个或多个子事件包括多种形式，包括但不限于按键按上/下/左右/、确定键、按键按住等中一个或组合。以及非实体按键的操作，如移动、按住、释放等操作。
界面布局管理模块2913，直接或间接接收来自于事件传输系统2914监听到各用户输 入事件或子事件，用于更新用户界面的布局，包括但不限于界面中各控件或子控件的位置，以及容器的大小或位置、层级等与界面布局相关各种执行操作。
由于A芯片的操作系统3911与N芯片的操作系统2911的功能比较相似，相关之处参见操作系统2911即可，在此就不再赘述。
如图6b中所示，显示设备的应用程序层包含可在显示设备200执行的各种应用程序。
N芯片的应用程序层2912可包含但不限于一个或多个应用程序，如：视频点播应用程序、应用程序中心、游戏应用等。A芯片的应用程序层3912可包含但不限于一个或多个应用程序，如：直播电视应用程序、媒体中心应用程序等。需要说明的是，A芯片和N芯片上分别包含什么应用程序是根据操作系统和其他设计确定的，本申请无需对A芯片和N芯片上所包含的应用程序做具体的限定和划分。
直播电视应用程序，可以通过不同的信号源提供直播电视。例如，直播电视应用程可以使用来自有线电视、无线广播、卫星服务或其他类型的直播电视服务的输入提供电视信号。以及，直播电视应用程序可在显示设备200上显示直播电视信号的视频。
视频点播应用程序，可以提供来自不同存储源的视频。不同于直播电视应用程序，视频点播提供来自某些存储源的视频显示。例如，视频点播可以来自云存储的服务器端、来自包含已存视频节目的本地硬盘储存器。
媒体中心应用程序，可以提供各种多媒体内容播放的应用程序。例如，媒体中心，可以为不同于直播电视或视频点播，用户可通过媒体中心应用程序访问各种图像或音频所提供服务。
应用程序中心，可以提供储存各种应用程序。应用程序可以是一种游戏、应用程序，或某些和计算机系统或其他设备相关但可以在显示设备中运行的其他应用程序。应用程序中心可从不同来源获得这些应用程序，将它们储存在本地储存器中，然后在显示设备200上可运行。
在一些实施例中，操作系统包括内核，数据库，框架层和应用层，上述应用均为应用层中的应用。
图7中示例性示出了根据示例性实施例中显示设备200中用户界面的示意图。如图7所示，用户界面包括多个视图显示区，示例的，第一视图显示区201和播放画面202，其中，播放画面包括布局一个或多个不同项目。以及用户界面中还包括指示项目被选择的选择器，可通过用户输入而移动选择器的位置，以改变选择不同的项目。
需要说明的是，多个视图显示区可以呈现不同层级的显示画面。如，第一视图显示区可呈现视频聊天项目内容，第二视图显示区可呈现应用层项目内容(如，网页视频、VOD展示、应用程序画面等)。
可选的，不同视图显示区的呈现存在优先级区别，优先级不同的视图显示区之间，视图显示区的显示优先级不同。如，系统层的优先级高于应用层的优先级，当用户在应用层使用获取选择器和画面切换时，不遮挡系统层的视图显示区的画面展示；以及，根据用户的选择使应用层的视图显示区的大小和位置发生变化时，系统层的视图显示区的大小和位置不受影响。
也可以呈现相同层级的显示画面，此时，选择器可以在第一视图显示区和第二视图显示区之间做切换，以及当第一视图显示区的大小和位置发生变化时，第二视图显示区的大小和位置可随及发生改变。
由于A芯片及N芯片中可能分别安装有独立的操作系统，从而使显示设备200中存在两个在独立但又存在相互关联的子系统。例如，A芯片和N均可以独立安装有安卓(Android)及各类APP，使得每个芯片均可以实现一定的功能，并且使A芯片和N芯片协同实现某项功能。
在一些实施例中，显示设备的控制器可以仅位于一个实体芯片内，以可以如上述实施例一样包含两个芯片，A芯片和N芯片中的控制器共同作为控制模块。
本申请一些实施例第一方面示出一种显示设备，包括：
显示屏，用于显示第一显示界面和所述第二显示界面，所述第一显示界面包括用于控制播放示范视频的播放控件，所述第二显示界面中包含用于播放所述示范视频的第一播放窗口和用于播放摄像头采集到的本地图像的第二播放窗口；
健身环境下的显示设备的第一界面如图8-1所示。其中，图8-1为第一界面200A的示意图，其中，第一界面200A可以滚动的方式显示多个示范视频，以便用户在多个示范视频找中选中一个目标示范视频。
请参阅图8-1，其中显示窗口用于显示用户选中的示范视频。接收用户输入的确认指令，根据选择器(焦点)在第一界面中的“开始训练”控件位置时，当用户在第一界面200A中选中开始训练的控件。响应于用户对开始训练控件的选中，控制器可基于API(Application Programming Interface，应用程序接口)从服务器获取并加载相应是示范视频片源。
请参阅图8-2，图8-2为根据一些实施例示出的健身视频第一界面的示意图，健身视频第一界面页可以被称作详情界面，其中，第一界面可以滚动的方式显示多个教练视频，以便用户在多个示范视频找中选中一个目标示范视频。例如：深蹲高抬腿、后退式弓箭步、四点式后踢腿……。用户在多个教练视频找中选中一个目标示范视频。
在第一界面中，设置有播放窗口，播放窗口用于播放默认训练视频或播放历史中上一次播放的训练视频，播放窗口的右侧设置有介绍展示控件，“开始训练”控件(即播放控件)，“收藏”控件中的至少一个，界面还包括训练列表控件，在训练列表控件中，显示有多个训练视频的展示控件。
在一些实施例中，在开始训练控件或播放窗口被选中后，示范视频也可以再校验后获取。具体的，显示预先下载示范视频并存储；然后建立示范视频与校验码之间的映射关系。响应于用户对开始训练控件的选中时，基于用户选择的示范视频，生成一个校验码。控制器可以基于检验码在存储的示范视频中，获取所述校验码对应的示范视频。由于示范视频预先存储，控制器在获取校验码后，可直接调取校验码对应的示范视频。通过上述方式调取示范视频，可避免由于网络等因素造成的卡顿问题，前获取能预先下载，提升示范视频流畅度。
摄像头，用于采集本地图像或本地视频；在未开启的时候，摄像头位于隐藏位置，以使的显示装置边缘保持平滑，在开启后，摄像头升起，探出在显示装置边缘的上方，以避免显示屏的遮挡来获取图像数据。
在一些实施例中，响应于用户对开始训练控件的选中，升起启动摄像头以获取图像数据，练习过程中摄像头始终处于开启的状态，并实时的采集本地视频，发送采集的本地视频至控制器，以在跟练界面显示用户的动作。这样用户可以实时观看自己的动作和示范视频的动作。
在一些实施例中，响应于用户对开始训练控件的选中，摄像头升起，但处于待机状态， 每当示范视频播放至预设时间点时，采集一个本地图像，发送采集的本地图像至控制器。这样可以减轻处理器的压力，并且维持本地图像在显示屏上显示到下一个时间点。
控制器，所述控制器被配置为：接收输入的对所述播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
在一种实现方式中，响应于确认操作，所述确认操作可以是对开始训练控件的选中。所述控制器还被进一步配置为控制显示器显示用于指示用户进入预定区域的提示界面200C(及引导界面)。具体的提示界面可以参阅图9-1和图9-2。具体的，图9-1为根据一些实施例示出的提示界面的示意图，用户根据提示界面调整自身的位置。当用户完成进入预定区域，控制器控制显示器展示第二界面。这是因为摄像头的采集区域是存在边缘的，为了能更好的采集本地数据，摄像头获取当前的图像，并在显示上显示的过程中在显示当前图像的图层上方新建一个浮层，浮层中根据摄像头的位置和角度确定最佳采集区域，并根据最佳采集区域在浮层中显示最佳位置边框。以引导用户进行位置的移动，使采集到的当前图像中的位置和浮层中最佳位置边框重合，在重叠程度达到预设阈值时，显示设备显示成功的提示消息，并取消浮层，并跳转到图10所示的跟练界面。
举例说明，在一些实施例中，提示界面200C中人物位于方框区域200C1的左边，相应的提示用户向右边移动。如果显示画面中人物位于方框区域的右边，相应的提示用户向左边移动，以使用户进入预定区域，其中预定区域摄像头可采集到的区域。本申请实施例示出的通过上述方式指示用户进入预定区域。在一些实施例中，提示界面还用于显示用提示信息，具体的可以参阅图9-2，图9-2为根据一些实施例示出的提示界面的示意图，所述提示信息为“请面对屏幕。保持身体自立”等提示信息。提示用户移动的消息可以是显示在浮层上的文字，也可以是语音提醒，还可以是指向最佳位置边框的指示标记。
在一种实现方式中，控制器也可以是响应于所述确认操作，直接展示所述第二界面，并在所述第一播放窗口播放所述示范视频，在所述第二视频窗口播放所述本地图像。用户可以根据第二界面中第二视频窗口显示的图像来调整自身的位置。
在一种实现方式中，控制器也可以是响应于所述确认操作，判断位置引导界面的出现次数，在引导界面展示次数不满足预设值时展示引导界面，在满足预设值时直接展示所述第二界面，并在所述第一播放窗口播放所述示范视频，在所述第二视频窗口播放所述本地图像。用户可以根据第二界面中第二视频窗口显示的图像来调整自身的位置。
具体的可以参阅图10，图10为根据一些实施例示出的第二显示界面200B的示意图，其中所述第二显示界面200B中包含用于播放所述示范视频的第一播放窗口200B1和用于播放摄像头采集到的本地图像的第二播放窗口200B2。
在一些实施例中，第一播放窗口中播放所述示范视频，所述示范视频中不展示关节点，第二播放串口播放本地图像数据包括控制器根据本地图像数据获取随影本地图像数据的关节点的位置，根据所述关节点的位置叠加所述本地图像数据和关节点标记后在第二播放窗口显示。
在一些实施例中叠加所述本地图像数据和关节点标记可以在在本地图像数据中根据关节点的位置在本地图像上增加关节点标记，然后在一个图层输出以显示叠加关节点后的本地图像。还可以在一个图层中显示摄像头获取的本地图像，在图层的上方增加浮层，并根据关节点的位置在浮层中增加关节面标记，将两个图层叠加后显示。
在一些实施例中第二播放窗口直接播放摄像头采集到的本地视频。
本申请实施例示出一种显示设备，所述显示设备包括显示器、摄像头及控制器。控制器被配置为响应于用户对显示界面中开始训练控件的选中，获取示范视频，升起并开启摄像头，所述摄像头用于采集本地图形；控制显示器的第一播放窗口播放所述示范视频，显示器的第二播放窗口展示所述本地图像。可见，本申请实施例示出的技术方案，通过第一播放窗口展示示范视频，第二播放窗口展示本地图片，用户在练习的过程中可以通过两个窗口内展示内容的比对及时的调整自身的动作，提高用户的体验感。
摄像头用于采集本地视频，本地视频为连续的本地图像的集合。在比对的过程中，如果针对每一帧图像都进行比对的话，控制器的数据处理量较大。
基于上述问题在一些可行性实施例中，控制器可将本地图像与示范视频帧进行比对，生成比对结果，用户健身结束后可以通过显示界面观看比对结果，进而帮助用户更好的了解自身的动作缺陷，以便用户在后续健身的过程中克服动作缺陷。其中，所述示范视频帧为示范视频中与所述本地图像相对应的图形。
在一些实施例中，采集本地图像的实现方式有多种。
例如，控制器可以控制摄像头在示范视频播放至预设时间点时采集本地图像；然后将摄像头采集的本地图像与预先存储的示范视频帧进行对比，进而得到对比结果。在一些实施例中可以是：当示范视频播放至预设时间点，控制器均会控制摄像头采集一个本地图像。其中，预设时间点可以是在示范视频出现第一图像为起点，每间隔T时间，均为一个预设时间点直到示范视频出现最后一帧图像。所述预设时间点也可以基于示范视频的内容生成，示范视频的内容中每个动作节点作为一个预设时间点。举例说明，对于一个示范视频，第一图像出现的起点为3s，T时间为10S，该示范视频的长度为53S。相应的预设时间点为：3S，13S，23S，33S，43S和53S，控制器会在释放视频播放至3S，13S，23S，33S，43S和53S时分别控制摄像头采集一个本地图像。示范视频中会按照预设时间节点增加标签，在播放时间出发标签时，采集本地图像。
再例如，摄像头一直处于开启的状态，并实时的录制本地视频，并本地视频发送至控制器。控制器可以在预设时间点处，抽取采集到的本地视频中对应的本地图像；然后将抽取的本地图像与预先存储的示范视频帧进行对比，进而得到对比结果。具体的实现过程：当示范视频播放至预设时间点，控制器在摄像头采集的本地视频中抽取一个或多个本地图像。其中，预设时间点可以是在示范视频出现第一图像为起点，每间隔T时间(即出现示范动作的时间)，均为一个预设时间点直到示范视频出现最后一帧图像。所述预设时间点也可以基于示范视频的内容生成或预先标记的，示范视频的内容中每个动作节点作为一个预设时间点。举例说明，对于一个示范视频，第一图像出现的起点为3S，预设时间点为：3S、16S、23S、45S和53S，控制器会在释放视频播放至3S、16S、23S、45S和53S时在本地视频中采集一个本地图像。可知的，示范动作的出现时间是任意的，待比对图像的获取是根据标识示范动作的标签或时间点触发的。
通常用户是在观看示范视频播放的教练动作后，模仿着教练动作出相应的动作。用户从接收到示范动作到作出相应的动作这段时间内天然的存在一定的延迟。为了抵消延迟，本申请实施例示出的技术方案示出了一种“延迟图像的采集方法”。本实施例中引入延迟采集时间点的概念，其中，延迟采集时间点＝预设时间点+预置反应时长。当示范视频播放至延迟采集时间点时，控制器控制摄像头采集本地视频。
本申请实施例示出的技术方案用过大量的实验数据统计。用户从接收到示范动作到作 出相应的动作这段时间内用户的反应时间为1S，相应的本申请实施例示出的技术方案将预置反应时长配置为1S。举例说明，预设时间点为：3S、13S、23S、33S、43S和53S，相应的延迟采集时间点为：4S、14S、24S、24S、44S和54S。在示范视频第一次出现图像帧为起点，在起点之后14S、14S、24S、24S、44S和54S控制器控制摄像头分别采集本地图像。
控制器，将本地图像与示范视频帧进行比对，生成比对结果；其中，示范视频帧为示范视频的中与本地图像相对应的图像，或示范视频中对应该预设时间点的标准图像帧。在本申请实施例示出的技术方案中，所述示范视频的中的图像可以为带有标志图像。所述标志可以为时间标志，但并不是限于时间标志。后续可以基于所述标志确定本地图像与示范视频帧的对应关系。举例说明，当示范视频播放至4S时，控制器控制摄像头采集本地视，所述本地视频对应的时间标志为3s，并将该本地视频与时间标志为3S的目标视频进行比对。
在一些实施例中，示范视频中存储有示范视频帧的关节点数据，该关节点数据是提前预置的，示范视频帧之外的其他图像帧由于不存在需求，因此可以不预置关节点数据。
在上述实施例示出的技术方案中，预置反应时长被配置为1S。但是1S为一个统计的数据，通常情况下用户的反应时长为1S，但是1S并不适用于所有的用户，在实际应用的过程中可以根据需求设置预置反应时长。
动作比对通过图像比对会造成较大的处理负担，为了进一步降低控制器的数据处理量本申请实施例示出的技术方案可以仅针对本地图像与示范视频中的中一些“关键部位”进行比对具体的实现过程：即，动作的比对是通过关节点的比对完成的。
在所述第二视频窗口播放所述本地图像之前，所述控制器还用于：识别所述本地图像中的关节点；比对所述本地图像中的关节点和示范视频中的关节点。
在一些可行性实施例中，所述控制器被配置为响应于用户对第一显示界面中开始训练控件的选中，控制器控制摄像头启动，以使得摄像头采集本地图像。摄像头将采集的本地图像传输至控制器。控制器识别所述本地图像中的关节点；在一些实施例中，控制器根据预设的模型识别本地图像中的关节点，所述关节点为人体关节对应的点以及人体的头部对应点，通常人体包括13个关节位。控制器通过对全身13个重要骨骼关节点位的标注。其中，标注了13个关节位的本地图像可以参阅图11。13个关节位分别为：左腕部、左肘部、左肩部、胸腔部、腰部、左膝盖部、左踝部、头部、右腕部、右肘部、右肩部、右膝盖部、右踝部。但在一些采集到的本地图像中人体有时出出现部分缺失的状况，此时可以仅识别图像中的人体部分。
所述控制器还用于，比对所述本地图像中的关节点和示范视频/示范视频帧中的关节点，确定本地图像中人体动作和示范视频中人体动作差异程度；在采集到的本地图像中标注识别出的关节点，其中，不同的颜色标注不同动作差异程度的关节点。
其中，确定本地图像中人体动作和示范视频中人体动作差异程度的实现方式有多种：
例如，比对方式可以是比对本地图像中人体的关节点所处的位置与示范视频中人体的关节点所处的相对位置。基于相对位置的差异得到比对结果。不同的比对结果采用不同的颜色进行标注。
举例说明：本地图像中人体的左腕部与示范视频中人体的左腕部位置相差10标准值，可以采用红色标注左腕部关节点。本地图像中人体的右腕部与示范视频中人体的右腕部位 置相差1标准值，可以采用绿色标注右腕部关节点。
再例如，比对方式可以计算两个关节位的匹配度，根据匹配度生成相应的结果。或者根据自身关节点之间的相对位置关系，确定动作的匹配程度。
在一些实施例中关节点的识别和匹配还可以采用相关技术中的其他可实现手段来代替。
在一些实施例中，预先在示范视频中标注出示范关节位，并将示范关节位与示范视频一同存储在本地的数据列表中。示范关节位的标注过程与前述一些实施例示出的标注过程类似，在此便不详细说明。
在一些实施例中，控制器将本地图像中的第一角度与对应的标准角度进行比对，生成比对结果。其中，第一角度为本地图像中每一个关节位与邻近关节位连线与躯干连线之间的夹角；标准角度为示范视频中每一个关节位与邻近关节位连线与躯干连线之间的夹角。
其中第一角度与标准角度之间的对应关系，可以基于时间标识生成。举例说明，本地图像的采集时间为10S，那么左踝部的第一角度对应的标准角度为示范视频中第10S出现的图像中的左踝部与邻近关节位连线与躯干连线之间的夹角。
举例说明，请参阅图12，图12为根据一些实施例示出的打了关节批注的本地图像。对于左腕部1A而言，左腕部1A临近的关节位为左肘部1B，相应的左腕部1A与左肘部1B连接线与躯干连线之间的夹角称之为的第一角度1a。通过上述方法可以分别计算左肘部，左肩部，左膝盖部，左踝部，头部，右腕部，右肘部，右肩部，右膝盖部，右踝部对应的第一角度。
其中，标准角度的生成方式可以参考第一角度的生成方式，在此便不赘述。
控制器计算第一角度与对应的标准角度的匹配度；根据所述匹配度可以用于评价用户动作完成的好坏程度。
可见本申请实施例示出的技术方可以计算出每个关节位所处的位置与标准位置之间的差异，进而帮助用户每个部位的完成情况提高用户的体验感。
为了帮助用户进一步了解各个部位动作完成的情况，本申请实施例示出的技术方案，控制器计算第一角度与对应的标准角度的匹配度，根据所述匹配度符合的区域，将关节点标注相应的颜色。
举例说明：在一些实施例示出的技术方案中，所述匹配度可以用角度偏差表示，匹配结果根据预设标准差值。对于角度偏差大于15度，可以将相应的关节位标记为红色；对于10度～15度偏差，可以将相应的关节位标记为黄色。对于10度以下的度偏差，可以将相应的关节位标记为绿色。
举例说明，10S采集的本地图像中应左腕关节的第一角度与示范示范视频中10s对应的标准角度的相差20度，可以将相应的左腕关节标记为红色；10S采集的本地图像中应左踝关节的第一角度与示范示范视频中10s对应的标准角度的相差12度，可以将相应的左腕关节标记为黄色；10S采集的本地图像中应头部的第一角度与示范示范视频中10s对应的标准角度的相差6度，可以将相应的左腕关节标记为绿色，相应的，标注后的本地图像可以参阅图13。
由于人体世界动作的不匹配新，本申请实施例示出的技术方案示出一个“范围”比对方式。即在示范视频的播放到示范视频帧的时候，显示设备从本地视频中获取该时间点邻近的多个图像帧，在一些实施例中当示范视频播放至预置时间点时，控制器在本地视频中 选取该时间点邻近的多个图像帧作为第一图像集，所述第一图像集至少包括第一本地图像和第二本地图像，其中第一本地图像为预置时间点对应的本地图像，第二本地图像为临近预设时间点对应的本地图像。
在一些实施例中，控制器计算所述第一图像集中的本地图像与示范视频帧匹配度，将匹配度最高的本地图像的比对结果作为该时间点的比对结果，将和示范视频帧匹配度最好的本地图像作为与该时间点对应的本地图像。
在一些实施例中还可以是：控制器计算第一本地图像与示范视频帧的匹配度(也可称之为人体动作差异程度)，在人体动作差异程度大于预设阈值的时候，控制器在所述第一图像集中筛选出与示范视频帧匹配度最高的图像作为替换图像，并根据替换图像与示范视频帧的比对结果，标注替换图像。
举例说明，对于一个10S采集的本地图像中应腕关节的第一角度与示范示范视频中10S对应的标准角度的匹配度为20％，预置匹配度(预设阈值)为25％。在此情况下，控制器确定所述目标数据集的第一图像集；第一图像集为目标数据集中1S-13S这一时间段内包含的本地图像。分别计算每个本地图像中腕关节的第一角度与10S示范视频帧中腕关节的标准角度的匹配度，计算结果为8S对应的数据与10S示范视频帧中腕关节的标准角度的匹配度为80％，80％最高匹配度。将10S对应腕关节的比对结果调整为80％，并用80％对用的颜色标注腕关节，控制器缓存标注后的本地视频。
在一些实施例中，当完成示范视频的播放，控制器可以控制显示器显示练习评价界面，所述练习评价界面用于展示标注后的本地图片。练习评价界面可以参阅图14-1，练习评价界面上可以同时展示用户的评分等级，用户的动作以及规范动作。其中评分等级可以基于本地图像与示范视频帧的匹配度生成。
在一些实施例中，练习评价界面可以以滚动的方式展示多个对应的用户的动作以及规范动作。其中，展示的先后顺序可以为：按照评分由低到高依次展示。其中，和示范视频帧匹配程度越高，评分越高。
在另一些实施例中，练习评价界面的展示形式可以参阅图14-1，练习评价界面置有两个显示窗口，一个显示用户动作中该时间点对应的本地图像，一个显示规范动作对应的示范视频帧。
在一些实施例中，为了进一步降低控制器的数据处理量，可以将‘关节点比对过程’放置在服务器端执行，具体的实现过程：
在一些实施例中，在所述第二视频窗口播放所述本地图像之前，所述控制器还用于：识别所述本地图像中的关节点；将所述本地图像中的关节点发送给服务器，所述服务器以使得服务器可以比对所述本地图像中的关节点和示范视频中的示范视频帧的关节点，确定本地图像中人体动作和示范视频中的示范视频帧的人体动作差异程度并生成反馈信息给所述显示设备。
在一些实施例中，显示设备中的关节点识别单元对摄像头采集的所有图像进行关节点的识别和标记，并在第二播放窗口进行显示。在示范视频播放至示范视频帧时，显示设备将此刻采集的本地图像的关节点数据和/或邻近时刻采集的本地图像的关节点数据上传给服务器已进行匹配程度的判断。
其中，本地图像中人体动作和示范视频中人体动作差异程度的比对方式，可以参阅上述实施例，在此便不赘述。
控制器被进一步配置为接收服务器发送的反馈消息，根据所述反馈消息在本地图像中标注识别出的关节点，其中，不同的颜色标注不同动作差异程度的关节点。
进一步的，本申请实施例示出的技术方案，采用不同的颜色来标记各关节位动作完成的情况。采用不同的颜色来区分用户各关节的完成情况，不同的颜色起到醒目的作用。可见采用本申请实施例示出的方案进一步帮助用户了解各个部位动作完成的情况。
在一些实施例中，如图14-2所述，在第二显示界面中，如果该时间点用户的动作和示范视频帧的匹配动作较高，增在第二播放窗口中增加浮层以显示提示语句，来对用户进行鼓励。
在一些实施例中，如图14-2所述，在第二显示界面中，第二播放窗口的上方还设置有训练进度控件，以展示用户动作的完成程度，控制器在检测到在用户动作和示范动作帧的匹配程度高于预设值时，控制训练进度控件中显示的完成程度值提高。在检测到在用户动作和示范动作帧的匹配程度低于预设值时，控制训练进度控件中显示的完成程度值维持不变。
为了减少服务器的数据处理量，在一些实施例中，服务器可以针对预设时间点对应的本地图像进行处理，具体的实现过程可以是：所述控制器将所述本地图像中的关节点发送给服务器，具体为：所述控制器，在所述示范视频的播放时间达到预设时间点时，缓存所述预设时间点前后预定时间段内采集到的本地图像。识别缓存的本地图像的关节点数据并将识别出的关节点数据发送给服务器。
其中，本地图像的缓存过程可以参阅上述实施在此便不赘述。
值得注意的是，由于图片在传输过程中占用的带宽较大，为了降低数据传输过程中占用的带宽本实施例示出的方案将本地视频的的关节点发送给服务器。
在一些实施例中，所述控制器还可以进一步被配置为，发送所述识别出的关节点数据发送给服务器的同时，将预设时间点发送个服务器，以使的所述服务器根据所述预设时间点确定用于比对的示范视频的图像帧(即目标图像)。
在一些可行性实施例中所述控制器，在人体动作差异程度大于预设阈值的时候，标注所述本地图像并缓存标注后的本地图像的图片和预设时间点对应的示范视频帧。以便在示范视频播放结束时，调取动作差异较大的本地视频。
在一些可行性实施例中所述控制器，在播放结束后，控制显示器显示练习评价界面，并在所述练习评价界面展示缓存的标注后的本地图像的图片和预设时间点对应的示范视频帧。
在一些实施例中，对预设时间点的示范视频帧和对应的本地图像按照匹配程度(或评分)进行排序，在示范视频播放结束后，选取匹配程度(或评分)低的预设个数的时间点的示范视频帧和对应的本地图像进行展示。示例性的，例如缓存了5个时间点的示范视频帧和对应的本地图像按照匹配程度，在示范视频播放结束后，选取匹配程度(或评分)低的3个的时间点的示范视频帧和对应的本地图像进行展示。
其中，练习评价界面的展示方式可以参阅上述实施例。
本行申请实施例还示出一种显示设备，包括：
显示屏，用于显示第一显示界面和所述第二显示界面，所述第一显示界面包括用于控制播放示范视频的播放控件，所述第二显示界面中包含用于播放所述示范视频的第一播放窗口和用于播放摄像头采集到的本地图像的第二播放窗口；
摄像头，用于采集本地图像；
控制器，所述控制器被配置为：
接收输入的对所述播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
响应于所述确认操作，展示所述第二界面；
在示范视频播放过程中，检测到表征所述示范视频的播放时间达到预设时间点的标签时，截取采集到的本地视频的当前视频帧及时间上邻近所述当前视频帧的邻近视频帧；
识别所述当前视频帧的关节点与所述邻近视频帧的关节点；
比对所述当前视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频帧的关节点；比对所述邻近视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频帧的关节点；
根据比对结果标注所述前视频帧或邻近视频帧的人体动作差异程度；
缓存标注后的人体动作差异程度低于差异阈值的所述当前视频帧或所述邻近视频帧，以及所述示范视频帧，以用于练习评价界面的展示。
本申请实施例还示出一种显示设备，包括：
显示屏，用于显示第一显示界面和所述第二显示界面，所述第一显示界面包括用于控制播放示范视频的播放控件，所述第二显示界面中包含用于播放所述示范视频的第一播放窗口和用于播放摄像头采集到的本地图像的第二播放窗口；
摄像头，用于采集本地图像；
控制器，所述控制器被配置为：
接收输入的对所述播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
响应于所述确认操作，展示所述第二界面，并在所述第一播放窗口播放所述示范视频，在所述第二视频窗口播放进行关节点标注后的所述本地图像，其中在所述关节点标注后的所述本地图像中，第一关节点被标注为第一颜色，第二关节点被标注为第二颜色，第一关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度大于第二关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度。
本申请实施例还示出一种界面显示方法，包括：
在显示第一界面时，接收输入的对所述第一界面中的播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
响应于所述确认操作，显示所述第二界面，并在所述第二界面中的第一播放窗口播放所述示范视频，在所述第二界面中的第二视频窗口播放所述本地图像。
本申请实施例还示出一种界面显示方法，包括：
在显示第一界面时，接收输入的对所述播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
响应于所述确认操作，展示所述第二界面；
在示范视频播放过程中，检测到表征所述示范视频的播放时间达到预设时间点的标签时，截取采集到的本地视频的当前视频帧及时间上邻近所述当前视频帧的邻近视频帧(视频帧在本申请实施例示出的方案中也可称之为图像)；
识别所述当前视频帧的关节点与所述邻近视频帧的关节点；
比对所述当前视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频帧的关节点；比对所述邻近视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频帧的关节点；
根据比对结果标注所述前视频帧或邻近视频帧的人体动作差异程度；
缓存标注后的人体动作差异程度低于差异阈值的所述当前视频帧或所述邻近视频帧，以及所述示范视频帧；
响应于所述示范视频播放结束，显示练习评价界面，其中所述练习评价界面显示所述标注后的人体动作差异程度较低的所述当前视频帧或所述邻近视频帧，以及所述示范视频帧。
本申请实施例还示出一种界面显示方法，包括：
在显示第一界面时，接收输入的对所述第一界面中的播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
响应于所述确认操作，展示所述第二界面，并在所述第二界面中的第一播放窗口播放所述示范视频，在所述第二界面中的第二视频窗口播放进行关节点标注后的所述本地图像，其中在所述关节点标注后的所述本地图像中，第一关节点被标注为第一颜色，第二关节点被标注为第二颜色，第一关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度大于第二关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度。
本实施例提供的方法可以适用于前述实施例提供的双芯片的显示装置，也可以适用于非双芯片的显示装置
基于本申请中示出的示例性实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。此外，虽然本申请中公开内容按照示范性一个或几个实例来介绍，但应理解，可以就这些公开内容的各个方面也可以单独构成一个完整技术方案。
应当理解，本申请中说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，例如能够根据本申请实施例图示或描述中给出那些以外的顺序实施。
此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖但不排他的包含，例如，包含了一系列组件的产品或设备不必限于清楚地列出的那些组件，而是可包括没有清楚地列出的或对于这些产品或设备固有的其它组件。
最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (12)

  1. 一种显示设备，其特征在于，包括：
    显示屏，用于显示第一显示界面和所述第二显示界面，所述第一显示界面包括用于控制播放示范视频的播放控件，所述第二显示界面中包含用于播放所述示范视频的第一播放窗口和用于播放摄像头采集到的本地图像的第二播放窗口；
    摄像头，用于采集本地图像；
    控制器，所述控制器被配置为：
    接收输入的对所述播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
    响应于所述确认操作，展示所述第二界面，并在所述第一播放窗口播放所述示范视频，在所述第二视频窗口播放所述本地图像。
  2. 如权利要求1所述的显示设备，其特征在于，在所述第二视频窗口播放所述本地图像之前，所述控制器还用于：
    识别所述本地图像中的关节点；
    比对所述本地图像中的关节点和示范视频中的关节点，确定本地图像中人体动作和示范视频中人体动作差异程度；
    在采集到的本地图像中标注识别出的关节点，其中，不同的颜色标注不同动作差异程度的关节点。
  3. 如权利要求1所述的显示设备，其特征在于，在所述第二视频窗口播放所述本地图像之前，所述控制器还用于：
    识别所述本地图像中的关节点；
    将所述本地图像中的关节点发送给服务器，所述服务器用于比对所述本地图像中的关节点和示范视频中的关节点，确定本地图像中人体动作和示范视频中人体动作差异程度并生成反馈信息给所述显示设备；
    接收服务器发送的反馈消息，根据所述反馈消息在本地图像中标注识别出的关节点，其中，不同的颜色标注不同动作差异程度的关节点。
  4. 如权利要求3所述的显示设备，其特征在于；
    所述控制器将所述本地图像中的关节点发送给服务器，具体为：所述控制器，
    在所述示范视频的播放时间达到预设时间点时，缓存所述预设时间点前后预定时间段内采集到的本地图像；
    识别缓存的本地图像的关节点数据并将识别出的关节点数据发送给服务器。
  5. 如权利要求4所述的显示设备，其特征在于；所述控制器，
    发送所述识别出的关节点数据发送给服务器的同时，将预设时间点发送个服务器，以使的所述服务器根据所述预设时间点确定用于比对的示范视频的图像帧。
  6. 如权利要求2或3所述的显示设备，其特征在于；所述控制器，
    在人体动作差异程度大于预设阈值的时候，标注所述本地图像并缓存标注后的本地图像的图片和预设时间点对应的示范视频帧。
  7. 如权利要求6所述的显示设备，其特征在于；所述控制器，
    在播放结束后，控制显示器显示练习评价界面，并在所述练习评价界面展示缓存的标 注后的本地图像的图片和预设时间点对应的示范视频帧。
  8. 一种显示设备，其特征在于，包括：
    显示屏，用于显示第一显示界面和所述第二显示界面，所述第一显示界面包括用于控制播放示范视频的播放控件，所述第二显示界面中包含用于播放所述示范视频的第一播放窗口和用于播放摄像头采集到的本地图像的第二播放窗口；
    摄像头，用于采集本地视频；
    控制器，所述控制器被配置为：
    接收输入的对所述播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
    响应于所述确认操作，展示所述第二界面；
    在示范视频播放过程中，检测到表征所述示范视频的播放时间达到预设时间点的标签时，截取采集到的本地视频的当前视频帧及时间上邻近所述当前视频帧的邻近视频帧；
    识别所述当前视频帧的关节点与所述邻近视频帧的关节点；
    比对所述当前视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频帧的关节点；比对所述邻近视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频帧的关节点；
    根据比对结果标注所述前视频帧或邻近视频帧的人体动作差异程度；
    缓存标注后的人体动作差异程度低于差异阈值的所述当前视频帧或所述邻近视频帧，以及所述示范视频帧，以用于练习评价界面的展示。
  9. 一种显示设备，其特征在于，包括：
    显示屏，用于显示第一显示界面和所述第二显示界面，所述第一显示界面包括用于控制播放示范视频的播放控件，所述第二显示界面中包含用于播放所述示范视频的第一播放窗口和用于播放摄像头采集到的本地图像的第二播放窗口；
    摄像头，用于采集本地图像；
    控制器，所述控制器被配置为：
    接收输入的对所述播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
    响应于所述确认操作，展示所述第二界面，并在所述第一播放窗口播放所述示范视频，在所述第二视频窗口播放进行关节点标注后的所述本地图像，其中在所述关节点标注后的所述本地图像中，第一关节点被标注为第一颜色，第二关节点被标注为第二颜色，第一关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度大于第二关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度。
  10. 一种界面显示方法，其特征在于，包括：
    在显示第一界面时，接收输入的对所述第一界面中的播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
    响应于所述确认操作，显示所述第二界面，并在所述第二界面中的第一播放窗口播放所述示范视频，在所述第二界面中的第二视频窗口播放所述本地图像。
  11. 一种界面显示方法，其特征在于，包括：
    在显示第一界面时，接收输入的对所述播放控件的确认操作，启动摄像头以使得所述摄像头采集本地视频，并加载所述示范视频的视频数据；
    响应于所述确认操作，展示所述第二界面；
    在示范视频播放过程中，检测到表征所述示范视频的播放时间达到预设时间点的标签时，截取采集到的本地视频的当前视频帧及时间上邻近所述当前视频帧的邻近视频帧；
    识别所述当前视频帧的关节点与所述邻近视频帧的关节点；
    比对所述当前视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频帧的关节点；比对所述邻近视频帧的关节点和所述示范视频中和所述预设时间点对应的示范视频帧的关节点；
    根据比对结果标注所述前视频帧或邻近视频帧的人体动作差异程度；
    缓存标注后的人体动作差异程度低于差异阈值的所述当前视频帧或所述邻近视频帧，以及所述示范视频帧；
    响应于所述示范视频播放结束，显示练习评价界面，其中所述练习评价界面显示所述标注后的人体动作差异程度较低的所述当前视频帧或所述邻近视频帧，以及所述示范视频帧。
  12. 一种界面显示方法，其特征在于，包括：
    在显示第一界面时，接收输入的对所述第一界面中的播放控件的确认操作，启动摄像头，并加载所述示范视频的视频数据；
    响应于所述确认操作，展示所述第二界面，并在所述第二界面中的第一播放窗口播放所述示范视频，在所述第二界面中的第二视频窗口播放进行关节点标注后的所述本地图像，其中在所述关节点标注后的所述本地图像中，第一关节点被标注为第一颜色，第二关节点被标注为第二颜色，第一关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度大于第二关节点所在的身体部位和示范视频中对应的身体部分的动作差异程度。
PCT/CN2020/105277 2019-08-18 2020-07-28 一种界面显示方法及显示设备 WO2021031809A1 (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201910761455 2019-08-18
CN201910761455.8 2019-08-18
CN202010386547.5A CN112399234B (zh) 2019-08-18 2020-05-09 一种界面显示方法及显示设备
CN202010386547.5 2020-05-09

Publications (1)

Publication Number Publication Date
WO2021031809A1 true WO2021031809A1 (zh) 2021-02-25

Family

ID=74603787

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/105277 WO2021031809A1 (zh) 2019-08-18 2020-07-28 一种界面显示方法及显示设备

Country Status (2)

Country Link
CN (2) CN116074564B (zh)
WO (1) WO2021031809A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114513694A (zh) * 2022-02-17 2022-05-17 平安国际智慧城市科技股份有限公司 评分确定方法、装置、电子设备和存储介质
CN115883748A (zh) * 2022-11-28 2023-03-31 中汽创智科技有限公司 一种数据回放的同步方法、装置、电子设备及存储介质

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113051432B (zh) * 2021-04-25 2022-07-19 聚好看科技股份有限公司 显示设备及媒资播放方法
CN112272324B (zh) * 2020-10-15 2023-03-14 聚好看科技股份有限公司 一种跟练模式控制方法及显示设备
CN114296668B (zh) * 2021-03-11 2024-08-23 海信视像科技股份有限公司 一种显示设备
CN113099308B (zh) * 2021-03-31 2023-10-27 聚好看科技股份有限公司 一种内容显示方法、显示设备及图像采集器
CN113556599A (zh) * 2021-07-07 2021-10-26 深圳创维－Rgb电子有限公司 视频教学方法、装置、电视机及存储介质
CN113794917A (zh) * 2021-09-15 2021-12-14 海信视像科技股份有限公司 一种显示设备和显示控制方法
CN115202531A (zh) * 2022-05-27 2022-10-18 当趣网络科技(杭州)有限公司 界面交互的方法、系统和电子装置
WO2023240973A1 (zh) * 2022-06-16 2023-12-21 聚好看科技股份有限公司 显示设备及投屏方法
CN116320583A (zh) * 2023-03-20 2023-06-23 抖音视界有限公司 视频通话方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9440134B2 (en) * 2010-12-06 2016-09-13 Full-Swing Golf, Inc. Microsoft kinect
CN107909060A (zh) * 2017-12-05 2018-04-13 前海健匠智能科技（深圳）有限公司 基于深度学习的健身房健身动作识别方法及装置
CN108853946A (zh) * 2018-07-10 2018-11-23 燕山大学 一种基于Kinect的健身指导训练系统及方法
CN109214231A (zh) * 2017-06-29 2019-01-15 深圳泰山体育科技股份有限公司 基于人体姿态识别的体育教学辅助系统和方法

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8620146B1 (en) * 2008-03-28 2013-12-31 Theresa Coleman Picture-in-picture video system for virtual exercise, instruction and entertainment
US9011293B2 (en) * 2011-01-26 2015-04-21 Flow-Motion Research And Development Ltd. Method and system for monitoring and feed-backing on execution of physical exercise routines
JP2013103010A (ja) * 2011-11-15 2013-05-30 Sony Corp 画像処理装置、画像処理方法及びプログラム
WO2014162787A1 (ja) * 2013-04-02 2014-10-09 Ｎｅｃソリューションイノベータ株式会社 身体動作採点装置、ダンス採点装置、カラオケ装置及びゲーム装置
CN105898133A (zh) * 2015-08-19 2016-08-24 乐视网信息技术（北京）股份有限公司 一种视频拍摄方法及装置
CN106919890A (zh) * 2015-12-25 2017-07-04 中国移动通信集团公司 一种评价用户动作标准性的方法及装置
CN106131611A (zh) * 2016-06-29 2016-11-16 乐视控股（北京）有限公司 一种识别节目频道的方法、终端及服务器
CN106228143A (zh) * 2016-08-02 2016-12-14 王国兴 一种教学视频与摄像头视频运动对比评分的方法
CN108960002A (zh) * 2017-05-17 2018-12-07 中兴通讯股份有限公司 一种动作调整信息提示方法及装置
CN107968921B (zh) * 2017-11-23 2020-02-28 香港乐蜜有限公司 视频生成方法、装置和电子设备
CN107952238B (zh) * 2017-11-23 2020-11-17 香港乐蜜有限公司 视频生成方法、装置和电子设备
CN107943291B (zh) * 2017-11-23 2021-06-08 卓米私人有限公司 人体动作的识别方法、装置和电子设备
CN108537284A (zh) * 2018-04-13 2018-09-14 东莞松山湖国际机器人研究院有限公司 基于计算机视觉深度学习算法的姿态评估打分方法和系统
CN109144247A (zh) * 2018-07-17 2019-01-04 尚晟 视频交互的方法以及基于可交互视频的运动辅助系统
CN109068081A (zh) * 2018-08-10 2018-12-21 北京微播视界科技有限公司 视频生成方法、装置、电子设备及存储介质
CN109432753B (zh) * 2018-09-26 2020-12-29 Oppo广东移动通信有限公司 动作矫正方法、装置、存储介质及电子设备
CN109376705A (zh) * 2018-11-30 2019-02-22 努比亚技术有限公司 舞蹈训练评分方法、装置及计算机可读存储介质
CN109451178B (zh) * 2018-12-27 2021-03-12 维沃移动通信有限公司 视频播放方法及终端
CN110008814A (zh) * 2019-01-25 2019-07-12 阿里巴巴集团控股有限公司 视频处理方法、视频处理装置和电子设备
CN110087106A (zh) * 2019-04-18 2019-08-02 杰哈思文化创意(杭州)有限公司 刷牙视频的播放方法、装置、存储介质和终端

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9440134B2 (en) * 2010-12-06 2016-09-13 Full-Swing Golf, Inc. Microsoft kinect
CN109214231A (zh) * 2017-06-29 2019-01-15 深圳泰山体育科技股份有限公司 基于人体姿态识别的体育教学辅助系统和方法
CN107909060A (zh) * 2017-12-05 2018-04-13 前海健匠智能科技（深圳）有限公司 基于深度学习的健身房健身动作识别方法及装置
CN108853946A (zh) * 2018-07-10 2018-11-23 燕山大学 一种基于Kinect的健身指导训练系统及方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114513694A (zh) * 2022-02-17 2022-05-17 平安国际智慧城市科技股份有限公司 评分确定方法、装置、电子设备和存储介质
CN115883748A (zh) * 2022-11-28 2023-03-31 中汽创智科技有限公司 一种数据回放的同步方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN116074564A (zh) 2023-05-05
CN112399234B (zh) 2022-12-16
CN116074564B (zh) 2024-10-01
CN112399234A (zh) 2021-02-23

Similar Documents

Publication Publication Date Title
WO2021031809A1 (zh) 一种界面显示方法及显示设备
CN113330736B (zh) 一种显示器及图像处理方法
WO2021088320A1 (zh) 显示设备和内容显示方法
WO2021032092A1 (zh) 显示设备
CN110708581B (zh) 显示设备及呈现多媒体屏保信息的方法
WO2021189358A1 (zh) 显示设备和音量调节方法
CN111464840B (zh) 显示设备及显示设备屏幕亮度的调节方法
CN112068741B (zh) 显示设备及显示设备蓝牙开关状态的显示方法
CN112333499A (zh) 寻找目标设备的方法和显示设备
CN112788422A (zh) 显示设备
CN112073662A (zh) 一种显示设备
WO2021031598A1 (zh) 视频聊天窗口位置的自适应调整方法及显示设备
CN112463267B (zh) 在显示设备屏幕上呈现屏保信息的方法及显示设备
WO2020248699A1 (zh) 一种声音处理法及显示设备
WO2020248654A1 (zh) 显示设备及应用共同显示的方法
CN112839254A (zh) 显示设备与内容显示方法
CN112073666B (zh) 一种显示设备的电源控制方法及显示设备
CN112073777B (zh) 一种语音交互方法及显示设备
CN112399245A (zh) 一种播放方法及显示设备
CN113301404A (zh) 显示设备和控制方法
CN112073773A (zh) 一种屏幕互动方法、装置及显示设备
WO2020248788A1 (zh) 一种语音控制方法及显示设备
CN112073779B (zh) 显示设备及按键传递的容错方法
CN112995762B (zh) 显示设备及网络状态同步方法
CN112995113B (zh) 一种显示设备、端口控制方法及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20855353

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20855353

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 16/09/2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20855353

Country of ref document: EP

Kind code of ref document: A1