CN116801019A - Screen projection method and display device - Google Patents

Abstract

The disclosure relates to a screen projection method and a display device, comprising: responding to the received starting instruction, triggering a display to display a heel training interface; acquiring a training video stream corresponding to a training interface from a server, and receiving a rendered training video stream sent by a terminal device; when the first training image corresponding to the first playing time in the training video stream is not matched with the first training image action corresponding to the first playing time in the rendered training video stream, searching a second training image matched with the first training image action in the training video stream; when a second training image exists in the training video stream, a first indicating signal is sent to the terminal equipment so as to indicate the terminal equipment to send an unrendered follow-up training video stream, the training video stream played by the display equipment is ensured to be consistent with the follow-up training video stream, and the use experience of a user is improved.

Description

Screen projection method and display device

Technical Field

The disclosure relates to the technical field of display, and in particular relates to a screen projection method and display equipment.

Background

With the rise of the body building exercises of the whole people, more and more users can select different heel training modes to punch cards for the body building of the whole people. For example, the exercise and card punching is realized through a training video stream played by the heel-exercise television, and the specific implementation mode is that the mobile phone terminal collects the user heel-exercise video stream and sends heel-exercise data to the television terminal, and the television terminal displays the training video stream and the user heel-exercise video stream collected by the mobile phone terminal respectively.

In the prior art, after a mobile phone terminal collects a user heel-exercise video stream, the collected heel-exercise video stream is generally subjected to face-beautifying treatment and then sent to a television terminal. Because the mobile phone end needs to make the face-beautifying treatment on the heel-training video stream, the training video stream played by the television end and the heel-training video stream cannot be synchronously played, the delay between the heel-training video stream and the training video stream from the screen of the mobile phone end to the television end is longer, and the use experience of a user is reduced.

Disclosure of Invention

In order to solve the technical problems or at least partially solve the technical problems, the present disclosure provides a screen projection method and a display device, which improve the use experience of a user.

In a first aspect, an embodiment of the present disclosure provides a screen projection method, including:

responding to a received starting instruction, triggering a display to display a heel training interface, wherein the heel training interface is provided with a display window and a local window, and the display window is used for displaying a training video stream;

acquiring a training video stream corresponding to the heel training interface from a server, and receiving a rendered heel training video stream sent by a terminal device, wherein the rendered heel training video stream is used for being directly displayed on the local window without being rendered by a display device;

Searching a second training image matched with the first training image action in the training video stream when the first training image corresponding to the first playing time in the training video stream is not matched with the first training image action corresponding to the first playing time in the rendered training video stream, wherein the first playing time is larger than a second playing time corresponding to the second training image;

and when the second training image exists in the training video stream, sending a first indication signal to the terminal equipment so as to indicate the terminal equipment to send an unrendered follow-up video stream.

As an implementation manner, optionally, when a first training image corresponding to a first playing time in the training video stream is not matched with a first training image action corresponding to the first playing time in the rendered training video stream, searching a second training image matched with the first training image action in the training video stream includes:

acquiring a first training image of the training video stream at a first playing time and a first heel-in image corresponding to the first playing time in the rendered heel-in video stream;

When the action matching degree of the first training image and the first training image is lower than a preset threshold, searching a second training image which meets the preset threshold with the action matching degree of the first training image from the training video stream;

and when the second training image exists in the training video stream, sending a first indication signal to the terminal equipment, wherein the method comprises the following steps:

and when the difference value between the first playing time and the second playing time is larger than or equal to a preset difference value, a first indicating signal is sent to the terminal equipment so as to indicate the terminal equipment to send the unrendered follow-up video stream.

As an alternative, after sending the first indication signal to the terminal device, the method further comprises:

receiving an unrendered heel-hold video stream sent by the terminal equipment;

determining a second training image which is matched with the first training image action of the training video stream in the unrendered training video stream;

and after rendering operation is carried out on the unrendered heel-exercise video stream, displaying the rendered heel-exercise video stream in a local window of the display, and playing the heel-exercise video stream displayed in the local window from the playing time corresponding to the second heel-exercise image.

As an alternative embodiment, the method further comprises:

and when the difference value between the first playing time and the second playing time is smaller than a preset difference value, not sending a first indication signal to the terminal equipment.

As an optional implementation manner, the first indication signal is further used to instruct the terminal device to suspend the rendering operation.

As an optional implementation manner, the searching the training video stream for the second training image matched with the first training image action includes:

in the training video stream, taking a first playing time as a starting point, acquiring a candidate training image of a preset time before the first playing time of the training video stream, and sending the candidate training image and the first training image to a server;

obtaining a comparison result of the candidate training image and the first training image by the server;

and determining the candidate training image, of which the matching degree with the first training image meets a preset threshold value, as the second training image.

As an optional implementation manner, the obtaining a comparison result of the candidate training image and the first training image by the server includes:

And acquiring a feature point comparison result of the server on the first image feature point of the candidate training image and the second image feature point of the first training image.

In a second aspect, embodiments of the present disclosure provide a display apparatus, including:

a display;

a controller configured to:

responding to a received starting instruction, triggering a display to display a heel training interface, wherein the heel training interface is provided with a display window and a local window, and the display window is used for displaying a training video stream;

acquiring a training video stream corresponding to the heel training interface from a server, and receiving a rendered heel training video stream sent by a terminal device, wherein the rendered heel training video stream is used for being directly displayed on the local window without being rendered by a display device;

searching a second training image matched with the first training image action in the training video stream when the first training image corresponding to the first playing time in the training video stream is not matched with the first training image action corresponding to the first playing time in the rendered training video stream, wherein the first playing time is larger than a second playing time corresponding to the second training image;

And when the second training image exists in the training video stream, sending a first indication signal to the terminal equipment so as to indicate the terminal equipment to send an unrendered follow-up video stream.

As an alternative, the device may, alternatively,

the controller is further configured to:

receiving an unrendered heel-hold video stream sent by the terminal equipment;

determining a second training image which is matched with the first training image action of the training video stream in the unrendered training video stream;

and after rendering operation is carried out on the unrendered heel-exercise video stream, displaying the rendered heel-exercise video stream in a local window of the display, and playing the heel-exercise video stream displayed in the local window from the playing time corresponding to the second heel-exercise image.

As an alternative, the device may, alternatively,

the controller is further configured to:

and when the difference value between the first playing time and the second playing time is smaller than a preset difference value, not sending a first indication signal to the terminal equipment.

Compared with the prior art, the technical scheme provided by the embodiment of the disclosure has the following advantages:

the screen projection method and the display device provided by the embodiment of the disclosure firstly respond to the received starting instruction, trigger the display to display the heel-and-exercise interface, then acquire the training video stream corresponding to the heel-and-exercise interface from the server, receive the rendered heel-and-exercise video stream sent by the terminal device, find the second training image matched with the first heel-and-exercise image action in the training video stream when the first training image corresponding to the first playing time in the training video stream is not matched with the first heel-and-exercise image action corresponding to the first playing time in the rendered heel-and-exercise video stream, finally send the first indication signal to the terminal device when the second training image exists in the training video stream to indicate the terminal device to send the unrendered heel-and-exercise video stream, when the display device receives the rendered heel-training video stream sent by the terminal device, and the first training image corresponding to the first playing time in the training video stream played by the display device is not matched with the first heel-training image corresponding to the first playing time in the rendered heel-training video stream, a first indication signal is sent to the terminal device to indicate the terminal device to send the unrendered heel-training video stream, namely the display device directly receives the unrendered heel-training video stream of the terminal device, so that display delay existing between the training video stream and the heel-training video stream and displayed by the display device due to time consumption of rendering operation of the heel-training video stream by the terminal device can be reduced, synchronization of the heel-training video stream and the training video stream displayed by the display device is ensured, and use experience of a user is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.

Fig. 1A is a network architecture diagram of a screen projection system of a screen projection method according to an embodiment of the present disclosure;

fig. 1B is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure;

fig. 2A is a schematic flow chart of a screen projection method according to an embodiment of the disclosure;

fig. 2B is a signal flow diagram of a screen projection method according to an embodiment of the disclosure;

FIG. 2C is a schematic illustration of interactions provided by an embodiment of the present disclosure;

FIG. 3A is a flow chart of another method of screen projection provided by an embodiment of the present disclosure;

FIG. 3B is a signal flow diagram of another screen projection method according to an embodiment of the present disclosure;

FIG. 3C is a flow chart of yet another method of screen projection provided by an embodiment of the present disclosure;

FIG. 3D is a signal flow diagram of yet another screen projection method provided by an embodiment of the present disclosure;

FIG. 4A is a flow chart of yet another method of screen projection provided by an embodiment of the present disclosure;

fig. 4B is a signal flow diagram of yet another screen projection method according to an embodiment of the disclosure;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

In order that the above objects, features and advantages of the present disclosure may be more clearly understood, a further description of aspects of the present disclosure will be provided below. It should be noted that, without conflict, the embodiments of the present disclosure and features in the embodiments may be combined with each other.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it will be apparent that the embodiments in the specification are only some, but not all, embodiments of the disclosure.

Fig. 1A schematically illustrates a network architecture diagram of a screen projection system of a screen projection method according to an embodiment of the present application. The network architecture diagram of the screen projection system comprises a terminal device 10, a display device 20, a router 30, a server 40 and the like. Wherein: the terminal equipment can be used as a transmitting end in the network architecture of the screen projection system and is used for transmitting the multimedia content and various instructions to the display equipment. In an exemplary embodiment, the terminal device includes, but is not limited to, a smart phone, a tablet, a gaming machine, a laptop computer, and the like. The display device can be used as a display end in a network architecture of the screen throwing system, and is used for receiving the multimedia content and various instructions of the terminal device and synchronously playing the received multimedia content to a display panel of the self device or a display panel of the associated device. The display device may be a related device with a larger screen, such as a virtual reality device, a set top box, a smart television, a vehicle terminal, a large open-air display panel, or the like, and of course, may also be a related device with a smaller screen, such as a smart phone, a tablet computer, a laptop computer, or the like.

The router can be used as a gateway in a network architecture of a screen throwing system for constructing a local area network and providing internet service. The router is capable of providing a local connection between the terminal device and the display device.

The server can be used as cloud equipment in a network architecture of the screen projection system and is used for providing cloud services, such as cloud account management services and the like. The server may be a rack server, a blade server, a tower server, or a rack server (including an independent server or a server cluster formed by a plurality of servers), etc.

The terminal device may discover the display device through the first network or the second network, and establish a network connection with the display device based on the first network or the second network. Through the established network connection, the terminal equipment can put the multimedia content needing to be put on the screen into the display equipment and play the multimedia content through the display equipment. The first network may be a local area network, which may be a wired local area network or a wireless local area network. As shown in fig. 1A, the wireless local area network may be a Wi-Fi network built by router 8. The second network may be a wide area network, such as the internet. The wide area network may include physical links such as coaxial cable links, twisted pair cable links, fiber optic links, combinations thereof, and the like. The wide area network may also include wireless links, such as cellular links, satellite links, and the like.

In an exemplary embodiment, as shown in fig. 1B, the terminal apparatus and the display apparatus include at least one of a modem 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

In some embodiments the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, a first interface for input/output to an nth interface.

The display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, a component for receiving an image signal from the controller output, displaying video content, image content, and a menu manipulation interface, and a user manipulation UI interface.

The display 260 may be a liquid crystal display, an OLED display, a projection device, or a projection screen.

The communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, or other network communication protocol chip or a near field communication protocol chip, and an infrared receiver. The display device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

A user interface, which may be used to receive control signals from the control device 100 (e.g., an infrared remote control, etc.).

The detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for capturing the intensity of ambient light; alternatively, the detector 230 includes an image collector such as a camera, which may be used to collect external environmental scenes, user attributes, or user interaction gestures, or alternatively, the detector 230 includes a sound collector such as a microphone, or the like, which is used to receive external sounds.

The external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, or the like. The input/output interface may be a composite input/output interface formed by a plurality of interfaces.

The modem 210 receives broadcast television signals through a wired or wireless reception manner, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, the controller 250 and the modem 210 may be located in separate devices, i.e., the modem 210 may also be located in an external device to the main device in which the controller 250 is located, such as an external set-top box or the like.

The controller 250 controls the operation of the display device and responds to the user's operations through various software control programs stored on the memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command to select a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments the controller includes at least one of a central processing unit (Central Processing Unit, CPU), video processor, audio processor, graphics processor (Graphics Processing Unit, GPU), RAM Random Access Memory, RAM), ROM (Read-Only Memory, ROM), first to nth interfaces for input/output, a communication Bus (Bus), and the like.

The user may input a user command through a Graphical User Interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.

A "user interface" is a media interface for interaction and exchange of information between an application or operating system and a user, which enables conversion between an internal form of information and a user-acceptable form. A commonly used presentation form of the user interface is a graphical user interface (Graphic User Interface, GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in a display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

In addition, the terminal device and the display device further comprise a rendering module for providing audio or video special effects rendering services, wherein the audio special effects can comprise audio reverberation, speed change, tone change, limitation, echo, compression, gain control, balance control and the like, and the video special effects can comprise beauty, make-up, face recognition, two-dimensional stickers, three-dimensional stickers and the like.

The respective functions of the terminal device and the display device are briefly described above.

The terminal device may implement a screen projection operation to the display device.

However, in the prior art, when the user follows the training video stream played by the display device, the terminal device collects the user follow-up video stream and sends the follow-up video stream to the display device after special effect rendering is performed on the collected user follow-up video stream, but the process of special effect rendering on the user follow-up video stream by the terminal device inevitably leads to incapability of synchronous playing between the training video stream played by the display device and the follow-up video stream, so that the use experience of the user is reduced.

The embodiment of the disclosure provides a screen projection method, when a display device receives a rendered heel-and-exercise video stream sent by a terminal device, by comparing a first training image corresponding to a first playing time in a training video stream played by the display device with a first heel-and-exercise image corresponding to a first playing time in the rendered heel-and-exercise video stream, by sending a first indication signal to the terminal device, the terminal device is instructed to send an unrendered heel-and-exercise video stream, i.e. the display device directly receives the unrendered heel-and-exercise video stream sent by the terminal device, so that display delay existing between the training video stream and the heel-and-exercise video stream and caused by time consumption of the rendering operation of the heel-and-exercise video stream by the terminal device can be reduced, synchronization between the heel-and-exercise video stream displayed by the display device is ensured, and use experience of a user is improved.

Fig. 2A is a schematic flow chart of a screen projection method provided by an embodiment of the present disclosure, fig. 2B is a schematic flow chart of a signal provided by an embodiment of the present disclosure, and fig. 2C is an interactive schematic flow chart provided by an embodiment of the present disclosure, where the screen projection method includes, in combination with fig. 2A, fig. 2B, and fig. 2C:

and S100, responding to the received starting instruction, and triggering a display to display the heel training interface.

The heel training interface is provided with a display window and a local window, and the display window is used for displaying training video streams.

The starting instruction can be a starting instruction based on voice, a starting instruction sent by the control device, or a starting instruction sent by the intelligent device.

In some embodiments, the control device may be a remote control, and the communication between the remote control and the display device includes infrared protocol communication or bluetooth protocol communication, and other short-range communication modes, and the display device is controlled by a wireless or wired mode. The user may control the display device by inputting user instructions through keys on the remote control, voice input, control panel input, etc.

In some embodiments, smart devices (e.g., mobile terminals, tablet computers, notebook computers, etc.) may also be used to control the display device. For example, the display device is controlled using an application running on the smart device.

In some embodiments, the display device may receive instructions not using the smart device or control device described above, but rather receive control of the user by touch or gesture, or the like.

In some embodiments, the display device may further perform control in a manner other than the control apparatus and the intelligent device, for example, the voice command control of the user may be directly received through a module configured inside the display device to obtain the voice command, or the voice command control of the user may be received through a voice control device configured outside the display device.

S200, acquiring a training video stream corresponding to the heel-and-toe training interface from a server, and receiving the rendered heel-and-toe training video stream sent by the terminal equipment.

Wherein the rendered heel-hold video stream is for display directly on the local window without rendering by the display device.

The training video stream displayed by the display interface of the display is obtained from a server of the display device.

In a specific embodiment, the terminal device includes an image acquisition module, which may be a camera, and the image acquisition module based on the terminal device acquires user video data.

As a specific implementation manner, the display device plays a training video stream, for example, a fitness training video stream, a dance training video stream, and the user acquires the user training video stream through the training video stream played by the training display device by the image acquisition module of the terminal device.

The method comprises the steps of setting a distance between a terminal device and a display device to be smaller than a preset distance threshold, and setting an angle between the orientation of an image acquisition module of the terminal device and the orientation of a display screen of the display device to be smaller than a preset angle threshold.

In the embodiment of the disclosure, when a user performs continuous limb movements, such as dance movements, gymnastics movements, etc., through a training video stream played by the heel training display device, an image acquisition module of the terminal device is utilized to acquire the user heel training video stream. It can be appreciated that, in order to keep the position of the image acquisition module fixed, jitter is prevented from occurring, the terminal device can be arranged on the fixed support, that is, the distance between the terminal device and the display device is smaller than a preset distance threshold, and the angle between the orientation of the image acquisition module of the terminal device and the orientation of the display screen of the display device is smaller than a preset angle threshold, so that the user can practice the training video stream played by the display device, and the image acquisition module of the terminal device can acquire the user practice video stream.

In order to ensure the use feeling of the user, in the prior art, after the image acquisition module of the terminal device acquires the heel-training video stream, the acquired heel-training video stream is sent to the rendering module, the heel-training video stream is rendered by the rendering module and then sent to the display device, so that the heel-training video stream played by the display device is rendered heel-training video stream.

S300, when a first training image corresponding to a first playing time in the training video stream is not matched with a first training image action corresponding to the first playing time in the rendered training video stream, searching a second training image matched with the first training image action in the training video stream.

The first playing time is greater than the second playing time corresponding to the second training image.

Because the terminal device needs to perform rendering operation on the acquired heel-and-toe video stream, the rendering operation takes a certain time, and therefore, there may be a problem that the rendered heel-and-toe video stream received by the display device is not synchronous with the training video stream, for example, the training video stream is played to the tenth frame picture at a certain playing time, and the rendered heel-and-toe video stream is only played to the eighth frame picture, so that the rendered heel-and-toe video stream watched by the user on the display device is different from the training video stream, the user cannot confirm whether the actions of the heel-and-toe video stream are consistent, and the user experience is reduced.

Based on the problems existing in the prior art, in the embodiment of the present disclosure, after receiving a rendered follow-up video stream sent by a terminal device, a first training image corresponding to a first playing time in a training video stream played by a display device is first obtained and a first follow-up image corresponding to the first playing time in the rendered follow-up video stream, then whether the first training image is matched with the first follow-up image is compared, when the first training image is not matched with the first follow-up image, the training video stream played by the display device is not synchronous with the follow-up video stream, and the training video stream is not synchronous with the follow-up video stream possibly because the terminal device needs to perform a rendering operation on the training video stream, and at this time, a second training image matched with the first follow-up image and a second playing time corresponding to the second training image are searched in the training video stream by searching for the second training image matched with the first follow-up image before the first playing time.

In a specific embodiment, searching the training video stream for a second training image that matches the first training image action includes: in the training video stream, taking the first playing time as a starting point, acquiring a candidate training image of a preset time before the first playing time of the training video stream, and transmitting the candidate training image and a first training image to a server; obtaining a comparison result of the candidate training image and the first training image by the server; and determining the candidate training image, the matching degree of which with the first training image meets a preset threshold value, as a second training image.

Specifically, a first playing time of a first training image of a training video stream played by a display device is taken as a starting point, a candidate training image of a preset time before the first playing time of the training video stream is obtained, the matching degree of the candidate training image of the previous playing time of the first playing time and a first training image is compared, and when the matching degree of the candidate training image of the previous playing time of the first playing time and the first training image meets a preset threshold value, a second training image is determined to be a candidate training image corresponding to the previous playing time of the first playing time; when the matching degree of the candidate training image at the previous playing time and the first training image at the first playing time does not meet the preset threshold, acquiring candidate training images corresponding to the previous two playing times of the training video stream, comparing the matching degree of the candidate training images at the previous two playing times and the first training image at the first playing time, when the matching degree of the candidate training images at the previous two playing times and the first training image at the first playing time meet the preset threshold, determining that the second training image is the candidate training image corresponding to the previous two playing times at the first playing time, and so on, comparing the candidate training images in turn until the second training image is determined, if the candidate training images are compared, and when the candidate image of which the matching degree with the first training image meets the preset threshold is not acquired, determining that the second training image does not exist in the training video stream.

In a specific embodiment, the specific process of obtaining the comparison result of the candidate training image and the first training image by the server includes:

after acquiring a candidate training image and a first training image sent by a controller of a display device, a server firstly extracts a first image feature point of the candidate training image and a second image feature point of the first training image, and then compares the extracted first image feature point of the candidate training image with the second image feature point of the first training image based on an image processing algorithm to determine a feature point comparison result.

And S400, when a second training image exists in the training video stream, sending a first indication signal to the terminal equipment so as to indicate the terminal equipment to send an unrendered follow-up training video stream.

When the first training image corresponding to the first playing time in the training video stream is not matched with the first heel training image corresponding to the first playing time in the rendered heel training video stream, and the second training image exists in the training video stream, the training video stream played by the characterization display device is not synchronous with the heel training video stream, and the first indication signal is sent to the terminal device to indicate the terminal device to send the unrendered heel training video stream.

According to the screen projection method provided by the embodiment of the disclosure, firstly, a display is triggered to display a heel-and-toe interface in response to a received starting instruction, then a training video stream corresponding to the heel-and-toe interface is obtained from a server, the rendered heel-and-toe video stream transmitted by a terminal device is received, when a first training image corresponding to a first playing moment in the training video stream is not matched with a first heel-and-toe image corresponding to a first playing moment in the rendered heel-and-toe video stream, a second training image matched with the first heel-and-toe image action is searched in the training video stream, finally, when a second training image exists in the training video stream, a first indicating signal is transmitted to the terminal device, so that the terminal device is instructed to transmit the unrendered heel video stream, namely, after the display device receives the rendered heel-and-toe video stream transmitted by the terminal device, the first training image corresponding to the first playing moment in the training video stream is determined to be not matched with the first heel-and-toe image action in the rendered heel-and-toe video stream, and the time consumption of the terminal device is reduced, and the time consumption of the display device is guaranteed to be shortened, and the display device is enabled to directly display the heel-and-toe video stream is rendered, and the display device is enabled to have a time consumption of the display device.

Fig. 3A is a schematic flow chart of another screen projection method provided by an embodiment of the present disclosure, and fig. 3B is a schematic flow chart of a signal of another screen projection method provided by an embodiment of the present disclosure, where the embodiment of the present disclosure is based on the foregoing embodiment, and in combination with fig. 3A and fig. 3B, when a specific implementation manner of step S300 includes:

s301, acquiring a first training image of a training video stream at a first playing time and a first training image corresponding to the first playing time in the rendered training video stream.

Because the heel-training video stream received by the display device is the rendered heel-training video stream, the first training image and the first heel-training image played by the terminal device may be different pictures, and at this time, the first training image of the training video stream at the first playing time and the first heel-training image corresponding to the first playing time in the rendered heel-training video stream are firstly obtained.

S302, when the action matching degree of the first training image and the first follow-up image is lower than a preset threshold, searching a second training image which meets the preset threshold with the action matching degree of the first training image from the training video stream.

After a first training image of a training video stream at a first playing time and a first training image of a rendered training video stream corresponding to the first playing time are obtained, determining the action matching degree of the first training image and the first training image first, and when the action matching degree of the first training image and the first training image is lower than a preset threshold value, characterizing that the training video stream played by a display device is not synchronous with the training video stream, and the reason that the training video stream is not synchronous with the training video stream is that the training video stream is sent to the display device for display after the terminal device renders the training video stream, so that the playing progress of the training video stream is slow compared with the playing progress of the training video stream, and the training video stream of the training image played by the training video stream is already played, therefore, a second training image matched with the action of the first training image can be searched from the training video stream.

When the screen projection method includes step S301 and step S302, a specific embodiment of step S400 includes:

s401, when the difference value between the first playing time and the second playing time is larger than or equal to a preset difference value, a first indication signal is sent to the terminal equipment so as to indicate the terminal equipment to send the unrendered follow-up video stream.

In a specific embodiment, after searching a second training image which meets a preset threshold value with the action matching degree of the first training image from the training video stream, a second playing time corresponding to the second training image can be determined, according to the first playing time of the first training image and the second playing time of the second training image, which playing time the training video stream played by the display device is played to and which playing time the rendered following video stream played by the display device is played to can be determined, according to the relation between the difference value of the first playing time and the second playing time and the preset difference value, the length of the delay time of the rendered following video stream played by the display device and the training video stream is determined, when the difference value of the first playing time and the second playing time is greater than or equal to the preset difference value, at this time, the delay time process between the rendered following video stream played by the display device and the training video stream is represented, and the reason that the delay time is caused between the rendered following video stream played by the display device and the training video stream may be the reason that the terminal device needs to process the following video stream played by the image acquisition module, the following video acquisition module is not processed, the following video stream is not transmitted to the display device by the display device, the display device is controlled by the display device, the display device is not rendered by the display device, and the display device is controlled to be controlled to display device, and the display device is not rendered, and the display device is controlled to receive the training stream is directly is controlled, the problem that the training video stream and the follow-up video stream played by the display device are not synchronous due to rendering operation is avoided.

When the screen projection method includes step S301 and step S302, another specific embodiment of step S400 includes:

and S402, when the difference value between the first playing time and the second playing time is smaller than a preset difference value, not sending a first indication signal to the terminal equipment.

Specifically, as shown in fig. 3C and fig. 3D, when the difference between the first playing time and the second playing time is smaller than the preset difference, the delay between the rendered heel-training video stream and the training video stream played by the display device is represented to be within the receivable range of the user, and the controller of the display device does not need to send the first indication signal to the terminal device.

Fig. 4A is a schematic flow chart of another screen projection method provided by an embodiment of the present disclosure, fig. 4B is a schematic signal flow chart of another screen projection method provided by an embodiment of the present disclosure, and on the basis of the foregoing embodiment, the screen projection method further includes, in combination with fig. 4A and fig. 4B:

s500, receiving the unrendered follow-up video stream sent by the terminal equipment.

When the first training image corresponding to the first playing time in the training video stream is not matched with the first training image corresponding to the first playing time in the rendered follow-up video stream, at this time, a delay exists between the rendered follow-up video stream played by the display device and the training video stream, and the reason for the delay between the rendered follow-up video stream played by the display device and the training video stream may be that the terminal device needs to perform rendering processing on the follow-up video stream acquired by the image acquisition module, so that delay of transmitting follow-up video stream data to the display device by the terminal device is caused, in order to ensure synchronization of the follow-up video stream played by the display device and the training video stream, at this time, the controller of the display device transmits a first indication signal to the terminal device, instructs the terminal device to transmit an unrendered follow-up video stream, and the controller of the display device receives the unrendered follow-up video stream transmitted by the terminal device.

S600, determining a second training image which is matched with the first training image action of the training video stream in the unrendered training video stream.

In the initial state, the controller receives the rendered heel-training video stream sent by the terminal device, when the actions of the first training image of the training video stream displayed by the display and the first heel-training image of the heel-training video stream are not matched, in order to ensure that the training video stream displayed by the display and the heel-training video stream are synchronously played, the controller firstly sends a first indication signal to the terminal device to indicate the terminal device to send the unrendered heel-training video stream, but because the playing progress of the rendered heel-training video stream played by the display is slower than that of the training video stream, after the unrendered heel-training video stream is obtained, if the controller directly performs rendering operation on the unrendered heel-training video stream from the second playing time and plays the heel-training video stream after the rendering operation from the second playing time, at this moment, the heel-training video stream played by the display and the training video stream are still asynchronous. In the embodiment of the present disclosure, after receiving the heel training video stream sent by the terminal device, a second heel training image that is matched with the first training image action of the training video stream is first determined in the unrendered heel training video stream.

And S700, after rendering operation is carried out on the unrendered heel-exercise video stream, displaying the rendered heel-exercise video stream in a local window of the display, and playing the heel-exercise video stream displayed in the local window from the playing moment corresponding to the second heel-exercise image.

After determining a second heel-training image which is matched with the first training image action of the training video stream from the unrendered heel-training video stream, rendering the unrendered heel-training video stream, displaying the rendered heel-training video stream in a local window of a display, and playing the heel-training video stream displayed in the local window from the playing moment corresponding to the second heel-training image.

However, in order to ensure the use feeling of the user, the controller of the display device performs rendering operation on the unrendered heel training video stream and then plays the unrendered heel training video stream on the display device, so that the unrendered heel training video stream is prevented from being directly played on the display device, and the use feeling of the user is reduced, namely, the heel training video stream played on the display device is the rendered heel training video stream, and the unrendered heel training video stream is played on the receiving terminal device.

As an implementation manner, when the second training image exists in the training video stream, a first indication signal is sent to the terminal device so as to instruct the terminal device to send the unrendered heel training video stream, and the first indication signal is further used for instructing the terminal device to pause the rendering operation.

In order to ensure synchronization of the follow-up video stream and the training video stream played by the display device, the controller of the display device acquires the follow-up video stream by directly receiving the image acquisition module of the terminal device, so that the problem that the training video stream and the follow-up video stream played by the display device are not synchronous due to rendering operation is avoided, and at the moment, a first indication signal which can be sent by the controller of the display device is also used for indicating the terminal device to pause rendering operation, and power consumption of the terminal device is reduced.

The embodiment of the disclosure also provides a display device, which includes:

a display;

a controller configured to:

responding to the received starting instruction, triggering a display to display a heel training interface, wherein the heel training interface is provided with a display window and a local window, and the display window is used for displaying a training video stream;

acquiring a training video stream corresponding to a heel-exercise interface from a server, and receiving a rendered heel-exercise video stream sent by a terminal device, wherein the rendered heel-exercise video stream is used for being directly displayed on a local window without being rendered by a display device;

when a first training image corresponding to a first playing time in the training video stream is not matched with a first training image action corresponding to the first playing time in the rendered training video stream, searching a second training image matched with the first training image action in the training video stream, wherein the first playing time is larger than a second playing time corresponding to the second training image;

And when the second training image exists in the training video stream, sending a first indication signal to the terminal equipment so as to indicate the terminal equipment to send the unrendered follow-up training video stream.

As an alternative embodiment, the controller is further configured to:

receiving an unrendered heel training video stream sent by a terminal device;

determining a second training image matched with the first training image action of the training video stream in the unrendered training video stream;

and after rendering the unrendered heel-exercise video stream, displaying the rendered heel-exercise video stream in a local window of the display, and playing the heel-exercise video stream displayed in the local window from the playing moment corresponding to the second heel-exercise image.

As an alternative embodiment, the controller is further configured to:

and when the difference value between the first playing time and the second playing time is smaller than the preset difference value, not sending the first indication signal to the terminal equipment.

The present disclosure also provides an electronic device, including: and a processor for executing a computer program stored in a memory, which when executed by the processor implements the steps of the method embodiments described above.

Fig. 5 is a schematic structural diagram of an electronic device provided in the present disclosure, and fig. 5 shows a block diagram of an exemplary electronic device suitable for implementing the embodiment of the present invention. The electronic device shown in fig. 5 is only an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

As shown in fig. 5, the electronic device 800 is embodied in the form of a general purpose computing device. Components of electronic device 800 may include, but are not limited to: one or more processors 810, a system memory 820, and a bus 830 that connects the different system components (including the system memory 820 and the processors).

Bus 830 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Electronic device 800 typically includes a variety of computer system readable media. Such media can be any medium that is accessible by electronic device 800 and includes both volatile and non-volatile media, removable and non-removable media.

The system memory 820 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 840 and/or cache memory 850. Electronic device 800 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, the storage system 860 may be used to read from and write to non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Disk drives for reading from and writing to removable nonvolatile magnetic disks (e.g., a "floppy disk"), and optical disk drives for reading from and writing to removable nonvolatile optical disks (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 830 through one or more data medium interfaces. The system memory 820 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of embodiments of the invention.

A program/utility 880 having a set (at least one) of program modules 870 may be stored, for example, in system memory 820, such program modules 870 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 870 generally perform the functions and/or methodologies of the embodiments described herein.

The processor 810 executes various functional applications and information processing, such as implementing method embodiments provided by embodiments of the present invention, by running at least one program of a plurality of programs stored in the system memory 820.

The present disclosure also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the above-described method embodiments.

Any combination of one or more computer readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The present disclosure also provides a computer program product which, when run on a computer, causes the computer to perform the steps of implementing the method embodiments described above.

It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A screen projection method, comprising:

responding to a received starting instruction, triggering a display to display a heel training interface, wherein the heel training interface is provided with a display window and a local window, and the display window is used for displaying a training video stream;

acquiring a training video stream corresponding to the heel training interface from a server, and receiving a rendered heel training video stream sent by a terminal device, wherein the rendered heel training video stream is used for being directly displayed on the local window without being rendered by a display device;

searching a second training image matched with the first training image action in the training video stream when the first training image corresponding to the first playing time in the training video stream is not matched with the first training image action corresponding to the first playing time in the rendered training video stream, wherein the first playing time is larger than a second playing time corresponding to the second training image;

and when the second training image exists in the training video stream, sending a first indication signal to the terminal equipment so as to indicate the terminal equipment to send an unrendered follow-up video stream.

2. The method of claim 1, wherein the searching for a second training image in the training video stream that matches the first training image action when the first training image in the training video stream that corresponds to a first play time does not match the first training image action in the rendered follow-up video stream that corresponds to the first play time comprises:

Acquiring a first training image of the training video stream at a first playing time and a first heel-in image corresponding to the first playing time in the rendered heel-in video stream;

when the action matching degree of the first training image and the first training image is lower than a preset threshold, searching a second training image which meets the preset threshold with the action matching degree of the first training image from the training video stream;

and when the second training image exists in the training video stream, sending a first indication signal to the terminal equipment, wherein the method comprises the following steps:

and when the difference value between the first playing time and the second playing time is larger than or equal to a preset difference value, a first indicating signal is sent to the terminal equipment so as to indicate the terminal equipment to send the unrendered follow-up video stream.

3. The method of claim 1, wherein after sending the first indication signal to the terminal device, the method further comprises:

receiving an unrendered heel-hold video stream sent by the terminal equipment;

determining a second training image which is matched with the first training image action of the training video stream in the unrendered training video stream;

And after rendering operation is carried out on the unrendered heel-exercise video stream, displaying the rendered heel-exercise video stream in a local window of the display, and playing the heel-exercise video stream displayed in the local window from the playing time corresponding to the second heel-exercise image.

4. The method according to claim 2, wherein the method further comprises:

and when the difference value between the first playing time and the second playing time is smaller than a preset difference value, not sending a first indication signal to the terminal equipment.

5. The method of claim 1, wherein the first indication signal is further used to instruct the terminal device to pause rendering operations.

6. The method of claim 1, wherein the searching the training video stream for a second training image that matches the first training image action comprises:

in the training video stream, taking a first playing time as a starting point, acquiring a candidate training image of a preset time before the first playing time of the training video stream, and sending the candidate training image and the first training image to a server;

obtaining a comparison result of the candidate training image and the first training image by the server;

And determining the candidate training image, of which the matching degree with the first training image meets a preset threshold value, as the second training image.

7. The method of claim 6, wherein the obtaining the comparison of the candidate training image and the first training image by the server comprises:

and acquiring a feature point comparison result of the server on the first image feature point of the candidate training image and the second image feature point of the first training image.

8. A display device, the display device comprising:

a display;

a controller configured to:

responding to a received starting instruction, triggering a display to display a heel training interface, wherein the heel training interface is provided with a display window and a local window, and the display window is used for displaying a training video stream;

acquiring a training video stream corresponding to the heel training interface from a server, and receiving a rendered heel training video stream sent by a terminal device, wherein the rendered heel training video stream is used for being directly displayed on the local window without being rendered by a display device;

searching a second training image matched with the first training image action in the training video stream when the first training image corresponding to the first playing time in the training video stream is not matched with the first training image action corresponding to the first playing time in the rendered training video stream, wherein the first playing time is larger than a second playing time corresponding to the second training image;

And when the second training image exists in the training video stream, sending a first indication signal to the terminal equipment so as to indicate the terminal equipment to send an unrendered follow-up video stream.

9. The apparatus of claim 8, wherein the device comprises a plurality of sensors,

the controller is further configured to:

receiving an unrendered heel-hold video stream sent by the terminal equipment;

determining a second training image which is matched with the first training image action of the training video stream in the unrendered training video stream;

and after rendering operation is carried out on the unrendered heel-exercise video stream, displaying the rendered heel-exercise video stream in a local window of the display, and playing the heel-exercise video stream displayed in the local window from the playing time corresponding to the second heel-exercise image.

10. The apparatus of claim 8, wherein the device comprises a plurality of sensors,

the controller is further configured to:

and when the difference value between the first playing time and the second playing time is smaller than a preset difference value, not sending a first indication signal to the terminal equipment.