CN111949237A

CN111949237A - Image display method and device

Info

Publication number: CN111949237A
Application number: CN202010699375.7A
Authority: CN
Inventors: 马权; 范志刚; 卢涛
Original assignee: Xian Wanxiang Electronics Technology Co Ltd
Current assignee: Xian Wanxiang Electronics Technology Co Ltd
Priority date: 2020-07-20
Filing date: 2020-07-20
Publication date: 2020-11-17

Abstract

The invention provides an image display method and device, relates to the technical field of image processing, and can solve the problem that pictures are torn due to the fact that sub-images decoded by a decoding end device are not synchronous when being displayed by a large-screen splicing device. The specific technical scheme is as follows: acquiring at least one current frame image to be displayed, first identification information of an image source of the current frame image to be displayed and frame identification information of the current frame image to be displayed; dividing each current frame image to be displayed to generate at least one current sub-image; performing time synchronization on decoding end equipment of at least one display by adopting reference time; generating a timestamp for each current sub-image; and sending the current sub-image, the timestamp, the first identification information and the frame identification information to a target decoding end device, so that the target decoding end device sends the current sub-image to a target display according to the timestamp, the reference time and the frame identification information. The invention is used for realizing the synchronization of the pictures and improving the image impression experience of the user.

Description

Image display method and device

Technical Field

The present disclosure relates to the field of image processing, and in particular, to an image display method and apparatus.

Background

In the prior art, an external large screen system is usually adopted to perform split screen display on an image source. The external large-screen system comprises an image source, a server such as an image processing server, decoding end equipment and external large-screen equipment. The external large-screen device can be formed by splicing a plurality of displays, and each display is called as a split screen. A large screen can be formed by a plurality of split screens, so that the external large screen equipment can be divided into at least one large screen. Each split screen is connected with the decoding end equipment through a video line; each decoding end device is connected with the image processing server through a network cable.

The process of split-screen display of the image source by adopting the external large-screen system comprises the following steps: the image processing server collects images of at least one image source, divides each image source to generate a plurality of sub-images, codes each sub-image to generate coded sub-images, respectively sends each coded sub-image to the decoding end equipment, and sends the coded sub-images to the corresponding display for display after the coded sub-images are decoded by the decoding end equipment. The image processing server divides an image of an image source acquired at the same time to generate a plurality of sub-images, and the plurality of sub-images form a logically complete image on the whole, under the condition, the display effect of the spliced large screen is directly influenced by the synchronism of each sub-image in the display process, however, due to the problems of network transmission delay, processor scheduling in the image processing server and the like, the sub-images decoded by the decoding end equipment are not synchronous in display, and the situation of image tearing is caused.

Disclosure of Invention

The embodiment of the disclosure provides an image display method and an image display device, which can solve the problem that in the prior art, when a sub-image decoded by a large-screen splicing device is displayed asynchronously, a picture is torn. The technical scheme is as follows:

according to a first aspect of embodiments of the present disclosure, there is provided an image display method including:

acquiring at least one current frame image to be displayed, first identification information of an image source of the current frame image to be displayed and frame identification information of the current frame image to be displayed;

dividing each current frame image to be displayed to generate at least one current sub-image;

performing time synchronization on decoding end equipment of at least one display by adopting reference time;

generating a timestamp for each of the current sub-images;

and sending the current sub-image, the timestamp, the first identification information and the frame identification information to a target decoding end device, so that the target decoding end device sends the current sub-image to the target display according to the timestamp, the reference time and the frame identification information, and the target decoding end device is a decoding end device of the target display corresponding to an image source of the frame image to be displayed.

The image display method provided by the embodiment of the disclosure can acquire at least one current frame image to be displayed, first identification information of an image source of the current frame image to be displayed and frame identification information of the current frame image to be displayed; dividing each current frame image to be displayed to generate at least one current sub-image; performing clock synchronization on decoding end equipment of at least one display by adopting reference time; time stamp of each current sub-picture; the current sub-image, the timestamp, the first identification information and the frame identification information are sent to a target decoding end device, so that the target decoding end device sends the current sub-image to the target display according to the timestamp, the reference time and the frame identification information, the target decoding end device is a decoding end device of the target display corresponding to an image source of the frame image to be displayed, clock synchronization can be carried out on the decoding end device of at least one display, the phenomenon that the sub-images decoded by a large-screen splicing device in the prior art are not synchronous when being displayed is avoided, and therefore the situation of image tearing is caused, and image viewing experience of a user is improved.

In one embodiment, before generating the timestamp of each sub-image, the method further comprises:

encoding the current sub-image to generate an encoded current sub-image;

the sending the current sub-image and the timestamp to a target decoding end device comprises:

and sending the encoded current sub-image and the timestamp to a target decoding end device.

By encoding the current sub-image and sending the encoded current sub-image to the target decoding terminal device, the data volume of the current sub-image in the transmission process can be reduced, and meanwhile, the safety of the current sub-image in the transmission process is improved.

In one embodiment, the method further comprises:

receiving an image display instruction sent by user equipment, wherein the image display instruction comprises first identification information of an image source of at least one frame image to be displayed currently and second identification information of a display to which the frame image to be displayed currently is to be displayed;

and determining a target display corresponding to the image source of each frame image to be displayed currently according to the image instruction.

By receiving an image display instruction sent by user equipment, a target display corresponding to each image source of the current frame image to be displayed can be determined according to the image instruction, and then the current frame image to be displayed is displayed through the target display.

According to a second aspect of the embodiments of the present disclosure, there is provided an image display method including:

receiving at least one current sub-image sent by a server, a timestamp of the current sub-image and frame identification information of the current frame image to be displayed, wherein the current sub-image is generated after the server divides the current frame image to be displayed;

if the time difference between the timestamp of the current sub-image and the current time is determined to be less than or equal to the preset time threshold, receiving frame identification information of other images to be displayed broadcasted by other decoding end equipment, wherein the other decoding end equipment is decoding end equipment except the target decoding end equipment;

determining target frame identification information according to the frame identification information of the current frame image to be displayed and the frame identification information of other frame images to be displayed;

and if the frame identification information of the frame image to be displayed is determined to be consistent with the target frame identification information, sending the current sub-image to the target display so that the target display can display the current sub-image.

The image display method provided by the embodiment of the disclosure can receive at least one current sub-image sent by a server, a timestamp of the current sub-image and frame identification information of a frame image to be displayed currently, wherein the current sub-image is generated after the frame image to be displayed currently is divided by the server; if the time difference between the timestamp of the current sub-image and the current time is less than or equal to the preset time threshold, receiving frame identification information of other images to be displayed broadcasted by other decoding end equipment; determining target frame identification information according to the frame identification information of the current frame image to be displayed and the frame identification information of other frame images to be displayed; if the frame identification information of the frame image to be displayed is determined to be consistent with the target frame identification information, the current sub-image is sent to the target display, so that the target display can display the current sub-image, the synchronization of the image can be realized, the condition that the image is torn due to the fact that the sub-images decoded by the decoding end device are not synchronous when the large-screen splicing device is used for displaying the sub-images in the prior art is avoided, and the image viewing experience of a user is improved.

In one embodiment, the frame identification information is a frame sequence number, and the method further includes:

if the frame identification information of the current frame image to be displayed is inconsistent with the target frame identification information and the frame number of the current frame image to be displayed is smaller than the target frame identification information, discarding the current subimage and receiving a next frame subimage sent by the server and the frame number of the next frame image to be displayed, wherein the next frame subimage is generated after the server divides the next frame image to be displayed;

and if the frame number of the next frame image to be displayed is consistent with the target frame identification information, sending the next frame sub-image to a target display so that the target display can display the next frame sub-image.

When the frame identification information of the current frame image to be displayed is inconsistent with the target frame identification information and the frame number of the current frame image to be displayed is less than the target frame identification information, the current subimage is discarded, so that the subimages of different frames can be prevented from being displayed on different displays at the same time, the situation of tearing the image frame is avoided, and the impression experience of a user is improved. When the current subimage is discarded and the next frame subimage sent by the server and the frame number of the next frame image to be displayed are received, and the frame number of the next frame image to be displayed is consistent with the target frame identification information, the next frame subimage is sent to the target display, so that all displays can display the subimage of the same frame image, the situation of tearing of the image frame is avoided, and the viewing experience of a user is improved.

In one embodiment, determining the target frame identification information according to the frame identification information of the current frame image to be displayed and the frame identification information of other frame images to be displayed includes:

determining the number of the same frame number in the frame number of the current frame image to be displayed and the frame numbers of other frame images to be displayed;

and taking the frame sequence number with the largest number as the identification information of the target frame.

The frame number of the sub-image to be displayed by each display can be determined by determining the number of the same frame number in the frame number of the current frame image to be displayed and the frame numbers of other frame images to be displayed and taking the frame number with the largest number as the target frame identification information.

In one embodiment, the method further comprises:

if the frame number of the current frame image to be displayed is inconsistent with the target frame identification information and the frame number of the current frame image to be displayed is greater than the target frame identification information, receiving the frame numbers of the other frame images to be displayed broadcasted by the other decoding end equipment;

and if the frame number of the current frame image to be displayed is consistent with the frame numbers of the other frame images to be displayed, sending the current sub-image to a target display so that the target display can display the current frame sub-image.

When the frame identification information of the current frame image to be displayed is inconsistent with the target frame identification information and the frame number of the current frame image to be displayed is greater than the target frame identification information, the frame numbers of other frame images to be displayed broadcasted by other decoding end equipment are received, and when the frame number of the current frame image to be displayed is consistent with the frame numbers of other frame images to be displayed, the current sub-images are sent to the target display, so that all the displays can display the sub-images of the same frame image, the situation of tearing of the image frame is avoided, and the viewing experience of a user is improved.

In an embodiment, the current sub-image is an encoded current sub-image, and after receiving the current sub-image sent by the server, the timestamp of the current sub-image, and the frame identification information of the current frame image to be displayed, the method further includes:

decoding the current sub-image to generate a decoded current sub-image;

the sending the current sub-image to the target display comprises:

and sending the decoded current sub-image to the target display.

The current sub-image can be restored by decoding the sub-image to generate a decoded current sub-image.

According to a third aspect of the embodiments of the present disclosure, there is provided an image display device including:

the device comprises a current frame image to be displayed acquisition module, a frame display module and a frame display module, wherein the current frame image to be displayed acquisition module is used for acquiring at least one current frame image to be displayed, first identification information of an image source of the current frame image to be displayed and frame identification information of the current frame image to be displayed;

the current frame image to be displayed dividing module is used for dividing each current frame image to be displayed to generate at least one current sub-image;

the time synchronization module is used for carrying out time synchronization on the decoding end equipment of at least one display by adopting reference time;

the timestamp generation module is used for generating a timestamp of each current sub-image;

and the current sub-image sending module is used for sending the current sub-image, the timestamp, the first identification information and the frame identification information to a target decoding end device, so that the target decoding end device sends the current sub-image to the target display according to the timestamp and the reference time, and the target decoding end device is a decoding end device of the target display corresponding to an image source of the frame image to be displayed.

In one embodiment, the apparatus further comprises:

the subimage coding module is used for coding the current subimage to generate a coded current subimage;

the current sub-image sending module is configured to:

In one embodiment, the apparatus further comprises:

a target display determination module to:

According to a fourth aspect of the embodiments of the present disclosure, there is provided an image display device including:

the current sub-image receiving module is used for receiving at least one current sub-image sent by a server, a timestamp of the current sub-image and frame identification information of the current frame image to be displayed, wherein the current sub-image is generated after the server divides the current frame image to be displayed;

a frame identification information receiving module, configured to receive frame identification information of other frame images to be displayed, which are broadcast by other decoding end devices, if it is determined that a time difference between the timestamp of the current sub-image and the current time is smaller than or equal to a preset time threshold, where the other decoding end devices are decoding end devices other than the target decoding end device;

the target frame identification information determining module is used for determining target frame identification information according to the frame identification information of the current frame image to be displayed and the frame identification information of other frame images to be displayed;

and the current sub-image sending module is used for sending the current sub-image to the target display if the frame identification information of the frame image to be displayed is determined to be consistent with the target frame identification information, so that the target display can display the current sub-image.

In one embodiment, the frame identification information is a frame sequence number, and the apparatus further includes:

a next frame sub-picture transmission module configured to:

In one embodiment, the target frame identification information determination module is configured to:

In one embodiment, the current sub-image sending module is further configured to:

In one embodiment, the current sub-image is an encoded current sub-image, the apparatus further comprising:

the current sub-image decoding module is used for decoding the current sub-image to generate a decoded current sub-image;

the current sub-image sending module is configured to:

and sending the decoded current sub-image to the target display.

According to a fifth aspect of the embodiments of the present disclosure, there is provided a server, the server comprising a processor and a memory, the memory having at least one computer instruction stored therein, the instruction being loaded and executed by the processor to implement the steps performed in the image display method of the first aspect and any of the embodiments of the first aspect.

According to a sixth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, in which at least one computer instruction is stored, the instruction being loaded and executed by a processor to implement the steps performed in the image display method according to the first aspect and any one of the embodiments of the first aspect.

According to a seventh aspect of the embodiments of the present disclosure, there is provided a decoding-side device, which includes a processor and a memory, where the memory stores at least one computer instruction, and the instruction is loaded and executed by the processor to implement the steps performed in the image display method according to the second aspect and any one of the embodiments of the second aspect.

According to an eighth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, in which at least one computer instruction is stored, the instruction being loaded and executed by a processor to implement the steps performed in the image display method according to the second aspect and any one of the embodiments of the second aspect.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

Fig. 1 is a structural diagram of an external large screen system provided in an embodiment of the present disclosure;

fig. 2 is a first flowchart of an image display method provided by an embodiment of the present disclosure;

fig. 3 is a flowchart ii of an image display method provided in the embodiment of the present disclosure;

fig. 4 is a flowchart three of an image display method provided by the embodiment of the present disclosure;

FIG. 5 is a flow chart of a pre-inspection method provided by an embodiment of the present disclosure;

fig. 6 is a first structural diagram of an image display device according to an embodiment of the disclosure;

fig. 7 is a second structural diagram of an image display device according to an embodiment of the disclosure;

fig. 8 is a third structural diagram of an image display device according to an embodiment of the present disclosure;

fig. 9 is a fourth structural diagram of an image display device according to an embodiment of the present disclosure;

FIG. 10 is a hardware block diagram of a server provided by an embodiment of the present disclosure;

fig. 11 is a hardware structure diagram of a decoding-side device according to an embodiment of the present disclosure.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

Fig. 1 is a structural diagram of an external large screen system provided in the prior art. As shown in fig. 1, the system includes:

at least one image source 101, a server, such as an image processing server 102, at least one decoding end device 103, an external large screen device 104, and a user device 105.

The external large-screen device 104 includes a plurality of displays, and the external large-screen device 104 is formed by splicing the plurality of displays, and each display is called a split screen. For example, as shown in fig. 2, the external large-screen device 104 includes eight split screens a to H. The external large-screen equipment can be divided into at least one large screen. Each split screen is connected with the decoding end device 103 through a video line; each decoding-end device 103 is connected to the image processing server 101 through a network cable; the user device 105 is connected to the image processing server 102, and a user who is externally connected to the large screen system can control the image processing server 102 on the user device 105.

The process of split-screen display of the image source by adopting the external large-screen system comprises the following steps: the image processing server 102 receives an image display instruction input by a user on the user equipment 105, wherein the image display instruction comprises first identification information of an image source of at least one frame image to be displayed and second identification information of a display to which the frame image to be displayed is to be displayed; after receiving the image display instruction, the image processing server 102 determines a target display corresponding to the image source of each frame image to be currently displayed according to the image display instruction.

Further, after the image processing server 102 collects the image of at least one image source 101, the image of each image source is divided to generate a plurality of sub-images, each sub-image is encoded to generate encoded sub-images, each encoded sub-image is sent to a target decoding end device, and the encoded sub-images are sent to a target display corresponding to the image source to be displayed after being decoded by the target decoding end device.

In this case, the display effect of the image source in the large spliced screen is directly affected by the synchronism of each subimage of the same image source in the display process, however, due to problems such as network transmission delay and processor scheduling in the image processing server, the subimages decoded by the large spliced screen device are not synchronized frequently, so that the image frame of the image source is torn.

The inventor has noticed this problem and proposes an image display method, specifically as follows:

fig. 2 is a flowchart of a first image display method provided by an embodiment of the present disclosure, and the method is applied to a server, for example, an image processing server. As shown in fig. 2, the method includes:

s201, acquiring at least one current frame image to be displayed, first identification information of an image source of the current frame image to be displayed and frame identification information of the current frame image to be displayed;

s202, dividing each frame image to be displayed currently to generate at least one current sub-image.

Illustratively, the image processing server may receive an image display instruction sent by the user equipment, where the image display instruction includes first identification information of an image source of at least one frame image to be currently displayed and second identification information of a display to which the frame image to be currently displayed is to be displayed;

further, after at least one current frame image to be displayed, the first identification information of the image source of the current frame image to be displayed and the frame identification information of the current frame image to be displayed are obtained, the image processing server determines a target display corresponding to the image source of each current frame image to be displayed according to the image instruction. For example, as shown in fig. 2, if the user wants to display the currently to-be-displayed frame image of the image source 1011 to A, B, C, D four displays, the user may input an image display instruction on the user device, where the image display instruction includes at least first identification information of the image source 1011 of the currently to-be-displayed frame image and second identification information of the currently to-be-displayed frame image to be displayed to A, B, C, D four displays, and the image display instruction is sent to the image processing server by the user device.

After receiving the image display instruction, the image processing server obtains a current frame image to be displayed of the image source 1011, first identification information of the image source 1011 of the current frame image to be displayed, and frame identification information of the current frame image to be displayed, and determines that a target display corresponding to the current frame image to be displayed is A, B, C, D four displays according to the image display instruction and the first identification information of the image source 1011 of the current frame image to be displayed.

Further, the image processing server divides the current frame image to be displayed into four current sub-images, and the four sub-images can be respectively displayed on A, B, C, D four displays.

Illustratively, after the frame image to be displayed is divided into four current sub-images, each current sub-image is encoded to generate an encoded current sub-image.

And S203, performing clock synchronization on the decoding end equipment of at least one display by adopting the reference time.

In one embodiment, the image processing server obtains a reference time from a Network Time Protocol (NTP) server and transmits the reference time to the decoding-side device of the at least one display, and the clock information of the image processing server is updated to the reference time. And after receiving the reference time, the decoding end device of the at least one display sends a first confirmation message to the image processing server, and meanwhile, the decoding end device of each display updates the clock information of the decoding end device of each display to the reference time, wherein the first confirmation message is used for confirming the reception of the reference time.

In another embodiment, the image processing server acquires reference time from the NTP server, and calibrates the current time of the image processing server by using the reference time; and after acquiring the reference time, the image processing server sends a time synchronization request to the decoding end equipment of the at least one display. When the decoding end equipment of at least one display receives the time synchronization request, a second confirmation message is returned to the image processing server, meanwhile, the decoding end equipment of at least one display acquires the reference time from the NTP server, the reference time is adopted to calibrate the current time of the decoding end equipment of each display, and the second confirmation message is used for confirming the receiving of the time synchronization request.

And S204, generating a time stamp of each current sub-image.

In this step, the image processing server encodes each current sub-image, generates an encoded current sub-image, and generates a timestamp for each encoded current sub-image.

S205, sending the current sub-image, the timestamp, the first identification information and the frame identification information to a target decoding end device, so that the target decoding end device sends the current sub-image to the target display according to the timestamp, the reference time and the frame identification information, wherein the target decoding end device is a decoding end device of the target display corresponding to an image source of the frame image to be displayed.

In this step, the image processing server sends each encoded current sub-image and the timestamp of the encoded current sub-image to the target decoding end device, so that the target decoding end device decodes the encoded current sub-image according to the timestamp, the reference time, and the frame identification information, and sends the decoded current sub-image to the target display. For example, the image processing server sends 4 encoded sub-images from the image source 1011 to four target decoding-side devices connected to A, B, C, D four target displays, respectively.

How the target display processes the sub-images is described below with reference to the embodiment of fig. 3. Fig. 3 is a flowchart ii of an image display method provided in an embodiment of the present disclosure, where the method is applied to a target decoding-side device. As shown in fig. 3, the method includes:

s301, receiving at least one current sub-image sent by a server, a timestamp of the current sub-image and frame identification information of the current frame image to be displayed, wherein the current sub-image is generated after the server divides the current frame image to be displayed.

Illustratively, the at least one current sub-image sent by the server, e.g. the image processing server, is the encoded current sub-image. And after receiving the at least current sub-image, the target display decodes each current sub-image to generate a decoded current sub-image.

S302, if the time difference between the time stamp of the current sub-image and the current time is determined to be smaller than or equal to a preset time threshold, receiving frame identification information of other images to be displayed broadcasted by other decoding end equipment, wherein the other decoding end equipment is decoding end equipment except the target decoding end equipment;

s303, determining target frame identification information according to the frame identification information of the current frame image to be displayed and the frame identification information of other frame images to be displayed.

Illustratively, the frame identification information is a frame sequence number. And if the time difference between the timestamp of the current sub-image and the current time is smaller than or equal to the preset time threshold, the target decoding end equipment determines the number of the same frame numbers from the frame number of the current frame image to be displayed and the frame numbers of other frame images to be displayed, and takes the frame number with the largest number as the target frame identification information.

For example, if the frame number of the current frame image to be displayed is 4, and the frame numbers of the other frame images to be displayed are 3, 4, and 4, respectively, then the number of the frame numbers 4 is 3, and the number of the frame numbers 3 is 2, then the target frame identification information is 4.

Illustratively, if it is determined that the time difference between the timestamp of the current sub-image and the current time is greater than the preset time threshold, the target decoding-side device discards the current sub-image and waits for receiving the next sub-image of the frame sent by the image processing server.

S304, if the frame identification information of the frame image to be displayed is determined to be consistent with the target frame identification information, the current sub-image is sent to the target display, so that the target display can display the current sub-image.

For example, if the frame number of the current frame image to be displayed is 4, the frame numbers of the other frame images to be displayed are 3, 4, and 4, respectively, then the number of the frame numbers 4 is 3, and the number of the frame numbers 3 is 2, then the target frame identification information is 4, and the frame number of the current frame image to be displayed is consistent with the target frame identification information, then the target decoding-side device decodes the encoded current sub-image, and sends the decoded current sub-image to the target display, so that the target display displays the current sub-image.

Exemplarily, if the frame number of the current frame image to be displayed is inconsistent with the target frame identification information and the frame number of the current frame image to be displayed is less than the target frame identification information, discarding the current sub-image and receiving a next sub-image sent by the server and the frame number of the next frame image to be displayed, where the next sub-image is generated after the image processing server divides the next frame image to be displayed; and if the frame number of the next frame image to be displayed is consistent with the target frame identification information, sending the next frame sub-image to a target display so that the target display can display the next frame sub-image.

For example, if the frame number of the current frame image to be displayed is 3, the frame numbers of the other frame images to be displayed are 3, 4, and 4, respectively, then the number of the frame number 4 is 3, and the number of the frame number 3 is 2, then the target frame identification information is 4, and if the frame number of the current frame image to be displayed is not consistent with the target frame identification information and the frame number of the current frame image to be displayed is less than the target frame identification information, then the current sub-image is discarded and the next sub-image sent by the server and the frame number of the next frame image to be displayed are received. And if the frame number of the next frame sub-image is 4, sending the next frame sub-image to a target display so that the target display can display the next frame sub-image.

For another example, if the frame number of the current frame image to be displayed is 2, the frame numbers of the other frame images to be displayed are 3, 4, and 4, respectively, then the number of the frame number 4 is 3, the number of the frame number 3 is 1, the number of the frame number 2 is 1, then the target frame identification information is 4, the frame number of the current frame image to be displayed is not consistent with the target frame identification information, and the frame number of the current frame image to be displayed is less than the target frame identification information, then the current sub-image is discarded, and the next frame sub-image sent by the server and the frame number of the next frame image to be displayed are received. And if the frame number of the next frame of subimage is 3, discarding the next frame of subimage and receiving the subimage to be displayed and the frame number of the frame image to be displayed, which are sent by the server later, until the frame number of the frame image to be displayed is consistent with the target frame identification information, and sending the subimage to be displayed to a target display.

Exemplarily, if the frame number of the current frame image to be displayed is inconsistent with the target frame identification information and the frame number of the frame image to be displayed is greater than the target frame identification information, receiving the frame number of the next frame image to be displayed broadcasted by the other decoding-end device; and if the frame number of the current frame image to be displayed is consistent with the frame numbers of the other frame images to be displayed, sending the current sub-image to a target display so that the target display can display the current sub-image.

For example, if the frame number of the current frame image to be displayed is 5, and the frame numbers of the other frame images to be displayed are 4, and 4, respectively, then the number of the frame number 4 is 4, and the number of the frame number 5 is 1, then the target frame identification information is 4, and if the frame number of the current frame image to be displayed is inconsistent with the target frame identification information and the frame number of the current frame image to be displayed is greater than the target frame identification information, then the frame number 5 of the other frame image to be displayed broadcast by the other decoding-end device is received; and if the frame number 5 of the current frame image to be displayed is consistent with the frame number 5 of the other frame image to be displayed, the target decoding end equipment decodes the encoded current sub-image and sends the encoded current sub-image to a target display so that the target display can display the current sub-image.

The following describes the image display method provided by the embodiment of the present disclosure in further detail with reference to the embodiment of fig. 4. Fig. 4 is a flowchart three of an image display method according to an embodiment of the present disclosure. As shown in fig. 4, the method includes:

s401, after the image processing server is respectively connected with a plurality of decoding end devices for receiving images in a splicing large-screen system, clock synchronization is carried out on all the decoding end devices and the image processing server; meanwhile, all the decoding end devices establish a multicast group;

in order to ensure that the reference time of the server side is consistent with that of the large screen splicing system, clock synchronization needs to be carried out on the equipment with the decoding end and the image processing server.

Specifically, there are many implementation manners for performing clock synchronization on the device having the decoding end and the image processing server, and in an optional embodiment of the present invention, the clock synchronization may be implemented by:

the image processing server acquires a reference time point (which can be acquired from the NTP server);

the image processing server sends the reference time points to a plurality of decoding end devices in the large screen splicing system;

after the image processing server receives the acknowledgement message sent by the plurality of decoding end devices, the image processing server updates the clock information of the image processing server to the reference time point, and meanwhile, the plurality of decoding end devices update the clock information of all the decoding end devices to the reference time point.

In another alternative embodiment, clock synchronization may be achieved by:

the image processing server acquires reference time from the NTP server and calibrates the time of the image processing server based on the acquired time; meanwhile, the image processing server sends time synchronization requests to the plurality of decoding end devices, the plurality of decoding end devices return a confirmation message to the image processing server when receiving the time synchronization requests, and meanwhile, the plurality of decoding end devices acquire reference time from the same NTP server and calibrate the time of all the decoding end devices based on the acquired time.

S402, dividing the acquired image by the image processing server to generate at least one sub-image, encoding each sub-image, and sending a data packet obtained by encoding to decoding end equipment;

in order to achieve the purpose of image synchronization, the image processing server needs to perform special setting on a header (head) of a data code stream obtained by encoding each sub-image, and specifically, a timestamp and an identification number (ID) need to be added to the header; specifically, the timestamp is used for identifying the current time; the ID is used to identify the current frame sequence number and the sequence number of the sub-picture. The numbering rule of the header information in each sub-image is uniform.

For example, the ID of the encoded data of the first sub-image after the third frame image is divided is: 3-1, the ID of the coded data of the second sub-image after the third frame image is divided is: 3-2, and so on.

The above is only an exemplary ID setting manner, and in practical applications, the setting rule of the ID may be adjusted as needed, and the set ID only needs to include the frame number of the current image and the number of the current sub-image.

And the image processing server sends the code stream obtained by coding each subimage to the corresponding decoding end equipment.

S403, after the decoding end equipment receives the coded data sent by the image processing server, analyzing a timestamp and an ID number from the coded data;

s404, the decoding end equipment performs pre-check processing on the coded data according to the analyzed time stamp, and determines whether the current coded data is discarded or sent to a decoding queue for decoding through the pre-check processing;

because the expired coding data has no processing value, in order to prevent the expired coding data from entering a queue to be decoded, the received coding data is pre-checked, whether the received coding data is expired is checked through the pre-check, and the expired coding data is directly discarded; and if the coded data is checked to have no expiration, the coded data is sent to a decoding queue to wait for decoding processing.

The specific implementation manner of the pre-inspection is shown in fig. 5, and fig. 5 is a flowchart of a pre-inspection method provided by the embodiment of the disclosure. As shown in fig. 5, the method includes:

s501, calculating a difference value between the current system time and a timestamp;

s502, judging whether the difference value is larger than a preset time threshold value or not; if the data packet is larger than a preset threshold value, determining that the data packet is an expired data packet and directly discarding the data packet; and if the data packet is not greater than the preset threshold, sending the data packet to a decoding queue for processing.

Specifically, the packet after the pre-check processing is inserted into the tail of the decoding queue to wait for the decoding processing.

Specifically, the preset threshold is a preset time threshold, and the time threshold is determined according to a maximum tolerable time delay value of the system. Due to the existence of network time delay, a time difference exists between the time when the coded data of one sub-image is sent to the decoding end device after the coding of the image processing server is completed and the time when the coded data is received by the decoding end device, and under the condition that other problems are not considered, the time difference is within a theoretical value range. For example, it may be set to 80ms, 90ms, 100ms, etc., and may also be set to other values as needed, where a higher set value indicates a higher tolerance of the system to the delay, and a lower set value indicates a lower tolerance of the system to the delay.

S405, the current decoding end equipment sequentially takes the coded data from the decoding queue for decoding; before the current decoding end equipment decodes the current coded data, the current decoding end equipment informs the decoding end equipment of a frame number contained in the current data to be decoded through multicast information; similarly, the decoding-end device also needs to notify the frame number contained in the current data to be decoded to other decoding-end devices in a multicast message manner;

in actual implementation, the current decoding end device can directly send the ID number in the data packet to other decoding end devices through a multicast message, and the other decoding end devices identify the current frame number from the received ID number; in order to save data amount, the current decoding end device may also directly extract the current frame number from the ID number, and notify the current frame number to other decoding end devices through a multicast message.

And S406, all the decoding end devices perform synchronous processing according to the received frame sequence numbers.

Through step S405, all decoding-end devices can obtain the frame number of the current image frame to be decoded; in step S406, synchronization processing is performed according to the following rule based on the acquired frame number:

1) if the frame numbers of the current image frames to be decoded of all the decoding end equipment are the same, all the decoding end equipment decodes the current image frames to be decoded and displays the decoded image frames;

2) if the frame numbers of the image frames to be decoded currently by all the decoding-end devices are different, and more than half (more than half) of the frame numbers are the same, the other decoding-end devices perform processing according to the principle of 'majority by minority', specifically perform processing according to the following mode:

if the frame number to be decoded is lagged behind the current same frame number, discarding the current frame until the frame to be decoded with the same frame number as the current frame number is obtained, and decoding the current frame together with other decoding end equipment;

if the frame number to be decoded is ahead of the current same frame number, waiting for a decoding period, namely, not decoding in the current decoding period; after the decoding of the r ends which do not perform decoding in the current decoding period is finished by other r ends, the frame sequence numbers of the respective image frames to be decoded are continuously sent through the multicast message in the next decoding period, and the processing is performed according to the comparison result of the frame sequence numbers and the rule;

3) if the frame numbers of the current image frames to be decoded of all the decoding end devices are different, and more than half of the same frame numbers do not exist in all the frame numbers, processing is carried out according to the following mode:

the processing is preferentially performed according to the principle of slow-to-fast, that is, all decoding end devices with lagging frame numbers perform image frame discarding until the frame number of the frame to be decoded is adjusted to be the same as the maximum frame number, and all r ends start decoding.

The above rules are explained below by the following specific examples:

example one:

assuming that there are 4 decoding end devices r1-r4, the frame numbers of the frames to be decoded sent by the multicast message are: 3. 4, 4 and 4.

Because the frame numbers to be decoded of the decoding end devices are not all the same, and more than half of the frame numbers (4) are the same, the following processing is carried out:

r1 discards the current frame to be decoded, and takes out the frame 4 to be decoded; thereafter, r1, r2, r3 decode the 4 th frame acquired respectively at the same time.

Example two:

assuming that there are 4 decoding end devices r1-r4, the frame numbers of the frames to be decoded sent by the multicast message are: 5. 4, 4 and 4.

r1 waits for a decoding period, i.e., no decoding is performed in the current decoding period; r1 discards the current frame to be decoded, and takes out the frame 4 to be decoded; then, r1, r2 and r3 decode the 4 th frame acquired respectively at the same time;

after r1, r2 and r3 are decoded, the next frame to be decoded (frame number 5) is taken out and notified to other decoding end devices, and the current frame to be encoded of r5 is still 5, so that the frame number 5 with the encoded frame is also notified to other decoding end devices, and thus, the frame numbers with the encoded frame of all the decoding end devices are the same and decoding is directly performed.

Example three:

assuming that the 4 decoding end devices r1-r4 have the frame sequence numbers of the frames to be decoded sent by the multicast message as follows: 3. 3, 4 and 4.

Because the frame numbers to be decoded by the decoding end devices are not all the same and do not exceed half of the frame numbers, according to the principle of 'with slow to fast', the decoding end devices with frame numbers lagging relative to the frame numbers, namely r1 and r2, discard the frame numbers, continue to fetch the image frame 4, inform the respective frame numbers through multicast messages, determine that the frame numbers of the 4 decoding end devices are all the same, and decode the 4 th frame obtained by the decoding end devices respectively.

Fig. 6 is a first structural diagram of an image display device provided in an embodiment of the present disclosure, where the device is applied to a server, such as an image processing server. As shown in fig. 6, the apparatus 60 includes:

a current frame image to be displayed obtaining module 601, configured to obtain at least one current frame image to be displayed, first identification information of an image source of the current frame image to be displayed, and frame identification information of the current frame image to be displayed;

a current to-be-displayed frame image dividing module 602, configured to divide each current to-be-displayed frame image to generate at least one current sub-image;

a time synchronization module 603, configured to perform time synchronization on a decoding-end device of at least one display by using a reference time;

a timestamp generation module 604, configured to generate a timestamp for each of the current sub-images;

a current sub-image sending module 605, configured to send the current sub-image, the timestamp, the first identification information, and the frame identification information to a target decoding end device, so that the target decoding end device sends the current sub-image to the target display according to the timestamp and the reference time, where the target decoding end device is a decoding end device of the target display corresponding to an image source of the frame image to be displayed.

In one embodiment, as shown in fig. 7, the apparatus 60 further comprises:

a sub-image encoding module 606, configured to encode the current sub-image to generate an encoded current sub-image;

the current sub-image sending module 605 is configured to:

In one embodiment, the apparatus 60 further comprises:

a target display determination module 607 for:

The implementation process and technical effects of the image display device provided in the embodiment of the present disclosure can be seen in the embodiments of fig. 2 and fig. 4, which are not described herein again.

Fig. 8 is a first structural diagram of an image display apparatus provided in an embodiment of the present disclosure, where the apparatus is applied to a target decoding-side device. As shown in fig. 8, the apparatus 80 includes:

a current sub-image receiving module 801, configured to receive at least one current sub-image sent by a server, a timestamp of the current sub-image, and frame identification information of the current frame image to be displayed, where the current sub-image is generated after the server divides the current frame image to be displayed;

a frame identifier receiving module 802, configured to receive frame identifier information of other frame images to be displayed broadcast by other decoding-end devices if it is determined that a time difference between the timestamp of the current sub-image and the current time is smaller than or equal to a preset time threshold, where the other decoding-end devices are decoding-end devices other than the target decoding-end device;

a target frame identification information determining module 803, configured to determine target frame identification information according to the frame identification information of the current frame image to be displayed and the frame identification information of other frame images to be displayed;

a current sub-image sending module 804, configured to send the current sub-image to the target display if it is determined that the frame identification information of the frame image to be displayed is consistent with the target frame identification information, so that the target display displays the current sub-image.

In one embodiment, the frame identification information is a frame sequence number, as shown in fig. 9, the apparatus 80 further includes:

a next frame sub-picture sending module 805 configured to:

In one embodiment, the target frame identification information determination module 803 is configured to:

In one embodiment, the current sub-image sending module 804 is further configured to:

In one embodiment, the current sub-image is an encoded current sub-image, and the apparatus 80 further comprises:

a current sub-image decoding module 806, configured to decode the current sub-image to generate a decoded current sub-image;

the current sub-image sending module 804 is configured to:

and sending the decoded current sub-image to the target display.

The implementation process and technical effects of the image display device provided in the embodiment of the present disclosure can be seen in the embodiments of fig. 3 and fig. 4, which are not described herein again.

Fig. 10 is a hardware structure diagram of a server according to an embodiment of the present disclosure, and as shown in fig. 10, the server 100 includes: a processor 1001 and a memory 1002, wherein the memory 1002 stores at least one computer instruction, and the instruction is loaded and executed by the processor 1001 to implement the steps executed in the image display method according to the embodiment shown in fig. 2.

Based on the image display method described in the embodiment corresponding to fig. 2, the present disclosure provides a computer-readable storage medium, where at least one computer instruction is stored, and the computer instruction is loaded and executed by a processor to implement the steps performed in the image display method described in the embodiment corresponding to fig. 2.

Fig. 11 is a hardware structure diagram of a decoding-end device provided in an embodiment of the present disclosure, and as shown in fig. 11, the decoding-end device 110 includes: comprising a processor 1101 and a memory 1102, said memory 1102 having stored therein at least one computer instruction, said instruction being loaded and executed by said processor 1101 to implement the steps performed in the image display method described in the corresponding embodiment of fig. 3.

Based on the image display method described in the embodiment corresponding to fig. 3, the present disclosure provides a computer-readable storage medium, in which at least one computer instruction is stored, and the computer instruction is loaded and executed by a processor to implement the steps performed in the image display method described in the embodiment corresponding to fig. 3.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. An image display method, comprising:

generating a timestamp for each of the current sub-images;

2. The method of claim 1, wherein prior to generating the timestamp for each sub-image, the method further comprises:

encoding the current sub-image to generate an encoded current sub-image;

3. The method of claim 1, further comprising:

4. An image display method, comprising:

5. The method of claim 4, wherein the frame identification information is a frame sequence number, the method further comprising:

6. The method according to claim 5, wherein determining target frame identification information according to the frame identification information of the current frame image to be displayed and the frame identification information of other frame images to be displayed comprises:

7. The method of claim 5, further comprising:

8. The method according to claim 4, wherein the current sub-image is an encoded current sub-image, and after receiving the current sub-image sent by the server, the timestamp of the current sub-image, and the frame identification information of the current frame image to be displayed, the method further comprises:

decoding the current sub-image to generate a decoded current sub-image;

the sending the current sub-image to the target display comprises:

and sending the decoded current sub-image to the target display.

9. An image display apparatus, comprising:

and the current sub-image sending module is used for sending the current sub-image, the timestamp, the first identification information and the frame identification information to a target decoding end device, so that the target decoding end device sends the current sub-image to the target display according to the timestamp, the reference time and the frame identification information, and the target decoding end device is a decoding end device of the target display corresponding to the image source of the frame image to be displayed.

10. An image display apparatus, comprising: