CN116468809A

CN116468809A - Image display method and device, storage medium and electronic equipment

Info

Publication number: CN116468809A
Application number: CN202210033916.1A
Authority: CN
Inventors: 毛峻岭
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-01-12
Filing date: 2022-01-12
Publication date: 2023-07-21

Abstract

The invention discloses an image display method and device, a storage medium and electronic equipment. Wherein the method comprises the following steps: and acquiring received first coded data, wherein the first coded data is coded data obtained by coding a first image in an original video stream, decoding the first coded data to obtain the first image, performing mark compensation on the first image according to the first mark information when the received first mark information matched with the first image is found, obtaining the first image added with the first mark, and rendering and displaying the first image added with the first mark. The invention solves the technical problem of larger display time delay of the image after the mark is added for the real-time acquisition of the image in the related technology.

Description

Image display method and device, storage medium and electronic equipment

Technical Field

The present invention relates to the field of computers, and in particular, to an image display method and apparatus, a storage medium, and an electronic device.

Background

At present, in the process of acquiring a video stream in real time by an image acquisition device, if object detection is required to be performed on the acquired video stream, generally, after the video stream is acquired, the object detection (for example, AI detection) is performed on the video stream first, and then, the detection result is marked on the video stream for encoding transmission.

For example, in the security monitoring field, after the image acquisition device acquires a video stream, AI detection is performed on a transmitting end frame by frame according to an acquired original image, a result of the AI detection is marked on the original image, then encoding and transmission are performed, a receiving end decodes after receiving encoded image data, the video stream after AI marking is recovered, and finally rendering and displaying are performed. AI detection and marking processes are added in series between video stream acquisition and rendering display, and because AI detection consumes a lot of time, the time delay of an end-to-end picture is often caused to be increased, the low-time-delay remote control requirement cannot be met, and further, the technical problem that the time delay of displaying the marked image in real time is large in the related technology is caused.

In view of the above problems, no effective solution has been proposed at present.

Disclosure of Invention

The embodiment of the invention provides an image display method and device, a storage medium and electronic equipment, which at least solve the technical problem of larger display time delay of images after marks are added for real-time acquisition of the images in the related technology.

According to an aspect of an embodiment of the present invention, there is provided an image display method including: acquiring received first encoded data, wherein the first encoded data is encoded data obtained by encoding a first image in an original video stream; decoding the first encoded data to obtain the first image; when the received first mark information matched with the first image is found, carrying out mark compensation on the first image according to the first mark information to obtain the first image added with a first mark, wherein the first mark information is mark information obtained by carrying out object detection on the first image, the first mark is used for marking a first object detected in the first image, and the operation of encoding the first image and the operation of carrying out object detection on the first image are operations which are executed in parallel; rendering and displaying the first image added with the first mark.

According to another aspect of the embodiment of the present invention, there is also provided an image transmission method, including: acquiring a first image in an original video stream, wherein the frame number of the first image in the original video stream is a first frame number; encoding the first image to obtain first encoded data; performing object detection on the first image to obtain first mark information, wherein when the object detection module detects a first object in the first image, the first mark information comprises position information of the first object in the first image, and the operation of encoding the first image and the operation of performing object detection on the first image are performed in parallel; and respectively transmitting the first coded data and the first marking information to target equipment.

According to another aspect of the embodiment of the present invention, there is also provided an image display apparatus including: the first acquisition module is used for acquiring received first coded data, wherein the first coded data is coded data obtained by coding a first image in an original video stream; the decoding module is used for decoding the first coded data to obtain the first image; the compensation module is used for carrying out mark compensation on the first image according to the first mark information when the received first mark information matched with the first image is found, so as to obtain the first image added with the first mark, wherein the first mark information is mark information obtained by carrying out object detection on the first image, the first mark is used for marking a first object detected in the first image, and the operation of encoding the first image and the operation of carrying out object detection on the first image are operations which are executed in parallel; and the display module is used for rendering and displaying the first image added with the first mark.

Optionally, the device is further configured to: acquiring a first frame number of the first image corresponding to the first encoded data; searching for mark information with a corresponding relation with the first frame number in a received mark information set, wherein the mark information set comprises one or more pairs of frame numbers with corresponding relation and mark information, the frame numbers in the mark information set are the frame numbers of images in the original video stream, and the mark information in the mark information set is mark information obtained by performing object detection on the images in the original video stream; when the mark information with the corresponding relation with the first frame number is found in the mark information set, the mark information with the corresponding relation with the first frame number in the mark information set is determined to be the first mark information matched with the first image.

Optionally, the device is further configured to: detecting whether the first mark information is received or not in a preset time length when the first mark information is not found; and when the first mark information is received within the preset time, carrying out mark compensation on the first image according to the first mark information to obtain the first image added with the first mark, and rendering and displaying the first image added with the first mark.

Optionally, the device is further configured to: rendering and displaying the first image without the mark when the first mark information is not received within the preset time length; or if the first marking information is not received within the preset time, searching a decoded second image matched with the first image, wherein the second marking information matched with the second image is the received marking information, the second marking information is the marking information obtained by performing object detection on the second image, and the operation of encoding the second image and the operation of performing object detection on the second image are performed in parallel; and under the condition that the second image is found, performing mark compensation on the first image according to the second mark information to obtain the first image added with a second mark, and rendering and displaying the first image added with the second mark, wherein the second mark is used for marking a second object predicted in the first image.

Optionally, the device is further configured to: and rendering and displaying the first image without the mark under the condition that the second image is not found.

Optionally, the device is further configured to: when the first mark information cannot be found, a decoded second image matched with the first image is found, wherein the second mark information matched with the second image is the received mark information, the second mark information is the mark information obtained by performing object detection on the second image, and the operation of encoding the second image and the operation of performing object detection on the second image are executed in parallel; and under the condition that the second image is found, performing mark compensation on the first image according to the second mark information to obtain the first image added with a second mark, and rendering and displaying the first image added with the second mark, wherein the second mark is used for marking a second object predicted in the first image.

Optionally, the device is configured to search for the decoded second image that matches the first image by: acquiring a first frame number of the first image corresponding to the first encoded data; searching a second frame number matched with the first frame number in a received mark information set, wherein the mark information set comprises one or more pairs of frame numbers and mark information with corresponding relation, the frame numbers in the mark information set are the frame numbers of images in the original video stream, the mark information in the mark information set is the mark information obtained by carrying out object detection on the images in the original video stream, the sequence number interval between the first frame number and the second frame number is smaller than or equal to a preset interval, and the second frame number is the frame number of the decoded second image; and under the condition that the second frame number is found in the marked information set, determining to find the decoded second image matched with the first image, wherein the second marked information is marked information with a corresponding relation with the second frame number in the marked information set.

Optionally, the device is configured to perform label compensation on the first image according to the second label information to obtain the first image to which the second label is added by: determining a second area where a target object in the second image is located according to the second marking information, and determining a second pixel point set in the second image, wherein the target object is an object detected in the second image, and the second pixel point set is a pixel point included in the second area; searching a first region matched with the second region in the first image, wherein the size of the first region is the same as that of the second region, the similarity between a first pixel point set and the second pixel point set is larger than or equal to a preset threshold value, and the first pixel point set is a pixel point included in the first region; and adding the second mark on the first area in the first image under the condition that the first area matched with the second area is found in the first image, wherein the object in the first area is the second object predicted in the first image.

Optionally, the device is configured to search the first image for a first region matching the second region by: searching the first area matched with the second area in the whole area of the first image; or determining a third region in the first image, wherein the position and size of the third region in the first image are the same as the position and size of the second region in the second image; expanding the boundary of the third region by a preset distance in the first image to obtain a target region, and searching the first region matched with the second region in the target region; or determining a third region in the first image, wherein the position and size of the third region in the first image are the same as the position and size of the second region in the second image; and expanding the boundary of the third region in the first image to obtain a target region, and searching the first region matched with the second region in the target region, wherein the target distance has a corresponding relation with the size of the first image or the target distance has a corresponding relation with the size of the storage space occupied by the first image.

Optionally, the device is further configured to: determining the classification information of the second object to be equal to the classification information of the target object when the second marking information includes the position information of the target object in the second image and the classification information of the target object, or configuring the second marking to be also used for marking the classification information of the second object; or when the second marking information includes the position information of the target object in the second image and the feature extraction information of the target object, determining the feature extraction information of the second object to be equal to the feature extraction information of the target object, or configuring the second marking to be also used for marking the feature extraction information of the second object; or when the second marking information includes the position information of the target object in the second image, the classification information of the target object, and the feature extraction information of the target object, determining the classification information of the second object to be equal to the classification information of the target object, and determining the feature extraction information of the second object to be equal to the feature extraction information of the target object, or configuring the second marking to be also used for marking the classification information of the second object and/or marking the feature extraction information of the second object.

Optionally, the device is configured to perform label compensation on the first image according to the first label information to obtain the first image to which the first label is added by: when the first mark information comprises the position information of the first object in the first image, determining a first area where the first object is located in the first image according to the position information of the first object in the first image; adding the first mark on the first region in the first image; or when the first marking information comprises the position information of the first object in the first image and the classification information of the first object, determining a first area where the first object is located in the first image according to the position information of the first object in the first image; adding the first mark on the first area in the first image according to the classification information of the first object, wherein the first mark is also used for marking the classification information of the first object; or when the first marking information comprises the position information of the first object in the first image and the feature extraction information of the first object, determining a first area where the first object is positioned in the first image according to the position information of the first object in the first image; adding the first mark on the first area in the first image according to the feature extraction information of the first object, wherein the first mark is also used for marking the feature extraction information of the first object; or when the first marking information comprises the position information of the first object in the first image, the classification information of the first object and the feature extraction information of the first object, determining a first area where the first object is located in the first image according to the position information of the first object in the first image; and adding the first mark on the first area in the first image according to the classification information of the first object and the feature extraction information of the first object, wherein the first mark is also used for marking the classification information of the first object and the feature extraction information of the first object.

According to another aspect of the embodiment of the present invention, there is also provided an image transmission apparatus including: the second acquisition module is used for acquiring a first image in an original video stream, wherein the frame sequence number of the first image in the original video stream is a first frame sequence number; the encoding module is used for encoding the first image to obtain first encoded data; the detection module is used for carrying out object detection on the first image to obtain first mark information, wherein the first mark information comprises position information of the first object in the first image when the object detection module detects the first object in the first image, and the operation of encoding the first image and the operation of carrying out object detection on the first image are executed in parallel; and the sending module is used for respectively sending the first coded data and the first marking information to target equipment.

Optionally, the device is further configured to: acquiring the received first coded data; decoding the first encoded data to obtain the first image; when the received first mark information matched with the first image is found, carrying out mark compensation on the first image according to the first mark information to obtain the first image added with a first mark, wherein the first mark is used for marking the first object detected in the first image; rendering and displaying the first image added with the first mark.

According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium having a computer program stored therein, wherein the computer program is configured to execute the above-described image display method when run.

According to yet another aspect of embodiments of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the image display method as above.

According to still another aspect of the embodiments of the present invention, there is also provided an electronic apparatus including a memory in which a computer program is stored, and a processor configured to execute the above-described image display method by the computer program.

In the embodiment of the invention, the received first encoded data is acquired, wherein the first encoded data is encoded data obtained by encoding a first image in an original video stream, the first encoded data is decoded to obtain a first image, when the received first mark information matched with the first image is found, the first image is subjected to mark compensation according to the first mark information to obtain a first image added with a first mark, the first mark information is mark information obtained by performing object detection on the first image, the first mark is used for marking a first object detected in the first image, the operation of encoding the first image and the operation of performing object detection on the first image are performed in parallel, the first image added with the first mark is rendered and displayed, and the first image added with the first mark is subjected to mark compensation based on the first mark information after the first mark information is acquired, so that the time delay of the first image added with the first mark is displayed in parallel, and the time delay of the first image added with the first mark is reduced, thereby achieving the purpose of displaying the real-time delay of the image added with the relevant technology.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention. In the drawings:

FIG. 1 is a schematic illustration of an application environment of an alternative image display method according to an embodiment of the present invention;

FIG. 2 is a flow chart of an alternative image display method according to an embodiment of the invention;

FIG. 3 is a schematic diagram of an alternative image display method according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of yet another alternative image display method according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of yet another alternative image display method according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of yet another alternative image display method according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of yet another alternative image display method according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of yet another alternative image display method according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of yet another alternative image display method according to an embodiment of the present invention;

FIG. 10 is a schematic diagram of an alternative image transmission method according to an embodiment of the invention;

Fig. 11 is a schematic structural view of an alternative image display device according to an embodiment of the present invention;

fig. 12 is a schematic structural view of an alternative image transmission apparatus according to an embodiment of the present invention;

FIG. 13 is a schematic diagram of an alternative image display product according to an embodiment of the invention;

fig. 14 is a schematic structural view of an alternative electronic device according to an embodiment of the present invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

First, partial terms or terminology appearing in describing embodiments of the present application are applicable to the following explanation:

video coding: the video encoding method is a method of converting a file in an original video format into a file in another video format by a compression technique.

The invention is illustrated below with reference to examples:

according to an aspect of the embodiment of the present invention, there is provided an image display method, alternatively, in the present embodiment, the above-described image display method may be applied to a hardware environment constituted by the server 101 and the terminal device 103 as shown in fig. 1. As shown in fig. 1, a server 101 is connected to a terminal 103 through a network, and may be used to provide services to a terminal device or an application installed on the terminal device, which may be a video application, an instant messaging application, a browser application, an educational application, a game application, or the like. The database 105 may be provided on or independent of the server for providing data storage services for the server 101, such as a video data storage server, which may include, but is not limited to: a wired network, a wireless network, wherein the wired network comprises: local area networks, metropolitan area networks, and wide area networks, the wireless network comprising: bluetooth, WIFI and other wireless communication networks, the terminal device 103 may be a terminal configured with an application program, and may include, but is not limited to, at least one of the following: the server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server for providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, big data and artificial intelligent platform. The terminal may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited herein. The application program 107 using the image display method displays the image through the terminal device 103, and can connect the acquisition device 109 with the server 101 through a network to acquire an original video stream, input the original video stream into the encoding module and the object detection module respectively, and send the obtained encoded data and the marking information to the terminal device 103.

As shown in fig. 1, the above image display method may be implemented in the terminal device 103 by:

s1, acquiring received first coded data on a terminal device 103, wherein the first coded data is coded data obtained by coding a first image in an original video stream;

s2, decoding the first coded data on the terminal equipment 103 to obtain a first image;

s3, when the received first mark information matched with the first image is found on the terminal equipment 103, carrying out mark compensation on the first image according to the first mark information to obtain a first image added with the first mark, wherein the first mark information is mark information obtained by carrying out object detection on the first image, the first mark is used for marking a first object detected in the first image, and the operation of encoding the first image and the operation of carrying out object detection on the first image are executed in parallel;

s4, rendering and displaying the first image added with the first mark on the terminal device 103.

Alternatively, in the present embodiment, the above-described image display method may also be implemented by a server, for example, in the server 101 shown in fig. 1; or by both the user terminal and the server.

The above is merely an example, and the present embodiment is not particularly limited.

Alternatively, as an optional embodiment, as shown in fig. 2, the image display method includes:

s202, acquiring received first coded data, wherein the first coded data is coded data obtained by coding a first image in an original video stream;

s204, decoding the first coded data to obtain a first image;

s206, when the received first mark information matched with the first image is found, carrying out mark compensation on the first image according to the first mark information to obtain the first image added with the first mark, wherein the first mark information is obtained by carrying out object detection on the first image, the first mark is used for marking a first object detected in the first image, and the operation of encoding the first image and the operation of carrying out object detection on the first image are executed in parallel;

and S208, rendering and displaying the first image added with the first mark.

Alternatively, in the present embodiment, the above-described image display method may include, but is not limited to, a cloud-based implementation.

Cloud technology (Cloud technology) is based on the general terms of network technology, information technology, integration technology, management platform technology, application technology and the like applied by Cloud computing business models, and can form a resource pool, so that the Cloud computing business model is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.

The internet of things (The Internet of Things, IOT for short) refers to collecting any object or process needing to be monitored, connected and interacted in real time through various devices and technologies such as various information sensors, radio frequency identification technologies, global positioning systems, infrared sensors and laser scanners, collecting various needed information such as sound, light, heat, electricity, mechanics, chemistry, biology and positions, and realizing ubiquitous connection of objects and people through various possible network access, and realizing intelligent sensing, identification and management of objects and processes. The internet of things is an information carrier based on the internet, a traditional telecommunication network and the like, and enables all common physical objects which can be independently addressed to form an interconnection network.

The Cloud IOT aims to connect information perceived by sensing equipment in the traditional IOT and accepted instructions into the Internet, networking is truly realized, mass data storage and operation are realized through a Cloud computing technology, the current running states of all 'objects' are perceived in real time due to the fact that the things are connected with each other, a large amount of data information can be generated in the process, how to collect the information, how to screen useful information in the mass information and make decision support for subsequent development, and the Cloud is a key problem affecting the development of the IOT, and the Internet of things Cloud based on Cloud computing and Cloud storage technology is also a powerful support for the technology and application of the IOT.

The cloud call center ((Cloud Call Center)) is a call center system built based on a cloud computing technology, enterprises can rapidly own the call center without purchasing any software and hardware systems and only by having basic conditions such as personnel, places and the like, and software and hardware platforms, communication resources, daily maintenance and services are provided by server providers. The method has the characteristics of short construction period, less investment, low risk, flexible deployment, strong system capacity scalability, low operation and maintenance cost and the like; whether a telephone marketing center or a customer service center, an enterprise can establish a call center system which has comprehensive, stable and reliable functions and can distribute the call access nationwide all over the country only by renting the service as required.

In this embodiment, the image display method can be applied to, for example, a cloud internet of things, and the video transmission delay in an AI detection scene is effectively reduced by combining with a cloud call center, so that the method is used for prompting pedestrians or dangerous objects in remote control scenes in industries such as mining areas, ports, logistics and the like, and assisting operators in remote control and the like.

Alternatively, in this embodiment, the first encoded data may include, but is not limited to, encoded data obtained by encoding an original video stream acquired by an image acquisition device, the first image may include, but is not limited to, any frame of image in the original video stream, and the encoding of the first image in the original video stream may include, but is not limited to, encoding methods based on the coding and decoding standards of international union, h.261, h.263, h.264, M-JPEG of a moving still image expert group, the MPEG series standard of an international organization for moving image expert group, and the like.

Alternatively, in this embodiment, the decoding of the first encoded data may include, but is not limited to, decoding the first encoded data based on the same coding standard, which corresponds to the coding mode, to obtain the first image.

It should be noted that, the receiving end continuously receives the first image obtained by encoding the original video stream, and at the same time, the receiving end continuously receives the mark information for performing object detection on the first image at the transmitting end, that is, the operation of encoding the first image and the operation of performing object detection on the first image are performed in parallel.

Optionally, in this embodiment, the first flag information is flag information obtained by performing object detection on the first image at the receiving end, where the object detection refers to detecting target objects existing in the image by using theories and methods in fields such as image processing and pattern recognition, determining semantic types of the target objects, and calibrating positions of the target objects in the image. The above-mentioned means of object detection may include, but are not limited to, face recognition, infrared detection, etc.

Alternatively, in this embodiment, the first flag information that matches the first image may include, but is not limited to, that the frame number of the first image is the same as the frame number corresponding to the first flag information, and it is determined that the first image matches the first flag information.

Alternatively, in this embodiment, the above-mentioned flag compensation method may include, but is not limited to, motion compensation, where the local image of the current frame is predicted and compensated by the local image of the previous frame.

Alternatively, in the present embodiment, the rendering and displaying the first image added with the first mark may include, but is not limited to, rendering and displaying the first image added with the first mark on the terminal device at a predetermined target rendering time.

For example, fig. 3 is a schematic diagram of an alternative image display method according to an embodiment of the present invention, as shown in fig. 3, in the prior art, an original video stream is generally collected in real time by an image collecting device, a first image in the collected original video stream is input to an AI detection module, after marking information is obtained, the first image with the mark added is input to an encoder, encoding is generated, and sent to a receiving end, after the receiving end receives the first encoding data, the first image with the mark added is decoded by a decoder, and finally, the display device is rendered and displayed, that is, the collected first image machine needs to be subjected to object detection operation, and then encoding is performed based on the result of the object detection operation, because the object detection time is relatively long, the time delay from the sending end to the receiving end tends to be large, and the low-delay remote control requirement cannot be met.

In this embodiment, fig. 4 is a schematic diagram of another alternative image display method according to an embodiment of the present invention, as shown in fig. 4, an image capturing device captures an original video stream in real time, inputs a first image in the captured original video stream into an AI detection module and an encoder, performs an object detection operation and an encoding operation on the first image in parallel, sends first encoded data obtained by the encoding operation to a decoder at a receiving end to decode, obtains the first image, inputs the first image and first tag information into a tag compensation module, performs tag compensation, and then displays the first image added with the first tag in a display.

Optionally, in this embodiment, fig. 5 is a schematic diagram of another optional image display method according to an embodiment of the present invention, as shown in fig. 5, taking a specific application scenario as a security scenario, when a monitoring camera starts to collect an original video stream, taking a security field as an example in a specific application process, the method may include, but is not limited to, collecting the original video stream in real time by the security camera, inputting each collected frame image into an AI detection module and an encoder to generate tag information and encoded data, receiving a first encoded data, decoding the received first encoded data to obtain a first image, and sending the first image to a tag compensation module, where the AI detection module also continuously sends the generated tag information to the tag compensation module, and the tag compensation module performs tag compensation on the first image based on the tag information to obtain a first image with the first tag added, and displaying the first image in a display.

The above is merely an example, and the present embodiment is not limited in any way.

According to the embodiment, the received first encoded data is obtained, wherein the first encoded data is encoded data obtained by encoding a first image in an original video stream, the first encoded data is decoded to obtain a first image, when the received first mark information matched with the first image is found, the first image is subjected to mark compensation according to the first mark information to obtain a first image added with a first mark, the first mark information is mark information obtained by performing object detection on the first image, the first mark is used for marking the first object detected in the first image, the operation of encoding the first image and the operation of performing object detection on the first image are performed in parallel, the first image added with the first mark is rendered and displayed, the encoding operation and the object detection operation are performed in parallel before image data transmission, and after the encoded image data is obtained, the first image added with the first mark is displayed based on the first mark information in a mark compensation mode, so that the technical problem of adding the first mark is achieved, and the real-time delay is relatively large after the real-time delay is achieved, and the time delay is displayed.

As an alternative, the method further comprises:

acquiring a first frame number of a first image corresponding to first encoded data;

searching for mark information with a corresponding relation with the first frame number in a received mark information set, wherein the mark information set comprises one or more pairs of frame numbers with the corresponding relation and mark information, the frame numbers in the mark information set are the frame numbers of images in an original video stream, and the mark information in the mark information set is mark information obtained by carrying out object detection on the images in the original video stream;

when the mark information with the corresponding relation with the first frame number is searched in the mark information set, the mark information with the corresponding relation with the first frame number in the mark information set is determined to be the first mark information matched with the first image.

Alternatively, in this embodiment, the received tag information set may include, but is not limited to, a tag information set formed by a plurality of tag information arranged by frame numbers generated by the AI detection module detecting the original video stream frame by frame.

It should be noted that, the frame number is a frame number corresponding to a frame image in the original video stream, and the first frame number of the first image corresponding to the first encoded data may be understood as the first frame number of the first image in the video stream acquired by the acquisition device.

Optionally, in this embodiment, the one or more pairs of frame numbers and tag information having a corresponding relationship may include, but are not limited to, that the frame numbers are the same or the frame numbers satisfy a preset relationship (for example, the difference between the frame numbers is closest to, is smaller than a threshold value, etc.), where the frame number in the tag information set is the frame number of an image in the original video stream, for example, the frame number of the first image is 1, the frame number of the first encoded data obtained by encoding the first image is 1, and the frame number of the first tag information obtained by inputting the first image into the AI detection module for object detection is also 1.

As an alternative, the method further comprises:

detecting whether the first mark information is received or not in a preset time length when the first mark information is not found;

And when the first mark information is received within the preset time, carrying out mark compensation on the first image according to the first mark information to obtain a first image added with the first mark, and rendering and displaying the first image added with the first mark.

Alternatively, in this embodiment, the above-mentioned searching for the first flag information may include, but is not limited to, not searching for flag information having a correspondence with the first frame number in the received flag information set, for example, fig. 6 is a schematic diagram of still another alternative image display method according to an embodiment of the present invention, and as shown in fig. 6, taking the first frame number as 3 as an example, only flag information corresponding to the frame number 1 and the frame number 2 is included in the received flag information set, and no flag information corresponding to the frame number 3 is included, and at this time, may be configured to wait for the above-mentioned predetermined period to detect whether the first flag information corresponding to the frame number 3 is received.

Alternatively, in this embodiment, the predetermined time period may be preconfigured by the system, or may be determined according to the rendering time of rendering the first image with the first mark added, for example, the current system time is 3:54:30, and the rendering time is configured to be 3:55:20, and then the predetermined time period may be configured as a time period from the current system time to the rendering time, that is, 30S.

As an alternative, the method further comprises:

rendering and displaying a first image without adding marks under the condition that the first mark information is not received within a preset time length; or alternatively

Searching for a decoded second image matched with the first image under the condition that the first mark information is not received within a preset time length, wherein the second mark information matched with the second image is the received mark information, the second mark information is the mark information obtained by carrying out object detection on the second image, and the operation of encoding the second image and the operation of carrying out object detection on the second image are executed in parallel; and under the condition that the second image is found, performing mark compensation on the first image according to the second mark information to obtain a first image added with the second mark, and rendering and displaying the first image added with the second mark, wherein the second mark is used for marking a second object predicted in the first image.

Optionally, in this embodiment, rendering and displaying the first image without adding the mark may include, but is not limited to, directly rendering and displaying the first image acquired by the image acquisition device when the first mark information is not received within the predetermined time period, searching for the decoded second image matched with the first image when the first mark information is not received within the predetermined time period may include, but is not limited to, searching for the second image according to the received mark information set when the first mark information is not received within the predetermined time period, specifically, searching for the closest second image according to the first frame number of the first image, for example, fig. 7 is a schematic diagram of an alternative image display method according to the embodiment of the present invention, taking the first frame number as 3 as an example, when only the mark information corresponding to the frame number 1 and the frame number 2 is included in the received mark information set, excluding the mark information corresponding to the frame number 3, waiting for the predetermined time period, determining whether the second image is a predicted mark corresponding to the first frame number 3 is detected, determining whether the predicted mark information corresponding to the second image is detected as a specific mark corresponding to the second frame number 2, determining that the first image is not detected as a predicted mark corresponding to the first frame number, determining whether the predicted image corresponding to the first image is detected as a predicted image corresponding to the first frame number 2, and determining that the second image is not detected according to the predicted image corresponding to the first frame number 2, and predicting the marking information from the first image based on the position and the classification information so as to carry out marking compensation on the first image.

As an alternative, the method further comprises:

and rendering and displaying the first image without the mark under the condition that the second image is not found.

Alternatively, in this embodiment, rendering and displaying the first image without the tag may include, but is not limited to, directly rendering and displaying the first image acquired by the image acquisition device when the first tag information is not found.

As an alternative, the method further comprises:

optionally, in this embodiment, when the first flag information is not found, a decoded second image that matches the first image is found, where the second flag information that matches the second image is the received flag information, and the second flag information is flag information obtained by performing object detection on the second image, and the operation of encoding the second image and the operation of performing object detection on the second image are performed in parallel; and under the condition that the second image is found, performing mark compensation on the first image according to the second mark information to obtain a first image added with the second mark, and rendering and displaying the first image added with the second mark, wherein the second mark is used for marking a second object predicted in the first image.

Alternatively, in this embodiment, in the case that the first flag information is not found, searching for the decoded second image that matches the first image may include, but is not limited to, searching for the second image according to the received set of flag information, specifically, searching for the nearest second image according to the first frame number of the first image, for example, as shown in fig. 7, taking the first frame number as 3 as an example, in the received set of flag information, only the flag information corresponding to the frame number 1 and the frame number 2 is included, and no flag information corresponding to the frame number 3 is included, at this time, may be configured to wait for the predetermined period of time to detect whether the first flag information corresponding to the frame number 3 is received, in the case that the first flag information corresponding to the frame number 3 is not received, determining the image corresponding to the frame number 2 as the second image, the flag information corresponding to the frame number 2 is the second flag information, for marking the second object predicted in the first image, and specifically, the prediction mode may include, but is not limited to, for example, the motion compensation mode, and the position compensation mode may be determined from the second flag information and the first image classification mode based on the first flag information.

As an alternative, searching for a decoded second image matching the first image includes:

searching a second frame number matched with the first frame number in a received mark information set, wherein the mark information set comprises one or more pairs of frame numbers and mark information with corresponding relations, the frame numbers in the mark information set are frame numbers of images in an original video stream, the mark information in the mark information set is mark information obtained by carrying out object detection on the images in the original video stream, the sequence number interval between the first frame number and the second frame number is smaller than or equal to a preset interval, and the second frame number is the frame number of the decoded second image;

and under the condition that the second frame number is found in the mark information set, determining to find the decoded second image matched with the first image, wherein the second mark information is mark information with a corresponding relation with the second frame number in the mark information set.

Optionally, in this embodiment, the frame number in the above-mentioned marking information set is the frame number of the image in the original video stream, which can be understood as that, at the transmitting end, after the image collecting device collects a frame image of the original video stream, the frame number is added to the frame image, and the frame number is sent to the object detecting module and the encoding module respectively to detect and encode the object.

Optionally, in this embodiment, the sequence number interval between the first frame sequence number and the second frame sequence number is less than or equal to the preset interval, which may include, but is not limited to, that the difference between the values of the first frame sequence number and the second frame sequence number is less than or equal to the preset interval, for example, taking the preset interval as 1, in the case of the first frame sequence number being 3, the second frame sequence number is 2, taking the preset interval as 2, in the case of the first frame sequence number being 3, the frame sequence number less than or equal to the preset interval is 1 or 2, and in the case that the frame sequence number includes a plurality of frame sequence numbers, determining that the value is closest to the first frame sequence number is the second frame sequence number, that is, the frame sequence number 2.

As an alternative, performing marker compensation on the first image according to the second marker information to obtain a first image to which the second marker is added, including:

determining a second region where a target object in the second image is located according to the second marking information, and determining a second pixel point set in the second image, wherein the target object is an object detected in the second image, and the second pixel point set is a pixel point included in the second region;

searching a first region matched with a second region in a first image, wherein the size of the first region is the same as that of the second region, the similarity between a first pixel point set and a second pixel point set is larger than or equal to a preset threshold value, and the first pixel point set is a pixel point included in the first region;

And adding a second mark on the first region in the first image under the condition that the first region matched with the second region is found in the first image, wherein the object in the first region is a second object predicted in the first image.

Alternatively, in the present embodiment, the target object may include, but is not limited to, an object that the object detection module is capable of detecting, and may include, but is not limited to, a person, an animal, a vehicle, and the like, for example.

Alternatively, in this embodiment, the second region may include, but is not limited to, a region where the target object is located in the second image, and the second pixel point set is a pixel point set included in the second region.

Alternatively, in this embodiment, the searching for the first area matching the second area in the first image may include, but is not limited to, determining coordinate information of the second area on the second image, and searching for the first area from the first image based on the coordinate information. The size of the first area is the same as the size of the second area, for example, when the size of the second area is 8×8, the first area is also 8×8, and the similarity between the first pixel point set and the second pixel point set is greater than or equal to a preset threshold value may include, but not limited to, generating first feature information according to pixels in the first pixel point set, generating second feature information according to pixels in the second pixel point set, and determining the similarity between the first feature information and the second feature information to determine whether the similarity between the first pixel point set and the second pixel point set is greater than or equal to the preset threshold value, where the preset threshold value may be preset by the system.

Alternatively, in the present embodiment, the above-described similarity may be determined by a distance between pixel sets, for example, a plurality of distance measures such as a euclidean distance, a manhattan distance, and the like.

For example, fig. 8 is a schematic diagram of still another alternative image display method according to an embodiment of the present invention, as shown in fig. 8, where the second flag information indicates that the second region of the second image detects the target object "person", the position of the second region and the second pixel point set corresponding to the second region are acquired, the first region is determined from the first image based on the position of the second region, the first pixel point set corresponding to the first region is acquired, whether the similarity between the first pixel point set and the second pixel point set satisfies the preset threshold is determined, and if yes, the second flag is added on the first image.

Alternatively, in the present embodiment, the above-described predictive markers may include, but are not limited to, frame markers, arrow markers, and the like.

The frame mark indicates that one frame is displayed on the boundary of the first region, and the arrow mark indicates that one arrow is displayed on the first region.

As an alternative, searching for a first region matching the second region in the first image includes:

Searching a first area matched with the second area in the whole area of the first image; or alternatively

Determining a third region in the first image, wherein the position and size of the third region in the first image are the same as the position and size of the second region in the second image; expanding the boundary of the third region by a preset distance in the first image to obtain a target region, and searching a first region matched with the second region in the target region; or alternatively

Determining a third region in the first image, wherein the position and size of the third region in the first image are the same as the position and size of the second region in the second image; and expanding the boundary of the third area by a target distance in the first image to obtain a target area, and searching the first area matched with the second area in the target area, wherein the target distance has a corresponding relation with the size of the first image, or the target distance has a corresponding relation with the size of a storage space occupied by the first image.

Alternatively, in this embodiment, the above-mentioned searching for the first area matching the second area in the entire area of the first image may include, but is not limited to, searching for the first area having the same size as the second area in the entire area of the first image, for example, when the first image is an image having a size of 4x4 and the second area has a size of 2x2, searching for the first area having a size of 2x2 when the first area corresponds to the position of the second area in the first image.

Optionally, in this embodiment, the determining a third area in the first image, where the position and the size of the third area in the first image are the same as the position and the size of the second area in the second image, the expanding the boundary of the third area may take the center of the third area as the center of expansion, expanding the third area by a preset distance according to the shape of the second area, taking the second area as a circular 2x2 area as an example, determining first coordinates of the center of the second area on the second image, finding the center of the third area on the first image according to the coordinates, determining a target area in the first image, expanding the third area by a preset distance with the center of the third area as the center, and finding the first area in the target area.

For example, fig. 9 is a schematic diagram of another alternative image display method according to an embodiment of the present invention, as shown in fig. 9, a position of a second area on the second image is determined, a third area corresponding to the position of the second area on the second image is determined on the first image, a preset distance is expanded outwards with a center of the third area as a center, a target area shown in fig. 9 is obtained, and a first area matched with the second area is searched in the target area.

Alternatively, in this embodiment, the target distance may be configured to have a correspondence with the size of the first image, where the larger the size of the first image is, the larger the target distance is, or the target distance may have a correspondence with the size of the storage space occupied by the first image, and where the larger the storage space occupied by the first image is, the larger the target distance is.

As an alternative, the method further comprises:

when the second marking information includes the position information of the target object in the second image and the classification information of the target object, determining the classification information of the second object to be equal to the classification information of the target object, or configuring the second marking to be also used for marking the classification information of the second object; or alternatively

When the second marking information includes the position information of the target object in the second image and the feature extraction information of the target object, determining the feature extraction information of the second object to be equal to the feature extraction information of the target object, or configuring the second marking to be also used for marking the feature extraction information of the second object; or alternatively

When the second marker information includes the position information of the target object in the second image, the classification information of the target object, and the feature extraction information of the target object, the classification information of the second object is determined to be equal to the classification information of the target object, and the feature extraction information of the second object is determined to be equal to the feature extraction information of the target object, or the second marker is configured to be further used to mark the classification information of the second object and/or the feature extraction information of the second object.

Alternatively, in this embodiment, the location information may include, but is not limited to, coordinate information, the classification information may include, but is not limited to, information for indicating a classification condition of the target object, for example, in the security field, the classification information may be used to indicate whether the identified person object meets the security requirement, and the classification information may include, but is not limited to, color indication of the arrow or the frame, for example, green indicates that the classification result is normal, and red indicates that the classification result is abnormal. The above classification information may also include, but is not limited to, sex, age, etc. of the target object.

Alternatively, in the present embodiment, the feature extraction information of the target object may include, but is not limited to, additional information of the target object, for example, when the target object is a human object, the feature extraction information may include, but is not limited to, information indicating whether the target object wears a hat, the body temperature of the target object, and the like.

Alternatively, in the present embodiment, the above-mentioned determining the classification information of the second object to be equal to the classification information of the target object may include, but is not limited to, directly configuring the classification information of the second object on the second image to be equal to the classification information of the target object, the above-mentioned determining the feature extraction information of the second object to be equal to the feature extraction information of the target object may include, but is not limited to, directly configuring the feature extraction information of the second object on the second image to be equal to the feature extraction information of the target object, the above-mentioned configuring the second mark to be further used for marking the classification information of the second object and/or the feature extraction information of the second object may include, but is not limited to, by the second mark marking may include, but is not limited to, also configuring the second mark to mark the second object when the second mark indicated by the second mark information is configured to mark the target object.

As an alternative, performing marker compensation on the first image according to the first marker information to obtain a first image to which the first marker is added, including:

when the first mark information comprises the position information of the first object in the first image, determining a first area where the first object is located in the first image according to the position information of the first object in the first image; adding a first mark on a first region in a first image; or alternatively

When the first marking information comprises the position information of the first object in the first image and the classification information of the first object, determining a first area where the first object is located in the first image according to the position information of the first object in the first image; adding a first mark on a first area in the first image according to the classification information of the first object, wherein the first mark is also used for marking the classification information of the first object; or alternatively

When the first mark information comprises the position information of the first object in the first image and the feature extraction information of the first object, determining a first area where the first object is located in the first image according to the position information of the first object in the first image; adding a first mark on a first area in the first image according to the feature extraction information of the first object, wherein the first mark is also used for marking the feature extraction information of the first object; or alternatively

When the first marking information comprises position information of the first object in the first image, classification information of the first object and feature extraction information of the first object, determining a first area where the first object is located in the first image according to the position information of the first object in the first image; and adding a first mark on a first area in the first image according to the classification information of the first object and the feature extraction information of the first object, wherein the first mark is also used for marking the classification information of the first object and the feature extraction information of the first object.

According to another aspect of the embodiment of the present invention, there is also provided an image transmission method, as shown in fig. 10, including:

s1002, acquiring a first image in an original video stream, wherein the frame number of the first image in the original video stream is a first frame number;

s1004, encoding the first image to obtain first encoded data;

s1006, performing object detection on the first image to obtain first mark information, wherein when the object detection module detects the first object in the first image, the first mark information comprises position information of the first object in the first image, and the operation of encoding the first image and the operation of performing object detection on the first image are performed in parallel;

and S1008, respectively transmitting the first coded data and the first marking information to the target equipment.

Alternatively, in the present embodiment, the above-described operation of encoding the first image and the operation of object detection of the first image are operations performed in parallel at the transmitting end. The sending the first encoded data and the first flag information to the target device respectively may include, but is not limited to, sending the first encoded data and the first flag information to the target device respectively using different threads, where the target device is the receiving end device.

As an alternative, the method further includes:

s1, acquiring received first coded data;

s2, decoding the first coded data to obtain a first image;

s3, when the received first mark information matched with the first image is found, carrying out mark compensation on the first image according to the first mark information to obtain a first image added with the first mark, wherein the first mark is used for marking a first object detected in the first image;

and S4, rendering and displaying the first image added with the first mark.

Alternatively, in the present embodiment, the method performed in steps S1 to S4 is that an operation step is performed on the target device, that is, steps S1 to S4 are performed after the first encoded data and the first flag information are respectively transmitted to the target device.

It will be appreciated that in the specific embodiments of the present application, related data such as user information, labeling information, etc. are referred to, and when the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required to be obtained, and the collection, use, and processing of related data is required to comply with related laws and regulations and standards of related countries and regions.

It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present invention. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present invention.

According to another aspect of the embodiment of the present invention, there is also provided an image display apparatus for implementing the above-described image display method. As shown in fig. 11, the apparatus includes:

a first obtaining module 1102, configured to obtain received first encoded data, where the first encoded data is encoded data obtained by encoding a first image in an original video stream;

a decoding module 1104, configured to decode the first encoded data to obtain the first image;

a compensation module 1106, configured to, when first mark information that matches the first image and is received is found, perform mark compensation on the first image according to the first mark information, to obtain the first image to which a first mark is added, where the first mark information is mark information obtained by performing object detection on the first image, the first mark is used to mark a first object detected in the first image, and the operation of encoding the first image and the operation of performing object detection on the first image are operations performed in parallel;

A display module 1108, configured to render and display the first image to which the first mark is added.

As an alternative, the device is further configured to:

acquiring a first frame number of the first image corresponding to the first encoded data;

searching for mark information with a corresponding relation with the first frame number in a received mark information set, wherein the mark information set comprises one or more pairs of frame numbers with corresponding relation and mark information, the frame numbers in the mark information set are the frame numbers of images in the original video stream, and the mark information in the mark information set is mark information obtained by performing object detection on the images in the original video stream;

when the mark information with the corresponding relation with the first frame number is found in the mark information set, the mark information with the corresponding relation with the first frame number in the mark information set is determined to be the first mark information matched with the first image.

As an alternative, the device is further configured to:

And when the first mark information is received within the preset time, carrying out mark compensation on the first image according to the first mark information to obtain the first image added with the first mark, and rendering and displaying the first image added with the first mark.

As an alternative, the device is further configured to:

rendering and displaying the first image without the mark when the first mark information is not received within the preset time length; or alternatively

Searching for a decoded second image matched with the first image under the condition that the first mark information is not received within the preset time length, wherein the second mark information matched with the second image is the received mark information, the second mark information is the mark information obtained by performing object detection on the second image, and the operation of encoding the second image and the operation of performing object detection on the second image are performed in parallel; and under the condition that the second image is found, performing mark compensation on the first image according to the second mark information to obtain the first image added with a second mark, and rendering and displaying the first image added with the second mark, wherein the second mark is used for marking a second object predicted in the first image.

As an alternative, the device is further configured to:

when the first mark information cannot be found, a decoded second image matched with the first image is found, wherein the second mark information matched with the second image is the received mark information, the second mark information is the mark information obtained by performing object detection on the second image, and the operation of encoding the second image and the operation of performing object detection on the second image are executed in parallel; and under the condition that the second image is found, performing mark compensation on the first image according to the second mark information to obtain the first image added with a second mark, and rendering and displaying the first image added with the second mark, wherein the second mark is used for marking a second object predicted in the first image.

As an alternative, the apparatus is configured to find a decoded second image that matches the first image by:

searching a second frame number matched with the first frame number in a received mark information set, wherein the mark information set comprises one or more pairs of frame numbers and mark information with corresponding relation, the frame numbers in the mark information set are the frame numbers of images in the original video stream, the mark information in the mark information set is the mark information obtained by carrying out object detection on the images in the original video stream, the sequence number interval between the first frame number and the second frame number is smaller than or equal to a preset interval, and the second frame number is the frame number of the decoded second image;

and under the condition that the second frame number is found in the marked information set, determining to find the decoded second image matched with the first image, wherein the second marked information is marked information with a corresponding relation with the second frame number in the marked information set.

As an alternative, the device is configured to perform label compensation on the first image according to the second label information to obtain the first image to which the second label is added by:

Determining a second area where a target object in the second image is located according to the second marking information, and determining a second pixel point set in the second image, wherein the target object is an object detected in the second image, and the second pixel point set is a pixel point included in the second area;

searching a first region matched with the second region in the first image, wherein the size of the first region is the same as that of the second region, the similarity between a first pixel point set and the second pixel point set is larger than or equal to a preset threshold value, and the first pixel point set is a pixel point included in the first region;

and adding the second mark on the first area in the first image under the condition that the first area matched with the second area is found in the first image, wherein the object in the first area is the second object predicted in the first image.

As an alternative, the device is configured to search the first image for a first region that matches the second region by:

Searching the first area matched with the second area in the whole area of the first image; or alternatively

Determining a third region in the first image, wherein the position and size of the third region in the first image are the same as the position and size of the second region in the second image; expanding the boundary of the third region by a preset distance in the first image to obtain a target region, and searching the first region matched with the second region in the target region; or alternatively

Determining a third region in the first image, wherein the position and size of the third region in the first image are the same as the position and size of the second region in the second image; and expanding the boundary of the third region in the first image to obtain a target region, and searching the first region matched with the second region in the target region, wherein the target distance has a corresponding relation with the size of the first image or the target distance has a corresponding relation with the size of the storage space occupied by the first image.

As an alternative, the device is further configured to:

Determining the classification information of the second object to be equal to the classification information of the target object when the second marking information includes the position information of the target object in the second image and the classification information of the target object, or configuring the second marking to be also used for marking the classification information of the second object; or alternatively

When the second marker information includes the position information of the target object in the second image and the feature extraction information of the target object, determining the feature extraction information of the second object to be equal to the feature extraction information of the target object, or configuring the second marker to be also used for marking the feature extraction information of the second object; or alternatively

When the second marker information includes the position information of the target object in the second image, the classification information of the target object, and the feature extraction information of the target object, the classification information of the second object is determined to be equal to the classification information of the target object, and the feature extraction information of the second object is determined to be equal to the feature extraction information of the target object, or the second marker is configured to be further used for marking the classification information of the second object and/or marking the feature extraction information of the second object.

As an alternative, the device is configured to perform label compensation on the first image according to the first label information to obtain the first image to which the first label is added by:

when the first mark information comprises the position information of the first object in the first image, determining a first area where the first object is located in the first image according to the position information of the first object in the first image; adding the first mark on the first region in the first image; or alternatively

When the first marking information comprises the position information of the first object in the first image and the classification information of the first object, determining a first area where the first object is located in the first image according to the position information of the first object in the first image; adding the first mark on the first area in the first image according to the classification information of the first object, wherein the first mark is also used for marking the classification information of the first object; or alternatively

When the first marking information comprises the position information of the first object in the first image and the feature extraction information of the first object, determining a first area where the first object is located in the first image according to the position information of the first object in the first image; adding the first mark on the first area in the first image according to the feature extraction information of the first object, wherein the first mark is also used for marking the feature extraction information of the first object; or alternatively

When the first marking information comprises position information of the first object in the first image, classification information of the first object and feature extraction information of the first object, determining a first area where the first object is located in the first image according to the position information of the first object in the first image; and adding the first mark on the first area in the first image according to the classification information of the first object and the feature extraction information of the first object, wherein the first mark is also used for marking the classification information of the first object and the feature extraction information of the first object.

According to another aspect of the embodiment of the present invention, there is also provided an image display apparatus for implementing the above-described image transmission method. As shown in fig. 12, the apparatus includes:

a second obtaining module 1202, configured to obtain a first image in an original video stream, where a frame number of the first image in the original video stream is a first frame number;

the encoding module 1204 is configured to encode the first image to obtain first encoded data;

a detection module 1206, configured to perform object detection on the first image to obtain first flag information, where in a case where the object detection module detects a first object in the first image, the first flag information includes location information of the first object in the first image, and the operation of encoding the first image and the operation of performing object detection on the first image are performed in parallel;

A sending module 1208, configured to send the first encoded data and the first flag information to a target device respectively.

As an alternative, the device is further configured to: acquiring the received first coded data; decoding the first encoded data to obtain the first image; when the received first mark information matched with the first image is found, carrying out mark compensation on the first image according to the first mark information to obtain the first image added with a first mark, wherein the first mark is used for marking the first object detected in the first image; rendering and displaying the first image added with the first mark.

According to one aspect of the present application, a computer program product is provided, comprising a computer program/instructions containing program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via the communication portion 1309 and/or installed from the removable medium 1311. When executed by the central processor 1301, performs the various functions provided by the embodiments of the present application.

The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

Fig. 13 schematically shows a block diagram of a computer system for implementing an electronic device according to an embodiment of the present application.

It should be noted that, the computer system 1300 of the electronic device shown in fig. 13 is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present application.

As shown in fig. 13, the computer system 1300 includes a central processing unit 1301 (Central Processing Unit, CPU) which can execute various appropriate actions and processes according to a program stored in a Read-Only Memory 1302 (ROM) or a program loaded from a storage portion 1308 into a random access Memory 1303 (Random Access Memory, RAM). In the random access memory 1303, various programs and data necessary for the system operation are also stored. The cpu 1301, the rom 1302, and the ram 1303 are connected to each other via a bus 1304. An Input/Output interface 1305 (i.e., an I/O interface) is also connected to bus 1304.

The following components are connected to the input/output interface 1305: an input section 1306 including a keyboard, a mouse, and the like; an output portion 1307 including a Cathode Ray Tube (CRT), a liquid crystal display (Liquid Crystal Display, LCD), and the like, a speaker, and the like; a storage portion 1308 including a hard disk or the like; and a communication section 1309 including a network interface card such as a local area network card, a modem, or the like. The communication section 1309 performs a communication process via a network such as the internet. The drive 1130 is also connected to the input/output interface 1305 as needed. Removable media 1311, such as magnetic disks, optical disks, magneto-optical disks, semiconductor memory, and the like, is mounted on drive 1130 as needed so that a computer program read therefrom is mounted into storage portion 1308 as needed.

In particular, according to embodiments of the present application, the processes described in the various method flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via the communication portion 1309 and/or installed from the removable medium 1311. The computer programs, when executed by the central processor 1301, perform the various functions defined in the system of the present application.

According to still another aspect of the embodiment of the present invention, there is also provided an electronic device for implementing the above image display method, which may be the terminal device or the server shown in fig. 1. The present embodiment is described taking the electronic device as a terminal device as an example. As shown in fig. 14, the electronic device comprises a memory 1402 and a processor 1404, the memory 1402 having stored therein a computer program, the processor 1404 being arranged to perform the steps of any of the method embodiments described above by means of the computer program.

Alternatively, in this embodiment, the electronic device may be located in at least one network device of a plurality of network devices of the computer network.

Alternatively, in the present embodiment, the above-described processor may be configured to execute the following steps by a computer program:

s1, acquiring received first coded data, wherein the first coded data is coded data obtained by coding a first image in an original video stream;

s2, decoding the first coded data to obtain a first image;

s3, when the received first mark information matched with the first image is found, carrying out mark compensation on the first image according to the first mark information to obtain the first image added with the first mark, wherein the first mark information is obtained by carrying out object detection on the first image, the first mark is used for marking a first object detected in the first image, and the operation of encoding the first image and the operation of carrying out object detection on the first image are executed in parallel;

and S4, rendering and displaying the first image added with the first mark.

Alternatively, it will be understood by those skilled in the art that the structure shown in fig. 14 is only schematic, and the electronic device may be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, a palm computer, a mobile internet device (Mobile Internet Devices, MID), a PAD, etc. Fig. 14 is not limited to the structure of the electronic device and the electronic apparatus described above. For example, the electronics can also include more or fewer components (e.g., network interfaces, etc.) than shown in fig. 14, or have a different configuration than shown in fig. 14.

The memory 1402 may be used to store software programs and modules, such as program instructions/modules corresponding to the image display method and apparatus in the embodiment of the present invention, and the processor 1404 executes the software programs and modules stored in the memory 1402 to perform various functional applications and data processing, i.e., implement the image display method described above. Memory 1402 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 1402 may further include memory located remotely from processor 1404, which may be connected to the terminal through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 1402 may be used for storing information such as an image and tag information, in particular, but not limited to. As an example, as shown in fig. 14, the memory 1402 may include, but is not limited to, the first acquisition module 1102, the decoding module 1104, the compensation module 1106, and the display module 1108 in the image display device. In addition, other module units in the image display apparatus may be included, but are not limited to, and are not described in detail in this example.

Optionally, the transmission device 1406 is used to receive or transmit data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1406 includes a network adapter (Network Interface Controller, NIC) that can connect to other network devices and routers via a network cable to communicate with the internet or a local area network. In one example, the transmission device 1406 is a Radio Frequency (RF) module that is used to communicate wirelessly with the internet.

In addition, the electronic device further includes: a display 1408 for displaying the first image with the first mark added thereto; and a connection bus 1410 for connecting the respective module parts in the above-described electronic device.

In other embodiments, the terminal device or the server may be a node in a distributed system, where the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the plurality of nodes through a network communication. Among them, the nodes may form a Peer-To-Peer (P2P) network, and any type of computing device, such as a server, a terminal, etc., may become a node in the blockchain system by joining the Peer-To-Peer network.

According to one aspect of the present application, there is provided a computer-readable storage medium, from which a processor of a computer device reads the computer instructions, the processor executing the computer instructions, so that the computer device performs the image display method provided in various alternative implementations of the above-described image display aspect.

Alternatively, in the present embodiment, the above-described computer-readable storage medium may be configured to store a computer program for performing the steps of:

s2, decoding the first coded data to obtain a first image;

And S4, rendering and displaying the first image added with the first mark.

Alternatively, in this embodiment, it will be understood by those skilled in the art that all or part of the steps in the methods of the above embodiments may be performed by a program for instructing a terminal device to execute the steps, where the program may be stored in a computer readable storage medium, and the storage medium may include: flash disk, read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), magnetic or optical disk, and the like.

The integrated units in the above embodiments may be stored in the above-described computer-readable storage medium if implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the technical solution of the present invention may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing one or more computer devices (which may be personal computers, servers or network devices, etc.) to perform all or part of the steps of the method described in the embodiments of the present invention.

In the foregoing embodiments of the present invention, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.

In several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, such as the division of the units, is merely a logical function division, and may be implemented in another manner, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.

Claims

1. An image display method, comprising:

acquiring received first encoded data, wherein the first encoded data is encoded data obtained by encoding a first image in an original video stream;

decoding the first encoded data to obtain the first image;

when the received first mark information matched with the first image is found, carrying out mark compensation on the first image according to the first mark information to obtain the first image added with a first mark, wherein the first mark information is mark information obtained by carrying out object detection on the first image, the first mark is used for marking a first object detected in the first image, and the operation of encoding the first image and the operation of carrying out object detection on the first image are operations which are executed in parallel;

Rendering and displaying the first image added with the first mark.

2. The method according to claim 1, wherein the method further comprises:

3. The method according to claim 1, wherein the method further comprises:

4. A method according to claim 3, characterized in that the method further comprises:

5. The method according to claim 4, wherein the method further comprises:

6. The method according to claim 1, wherein the method further comprises:

7. The method according to claim 4 or 6, wherein said finding a decoded second image matching said first image comprises:

8. The method according to claim 4 or 6, wherein the performing marker compensation on the first image according to the second marker information to obtain the first image to which the second marker is added includes:

9. The method of claim 8, wherein the looking up in the first image a first region that matches the second region comprises:

10. The method of claim 8, wherein the method further comprises:

11. The method according to any one of claims 1 to 6, wherein performing marker compensation on the first image according to the first marker information to obtain the first image to which the first marker is added, includes:

12. An image transmission method, comprising:

acquiring a first image in an original video stream, wherein the frame number of the first image in the original video stream is a first frame number;

encoding the first image to obtain first encoded data;

performing object detection on the first image to obtain first mark information, wherein when the object detection module detects a first object in the first image, the first mark information comprises position information of the first object in the first image, and the operation of encoding the first image and the operation of performing object detection on the first image are performed in parallel;

And respectively transmitting the first coded data and the first marking information to target equipment.

13. The method according to claim 12, wherein the method further comprises:

acquiring the received first coded data;

decoding the first encoded data to obtain the first image;

when the received first mark information matched with the first image is found, carrying out mark compensation on the first image according to the first mark information to obtain the first image added with a first mark, wherein the first mark is used for marking the first object detected in the first image;

rendering and displaying the first image added with the first mark.

14. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program is executable by a terminal device or a computer to perform the method of any one of claims 1 to 11 or 12 to 13.

15. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the method of any one of claims 1 to 11 or 12 to 13.

16. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method according to any of the claims 1-11 or 12-13 by means of the computer program.