CN115086778A - AI processing method and device for video stream - Google Patents
AI processing method and device for video stream Download PDFInfo
- Publication number
- CN115086778A CN115086778A CN202110276815.2A CN202110276815A CN115086778A CN 115086778 A CN115086778 A CN 115086778A CN 202110276815 A CN202110276815 A CN 202110276815A CN 115086778 A CN115086778 A CN 115086778A
- Authority
- CN
- China
- Prior art keywords
- processing
- target
- processing result
- video frame
- message
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 293
- 238000000034 method Methods 0.000 claims abstract description 48
- 238000004590 computer program Methods 0.000 claims description 8
- 238000013473 artificial intelligence Methods 0.000 description 165
- 238000001514 detection method Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000012795 verification Methods 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/637—Control signals issued by the client directed to the server or network components
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/65—Transmission of management data between client and server
- H04N21/658—Transmission by the client directed to the server
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention provides an AI processing method and a device of a video stream, wherein the method comprises the following steps: acquiring an identifier of a target AI algorithm configured by a user; acquiring a target video frame from the acquired video stream according to the requirement of the target AI algorithm; generating a request message, wherein the request message at least comprises: an identification of the target video frame and the target AI algorithm; the request message is sent to an AI processing module for AI processing; obtaining a processing result message returned by the AI processing module, wherein the processing result message at least comprises: and processing the result data. In the invention, the user can flexibly configure the required AI algorithm according to the requirement and carry the configured AI algorithm identifier in the request message, so that the AI processing module can execute the corresponding AI algorithm processing to meet various AI processing requirements of the user on the video.
Description
Technical Field
The embodiment of the invention relates to the technical field of information, in particular to an AI processing method and device of a video stream.
Background
In scenes such as an intelligent park and an industrial park, video monitoring and AI (Artificial Intelligence) processing are indispensable functions. Different enterprise users can put forward various AI processing requirements of the video stream according to respective requirements, such as safety helmet wearing detection, safety shoe wearing detection, goggles detection, firework detection or dangerous area entrance detection. How to implement AI processing of a video stream according to various AI processing requirements of users is a problem to be solved.
Disclosure of Invention
Embodiments of the present invention provide an AI processing method and an AI processing device for a video stream, which are used to solve the problem of how to implement AI processing of a video stream according to various AI processing requirements of a user.
In order to solve the technical problem, the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides an AI processing method for a video stream, including:
acquiring an identifier of a target AI algorithm configured by a user;
acquiring a target video frame from the acquired video stream according to the requirement of the target AI algorithm;
generating a request message, wherein the request message at least comprises: an identification of the target video frame and the target AI algorithm;
the request message is sent to an AI processing module for AI processing;
acquiring a processing result message returned by the AI processing module, wherein the processing result message at least comprises: and processing the result data.
Optionally, the sending the request packet to an AI processing module for AI processing includes: the request message is sent to an AI processing module through a message queue for AI processing;
the obtaining of the processing result message returned by the AI processing module includes: and acquiring a processing result message returned by the AI processing module from the message queue in a subscription mode.
Optionally, the request packet and the processing result packet further include: identifying a task; before obtaining the processing result message returned by the AI processing module, the method further includes:
and subscribing a processing result by taking the task identifier as a topic.
Optionally, the request packet further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, indication information of whether an extension field exists or not, indication information of whether a check field exists or not, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the AI processing module comprises an identifier of an AI processing module, an identifier of a manufacturer, a length of a processing result, a format of the processing result, indication information of whether an extension field exists, indication information of whether a check field exists, a length of the extension field, the extension field and a check bit.
Optionally, after acquiring the target video frame from the captured video stream according to the AI algorithm, the method further includes: caching the target video frame;
after the processing result message returned by the AI processing module is obtained, the method further comprises the following steps: merging the cached target video frame and the processing result data to obtain a new video frame; and returning the new video frame to the user.
In a second aspect, an embodiment of the present invention provides a method for processing an AI of a video stream, including:
obtaining a request message in a subscription mode, wherein the request message at least comprises: identification of a target video frame and a target AI algorithm;
acquiring a corresponding target AI algorithm according to the identification of the target AI algorithm, and processing the target video frame by adopting the acquired target AI algorithm to obtain processing result data;
generating a processing result message, wherein the processing result message at least comprises: processing the result data;
and returning the processing result message.
Optionally, the obtaining the request packet in a subscription manner includes: acquiring a request message from a message queue in a subscription mode;
returning the processing result message comprises: and sending the processing result message to the message queue.
Optionally, the request packet and the processing result packet further include: and identifying the task.
Optionally, the request packet further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, indication information of whether an extension field exists or not, indication information of whether a check field exists or not, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the AI processing module comprises an identifier of an AI processing module, an identifier of a manufacturer, a length of a processing result, a format of the processing result, indication information of whether an extension field exists, indication information of whether a check field exists, a length of the extension field, the extension field and a check bit.
In a third aspect, an embodiment of the present invention provides an AI processing apparatus for a video stream, including:
the first acquisition module is used for acquiring the identifier of a target AI algorithm configured by a user;
the second acquisition module is used for acquiring a target video frame from the acquired video stream according to the requirement of the target AI algorithm;
a generating module, configured to generate a request packet, where the request packet at least includes: an identification of the target video frame and the target AI algorithm;
the distribution module is used for distributing the request message to an AI processing module for AI processing;
a third obtaining module, configured to obtain a processing result message returned by the AI processing module, where the processing result message at least includes: and processing the result data.
Optionally, the distribution module is configured to distribute the request packet to an AI processing module through a message queue for AI processing;
and the third obtaining module is configured to obtain, in a subscription manner, a processing result message returned by the AI processing module from the message queue.
Optionally, the request packet and the processing result packet further include: identifying a task; the device further comprises:
and the subscription module is used for subscribing the processing result by taking the task identifier as a theme.
Optionally, the request packet further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, indication information of whether an extension field exists or not, indication information of whether a check field exists or not, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the identification of the AI processing module, the identification of a manufacturer, the length of a processing result, the format of the processing result, the indication information of whether an extension field exists, the indication information of whether a check field exists, the length of the extension field, the extension field and the check bit.
Optionally, the apparatus further comprises:
the cache module is used for caching the target video frame;
the merging module is used for merging the cached target video frame and the processing result data to obtain a new video frame;
and the sending module is used for returning the new video frame to the user.
In a fourth aspect, an embodiment of the present invention provides an AI processing apparatus for a video stream, including:
an obtaining module, configured to obtain, through subscription, a request packet sent by a video processing requester, where the request packet at least includes: identification of a target video frame and a target AI algorithm;
the processing module is used for acquiring a corresponding target AI algorithm according to the identification of the target AI algorithm and processing the target video frame by adopting the acquired target AI algorithm to obtain processing result data;
a generating module, configured to generate a processing result message, where the processing result message at least includes: processing the result data;
and the sending module is used for returning the processing result message.
Optionally, the obtaining module is configured to obtain the request packet from the message queue in a subscription manner;
and the sending module is used for sending the processing result message to the message queue.
Optionally, the request packet and the processing result packet further include: and identifying the task.
Optionally, the request packet further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, indication information of whether an extension field exists or not, indication information of whether a check field exists or not, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the AI processing module comprises an identifier of an AI processing module, an identifier of a manufacturer, a length of a processing result, a format of the processing result, indication information of whether an extension field exists, indication information of whether a check field exists, a length of the extension field, the extension field and a check bit.
In a fifth aspect, an embodiment of the present invention provides a server, including: a processor, a memory and a program stored on the memory and executable on the processor, the program implementing the steps of the AI processing method of a video stream according to the first or second aspect when executed by the processor.
In a sixth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the AI processing method for video streams according to the first aspect or the second aspect.
In the embodiment of the invention, the user can flexibly configure the required AI algorithm according to the requirement and carry the configured AI algorithm identifier in the request message, so that the AI processing module can execute the corresponding AI algorithm processing to meet various AI processing requirements of the user on the video.
Drawings
Various additional advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a flowchart illustrating an AI processing method for a video stream according to an embodiment of the invention;
fig. 2 is a schematic diagram of a format of a request message according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a format of a processing result message according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating an AI processing method of a video stream according to another embodiment of the invention;
FIG. 5 is a flowchart illustrating an AI processing apparatus for video stream according to an embodiment of the invention;
FIG. 6 is a flowchart illustrating an AI processing device of a video stream according to another embodiment of the invention;
FIG. 7 is a schematic diagram of a processing system according to an embodiment of the present invention;
FIG. 8 is a schematic diagram illustrating the interaction flow of modules in the processing system according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, an embodiment of the invention provides an AI processing method for a video stream, including:
step 11: acquiring an identifier of a target AI algorithm configured by a user;
the identity of the AI algorithm may be the name or ID of the AI algorithm.
In the embodiment of the present invention, the identifiers of all AI algorithms that can be supported may be stored in the database. Additionally, a user interface may also be provided to display an identification of all AI algorithms that can be supported for user configuration.
Step 12: acquiring a target video frame from the acquired video stream according to the requirement of the target AI algorithm;
for example, if the target AI algorithm is a trajectory recognition algorithm, the algorithm needs multiple frames of consecutive video frames for analysis, and at this time, multiple frames of consecutive target video frames are obtained from the collected video stream. If the target AI algorithm is safety helmet wearing detection or safety shoe wearing detection and the like, the algorithm can be analyzed and processed by only one frame of video frame, and at the moment, one frame of target video frame is obtained from the acquired video stream.
According to the client requirement, the target video frames can be periodically acquired from the acquired video stream, for example, when the safety helmet wearing detection is performed, one frame of target video frame can be acquired from the acquired video stream every one minute to perform the safety helmet wearing detection analysis.
Step 13: generating a request message, wherein the request message at least comprises: an identification of the target video frame and the target AI algorithm;
step 14: the request message is sent to an AI processing module for AI processing;
in the embodiment of the present invention, the AI processing module may be one or more. Optionally, each AI processing module may support all the AI algorithms that can be supported and stored in the database. When there are a plurality of AI processing modules, the AI processing module for processing the request packet may be selected in consideration of the load of the AI processing module and other factors.
Step 15: obtaining a processing result message returned by the AI processing module, wherein the processing result message at least comprises: and processing the result data.
In the embodiment of the invention, the user can flexibly configure the required AI algorithm according to the requirement and carry the configured AI algorithm identifier in the request message, so that the AI processing module can execute the corresponding AI algorithm processing to meet various AI processing requirements of the user on the video.
In this embodiment of the present invention, optionally, the sending the request packet to an AI processing module for AI processing includes: distributing the request message to an AI processing module through a message queue for AI processing; namely, the request message is sent to the message queue, and then the message queue distributes the request message to be processed to the AI processing module. At this time, obtaining the processing result message returned by the AI processing module includes: and acquiring a processing result message returned by the AI processing module from the message queue in a subscription mode. That is, after the request message is sent to the message queue, the processing result message for the request message pair is subscribed to the message queue in a subscription mode, and when the message queue receives the processing result message returned by the AI processing module, the processing result message can be returned according to the subscription. Distributed, asynchronous video frame processing can be achieved using message queues.
In this embodiment of the present invention, optionally, the request packet and the processing result packet further include: identifying a task; before obtaining the processing result message returned by the AI processing module, the method further includes: and subscribing a processing result by taking the task identifier as a topic. That is, one request message and one processing result message correspond to one unique task identifier, so that the request message corresponding to the processing result message can be identified according to the task identifier. In the embodiment of the present invention, the task identifier may be a task ID.
In the embodiment of the present invention, a video frame AI Processing Protocol (MAIPP) may be used to perform video frame AI Processing, where the video frame AI Processing Protocol includes a request Message and a Processing result Message, and formats of the request Message and the Processing result Message are described in detail below.
In this embodiment of the present invention, optionally, the request packet further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, indication information of whether an extension field exists or not, indication information of whether a check field exists or not, extension field length, extension field and check bit. In the embodiment of the invention, the expansibility for processing the video frame can be improved by configuring the extension field in the request message.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating a format of a request message according to an embodiment of the present invention, where the request message includes: task identification (namely, task UUID), identification (name or ID) of AI algorithm, video frame length, video frame number, video frame format, indication information of whether an extension field or a check field exists, extension field length, extension field (optional) and check bit (optional).
Wherein:
1) the task Identifier may be a UUID (universal Unique Identifier) type, represented by 16 bytes. Because tasks are processed asynchronously, it is necessary to distinguish which processing task the return value belongs to by a unique UUID.
2) The algorithm name can be a name composed of English characters, the limited length does not exceed 32 characters, and the limited length is expressed by 32 bytes. Such as FACE _ recording, FIRE _ DETECTION, etc.
3) The video frame length is integer, can be represented by 4 bytes, and is the length of a single video frame.
4) The number of video frames is integer and can be represented by 1 byte, and when the AI algorithm needs a plurality of consecutive video frames, the corresponding number of video frames needs to be filled.
5) The video frame format may be represented by an integer of 1 byte, e.g., 1 for jpeg, 2 for png;
6) whether an extension field is needed or not can be represented by 0 or 1, and the highest bit of the byte is positioned; whether the check is needed or not can be represented by 0 or 1, and the second highest bit of the byte is set; leaving 6 bits and the next byte, and filling the length of the extension field when the extension is needed, wherein 16384 is the maximum length of the extension field not exceeding 2^ (6+ 8);
7) the data field of the video frame needs to be matched with the length field of the video frame; if there are multiple video frames, the multiple video frames are placed consecutively.
8) When an extension field exists, filling extension field data; for example, when risk detection is performed, a coordinate list of a risk area needs to be introduced into the extension field.
9) When the verification is started, filling verification bit data; the SHA128 may be used for verification.
In this embodiment of the present invention, optionally, the processing result message further includes at least one of the following: the AI processing module comprises an identifier of an AI processing module, an identifier of a manufacturer, a length of a processing result, a format of the processing result, indication information of whether an extension field exists, indication information of whether a check field exists, a length of the extension field, the extension field and a check bit.
Referring to fig. 3, fig. 3 is a schematic diagram of a format of a processing result message according to an embodiment of the present invention, where the processing result message includes: the method comprises the steps of task identification, identification of an AI processing module, identification of a manufacturer, length of a processing result, format of the processing result, indication information of whether an extension field exists, indication information of whether a check field exists, length of the extension field, the extension field and a check bit.
Wherein:
1) the task ID may be of the UUID type, represented by 16 bytes. Because tasks are processed asynchronously, a unique UUID is needed to distinguish which processing task the returned packet belongs to.
2) The identifier (ClientID, which may also be referred to as AI processing service ID) of the AI processing module is an integer, may be represented by 4 bytes, and is a client that indicates that AI processing is performed this time. The central control module can count the processing capacity and the processing amount of the Client according to the Client ID and adjust the load condition according to the counting result.
3) The vendor ID is an integer and can be represented by 4 bytes, which indicates the vendor to which the client performing the AI process belongs, and can be used to count the processing capacity and throughput of the vendor.
4) The processing result length is an integer, can be expressed by 4 bytes, and is a result length indicating the current AI processing.
5) The format of the processing result can be expressed by 1-byte integer, such as 0 for json, 1 for jpeg, and 2 for png;
6) whether an extension field is needed or not can be represented by 0 or 1, and the highest bit of the byte is positioned; if the check is needed, the check can be represented by 0 or 1, and the second high order bit of the byte is set; leaving 6 bits and the next byte, and filling the length of the extension field when the extension field is needed, wherein 16384 is the maximum length of the extension field not exceeding 2^ (6+ 8);
7) the processing result data may be JSON or picture data of the composite AI result (for example, a worker who does not wear a helmet, a worker who does not wear safety shoes, a region where fireworks are present, or a dangerous region, etc. is selected).
8) When an extension field exists, filling extension field data; for example, in performing trajectory analysis, the predicted results are returned in the extension field.
9) When the verification is started, filling verification bit data; the SHA128 may be used for verification.
In this embodiment of the present invention, optionally, after acquiring the target video frame from the acquired video stream according to the requirement of the AI algorithm, the method further includes: caching the target video frame; at this time, after obtaining the processing result message returned by the AI processing module, the method further includes: merging the cached target video frame and the processing result data to obtain a new video frame; and returning the new video frame to the user. In the merging in the embodiment of the present invention, for example, when the returned processing result data is the coordinates of the mark frame of a worker without a helmet, the coordinates of the frame are merged with the original video frame, and the mark frame is displayed in the original video frame.
Referring to fig. 4, an embodiment of the present invention further provides an AI processing method for a video stream, including:
step 41: obtaining a request message in a subscription mode, wherein the request message at least comprises: identification of a target video frame and a target AI algorithm;
step 42: acquiring a corresponding target AI algorithm according to the identification of the target AI algorithm, and processing the target video frame by adopting the acquired target AI algorithm to obtain processing result data;
step 43: generating a processing result message, wherein the processing result message at least comprises: processing the result data;
step 44: and returning the processing result message.
In the invention, a user can flexibly configure the required AI algorithm according to the requirement and carry the configured AI algorithm identifier in the request message, so that the AI processing module can execute the corresponding AI algorithm processing to meet various AI processing requirements of the user on the video.
In this embodiment of the present invention, optionally, the obtaining of the request packet in a subscription manner includes: acquiring a request message from a message queue in a subscription mode; at this time, returning the processing result message includes: and sending the processing result message to the message queue. Distributed, asynchronous video frame processing can be achieved using message queues.
In this embodiment of the present invention, optionally, the request packet and the processing result packet further include: and identifying the task.
In this embodiment of the present invention, optionally, the request packet further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, indication information of whether an extension field exists or not, indication information of whether a check field exists or not, extension field length, extension field and check bit.
In this embodiment of the present invention, optionally, the processing result message further includes at least one of the following: the AI processing module comprises an identifier of an AI processing module, an identifier of a manufacturer, a length of a processing result, a format of the processing result, indication information of whether an extension field exists, indication information of whether a check field exists, a length of the extension field, the extension field and a check bit.
Referring to fig. 5, an AI processing apparatus 50 for video stream according to an embodiment of the present invention includes:
a first obtaining module 51, configured to obtain an identifier of a target AI algorithm configured by a user;
a second obtaining module 52, configured to obtain a target video frame from the acquired video stream according to the requirement of the target AI algorithm;
a generating module 53, configured to generate a request message, where the request message at least includes: an identification of the target video frame and the target AI algorithm;
the distribution module 54 is configured to distribute the request packet to an AI processing module for AI processing;
a third obtaining module 55, configured to obtain a processing result message returned by the AI processing module, where the processing result message at least includes: and processing the result data.
Optionally, the distributing module 54 is configured to distribute the request packet to an AI processing module through a message queue for AI processing;
the third obtaining module 55 is configured to obtain, in a subscription manner, a processing result message returned by the AI processing module from the message queue.
Optionally, the request packet and the processing result packet further include: identifying a task; the device further comprises:
and the subscription module is used for subscribing the processing result by taking the task identifier as a theme.
Optionally, the request packet further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, indication information of whether an extension field exists or not, indication information of whether a check field exists or not, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the AI processing module comprises an identifier of an AI processing module, an identifier of a manufacturer, a length of a processing result, a format of the processing result, indication information of whether an extension field exists, indication information of whether a check field exists, a length of the extension field, the extension field and a check bit.
Optionally, the apparatus further comprises:
the cache module is used for caching the target video frame;
the merging module is used for merging the cached target video frame and the processing result data to obtain a new video frame;
and the sending module is used for returning the new video frame to the user.
Referring to fig. 6, an AI processing apparatus 60 for video stream according to an embodiment of the present invention includes:
an obtaining module 61, configured to obtain, by subscription, a request packet sent by a video processing requester, where the request packet at least includes: identification of a target video frame and a target AI algorithm;
the processing module 62 is configured to obtain a corresponding target AI algorithm according to the identifier of the target AI algorithm, and process the target video frame by using the obtained target AI algorithm to obtain processing result data;
a generating module 63, configured to generate a processing result message, where the processing result message at least includes: processing the result data;
and a sending module 64, configured to return the processing result message.
Optionally, the obtaining module 61 is configured to obtain the request packet from the message queue in a subscription manner;
the sending module 64 is configured to send the processing result message to the message queue.
Optionally, the request packet and the processing result packet further include: and identifying the task.
Optionally, the request packet further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, indication information of whether an extension field exists or not, indication information of whether a check field exists or not, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the AI processing module comprises an identifier of an AI processing module, an identifier of a manufacturer, a length of a processing result, a format of the processing result, indication information of whether an extension field exists, indication information of whether a check field exists, a length of the extension field, the extension field and a check bit.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a processing system according to an embodiment of the present invention, the processing system including: the central control module is responsible for processing the video stream, processing the video frames and performing the overall control process, i.e. executing the functions executed by the AI processing device 50 of the video stream in fig. 5, the database is used for storing an identifier list of the rumor AI algorithm supported by the system, the buffer module is used for buffering the video frames, and the central control module interacts with the AI processing module through the message queue.
Referring to fig. 8, fig. 8 is a schematic view illustrating an interaction flow of modules in a processing system according to an embodiment of the present invention, where the flow includes:
step 81: the central control module inquires out the identification of the target AI algorithm configured by the user from the database;
step 82: the central control module takes one or more frames of target video frames (track identification and the like need the multiple frames of video frames) from the video stream collected by the camera according to the requirements of a target AI algorithm;
step 83: the central control module caches the acquired target video frame to a cache module; the cache module can be a local cache module or a cloud cache module;
step 84: the central control module generates a request message, wherein the request message comprises: the specific format of the target video frame, the name of the target AI algorithm, the task ID, etc. can be seen in fig. 2;
step 85: the central control module sends the request message to a message queue and subscribes a processing result to the message queue by taking the task ID as a theme;
step 86: the AI processing module acquires a request message from the message queue through subscription;
step 87: the AI processing module acquires the target video frame to be processed and the identifier of the AI algorithm to be operated by analyzing the request message;
step 88: the AI processing module executes a corresponding target AI algorithm on the target video frame and generates a processing result message;
step 89: the AI processing module pushes a processing result message to a message queue;
step 810: the central control module acquires a processing result message through the message queue;
step 811: the central control module acquires the original video frame from the cache again;
step 812: the central control module combines the processing result with the original video frame to form a new video frame;
step 813: the central control module returns the new video frame to the user or other modules.
The method of the invention has the following advantages:
1. the method can flexibly configure the AI algorithm and the video frame format according to the user requirements, is fast adapted to various AI algorithm manufacturers, has better flexibility and adaptability, and can shorten the time for originally adapting to the AI manufacturers by more than 50%.
2. Based on the message queue and MAIPP protocol video stream AI processing process, the method can meet the diversified AI processing requirements of users and realize the video stream AI analysis processing with dynamic extension and retraction in minutes and second level through the message queue. Referring to fig. 9, an embodiment of the present invention further provides a server 90, including a processor 91, a memory 92, and a computer program stored in the memory 92 and capable of running on the processor 91, where the computer program, when executed by the processor 91, implements the processes of the above-mentioned AI processing method for video streams, and can achieve the same technical effects, and therefore, for avoiding repetition, no further description is provided here.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when executed by a processor, the computer program implements each process of the above-mentioned AI processing method for video streams, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (15)
1. An AI processing method for a video stream, comprising:
acquiring an identifier of a target AI algorithm configured by a user;
acquiring a target video frame from the acquired video stream according to the requirement of the target AI algorithm;
generating a request message, wherein the request message at least comprises: an identification of the target video frame and the target AI algorithm;
the request message is sent to an AI processing module for AI processing;
obtaining a processing result message returned by the AI processing module, wherein the processing result message at least comprises: and processing the result data.
2. The method of claim 1,
the step of sending the request message to an AI processing module for AI processing comprises the following steps: the request message is sent to an AI processing module through a message queue for AI processing;
the obtaining of the processing result message returned by the AI processing module includes: and acquiring a processing result message returned by the AI processing module from the message queue in a subscription mode.
3. The method of claim 1, wherein the request message and the processing result message further comprise: identifying a task; before obtaining the processing result message returned by the AI processing module, the method further includes:
and subscribing a processing result by taking the task identifier as a topic.
4. The method according to claim 1 or 3, wherein the request message further comprises at least one of: the method comprises the steps of video frame length, video frame number, video frame format, indication information of whether an extension field exists or not, indication information of whether a check field exists or not, extension field length, extension field and check bit.
5. The method according to claim 1 or 3, wherein the processing result message further comprises at least one of the following: the AI processing module comprises an identifier of an AI processing module, an identifier of a manufacturer, a length of a processing result, a format of the processing result, indication information of whether an extension field exists, indication information of whether a check field exists, a length of the extension field, the extension field and a check bit.
6. The method of claim 1,
the method also comprises the following steps after the target video frame is acquired from the acquired video stream according to the requirements of the AI algorithm: caching the target video frame;
after the processing result message returned by the AI processing module is obtained, the method further comprises the following steps: merging the cached target video frame and the processing result data to obtain a new video frame; and returning the new video frame to the user.
7. An AI processing method for a video stream, comprising:
obtaining a request message in a subscription mode, wherein the request message at least comprises: identification of a target video frame and a target AI algorithm;
acquiring a corresponding target AI algorithm according to the identification of the target AI algorithm, and processing the target video frame by adopting the acquired target AI algorithm to obtain processing result data;
generating a processing result message, wherein the processing result message at least comprises: processing the result data;
and returning the processing result message.
8. The method of claim 7,
the obtaining of the request message in a subscription manner includes: acquiring a request message from a message queue in a subscription mode;
returning the processing result message comprises: and sending the processing result message to the message queue.
9. The method of claim 7, wherein the request message and the processing result message further comprise: and identifying the task.
10. The method according to claim 7 or 9, wherein the request message further comprises at least one of: the method comprises the steps of video frame length, video frame number, video frame format, indication information of whether an extension field exists or not, indication information of whether a check field exists or not, extension field length, extension field and check bit.
11. The method according to claim 7 or 9, wherein the processing result message further comprises at least one of the following: the AI processing module comprises an identifier of an AI processing module, an identifier of a manufacturer, a length of a processing result, a format of the processing result, indication information of whether an extension field exists, indication information of whether a check field exists, a length of the extension field, the extension field and a check bit.
12. An AI processing apparatus of a video stream, comprising:
the first acquisition module is used for acquiring the identifier of a target AI algorithm configured by a user;
the second acquisition module is used for acquiring a target video frame from the acquired video stream according to the requirement of the target AI algorithm;
a generating module, configured to generate a request packet, where the request packet at least includes: an identification of the target video frame and the target AI algorithm;
the distribution module is used for distributing the request message to an AI processing module for AI processing;
a third obtaining module, configured to obtain a processing result message returned by the AI processing module, where the processing result message at least includes: and processing the result data.
13. An AI processing apparatus of a video stream, comprising:
an obtaining module, configured to obtain, through subscription, a request packet sent by a video processing requester, where the request packet at least includes: identification of a target video frame and a target AI algorithm;
the processing module is used for acquiring a corresponding target AI algorithm according to the identification of the target AI algorithm and processing the target video frame by adopting the acquired target AI algorithm to obtain processing result data;
a generating module, configured to generate a processing result message, where the processing result message at least includes: processing the result data;
and the sending module is used for returning the processing result message.
14. A server, comprising: processor, memory and program stored on the memory and executable on the processor, which program, when executed by the processor, carries out the steps of the AI processing method of a video stream according to any one of claims 1 to 6 or which program, when executed by the processor, carries out the steps of the AI processing method of a video stream according to any one of claims 7 to 11.
15. A computer-readable storage medium, characterized in that a computer program is stored thereon, which, when being executed by a processor, implements the steps of the AI processing method of a video stream according to any one of claims 1 to 6; alternatively, the computer program when executed by a processor implements the steps of the AI processing method of a video stream according to any one of claims 7 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110276815.2A CN115086778B (en) | 2021-03-15 | 2021-03-15 | AI processing method and device for video stream |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110276815.2A CN115086778B (en) | 2021-03-15 | 2021-03-15 | AI processing method and device for video stream |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115086778A true CN115086778A (en) | 2022-09-20 |
| CN115086778B CN115086778B (en) | 2024-05-24 |
Family
ID=83240683
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110276815.2A Active CN115086778B (en) | 2021-03-15 | 2021-03-15 | AI processing method and device for video stream |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115086778B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160286244A1 (en) * | 2015-03-27 | 2016-09-29 | Twitter, Inc. | Live video streaming services |
| JP2017085196A (en) * | 2015-10-22 | 2017-05-18 | シャープ株式会社 | Receiving device, broadcasting system, reception method, and reception program |
| CN107222427A (en) * | 2016-03-22 | 2017-09-29 | 华为技术有限公司 | The method and relevant device of a kind of Message processing |
| CN107547938A (en) * | 2016-06-29 | 2018-01-05 | 北京信威通信技术股份有限公司 | A kind of method, apparatus and system for analyzing video data |
| US20180131975A1 (en) * | 2016-11-09 | 2018-05-10 | Charter Communications Operating, Llc | Apparatus and methods for selective secondary content insertion in a digital network |
| US20180176526A1 (en) * | 2016-12-19 | 2018-06-21 | Thomson Licensing | Method and apparatus for contextual color remapping transformation in content rendering |
| CN108319926A (en) * | 2018-02-12 | 2018-07-24 | 安徽金禾软件股份有限公司 | A kind of the safety cap wearing detecting system and detection method of building-site |
| CN109168031A (en) * | 2018-11-06 | 2019-01-08 | 杭州云英网络科技有限公司 | Streaming Media method for pushing and device, steaming media platform |
| CN109218752A (en) * | 2017-06-30 | 2019-01-15 | 中国电信股份有限公司 | Video playing, offer and detection method and set-top box, media server and system |
| CN110545242A (en) * | 2018-05-29 | 2019-12-06 | 杭州海康威视数字技术股份有限公司 | target analysis method and intelligent analysis equipment |
| CN111310638A (en) * | 2019-12-31 | 2020-06-19 | 深圳云天励飞技术有限公司 | Data processing method and device and computer readable storage medium |
| CN112001351A (en) * | 2020-09-01 | 2020-11-27 | 城云科技（中国）有限公司 | Method, system, computer device and storage medium for processing multiple video streams |
-
2021
- 2021-03-15 CN CN202110276815.2A patent/CN115086778B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160286244A1 (en) * | 2015-03-27 | 2016-09-29 | Twitter, Inc. | Live video streaming services |
| JP2017085196A (en) * | 2015-10-22 | 2017-05-18 | シャープ株式会社 | Receiving device, broadcasting system, reception method, and reception program |
| CN107222427A (en) * | 2016-03-22 | 2017-09-29 | 华为技术有限公司 | The method and relevant device of a kind of Message processing |
| CN107547938A (en) * | 2016-06-29 | 2018-01-05 | 北京信威通信技术股份有限公司 | A kind of method, apparatus and system for analyzing video data |
| US20180131975A1 (en) * | 2016-11-09 | 2018-05-10 | Charter Communications Operating, Llc | Apparatus and methods for selective secondary content insertion in a digital network |
| US20180176526A1 (en) * | 2016-12-19 | 2018-06-21 | Thomson Licensing | Method and apparatus for contextual color remapping transformation in content rendering |
| CN109218752A (en) * | 2017-06-30 | 2019-01-15 | 中国电信股份有限公司 | Video playing, offer and detection method and set-top box, media server and system |
| CN108319926A (en) * | 2018-02-12 | 2018-07-24 | 安徽金禾软件股份有限公司 | A kind of the safety cap wearing detecting system and detection method of building-site |
| CN110545242A (en) * | 2018-05-29 | 2019-12-06 | 杭州海康威视数字技术股份有限公司 | target analysis method and intelligent analysis equipment |
| CN109168031A (en) * | 2018-11-06 | 2019-01-08 | 杭州云英网络科技有限公司 | Streaming Media method for pushing and device, steaming media platform |
| CN111310638A (en) * | 2019-12-31 | 2020-06-19 | 深圳云天励飞技术有限公司 | Data processing method and device and computer readable storage medium |
| CN112001351A (en) * | 2020-09-01 | 2020-11-27 | 城云科技（中国）有限公司 | Method, system, computer device and storage medium for processing multiple video streams |
Non-Patent Citations (3)
| Title |
|---|
| RAQUEL RAMOS LÓPEZ ET AL.: ""Digital Video Source Identification Based on Container’s Structure Analysis"", 《IEEE ACCESS》 * |
| 朱明凌: ""面向智能视频监控系统的运动目标检测与异常行为分析"", 《中国学位论文全文数据库》 * |
| 谢洪茂: ""智能视频分析服务器集群设计与实现"", 《中国优秀硕士学位论文全文数据库（电子期刊）》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115086778B (en) | 2024-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111510395B (en) | Service message reporting method, device, equipment and medium | |
| CN109672580B (en) | Full link monitoring method, device, terminal equipment and storage medium | |
| CN110826799B (en) | Service prediction method, device, server and readable storage medium | |
| CN110753112A (en) | Elastic expansion method and device of cloud service | |
| JP4205323B2 (en) | Distribution system, distribution server and distribution method, distribution program | |
| EP3267323A1 (en) | Thin client system, connection management device, virtual machine operating device, method, and storage medium | |
| CN111861572A (en) | Advertisement putting method and device, electronic equipment and computer readable storage medium | |
| CN109933610B (en) | Data processing method, device, computer equipment and storage medium | |
| CN112165436A (en) | Flow control method, device and system | |
| CN111917586A (en) | Container bandwidth adjusting method, server and storage medium | |
| CN109144782B (en) | Data recovery method and device | |
| CN111328032B (en) | Method and device for sending video short message | |
| CN113422808B (en) | Internet of things platform HTTP information pushing method, system, device and medium | |
| CN113329139A (en) | Video stream processing method, device and computer readable storage medium | |
| CN109150927B (en) | File issuing method and device for file storage system | |
| CN112988337A (en) | Task processing system, method, device, electronic equipment and storage medium | |
| CN115086778A (en) | AI processing method and device for video stream | |
| CN111277626A (en) | Server upgrading method and device, electronic equipment and medium | |
| CN106293996B (en) | Method of data synchronization, main equipment, alternate device and system | |
| CN108989244B (en) | Data processing method, data processing device, storage medium and electronic equipment | |
| CN108924653B (en) | Bullet screen message distribution method, device, equipment and storage medium | |
| CN110610164A (en) | Face image processing method, system, server and readable storage medium | |
| CN113055493B (en) | Data packet processing method, device, system, scheduling device and storage medium | |
| CN114285906B (en) | Message processing method and device, electronic equipment and storage medium | |
| CN115686746A (en) | Access method, task processing method, computing device, and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |