CN115086778B - AI processing method and device for video stream - Google Patents
AI processing method and device for video stream Download PDFInfo
- Publication number
- CN115086778B CN115086778B CN202110276815.2A CN202110276815A CN115086778B CN 115086778 B CN115086778 B CN 115086778B CN 202110276815 A CN202110276815 A CN 202110276815A CN 115086778 B CN115086778 B CN 115086778B
- Authority
- CN
- China
- Prior art keywords
- processing
- message
- target
- module
- processing result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 21
- 238000000034 method Methods 0.000 claims abstract description 50
- 238000004590 computer program Methods 0.000 claims description 7
- 239000000758 substrate Substances 0.000 claims 1
- 238000013473 artificial intelligence Methods 0.000 description 164
- 238000001514 detection method Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 4
- 230000003139 buffering effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000000779 smoke Substances 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 231100001261 hazardous Toxicity 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/637—Control signals issued by the client directed to the server or network components
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/65—Transmission of management data between client and server
- H04N21/658—Transmission by the client directed to the server
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention provides an AI processing method and device for video stream, the method includes: acquiring the identification of a target AI algorithm configured by a user; acquiring a target video frame from the acquired video stream according to the requirements of the target AI algorithm; generating a request message, wherein the request message at least comprises: identification of the target video frame and the target AI algorithm; the request message is distributed to an AI processing module for AI processing; obtaining a processing result message returned by the AI processing module, wherein the processing result message at least comprises: and processing the result data. In the invention, a user can flexibly configure a required AI algorithm according to the requirement, and the request message carries the identifier of the configured AI algorithm, so that the AI processing module can execute corresponding AI algorithm processing to meet various AI processing requirements of the user on videos.
Description
Technical Field
The embodiment of the invention relates to the technical field of information, in particular to an AI processing method and device for video stream.
Background
In the scenes of smart parks, industrial parks, etc., video monitoring and AI (ARTIFICIAL INTELLIGENCE ) processing are indispensable functions. Different enterprise users may set forth a variety of AI processing requirements for video streams for respective needs, such as headgear wear detection, safety shoe wear detection, goggles detection, smoke detection, or hazardous area entry detection, etc. How to implement AI processing of video streams according to various AI processing demands of users is a problem to be solved.
Disclosure of Invention
The embodiment of the invention provides an AI processing method and device for video streams, which are used for solving the problem of how to realize the AI processing of the video streams according to various AI processing requirements of users.
In order to solve the technical problems, the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides an AI processing method for a video stream, including:
acquiring the identification of a target AI algorithm configured by a user;
Acquiring a target video frame from the acquired video stream according to the requirements of the target AI algorithm;
Generating a request message, wherein the request message at least comprises: identification of the target video frame and the target AI algorithm;
the request message is distributed to an AI processing module for AI processing;
Obtaining a processing result message returned by the AI processing module, wherein the processing result message at least comprises: and processing the result data.
Optionally, distributing the request packet to an AI processing module for AI processing includes: distributing the request message to an AI processing module through a message queue for AI processing;
The step of obtaining the processing result message returned by the AI processing module comprises the following steps: and acquiring a processing result message returned by the AI processing module from the message queue in a subscription mode.
Optionally, the request message and the processing result message further include: a task identifier; before the processing result message returned by the AI processing module is obtained, the method further comprises the following steps:
and subscribing the processing result by taking the task identifier as a theme.
Optionally, the request message further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, whether the indication information of an extension field exists, whether the indication information of a check field exists, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the AI processing module comprises an AI processing module identifier, a manufacturer identifier, a processing result length, a processing result format, whether the indicating information of an extension field exists or not, whether the indicating information of a check field exists or not, an extension field length, an extension field and a check bit.
Optionally, after the target video frame is acquired from the acquired video stream according to the requirements of the AI algorithm, the method further includes: caching the target video frame;
the process result message returned by the AI processing module is obtained, and then the process result message comprises: combining the cached target video frame with the processing result data to obtain a new video frame; and returning the new video frame to the user.
In a second aspect, an embodiment of the present invention provides an AI processing method for a video stream, including:
acquiring a request message in a subscription mode, wherein the request message at least comprises: identification of a target video frame and a target AI algorithm;
Acquiring a corresponding target AI algorithm according to the identification of the target AI algorithm, and processing the target video frame by adopting the acquired target AI algorithm to obtain processing result data;
Generating a processing result message, wherein the processing result message at least comprises: processing the result data;
And returning the processing result message.
Optionally, obtaining the request message by a subscription method includes: acquiring a request message from a message queue in a subscription mode;
the step of returning the processing result message comprises the following steps: and sending the processing result message to the message queue.
Optionally, the request message and the processing result message further include: and (5) task identification.
Optionally, the request message further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, whether the indication information of an extension field exists, whether the indication information of a check field exists, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the AI processing module comprises an AI processing module identifier, a manufacturer identifier, a processing result length, a processing result format, whether the indicating information of an extension field exists or not, whether the indicating information of a check field exists or not, an extension field length, an extension field and a check bit.
In a third aspect, an embodiment of the present invention provides an AI processing apparatus for a video stream, including:
The first acquisition module is used for acquiring the identification of a target AI algorithm configured by a user;
the second acquisition module is used for acquiring a target video frame from the acquired video stream according to the requirements of the target AI algorithm;
the generating module is used for generating a request message, and the request message at least comprises: identification of the target video frame and the target AI algorithm;
The distribution module is used for distributing the request message to the AI processing module for AI processing;
The third obtaining module is configured to obtain a processing result message returned by the AI processing module, where the processing result message at least includes: and processing the result data.
Optionally, the distributing module is configured to distribute the request packet to an AI processing module through a message queue for AI processing;
the third obtaining module is configured to obtain, from the message queue, a processing result packet returned by the AI processing module in a subscription manner.
Optionally, the request message and the processing result message further include: a task identifier; the apparatus further comprises:
and the subscription module is used for subscribing the processing result by taking the task identifier as a theme.
Optionally, the request message further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, whether the indication information of an extension field exists, whether the indication information of a check field exists, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the AI processing module comprises an AI processing module identifier, a manufacturer identifier, a processing result length, a processing result format, whether the indicating information of an extension field exists or not, whether the indicating information of a check field exists or not, an extension field length, an extension field and a check bit.
Optionally, the apparatus further includes:
The buffer module is used for buffering the target video frame;
The merging module is used for merging the cached target video frame with the processing result data to obtain a new video frame;
And the sending module is used for returning the new video frame to the user.
In a fourth aspect, an embodiment of the present invention provides an AI processing apparatus for a video stream, including:
The acquisition module is used for acquiring a request message sent by a video processing requester through subscription, wherein the request message at least comprises: identification of a target video frame and a target AI algorithm;
The processing module is used for acquiring a corresponding target AI algorithm according to the identification of the target AI algorithm, and processing the target video frame by adopting the acquired target AI algorithm to obtain processing result data;
The generating module is used for generating a processing result message, and the processing result message at least comprises: processing the result data;
and the sending module is used for returning the processing result message.
Optionally, the acquiring module is configured to acquire the request message from the message queue in a subscription manner;
The sending module is used for sending the processing result message to the message queue.
Optionally, the request message and the processing result message further include: and (5) task identification.
Optionally, the request message further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, whether the indication information of an extension field exists, whether the indication information of a check field exists, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the AI processing module comprises an AI processing module identifier, a manufacturer identifier, a processing result length, a processing result format, whether the indicating information of an extension field exists or not, whether the indicating information of a check field exists or not, an extension field length, an extension field and a check bit.
In a fifth aspect, an embodiment of the present invention provides a server, including: a processor, a memory, and a program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the AI processing method for a video stream of the first aspect or the second aspect.
In a sixth aspect, an embodiment of the present invention provides a computer readable storage medium, where a computer program is stored, where the computer program is executed by a processor to implement the steps of the AI processing method for video streams in the first aspect or the second aspect.
In the embodiment of the invention, a user can flexibly configure the required AI algorithm according to the requirement, and the request message carries the identifier of the configured AI algorithm, so that the AI processing module can execute corresponding AI algorithm processing to meet various AI processing requirements of the user on the video.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flowchart illustrating an AI processing method for video stream according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a format of a request message according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a format of a processing result message according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating an AI processing method for video stream according to another embodiment of the invention;
FIG. 5 is a flowchart of an AI processing apparatus for video stream according to an embodiment of the invention;
Fig. 6 is a flowchart of an AI processing device for video stream according to another embodiment of the invention;
FIG. 7 is a schematic diagram of a processing system according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of an interaction flow of modules within a processing system according to an embodiment of the present invention;
Fig. 9 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, an embodiment of the present invention provides an AI processing method for a video stream, including:
Step 11: acquiring the identification of a target AI algorithm configured by a user;
the identification of the AI algorithm may be the name or ID of the AI algorithm.
In the embodiment of the invention, the identification of all the AI algorithms which can be supported can be stored in the database. In addition, the user interface may also be provided to display the identity of all AI algorithms that can be supported for configuration by the user.
Step 12: acquiring a target video frame from the acquired video stream according to the requirements of the target AI algorithm;
For example, if the target AI algorithm is a track recognition algorithm, the algorithm needs multiple frames of continuous video frames to perform analysis processing, and at this time, multiple frames of continuous target video frames are acquired from the acquired video stream. If the target AI algorithm is a helmet wearing detection or a safety shoe wearing detection, the algorithm can analyze and process only one frame of video frame, and at the moment, one frame of target video frame is obtained from the collected video stream.
According to the customer demand, can obtain the goal video frame from gathering the video stream periodically, for example when carrying on the wearing detection of the helmet, can obtain a frame goal video frame from gathering the video stream every one minute and carry on the wearing detection analysis of the helmet.
Step 13: generating a request message, wherein the request message at least comprises: identification of the target video frame and the target AI algorithm;
step 14: the request message is distributed to an AI processing module for AI processing;
In the embodiment of the invention, the AI processing module may be one or more. Alternatively, each AI processing module may support all AI algorithms stored in the database that can be supported. When there are a plurality of AI processing modules, the AI processing module for processing the request message may be selected in consideration of factors such as the load of the AI processing module.
Step 15: obtaining a processing result message returned by the AI processing module, wherein the processing result message at least comprises: and processing the result data.
In the embodiment of the invention, a user can flexibly configure the required AI algorithm according to the requirement, and the request message carries the identifier of the configured AI algorithm, so that the AI processing module can execute corresponding AI algorithm processing to meet various AI processing requirements of the user on the video.
In the embodiment of the present invention, optionally, distributing the request packet to the AI processing module for AI processing includes: distributing the request message to an AI processing module through a message queue for AI processing; that is, the request message is sent to the message queue, and then the request message to be processed is distributed to the AI processing module by the message queue. At this time, the obtaining the processing result message returned by the AI processing module includes: and acquiring a processing result message returned by the AI processing module from the message queue in a subscription mode. That is, after the request message is sent to the message queue, the message queue is subscribed to the processing result message for the request message by a subscription mode, and when the message queue receives the processing result message returned by the AI processing module, the processing result message can be returned according to the subscription. Distributed, asynchronous video frame processing can be achieved using message queues.
In the embodiment of the present invention, optionally, the request message and the processing result message further include: a task identifier; before the processing result message returned by the AI processing module is obtained, the method further comprises the following steps: and subscribing the processing result by taking the task identifier as a theme. Namely, a request message and a processing result message correspond to a unique task identifier, so that the request message corresponding to the processing result message can be identified according to the task identifier. In the embodiment of the invention, the task identifier may be a task ID.
In the embodiment of the present invention, a video frame AI process protocol (MAIPP, message queue based AI Processing Protocol) may be used to perform a video frame AI process, where the video frame AI process protocol includes a request packet and a processing result packet, and the formats of the request packet and the processing result packet are described in detail below.
In an embodiment of the present invention, optionally, the request message further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, whether the indication information of an extension field exists, whether the indication information of a check field exists, extension field length, extension field and check bit. In the embodiment of the invention, the expansibility of video frame processing can be improved by configuring the expansion field in the request message.
Referring to fig. 2, fig. 2 is a schematic diagram of a format of a request message according to an embodiment of the present invention, where the request message includes: task identification (i.e., task UUID), identification (name or ID) of AI algorithm, video frame length, number of video frames, video frame format, indication of whether there is an extension field or check field, extension field length, extension field (optional) and check bit (optional).
Wherein:
1) The task identity may be a UUID (Universally Unique Identifier, universally unique identification code) type, expressed in 16 bytes. Because tasks are processed asynchronously, it is necessary to distinguish to which processing task the return value belongs by a unique UUID.
2) The algorithm name can be a name consisting of English characters, the limiting length is not more than 32 characters, and the algorithm name is expressed by 32 bytes. Such as FACE _ RECOGNITION, FIRE _ DETECTION, etc.
3) The video frame length is integer, which can be represented by 4 bytes, representing the length of a single video frame.
4) The number of video frames is integer and can be represented by 1 byte, and when the AI algorithm requires a number of consecutive video frames, the corresponding number of video frames needs to be filled.
5) The video frame format may be represented by an integer of 1 byte, e.g., 1 represents jpeg and 2 represents png;
6) Whether an extension field is required or not can be represented by 0 or 1, and the most significant bit of the byte is set; whether the check is needed or not can be represented by 0 or 1, and the second highest bit of the byte is set; the remaining 6 bits and the next byte, when expansion is needed, filling the length of the expansion field, which is 16384, which is not more than 2 (6+8);
7) The data field of the video frame needs to be matched with the length field of the video frame; if there are multiple video frames, then the multiple video frames are placed in succession.
8) Filling extension field data when an extension field exists; for example, in the case of hazard recognition, a coordinate list of the hazard area is entered in the extension field.
9) Filling check bit data when checking is started; the verification may be performed using SHA 128.
In an embodiment of the present invention, optionally, the processing result packet further includes at least one of the following: the AI processing module comprises an AI processing module identifier, a manufacturer identifier, a processing result length, a processing result format, whether the indicating information of an extension field exists or not, whether the indicating information of a check field exists or not, an extension field length, an extension field and a check bit.
Referring to fig. 3, fig. 3 is a schematic diagram of a format of a processing result message according to an embodiment of the present invention, where the processing result message includes: the method comprises the steps of task identification, identification of an AI processing module, identification of a manufacturer, processing result length, processing result format, whether the indicating information of an extension field exists or not, whether the indicating information of a check field exists or not, extension field length, extension field and check bit.
Wherein:
1) The task ID may be a UUID type, represented in 16 bytes. Because tasks are processed asynchronously, it is necessary to distinguish to which processing task the returned message belongs by a unique UUID.
2) The identifier (ClientID, which may also be referred to as AI-processing service ID) of the AI-processing module is an integer, which may be represented by 4 bytes, and is a client that performs AI processing this time. The central control module can count the processing capacity and the processing capacity of the clients according to the clients ID, and adjust the load condition according to the statistics result.
3) The vendor ID is an integer and can be expressed by 4 bytes, and represents the vendor to which the client performing the AI processing belongs, and can be used to count the processing capacity and throughput of the vendor.
4) The processing result length is integer, which can be expressed by 4 bytes, and is the result length of the current AI processing.
5) The processing result format may be expressed in terms of an integer of 1 byte, for example, 0 represents json,1 represents jpeg, and 2 represents png;
6) Whether an extension field is needed or not, which can be represented by 0 or 1, the most significant bit of the byte is set; whether checking is needed or not can be represented by 0 or 1, and the second highest bit of the byte is set; leaving 6 bits and the next byte, filling the length of the extension field when the extension field is needed, wherein the length is not more than 2 (6+8), namely 16384 at most;
7) The processing result data may be JSON or picture data of a composite AI result (for example, a worker without a helmet is selected in a box, a worker without safety shoes is selected in a box, a region where smoke exists is selected in a box, a dangerous region is selected in a box, or the like).
8) Filling extension field data when an extension field exists; for example, when track analysis is performed, a predicted result is returned in the extension field.
9) Filling check bit data when checking is started; the verification may be performed using SHA 128.
In the embodiment of the present invention, optionally, after obtaining the target video frame from the collected video stream according to the requirements of the AI algorithm, the method further includes: caching the target video frame; at this time, after obtaining the processing result message returned by the AI processing module, the method further includes: combining the cached target video frame with the processing result data to obtain a new video frame; and returning the new video frame to the user. In the embodiment of the invention, for example, when the returned processing result data is the coordinates of the marked frame of the worker without the safety helmet, the coordinates of the frame are combined with the original video frame, and the marked frame is displayed in the original video frame.
Referring to fig. 4, the embodiment of the present invention further provides an AI processing method for a video stream, including:
Step 41: acquiring a request message in a subscription mode, wherein the request message at least comprises: identification of a target video frame and a target AI algorithm;
step 42: acquiring a corresponding target AI algorithm according to the identification of the target AI algorithm, and processing the target video frame by adopting the acquired target AI algorithm to obtain processing result data;
Step 43: generating a processing result message, wherein the processing result message at least comprises: processing the result data;
step 44: and returning the processing result message.
In the invention, a user can flexibly configure a required AI algorithm according to the requirement, and the request message carries the identifier of the configured AI algorithm, so that the AI processing module can execute corresponding AI algorithm processing to meet various AI processing requirements of the user on videos.
In the embodiment of the present invention, optionally, obtaining the request packet by a subscription method includes: acquiring a request message from a message queue in a subscription mode; at this time, returning the processing result message includes: and sending the processing result message to the message queue. Distributed, asynchronous video frame processing can be achieved using message queues.
In the embodiment of the present invention, optionally, the request message and the processing result message further include: and (5) task identification.
In an embodiment of the present invention, optionally, the request message further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, whether the indication information of an extension field exists, whether the indication information of a check field exists, extension field length, extension field and check bit.
In an embodiment of the present invention, optionally, the processing result packet further includes at least one of the following: the AI processing module comprises an AI processing module identifier, a manufacturer identifier, a processing result length, a processing result format, whether the indicating information of an extension field exists or not, whether the indicating information of a check field exists or not, an extension field length, an extension field and a check bit.
Referring to fig. 5, an embodiment of the present invention further provides an AI processing device 50 for video streaming, including:
A first obtaining module 51, configured to obtain an identifier of a target AI algorithm configured by a user;
A second obtaining module 52, configured to obtain a target video frame from the collected video stream according to the requirement of the target AI algorithm;
the generating module 53 is configured to generate a request packet, where the request packet at least includes: identification of the target video frame and the target AI algorithm;
a distribution module 54, configured to distribute the request packet to an AI processing module for AI processing;
The third obtaining module 55 is configured to obtain a processing result packet returned by the AI processing module, where the processing result packet at least includes: and processing the result data.
Optionally, the distributing module 54 is configured to distribute the request packet to an AI processing module for AI processing through a message queue;
The third obtaining module 55 is configured to obtain, by means of subscription, the processing result packet returned by the AI processing module from the message queue.
Optionally, the request message and the processing result message further include: a task identifier; the apparatus further comprises:
and the subscription module is used for subscribing the processing result by taking the task identifier as a theme.
Optionally, the request message further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, whether the indication information of an extension field exists, whether the indication information of a check field exists, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the AI processing module comprises an AI processing module identifier, a manufacturer identifier, a processing result length, a processing result format, whether the indicating information of an extension field exists or not, whether the indicating information of a check field exists or not, an extension field length, an extension field and a check bit.
Optionally, the apparatus further includes:
The buffer module is used for buffering the target video frame;
The merging module is used for merging the cached target video frame with the processing result data to obtain a new video frame;
And the sending module is used for returning the new video frame to the user.
Referring to fig. 6, an embodiment of the present invention further provides an AI processing device 60 for video streaming, including:
the obtaining module 61 is configured to obtain, through subscription, a request packet sent by a video processing requester, where the request packet at least includes: identification of a target video frame and a target AI algorithm;
the processing module 62 is configured to obtain a corresponding target AI algorithm according to the identifier of the target AI algorithm, and process the target video frame by using the obtained target AI algorithm to obtain processing result data;
The generating module 63 is configured to generate a processing result packet, where the processing result packet at least includes: processing the result data;
And a sending module 64, configured to return the processing result message.
Optionally, the acquiring module 61 is configured to acquire, by means of subscription, a request packet from the message queue;
the sending module 64 is configured to send the processing result packet to the message queue.
Optionally, the request message and the processing result message further include: and (5) task identification.
Optionally, the request message further includes at least one of the following: the method comprises the steps of video frame length, video frame number, video frame format, whether the indication information of an extension field exists, whether the indication information of a check field exists, extension field length, extension field and check bit.
Optionally, the processing result message further includes at least one of the following: the AI processing module comprises an AI processing module identifier, a manufacturer identifier, a processing result length, a processing result format, whether the indicating information of an extension field exists or not, whether the indicating information of a check field exists or not, an extension field length, an extension field and a check bit.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a processing system according to an embodiment of the invention, the processing system includes: the central control module is responsible for processing video streams, processing video frames and overall control flow, namely executing the functions executed by the AI processing device 50 of the video streams in FIG. 5, the database is used for storing an identification list of You AI algorithms supported by the system, the buffer module is used for buffering the video frames, and the central control module interacts with the AI processing module through the message queue.
Referring to fig. 8, fig. 8 is a schematic diagram of an interaction flow of each module in the processing system according to an embodiment of the present invention, where the flow includes:
step 81: the central control module inquires the identification of a target AI algorithm configured by a user from a database;
Step 82: the central control module extracts one or more target video frames (multiple frames are needed for track recognition and the like) from a video stream acquired by a camera according to the requirements of a target AI algorithm;
Step 83: the central control module caches the acquired target video frames to the cache module; the cache module can be a local cache module or a cloud cache module;
Step 84: the central control module generates a request message, wherein the request message comprises: the specific format of the target video frame, the name of the target AI algorithm, the task ID, etc. can be seen in fig. 2;
step 85: the central control module sends the request message to the message queue, and subscribes a processing result to the message queue by taking the task ID as a theme;
step 86: the AI processing module acquires a request message from the message queue through subscription;
step 87: the AI processing module acquires the target video frame to be processed and the identification of the AI algorithm to be operated by analyzing the request message;
Step 88: the AI processing module executes a corresponding target AI algorithm on the target video frame and generates a processing result message;
step 89: the AI processing module pushes the processing result message to a message queue;
Step 810: the central control module obtains a processing result message through a message queue;
step 811: the central control module re-acquires the original video frame from the cache;
Step 812: the central control module combines the processing result with the original video frame to form a new video frame;
Step 813: the central control module returns the new video frame to the user or other module.
The method of the invention has the following advantages:
1. The method can flexibly configure the AI algorithm and the video frame format according to the user requirements, quickly adapt to various AI algorithm manufacturers, has better flexibility and adaptability, and can shorten the original adaptation time with the AI manufacturers by more than 50%.
2. Based on the message queue and MAIPP protocol video stream AI processing process, the video stream AI analysis processing of minute dynamic expansion and second level can be realized through the message queue while the user diversified AI processing requirements are met. Referring to fig. 9, the embodiment of the present invention further provides a server 90, which includes a processor 91, a memory 92, and a computer program stored in the memory 92 and capable of running on the processor 91, where the computer program when executed by the processor 91 implements the processes of the AI processing method embodiment of video stream, and the same technical effects can be achieved, and for avoiding repetition, a detailed description is omitted herein.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements each process of the AI processing method embodiment of a video stream, and can achieve the same technical effects, so that repetition is avoided, and no further description is provided herein. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.
Claims (13)
1. An AI processing method for a video stream, comprising:
acquiring the identification of a target AI algorithm configured by a user;
Acquiring a target video frame from the acquired video stream according to the requirements of the target AI algorithm;
Generating a request message, wherein the request message at least comprises: identification of the target video frame and the target AI algorithm;
the request message is distributed to an AI processing module for AI processing;
obtaining a processing result message returned by the AI processing module, wherein the processing result message at least comprises: processing the result data;
the step of distributing the request message to an AI processing module for AI processing comprises the following steps: distributing the request message to an AI processing module through a message queue for AI processing;
The step of obtaining the processing result message returned by the AI processing module comprises the following steps: and acquiring a processing result message returned by the AI processing module from the message queue in a subscription mode.
2. The method of claim 1, wherein the request message and the processing result message further comprise: a task identifier; before the processing result message returned by the AI processing module is obtained, the method further comprises the following steps:
and subscribing the processing result by taking the task identifier as a theme.
3. The method according to claim 1 or 2, wherein the request message further comprises at least one of: the method comprises the steps of video frame length, video frame number, video frame format, whether the indication information of an extension field exists, whether the indication information of a check field exists, extension field length, extension field and check bit.
4. The method according to claim 1 or 2, wherein the processing result message further comprises at least one of the following: the AI processing module comprises an AI processing module identifier, a manufacturer identifier, a processing result length, a processing result format, whether the indicating information of an extension field exists or not, whether the indicating information of a check field exists or not, an extension field length, an extension field and a check bit.
5. The method of claim 1, wherein the step of determining the position of the substrate comprises,
The method further comprises the following steps of: caching the target video frame;
the process result message returned by the AI processing module is obtained, and then the process result message comprises: combining the cached target video frame with the processing result data to obtain a new video frame; and returning the new video frame to the user.
6. An AI processing method for a video stream, comprising:
acquiring a request message in a subscription mode, wherein the request message at least comprises: identification of a target video frame and a target AI algorithm;
Acquiring a corresponding target AI algorithm according to the identification of the target AI algorithm, and processing the target video frame by adopting the acquired target AI algorithm to obtain processing result data;
Generating a processing result message, wherein the processing result message at least comprises: processing the result data;
Returning the processing result message;
The obtaining the request message through the subscription mode comprises the following steps: acquiring a request message from a message queue in a subscription mode;
the step of returning the processing result message comprises the following steps: and sending the processing result message to the message queue.
7. The method of claim 6, wherein the request message and the processing result message further comprise: and (5) task identification.
8. The method according to claim 6 or 7, wherein the request message further comprises at least one of: the method comprises the steps of video frame length, video frame number, video frame format, whether the indication information of an extension field exists, whether the indication information of a check field exists, extension field length, extension field and check bit.
9. The method according to claim 6 or 7, wherein the processing result message further comprises at least one of the following: the AI processing module comprises an AI processing module identifier, a manufacturer identifier, a processing result length, a processing result format, whether the indicating information of an extension field exists or not, whether the indicating information of a check field exists or not, an extension field length, an extension field and a check bit.
10. An AI processing apparatus for a video stream, comprising:
The first acquisition module is used for acquiring the identification of a target AI algorithm configured by a user;
the second acquisition module is used for acquiring a target video frame from the acquired video stream according to the requirements of the target AI algorithm;
the generating module is used for generating a request message, and the request message at least comprises: identification of the target video frame and the target AI algorithm;
The distribution module is used for distributing the request message to the AI processing module for AI processing;
the third obtaining module is configured to obtain a processing result message returned by the AI processing module, where the processing result message at least includes: processing the result data;
the distribution module is used for distributing the request message to the AI processing module for AI processing through a message queue;
the third obtaining module is configured to obtain, from the message queue, a processing result packet returned by the AI processing module in a subscription manner.
11. An AI processing apparatus for a video stream, comprising:
The acquisition module is used for acquiring a request message sent by a video processing requester through subscription, wherein the request message at least comprises: identification of a target video frame and a target AI algorithm;
The processing module is used for acquiring a corresponding target AI algorithm according to the identification of the target AI algorithm, and processing the target video frame by adopting the acquired target AI algorithm to obtain processing result data;
The generating module is used for generating a processing result message, and the processing result message at least comprises: processing the result data;
the sending module is used for returning the processing result message;
The acquisition module is used for acquiring a request message from the message queue in a subscription mode;
The sending module is used for sending the processing result message to the message queue.
12. A server, comprising: a processor, a memory, and a program stored on the memory and executable on the processor, the program implementing the steps of the AI processing method of a video stream according to any one of claims 1 to 5 when executed by the processor, or the program implementing the steps of the AI processing method of a video stream according to any one of claims 6 to 9 when executed by the processor.
13. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, which when executed by a processor, implements the steps of the AI-processing method of a video stream according to any one of claims 1 to 5; or the computer program when executed by a processor implements the steps of the AI processing method of a video stream as claimed in any one of claims 6 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110276815.2A CN115086778B (en) | 2021-03-15 | 2021-03-15 | AI processing method and device for video stream |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110276815.2A CN115086778B (en) | 2021-03-15 | 2021-03-15 | AI processing method and device for video stream |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115086778A CN115086778A (en) | 2022-09-20 |
| CN115086778B true CN115086778B (en) | 2024-05-24 |
Family
ID=83240683
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110276815.2A Active CN115086778B (en) | 2021-03-15 | 2021-03-15 | AI processing method and device for video stream |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115086778B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2017085196A (en) * | 2015-10-22 | 2017-05-18 | シャープ株式会社 | Receiving device, broadcasting system, reception method, and reception program |
| CN107222427A (en) * | 2016-03-22 | 2017-09-29 | 华为技术有限公司 | The method and relevant device of a kind of Message processing |
| CN107547938A (en) * | 2016-06-29 | 2018-01-05 | 北京信威通信技术股份有限公司 | A kind of method, apparatus and system for analyzing video data |
| CN108319926A (en) * | 2018-02-12 | 2018-07-24 | 安徽金禾软件股份有限公司 | A kind of the safety cap wearing detecting system and detection method of building-site |
| CN109168031A (en) * | 2018-11-06 | 2019-01-08 | 杭州云英网络科技有限公司 | Streaming Media method for pushing and device, steaming media platform |
| CN109218752A (en) * | 2017-06-30 | 2019-01-15 | 中国电信股份有限公司 | Video playing, offer and detection method and set-top box, media server and system |
| CN110545242A (en) * | 2018-05-29 | 2019-12-06 | 杭州海康威视数字技术股份有限公司 | target analysis method and intelligent analysis equipment |
| CN111310638A (en) * | 2019-12-31 | 2020-06-19 | 深圳云天励飞技术有限公司 | Data processing method and device and computer readable storage medium |
| CN112001351A (en) * | 2020-09-01 | 2020-11-27 | 城云科技（中国）有限公司 | Method, system, computer device and storage medium for processing multiple video streams |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016160744A1 (en) * | 2015-03-27 | 2016-10-06 | Twitter, Inc. | Live video streaming services |
| US10911794B2 (en) * | 2016-11-09 | 2021-02-02 | Charter Communications Operating, Llc | Apparatus and methods for selective secondary content insertion in a digital network |
| US20180176526A1 (en) * | 2016-12-19 | 2018-06-21 | Thomson Licensing | Method and apparatus for contextual color remapping transformation in content rendering |
-
2021
- 2021-03-15 CN CN202110276815.2A patent/CN115086778B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2017085196A (en) * | 2015-10-22 | 2017-05-18 | シャープ株式会社 | Receiving device, broadcasting system, reception method, and reception program |
| CN107222427A (en) * | 2016-03-22 | 2017-09-29 | 华为技术有限公司 | The method and relevant device of a kind of Message processing |
| CN107547938A (en) * | 2016-06-29 | 2018-01-05 | 北京信威通信技术股份有限公司 | A kind of method, apparatus and system for analyzing video data |
| CN109218752A (en) * | 2017-06-30 | 2019-01-15 | 中国电信股份有限公司 | Video playing, offer and detection method and set-top box, media server and system |
| CN108319926A (en) * | 2018-02-12 | 2018-07-24 | 安徽金禾软件股份有限公司 | A kind of the safety cap wearing detecting system and detection method of building-site |
| CN110545242A (en) * | 2018-05-29 | 2019-12-06 | 杭州海康威视数字技术股份有限公司 | target analysis method and intelligent analysis equipment |
| CN109168031A (en) * | 2018-11-06 | 2019-01-08 | 杭州云英网络科技有限公司 | Streaming Media method for pushing and device, steaming media platform |
| CN111310638A (en) * | 2019-12-31 | 2020-06-19 | 深圳云天励飞技术有限公司 | Data processing method and device and computer readable storage medium |
| CN112001351A (en) * | 2020-09-01 | 2020-11-27 | 城云科技（中国）有限公司 | Method, system, computer device and storage medium for processing multiple video streams |
Non-Patent Citations (3)
| Title |
|---|
| "Digital Video Source Identification Based on Container’s Structure Analysis";Raquel Ramos López et al.;《IEEE Access》;全文 * |
| "智能视频分析服务器集群设计与实现";谢洪茂;《中国优秀硕士学位论文全文数据库（电子期刊）》;全文 * |
| "面向智能视频监控系统的运动目标检测与异常行为分析";朱明凌;《中国学位论文全文数据库》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115086778A (en) | 2022-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109672580B (en) | Full link monitoring method, device, terminal equipment and storage medium | |
| CN110856018B (en) | Rapid transcoding method and system in monitoring system based on cloud computing | |
| CN111459986B (en) | Data computing system and method | |
| CN108062499B (en) | Face recognition method, system and platform | |
| CN111510395A (en) | Service message reporting method, device, equipment and medium | |
| CN110826799B (en) | Service prediction method, device, server and readable storage medium | |
| CN105100032A (en) | Method and apparatus for preventing resource steal | |
| CN113453025B (en) | Data acquisition method and device | |
| CN111314747B (en) | Resource outage control method, device, equipment and storage medium | |
| CN113329139A (en) | Video stream processing method, device and computer readable storage medium | |
| CN111328032B (en) | Method and device for sending video short message | |
| CN115086778B (en) | AI processing method and device for video stream | |
| CN110611937B (en) | Data distribution method and device, edge data center and readable storage medium | |
| CN112988337A (en) | Task processing system, method, device, electronic equipment and storage medium | |
| CN111263113B (en) | Data packet sending method and device and data packet processing method and device | |
| CN112910987A (en) | Message pushing method, system, device, equipment and storage medium | |
| CN110009273B (en) | Information processing method and device, storage medium, and electronic device | |
| CN111277626A (en) | Server upgrading method and device, electronic equipment and medium | |
| CN112954374B (en) | Video data processing method and device, electronic equipment and storage medium | |
| CN111405325B (en) | Video content distribution method and device and electronic equipment | |
| CN111401170B (en) | Face detection method and device | |
| CN113055493A (en) | Data packet processing method, device, system, scheduling device and storage medium | |
| CN113407643A (en) | Data routing method, device, equipment and computer readable storage medium | |
| CN112203155B (en) | Stream taking method, system and equipment | |
| CN116524417B (en) | Method and device for extracting distributed real-time video key frames based on Flink |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |