CN114187541A - Intelligent video analysis method and storage device for user-defined service scene - Google Patents
Intelligent video analysis method and storage device for user-defined service scene Download PDFInfo
- Publication number
- CN114187541A CN114187541A CN202111254845.XA CN202111254845A CN114187541A CN 114187541 A CN114187541 A CN 114187541A CN 202111254845 A CN202111254845 A CN 202111254845A CN 114187541 A CN114187541 A CN 114187541A
- Authority
- CN
- China
- Prior art keywords
- video analysis
- service
- video
- analysis
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44521—Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
- G06F9/44526—Plug-ins; Add-ons
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of artificial intelligence, in particular to an intelligent video analysis method and storage equipment for a user-defined service scene. The method for analyzing the intelligent video facing to the user-defined service scene comprises the following steps: splitting the scene application category by the minimum granularity; dividing the function of video analysis into different recognition modes; after the minimum granularity of the scene application category is divided and the video analysis function is divided, when a video analysis instruction is responded, a business scene can be customized according to the video analysis requirement, the requirement can be met by the combination of the minimum granularity scene application category and the video analysis function no matter how the video analysis requirement is, and video analysis is carried out based on a preset flow-line arrangement tool. The whole method can meet the video analysis requirements of different industries in different scenes, and can implement customized products and customized services for clients of all industries.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an intelligent video analysis method and storage equipment for a user-defined service scene.
Background
With the rapid development of technologies such as internet of things, artificial intelligence, cloud computing and big data, the way of transmitting information through vision, such as pictures and videos, is continuously developed and optimized, and the applications of intelligent videos, such as face snapshot recognition, behavior analysis, regional intrusion detection, video capture, etc., are increasingly touched in daily life and work of people,
Flame and smoke detection, etc. The development of the deep learning technology, the processing of videos and images is widely concerned, the intelligent video analysis technology gradually develops towards intellectualization and rapidness, wireless video monitoring is easier to deploy along with the arrival of the 5G era, the more convenient advantage promotes the exertion of more fields, the intellectualization becomes the mainstream configuration of video analysis, the future application extends to the fields of government, public utilities, education, communities, financial industry and the like, and the development space is wide.
With the popularization of high-definition cameras, the video data volume is increased explosively, video analysis technology is continuously developed and changed, the requirements of more and more industries on intelligent video analysis are higher and higher, and the application scenes are wider and more detailed compared with the prior art, so that higher requirements are provided for intelligent analysis. Most of the current video analysis mainly develops different algorithm models and matched application analysis tasks for different service scenes, and needs to face challenges of various complex factors under real scenes, such as illumination change, blurring, deformation, crowded scenes, lens jitter, delay and the like, and needs to continuously adjust strategies, cut in from a more specific and more practical angle and customize different solutions for different service scenes and different environments, so as to achieve the purposes of different requirements, different functions and stronger specialty.
The current intelligent video analysis technology mainly utilizes a camera to shoot images/videos for intelligent analysis, realizes real-time analysis of picture information, automatically identifies moving targets in an area, and simultaneously tracks and alarms the identified targets. The prior art mainly has the following problems.
1. Cannot be compatible with complex and changeable service scenes simultaneously
At present, most video analysis needs customized development aiming at different service scenes, the sharing, intelligence and integration degrees are not enough, the video analysis cannot be well adapted to complex and variable service scenes, customized development needs to be carried out aiming at different service scenes, and the product customization capability is weak.
2. Business scenario flow arrangement inflexibility
The intelligent video analysis has the requirements of industrialization, specialization, concretization and the like, has own characteristics aiming at various industries, needs to make different solutions according to different special requirements and different emphasis points, needs to provide a flexible flow arrangement, an automatic event triggering mechanism and a message notification mechanism, can provide a more convenient and easier personalized customization mechanism for customers, and most of the current platforms cannot be flexibly expanded and lack graphical arrangement capability.
3. Video algorithm model cannot be adjusted conveniently and dynamically and online self-defined business rules
Because the video algorithm model has large unconfirmability and needs to be adjusted and tested frequently, factors influencing the accuracy rate of video analysis are very many, including shaking of an object or a camera, change of light, dynamic objects, blurring, deformation, crowded scenes and the like, environmental factors cannot be expected, the model algorithm has a lot of uncertainty, continuous adjustment and change in different service scenes are needed, a tool for self-defining plug-ins and self-defining rules is needed to be provided to adapt to different service scene requirements, most products do not provide an online self-defining expansion function at present, and only personalized function development can be carried out.
Disclosure of Invention
Therefore, it is necessary to provide an intelligent video analysis method for a user-defined service scene to solve the technical problems that a video analysis technology cannot be compatible with a complex and changeable service scene at the same time, and the service scene process is not flexible, and the like, and the specific technical scheme is as follows:
a method for intelligent video analysis facing to user-defined service scene comprises the following steps:
splitting the scene application category by the minimum granularity;
dividing the function of video analysis into different recognition modes;
responding to the video analysis instruction, customizing a service scene according to the video analysis requirement, and performing video analysis based on a preset flow line arrangement tool.
Further, the "performing video analysis based on a preset streamlined assembly line editing tool" specifically includes the steps of:
and constructing a video analysis pipeline based on the Deepstream, and processing the input video stream.
Further, the method for constructing the video analysis pipeline based on the Deepstream specifically comprises the following steps
Accessing a video terminal device, and forming a batch of buffer areas from a plurality of input sources through a video stream mixing plug-in;
combining a model algorithm defined by a flow link customized by a service scene, and performing multi-level model inference by using a model inference plug-in to generate a corresponding target frame or classification information;
carrying out object tracking of the unique ID through a tracker plug-in of image analysis to generate tracking target information;
forming a 2D frame array through a multi-data stream tiling plug-in;
generating and identifying a video stream and pushing the video stream to the RTSP streaming media service by using the generated metadata on the synthesized frame through the screen display plug-in;
the analysis data is sent to the corresponding message queue by combining the message converter and the message agent plug-in.
Further, the "processing an input video stream" specifically includes the steps of:
the consumption video service analyzes the pushed analysis result and performs data analysis by combining the analysis strategy and the related rule defined by the service scene flow node;
if the human face needs to be recognized, calling a human face recognition service, and integrating a recognition result into the service rule judgment of the analysis service;
pushing the result of the data analysis to a message queue for consumption in the next process link;
if the alarm strategy of the process link needs to perform recording service, generating a recording calling message, pushing the recording calling message to a message queue for consumption of the video recording service, consuming the recording service message queue by the video recording service, acquiring violation information, generating alarm information, and pushing the alarm information to the message queue for calling of other modules.
Further, before "responding to the video analysis instruction and customizing the service scene according to the video analysis requirement", the method specifically includes the following steps:
an overhead visual business scenario programming tool;
the method for customizing the service scene according to the video analysis requirement specifically comprises the following steps:
and realizing a logical process on a visual business scene programming tool through a dragging component.
Further, after the "responding to the video analysis instruction", the method specifically includes the following steps:
setting a movement detection threshold value, and reading a movement detection result, the number of movement areas, the total alarm pixel number and the coordinate information of each movement area, which are output by video analysis;
reading the number of the detected motion areas, judging the next step if the number is not zero, and waiting for judging the next frame if the number is zero and the current frame does not have a moving object;
judging whether the motion area is in a detection frame defined by a user, giving motion area information in the detection frame, and then making the same judgment on other detected motion areas;
after all motion areas in a detection area defined by a user are determined, summing the areas of all the motion areas in the detection area to obtain the total area of the motion area, solving the ratio of the total area of the motion area to the area of the detection area, and entering the next video analysis if the ratio is larger than or equal to an area ratio threshold set by the user; if the ratio is smaller than the area-to-area ratio threshold set by the user, no further video analysis is performed.
Further, the different recognition modes include, but are not limited to: general recognition, pattern recognition that customizes a particular environment.
Further, the processing includes one or more of: decoding the video stream, preprocessing, reasoning, pushing the RTSP video stream, pushing a target detection result to a message queue, capturing a picture, and analyzing message development warning information related to service consumption.
In order to solve the technical problem, the storage device is further provided, and the specific technical scheme is as follows:
a storage device having stored therein a set of instructions for performing: any step of the above mentioned method for intelligent video analysis oriented to custom service scenes.
The invention has the beneficial effects that: a method for intelligent video analysis facing to user-defined service scene comprises the following steps: splitting the scene application category by the minimum granularity; the functionality of the video analytics is divided into different recognition modes, including but not limited to: general recognition, pattern recognition of customized specific environments; after the minimum granularity of the scene application category is divided and the video analysis function is divided, when a video analysis instruction is responded, a business scene can be customized according to the video analysis requirement, the requirement can be met by the combination of the minimum granularity scene application category and the video analysis function no matter how the video analysis requirement is, and video analysis is carried out based on a preset flow-line arrangement tool. The whole method can meet the video analysis requirements of different industries in different scenes, and can implement customized products and customized services for clients of all industries.
Further, the plug-and-play architecture realized based on the Gstreamer platform enables developers to flexibly use not only the plug-in of NVIDIA, but also other open-source plug-ins or create own plug-ins and use the plug-ins in the pipeline.
Drawings
Fig. 1 is a flowchart of a method for intelligent video analysis oriented to a customized service scenario according to an embodiment;
fig. 2 is a schematic diagram of an application module of an intelligent video analysis oriented to a custom service scenario according to an embodiment;
FIG. 3 is a schematic diagram of a graphical programming tool interface in accordance with an embodiment;
FIG. 4 is a schematic diagram illustrating an overall architecture of a smart recognition algorithm service module according to an embodiment;
FIG. 5 is a diagram illustrating a process for implementing a model service of the smart recognition algorithm service module according to an embodiment;
FIG. 6 is a schematic diagram illustrating a video analysis service implementation process of the intelligent recognition algorithm service module according to an embodiment;
FIG. 7 is a diagram illustrating an analysis service implementation process of the smart recognition algorithm service module according to an embodiment;
FIG. 8 is a diagram illustrating an intelligent recognition algorithm service module generating an alert message and pushing the alert message to a main body software module according to an embodiment of the present invention;
fig. 9 is a block diagram of a storage device according to an embodiment.
Description of reference numerals:
900. a storage device.
Detailed Description
To explain technical contents, structural features, and objects and effects of the technical solutions in detail, the following detailed description is given with reference to the accompanying drawings in conjunction with the embodiments.
Referring to fig. 1 to 8, in the present embodiment, a method for intelligent video analysis oriented to a customized service scenario may be applied to a storage device, where the storage device includes but is not limited to: personal computers, servers, general purpose computers, special purpose computers, network devices, embedded devices, programmable devices, and the like.
The core technical idea of the application is as follows: aiming at different service scenes, a video identification capability system model is established, different capability indexes are customized based on the capability of different model algorithms, and different service scenes are customized through a visual process arrangement tool to meet the requirements of different scenes. The definition of the whole service scene process is completed in a dragging and pulling mode, the expansion of service capability is realized by combining a rule engine and a message platform based on a plug-in type and streaming type component assembly type program framework, and the running-through of the whole pipeline is completed so as to adapt to the complex and variable requirements of different service scenes.
As will be described in detail below, above the software module, as shown in fig. 2, the video analysis of the video collected by the video detection terminal is mainly performed by matching the intelligent recognition algorithm service module and the ontology software module.
The method is mainly used for constructing an AI-driven intelligent video analysis application service based on a flow arrangement and plug-in mode, providing real-time video stream analysis, and comprising real-time video encoding and decoding, neural network reasoning and related analysis services. The AI model and the service analysis rules are decoupled, various training frame adaptations and uniform calling interfaces are provided, data transmission protocols such as HTTP, RTSP, MQTT, GB/28181 and the like are adopted to carry out uniform access on various video acquisition terminals, and the mechanisms such as a rule engine, function calculation domain flow processing and the like are combined to realize data interactive sharing, so that the real-time intelligent video analysis service of the AI is realized.
The following specific description will be developed:
step S101: and splitting the scene application category with minimum granularity.
Step S102: the functionality of the video analysis is divided into different recognition modes. Wherein the different recognition modes include, but are not limited to: general recognition, pattern recognition that customizes a particular environment.
Step S103: responding to the video analysis instruction, customizing a service scene according to the video analysis requirement, and performing video analysis based on a preset flow line arrangement tool.
The above steps S101 to S103 are explained in detail below from the main blocks of the entire storage device:
it should be noted that the video monitoring terminal aimed at by the application is composed of various intelligent video terminals, full-coverage acquisition of multi-dimensional video data of an operation field is achieved, the perception of panorama of the operation field is met, and a foundation of digital safety control video analysis service is laid. The method mainly provides management of the terminal equipment through a body software module, adopts data transmission protocols such as HTTP, RTSP, MQTT, GB/28181 and the like to carry out unified access to various types of video acquisition terminals, can flexibly and self-define and maintain the video monitoring terminals, mainly provides classification management of the terminal equipment, and maintains basic information and relevant strategy information of the terminal equipment, and has the following specific implementation modes:
the management of the device mainly comprises the device type, an access protocol, a device IP, a device number, a device position, a mobile detection threshold, scene similarity, a detection interval, a default behavior identification type, an identification area, whether the device is started or not and the like.
And motion detection, wherein video detection analysis is mainly carried out to obtain a video detection analysis result by detecting the brightness change of the video. The color image of luminance method represents using the YCbCr color space and its variations, Y being the luminance component of the image, calculated as a weighted average of the R, G, B components using the following formula: it is proposed that in calculating Y, the empirically recommended weight value chosen is kr-0.299, kg-0.587, and kb-0.114. The usual conversion formula is then as follows:
Y＝0.299R+0.587G+0.114B
and the brightness Y of the image is the only contrast parameter of the video frame in the motion detection algorithm, and the subsequent motion detection judgment is carried out by depending on the brightness Y change of each pixel point.
Therefore, whether the video is analyzed in the next step or not specifically comprises the following steps after the video analysis instruction is responded:
1) step 1: setting a movement detection threshold (the average value of the brightness of pixel points), and reading a movement detection result output by video analysis, the number of movement areas, the total alarm pixel number and the coordinate information of each movement area;
2) step 2: reading the number of the detected motion areas, judging the next step if the number is not zero, and waiting for judging the next frame if the number is zero and the current frame does not have a moving object;
3) and step 3: judging whether the motion area is in a detection frame defined by a user, giving motion area information in the detection frame, and then making the same judgment on other detected motion areas;
4) and 4, step 4: after all motion areas in a detection area defined by a user are determined, summing the areas of all the motion areas in the detection area to obtain the total area of the motion area, then calculating the ratio of the total area of the motion area to the area of the detection area, and entering the next video analysis if the ratio is larger than or equal to an area ratio threshold set by the user; if the ratio is smaller than the area-to-area ratio threshold set by the user, no further analysis is performed.
(3) And the definition of the identification area can define the identified video area by self aiming at different model behaviors, and after people or targets enter the designated area, the intelligent analysis algorithm can automatically identify the moving targets in the area.
Wherein step S103 is completed by the mutual cooperation of the intelligent recognition algorithm service module and the body software module, and the following description is made for step S103 from these two modules:
looking at the ontology software module first:
ontology software: the method provides services such as business scene management and algorithm model updating, provides information such as face feature information, video sources, analysis strategies, models, alarm rules and strategies for an intelligent recognition algorithm service module, and is specifically realized as follows:
step 1: the method comprises the steps of constructing a capability system model, defining the capability system model by combining the model capabilities of target detection, classification and the like provided by the model service of the intelligent algorithm service, dividing the video analysis function into a general identification capability index and a capability index for customizing the mode identification of a specific environment, and providing the capability of performing custom rule analysis or conversion on the identification result of the algorithm model or integrating a plurality of indexes for secondary forwarding. Examples of capability indicators are as follows:
for example, the target detection capability index refers to a function of intelligently identifying human or object intrusion behaviors and alarming potential dangerous behaviors by a front-end product. When a person or an object enters a designated area or a video area, the intelligent camera can automatically recognize a moving target appearing in the video area, and simultaneously track and alarm the recognized target, wherein general capability indexes include identification of the person, identification of an unworn mask, identification of a signboard, identification of a safety helmet, identification of instruments and other equipment, and the like. The capability indexes of the specific field operation include that the climbing operation does not wear a safety belt, the detection of traffic events, routing inspection detection and the like.
Step 2: provides a graphical programming tool and a simple and powerful flow arrangement tool. The method is characterized in that visual service scene customization work is developed based on graphical customization, a logical process is realized through a dragging component, connection equipment hardware, WebAPI, function functions and various online services are provided, and functions are expanded and personalized service scenes are customized through abundant components. As shown in fig. 3.
And step 3: and customizing a service scene, splitting the scene application according to the category of the scene application with minimum granularity, and dividing the capability index of the capability model into a set of capabilities which can be supported or assembled according to the functional requirements of video analysis. Taking a service scene of digital safety control of electric power field operation as an example, the service scene mainly integrates various equipment data, personnel information and video images of an operation field, depends on a series of judgment rules (judgment rule base) including service logic, safety standards or constraint conditions, combines with an intelligent identification algorithm model, carries out real-time intelligent analysis on video terminal data, timely reminds operation risks, automatically alarms illegal behaviors and realizes operation local safety control. Combine the management and control point of business to comb, whether the video analysis management and control point of having mainly that the operation scene arranges the signboard, whether arranges the security fence, whether the operation personnel dress is normal, whether correctly wears the safety helmet, whether the personnel of the operation of ascending a height worn the safety belt, test the electricity, whether wear insulating gloves when articulating the ground wire, unmanned staircase when someone works on the insulating ladder, etc.
And 4, step 4: and arranging the service scene video analysis processes, and defining a plurality of sub-control processes by combining different service control points. The following control flow of "whether to wear an insulating glove when inspecting electricity and hanging a ground wire" is taken as an example.
1) Selecting 'hardware equipment' process nodes, configuring the connection information set by the corresponding video terminal, and inputting one or more videos.
2) And selecting a plurality of target identification and detection flow nodes, configuring corresponding flow nodes to correspond to the capacity indexes, and mainly comprising electroscope identification, operating rod identification, human identification, no insulating gloves wearing and the like.
3) Establishing a 'data analysis' flow link, combining detection information of target identification and a business safety control strategy online definition rule, and identifying a current business scene as 'live working' when identifying targets such as an electroscope pen or an operating rod and the like and target frame information and a target frame position of a person have a proportion intersection exceeding a threshold value; when the target frame information of a person and the target frame information of the person without the insulating gloves exceed the intersection of the threshold proportion, the current video frame is identified as a video frame with illegal operation; when the violation of a certain proportion (for example 0.5) exists in the video frames (for example 20 frames) which are continuously connected with a certain threshold value, and the distance and the time from the last similar alarm do not exceed a certain threshold value (for example 10S), the violation information is generated,
4) connecting 'message sending' process link, pushing the generated violation information to a recording service message queue, and entering the next service process
And 5: the graphical arrangement tool is used for finishing the arrangement definition of the process of the whole business scene business, and the communication of each process link is realized by means of an intelligent recognition algorithm service module and the self-definition of the process link.
The following describes the intelligent recognition algorithm service module in detail:
the access of multiple paths of video equipment is realized, the uniform aggregation of data is realized, intelligent analysis services such as face snapshot identification, behavior analysis, regional intrusion detection, flame and smoke detection and the like are provided, the identification and alarm of real-time video violation behaviors and risks are met, and the overall architecture is shown in fig. 4.
The intelligent recognition algorithm service module mainly comprises a model service, a video service, an analysis service, a recording service and a face recognition service, and the specific implementation process is as follows:
1. the model service is used for realizing an algorithm model of a related business scene through the model service, constructing a capacity index system of the business scene, providing support for providing analysis capacity through intelligent analysis, and mainly providing flow line tools such as model training, model compression, model quantization, model conversion, edge equipment model deployment and the like. As shown in fig. 5.
2. The video analysis service is used for constructing a video analysis assembly line based on the Deepstream, realizing the pushing of modules for decoding, preprocessing and reasoning the input video stream and the RTSP video stream, pushing a target detection result to a message queue and capturing a screen, and analyzing messages related to service consumption to carry out alarm analysis. As shown in fig. 6, the following development details:
the method comprises the following specific steps of constructing a video analysis pipeline based on a GSTreamer, wherein the video analysis pipeline consists of a group of GSTreamer plug-ins to form a complete pipeline frame, and the specific implementation process is as follows:
step 1: an access video terminal device forms a batch of buffers from a plurality of input sources by means of a video stream mixing plug-in (Gst-nvstreammux).
Step 2: and performing multi-level model inference by combining a model algorithm defined by a flow link customized by a service scene and using a model inference plug-in (Gst-nvinfer), wherein the model inference plug-in is mainly used for target detection and classification to generate a corresponding target frame or classification information.
And step 3: and tracking the object with the unique ID by using a tracker plug-in (Gst-nvtracker) for image analysis to generate tracking target information.
And 4, step 4: a multiple data stream tiling plug-in (Gst-nvmultistreamtier) is utilized to form the 2D frame array.
And 5: drawing a shadow frame, a rectangle and a text on the synthesized frame by using an on-screen display (OSD) plug-in (Gst-nvdsosd) and using the generated metadata, and generating a recognition video stream to be pushed to the RTSP streaming media service.
Step 6: and sending the analysis data to a corresponding message queue by using a message converter (Gst-nvmsgconv) and a message broker (Gst-nvmsgbbrooker) plug-in combination.
3. Analyzing service, performing comprehensive identification and analysis on real-time video targets, behaviors and risks by combining target detection results and service scene self-defined alarm rules, generating corresponding alarm information, and pushing the alarm information to a message queue, wherein the specific implementation process is as follows by referring to fig. 7:
step 1: and the consumption video service analyzes the pushed analysis result and performs data analysis by combining the analysis strategy and the related rule defined by the service scene flow node.
Step 2: if the face recognition is needed, the face recognition service is needed to be called. And the face recognition service is combined with the face feature library to provide a human experience recognition service, and the generated recognition result is integrated into the service rule judgment of the analysis service.
And step 3: and pushing the result of the data analysis to a message queue for consumption in the next process link.
And 4, step 4: and if the alarm strategy of the flow link needs to perform recording service, generating a recording call message, and pushing the recording call message to a message queue for consumption of the video recording service. And recording service, consuming a recording service message queue, and carrying out small video recording or screenshot on the pushed RTSP video stream. Through consuming the violation information of the message queue, screenshot, drawing an identification box, marking characters and searching a matched video are carried out on the video stream, warning information is generated and pushed to the message queue to be called by a body software module, and the specific implementation flow is shown in fig. 8.
The method is suitable for development requirements of sharing, intellectualization and integration, and meets video analysis requirements of different industries in different scenes. The whole video analysis process can be flexibly arranged based on a self-defined process model of a business scene, and products and customized services are customized for clients of various industries by combining standard configuration and professional customization and aiming at universality and specificity of the video analysis technology.
Secondly, the following steps: provides a graphical programming tool and a simple and powerful flow arrangement tool. The method is characterized in that the visual service scene customization work is developed based on Node-red customization, the logical process is realized by dragging components, the hardware of the connecting equipment, the WebAPI, the function and various online services are provided, and the function is expanded and the personalized service scene is customized by abundant components.
Thirdly, the method comprises the following steps: based on plug-in type, flow type assembly assembling type program frame, flexible and efficient expansion and customization. The plug-and-play architecture realized based on the Gstreamer platform enables developers to flexibly use the NVIDIA plug-in, and also can use other open-source plug-ins or create own plug-ins and use the plug-ins in the pipeline.
Referring to fig. 9, a specific implementation of the memory device 900 is described as follows:
a memory device 900 having stored therein a set of instructions for performing: any step of the above mentioned method for intelligent video analysis oriented to custom service scenes.
It should be noted that, although the above embodiments have been described herein, the invention is not limited thereto. Therefore, based on the innovative concepts of the present invention, the technical solutions of the present invention can be directly or indirectly applied to other related technical fields by making changes and modifications to the embodiments described herein, or by using equivalent structures or equivalent processes performed in the content of the present specification and the attached drawings, which are included in the scope of the present invention.
Claims (9)
1. A method for intelligent video analysis facing to user-defined service scenes is characterized by comprising the following steps:
splitting the scene application category by the minimum granularity;
dividing the function of video analysis into different recognition modes;
responding to the video analysis instruction, customizing a service scene according to the video analysis requirement, and performing video analysis based on a preset flow line arrangement tool.
2. The method for intelligent video analysis oriented to custom service scenes according to claim 1, wherein the "performing video analysis based on a preset streamlined assembly line editing tool" specifically comprises the following steps:
and constructing a video analysis pipeline based on the Deepstream, and processing the input video stream.
3. The method for intelligent video analysis oriented to the custom service scene according to claim 2, wherein the "constructing a video analysis pipeline based on stream" specifically further comprises the steps of:
accessing a video terminal device, and forming a batch of buffer areas from a plurality of input sources through a video stream mixing plug-in;
combining a model algorithm defined by a flow link customized by a service scene, and performing multi-level model inference by using a model inference plug-in to generate a corresponding target frame or classification information;
carrying out object tracking of the unique ID through a tracker plug-in of image analysis to generate tracking target information;
forming a 2D frame array through a multi-data stream tiling plug-in;
generating and identifying a video stream and pushing the video stream to the RTSP streaming media service by using the generated metadata on the synthesized frame through the screen display plug-in;
the analysis data is sent to the corresponding message queue by combining the message converter and the message agent plug-in.
4. The method for intelligent video analysis oriented to custom service scenes according to claim 2, wherein the "processing the input video stream" specifically comprises the following steps:
the consumption video service analyzes the pushed analysis result and performs data analysis by combining the analysis strategy and the related rule defined by the service scene flow node;
if the human face needs to be recognized, calling a human face recognition service, and integrating a recognition result into the service rule judgment of the analysis service;
pushing the result of the data analysis to a message queue for consumption in the next process link;
if the alarm strategy of the process link needs to perform recording service, generating a recording calling message, pushing the recording calling message to a message queue for consumption of the video recording service, consuming the recording service message queue by the video recording service, acquiring violation information, generating alarm information, and pushing the alarm information to the message queue for calling of other modules.
5. The method for intelligent video analysis oriented to custom service scenes according to any one of claims 1 to 4, wherein before "responding to a video analysis instruction and customizing a service scene according to a video analysis requirement", the method specifically comprises the following steps:
an overhead visual business scenario programming tool;
the method for customizing the service scene according to the video analysis requirement specifically comprises the following steps:
and realizing a logical process on a visual business scene programming tool through a dragging component.
6. The method for intelligent video analysis oriented to custom service scenes according to any one of claims 1 to 5, wherein after responding to the video analysis command, the method further comprises the following steps:
setting a movement detection threshold value, and reading a movement detection result, the number of movement areas, the total alarm pixel number and the coordinate information of each movement area, which are output by video analysis;
reading the number of the detected motion areas, judging the next step if the number is not zero, and waiting for judging the next frame if the number is zero and the current frame does not have a moving object;
judging whether the motion area is in a detection frame defined by a user, giving motion area information in the detection frame, and then making the same judgment on other detected motion areas;
after all motion areas in a detection area defined by a user are determined, summing the areas of all the motion areas in the detection area to obtain the total area of the motion area, solving the ratio of the total area of the motion area to the area of the detection area, and entering the next video analysis if the ratio is larger than or equal to an area ratio threshold set by the user; if the ratio is smaller than the area-to-area ratio threshold set by the user, no further video analysis is performed.
7. The method for intelligent video analysis oriented to custom service scenes according to any one of claims 1 to 6, wherein the different recognition modes include but are not limited to: general recognition, pattern recognition that customizes a particular environment.
8. The method for intelligent video analysis oriented to custom service scenes according to claim 2, wherein the processing comprises one or more of the following: decoding the video stream, preprocessing, reasoning, pushing the RTSP video stream, pushing a target detection result to a message queue, capturing a picture, and analyzing message development warning information related to service consumption.
9. A storage device having a set of instructions stored therein, the set of instructions being operable to perform: any of the steps of a method of intelligent video analytics oriented to custom business scenarios as claimed in any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111254845.XA CN114187541A (en) | 2021-10-27 | 2021-10-27 | Intelligent video analysis method and storage device for user-defined service scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111254845.XA CN114187541A (en) | 2021-10-27 | 2021-10-27 | Intelligent video analysis method and storage device for user-defined service scene |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114187541A true CN114187541A (en) | 2022-03-15 |
Family
ID=80601571
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111254845.XA Pending CN114187541A (en) | 2021-10-27 | 2021-10-27 | Intelligent video analysis method and storage device for user-defined service scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114187541A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116382813A (en) * | 2023-03-16 | 2023-07-04 | 成都考拉悠然科技有限公司 | Video real-time processing AI engine system for smart city management |
| CN117009164A (en) * | 2023-08-15 | 2023-11-07 | 江苏流枢阁科技有限公司 | Method and device for evaluating artificial intelligence solution |
| WO2023241693A1 (en) * | 2022-06-17 | 2023-12-21 | 中兴通讯股份有限公司 | Media service orchestration method and apparatus, and media server and storage medium |
-
2021
- 2021-10-27 CN CN202111254845.XA patent/CN114187541A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023241693A1 (en) * | 2022-06-17 | 2023-12-21 | 中兴通讯股份有限公司 | Media service orchestration method and apparatus, and media server and storage medium |
| CN116382813A (en) * | 2023-03-16 | 2023-07-04 | 成都考拉悠然科技有限公司 | Video real-time processing AI engine system for smart city management |
| CN116382813B (en) * | 2023-03-16 | 2024-04-19 | 成都考拉悠然科技有限公司 | Video real-time processing AI engine system for smart city management |
| CN117009164A (en) * | 2023-08-15 | 2023-11-07 | 江苏流枢阁科技有限公司 | Method and device for evaluating artificial intelligence solution |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tsakanikas et al. | Video surveillance systems-current status and future trends | |
| CN114187541A (en) | Intelligent video analysis method and storage device for user-defined service scene | |
| US11328163B2 (en) | Methods and apparatus for automated surveillance systems | |
| Nam et al. | Intelligent video surveillance system: 3-tier context-aware surveillance system with metadata | |
| CN107222660B (en) | Distributed network vision monitoring system | |
| CN103069434A (en) | Multi-mode video event indexing | |
| CN110659391A (en) | Video detection method and device | |
| CN110852222A (en) | Campus corridor scene intelligent monitoring method based on target detection | |
| CN111653023A (en) | Intelligent factory supervision method | |
| US11720627B2 (en) | Systems and methods for efficiently sending video metadata | |
| Li et al. | Improved YOLOv4 network using infrared images for personnel detection in coal mines | |
| Fawzi et al. | Embedded real-time video surveillance system based on multi-sensor and visual tracking | |
| CN111429583A (en) | Space-time situation perception method and system based on three-dimensional geographic information | |
| CN115002414A (en) | Monitoring method, monitoring device, server and computer readable storage medium | |
| US10860865B2 (en) | Predictive security camera system | |
| CN115620208A (en) | Power grid safety early warning method and device, computer equipment and storage medium | |
| CN108900895B (en) | Method and device for shielding target area of video stream | |
| AU2569801A (en) | Surveillance method, system and module | |
| CN114360064B (en) | Office place personnel behavior lightweight target detection method based on deep learning | |
| Bouma et al. | WPSS: Watching people security services | |
| US10650275B2 (en) | Method for detection of temporal pattern anomalies in video streams | |
| Casanova et al. | Analysis of video surveillance images using computer vision in a controlled security environment | |
| CN116156149B (en) | Detection method and device for detecting camera movement | |
| CN112700657B (en) | Method and device for generating detection information, road side equipment and cloud control platform | |
| Borra et al. | Deep CNN Framework for Object Detection and Classification System from Real Time Videos |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |