CN112541391A - Violation behavior identification method and system based on examination video analysis - Google Patents
Violation behavior identification method and system based on examination video analysis Download PDFInfo
- Publication number
- CN112541391A CN112541391A CN202011199820.XA CN202011199820A CN112541391A CN 112541391 A CN112541391 A CN 112541391A CN 202011199820 A CN202011199820 A CN 202011199820A CN 112541391 A CN112541391 A CN 112541391A
- Authority
- CN
- China
- Prior art keywords
- picture
- task
- algorithm
- video
- identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/54—Browsing; Visualisation therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
Abstract
The invention discloses an illegal behavior identification method and system based on examination video analysis, wherein the illegal behavior identification method based on the examination video analysis comprises the following steps: receiving externally input video stream data, decoding and frame extracting the input video file, extracting pictures through an RGB data channel, and generating and sending task messages and task pictures; performing feature processing on the task picture by calling an algorithm to obtain a feature picture; calling an illegal behavior recognition algorithm to judge whether an illegal behavior exists in the feature picture; and counting task picture identification results and messages, and displaying violation behavior identification results. According to the method and the device, the artificial neural network technology is utilized to carry out algorithm identification on the decoded and framed pictures and obtain the identification result, the whole process does not need manual participation, on one hand, the human resource investment is saved, on the other hand, the error rate caused by fatigue work and carelessness of workers is reduced, and the work efficiency is improved.
Description
Technical Field
The invention relates to the field of algorithm identification, in particular to an illegal behavior identification method and system based on examination video analysis.
Background
At present, more and more manufacturers pay attention to the field of algorithm recognition, with the investment and deep research of various manufacturers and researchers in the field, the development of the field of algorithm recognition tends to be mature, and the application of the field of violation behavior recognition algorithm based on the artificial neural network technology is wider.
The traditional examination site uses a manual examination mode, the process of rechecking the stored examination video also needs the staff to check frame by frame, the manual checking mode has high efficiency, the error rate is high, the labor and the time are wasted, and the work progress is delayed. Therefore, most of the current conditional cases are examined by using algorithm identification.
Disclosure of Invention
Based on the above, the invention aims to solve the problems of low efficiency, high error rate, serious waste of human resources and long time consumption of the current examination video manual examination mode.
In order to achieve the above purpose, the invention provides an illegal behavior recognition method and system based on examination video analysis, wherein the illegal behavior recognition method based on examination video analysis comprises the following steps:
s1: receiving externally input video stream data, decoding and frame extracting the input video file, extracting pictures through an RGB data channel, and generating and sending task messages and task pictures;
s2: performing feature processing on the task picture by calling an algorithm to obtain a feature picture;
s3: calling an illegal behavior recognition algorithm to judge whether an illegal behavior exists in the feature picture;
s4: and counting task picture identification results and messages, and displaying violation behavior identification results.
The video decoding process in S1 includes the following sub-steps:
s11, reading the corresponding video file or video stream according to the type of the video transmitted from the outside;
s12, acquiring the coding format of the video from the video stream information;
and S13, calling a corresponding video decoder to decode to obtain the video frame.
The frame extraction process in S1 is completed by a frame counter.
The RGB data channel extraction in S1 is formulated as follows, wherein the video frame is represented in YUV format:
R＝Y+1.4075*(V-128) (1)
G＝Y-0.3455*(U-128)-0.7169*(V-128) (2)
B＝Y+1.779*(U-128) (3)
the violation identification process in S3 includes the following substeps:
s31: initializing an illegal behavior recognition algorithm;
s32: transmitting the characteristic picture to an algorithm model example;
s33: calling an identification algorithm;
s34: receiving the identification result and judging whether illegal behaviors exist or not;
s35: if no violation is detected, the current picture processing flow is ended; if yes, continuing to execute S36 and S37;
s36: storing the current characteristic picture to a super-fusion storage module;
s37: and sending the related information such as the current violation, the current characteristic picture super-fusion address, the original picture and the like to the service terminal.
The violation identification algorithm in S2 employs an artificial neural network computational model.
The transfer between the data streams is completed through Kafka message middleware.
The violation behavior recognition system based on examination video analysis comprises: the video decoding service module, at least one algorithm service module and the service processing module:
the video decoding service module includes: the video stream receiving submodule, the video decoding submodule, the picture frame extracting submodule and the data sending submodule are used for receiving and processing an externally input video stream and sending a task message and a task picture to the algorithm service module;
the algorithm service module comprises: the data receiving submodule, the characteristic processing submodule, the algorithm processing submodule and the data sending submodule are used for receiving and reading the task message and the picture sent by the video decoding service module and sending the identification result message and the task result picture to the service platform;
and the business processing module is used for receiving the identification result message and the task result picture sent by the algorithm service platform, connecting the identification result message and the task result picture with the user terminal and displaying the violation identification result.
The beneficial effect of this application: the artificial neural network technology is utilized to carry out algorithm recognition on the decoded and frame-extracted picture and obtain a recognition result, the whole process does not need manual participation, on one hand, the human resource investment is saved, on the other hand, the error rate caused by the fatigue work and carelessness of workers is reduced, the work efficiency is improved, and the work duration is greatly shortened.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and for those skilled in the art, other drawings can be obtained according to the structures of the drawings without creative efforts.
FIG. 1 is a general flow diagram of the system;
FIG. 2 is a video decoding flow diagram;
FIG. 3 is a flow chart of picture framing;
FIG. 4 is an image feature processing flow diagram;
FIG. 5 is a flowchart of violation identification;
FIG. 6 is a super-fused computing cluster description diagram;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to more clearly understand the technical features, objects, and effects of the present invention, embodiments of the present invention will now be described with reference to the accompanying drawings.
As shown in fig. 1, in this embodiment, an illegal behavior recognition system based on examination video analysis includes the following steps:
s1: receiving externally input video stream data, decoding and frame extracting the input video file, extracting pictures through an RGB data channel, and generating and sending task messages and task pictures;
s2: performing feature processing on the task picture by calling an algorithm to obtain a feature picture;
s3: calling an illegal behavior recognition algorithm to judge whether an illegal behavior exists in the feature picture;
s4: and counting task picture identification results and messages, and displaying violation behavior identification results.
As shown in fig. 2, preferably, the video decoding process in S1 includes the following sub-steps:
s11, reading the corresponding video file or video stream according to the type of the video transmitted from the outside;
s12, acquiring the coding format of the video from the video stream information;
and S13, calling a corresponding video decoder to decode to obtain the video frame.
Preferably, the frame extraction process in S1 is completed by a frame counter;
as shown in fig. 3, specifically, a frame counter is used to perform frame selection determination, and when a specified frame interval is reached, the latest picture frame is selected.
Preferably, the RGB data channel extraction in S1 is formulated as follows, wherein the video frame is represented in YUV format:
R＝Y+1.4075*(V-128) (1)
G＝Y-0.3455*(U-128)-0.7169*(V-128) (2)
B＝Y+1.779*(U-128) (3)
as shown in fig. 4, the picture feature processing is specifically described as: the color numerical values of three channels of the picture RGB are represented by integers between 0-255, and are compressed to be represented by single-precision floating point numbers between 0-1.
As shown in fig. 5, preferably, the violation identification process in S3 includes the following sub-steps:
s31: initializing an illegal behavior recognition algorithm;
s32: transmitting the characteristic picture to an algorithm model example;
s33: calling an identification algorithm;
s34: receiving the identification result and judging whether illegal behaviors exist or not;
s35: if no violation is detected, the current picture processing flow is ended; if yes, continuing to execute S36 and S37;
s36: storing the current characteristic picture to a super-fusion storage module;
s37: and sending the related information such as the current violation, the current characteristic picture super-fusion address, the original picture and the like to the service terminal.
Preferably, the violation identification algorithm in S2 employs an artificial neural network computational model.
Preferably, the transfer between the data streams is accomplished through Kafka message middleware.
Preferably, the violation behavior recognition system based on examination video analysis comprises: the video decoding service module, at least one algorithm service module and the service processing module:
the video decoding service module includes: the video stream receiving submodule, the video decoding submodule, the picture frame extracting submodule and the data sending submodule are used for receiving and processing an externally input video stream and sending a task message and a task picture to the algorithm service module;
the algorithm service module comprises: the data receiving submodule, the characteristic processing submodule, the algorithm processing submodule and the data sending submodule are used for receiving and reading the task message and the picture sent by the video decoding service module and sending the identification result message and the task result picture to the service platform;
and the business processing module is used for receiving the identification result message and the task result picture sent by the algorithm service platform, connecting the identification result message and the task result picture with the user terminal and displaying the violation identification result.
The Kafka message middleware is an open source stream processing platform developed by the Apache software foundation and written by Scala and Java. Kafka is a high-throughput distributed publish-subscribe message system, and is used for connecting task message sending of a video decoding service module and task message receiving of an algorithm service in the system, and identifying result message sending of the algorithm service module and identifying result receiving of a service platform;
the super-fusion storage module is connected with the task picture writing and the task picture reading of the algorithm service of the video decoding service module, and the task result picture writing and the task result picture reading of the service platform of the algorithm service module are carried out.
It should be noted that the artificial neural network technology is a computational model, and is formed by a large number of nodes (or neurons) connected with each other.
It is added that if the picture frame is obtained from inside the video file, the task information includes the following necessary elements:
raw data type (value fixed as file);
a picture storage path (a path in which pictures are stored in the super-fusion file storage);
device UUID (device unique identification);
disk serial number (disk unique identifier);
video file addresses (original video file saving directory and file name);
picture frame position (offset of current frame relative to file start frame);
if the picture frame is extracted from the real-time video stream, the task information includes the following necessary elements:
raw data type (value fixed as real-time stream);
real-time streaming sources (external input sources, provided by parameters);
a picture storage path (a path in which pictures are stored in the super-fusion file storage);
device UUID (device unique identification);
device time (time of recording frame generation).
And the task message sending is to encode the task information into a JSON character string and send the JSON character string to the Kafka server.
And the writing task picture is to write the generated picture data in the jpg format into the super-fusion file for storage.
It should be added that the violation identification system based on examination video analysis further includes the following steps:
receiving and counting violation identification behaviors: after receiving the violation identification result, the business system writes the violation identification result into a database for storage; carrying out result statistics according to the original video file of the illegal action;
and displaying an illegal behavior recognition result: and the service end provides a web interface to display the statistical result and the details of the violation behavior.
It is added that, as shown in fig. 6, the algorithm used by the violation identification system based on the examination video analysis further includes a high-performance GPU algorithm matrix based on the super-fusion technology; the super-fusion technology can bind the GPU to one or more containers for use, can manage algorithm services in the containers, and each algorithm service instance is an independent service and can receive tasks from a Kafka message queue and perform calculation; after the calculation is finished, sending result data to a specified kafka result queue; the method has the advantages that the resource optimization space can be provided while high performance is provided, and in a high-load scene, all the computing nodes are automatically started in a super-fusion mode and computing containers are deployed for computing; in a low-load scene, the super-fusion system automatically transfers the active containers to a small number of computing nodes in a container transfer mode, and resources are released.
The foregoing is illustrative of the preferred embodiments of this invention, and it is to be understood that the invention is not limited to the precise form disclosed herein and that various other combinations, modifications, and environments may be resorted to, falling within the scope of the concept as disclosed herein, either as described above or as apparent to those skilled in the relevant art. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (8)
1. An illegal behavior identification method based on examination video analysis is characterized by comprising the following steps of:
s1: receiving externally input video stream data, decoding and frame extracting the input video file, extracting pictures through an RGB data channel, and generating and sending task messages and task pictures;
s2: performing feature processing on the task picture by calling an algorithm to obtain a feature picture;
s3: calling an illegal behavior recognition algorithm to judge whether an illegal behavior exists in the feature picture;
s4: and counting task picture identification results and messages, and displaying violation behavior identification results.
2. The method according to claim 1, wherein the video decoding process in S1 includes the following sub-steps:
s11, reading the corresponding video file or video stream according to the type of the video transmitted from the outside;
s12, acquiring the coding format of the video from the video stream information;
and S13, calling a corresponding video decoder to decode to obtain the video frame.
3. The method of claim 1, wherein the frame extraction process in S1 is performed by a frame counter.
4. The method of claim 1, wherein the RGB data channel extraction in S1 is formulated as follows, wherein the video frames are represented in YUV format:
R＝Y+1.4075*(V-128) (1)
G＝Y-0.3455*(U-128)-0.7169*(V-128) (2)
B＝Y+1.779*(U-128) (3)
5. the method according to claim 1, wherein the violation identification process in S3 comprises the following sub-steps:
s31: initializing an illegal behavior recognition algorithm;
s32: transmitting the characteristic picture to an algorithm model example;
s33: calling an identification algorithm;
s34: receiving the identification result and judging whether illegal behaviors exist or not;
s35: if no violation is detected, the current picture processing flow is ended; if yes, continuing to execute S36 and S37;
s36: storing the current characteristic picture to a super-fusion storage module;
s37: and sending the related information such as the current violation, the current characteristic picture super-fusion address, the original picture and the like to the service terminal.
6. The method according to claim 1, wherein the violation identification algorithm in S2 employs an artificial neural network computational model.
7. The method of claim 1, wherein the data stream is passed through Kafka messaging middleware.
8. An violation identification system based on examination video analysis, comprising: the video decoding service module, at least one algorithm service module and the service processing module:
the video decoding service module includes: the video stream receiving submodule, the video decoding submodule, the picture frame extracting submodule and the data sending submodule are used for receiving and processing an externally input video stream and sending a task message and a task picture to the algorithm service module;
the algorithm service module comprises: the data receiving submodule, the characteristic processing submodule, the algorithm processing submodule and the data sending submodule are used for receiving and reading the task message and the picture sent by the video decoding service module and sending the identification result message and the task result picture to the service platform;
and the business processing module is used for receiving the identification result message and the task result picture sent by the algorithm service platform, connecting the identification result message and the task result picture with the user terminal and displaying the violation identification result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011199820.XA CN112541391A (en) | 2020-10-30 | 2020-10-30 | Violation behavior identification method and system based on examination video analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011199820.XA CN112541391A (en) | 2020-10-30 | 2020-10-30 | Violation behavior identification method and system based on examination video analysis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112541391A true CN112541391A (en) | 2021-03-23 |
Family
ID=75014925
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011199820.XA Pending CN112541391A (en) | 2020-10-30 | 2020-10-30 | Violation behavior identification method and system based on examination video analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112541391A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114339299A (en) * | 2021-12-27 | 2022-04-12 | 司法鉴定科学研究院 | Video evidence obtaining method for automobile driving recorder |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102081844A (en) * | 2011-01-25 | 2011-06-01 | 华中科技大学 | Traffic video behavior analyzing and alarming server |
| CN105590454A (en) * | 2016-01-27 | 2016-05-18 | 福建工程学院 | Vehicle violation behavior proof-providing method and system thereof |
| CN105632183A (en) * | 2016-01-27 | 2016-06-01 | 福建工程学院 | Vehicle violation behavior proof method and system thereof |
| CN105791888A (en) * | 2016-03-09 | 2016-07-20 | 浪潮软件股份有限公司 | Video analyzing method and video analyzing device |
| CN108683877A (en) * | 2018-03-30 | 2018-10-19 | 中国科学院自动化研究所 | Distributed massive video resolution system based on Spark |
| CN109089160A (en) * | 2018-09-19 | 2018-12-25 | 中国石油大学(华东) | A kind of colleges and universities dining room food processing unlawful practice video analytic system and method |
| CN110189524A (en) * | 2019-06-19 | 2019-08-30 | 刘先进 | A kind of active violation recognition methods, system and electronic equipment violating the regulations |
| CN110399766A (en) * | 2019-01-28 | 2019-11-01 | 浙江浩腾电子科技股份有限公司 | Smoking testing and analysis system based on deep learning |
| CN111814594A (en) * | 2020-06-19 | 2020-10-23 | 上海东普信息科技有限公司 | Logistics violation identification method, device, equipment and storage medium |
-
2020
- 2020-10-30 CN CN202011199820.XA patent/CN112541391A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102081844A (en) * | 2011-01-25 | 2011-06-01 | 华中科技大学 | Traffic video behavior analyzing and alarming server |
| CN105590454A (en) * | 2016-01-27 | 2016-05-18 | 福建工程学院 | Vehicle violation behavior proof-providing method and system thereof |
| CN105632183A (en) * | 2016-01-27 | 2016-06-01 | 福建工程学院 | Vehicle violation behavior proof method and system thereof |
| CN105791888A (en) * | 2016-03-09 | 2016-07-20 | 浪潮软件股份有限公司 | Video analyzing method and video analyzing device |
| CN108683877A (en) * | 2018-03-30 | 2018-10-19 | 中国科学院自动化研究所 | Distributed massive video resolution system based on Spark |
| CN109089160A (en) * | 2018-09-19 | 2018-12-25 | 中国石油大学(华东) | A kind of colleges and universities dining room food processing unlawful practice video analytic system and method |
| CN110399766A (en) * | 2019-01-28 | 2019-11-01 | 浙江浩腾电子科技股份有限公司 | Smoking testing and analysis system based on deep learning |
| CN110189524A (en) * | 2019-06-19 | 2019-08-30 | 刘先进 | A kind of active violation recognition methods, system and electronic equipment violating the regulations |
| CN111814594A (en) * | 2020-06-19 | 2020-10-23 | 上海东普信息科技有限公司 | Logistics violation identification method, device, equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114339299A (en) * | 2021-12-27 | 2022-04-12 | 司法鉴定科学研究院 | Video evidence obtaining method for automobile driving recorder |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113473214B (en) | Screen recording method and device, computing equipment and readable storage medium | |
| CN109840879B (en) | Image rendering method and device, computer storage medium and terminal | |
| CN113457160A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| WO2024061005A1 (en) | Read processing method and apparatus for audio and video buffer | |
| WO2021197157A1 (en) | Video stream processing method and apparatus, and electronic device and computer-readable medium | |
| CN109862381A (en) | Promotion method, system, device and storage medium based on dynamic video poster | |
| CN115037944B (en) | Cloud streaming media hard decoding method and device and storage medium | |
| CN112689197B (en) | File format conversion method and device and computer storage medium | |
| CN112541391A (en) | Violation behavior identification method and system based on examination video analysis | |
| WO2017162015A1 (en) | Data processing method and apparatus, and storage medium | |
| CN111263164A (en) | High frame frequency video parallel coding and recombination method | |
| CN110662080A (en) | Machine-oriented universal coding method | |
| CN115729683A (en) | Task processing method, device, system, computer equipment and storage medium | |
| CN105872731A (en) | Data processing method and device | |
| CN114222185B (en) | Video playing method, terminal equipment and storage medium | |
| CN113099270B (en) | File storage method, decoding method, device, storage medium and electronic device | |
| CN115546369A (en) | Workflow scheduling interface construction management method and system | |
| CN113411661B (en) | Method, apparatus, device, storage medium and program product for recording information | |
| CN112911306B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN114938408A (en) | Data transmission method, system, equipment and medium of cloud mobile phone | |
| CN114217758A (en) | Image display method, image display device, electronic equipment and computer readable storage medium | |
| CN110062296B (en) | Real-time streaming media player and architecture method thereof | |
| CN115118922B (en) | Method and device for inserting motion picture in real-time video screen combination in cloud conference | |
| WO2020155538A1 (en) | Video processing method and system, computer device and storage medium | |
| CN112131423A (en) | Picture acquisition method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220428 Address after: 610000 third floor, No. 20, Jinxing Road, Jinjiang District, Chengdu, Sichuan Applicant after: Sichuan enrollment and Examination Information Co.,Ltd. Address before: Third floor, No.10, Jiuxing Avenue, high tech Zone, Chengdu, Sichuan 610041 Applicant before: SICHUAN TIANYI NETWORK SERVICE Co.,Ltd. |