CN116886955B - Video analysis method and system based on ffmpeg and yolov5 - Google Patents

Video analysis method and system based on ffmpeg and yolov5 Download PDF

Info

Publication number
CN116886955B
CN116886955B CN202310910520.5A CN202310910520A CN116886955B CN 116886955 B CN116886955 B CN 116886955B CN 202310910520 A CN202310910520 A CN 202310910520A CN 116886955 B CN116886955 B CN 116886955B
Authority
CN
China
Prior art keywords
video
depth
network model
ffmpeg
yolov5
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310910520.5A
Other languages
Chinese (zh)
Other versions
CN116886955A (en
Inventor
史普力
马晓雨
杨文博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Testor Technology Co ltd
Original Assignee
Beijing Testor Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Testor Technology Co ltd filed Critical Beijing Testor Technology Co ltd
Priority to CN202310910520.5A priority Critical patent/CN116886955B/en
Publication of CN116886955A publication Critical patent/CN116886955A/en
Application granted granted Critical
Publication of CN116886955B publication Critical patent/CN116886955B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/11—File system administration, e.g. details of archiving or snapshots
    • G06F16/116—Details of conversion of file system types or formats
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a video analysis method and a system based on ffmpeg and yolov5, comprising the following steps: acquiring a multi-type video source file through multi-type video shooting equipment; converting the multi-type video source file into a video file processed by ffmpeg in a cross-platform manner; according to the video files processed in a cross-platform manner, converting and analyzing the video files processed in the cross-platform manner into picture set files through a multipath video source file ffmpeg processing process to obtain a multi-type analysis picture set; detecting a multi-type analysis picture set, automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing a self-selected multi-depth yolov5 network model; and analyzing the multi-type analysis picture set in real time through a self-selection multi-depth yolov5 network model to obtain a real-time analysis result of the multi-type analysis network model.

Description

Video analysis method and system based on ffmpeg and yolov5
Technical Field
The invention relates to the technical field of intelligent model detection and analysis, in particular to a video analysis method and a video analysis system based on ffmpeg and yolov 5.
Background
ffmpeg is an open source computer program for recording, converting digital audio video and converting to information streams; yolov5 is an object identification positioning model based on a deep neural network; at present, the ffmpeg can only perform file processing of a source development platform, so that cross-platform processing is difficult, and the conventional yolov5 can only perform video analysis by a single depth model, so that intelligent selection analysis of multiple depth models is difficult; the specific problems include: how to obtain multi-type video source files and ffmpeg multi-platform conversion, how to efficiently convert and analyze cross-platform processed video files through a ffmpeg processing process, how to automatically select the depth of a yolov5 network model and monitor the yolov5 network model, how to rapidly analyze multi-type analysis pictures in real time and output real-time analysis results on a large scale, and the like are yet to be solved; therefore, there is a need to propose a video analysis method and system based on ffmpeg and yolov5 to at least partially solve the problems existing in the prior art.
Disclosure of Invention
A series of concepts in simplified form are introduced in the summary section, which will be described in further detail in the detailed description section; the summary of the invention is not intended to define the key features and essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
To at least partially solve the above problems, the present invention provides a video analysis method based on ffmpeg and yolov5, comprising:
s100, acquiring a multi-type video source file through multi-type video shooting equipment; converting the multi-type video source file into a video file processed by ffmpeg in a cross-platform manner;
s200, converting and analyzing the video files processed in a cross-platform manner into picture set files through a multi-channel video source file ffmpeg processing process according to the video files processed in the cross-platform manner, and acquiring multi-type analysis picture sets;
s300, detecting a multi-type analysis picture set, automatically selecting the depth of the yolov5 network model according to a multi-type analysis picture pixel depth comparison result, and constructing a self-selected multi-depth yolov5 network model;
s400, analyzing the multi-type analysis picture set in real time through a self-selection multi-depth yolov5 network model, and obtaining a real-time analysis result of the multi-type analysis network model.
Preferably, S100 includes:
s101, shooting multiple types of videos through multiple types of video shooting equipment to acquire multiple types of video source files; the multi-type video source file includes: MPEG, WMV, AVI, MKV or OGG;
s102, setting a cross-platform conversion tool of a video source file according to a plurality of types of video source files;
s103, converting the multi-type video source file into a video file processed by the ffmpeg in a cross-platform manner through a cross-platform conversion tool of the video source file;
the video source file cross-platform conversion tool comprises: the source development platform converts the non-source development platform tool and the non-source development platform converts the source development platform tool; the source development platform conversion non-source development platform tool comprises: a first source video window drive conversion model and a second source video window drive conversion model; the non-source development platform conversion source development platform tool comprises: a first target video window drive conversion model and a second target video window drive conversion model; when a source development platform is required to convert the video source file into a non-source development platform, converting the video source file through a first source video window driving conversion model or a second source video window driving conversion model; when a source development platform is required to be converted from a non-source development platform to a source development platform, converting the video source file through a first target video window driving conversion model or a second target video window driving conversion model; when the inter-platform conversion is required, the first source video window driving conversion model corresponds to the first target video window driving conversion model, and the second source video window driving conversion model corresponds to the second target video window driving conversion model.
Preferably, S200 includes:
s201, setting a multi-path video source file ffmpeg processing process according to a video file processed in a cross-platform manner; the multipath video source file ffmpeg processing process comprises the following steps: a ffmpeg video source file catalog monitoring process, a ffmpeg video source file processing control process and a ffmpeg processing output image frame reading timing release process;
s202, converting and analyzing the video files processed in a cross-platform manner into picture set files through a multipath video source file ffmpeg processing process to obtain a multi-type analysis picture set.
Preferably, S300 includes:
s301, setting a multi-type analysis picture set pixel detection unit, and detecting the picture pixel depth in the multi-type analysis picture set through the multi-type analysis picture set pixel detection unit;
s302, according to the pixel depth of the picture, comparing the pixel depth of the picture with a pixel depth comparison standard by setting the pixel depth comparison standard to obtain a multi-type analysis picture pixel depth comparison result;
s303, automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing a self-selected multi-depth yolov5 network model; displaying a self-selected multi-depth yolov5 network model through a network model visualization tool, and monitoring the self-selected multi-depth yolov5 network model;
The pixel depth contrast criteria include: a first pixel depth contrast criterion, a second pixel depth contrast criterion, and a third pixel depth contrast criterion;
the multi-depth yolov5 network model includes: a first depth yolov5 network model, a second depth yolov5 network model, a third depth yolov5 network model, and a fourth depth yolov5 network model; automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing the self-selected multi-depth yolov5 network model comprises the following steps: according to the pixel depth comparison result of the multi-type analysis picture, when the pixel depth comparison result of the multi-type analysis picture is lower than a first pixel depth comparison standard, automatically selecting a first depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is not lower than the first pixel depth comparison standard and not higher than the second pixel depth comparison standard, automatically selecting a second depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is higher than the second pixel depth comparison standard and not higher than the third pixel depth comparison standard, automatically selecting a third depth yolov5 network model; and when the pixel depth comparison result of the multi-type analysis picture is higher than the third pixel depth comparison standard, automatically selecting a fourth depth yolov5 network model.
Preferably, S400 includes:
s401, automatically adjusting the size of an input picture in a multi-type analysis picture set to a preset picture size through a pre-training weight, and combining a plurality of picture parts in the multi-type analysis picture set to form an input picture with the preset picture size;
s402, inputting pictures according to preset picture sizes, analyzing a multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model, and outputting a deep feature map;
s403, carrying out high-speed large-target real-time detection analysis according to the deep feature map, and obtaining real-time analysis results of the multi-type analysis network model.
The invention provides a video analysis system based on ffmpeg and yolov5, comprising:
the video shooting and recording multi-platform conversion subsystem acquires multi-type video source files through multi-type video shooting and recording equipment; converting the multi-type video source file into a video file processed by ffmpeg in a cross-platform manner;
the ffmpeg multi-path processing analysis subsystem converts and analyzes the video files processed in a cross-platform manner into a picture set file through a multi-path video source file ffmpeg processing process according to the video files processed in the cross-platform manner, and acquires a multi-type analysis picture set;
a self-selection multi-depth yolov5 model subsystem detects a multi-type analysis picture set and automatically selects according to a multi-type analysis picture pixel depth comparison result
Constructing a self-selected multi-depth yolov5 network model according to the depth of the yolov5 network model;
and the target detection real-time analysis subsystem is used for analyzing the multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model to obtain a real-time analysis result of the multi-type analysis network model.
Preferably, the video recording multi-platform conversion subsystem comprises:
the multi-type video shooting subsystem shoots and records multi-type videos through multi-type video shooting equipment to acquire multi-type video source files; the multi-type video source file includes: MPEG, WMV, AVI, MKV or OGG;
the video file cross-platform conversion subsystem sets a video source file cross-platform conversion tool according to the multi-type video source files;
the file conversion acquisition subsystem converts the multi-type video source files into video files processed by the ffmpeg in a cross-platform manner through a cross-platform conversion tool of the video source files;
the video source file cross-platform conversion tool comprises: the source development platform converts the non-source development platform tool and the non-source development platform converts the source development platform tool; the source development platform conversion non-source development platform tool comprises: a first source video window drive conversion model and a second source video window drive conversion model; the non-source development platform conversion source development platform tool comprises: a first target video window drive conversion model and a second target video window drive conversion model; when a source development platform is required to convert the video source file into a non-source development platform, converting the video source file through a first source video window driving conversion model or a second source video window driving conversion model; when a source development platform is required to be converted from a non-source development platform to a source development platform, converting the video source file through a first target video window driving conversion model or a second target video window driving conversion model; when the inter-platform conversion is required, the first source video window driving conversion model corresponds to the first target video window driving conversion model, and the second source video window driving conversion model corresponds to the second target video window driving conversion model.
Preferably, the ffmpeg multiprocessing parsing subsystem comprises:
the ffmpeg processing process subsystem sets a multi-path video source file ffmpeg processing process according to the cross-platform processed video file; the multipath video source file ffmpeg processing process comprises the following steps: a ffmpeg video source file catalog monitoring process, a ffmpeg video source file processing control process and a ffmpeg processing output image frame reading timing release process;
and the file conversion analysis subsystem converts and analyzes the video file processed in a cross-platform manner into a picture set file through a multipath video source file ffmpeg processing process to obtain a multi-type analysis picture set.
Preferably, the discretionary multi-depth yolov5 model subsystem comprises:
the image pixel depth detection subsystem is provided with a multi-type analysis image set pixel detection unit, and the image pixel depth in the multi-type analysis image set is detected through the multi-type analysis image set pixel detection unit;
the pixel depth comparison standard subsystem is used for comparing the pixel depth of the picture with the pixel depth comparison standard according to the pixel depth of the picture by setting the pixel depth comparison standard so as to obtain a multi-type analysis picture pixel depth comparison result;
the self-selection multi-depth yolov5 network monitoring subsystem automatically selects the depth of a yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructs a self-selection multi-depth yolov5 network model; displaying a self-selected multi-depth yolov5 network model through a network model visualization tool, and monitoring the self-selected multi-depth yolov5 network model;
The pixel depth contrast criteria include: a first pixel depth contrast criterion, a second pixel depth contrast criterion, and a third pixel depth contrast criterion;
the multi-depth yolov5 network model includes: a first depth yolov5 network model, a second depth yolov5 network model, a third depth yolov5 network model, and a fourth depth yolov5 network model; automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing the self-selected multi-depth yolov5 network model comprises the following steps: according to the pixel depth comparison result of the multi-type analysis picture, when the pixel depth comparison result of the multi-type analysis picture is lower than a first pixel depth comparison standard, automatically selecting a first depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is not lower than the first pixel depth comparison standard and not higher than the second pixel depth comparison standard, automatically selecting a second depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is higher than the second pixel depth comparison standard and not higher than the third pixel depth comparison standard, automatically selecting a third depth yolov5 network model; and when the pixel depth comparison result of the multi-type analysis picture is higher than the third pixel depth comparison standard, automatically selecting a fourth depth yolov5 network model.
Preferably, the target detection real-time analysis subsystem comprises:
the automatic picture size adjustment subsystem automatically adjusts the size of the input picture in the multi-type analysis picture set to a preset picture size through a pre-training weight, and combines a plurality of picture parts in the multi-type analysis picture set to form an input picture with the preset picture size;
the deep feature map real-time analysis subsystem inputs pictures according to preset picture sizes, analyzes a multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model, and outputs a deep feature map;
and the large target real-time detection and analysis subsystem is used for carrying out high-speed large target real-time detection and analysis according to the deep feature map to obtain real-time analysis results of the multi-type analysis network model.
Compared with the prior art, the invention at least comprises the following beneficial effects:
the invention provides a video analysis method and a system based on ffmpeg and yolov5, which acquire a plurality of types of video source files through a plurality of types of video shooting equipment; converting the multi-type video source file into a video file processed by ffmpeg in a cross-platform manner; according to the video files processed in a cross-platform manner, converting and analyzing the video files processed in the cross-platform manner into picture set files through a multipath video source file ffmpeg processing process to obtain a multi-type analysis picture set; detecting a multi-type analysis picture set, automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing a self-selected multi-depth yolov5 network model; real-time analyzing the multi-type analysis picture set through a self-selected multi-depth yolov5 network model to obtain a real-time analysis result of the multi-type analysis network model; the method can solve the problem of file processing between the ffmpeg source development platform and the cross-platform, and can perform yolov5 multi-depth model automatic selection analysis; the cross-platform file processing speed and the model running efficiency are greatly improved, and multi-type video source files can be obtained and ffmpeg multi-platform conversion can be rapidly carried out; the multi-line ffmpeg processing process can be run in batches, and the cross-platform processed video files are efficiently converted and analyzed; the depth of the yolov5 network model can be automatically selected, the yolov5 network model is monitored, multiple types of analysis pictures can be rapidly analyzed in real time, and real-time analysis results can be output in a large scale; the efficiency and the analysis depth of the video multi-platform multi-type video analysis are greatly improved.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
fig. 1 is a diagram illustrating steps of a video analysis method based on ffmpeg and yolov5 according to an embodiment of the present invention.
Fig. 2 is a block diagram of a video analysis system based on ffmpeg and yolov5 according to the present invention.
FIG. 3 is a diagram of an embodiment of a video analysis system based on ffmpeg and yolov5 according to the present invention.
Detailed Description
The present invention is described in further detail below with reference to the drawings and examples to enable those skilled in the art to practice the same and to refer to the description; as shown in fig. 1 to 3, the present invention provides a video analysis method based on ffmpeg and yolov5, comprising:
s100, acquiring a multi-type video source file through multi-type video shooting equipment; converting the multi-type video source file into a video file processed by ffmpeg in a cross-platform manner;
S200, converting and analyzing the video files processed in a cross-platform manner into picture set files through a multi-channel video source file ffmpeg processing process according to the video files processed in the cross-platform manner, and acquiring multi-type analysis picture sets;
s300, detecting a multi-type analysis picture set, automatically selecting the depth of the yolov5 network model according to a multi-type analysis picture pixel depth comparison result, and constructing a self-selected multi-depth yolov5 network model;
s400, analyzing the multi-type analysis picture set in real time through a self-selection multi-depth yolov5 network model, and obtaining a real-time analysis result of the multi-type analysis network model.
The principle and effect of the technical scheme are as follows: the invention provides a video analysis method based on ffmpeg and yolov5, which comprises the following steps: acquiring a multi-type video source file through multi-type video shooting equipment; converting the multi-type video source file into a video file processed by ffmpeg in a cross-platform manner; according to the video files processed in a cross-platform manner, converting and analyzing the video files processed in the cross-platform manner into picture set files through a multipath video source file ffmpeg processing process to obtain a multi-type analysis picture set; detecting a multi-type analysis picture set, automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing a self-selected multi-depth yolov5 network model; real-time analyzing the multi-type analysis picture set through a self-selected multi-depth yolov5 network model to obtain a real-time analysis result of the multi-type analysis network model; the method can solve the problem of file processing between the ffmpeg source development platform and the cross-platform, and can perform yolov5 multi-depth model automatic selection analysis; the cross-platform file processing speed and the model running efficiency are greatly improved, and multi-type video source files can be obtained and ffmpeg multi-platform conversion can be rapidly carried out; the multi-line ffmpeg processing process can be run in batches, and the cross-platform processed video files are efficiently converted and analyzed; the depth of the yolov5 network model can be automatically selected, the yolov5 network model is monitored, multiple types of analysis pictures can be rapidly analyzed in real time, and real-time analysis results can be output in a large scale; the efficiency and the analysis depth of the video multi-platform multi-type video analysis are greatly improved.
In one embodiment, S100 comprises:
s101, shooting multiple types of videos through multiple types of video shooting equipment to acquire multiple types of video source files; the multi-type video source file includes: MPEG, WMV, AVI, MKV or OGG;
s102, setting a cross-platform conversion tool of a video source file according to a plurality of types of video source files;
s103, converting the multi-type video source file into a video file processed by the ffmpeg in a cross-platform manner through a cross-platform conversion tool of the video source file;
the video source file cross-platform conversion tool comprises: the source development platform converts the non-source development platform tool and the non-source development platform converts the source development platform tool; the source development platform conversion non-source development platform tool comprises: a first source video window drive conversion model and a second source video window drive conversion model; the non-source development platform conversion source development platform tool comprises: a first target video window drive conversion model and a second target video window drive conversion model; when a source development platform is required to convert the video source file into a non-source development platform, converting the video source file through a first source video window driving conversion model or a second source video window driving conversion model; when a source development platform is required to be converted from a non-source development platform to a source development platform, converting the video source file through a first target video window driving conversion model or a second target video window driving conversion model; when the inter-platform conversion is required, the first source video window driving conversion model corresponds to the first target video window driving conversion model, and the second source video window driving conversion model corresponds to the second target video window driving conversion model.
The principle and effect of the technical scheme are as follows: recording the multi-type video through multi-type video recording equipment to obtain a multi-type video source file; the multi-type video source file includes: MPEG, WMV, AVI, MKV or OGG; setting a cross-platform conversion tool of the video source file according to the multi-type video source file; converting the multi-type video source file into a video file processed by a ffmpeg cross-platform through a video source file cross-platform conversion tool; the video source file cross-platform conversion tool comprises: the source development platform converts the non-source development platform tool and the non-source development platform converts the source development platform tool; the source development platform conversion non-source development platform tool comprises: a first source video window drive conversion model and a second source video window drive conversion model; the non-source development platform conversion source development platform tool comprises: a first target video window drive conversion model and a second target video window drive conversion model; when a source development platform is required to convert the video source file into a non-source development platform, converting the video source file through a first source video window driving conversion model or a second source video window driving conversion model; when a source development platform is required to be converted from a non-source development platform to a source development platform, converting the video source file through a first target video window driving conversion model or a second target video window driving conversion model; when the mutual conversion between the platforms is needed, the first source video window driving conversion model corresponds to the first target video window driving conversion model, and the second source video window driving conversion model corresponds to the second target video window driving conversion model; the method can solve the problem of file processing between the ffmpeg source development platform and the cross-platform, and can perform yolov5 multi-depth model automatic selection analysis; the method greatly improves the cross-platform file processing speed and the model running efficiency, and can acquire multi-type video source files and rapidly perform ffmpeg multi-platform conversion.
In one embodiment, S200 includes:
s201, setting a multi-path video source file ffmpeg processing process according to a video file processed in a cross-platform manner; the multipath video source file ffmpeg processing process comprises the following steps: a ffmpeg video source file catalog monitoring process, a ffmpeg video source file processing control process and a ffmpeg processing output image frame reading timing release process;
s202, converting and analyzing the video files processed in a cross-platform manner into picture set files through a multipath video source file ffmpeg processing process to obtain a multi-type analysis picture set.
The principle and effect of the technical scheme are as follows: setting a multipath video source file ffmpeg processing process according to the video files processed in a cross-platform manner; the multipath video source file ffmpeg processing process comprises the following steps: a ffmpeg video source file catalog monitoring process, a ffmpeg video source file processing control process and a ffmpeg processing output image frame reading timing release process; converting and analyzing the video files processed in a cross-platform manner into picture set files through a multipath video source file ffmpeg processing process to obtain a multi-type analysis picture set; the multi-line ffmpeg processing process can be run in batches, and the cross-platform processed video files can be converted and analyzed efficiently.
In one embodiment, S300 includes:
s301, setting a multi-type analysis picture set pixel detection unit, and detecting the picture pixel depth in the multi-type analysis picture set through the multi-type analysis picture set pixel detection unit;
s302, according to the pixel depth of the picture, comparing the pixel depth of the picture with a pixel depth comparison standard by setting the pixel depth comparison standard to obtain a multi-type analysis picture pixel depth comparison result;
s303, automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing a self-selected multi-depth yolov5 network model; displaying a self-selected multi-depth yolov5 network model through a network model visualization tool, and monitoring the self-selected multi-depth yolov5 network model;
the pixel depth contrast criteria include: a first pixel depth contrast criterion, a second pixel depth contrast criterion, and a third pixel depth contrast criterion;
the multi-depth yolov5 network model includes: a first depth yolov5 network model, a second depth yolov5 network model, a third depth yolov5 network model, and a fourth depth yolov5 network model; automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing the self-selected multi-depth yolov5 network model comprises the following steps: according to the pixel depth comparison result of the multi-type analysis picture, when the pixel depth comparison result of the multi-type analysis picture is lower than a first pixel depth comparison standard, automatically selecting a first depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is not lower than the first pixel depth comparison standard and not higher than the second pixel depth comparison standard, automatically selecting a second depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is higher than the second pixel depth comparison standard and not higher than the third pixel depth comparison standard, automatically selecting a third depth yolov5 network model; and when the pixel depth comparison result of the multi-type analysis picture is higher than the third pixel depth comparison standard, automatically selecting a fourth depth yolov5 network model.
The principle and effect of the technical scheme are as follows: setting a multi-type analysis picture set pixel detection unit, and detecting the picture pixel depth in the multi-type analysis picture set through the multi-type analysis picture set pixel detection unit; according to the pixel depth of the picture, comparing the pixel depth of the picture with a pixel depth comparison standard by setting the pixel depth comparison standard to obtain a multi-type analysis picture pixel depth comparison result; automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing a self-selected multi-depth yolov5 network model; displaying a self-selected multi-depth yolov5 network model through a network model visualization tool, and monitoring the self-selected multi-depth yolov5 network model; the pixel depth contrast criteria include: a first pixel depth contrast criterion, a second pixel depth contrast criterion, and a third pixel depth contrast criterion; the multi-depth yolov5 network model includes: a first depth yolov5 network model, a second depth yolov5 network model, a third depth yolov5 network model, and a fourth depth yolov5 network model; automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing the self-selected multi-depth yolov5 network model comprises the following steps: according to the pixel depth comparison result of the multi-type analysis picture, when the pixel depth comparison result of the multi-type analysis picture is lower than a first pixel depth comparison standard, automatically selecting a first depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is not lower than the first pixel depth comparison standard and not higher than the second pixel depth comparison standard, automatically selecting a second depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is higher than the second pixel depth comparison standard and not higher than the third pixel depth comparison standard, automatically selecting a third depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is higher than the third pixel depth comparison standard, automatically selecting a fourth depth yolov5 network model; the depth of the yolov5 network model can be automatically selected, the yolov5 network model is monitored, multiple types of analysis pictures can be rapidly analyzed in real time, and real-time analysis results can be output in a large scale.
In one embodiment, S400 includes:
s401, automatically adjusting the size of an input picture in a multi-type analysis picture set to a preset picture size through a pre-training weight, and combining a plurality of picture parts in the multi-type analysis picture set to form an input picture with the preset picture size;
s402, inputting pictures according to preset picture sizes, analyzing a multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model, and outputting a deep feature map;
s403, carrying out high-speed large-target real-time detection analysis according to the deep feature map, and obtaining real-time analysis results of the multi-type analysis network model.
The principle and effect of the technical scheme are as follows: automatically adjusting the size of an input picture in the multi-type analysis picture set to a preset picture size through a pre-training weight, and combining a plurality of picture parts in the multi-type analysis picture set to form an input picture with the preset picture size; inputting pictures according to preset picture sizes, analyzing a multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model, and outputting a deep feature map; according to the deep feature map, performing high-speed large-target real-time detection analysis to obtain a real-time analysis result of the multi-type analysis network model; the high-speed large target real-time detection analysis comprises the following steps: carrying out high-speed large-target real-time detection and analysis through a deep feature map high-speed large-target detection and analysis group; the deep feature map high-speed large target detection analysis group comprises: the device comprises a deep feature map processing and converting unit, a data storage unit, a deep feature map ffmpeg processing unit, a ffmpeg stream executing unit and a network model output unit; a multi-layer deep feature map processing and converting unit for dynamically selecting a multi-layer deep feature map ffmpeg processing unit; the data storage unit is used for storing the multi-layer deep feature map processing result of the multi-layer deep feature map ffmpeg processing unit;
The multi-layer deep feature map ffmpeg processing unit includes: the system comprises a rapid resource locator decoding subunit, a multi-layer deep feature map processing result writing subunit, a multi-layer deep feature map encoding subunit and a multi-layer deep feature map transmitting subunit; the rapid resource locator decoding subunit decodes the multi-layer deep feature map according to the rapid resource locator address; the multi-layer deep feature map processing subunit reads multi-layer deep feature map processing instruction parameters from the data storage unit to perform corresponding multi-layer deep feature map processing, and writes the processing result into the data storage unit through the multi-layer deep feature map processing result writing subunit; the fast resource locator address custom format employed in the fast resource locator decoding subunit includes: the rapid resource locator identifies a head, a first layer of characteristic diagrams, a second layer of characteristic diagrams, a third layer of characteristic diagrams and a fourth layer of characteristic diagrams; the ffmpeg stream execution unit converts the processing address dynamically allocated by the ffmpeg into a quick resource locator for the network model output unit to access, and a multi-type multi-layer deep feature map shooting device corresponds to the unique quick resource locator; the ffmpeg stream execution unit generates a multi-layer deep feature map stream address prefix, a multi-type multi-layer deep feature map recording device number and a multi-layer deep feature map stream address in a fixed domain name suffix format according to a ffmpeg dynamic selection result and an original RTSP address generation rule, wherein the multi-type multi-layer deep feature map recording device number is unique in a system; the network model output unit is used for displaying the real-time multilayer deep feature map, the processing result data of the multilayer deep feature map and the process state; the multi-type analysis pictures can be rapidly analyzed in real time, and real-time analysis results can be output in a large scale; the efficiency and the analysis depth of the video multi-platform multi-type video analysis are greatly improved.
The invention provides a video analysis system based on ffmpeg and yolov5, comprising:
the video shooting and recording multi-platform conversion subsystem acquires multi-type video source files through multi-type video shooting and recording equipment; converting the multi-type video source file into a video file processed by ffmpeg in a cross-platform manner;
the ffmpeg multi-path processing analysis subsystem converts and analyzes the video files processed in a cross-platform manner into a picture set file through a multi-path video source file ffmpeg processing process according to the video files processed in the cross-platform manner, and acquires a multi-type analysis picture set;
a self-selection multi-depth yolov5 model subsystem detects a multi-type analysis picture set, automatically selects the depth of a yolov5 network model according to a multi-type analysis picture pixel depth comparison result, and constructs a self-selection multi-depth yolov5 network model;
and the target detection real-time analysis subsystem is used for analyzing the multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model to obtain a real-time analysis result of the multi-type analysis network model.
The principle and effect of the technical scheme are as follows: the invention provides a video analysis system based on ffmpeg and yolov5, comprising: the video shooting and recording multi-platform conversion subsystem acquires multi-type video source files through multi-type video shooting and recording equipment; converting the multi-type video source file into a video file processed by ffmpeg in a cross-platform manner; the ffmpeg multi-path processing analysis subsystem converts and analyzes the video files processed in a cross-platform manner into a picture set file through a multi-path video source file ffmpeg processing process according to the video files processed in the cross-platform manner, and acquires a multi-type analysis picture set; a self-selection multi-depth yolov5 model subsystem detects a multi-type analysis picture set, automatically selects the depth of a yolov5 network model according to a multi-type analysis picture pixel depth comparison result, and constructs a self-selection multi-depth yolov5 network model; the target detection real-time analysis subsystem is used for analyzing the multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model to obtain a real-time analysis result of the multi-type analysis network model; the method can solve the problem of file processing between the ffmpeg source development platform and the cross-platform, and can perform yolov5 multi-depth model automatic selection analysis; the cross-platform file processing speed and the model running efficiency are greatly improved, and multi-type video source files can be obtained and ffmpeg multi-platform conversion can be rapidly carried out; the multi-line ffmpeg processing process can be run in batches, and the cross-platform processed video files are efficiently converted and analyzed; the depth of the yolov5 network model can be automatically selected, the yolov5 network model is monitored, multiple types of analysis pictures can be rapidly analyzed in real time, and real-time analysis results can be output in a large scale; the efficiency and the analysis depth of the video multi-platform multi-type video analysis are greatly improved.
In one embodiment, a video capture multi-platform conversion subsystem includes:
the multi-type video shooting subsystem shoots and records multi-type videos through multi-type video shooting equipment to acquire multi-type video source files; the multi-type video source file includes: MPEG, WMV, AVI, MKV or OGG;
the video file cross-platform conversion subsystem sets a video source file cross-platform conversion tool according to the multi-type video source files;
the file conversion acquisition subsystem converts the multi-type video source files into video files processed by the ffmpeg in a cross-platform manner through a cross-platform conversion tool of the video source files;
the video source file cross-platform conversion tool comprises: the source development platform converts the non-source development platform tool and the non-source development platform converts the source development platform tool; the source development platform conversion non-source development platform tool comprises: a first source video window drive conversion model and a second source video window drive conversion model; the non-source development platform conversion source development platform tool comprises: a first target video window drive conversion model and a second target video window drive conversion model; when a source development platform is required to convert the video source file into a non-source development platform, converting the video source file through a first source video window driving conversion model or a second source video window driving conversion model; when a source development platform is required to be converted from a non-source development platform to a source development platform, converting the video source file through a first target video window driving conversion model or a second target video window driving conversion model; when the inter-platform conversion is required, the first source video window driving conversion model corresponds to the first target video window driving conversion model, and the second source video window driving conversion model corresponds to the second target video window driving conversion model.
The principle and effect of the technical scheme are as follows: the video recording multi-platform conversion subsystem comprises: the multi-type video shooting subsystem shoots and records multi-type videos through multi-type video shooting equipment to acquire multi-type video source files; the multi-type video source file includes: MPEG, WMV, AVI, MKV or OGG; the video file cross-platform conversion subsystem sets a video source file cross-platform conversion tool according to the multi-type video source files; the file conversion acquisition subsystem converts the multi-type video source files into video files processed by the ffmpeg in a cross-platform manner through a cross-platform conversion tool of the video source files; the video source file cross-platform conversion tool comprises: the source development platform converts the non-source development platform tool and the non-source development platform converts the source development platform tool; the source development platform conversion non-source development platform tool comprises: a first source video window drive conversion model and a second source video window drive conversion model; the non-source development platform conversion source development platform tool comprises: a first target video window drive conversion model and a second target video window drive conversion model; when a source development platform is required to convert the video source file into a non-source development platform, converting the video source file through a first source video window driving conversion model or a second source video window driving conversion model; when a source development platform is required to be converted from a non-source development platform to a source development platform, converting the video source file through a first target video window driving conversion model or a second target video window driving conversion model; when the mutual conversion between the platforms is needed, the first source video window driving conversion model corresponds to the first target video window driving conversion model, and the second source video window driving conversion model corresponds to the second target video window driving conversion model; the method can solve the problem of file processing between the ffmpeg source development platform and the cross-platform, and can perform yolov5 multi-depth model automatic selection analysis; the method greatly improves the cross-platform file processing speed and the model running efficiency, and can acquire multi-type video source files and rapidly perform ffmpeg multi-platform conversion.
In one embodiment, the ffmpeg multiprocessing parsing subsystem includes:
the ffmpeg processing process subsystem sets a multi-path video source file ffmpeg processing process according to the cross-platform processed video file; the multipath video source file ffmpeg processing process comprises the following steps: a ffmpeg video source file catalog monitoring process, a ffmpeg video source file processing control process and a ffmpeg processing output image frame reading timing release process;
and the file conversion analysis subsystem converts and analyzes the video file processed in a cross-platform manner into a picture set file through a multipath video source file ffmpeg processing process to obtain a multi-type analysis picture set.
The principle and effect of the technical scheme are as follows: the ffmpeg multiprocessing parsing subsystem includes: the ffmpeg processing process subsystem sets a multi-path video source file ffmpeg processing process according to the cross-platform processed video file; the multipath video source file ffmpeg processing process comprises the following steps: a ffmpeg video source file catalog monitoring process, a ffmpeg video source file processing control process and a ffmpeg processing output image frame reading timing release process; the file conversion analysis subsystem converts and analyzes the video files processed in a cross-platform manner into picture set files through a multipath video source file ffmpeg processing process to obtain a multi-type analysis picture set; the multi-line ffmpeg processing process can be run in batches, and the cross-platform processed video files can be converted and analyzed efficiently.
In one embodiment, the discretionary multi-depth yolov5 model subsystem comprises:
the image pixel depth detection subsystem is provided with a multi-type analysis image set pixel detection unit, and the image pixel depth in the multi-type analysis image set is detected through the multi-type analysis image set pixel detection unit;
the pixel depth comparison standard subsystem is used for comparing the pixel depth of the picture with the pixel depth comparison standard according to the pixel depth of the picture by setting the pixel depth comparison standard so as to obtain a multi-type analysis picture pixel depth comparison result;
the self-selection multi-depth yolov5 network monitoring subsystem automatically selects the depth of a yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructs a self-selection multi-depth yolov5 network model; displaying a self-selected multi-depth yolov5 network model through a network model visualization tool, and monitoring the self-selected multi-depth yolov5 network model;
the pixel depth contrast criteria include: a first pixel depth contrast criterion, a second pixel depth contrast criterion, and a third pixel depth contrast criterion;
the multi-depth yolov5 network model includes: a first depth yolov5 network model, a second depth yolov5 network model, a third depth yolov5 network model, and a fourth depth yolov5 network model; automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing the self-selected multi-depth yolov5 network model comprises the following steps: according to the pixel depth comparison result of the multi-type analysis picture, when the pixel depth comparison result of the multi-type analysis picture is lower than a first pixel depth comparison standard, automatically selecting a first depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is not lower than the first pixel depth comparison standard and not higher than the second pixel depth comparison standard, automatically selecting a second depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is higher than the second pixel depth comparison standard and not higher than the third pixel depth comparison standard, automatically selecting a third depth yolov5 network model; and when the pixel depth comparison result of the multi-type analysis picture is higher than the third pixel depth comparison standard, automatically selecting a fourth depth yolov5 network model.
The principle and effect of the technical scheme are as follows: the self-selected multi-depth yolov5 model subsystem comprises: the image pixel depth detection subsystem is provided with a multi-type analysis image set pixel detection unit, and the image pixel depth in the multi-type analysis image set is detected through the multi-type analysis image set pixel detection unit; the pixel depth comparison standard subsystem is used for comparing the pixel depth of the picture with the pixel depth comparison standard according to the pixel depth of the picture by setting the pixel depth comparison standard so as to obtain a multi-type analysis picture pixel depth comparison result; the self-selection multi-depth yolov5 network monitoring subsystem automatically selects the depth of a yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructs a self-selection multi-depth yolov5 network model; displaying a self-selected multi-depth yolov5 network model through a network model visualization tool, and monitoring the self-selected multi-depth yolov5 network model; the pixel depth contrast criteria include: a first pixel depth contrast criterion, a second pixel depth contrast criterion, and a third pixel depth contrast criterion; the multi-depth yolov5 network model includes: a first depth yolov5 network model, a second depth yolov5 network model, a third depth yolov5 network model, and a fourth depth yolov5 network model; automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing the self-selected multi-depth yolov5 network model comprises the following steps: according to the pixel depth comparison result of the multi-type analysis picture, when the pixel depth comparison result of the multi-type analysis picture is lower than a first pixel depth comparison standard, automatically selecting a first depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is not lower than the first pixel depth comparison standard and not higher than the second pixel depth comparison standard, automatically selecting a second depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is higher than the second pixel depth comparison standard and not higher than the third pixel depth comparison standard, automatically selecting a third depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is higher than the third pixel depth comparison standard, automatically selecting a fourth depth yolov5 network model; the depth of the yolov5 network model can be automatically selected, the yolov5 network model is monitored, multiple types of analysis pictures can be rapidly analyzed in real time, and real-time analysis results can be output in a large scale.
In one embodiment, the target detection real-time analysis subsystem comprises:
the automatic picture size adjustment subsystem automatically adjusts the size of the input picture in the multi-type analysis picture set to a preset picture size through a pre-training weight, and combines a plurality of picture parts in the multi-type analysis picture set to form an input picture with the preset picture size;
the deep feature map real-time analysis subsystem inputs pictures according to preset picture sizes, analyzes a multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model, and outputs a deep feature map;
and the large target real-time detection and analysis subsystem is used for carrying out high-speed large target real-time detection and analysis according to the deep feature map to obtain real-time analysis results of the multi-type analysis network model.
The principle and effect of the technical scheme are as follows: the target detection real-time analysis subsystem comprises: the automatic picture size adjustment subsystem automatically adjusts the size of the input picture in the multi-type analysis picture set to a preset picture size through a pre-training weight, and combines a plurality of picture parts in the multi-type analysis picture set to form an input picture with the preset picture size; the deep feature map real-time analysis subsystem inputs pictures according to preset picture sizes, analyzes a multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model, and outputs a deep feature map; the large target real-time detection analysis subsystem is used for carrying out high-speed large target real-time detection analysis according to the deep feature map to obtain real-time analysis results of the multi-type analysis network model; the high-speed large target real-time detection analysis comprises the following steps: carrying out high-speed large-target real-time detection and analysis through a deep feature map high-speed large-target detection and analysis group; the deep feature map high-speed large target detection analysis group comprises: the device comprises a deep feature map processing and converting unit, a data storage unit, a deep feature map ffmpeg processing unit, a ffmpeg stream executing unit and a network model output unit; a multi-layer deep feature map processing and converting unit for dynamically selecting a multi-layer deep feature map ffmpeg processing unit; the data storage unit is used for storing the multi-layer deep feature map processing result of the multi-layer deep feature map ffmpeg processing unit;
The multi-layer deep feature map ffmpeg processing unit includes: the system comprises a rapid resource locator decoding subunit, a multi-layer deep feature map processing result writing subunit, a multi-layer deep feature map encoding subunit and a multi-layer deep feature map transmitting subunit; the rapid resource locator decoding subunit decodes the multi-layer deep feature map according to the rapid resource locator address; the multi-layer deep feature map processing subunit reads multi-layer deep feature map processing instruction parameters from the data storage unit to perform corresponding multi-layer deep feature map processing, and writes the processing result into the data storage unit through the multi-layer deep feature map processing result writing subunit; the fast resource locator address custom format employed in the fast resource locator decoding subunit includes: the rapid resource locator identifies a head, a first layer of characteristic diagrams, a second layer of characteristic diagrams, a third layer of characteristic diagrams and a fourth layer of characteristic diagrams; the ffmpeg stream execution unit converts the processing address dynamically allocated by the ffmpeg into a quick resource locator for the network model output unit to access, and a multi-type multi-layer deep feature map shooting device corresponds to the unique quick resource locator; the ffmpeg stream execution unit generates a multi-layer deep feature map stream address prefix, a multi-type multi-layer deep feature map recording device number and a multi-layer deep feature map stream address in a fixed domain name suffix format according to a ffmpeg dynamic selection result and an original RTSP address generation rule, wherein the multi-type multi-layer deep feature map recording device number is unique in a system; the network model output unit is used for displaying the real-time multilayer deep feature map, the processing result data of the multilayer deep feature map and the process state; the multi-type analysis pictures can be rapidly analyzed in real time, and real-time analysis results can be output in a large scale; the efficiency and the analysis depth of the video multi-platform multi-type video analysis are greatly improved.
Although embodiments of the present invention have been disclosed above, it is not limited to the details and embodiments shown and described, it is well suited to various fields of use for which the invention would be readily apparent to those skilled in the art, and accordingly, the invention is not limited to the specific details and illustrations shown and described herein, without departing from the general concepts defined in the claims and their equivalents.

Claims (8)

1. A video analysis method based on ffmpeg and yolov5, comprising:
s100, acquiring a multi-type video source file through multi-type video shooting equipment; converting the multi-type video source file into a video file processed by ffmpeg in a cross-platform manner;
s200, converting and analyzing the video files processed in a cross-platform manner into picture set files through a multi-channel video source file ffmpeg processing process according to the video files processed in the cross-platform manner, and acquiring multi-type analysis picture sets;
s300, detecting a multi-type analysis picture set, automatically selecting the depth of the yolov5 network model according to a multi-type analysis picture pixel depth comparison result, and constructing a self-selected multi-depth yolov5 network model;
s400, analyzing a multi-type analysis picture set in real time through a self-selection multi-depth yolov5 network model to obtain a real-time analysis result of the multi-type analysis network model;
S300 includes:
s301, setting a multi-type analysis picture set pixel detection unit, and detecting the picture pixel depth in the multi-type analysis picture set through the multi-type analysis picture set pixel detection unit;
s302, according to the pixel depth of the picture, comparing the pixel depth of the picture with a pixel depth comparison standard by setting the pixel depth comparison standard to obtain a multi-type analysis picture pixel depth comparison result;
s303, automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing a self-selected multi-depth yolov5 network model; displaying a self-selected multi-depth yolov5 network model through a network model visualization tool, and monitoring the self-selected multi-depth yolov5 network model;
the pixel depth contrast criteria include: a first pixel depth contrast criterion, a second pixel depth contrast criterion, and a third pixel depth contrast criterion;
the multi-depth yolov5 network model includes: a first depth yolov5 network model, a second depth yolov5 network model, a third depth yolov5 network model, and a fourth depth yolov5 network model; automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing the self-selected multi-depth yolov5 network model comprises the following steps: according to the pixel depth comparison result of the multi-type analysis picture, when the pixel depth comparison result of the multi-type analysis picture is lower than a first pixel depth comparison standard, automatically selecting a first depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is not lower than the first pixel depth comparison standard and not higher than the second pixel depth comparison standard, automatically selecting a second depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is higher than the second pixel depth comparison standard and not higher than the third pixel depth comparison standard, automatically selecting a third depth yolov5 network model; and when the pixel depth comparison result of the multi-type analysis picture is higher than the third pixel depth comparison standard, automatically selecting a fourth depth yolov5 network model.
2. The video analysis method based on ffmpeg and yolov5 of claim 1, wherein S100 comprises:
s101, shooting multiple types of videos through multiple types of video shooting equipment to acquire multiple types of video source files; the multi-type video source file includes: MPEG, WMV, AVI, MKV or OGG;
s102, setting a cross-platform conversion tool of a video source file according to a plurality of types of video source files;
s103, converting the multi-type video source file into a video file processed by the ffmpeg in a cross-platform manner through a cross-platform conversion tool of the video source file;
the video source file cross-platform conversion tool comprises: the source development platform converts the non-source development platform tool and the non-source development platform converts the source development platform tool; the source development platform conversion non-source development platform tool comprises: a first source video window drive conversion model and a second source video window drive conversion model; the non-source development platform conversion source development platform tool comprises: a first target video window drive conversion model and a second target video window drive conversion model; when a source development platform is required to convert the video source file into a non-source development platform, converting the video source file through a first source video window driving conversion model or a second source video window driving conversion model; when a source development platform is required to be converted from a non-source development platform to a source development platform, converting the video source file through a first target video window driving conversion model or a second target video window driving conversion model; when the inter-platform conversion is required, the first source video window driving conversion model corresponds to the first target video window driving conversion model, and the second source video window driving conversion model corresponds to the second target video window driving conversion model.
3. The video analysis method based on ffmpeg and yolov5 of claim 1, wherein S200 comprises:
s201, setting a multi-path video source file ffmpeg processing process according to a video file processed in a cross-platform manner; the multipath video source file ffmpeg processing process comprises the following steps: a ffmpeg video source file catalog monitoring process, a ffmpeg video source file processing control process and a ffmpeg processing output image frame reading timing release process;
s202, converting and analyzing the video files processed in a cross-platform manner into picture set files through a multipath video source file ffmpeg processing process to obtain a multi-type analysis picture set.
4. The video analysis method based on ffmpeg and yolov5 of claim 1, wherein S400 comprises:
s401, automatically adjusting the size of an input picture in a multi-type analysis picture set to a preset picture size through a pre-training weight, and combining a plurality of picture parts in the multi-type analysis picture set to form an input picture with the preset picture size;
s402, inputting pictures according to preset picture sizes, analyzing a multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model, and outputting a deep feature map;
s403, carrying out high-speed large-target real-time detection analysis according to the deep feature map, and obtaining real-time analysis results of the multi-type analysis network model.
5. A video analytics system based on ffmpeg and yolov5, comprising:
the video shooting and recording multi-platform conversion subsystem acquires multi-type video source files through multi-type video shooting and recording equipment; converting the multi-type video source file into a video file processed by ffmpeg in a cross-platform manner;
the ffmpeg multi-path processing analysis subsystem converts and analyzes the video files processed in a cross-platform manner into a picture set file through a multi-path video source file ffmpeg processing process according to the video files processed in the cross-platform manner, and acquires a multi-type analysis picture set;
a self-selection multi-depth yolov5 model subsystem detects a multi-type analysis picture set, automatically selects the depth of a yolov5 network model according to a multi-type analysis picture pixel depth comparison result, and constructs a self-selection multi-depth yolov5 network model;
the target detection real-time analysis subsystem is used for analyzing the multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model to obtain a real-time analysis result of the multi-type analysis network model;
the self-selected multi-depth yolov5 model subsystem comprises:
the image pixel depth detection subsystem is provided with a multi-type analysis image set pixel detection unit, and the image pixel depth in the multi-type analysis image set is detected through the multi-type analysis image set pixel detection unit;
The pixel depth comparison standard subsystem is used for comparing the pixel depth of the picture with the pixel depth comparison standard according to the pixel depth of the picture by setting the pixel depth comparison standard so as to obtain a multi-type analysis picture pixel depth comparison result;
the self-selection multi-depth yolov5 network monitoring subsystem automatically selects the depth of a yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructs a self-selection multi-depth yolov5 network model; displaying a self-selected multi-depth yolov5 network model through a network model visualization tool, and monitoring the self-selected multi-depth yolov5 network model;
the pixel depth contrast criteria include: a first pixel depth contrast criterion, a second pixel depth contrast criterion, and a third pixel depth contrast criterion;
the multi-depth yolov5 network model includes: a first depth yolov5 network model, a second depth yolov5 network model, a third depth yolov5 network model, and a fourth depth yolov5 network model; automatically selecting the depth of the yolov5 network model according to the pixel depth comparison result of the multi-type analysis picture, and constructing the self-selected multi-depth yolov5 network model comprises the following steps: according to the pixel depth comparison result of the multi-type analysis picture, when the pixel depth comparison result of the multi-type analysis picture is lower than a first pixel depth comparison standard, automatically selecting a first depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is not lower than the first pixel depth comparison standard and not higher than the second pixel depth comparison standard, automatically selecting a second depth yolov5 network model; when the pixel depth comparison result of the multi-type analysis picture is higher than the second pixel depth comparison standard and not higher than the third pixel depth comparison standard, automatically selecting a third depth yolov5 network model; and when the pixel depth comparison result of the multi-type analysis picture is higher than the third pixel depth comparison standard, automatically selecting a fourth depth yolov5 network model.
6. The ffmpeg and yolov 5-based video analytics system of claim 5, wherein the video camcorder multi-platform conversion subsystem comprises:
the multi-type video shooting subsystem shoots and records multi-type videos through multi-type video shooting equipment to acquire multi-type video source files; the multi-type video source file includes: MPEG, WMV, AVI, MKV or OGG;
the video file cross-platform conversion subsystem sets a video source file cross-platform conversion tool according to the multi-type video source files;
the file conversion acquisition subsystem converts the multi-type video source files into video files processed by the ffmpeg in a cross-platform manner through a cross-platform conversion tool of the video source files;
the video source file cross-platform conversion tool comprises: the source development platform converts the non-source development platform tool and the non-source development platform converts the source development platform tool; the source development platform conversion non-source development platform tool comprises: a first source video window drive conversion model and a second source video window drive conversion model; the non-source development platform conversion source development platform tool comprises: a first target video window drive conversion model and a second target video window drive conversion model; when a source development platform is required to convert the video source file into a non-source development platform, converting the video source file through a first source video window driving conversion model or a second source video window driving conversion model; when a source development platform is required to be converted from a non-source development platform to a source development platform, converting the video source file through a first target video window driving conversion model or a second target video window driving conversion model; when the inter-platform conversion is required, the first source video window driving conversion model corresponds to the first target video window driving conversion model, and the second source video window driving conversion model corresponds to the second target video window driving conversion model.
7. The ffmpeg and yolov 5-based video analytics system of claim 5, wherein the ffmpeg multiprocessing analytics subsystem comprises:
the ffmpeg processing process subsystem sets a multi-path video source file ffmpeg processing process according to the cross-platform processed video file; the multipath video source file ffmpeg processing process comprises the following steps: a ffmpeg video source file catalog monitoring process, a ffmpeg video source file processing control process and a ffmpeg processing output image frame reading timing release process;
and the file conversion analysis subsystem converts and analyzes the video file processed in a cross-platform manner into a picture set file through a multipath video source file ffmpeg processing process to obtain a multi-type analysis picture set.
8. The ffmpeg and yolov 5-based video analytics system of claim 5, wherein the object detection real-time analytics subsystem comprises:
the automatic picture size adjustment subsystem automatically adjusts the size of the input picture in the multi-type analysis picture set to a preset picture size through a pre-training weight, and combines a plurality of picture parts in the multi-type analysis picture set to form an input picture with the preset picture size;
the deep feature map real-time analysis subsystem inputs pictures according to preset picture sizes, analyzes a multi-type analysis picture set in real time through a self-selected multi-depth yolov5 network model, and outputs a deep feature map;
And the large target real-time detection and analysis subsystem is used for carrying out high-speed large target real-time detection and analysis according to the deep feature map to obtain real-time analysis results of the multi-type analysis network model.
CN202310910520.5A 2023-07-24 2023-07-24 Video analysis method and system based on ffmpeg and yolov5 Active CN116886955B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310910520.5A CN116886955B (en) 2023-07-24 2023-07-24 Video analysis method and system based on ffmpeg and yolov5

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310910520.5A CN116886955B (en) 2023-07-24 2023-07-24 Video analysis method and system based on ffmpeg and yolov5

Publications (2)

Publication Number Publication Date
CN116886955A CN116886955A (en) 2023-10-13
CN116886955B true CN116886955B (en) 2024-04-16

Family

ID=88271259

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310910520.5A Active CN116886955B (en) 2023-07-24 2023-07-24 Video analysis method and system based on ffmpeg and yolov5

Country Status (1)

Country Link
CN (1) CN116886955B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109816118A (en) * 2019-01-25 2019-05-28 上海深杳智能科技有限公司 A kind of method and terminal of the creation structured document based on deep learning model
CN114723944A (en) * 2022-03-28 2022-07-08 长春长光博翔无人机有限公司 Image analysis method, storage medium, and electronic device
CN114885204A (en) * 2021-02-05 2022-08-09 山西云芯新一代信息技术研究院有限公司 Web-based AR scene interaction sdk
CN115514982A (en) * 2022-08-09 2022-12-23 慧之安信息技术股份有限公司 FFmpeg-based education live broadcast method and system
CN115512333A (en) * 2022-10-09 2022-12-23 北京无线电测量研究所 Vehicle target detection method and system of SAR image
CN116385854A (en) * 2023-02-27 2023-07-04 深圳市华普微电子股份有限公司 YOLO vehicle detection method based on TrustZone technology

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10818033B2 (en) * 2018-01-18 2020-10-27 Oath Inc. Computer vision on broadcast video

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109816118A (en) * 2019-01-25 2019-05-28 上海深杳智能科技有限公司 A kind of method and terminal of the creation structured document based on deep learning model
CN114885204A (en) * 2021-02-05 2022-08-09 山西云芯新一代信息技术研究院有限公司 Web-based AR scene interaction sdk
CN114723944A (en) * 2022-03-28 2022-07-08 长春长光博翔无人机有限公司 Image analysis method, storage medium, and electronic device
CN115514982A (en) * 2022-08-09 2022-12-23 慧之安信息技术股份有限公司 FFmpeg-based education live broadcast method and system
CN115512333A (en) * 2022-10-09 2022-12-23 北京无线电测量研究所 Vehicle target detection method and system of SAR image
CN116385854A (en) * 2023-02-27 2023-07-04 深圳市华普微电子股份有限公司 YOLO vehicle detection method based on TrustZone technology

Also Published As

Publication number Publication date
CN116886955A (en) 2023-10-13

Similar Documents

Publication Publication Date Title
US20220046292A1 (en) Networked system for real-time computer-aided augmentation of live input video stream
WO2019237657A1 (en) Method and device for generating model
CN107707931B (en) Method and device for generating interpretation data according to video data, method and device for synthesizing data and electronic equipment
WO2019047789A1 (en) Augmented reality scene related processing method, terminal device and system and computer storage medium
CN111476871B (en) Method and device for generating video
CN113691836B (en) Video template generation method, video generation method and device and electronic equipment
CN111523566A (en) Target video clip positioning method and device
US20110050939A1 (en) Image processing apparatus, image processing method, program, and electronic device
US20100289913A1 (en) Video processing apparatus, and control method and program therefor
CN112511818B (en) Video playing quality detection method and device
CN110310299B (en) Method and apparatus for training optical flow network, and method and apparatus for processing image
WO2020052062A1 (en) Detection method and device
JP7204786B2 (en) Visual search method, device, computer equipment and storage medium
CN107133631A (en) A kind of method and device for recognizing TV station's icon
US11776272B2 (en) Method for comparing videos of surgical techniques
CN116886955B (en) Video analysis method and system based on ffmpeg and yolov5
CN114419502A (en) Data analysis method and device and storage medium
WO2019214019A1 (en) Online teaching method and apparatus based on convolutional neural network
CN111611973B (en) Target user identification method, device and storage medium
CN113570689A (en) Portrait cartoon method, apparatus, medium and computing device
JP2022526473A (en) Methods and devices for acquiring information, electronic devices, storage media and computer programs
CN109871465B (en) Time axis calculation method and device, electronic equipment and storage medium
CN115499677A (en) Audio and video synchronization detection method and device based on live broadcast
US11042274B2 (en) Extracting demonstrations from in-situ video content
EP4165635A1 (en) Automatic non-linear editing style transfer

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant