CN114708643B - Computing power improving method for edge video analysis device and edge video analysis device - Google Patents
Computing power improving method for edge video analysis device and edge video analysis device Download PDFInfo
- Publication number
- CN114708643B CN114708643B CN202210621368.4A CN202210621368A CN114708643B CN 114708643 B CN114708643 B CN 114708643B CN 202210621368 A CN202210621368 A CN 202210621368A CN 114708643 B CN114708643 B CN 114708643B
- Authority
- CN
- China
- Prior art keywords
- module
- algorithm
- configuration information
- computing power
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Picture Signal Circuits (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention relates to a computing power improving method of an edge video analysis device and the edge video analysis device, the edge video analysis device provided by the embodiment of the invention comprises more than two computing power modules, and the more than two computing power modules realize cascade connection through an internal network; the configuration information of each calculation force module is obtained, and then the corresponding module is controlled to run the appointed algorithm model according to the configuration information; and then different computational power modules analyze the video stream or the picture stream according to the running algorithm model. The method and the device of the invention preset a plurality of algorithm models in the computing power module added in the video analysis device, and then call the corresponding module according to different visual tasks in the configuration information, thereby achieving the effect of improving the computing power of the edge computing equipment and realizing the function of simultaneously operating a plurality of algorithm network models by the same edge computing equipment.
Description
Technical Field
The present invention relates to the field of edge computing technologies, and in particular, to a computing power improving method for an edge video analysis device and an edge video analysis device.
Background
At present, an AI chip is generally adopted as a main control chip in an edge video intelligent analysis device, and detection tasks such as specific targets, intelligent events and the like in videos and pictures are executed by loading an algorithm model on the chip. The conventional AI chip on the edge side is limited in working environment, design positioning and other conditions, so that the calculation power is limited generally, and the economic scheme is a level below 4T calculation power.
However, for designs that employ lower computational power levels and localized edges, it is not possible for the device to run multiple algorithmic models simultaneously to perform different algorithmic parameters; moreover, under the condition of multi-video input, the real-time performance of the operation of the equipment cannot be effectively ensured, so that the problems of target detection omission, incapability of tracking in real time, low detection accuracy and the like occur. On the other hand, for the computational power chip with higher performance, the extremely severe working environment and stability requirements of the edge side equipment cannot be met, and the cost is greatly increased.
Disclosure of Invention
The embodiment of the invention provides a computing power improving method of an edge video analysis device and the edge video analysis device, and at least solves the problem that a plurality of algorithm network models cannot be operated simultaneously due to low computing power of edge computing equipment in the related art.
In a first aspect, an embodiment of the present invention provides a computing power improving method for an edge video analysis device, including a main control module and two or more computing power modules, where the main control module performs data interaction with each computing power module through a private protocol, and the two or more computing power modules implement cascade connection through an internal network; the main control module is accessed to a peripheral data source and interacts with an upper platform through network communication, and the method comprises the following steps:
acquiring first configuration information of each computing module through the main control module, and sending the first configuration information to a human-computer interaction interface for a user to change to acquire second configuration information;
sending the second configuration information to a corresponding force calculation module through the main control module so that the force calculation module can operate a corresponding algorithm model according to the second configuration information;
and acquiring a video stream or a picture stream according to the second configuration information and sending the video stream or the picture stream to the force calculation module so that the force calculation module can analyze the video stream or the picture stream according to an algorithm model operated by the force calculation module.
More specifically, each computing force module is preset with a plurality of algorithm models, and configuration information comprises video channel information, algorithm model information and algorithm parameters of each computing force module;
the video channel information is the stream taking information of the video stream or the picture stream acquired by the computing module; the algorithm model information comprises the type and the version of an algorithm model, and the algorithm model comprises three types of face recognition, behavior analysis and target detection; the algorithm parameters include monitoring area, sensitivity threshold, time threshold, task status and target type.
In some of these embodiments, the method further comprises:
the second configuration information is sent to the force calculation module for operation verification;
obtaining verification information fed back by each force calculation module, wherein the verification information is a result fed back by the force calculation module after the force calculation module performs conformance verification according to the second configuration information; if the configuration information does not accord with the operation condition of the computing power module, the verification information comprises a verification result and error information;
and prompting a user to modify the configuration information according to the verification information.
More specifically, the main control module performs data interaction with the computing power module through a private protocol and performs data interaction with a peripheral data source and an upper platform through a public protocol;
the acquiring a video stream or a picture stream according to the second configuration information and sending the video stream or the picture stream to the computing power module includes:
extracting the video channel information of each computing module and the equipment ID of an external data source thereof from the second configuration information;
acquiring a video stream or a picture stream of the peripheral data source according to the equipment ID;
and transmitting the video stream or the picture stream to a corresponding force calculation module through the video channel information.
More specifically, the algorithm module executes a corresponding algorithm model according to the second configuration information, including:
extracting an algorithm model to be executed by each computing power module from the second configuration information; and the computing force module calls a corresponding algorithm according to the algorithm model and starts to operate.
More specifically, the algorithm module analyzes the video stream or the picture stream according to an algorithm model operated by the algorithm module, and the method comprises the following steps:
extracting algorithm parameters to be executed by each computing power module from the second configuration information;
the calculation module analyzes and judges each frame image of the received video stream or picture stream according to the algorithm parameters;
and when the images in the video stream or the picture stream accord with the algorithm parameters, feeding back an analysis judgment result to the user.
In some of these embodiments, the method further comprises:
obtaining an algorithm upgrading file, and extracting algorithm model information from the algorithm upgrading file to judge the type and upgrading parameters of an algorithm model to be upgraded;
upgrading the corresponding algorithm model according to the upgrading parameters according to the types;
and after each algorithm model is upgraded, feeding back the type and version information of the algorithm model of each algorithm module to a user.
In a second aspect, an embodiment of the present invention provides an edge video analysis apparatus, including a main control module and two or more force calculation modules, where the main control module performs data interaction with each force calculation module through a private protocol, and the two or more force calculation modules implement cascade connection through an internal network; the main control module is accessed to a peripheral data source and interacts with the upper layer platform through network communication, wherein,
the main control module is used for acquiring first configuration information of each computing module, and sending the first configuration information to a human-computer interaction interface for a user to change so as to acquire second configuration information; sending the second configuration information to a corresponding computing power module; acquiring a video stream or a picture stream according to the second configuration information and sending the video stream or the picture stream to the computing power module;
and the computing force module is used for operating a corresponding algorithm model according to the second configuration information and analyzing the video stream or the picture stream according to the operated algorithm model.
In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, where the memory stores a computer program, and the processor is configured to execute the computer program to perform the computing power improving method of the edge video analysis apparatus described in any one of the above.
In a fourth aspect, an embodiment of the present invention provides a storage medium, where a computer program is stored, where the computer program is configured to execute the computing power improving method of the edge video analysis apparatus according to any one of the above embodiments when the computer program runs.
Compared with the related art, the edge video analysis device provided by the embodiment of the invention comprises more than two force calculation modules, wherein the more than two force calculation modules are cascaded through an internal network; the configuration information of each calculation force module is obtained, and then the corresponding module is controlled to run the appointed algorithm model according to the configuration information; and then different calculation modules analyze the video stream or the picture stream according to the running algorithm model. The method and the device of the invention preset a plurality of algorithm models in the computing power module added in the video analysis device, and then call the corresponding module according to different visual tasks in the configuration information, thereby achieving the effect of improving the computing power of the edge computing equipment and realizing the function of simultaneously operating a plurality of algorithm network models by the same edge computing equipment.
The invention can not only realize different operation requirements by adjusting the algorithm models in each calculation force module, but also improve the access analysis capability of the whole device by calling the same algorithm model in each calculation force module, namely improve the whole calculation force of the device; for example, products of the original 4-way behavior analysis scheme can be promoted and developed by the method to support 16-way behavior analysis. Different algorithm models are preset in each calculation force module, so that concurrence of different calculation tasks of the system can be realized; for example, the original product only supporting behavior analysis can be promoted and developed to support the integrated products of behavior analysis, target detection and face recognition through the technical scheme.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention and not to limit the invention. In the drawings:
fig. 1 is a flowchart of a computing power improvement method of an edge video analysis apparatus according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an edge video analysis apparatus according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of data flow of an edge video analysis apparatus according to an embodiment of the present invention;
fig. 4 is a schematic diagram of the internal structure of the electronic device according to the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments provided by the present invention, belong to the protection scope of the present invention. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is to be expressly and implicitly understood by one of ordinary skill in the art that the described embodiments of the present invention may be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The use of the terms "a" and "an" and "the" and similar referents in the context of describing the invention are not to be construed as limiting in number, and may be construed to cover both the singular and the plural. The present invention relates to the terms "comprises," "comprising," "includes," "including," "has," "having" and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. The terms "connected," "coupled," and the like, as used herein, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as used herein means greater than or equal to two. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. The terms "first," "second," "third," and the like in reference to the present invention are used merely to distinguish between similar objects and not necessarily to represent a particular ordering for the objects.
The invention provides a computing power improving method of an edge video analysis device, which is mainly applied to the edge video analysis device. Data flow and processing are carried out through a private protocol F6 (the private protocol is a protocol which is independently developed and used in the interior and mainly used for avoiding the problem of data leakage) and a computing power module, and code stream distribution, algorithm parameter configuration, algorithm result subscription, alarm capture, algorithm model upgrading and the like are specifically packaged. The main control module is responsible for accessing a peripheral data source and interacting with an upper layer platform, wherein the peripheral data source such as an IPC (IP Camera, a network Camera, a digital video device integrating the functions of a video server and a Camera) is accessed through a standard ONVIF/RTSP/GB28181 protocol; and the upper-layer platform interacts with the upper-layer platform through a ZNLINK protocol developed based on MQTT, and the interactive content comprises the functions of alarm subscription and video stream pushing. The main control module and the force calculation module realize cascade communication through a built-in Switch chip.
In the invention, various operational algorithm models are preset in each computing power module, namely the algorithm models on each computing power module can be the same and are used for increasing the concurrency capability of the same visual task; or may be different for running different visual tasks. The main control module accesses external video or picture data (i.e. video stream or picture stream) through a peripheral network and a related video protocol. The main control module obtains the configuration information and the work tasks of the force calculation modules through a private interaction protocol, and the configuration information and the task information of the force calculation modules are modified and changed on the main control module and are updated to the force calculation modules.
Referring to fig. 1, the present invention provides a method for expanding the computation power of an edge video analysis apparatus according to the present invention, which mainly includes the following steps.
Step S1, the main control module obtains the first configuration information of each computation power module through the private protocol, and sends the first configuration information to the human-computer interaction interface for the user to change, so as to obtain the second configuration information. The configuration information of this embodiment includes video channel information, algorithm model information, and algorithm parameters of each computation module, and the video channel information of the present invention is stream fetching information of each computation module for obtaining a video stream or a picture stream, and includes information such as communication interface information between the computation module and the main control module, direct link information between the main control module and a peripheral data source, or device ID of the peripheral data source; the algorithm model information comprises the type and the version of the algorithm model; the algorithm parameters are task information of different visual tasks, such as a falling detection task of a person, and the algorithm parameters of the task comprise a detection area, detection sensitivity, falling time and the like.
In this embodiment, the first configuration information is the configuration information in the force calculation module before modification by the user, and the second information is the configuration information that is sent to the force calculation module by the main control module after modification by the user. For the step of obtaining the second configuration information, first, the main control module of this embodiment obtains video channel information (stream taking information), algorithm model information (such as types: POSE, FACE, OBJDCT, etc.) and algorithm parameters (monitoring area, sensitivity threshold, time threshold, task state, target type, etc.) of a certain computation module according to a preset video channel. Then, the main control module provides the acquired algorithm parameters to a user through a human-computer interaction interface to edit and modify parameter contents, for example, the parameter contents include algorithm tasks to be executed, detection areas are configured through a real-time monitoring picture, an alarm time threshold value is set, target types to be detected, and the like.
The algorithm model of the embodiment of the invention mainly protects the three types: POSE, FACE and OBJDCT, these algorithm models can be developed according to the needs of customers, and also can adopt the commonly used image processing algorithm or video processing algorithm in the prior art, etc. The POSE algorithm model generally adopts a human skeleton key point algorithm, is adaptively adjusted or researched and developed based on an OPEPOSE algorithm and combined with the industry, and is additionally provided with a target tracking algorithm and a post-processing algorithm on the basis of detecting human skeleton key points by the OPEPOSE algorithm, so that the functions of crossing warning lines, breaking into forbidden areas, falling down people, target number of people and the like of target human bodies in a video are realized. The FACE algorithm model is an algorithm of a third party company and mainly integrates a human FACE and human figure structural algorithm. The method comprises the following steps of (1) a block face picture detection algorithm and a face characteristic value comparison algorithm, wherein the algorithm is used for realizing the functions of capturing faces in a video stream, extracting face characteristic values, carrying out face recognition and the like; the human body characteristics are recognized by the human body shape structural algorithm, such as whether a hat is worn or not, whether glasses are worn or not, the color of the upper garment, the color of the lower garment, the long-short sleeves and the like, and the human body shape structural algorithm is used for realizing the recognition functions of whether a target human body wears a safety helmet or not, whether a work garment is worn or not and the like. The OBJDCT algorithm model is a target detection algorithm and is an algorithm developed autonomously based on Segnet semantic segmentation, and the algorithm is mainly used for detecting events such as flame, smoke and the like in a video picture.
And step S2, the main control module sends the second configuration information to the corresponding force calculation module, so that the force calculation module runs the corresponding algorithm model according to the second configuration information. Specifically, the main control software issues the second configuration information modified by the user to the force calculation module, and in this embodiment, the configuration information may be issued to the designated force calculation module, or may be selected by the main control module according to the state of the force calculation module. For the former, the main control module can send the second configuration information to the force calculation module waiting for the ID only by adding the chip ID of the force calculation module needing to be specified in the configuration information; for the latter, before the main control module issues the configuration information to the force calculation modules, the main control module actively acquires the working condition of each force calculation module, selects the idle force calculation modules in the front sequence according to the preset serial number, and then sends the second configuration information to the force calculation modules. After receiving the second configuration information, the force calculation module extracts the algorithm model information to be executed by each force calculation module from the second configuration information, and then calls a corresponding algorithm according to the algorithm model information (the type and the version number of the algorithm model) and starts to operate.
And step S3, acquiring the video stream or the picture stream according to the configuration information and sending the video stream or the picture stream to the force calculation module, so that the force calculation module analyzes the video stream or the picture stream according to the running algorithm model. Specifically, first, video channel information of any one of the computation module is extracted from the second configuration information, for example, a device ID of an external data source for which the computation module needs to acquire video data, and a video stream or a picture stream of the corresponding external data source is acquired according to the video channel; and then transmitting the video stream or the picture stream to a corresponding force calculation module through video channel information.
In this embodiment, the algorithm parameters include monitoring area, sensitivity threshold, time threshold, task status, and target type. In the process that the computing power module analyzes the video stream or the picture stream according to the running algorithm model thereof, firstly, extracting algorithm parameters to be executed by each computing power module from second configuration information; the calculation module analyzes and judges each frame image of the received video stream or picture stream according to the algorithm parameters; and when the images in the video stream or the picture stream accord with the algorithm parameters, feeding back the analysis judgment result to the user.
Compared with the related art, the edge video analysis device provided by the embodiment of the invention comprises more than two force calculation modules, wherein the more than two force calculation modules are cascaded through an internal network; the configuration information of each calculation force module is obtained, and then the corresponding module is controlled to run the appointed algorithm model according to the configuration information; and then different calculation modules analyze the video stream or the picture stream according to the running algorithm model. The method and the device of the invention preset a plurality of algorithm models in the computing power module added in the video analysis device, and then call the corresponding module according to different visual tasks in the configuration information, thereby achieving the effect of improving the computing power of the edge computing equipment and realizing the function of simultaneously operating a plurality of algorithm network models by the same edge computing equipment.
The invention can not only realize different operation requirements by adjusting the algorithm models in each calculation force module, but also improve the access analysis capability of the whole device by calling the same algorithm model in each calculation force module, namely improve the whole calculation force of the device. Specifically, when four computational power modules are arranged in one analysis device and only one computational power module supports a behavior analysis algorithm, algorithm models of other three computational power modules can be modified into the behavior analysis algorithm by the method of the scheme, and products of the original 4-path behavior analysis scheme can be promoted and developed to support 16-path behavior analysis by the method. Different algorithm models are preset in each calculation force module, so that concurrence of different calculation tasks of the system can be realized; for example, the original product only supporting behavior analysis can be promoted and developed to support the integrated products of behavior analysis, target detection and face recognition through the technical scheme.
In other embodiments of the present invention, the force calculation module may monitor the configuration information issued by the main control module in real time, and after receiving the configuration information, the force calculation module performs a compliance check on the configuration parameters issued by the main control module, for example, whether the force calculation module itself can operate the force calculation module or execute the task type, and then matches the algorithm parameters, and feeds back error information that may occur to the main control module after the check is completed. Specifically, verification information fed back by each force calculation module is obtained, and the verification information is a result fed back by the force calculation module after the second configuration information is subjected to conformance verification. If the configuration information does not accord with the operation condition of the computing power module, the verification information comprises a verification result and error information; and if the configuration information meets the operation condition of the computing power module, the verification information comprises contents of 'verification pass' and the like. And when receiving the verification signal, the main control module directly sends the verification signal to a man-machine interaction interface to be presented to a user, so that the user can check and modify parameters corresponding to the error information returned by the force calculation module. If error information exists, so that the calculation module cannot execute the video analysis task, a user is required to modify relevant error parameters according to prompts. Even if the verification is passed, the computing power module can feed back some operation parameters when the algorithm is executed, so that a user can optimize and adjust the configuration parameters of the computing power module according to the operation parameters.
In another embodiment of the present invention, a visual task may be added to the same computing power module, for example, the main control module obtains, through a private protocol, that a task of detecting a safety helmet is being performed on any computing power module, that is, whether a person entering a worksite wears a safety helmet is detected. The main control module sends the task and the operation parameter which are being executed by the computing power module to a human-computer interaction interface, and the task and the operation parameter are presented to a user through the human-computer interaction interface; then, the user can modify the operation parameters of the computing power module through the man-machine interaction interface, and a work clothes detection task is added. And then the main control module issues the new configuration information to the corresponding force calculation module, and at the moment, the same force calculation module simultaneously executes the tasks of safety helmet detection and work clothes detection on the received video or picture data according to the new parameters.
In another embodiment of the invention, the algorithm configuration of the force calculation module can be modified, specifically, the main control module obtains the algorithm parameters of each force calculation module, the main control module sends the algorithm parameters to the human-computer interaction interface, the algorithm parameters are presented to the user through the human-computer interaction interface, and the user adjusts the parameters of a certain algorithm through the configuration interface of the human-computer interaction terminal, for example, for a detection task of falling down of a person, the parameters of a detection area, detection sensitivity, falling down confirmation time and the like of the falling down of the person can be adjusted. And then the main control module issues the new parameters to the corresponding force calculation module through a private protocol, and the force calculation module executes the corresponding algorithm task according to the new algorithm configuration parameters.
In another embodiment of the invention, the method of the invention can also upgrade the algorithm model, specifically, firstly, the algorithm upgrade file is obtained, and the algorithm model information is extracted from the algorithm upgrade file to judge the type and upgrade parameters of the algorithm model to be upgraded; then upgrading the corresponding algorithm model according to the upgrading parameters according to the types; and finally, after the upgrading of each algorithm model is completed, feeding back the type and version information of the algorithm model of each algorithm module to a user.
In another embodiment of the present invention, an edge video analysis apparatus is provided, which includes a main control module and two or more force calculation modules, where the main control module performs data interaction with each force calculation module through a private protocol, and the two or more force calculation modules implement cascade connection through an internal network. The main control module is used for acquiring first configuration information of each computing power module through a private protocol, and sending the first configuration information to a human-computer interaction interface for a user to change so as to acquire second configuration information; then sending the second configuration information to a corresponding force calculation module; and finally, acquiring a video stream or a picture stream according to the second configuration information and sending the video stream or the picture stream to the force calculation module. Each computational force module is used for operating a corresponding algorithm model according to the second configuration information and then analyzing the received video stream or the picture stream according to the operated algorithm model.
In an embodiment of the present invention, referring to fig. 2, the edge video analysis device of this embodiment has four computation power modules, the main control module is connected to the four computation power modules through an RTL8367 switch chip, and the four computation power modules also implement cascade connection through the RTL8367 switch chip (internal network). More specifically, the HDMI interface and the VGA interface in fig. 2 are device local GUI (graphical user interface) output interfaces for real-time video preview and real-time waterfall stream display of algorithm analysis results; the GPIO interface is a relay output terminal interface and is used for linkage output of algorithm analysis results, such as alarm signal buzzing control, lighting switches and the like; the USB Host port is a USB expansion interface and is used for expanding a USB interface of the device and downloading event pictures and video clips through a USB upgradable device software program; the EMIF port is an internal storage chip interface, and an SPI FLASH type memory can be configured to be used as a storage hard disk of a software program of the device; the DDR CTRL port is a DDR memory control interface of the main control chip; the PHY port is a network communication interface and is used for installing an external network, such as a video source connected with a network camera and the like; the I2S port audio bus interface can be used for configuring an audio coding and decoding chip, and is provided with an audio input and output interface; the PCIE port is a PCIE bus interface and is used for expanding an internal network, the invention uses RTL8111H to expand the PCIE into an internal network, and configures an RTL8367 exchange chip as a network access interface of the computing power module; the SATA port is a data interface of an external storage hard disk, and software stores data such as video, pictures of algorithm analysis results, time information and the like through the interface.
The main control module of the invention has the following functions:
code stream configuration, wherein the main control module configures the code stream configuration of each computational power module through a private protocol, so that the computational power modules take streams from the main control module;
the main control module issues algorithm requirements of a user to a force calculation module with a corresponding algorithm model through a private protocol through configuration parameters, so that the force calculation module performs algorithm analysis on video data output by a video source specified by the user, and performs real-time intelligent analysis on the video source according to parameters such as a detection area, a target detection size, detection frequency, confidence coefficient, a trigger rule and a time threshold;
the system comprises an alarm subscription and a snapshot, wherein the master control module performs alarm subscription on a force calculation module through a private protocol, the force calculation module informs the master control module of a detection result through an intelligent event according to an algorithm parameter, and the master control module receives alarm event information and performs intelligent event snapshot;
and (3) upgrading the algorithm model, wherein the main control module receives an algorithm model upgrading file designated by a user, judges the type of the algorithm model, calls an interface of the force calculation module through a private protocol to verify and upgrade the algorithm model, and feeds back the type and version information of the algorithm model to the main control module after the force calculation module finishes upgrading the algorithm model.
Referring to fig. 3, fig. 3 is a data flow diagram between a main control module and a force calculation module and an external device, in an operation process of the edge video analysis device of the present invention, the main control module obtains a video stream or a picture stream from an external data source, the force calculation module pulls video or picture data from the main control module and loads a corresponding algorithm network model for analysis according to configuration information received by the main control module, and feeds back an analysis result to the main control module, and the main control module sends the algorithm analysis result data of the force calculation module to a WEB interface or a third party platform through an interaction protocol. The four force calculation modules of this embodiment: the force calculation module 1 and the force calculation module 2 call an algorithm model X for face recognition; the calculation force module 3 calls an algorithm model Y for analyzing human behaviors; the calculation force module 4 calls an algorithm model Z for target detection.
The main control module also manages the link of the hardware main control scheme and the force calculation module and is responsible for IP distribution, communication and real-time monitoring of working states. And providing a human-computer interaction interface and a protocol interaction entrance of a third-party platform for a user outside the system. The force calculation module is mainly responsible for data interaction with the main control module and management of hardware of each force calculation module, receives protocol instructions of the main control module, such as flow taking, parameter configuration, task distribution, model upgrading and the like, and executes algorithm detection and analysis tasks.
The invention takes a main control chip (main control module) and a plurality of force calculation modules which are in network cascade connection as hardware bases, and the force calculation capability of the whole device is expanded through the mutual cooperation of the main control module and the plurality of force calculation modules, so that the system can process a plurality of algorithm models in parallel to execute complex visual tasks.
It should be noted that the above modules may be functional modules or program modules, and may be implemented by software or hardware. For a module implemented by hardware, the modules may be located in the same processor; or the modules may be located in different processors in any combination.
It should be noted that, for specific examples in this embodiment, reference may be made to examples described in the foregoing embodiments and optional implementations, and details of this embodiment are not described herein again.
In addition, in combination with the calculation capability improving method of the edge video analysis device in the above embodiment, the embodiment of the present invention may provide a storage medium to implement. The storage medium having stored thereon a computer program; the computer program, when executed by a processor, implements the computing power improving method of any one of the edge video analysis devices in the above embodiments.
An embodiment of the invention also provides an electronic device, which can be a terminal. The electronic device comprises a processor, a memory, a network interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the electronic device is used for connecting and communicating with an external terminal through a network. The computer program is executed by a processor to implement a computing power improving method of an edge video analysis apparatus. The display screen of the electronic equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the electronic equipment, an external keyboard, a touch pad or a mouse and the like.
In an embodiment, fig. 4 is a schematic diagram of an internal structure of an electronic device according to an embodiment of the present invention, and as shown in fig. 4, there is provided an electronic device, which may be a server, and an internal structure diagram of which may be as shown in fig. 4. The electronic device includes a processor, a network interface, an internal memory, and a non-volatile memory, which stores an operating system, a computer program, and a database, connected by an internal bus. The processor is used for providing calculation and control capabilities, the network interface is used for communicating with an external terminal through network connection, the internal memory is used for providing an environment for an operating system and running of a computer program, the computer program is executed by the processor to realize a calculation capacity improving method of the edge video analysis device, and the database is used for storing data.
It will be appreciated by those skilled in the art that the configuration shown in fig. 4 is a block diagram of only a portion of the configuration associated with the inventive arrangements and does not constitute a limitation on the electronic device to which the inventive arrangements may be applied, and that a particular electronic device may include more or less components than those shown, or combine certain components, or have a different arrangement of components.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, databases, or other media used in embodiments provided herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It should be understood by those skilled in the art that various features of the above-described embodiments can be combined in any combination, and for the sake of brevity, all possible combinations of features in the above-described embodiments are not described in detail, but rather, all combinations of features which are not inconsistent with each other should be construed as being within the scope of the present disclosure.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is specific and detailed, but not to be understood as limiting the scope of the invention. It should be noted that various changes and modifications can be made by those skilled in the art without departing from the spirit of the invention, and these changes and modifications are all within the scope of the invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (9)
1. The computing power improving method of the edge video analysis device is characterized by comprising a main control module and more than two computing power modules, wherein the main control module and each computing power module perform data interaction through a private protocol, and the more than two computing power modules realize cascade connection through an internal network; the main control module is accessed to a peripheral data source and interacts with an upper platform through network communication, and the method comprises the following steps:
acquiring first configuration information of each computing module through the main control module, and sending the first configuration information to a human-computer interaction interface for a user to change to acquire second configuration information;
sending the second configuration information to a corresponding force calculation module through the main control module so that the force calculation module can operate a corresponding algorithm model according to the second configuration information;
acquiring a video stream or a picture stream according to the second configuration information and sending the video stream or the picture stream to the force calculation module so that the force calculation module can analyze the video stream or the picture stream according to an algorithm model operated by the force calculation module;
each computing force module is preset with a plurality of algorithm models, and configuration information comprises video channel information, algorithm model information and algorithm parameters of each computing force module;
the video channel information is the stream taking information of the video stream or the picture stream acquired by the computing power module; the algorithm model information comprises the type and the version of an algorithm model, and the algorithm model comprises three types of face recognition, behavior analysis and target detection; the algorithm parameters include monitoring area, sensitivity threshold, time threshold, task status and target type.
2. The method of claim 1, further comprising:
the second configuration information is sent to the force calculation module for operation verification;
obtaining verification information fed back by each force calculation module, wherein the verification information is a result fed back by the force calculation module after the force calculation module performs compliance verification according to the second configuration information; if the configuration information does not accord with the operation condition of the computing power module, the verification information comprises a verification result and error information;
and prompting a user to modify the configuration information according to the verification information.
3. The method of claim 1, wherein the master control module performs data interaction with the computing power module through a private protocol and performs data interaction with a peripheral data source and an upper platform through a public protocol;
the acquiring a video stream or a picture stream according to the second configuration information and sending the video stream or the picture stream to the computing power module includes:
extracting the video channel information of each computing module and the equipment ID of an external data source thereof from the second configuration information;
acquiring a video stream or a picture stream of the peripheral data source according to the equipment ID;
and transmitting the video stream or the picture stream to a corresponding force calculation module through the video channel information.
4. The method of claim 1, wherein the algorithm module runs a corresponding algorithm model according to the second configuration information, comprising:
extracting an algorithm model to be executed by each computing power module from the second configuration information; and the calculation force module calls a corresponding algorithm according to the algorithm model and starts to operate.
5. The method of claim 1, wherein the algorithmic model in which the computational power module operates analyzes the video or picture stream comprises:
extracting algorithm parameters to be executed by each computing power module from the second configuration information;
the calculation module analyzes and judges each frame image of the received video stream or picture stream according to the algorithm parameters;
and when the images in the video stream or the picture stream accord with the algorithm parameters, feeding back the analysis judgment result to the user.
6. The method of claim 5, further comprising:
obtaining an algorithm upgrading file, and extracting algorithm model information from the algorithm upgrading file to judge the type and upgrading parameters of an algorithm model to be upgraded;
upgrading the corresponding algorithm model according to the upgrading parameters according to the types;
and after the upgrading of each algorithm model is completed, feeding back the type and version information of the algorithm model of each algorithm module to a user.
7. The edge video analysis device is characterized by comprising a main control module and more than two force calculation modules, wherein the main control module and each force calculation module perform data interaction through a private protocol, and the more than two force calculation modules realize cascade connection through an internal network; the main control module is accessed to a peripheral data source and interacts with an upper platform through network communication, wherein,
the main control module is used for acquiring first configuration information of each computing power module and sending the first configuration information to a human-computer interaction interface for a user to change so as to acquire second configuration information; sending the second configuration information to a corresponding computing power module; acquiring a video stream or a picture stream according to the second configuration information and sending the video stream or the picture stream to the force calculation module;
the computing power module is used for operating a corresponding algorithm model according to the second configuration information and analyzing the video stream or the picture stream according to the operating algorithm model;
each computing power module is preset with a plurality of algorithm models, and configuration information comprises video channel information, algorithm model information and algorithm parameters of each computing power module;
the video channel information is the stream taking information of the video stream or the picture stream acquired by the computation module; the algorithm model information comprises the type and the version of an algorithm model, and the algorithm model comprises three types of face recognition, behavior analysis and target detection; the algorithm parameters include monitoring area, sensitivity threshold, time threshold, task status and target type.
8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to perform the computing power improving method of the edge video analysis apparatus according to any one of claims 1 to 6.
9. A storage medium having a computer program stored therein, wherein the computer program is configured to execute the method for enhancing computing power of an edge video analysis apparatus according to any one of claims 1 to 6 when the computer program runs.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210621368.4A CN114708643B (en) | 2022-06-02 | 2022-06-02 | Computing power improving method for edge video analysis device and edge video analysis device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210621368.4A CN114708643B (en) | 2022-06-02 | 2022-06-02 | Computing power improving method for edge video analysis device and edge video analysis device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114708643A CN114708643A (en) | 2022-07-05 |
| CN114708643B true CN114708643B (en) | 2022-09-13 |
Family
ID=82177978
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210621368.4A Active CN114708643B (en) | 2022-06-02 | 2022-06-02 | Computing power improving method for edge video analysis device and edge video analysis device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114708643B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115378826B (en) * | 2022-10-26 | 2023-01-31 | 北京网藤科技有限公司 | Network vulnerability identification method and system for multiple workflows |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110474790A (en) * | 2018-05-11 | 2019-11-19 | 西门子股份公司 | System, cloud platform, the device and method that edge device is configured |
| WO2021009155A1 (en) * | 2019-07-17 | 2021-01-21 | Koninklijke Kpn N.V. | Facilitating video streaming and processing by edge computing |
| CN113590336A (en) * | 2021-08-11 | 2021-11-02 | 上海仁童电子科技有限公司 | Algorithm management method and device of edge computing equipment |
| CN113711243A (en) * | 2019-04-09 | 2021-11-26 | 雾角系统公司 | Intelligent edge computing platform with machine learning capability |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110312057B (en) * | 2018-03-27 | 2021-01-29 | 华为技术有限公司 | Intelligent video processing device |
| CN111479048A (en) * | 2020-04-22 | 2020-07-31 | 安徽大学 | Intelligent video image processing equipment based on edge calculation |
| CN111629120A (en) * | 2020-05-26 | 2020-09-04 | 南京毫末科技有限公司 | Artificial intelligence video analysis platform based on edge calculation |
| CN111901573A (en) * | 2020-08-17 | 2020-11-06 | 泽达易盛（天津）科技股份有限公司 | Fine granularity real-time supervision system based on edge calculation |
| CN113011833A (en) * | 2021-01-15 | 2021-06-22 | 广州穗能通能源科技有限责任公司 | Safety management method and device for construction site, computer equipment and storage medium |
| CN113568724A (en) * | 2021-07-06 | 2021-10-29 | 广州衡昊数据科技有限公司 | Edge computing node control method and system |
| CN113596158A (en) * | 2021-07-29 | 2021-11-02 | 杭州海康威视系统技术有限公司 | Scene-based algorithm configuration method and device |
| CN113691783A (en) * | 2021-10-27 | 2021-11-23 | 中国南方电网有限责任公司超高压输电公司广州局 | Converter station video monitoring method, system, device and computer equipment |
| CN114387549A (en) * | 2022-01-11 | 2022-04-22 | 山东华夏高科信息股份有限公司 | Zebra crossing gift pedestrian visual detection system and method based on deep learning |
-
2022
- 2022-06-02 CN CN202210621368.4A patent/CN114708643B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110474790A (en) * | 2018-05-11 | 2019-11-19 | 西门子股份公司 | System, cloud platform, the device and method that edge device is configured |
| CN113711243A (en) * | 2019-04-09 | 2021-11-26 | 雾角系统公司 | Intelligent edge computing platform with machine learning capability |
| WO2021009155A1 (en) * | 2019-07-17 | 2021-01-21 | Koninklijke Kpn N.V. | Facilitating video streaming and processing by edge computing |
| CN113590336A (en) * | 2021-08-11 | 2021-11-02 | 上海仁童电子科技有限公司 | Algorithm management method and device of edge computing equipment |
Non-Patent Citations (3)
| Title |
|---|
| Deep Learning With Edge Computing: A Review;Jiasi Chen 等;《Proceedings of the IEEE》;20190831;第107卷(第8期);全文 * |
| 面向实时视频流分析的边缘计算技术;杨铮 等;《中国科学: 信息科学》;20220110;第52卷(第1期);全文 * |
| 面向智慧高速网络节点边缘处理器的资源配置优化;陈鸣 等;《无线电通信技术》;20211231;第48卷(第1期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114708643A (en) | 2022-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109388532B (en) | Test method, test device, electronic equipment and computer readable storage medium | |
| CN108021259B (en) | False touch prevention method and electronic equipment | |
| CN109831419A (en) | The determination method and device of shell program authority | |
| CN108037863A (en) | A kind of method and apparatus for showing image | |
| CN108965981B (en) | Video playing method and device, storage medium and electronic equipment | |
| SG191954A1 (en) | An integrated intelligent server based system and method/systems adapted to facilitate fail-safe integration and /or optimized utilization of various sensory inputs | |
| CN112379963B (en) | Remote application window control method and device and computer equipment | |
| CN114708643B (en) | Computing power improving method for edge video analysis device and edge video analysis device | |
| CN107292158A (en) | Mobile terminal and pattern triggering method, computer-readable recording medium | |
| CN109491736B (en) | Display method and device of pop-up frame window | |
| CN114302185A (en) | Display device and information association method | |
| CN109144834B (en) | User behavior data acquisition method and device, android system and terminal equipment | |
| CN112949172B (en) | Data processing method, device, machine-readable medium and equipment | |
| US11507244B2 (en) | Window adjustment method, window adjustment device and mobile terminal | |
| CN111338910A (en) | Log data processing method, log data display method, log data processing device, log data display device, log data processing equipment and log data storage medium | |
| CN111625383B (en) | Process exception event processing method and device, electronic equipment and storage medium | |
| CN107249084A (en) | Mobile terminal and routine call method, computer-readable recording medium | |
| CN108664818B (en) | Unlocking control method and device | |
| CN111338745A (en) | Deployment method and device of virtual machine and intelligent equipment | |
| CN114625510A (en) | Task processing system, method, device and storage medium | |
| CN106303371A (en) | Take pictures monitoring system, method and mobile terminal | |
| WO2021190336A1 (en) | Device control method, apparatus and system | |
| CN117909197A (en) | Operation monitoring method, program updating method, terminal, server and electronic equipment | |
| CN113470013A (en) | Method and device for detecting moved article | |
| CN108960213A (en) | Method for tracking target, device, storage medium and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |