CN111541864A - Digital retina software defined camera method and system - Google Patents
Digital retina software defined camera method and system Download PDFInfo
- Publication number
- CN111541864A CN111541864A CN201910803145.8A CN201910803145A CN111541864A CN 111541864 A CN111541864 A CN 111541864A CN 201910803145 A CN201910803145 A CN 201910803145A CN 111541864 A CN111541864 A CN 111541864A
- Authority
- CN
- China
- Prior art keywords
- video stream
- intelligent
- camera
- calculation
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/71—Indexing; Data structures therefor; Storage structures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/46—Interconnection of networks
- H04L12/4633—Interconnection of networks using encapsulation techniques, e.g. tunneling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/08—Protocols for interworking; Protocol conversion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
- H04N21/6437—Real-time Transport Protocol [RTP]
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Computer Security & Cryptography (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention relates to the field of security monitoring and artificial intelligence, in particular to a digital retina software definition camera method. The method comprises the steps of accessing a monitoring video stream, and inputting the monitoring video stream into an intelligent converter; processing the monitoring video stream according to a software defined camera protocol; monitoring system resources of the intelligent converter; when the system resources of the intelligent converter are available, calling a corresponding calculation model for the processed monitoring video stream to perform intelligent calculation; sending the intelligent calculation result to a cloud platform; the cloud platform stores the received data, receives retrieval and feature calculation of the terminal, and returns results obtained by the retrieval and the feature calculation to the terminal for display. The problem that the traditional video monitoring camera cannot extract and analyze dynamic targets such as people and vehicles in a monitored scene at present and the problem that a video analysis algorithm of the camera cannot be defined and updated when a monitoring video is applied in a large-scale convergence mode are solved.
Description
Technical Field
The invention relates to the field of security monitoring and artificial intelligence, in particular to a digital retina software defined camera method and a digital retina software defined camera system.
Background
The video monitoring system deployed at present adopts the technical standard H.264 more than ten years ago, has low data compression efficiency, high construction cost and poor application effect, and is mainly expressed as follows:
1) early standards compressed inefficient. Under the condition of ensuring the video quality, the estimated memory cost of hundreds of millions of cameras deployed in China needs, and under the condition that the memory space is insufficient, the videos are often over-compressed, so that the quality of a large number of video images is seriously degraded, and key people and vehicles cannot be clearly seen;
2) and the monitoring video is difficult to network. Cameras deployed in many provinces and cities exceed millions, but the cameras adopt old standard codes, only hundreds of videos can be transmitted in real time under the existing communication bandwidth condition, and most monitoring videos cannot be effectively utilized;
3) highly dense cameras cannot cover the full scene. Although the cameras in partial areas are high in distribution density, the full scene coverage still cannot be carried out, the information shot by the ground cameras in the area covered by the cameras is limited, and meanwhile, the redundancy of video data acquired all weather is high, the valuable information of the whole situation is difficult to extract, and huge information waste is caused;
4) and massive videos are difficult to retrieve. The traditional video monitoring system realizes the playback and evidence obtaining of an event by monitoring personnel looking up and reading a historical video, the manual playback and evidence obtaining mode of the video has low efficiency, and although the image retrieval technology is rapidly developed, the traditional video monitoring system is applied in the industrial field, particularly the large-scale application in the security field is still in need of solution;
5) video precision analysis is lacking. In actual combat application, the video monitoring technology has the problems of slow video retrieval and difficult analysis, and has positive significance on how to find important and valuable clues from massive videos, such as quickly identifying and positioning targets and excavating action tracks of the targets, shortening event processing time, reducing working intensity and improving working efficiency.
Disclosure of Invention
The embodiment of the invention provides a method and a system for defining a camera by digital retina software, which solve the problems that the traditional video monitoring camera cannot extract and analyze dynamic targets such as people and vehicles in a monitoring scene at present and the video analysis algorithm of the camera cannot be defined and updated when the monitoring video is applied in a large-scale convergence manner.
According to a first aspect of embodiments of the present invention, a digital retina software defined camera method comprises:
the intelligent converter receives the video stream of the camera and decodes the video stream;
the cloud platform monitors system resources of the intelligent converter and judges whether the system resources are available, when the system resources of the intelligent converter are available, the intelligent converter calls a calculation model of the camera for calculation on the processed video stream, and when the system resources of the intelligent converter are unavailable, an error is reported;
the intelligent converter packages a calculation result and a video stream and then sends the calculation result and the video stream to the cloud platform;
the cloud platform receives the encapsulated computation results and video stream and stores the encapsulated computation results and video stream,
and the cloud platform receives retrieval and feature calculation instructions sent by the terminal, retrieves and calculates the calculation results and the video stream, and returns the results obtained by the retrieval and feature calculation to the terminal for display.
When the cloud platform receives a request sent by a terminal and needing to change the calculation model of the camera, the parameter information of the camera, the running state information of the calculation model of the camera and the running support information of the model to be run are compared,
judging whether the parameter information of the camera and the running state information of the calculation model of the camera meet the running support information change condition of the model to be run,
if so, then
The cloud platform sends a model to be operated to the intelligent converter; the intelligent converter receives and finishes decapsulation and then stops the running calculation model of the camera, and the camera starts the model to be run to finish model updating;
if not, then
And returning the change failure to the terminal.
The cloud platform receives the encapsulated calculation result and video stream and stores the encapsulated calculation result and video stream, and the method specifically comprises the following steps:
and the cloud platform decapsulates the received data to obtain a feature stream and a video stream, stores the video stream into a video file database, and stores the feature stream into a corresponding structured and/or unstructured database.
The cloud monitors system resources of the intelligent converter, and judges whether the system resources are available, and the method specifically comprises the following steps:
the intelligent converter reads the occupation rate of a CPU, a memory and a network I/O, GPU of the intelligent converter resource manager and sends the occupation rate to the cloud, and the cloud judges whether the system resources are available.
The intelligent converter decodes the video stream, including
And the intelligent converter detects the format of the video stream, decodes the accessed video stream and extracts the ID information of the monitoring camera and the video coding format parameters.
A digital retina software definition camera system comprises an intelligent converter, a cloud platform and a terminal, wherein the intelligent converter comprises an intelligent analysis module, the intelligent analysis module receives and processes a video stream, reads system resources of the intelligent converter, sends a reading result to a cloud, and calls a corresponding calculation model to perform intelligent calculation on the processed video stream when the resources are available;
the cloud platform stores the received intelligent calculation result and the video stream, receives retrieval and feature calculation of the terminal, and returns the results obtained by the retrieval and the feature calculation to the terminal for display;
and the terminal realizes user interaction and sends an instruction to the cloud platform.
The cloud platform comprises an access scheduling middleware, a data storage module, a business application service module and a calculation algorithm pool module, wherein,
the access scheduling middleware receives the intelligent calculation result and the video stream of the intelligent analysis module and sends the intelligent calculation result and the video stream to the data storage module; the business application service module sends an instruction to the access scheduling middleware, receives data transmitted by the access scheduling middleware and transmits the data to the terminal; and the calculation algorithm pool module stores calculation algorithms, accesses the scheduling middleware to call the model to be operated from the calculation model algorithm pool, and sends the model to be operated to the intelligent converter.
The intelligent analysis module comprises a video decoding module, a system resource module and an intelligent calculation module, wherein
The video decoding module processes the monitoring video stream, specifically comprises the steps of decoding the accessed video stream and extracting ID information and video coding format parameters of the monitoring camera;
the system resource module reads system resources of the intelligent converter, and specifically comprises reading occupancy rates of a CPU, a memory and a network I/O, GPU of a resource manager of the intelligent converter;
and the intelligent computing module calls a corresponding computing model for the processed monitoring video stream to perform intelligent computing, receives the to-be-operated model transmitted by the access scheduling middleware and completes updating of the intelligent computing model.
The data storage module comprises a video file database and structured and unstructured databases, wherein the data storage module decapsulates the received intelligent calculation result and the video stream to obtain a feature stream and a video stream, stores the video stream to the video file database, and stores the feature stream to the structured and unstructured databases.
The system further comprises a third-party service engine module, wherein the third-party service engine module is connected with the access scheduling middleware, the access scheduling middleware receives data services provided by the third-party service engine, and the application services are realized based on the data.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
by adding the intelligent converter on the traditional video monitoring camera, the video intelligent analysis module of the intelligent converter finishes upgrading the traditional video monitoring camera into the digital retina intelligent camera, realizes extraction and analysis of dynamic targets such as people and vehicles in a monitoring scene, enables the single-function camera to have more flexible video processing and analysis functions, and supports better application reliability along with model updating.
A computational model algorithm pool is established for a video analysis platform, so that a video analysis algorithm of a camera can be effectively defined and updated, the single-function camera has more flexible video processing and analysis functions, and better application reliability is supported along with model updating.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
FIG. 1 is a flow chart of a digital retinal software defined camera method of the present invention;
fig. 2 is a block diagram of a digital retinal software defined camera system of the present invention.
Detailed Description
Example one
In the process of rapid development, the video monitoring industry is continuously developing towards networking, high-definition, intellectualization and diversification. With the deep application of artificial intelligence, cloud computing, big data and unmanned aerial vehicle technology, the diversification of video intelligent analysis becomes the most distinctive feature of a new generation of video monitoring system. The digital retina end-to-end system is an intelligent video monitoring platform (short for "platform") which is developed by Beijing university digital video coding and decoding national engineering laboratories and simultaneously supports functions of concentration transcoding, image analysis, feature retrieval, application display and the like of monitoring videos, the platform integrates a plurality of front edge technologies such as visual content analysis, visual feature retrieval, big data analysis, cloud storage, deep learning and the like, develops a plurality of technologies such as parallel multi-channel video concentration transcoding, human and vehicle visual feature extraction, massive visual big data fast retrieval, double-current remote communication, a software defined camera network, a real-time service application middleware, an online camera and an offline video file, is suitable for efficient storage, fast retrieval and intelligent application of city-level large-scale monitoring videos, and can provide an overall large-scale monitoring video intelligent application solution for users, the method can be widely applied to intelligent video processing of various bayonet, micro-bayonet and security monitoring video scenes. An effective technical means is provided for the comprehensive management of the city.
As shown in fig. 1, the present invention provides a digital retina software defined camera method, comprising:
accessing a monitoring video stream, and inputting the monitoring video stream into an intelligent converter;
processing the monitoring video stream, specifically detecting the format of an input video, extracting camera parameters, namely decoding the accessed rtsp monitoring video stream, and extracting monitoring camera ID information (such as IP address) and video coding format (resolution, frame rate and code rate) parameters; rtsp is real time Streaming Protocol; the ID is an Identity document and is an Identity identification number. The software-defined camera protocol is a communication protocol of the front-end intelligent converter and the cloud, and information exchange and resource sharing are realized through cooperative work.
Reading the occupancy rates of a CPU, a memory and a network I/O, GPU of the resource manager of the intelligent converter;
when the resources of the intelligent converter are available, calling corresponding calculation models (such as detection tracking and target structuring) for the decoded video sequence to perform intelligent calculation, wherein the intelligent calculation can comprise detection, tracking, target selection, target structuring and feature calculation; if the resources of the intelligent converter are unavailable, the intelligent converter reports errors to the terminal by accessing a scheduling middleware and a business application service;
synchronizing the intelligent calculation result with the video stream, packaging according to a software defined camera interaction protocol, and sending to a cloud platform; receiving an intelligent calculation result and a video stream by an access scheduling middleware of the cloud platform; the access scheduling middleware decapsulates the received data to obtain a feature stream and a video stream, stores the video stream into a video file database, and stores analysis results of the decapsulated feature stream into corresponding structured and unstructured databases; in addition, the cloud platform receives a service request of the business application, retrieves the corresponding database, completes operations such as statistics, analysis and calculation of data and returns an obtained result to the terminal for display;
preferably, the third-party service engine is connected to the access scheduling middleware, and the third-party service engine may include a geographic information database and a blacklist information database; the third-party service engine is connected with the access scheduling middleware, and the access scheduling middleware receives the data service provided by the third-party service engine and makes application service based on the data.
The access scheduling middleware is responsible for protocol analysis, data storage scheduling and data retrieval scheduling;
the business application service is responsible for sending instructions to the access scheduling middleware and receiving data transmitted by the access scheduling middleware.
Preferably, the cloud platform receives a camera calculation model change request of the service application, retrieves the corresponding database to obtain information such as camera parameters and calculation model running states, and retrieves running support information of a model to be run in the calculation model algorithm pool; comparing the camera parameter information, the calculation model running state information and the running support information of the model to be run, wherein the camera parameter information can comprise resolution, frame rate and brightness, and the calculation model running state information comprises CPU (central processing unit), GPU (graphic processing unit), internal memory and video memory information; if the parameter information of the camera and the running state information of the calculation model accord with the changing condition of the running support information of the model to be run, continuing to change, otherwise, returning to fail to change; calling a model to be operated from a calculation model algorithm pool, packaging the calculation model according to a software defined camera interaction protocol, and transmitting the calculation model to an intelligent converter; and the intelligent converter receives the model to be operated, stops the calculation model of the specified camera in operation according to the camera parameters after the model to be operated is received and unpacked, starts the model to be operated by the specified camera, and finishes model updating.
Example two
A digital retina software defined camera method specifically comprises the following steps:
step 1, adding a video intelligent analysis module (including but not limited to Nvidia Jetson TX2) at a video stream output port of a video monitoring camera, wherein the Nvidia Jetson TX2 is an AI super computer in the embedded field of Invida. .
Step 2, processing the monitoring video stream according to the software defined camera protocol, comprising: detecting the format of an input video, wherein the step of extracting the parameters of the camera refers to decapsulating an accessed rtsp monitoring video stream, and extracting the ID information (such as an IP address) of the monitoring camera, the parameters of a video coding format (such as resolution, frame rate and code rate) and the like;
opening a path of monitoring video, extracting camera parameters and decoding
Protocol: zmq-req/rep
The functions are as follows: prompting to open a path of video, extracting and decoding camera parameters, and returning a handle if the path of video is successful.
Description of the drawings:
Request：
the initiator: a scheduling program;
cmd is the command word. Each command has its own independent command word for distinguishing different commands;
camera _ param: the address of the req/rep server for asynchronously receiving camera parameters and video format (json format);
pic _ receiver: zmq-req/rep server, for asynchronously receiving a target image (binary) of a video frame.
Response：
Errcode, errmsg: if the error occurs, the error code is not 0, and the errmsg is set as specific error information;
handle: if the open is successful, handle represents the unique ID of the open channel. Can be followed up
To use this ID for channel closing and the like.
Video stream data delivery
Protocol: zmq-req/rep
The functions are as follows: decoding the video and inputting the video into an intelligent calculation model;
if the transfer fails, then:
description of the drawings:
"error" of-1 means that the shared memory device was created with an error;
an "error" of-2 means that there is no corresponding decoder or no decoder running.
And step 3: reading the occupation rates of a CPU, a memory and a network I/O, GPU of a system resource manager of the intelligent converter;
protocol: zmq-req/rep;
the functions are as follows: counting the occupancy rate of system resources:
and 4, step 4: calling a corresponding calculation model (such as detection tracking and target structuring) for the decoded video sequence to perform intelligent calculation under the condition that system resources are available;
protocol: zmq-req/rep;
the functions are as follows: detection and tracking of visual targets:
protocol: zmq-req/rep;
the functions are as follows: object structuring
And 5: synchronizing the intelligent calculation result with the video stream, packaging according to a software defined camera interaction protocol, and sending to a cloud platform;
video stream data delivery
Protocol: zmq-req/rep
Request：
The functions are as follows: for synchronization with the feature stream and delivery to the cloud;
the format is as follows: adding a data structure behind a binary data system head 'b';
cmd: command word 1101(int32 little endian);
dev _ id: camera ID (64 byte string);
handle: task ID (session _ ID) (36-byte character string) of the camera;
pts (pts) is as follows: start pts of transcoded video stream, (8 bytes int64, little endian);
filename _ pts: the file name pts is used for segmenting the current system time (int64 little endian) of the first frame of the file, the file name is named according to the storage strategy of the video stream and the absolute time of the initial frame, and the same file can be sent in several times;
length code stream length (4 bytes int 32);
Response：
the format is as follows: adding a data structure behind a binary data system head 'b';
cmd: command word 1100(int32 little endian);
dev _ id: camera ID (64 byte string);
handle: task ID (session _ ID) (36-byte character string) of the camera;
return _ value: return value (int32 little endian), 0 is correct, not 0 error.
Video analytics results delivery
Protocol: zmq-req/rep
The functions are as follows: transmitting video analysis results to cloud
Step 6: the cloud platform decapsulates the received data to obtain a feature stream and a video stream, stores the video stream into a video file database, and stores analysis results obtained after feature stream decapsulation into corresponding structured and unstructured databases; in addition, the cloud platform receives a service request of the business application, retrieves the corresponding database, completes operations such as data statistics, analysis and calculation, and returns an obtained result to the terminal for display;
protocol: http post
The functions are as follows: and searching the pedestrian database according to the search condition, and if the search condition is successful, returning the pedestrian structured information.
If correctly returning:
otherwise, the error returns:
and 7: the cloud platform receives a camera calculation model change request of business application, retrieves a corresponding database to obtain information such as camera parameters and calculation model operation states, and retrieves operation support information of a model to be operated in a calculation model algorithm pool; comparing the camera parameter information, the calculation model running state information and the running support information of the model to be run, if the camera parameter information and the calculation model running state information meet the changing conditions, continuing to change, and if not, returning to the state that the camera parameter information and the calculation model running state information fail to change; calling a calculation model to be changed from a calculation model algorithm pool, packaging the calculation model according to a software defined camera interaction protocol, and transmitting the calculation model to an intelligent converter; the intelligent converter receives the model to be operated, after decapsulation is completed, the calculation model in which the specified camera is operating is stopped according to the camera parameters, the specified camera starts the model to be operated, and model updating is completed;
protocol: zmq-req/rep
The functions are as follows: transfer of computational model to be altered to converter
A digital retinal software defined camera system comprising: an intelligent converter, a cloud platform and a terminal,
the intelligent converter comprises an intelligent analysis module, the intelligent analysis module receives the monitoring video stream, processes the monitoring video stream according to a software definition camera protocol, monitors system resources of the intelligent converter, and calls a corresponding calculation model for the processed monitoring video stream to perform intelligent calculation when the resources are available;
the cloud platform stores the received intelligent calculation and video stream, receives retrieval and calculation services of the terminal, and returns the results obtained by the retrieval and calculation to the terminal for display;
and the terminal realizes user interaction and sends an instruction to the cloud platform.
Preferably, the cloud platform comprises an access scheduling middleware, a data storage module, a business application service module and a calculation algorithm pool module, wherein the access scheduling middleware is responsible for protocol analysis, data storage scheduling and data retrieval scheduling, receives an intelligent calculation result and a video stream of the intelligent analysis module and transmits the intelligent calculation result and the video stream to the data storage module; the business application service module sends an instruction to the access scheduling middleware, receives data transmitted by the access scheduling middleware and transmits the data to the terminal; and the calculation algorithm pool module stores calculation algorithms, accesses the scheduling middleware to call the model to be operated from the calculation model algorithm pool, packages the calculation model according to the software defined camera interaction protocol and transmits the calculation model to the intelligent converter.
Preferably, the intelligent analysis module comprises a video decoding module, a system resource module and an intelligent calculation module, wherein the video decoding module processes the monitoring video stream according to a software defined camera protocol, and specifically comprises the steps of decoding the accessed video stream and extracting the ID information and the video coding format parameters of the monitoring camera;
the system resource module monitors system resources of the intelligent converter, and specifically comprises reading the occupancy rates of a CPU, a memory and a network I/O, GPU of a resource manager of the intelligent converter;
and the intelligent computing module calls a corresponding computing model for the processed monitoring video stream to perform intelligent computing, receives the to-be-operated model transmitted by the access scheduling middleware and completes updating of the intelligent computing model.
Preferably, the data storage module comprises a video file database, a structured database and an unstructured database, the received intelligent calculation result and the video stream are unpacked to obtain a feature stream and a video stream, the video stream is stored in the video file database, and the feature stream is stored in the structured database and the unstructured database.
Preferably, the system further comprises a third-party service engine module, the third-party service engine module is connected with the access scheduling middleware, and the access scheduling middleware receives the data service provided by the third-party service engine and makes an application service based on the data.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the features described above or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (10)
1. A digital retinal software defined camera method, comprising:
the intelligent converter receives the video stream of the camera and decodes the video stream;
the cloud platform monitors system resources of the intelligent converter and judges whether the system resources are available, when the system resources of the intelligent converter are available, the intelligent converter calls a calculation model of the camera to calculate the processed video stream, and when the system resources of the intelligent converter are unavailable, an error is reported;
the intelligent converter packages a calculation result and a video stream and then sends the calculation result and the video stream to the cloud platform;
the cloud platform receives the encapsulated computation results and video stream and stores the encapsulated computation results and video stream,
and the cloud platform receives retrieval and feature calculation instructions sent by the terminal, retrieves and calculates the calculation results and the video stream, and returns the results obtained by the retrieval and feature calculation to the terminal for display.
2. The digital retinal software defined camera method of claim 1,
when the cloud platform receives a request sent by a terminal and needing to change the calculation model of the camera, the parameter information of the camera, the running state information of the calculation model of the camera and the running support information of the model to be run are compared,
judging whether the parameter information of the camera and the running state information of the calculation model of the camera meet the running support information change condition of the model to be run,
if so, then
The cloud platform sends a model to be operated to the intelligent converter; the intelligent converter receives and finishes decapsulation and then stops the running calculation model of the camera, and the camera starts the model to be run to finish model updating;
if not, then
And returning the change failure to the terminal.
3. The method as claimed in claim 2, wherein the cloud platform receives the encapsulated computation results and video stream and stores the encapsulated computation results and video stream, specifically:
and the cloud platform decapsulates the received data to obtain a feature stream and a video stream, stores the video stream into a video file database, and stores the feature stream into a corresponding structured and/or unstructured database.
4. The method as claimed in claim 3, wherein the cloud end monitors system resources of the intelligent converter and determines whether the system resources are available, specifically comprising:
the intelligent converter reads the occupancy rates of the CPU, the memory and the network I/O, GPU of the intelligent converter resource manager and sends the occupancy rates to the cloud, and the cloud judges whether the system resources are available.
5. The digital retinal software defined camera method of claim 4 wherein the intelligent converter decodes the video stream comprising
And the intelligent converter detects the format of the video stream, decodes the accessed video stream and extracts the ID information of the monitoring camera and the video coding format parameters.
6. A digital retina software-defined camera system is characterized by comprising an intelligent converter, a cloud platform and a terminal,
the intelligent converter comprises an intelligent analysis module, the intelligent analysis module receives the video stream, processes the video stream, reads system resources of the intelligent converter, sends a reading result to the cloud, and calls a corresponding calculation model for the processed video stream to perform intelligent calculation when the resources are available;
the cloud platform stores the received intelligent calculation result and the video stream, receives retrieval and feature calculation of the terminal, and returns the results obtained by the retrieval and the feature calculation to the terminal for display;
and the terminal realizes user interaction and sends an instruction to the cloud platform.
7. The digital retinal software defined camera system of claim 6, wherein the cloud platform comprises access scheduling middleware, a data storage module, a business application service module, a computational algorithm pool module, wherein,
the access scheduling middleware receives the intelligent calculation result and the video stream of the intelligent analysis module and sends the intelligent calculation result and the video stream to the data storage module; the business application service module sends an instruction to the access scheduling middleware, receives data transmitted by the access scheduling middleware and transmits the data to the terminal; and the calculation algorithm pool module stores calculation algorithms, accesses the scheduling middleware to call the model to be operated from the calculation model algorithm pool, and sends the model to be operated to the intelligent converter.
8. The digital retinal software defined camera system of claim 7, wherein the intelligent analysis module comprises a video decoding module, a system resource module, an intelligent computing module, wherein
The video decoding module processes the monitoring video stream, specifically comprises the steps of decoding the accessed video stream and extracting ID information and video coding format parameters of the monitoring camera;
the system resource module reads system resources of the intelligent converter, and specifically comprises reading the occupancy rates of a CPU, a memory and a network I/O, GPU of a resource manager of the intelligent converter;
and the intelligent computing module calls a corresponding computing model for the processed monitoring video stream to perform intelligent computing, receives the to-be-operated model transmitted by the access scheduling middleware and completes updating of the intelligent computing model.
9. The digital retinal software defined camera system of claim 8, wherein the data storage module comprises a video file database, a structured and unstructured database, wherein,
and the data storage module decapsulates the received intelligent calculation result and the video stream to obtain a feature stream and a video stream, stores the video stream to a video file database, and stores the feature stream to a structured database and an unstructured database.
10. The digital retina software defined camera system of claim 9, further comprising a third party service engine module, wherein the third party service engine module is connected to the access scheduling middleware, and the access scheduling middleware receives the data service provided by the third party service engine and implements the application service based on the data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910803145.8A CN111541864B (en) | 2019-08-28 | 2019-08-28 | Digital retina software defined camera method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910803145.8A CN111541864B (en) | 2019-08-28 | 2019-08-28 | Digital retina software defined camera method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111541864A true CN111541864A (en) | 2020-08-14 |
| CN111541864B CN111541864B (en) | 2021-03-23 |
Family
ID=71976629
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910803145.8A Active CN111541864B (en) | 2019-08-28 | 2019-08-28 | Digital retina software defined camera method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111541864B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115984675A (en) * | 2022-12-01 | 2023-04-18 | 扬州万方科技股份有限公司 | System and method for realizing multi-channel video decoding and AI intelligent analysis |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160320837A1 (en) * | 2015-05-01 | 2016-11-03 | Massachusetts Institute Of Technology | Methods and Apparatus for Retinal Retroreflection Imaging |
| CN106790367A (en) * | 2016-11-15 | 2017-05-31 | 山东省科学院自动化研究所 | The vehicle safety hidden danger early warning of big data treatment and accident reproduction system and method |
| CN108984799A (en) * | 2018-08-21 | 2018-12-11 | 北京深瞐科技有限公司 | A kind of video data handling procedure and device |
-
2019
- 2019-08-28 CN CN201910803145.8A patent/CN111541864B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160320837A1 (en) * | 2015-05-01 | 2016-11-03 | Massachusetts Institute Of Technology | Methods and Apparatus for Retinal Retroreflection Imaging |
| CN106790367A (en) * | 2016-11-15 | 2017-05-31 | 山东省科学院自动化研究所 | The vehicle safety hidden danger early warning of big data treatment and accident reproduction system and method |
| CN108984799A (en) * | 2018-08-21 | 2018-12-11 | 北京深瞐科技有限公司 | A kind of video data handling procedure and device |
Non-Patent Citations (1)
| Title |
|---|
| 高文、田永鸿、王坚: "数字视网膜:智慧城市系统演进的关键环节", 《中国科学：信息科学》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115984675A (en) * | 2022-12-01 | 2023-04-18 | 扬州万方科技股份有限公司 | System and method for realizing multi-channel video decoding and AI intelligent analysis |
| CN115984675B (en) * | 2022-12-01 | 2023-10-13 | 扬州万方科技股份有限公司 | System and method for realizing multipath video decoding and AI intelligent analysis |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111541864B (en) | 2021-03-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108255605B (en) | Image recognition cooperative computing method and system based on neural network | |
| CN104137146A (en) | Method and system for video coding with noise filtering of foreground object segmentation | |
| CN103475882B (en) | Surveillance video encoding and recognizing method and surveillance video encoding and recognizing system | |
| CN106254458B (en) | A kind of image processing method based on cloud robot vision, platform and system | |
| CN108040221A (en) | A kind of intelligent video analysis and monitoring system | |
| CN111090773B (en) | Digital retina system structure and software architecture method and system | |
| CN110662017B (en) | Video playing quality detection method and device | |
| CN105407316A (en) | Implementation method for intelligent camera system, intelligent camera system, and network camera | |
| WO2022105090A1 (en) | Fine-grained data stream reliable offloading method based on callback mechanism | |
| WO2023179161A1 (en) | Video frame rate control method and apparatus, and electronic device and storage medium | |
| CN111541864B (en) | Digital retina software defined camera method and system | |
| CN111259839A (en) | Target object behavior monitoring method, device, equipment, system and storage medium | |
| CN112312070A (en) | Digital retina cloud software scheduling method | |
| US10506021B2 (en) | Method and device for providing communication connection for a plurality of candidate applications in a mobile device | |
| CN111209566A (en) | Intelligent anti-crawler system and method for multi-layer threat interception | |
| CN112087632A (en) | Video processing system, method, storage medium and computer device | |
| CN114640854A (en) | Real-time high-speed decoding method for multi-channel video stream | |
| CN107194013B (en) | Processing method and processing system based on Distributed Message Queue middleware | |
| CN107295340A (en) | A kind of method and device of remote desktop Video coding | |
| CN110727722A (en) | Massive concurrent radar data storage method | |
| CN112565800B (en) | Video positioning method, device, equipment, system and storage medium | |
| CN112202786B (en) | Illegal data identification method and device and electronic equipment | |
| CN113596396B (en) | GB 28181-based target tracking system | |
| CN112188244B (en) | Front-end camera real-time video-on-demand method and device and electronic equipment | |
| CN117176962B (en) | Video encoding and decoding method and device and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |