CN111541864A - Digital retina software defined camera method and system - Google Patents

Digital retina software defined camera method and system Download PDF

Info

Publication number
CN111541864A
CN111541864A CN201910803145.8A CN201910803145A CN111541864A CN 111541864 A CN111541864 A CN 111541864A CN 201910803145 A CN201910803145 A CN 201910803145A CN 111541864 A CN111541864 A CN 111541864A
Authority
CN
China
Prior art keywords
video stream
intelligent
camera
calculation
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910803145.8A
Other languages
Chinese (zh)
Other versions
CN111541864B (en
Inventor
贾惠柱
李源
杨长水
崔玥
解晓东
高文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University
Original Assignee
Peking University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University filed Critical Peking University
Priority to CN201910803145.8A priority Critical patent/CN111541864B/en
Publication of CN111541864A publication Critical patent/CN111541864A/en
Application granted granted Critical
Publication of CN111541864B publication Critical patent/CN111541864B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/71Indexing; Data structures therefor; Storage structures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/46Interconnection of networks
    • H04L12/4633Interconnection of networks using encapsulation techniques, e.g. tunneling
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/06Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/08Protocols for interworking; Protocol conversion
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643Communication protocols
    • H04N21/6437Real-time Transport Protocol [RTP]

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Security & Cryptography (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

The invention relates to the field of security monitoring and artificial intelligence, in particular to a digital retina software definition camera method. The method comprises the steps of accessing a monitoring video stream, and inputting the monitoring video stream into an intelligent converter; processing the monitoring video stream according to a software defined camera protocol; monitoring system resources of the intelligent converter; when the system resources of the intelligent converter are available, calling a corresponding calculation model for the processed monitoring video stream to perform intelligent calculation; sending the intelligent calculation result to a cloud platform; the cloud platform stores the received data, receives retrieval and feature calculation of the terminal, and returns results obtained by the retrieval and the feature calculation to the terminal for display. The problem that the traditional video monitoring camera cannot extract and analyze dynamic targets such as people and vehicles in a monitored scene at present and the problem that a video analysis algorithm of the camera cannot be defined and updated when a monitoring video is applied in a large-scale convergence mode are solved.

Description

Digital retina software defined camera method and system
Technical Field
The invention relates to the field of security monitoring and artificial intelligence, in particular to a digital retina software defined camera method and a digital retina software defined camera system.
Background
The video monitoring system deployed at present adopts the technical standard H.264 more than ten years ago, has low data compression efficiency, high construction cost and poor application effect, and is mainly expressed as follows:
1) early standards compressed inefficient. Under the condition of ensuring the video quality, the estimated memory cost of hundreds of millions of cameras deployed in China needs, and under the condition that the memory space is insufficient, the videos are often over-compressed, so that the quality of a large number of video images is seriously degraded, and key people and vehicles cannot be clearly seen;
2) and the monitoring video is difficult to network. Cameras deployed in many provinces and cities exceed millions, but the cameras adopt old standard codes, only hundreds of videos can be transmitted in real time under the existing communication bandwidth condition, and most monitoring videos cannot be effectively utilized;
3) highly dense cameras cannot cover the full scene. Although the cameras in partial areas are high in distribution density, the full scene coverage still cannot be carried out, the information shot by the ground cameras in the area covered by the cameras is limited, and meanwhile, the redundancy of video data acquired all weather is high, the valuable information of the whole situation is difficult to extract, and huge information waste is caused;
4) and massive videos are difficult to retrieve. The traditional video monitoring system realizes the playback and evidence obtaining of an event by monitoring personnel looking up and reading a historical video, the manual playback and evidence obtaining mode of the video has low efficiency, and although the image retrieval technology is rapidly developed, the traditional video monitoring system is applied in the industrial field, particularly the large-scale application in the security field is still in need of solution;
5) video precision analysis is lacking. In actual combat application, the video monitoring technology has the problems of slow video retrieval and difficult analysis, and has positive significance on how to find important and valuable clues from massive videos, such as quickly identifying and positioning targets and excavating action tracks of the targets, shortening event processing time, reducing working intensity and improving working efficiency.
Disclosure of Invention
The embodiment of the invention provides a method and a system for defining a camera by digital retina software, which solve the problems that the traditional video monitoring camera cannot extract and analyze dynamic targets such as people and vehicles in a monitoring scene at present and the video analysis algorithm of the camera cannot be defined and updated when the monitoring video is applied in a large-scale convergence manner.
According to a first aspect of embodiments of the present invention, a digital retina software defined camera method comprises:
the intelligent converter receives the video stream of the camera and decodes the video stream;
the cloud platform monitors system resources of the intelligent converter and judges whether the system resources are available, when the system resources of the intelligent converter are available, the intelligent converter calls a calculation model of the camera for calculation on the processed video stream, and when the system resources of the intelligent converter are unavailable, an error is reported;
the intelligent converter packages a calculation result and a video stream and then sends the calculation result and the video stream to the cloud platform;
the cloud platform receives the encapsulated computation results and video stream and stores the encapsulated computation results and video stream,
and the cloud platform receives retrieval and feature calculation instructions sent by the terminal, retrieves and calculates the calculation results and the video stream, and returns the results obtained by the retrieval and feature calculation to the terminal for display.
When the cloud platform receives a request sent by a terminal and needing to change the calculation model of the camera, the parameter information of the camera, the running state information of the calculation model of the camera and the running support information of the model to be run are compared,
judging whether the parameter information of the camera and the running state information of the calculation model of the camera meet the running support information change condition of the model to be run,
if so, then
The cloud platform sends a model to be operated to the intelligent converter; the intelligent converter receives and finishes decapsulation and then stops the running calculation model of the camera, and the camera starts the model to be run to finish model updating;
if not, then
And returning the change failure to the terminal.
The cloud platform receives the encapsulated calculation result and video stream and stores the encapsulated calculation result and video stream, and the method specifically comprises the following steps:
and the cloud platform decapsulates the received data to obtain a feature stream and a video stream, stores the video stream into a video file database, and stores the feature stream into a corresponding structured and/or unstructured database.
The cloud monitors system resources of the intelligent converter, and judges whether the system resources are available, and the method specifically comprises the following steps:
the intelligent converter reads the occupation rate of a CPU, a memory and a network I/O, GPU of the intelligent converter resource manager and sends the occupation rate to the cloud, and the cloud judges whether the system resources are available.
The intelligent converter decodes the video stream, including
And the intelligent converter detects the format of the video stream, decodes the accessed video stream and extracts the ID information of the monitoring camera and the video coding format parameters.
A digital retina software definition camera system comprises an intelligent converter, a cloud platform and a terminal, wherein the intelligent converter comprises an intelligent analysis module, the intelligent analysis module receives and processes a video stream, reads system resources of the intelligent converter, sends a reading result to a cloud, and calls a corresponding calculation model to perform intelligent calculation on the processed video stream when the resources are available;
the cloud platform stores the received intelligent calculation result and the video stream, receives retrieval and feature calculation of the terminal, and returns the results obtained by the retrieval and the feature calculation to the terminal for display;
and the terminal realizes user interaction and sends an instruction to the cloud platform.
The cloud platform comprises an access scheduling middleware, a data storage module, a business application service module and a calculation algorithm pool module, wherein,
the access scheduling middleware receives the intelligent calculation result and the video stream of the intelligent analysis module and sends the intelligent calculation result and the video stream to the data storage module; the business application service module sends an instruction to the access scheduling middleware, receives data transmitted by the access scheduling middleware and transmits the data to the terminal; and the calculation algorithm pool module stores calculation algorithms, accesses the scheduling middleware to call the model to be operated from the calculation model algorithm pool, and sends the model to be operated to the intelligent converter.
The intelligent analysis module comprises a video decoding module, a system resource module and an intelligent calculation module, wherein
The video decoding module processes the monitoring video stream, specifically comprises the steps of decoding the accessed video stream and extracting ID information and video coding format parameters of the monitoring camera;
the system resource module reads system resources of the intelligent converter, and specifically comprises reading occupancy rates of a CPU, a memory and a network I/O, GPU of a resource manager of the intelligent converter;
and the intelligent computing module calls a corresponding computing model for the processed monitoring video stream to perform intelligent computing, receives the to-be-operated model transmitted by the access scheduling middleware and completes updating of the intelligent computing model.
The data storage module comprises a video file database and structured and unstructured databases, wherein the data storage module decapsulates the received intelligent calculation result and the video stream to obtain a feature stream and a video stream, stores the video stream to the video file database, and stores the feature stream to the structured and unstructured databases.
The system further comprises a third-party service engine module, wherein the third-party service engine module is connected with the access scheduling middleware, the access scheduling middleware receives data services provided by the third-party service engine, and the application services are realized based on the data.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
by adding the intelligent converter on the traditional video monitoring camera, the video intelligent analysis module of the intelligent converter finishes upgrading the traditional video monitoring camera into the digital retina intelligent camera, realizes extraction and analysis of dynamic targets such as people and vehicles in a monitoring scene, enables the single-function camera to have more flexible video processing and analysis functions, and supports better application reliability along with model updating.
A computational model algorithm pool is established for a video analysis platform, so that a video analysis algorithm of a camera can be effectively defined and updated, the single-function camera has more flexible video processing and analysis functions, and better application reliability is supported along with model updating.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
FIG. 1 is a flow chart of a digital retinal software defined camera method of the present invention;
fig. 2 is a block diagram of a digital retinal software defined camera system of the present invention.
Detailed Description
Example one
In the process of rapid development, the video monitoring industry is continuously developing towards networking, high-definition, intellectualization and diversification. With the deep application of artificial intelligence, cloud computing, big data and unmanned aerial vehicle technology, the diversification of video intelligent analysis becomes the most distinctive feature of a new generation of video monitoring system. The digital retina end-to-end system is an intelligent video monitoring platform (short for "platform") which is developed by Beijing university digital video coding and decoding national engineering laboratories and simultaneously supports functions of concentration transcoding, image analysis, feature retrieval, application display and the like of monitoring videos, the platform integrates a plurality of front edge technologies such as visual content analysis, visual feature retrieval, big data analysis, cloud storage, deep learning and the like, develops a plurality of technologies such as parallel multi-channel video concentration transcoding, human and vehicle visual feature extraction, massive visual big data fast retrieval, double-current remote communication, a software defined camera network, a real-time service application middleware, an online camera and an offline video file, is suitable for efficient storage, fast retrieval and intelligent application of city-level large-scale monitoring videos, and can provide an overall large-scale monitoring video intelligent application solution for users, the method can be widely applied to intelligent video processing of various bayonet, micro-bayonet and security monitoring video scenes. An effective technical means is provided for the comprehensive management of the city.
As shown in fig. 1, the present invention provides a digital retina software defined camera method, comprising:
accessing a monitoring video stream, and inputting the monitoring video stream into an intelligent converter;
processing the monitoring video stream, specifically detecting the format of an input video, extracting camera parameters, namely decoding the accessed rtsp monitoring video stream, and extracting monitoring camera ID information (such as IP address) and video coding format (resolution, frame rate and code rate) parameters; rtsp is real time Streaming Protocol; the ID is an Identity document and is an Identity identification number. The software-defined camera protocol is a communication protocol of the front-end intelligent converter and the cloud, and information exchange and resource sharing are realized through cooperative work.
Reading the occupancy rates of a CPU, a memory and a network I/O, GPU of the resource manager of the intelligent converter;
when the resources of the intelligent converter are available, calling corresponding calculation models (such as detection tracking and target structuring) for the decoded video sequence to perform intelligent calculation, wherein the intelligent calculation can comprise detection, tracking, target selection, target structuring and feature calculation; if the resources of the intelligent converter are unavailable, the intelligent converter reports errors to the terminal by accessing a scheduling middleware and a business application service;
synchronizing the intelligent calculation result with the video stream, packaging according to a software defined camera interaction protocol, and sending to a cloud platform; receiving an intelligent calculation result and a video stream by an access scheduling middleware of the cloud platform; the access scheduling middleware decapsulates the received data to obtain a feature stream and a video stream, stores the video stream into a video file database, and stores analysis results of the decapsulated feature stream into corresponding structured and unstructured databases; in addition, the cloud platform receives a service request of the business application, retrieves the corresponding database, completes operations such as statistics, analysis and calculation of data and returns an obtained result to the terminal for display;
preferably, the third-party service engine is connected to the access scheduling middleware, and the third-party service engine may include a geographic information database and a blacklist information database; the third-party service engine is connected with the access scheduling middleware, and the access scheduling middleware receives the data service provided by the third-party service engine and makes application service based on the data.
The access scheduling middleware is responsible for protocol analysis, data storage scheduling and data retrieval scheduling;
the business application service is responsible for sending instructions to the access scheduling middleware and receiving data transmitted by the access scheduling middleware.
Preferably, the cloud platform receives a camera calculation model change request of the service application, retrieves the corresponding database to obtain information such as camera parameters and calculation model running states, and retrieves running support information of a model to be run in the calculation model algorithm pool; comparing the camera parameter information, the calculation model running state information and the running support information of the model to be run, wherein the camera parameter information can comprise resolution, frame rate and brightness, and the calculation model running state information comprises CPU (central processing unit), GPU (graphic processing unit), internal memory and video memory information; if the parameter information of the camera and the running state information of the calculation model accord with the changing condition of the running support information of the model to be run, continuing to change, otherwise, returning to fail to change; calling a model to be operated from a calculation model algorithm pool, packaging the calculation model according to a software defined camera interaction protocol, and transmitting the calculation model to an intelligent converter; and the intelligent converter receives the model to be operated, stops the calculation model of the specified camera in operation according to the camera parameters after the model to be operated is received and unpacked, starts the model to be operated by the specified camera, and finishes model updating.
Example two
A digital retina software defined camera method specifically comprises the following steps:
step 1, adding a video intelligent analysis module (including but not limited to Nvidia Jetson TX2) at a video stream output port of a video monitoring camera, wherein the Nvidia Jetson TX2 is an AI super computer in the embedded field of Invida. .
Step 2, processing the monitoring video stream according to the software defined camera protocol, comprising: detecting the format of an input video, wherein the step of extracting the parameters of the camera refers to decapsulating an accessed rtsp monitoring video stream, and extracting the ID information (such as an IP address) of the monitoring camera, the parameters of a video coding format (such as resolution, frame rate and code rate) and the like;
opening a path of monitoring video, extracting camera parameters and decoding
Protocol: zmq-req/rep
The functions are as follows: prompting to open a path of video, extracting and decoding camera parameters, and returning a handle if the path of video is successful.
Figure BDA0002182883990000071
Description of the drawings:
Request:
the initiator: a scheduling program;
cmd is the command word. Each command has its own independent command word for distinguishing different commands;
camera _ param: the address of the req/rep server for asynchronously receiving camera parameters and video format (json format);
pic _ receiver: zmq-req/rep server, for asynchronously receiving a target image (binary) of a video frame.
Response:
Errcode, errmsg: if the error occurs, the error code is not 0, and the errmsg is set as specific error information;
handle: if the open is successful, handle represents the unique ID of the open channel. Can be followed up
To use this ID for channel closing and the like.
Video stream data delivery
Protocol: zmq-req/rep
The functions are as follows: decoding the video and inputting the video into an intelligent calculation model;
Figure BDA0002182883990000081
if the transfer fails, then:
Figure BDA0002182883990000082
description of the drawings:
"error" of-1 means that the shared memory device was created with an error;
an "error" of-2 means that there is no corresponding decoder or no decoder running.
And step 3: reading the occupation rates of a CPU, a memory and a network I/O, GPU of a system resource manager of the intelligent converter;
protocol: zmq-req/rep;
the functions are as follows: counting the occupancy rate of system resources:
Figure BDA0002182883990000091
and 4, step 4: calling a corresponding calculation model (such as detection tracking and target structuring) for the decoded video sequence to perform intelligent calculation under the condition that system resources are available;
protocol: zmq-req/rep;
the functions are as follows: detection and tracking of visual targets:
Figure BDA0002182883990000092
Figure BDA0002182883990000101
protocol: zmq-req/rep;
the functions are as follows: object structuring
Figure BDA0002182883990000102
And 5: synchronizing the intelligent calculation result with the video stream, packaging according to a software defined camera interaction protocol, and sending to a cloud platform;
video stream data delivery
Protocol: zmq-req/rep
Request:
The functions are as follows: for synchronization with the feature stream and delivery to the cloud;
the format is as follows: adding a data structure behind a binary data system head 'b';
cmd: command word 1101(int32 little endian);
dev _ id: camera ID (64 byte string);
handle: task ID (session _ ID) (36-byte character string) of the camera;
pts (pts) is as follows: start pts of transcoded video stream, (8 bytes int64, little endian);
filename _ pts: the file name pts is used for segmenting the current system time (int64 little endian) of the first frame of the file, the file name is named according to the storage strategy of the video stream and the absolute time of the initial frame, and the same file can be sent in several times;
length code stream length (4 bytes int 32);
Response:
the format is as follows: adding a data structure behind a binary data system head 'b';
cmd: command word 1100(int32 little endian);
dev _ id: camera ID (64 byte string);
handle: task ID (session _ ID) (36-byte character string) of the camera;
return _ value: return value (int32 little endian), 0 is correct, not 0 error.
Video analytics results delivery
Protocol: zmq-req/rep
The functions are as follows: transmitting video analysis results to cloud
Figure BDA0002182883990000111
Figure BDA0002182883990000121
Step 6: the cloud platform decapsulates the received data to obtain a feature stream and a video stream, stores the video stream into a video file database, and stores analysis results obtained after feature stream decapsulation into corresponding structured and unstructured databases; in addition, the cloud platform receives a service request of the business application, retrieves the corresponding database, completes operations such as data statistics, analysis and calculation, and returns an obtained result to the terminal for display;
protocol: http post
The functions are as follows: and searching the pedestrian database according to the search condition, and if the search condition is successful, returning the pedestrian structured information.
Figure BDA0002182883990000122
If correctly returning:
Figure BDA0002182883990000123
Figure BDA0002182883990000131
otherwise, the error returns:
Figure BDA0002182883990000132
and 7: the cloud platform receives a camera calculation model change request of business application, retrieves a corresponding database to obtain information such as camera parameters and calculation model operation states, and retrieves operation support information of a model to be operated in a calculation model algorithm pool; comparing the camera parameter information, the calculation model running state information and the running support information of the model to be run, if the camera parameter information and the calculation model running state information meet the changing conditions, continuing to change, and if not, returning to the state that the camera parameter information and the calculation model running state information fail to change; calling a calculation model to be changed from a calculation model algorithm pool, packaging the calculation model according to a software defined camera interaction protocol, and transmitting the calculation model to an intelligent converter; the intelligent converter receives the model to be operated, after decapsulation is completed, the calculation model in which the specified camera is operating is stopped according to the camera parameters, the specified camera starts the model to be operated, and model updating is completed;
protocol: zmq-req/rep
The functions are as follows: transfer of computational model to be altered to converter
Figure BDA0002182883990000133
Figure BDA0002182883990000141
A digital retinal software defined camera system comprising: an intelligent converter, a cloud platform and a terminal,
the intelligent converter comprises an intelligent analysis module, the intelligent analysis module receives the monitoring video stream, processes the monitoring video stream according to a software definition camera protocol, monitors system resources of the intelligent converter, and calls a corresponding calculation model for the processed monitoring video stream to perform intelligent calculation when the resources are available;
the cloud platform stores the received intelligent calculation and video stream, receives retrieval and calculation services of the terminal, and returns the results obtained by the retrieval and calculation to the terminal for display;
and the terminal realizes user interaction and sends an instruction to the cloud platform.
Preferably, the cloud platform comprises an access scheduling middleware, a data storage module, a business application service module and a calculation algorithm pool module, wherein the access scheduling middleware is responsible for protocol analysis, data storage scheduling and data retrieval scheduling, receives an intelligent calculation result and a video stream of the intelligent analysis module and transmits the intelligent calculation result and the video stream to the data storage module; the business application service module sends an instruction to the access scheduling middleware, receives data transmitted by the access scheduling middleware and transmits the data to the terminal; and the calculation algorithm pool module stores calculation algorithms, accesses the scheduling middleware to call the model to be operated from the calculation model algorithm pool, packages the calculation model according to the software defined camera interaction protocol and transmits the calculation model to the intelligent converter.
Preferably, the intelligent analysis module comprises a video decoding module, a system resource module and an intelligent calculation module, wherein the video decoding module processes the monitoring video stream according to a software defined camera protocol, and specifically comprises the steps of decoding the accessed video stream and extracting the ID information and the video coding format parameters of the monitoring camera;
the system resource module monitors system resources of the intelligent converter, and specifically comprises reading the occupancy rates of a CPU, a memory and a network I/O, GPU of a resource manager of the intelligent converter;
and the intelligent computing module calls a corresponding computing model for the processed monitoring video stream to perform intelligent computing, receives the to-be-operated model transmitted by the access scheduling middleware and completes updating of the intelligent computing model.
Preferably, the data storage module comprises a video file database, a structured database and an unstructured database, the received intelligent calculation result and the video stream are unpacked to obtain a feature stream and a video stream, the video stream is stored in the video file database, and the feature stream is stored in the structured database and the unstructured database.
Preferably, the system further comprises a third-party service engine module, the third-party service engine module is connected with the access scheduling middleware, and the access scheduling middleware receives the data service provided by the third-party service engine and makes an application service based on the data.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the features described above or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims (10)

1. A digital retinal software defined camera method, comprising:
the intelligent converter receives the video stream of the camera and decodes the video stream;
the cloud platform monitors system resources of the intelligent converter and judges whether the system resources are available, when the system resources of the intelligent converter are available, the intelligent converter calls a calculation model of the camera to calculate the processed video stream, and when the system resources of the intelligent converter are unavailable, an error is reported;
the intelligent converter packages a calculation result and a video stream and then sends the calculation result and the video stream to the cloud platform;
the cloud platform receives the encapsulated computation results and video stream and stores the encapsulated computation results and video stream,
and the cloud platform receives retrieval and feature calculation instructions sent by the terminal, retrieves and calculates the calculation results and the video stream, and returns the results obtained by the retrieval and feature calculation to the terminal for display.
2. The digital retinal software defined camera method of claim 1,
when the cloud platform receives a request sent by a terminal and needing to change the calculation model of the camera, the parameter information of the camera, the running state information of the calculation model of the camera and the running support information of the model to be run are compared,
judging whether the parameter information of the camera and the running state information of the calculation model of the camera meet the running support information change condition of the model to be run,
if so, then
The cloud platform sends a model to be operated to the intelligent converter; the intelligent converter receives and finishes decapsulation and then stops the running calculation model of the camera, and the camera starts the model to be run to finish model updating;
if not, then
And returning the change failure to the terminal.
3. The method as claimed in claim 2, wherein the cloud platform receives the encapsulated computation results and video stream and stores the encapsulated computation results and video stream, specifically:
and the cloud platform decapsulates the received data to obtain a feature stream and a video stream, stores the video stream into a video file database, and stores the feature stream into a corresponding structured and/or unstructured database.
4. The method as claimed in claim 3, wherein the cloud end monitors system resources of the intelligent converter and determines whether the system resources are available, specifically comprising:
the intelligent converter reads the occupancy rates of the CPU, the memory and the network I/O, GPU of the intelligent converter resource manager and sends the occupancy rates to the cloud, and the cloud judges whether the system resources are available.
5. The digital retinal software defined camera method of claim 4 wherein the intelligent converter decodes the video stream comprising
And the intelligent converter detects the format of the video stream, decodes the accessed video stream and extracts the ID information of the monitoring camera and the video coding format parameters.
6. A digital retina software-defined camera system is characterized by comprising an intelligent converter, a cloud platform and a terminal,
the intelligent converter comprises an intelligent analysis module, the intelligent analysis module receives the video stream, processes the video stream, reads system resources of the intelligent converter, sends a reading result to the cloud, and calls a corresponding calculation model for the processed video stream to perform intelligent calculation when the resources are available;
the cloud platform stores the received intelligent calculation result and the video stream, receives retrieval and feature calculation of the terminal, and returns the results obtained by the retrieval and the feature calculation to the terminal for display;
and the terminal realizes user interaction and sends an instruction to the cloud platform.
7. The digital retinal software defined camera system of claim 6, wherein the cloud platform comprises access scheduling middleware, a data storage module, a business application service module, a computational algorithm pool module, wherein,
the access scheduling middleware receives the intelligent calculation result and the video stream of the intelligent analysis module and sends the intelligent calculation result and the video stream to the data storage module; the business application service module sends an instruction to the access scheduling middleware, receives data transmitted by the access scheduling middleware and transmits the data to the terminal; and the calculation algorithm pool module stores calculation algorithms, accesses the scheduling middleware to call the model to be operated from the calculation model algorithm pool, and sends the model to be operated to the intelligent converter.
8. The digital retinal software defined camera system of claim 7, wherein the intelligent analysis module comprises a video decoding module, a system resource module, an intelligent computing module, wherein
The video decoding module processes the monitoring video stream, specifically comprises the steps of decoding the accessed video stream and extracting ID information and video coding format parameters of the monitoring camera;
the system resource module reads system resources of the intelligent converter, and specifically comprises reading the occupancy rates of a CPU, a memory and a network I/O, GPU of a resource manager of the intelligent converter;
and the intelligent computing module calls a corresponding computing model for the processed monitoring video stream to perform intelligent computing, receives the to-be-operated model transmitted by the access scheduling middleware and completes updating of the intelligent computing model.
9. The digital retinal software defined camera system of claim 8, wherein the data storage module comprises a video file database, a structured and unstructured database, wherein,
and the data storage module decapsulates the received intelligent calculation result and the video stream to obtain a feature stream and a video stream, stores the video stream to a video file database, and stores the feature stream to a structured database and an unstructured database.
10. The digital retina software defined camera system of claim 9, further comprising a third party service engine module, wherein the third party service engine module is connected to the access scheduling middleware, and the access scheduling middleware receives the data service provided by the third party service engine and implements the application service based on the data.
CN201910803145.8A 2019-08-28 2019-08-28 Digital retina software defined camera method and system Active CN111541864B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910803145.8A CN111541864B (en) 2019-08-28 2019-08-28 Digital retina software defined camera method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910803145.8A CN111541864B (en) 2019-08-28 2019-08-28 Digital retina software defined camera method and system

Publications (2)

Publication Number Publication Date
CN111541864A true CN111541864A (en) 2020-08-14
CN111541864B CN111541864B (en) 2021-03-23

Family

ID=71976629

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910803145.8A Active CN111541864B (en) 2019-08-28 2019-08-28 Digital retina software defined camera method and system

Country Status (1)

Country Link
CN (1) CN111541864B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115984675A (en) * 2022-12-01 2023-04-18 扬州万方科技股份有限公司 System and method for realizing multi-channel video decoding and AI intelligent analysis

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160320837A1 (en) * 2015-05-01 2016-11-03 Massachusetts Institute Of Technology Methods and Apparatus for Retinal Retroreflection Imaging
CN106790367A (en) * 2016-11-15 2017-05-31 山东省科学院自动化研究所 The vehicle safety hidden danger early warning of big data treatment and accident reproduction system and method
CN108984799A (en) * 2018-08-21 2018-12-11 北京深瞐科技有限公司 A kind of video data handling procedure and device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160320837A1 (en) * 2015-05-01 2016-11-03 Massachusetts Institute Of Technology Methods and Apparatus for Retinal Retroreflection Imaging
CN106790367A (en) * 2016-11-15 2017-05-31 山东省科学院自动化研究所 The vehicle safety hidden danger early warning of big data treatment and accident reproduction system and method
CN108984799A (en) * 2018-08-21 2018-12-11 北京深瞐科技有限公司 A kind of video data handling procedure and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
高文、田永鸿、王坚: "数字视网膜:智慧城市系统演进的关键环节", 《中国科学:信息科学》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115984675A (en) * 2022-12-01 2023-04-18 扬州万方科技股份有限公司 System and method for realizing multi-channel video decoding and AI intelligent analysis
CN115984675B (en) * 2022-12-01 2023-10-13 扬州万方科技股份有限公司 System and method for realizing multipath video decoding and AI intelligent analysis

Also Published As

Publication number Publication date
CN111541864B (en) 2021-03-23

Similar Documents

Publication Publication Date Title
CN108255605B (en) Image recognition cooperative computing method and system based on neural network
CN104137146A (en) Method and system for video coding with noise filtering of foreground object segmentation
CN103475882B (en) Surveillance video encoding and recognizing method and surveillance video encoding and recognizing system
CN106254458B (en) A kind of image processing method based on cloud robot vision, platform and system
CN111770014A (en) Data information processing method, edge gateway and system for processing data information
CN108040221A (en) A kind of intelligent video analysis and monitoring system
CN111090773B (en) Digital retina system structure and software architecture method and system
CN110662017B (en) Video playing quality detection method and device
CN105407316A (en) Implementation method for intelligent camera system, intelligent camera system, and network camera
WO2023179161A1 (en) Video frame rate control method and apparatus, and electronic device and storage medium
CN111935663B (en) Sensor data stream processing method, device, medium and electronic equipment
CN111541864B (en) Digital retina software defined camera method and system
CN112312070A (en) Digital retina cloud software scheduling method
US10506021B2 (en) Method and device for providing communication connection for a plurality of candidate applications in a mobile device
CN111209566A (en) Intelligent anti-crawler system and method for multi-layer threat interception
CN112087632A (en) Video processing system, method, storage medium and computer device
CN114640854A (en) Real-time high-speed decoding method for multi-channel video stream
CN107194013B (en) Processing method and processing system based on Distributed Message Queue middleware
CN107295340A (en) A kind of method and device of remote desktop Video coding
CN110727722A (en) Massive concurrent radar data storage method
CN112565800B (en) Video positioning method, device, equipment, system and storage medium
CN112202786B (en) Illegal data identification method and device and electronic equipment
CN113596396B (en) GB 28181-based target tracking system
CN117176962B (en) Video encoding and decoding method and device and related equipment
CN114125502B (en) Video stream management method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant