CN115209111A - Home and office video monitoring method and system supporting real-time background replacement - Google Patents
Home and office video monitoring method and system supporting real-time background replacement Download PDFInfo
- Publication number
- CN115209111A CN115209111A CN202210884211.0A CN202210884211A CN115209111A CN 115209111 A CN115209111 A CN 115209111A CN 202210884211 A CN202210884211 A CN 202210884211A CN 115209111 A CN115209111 A CN 115209111A
- Authority
- CN
- China
- Prior art keywords
- module
- real
- monitoring
- time
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/08—Protocols for interworking; Protocol conversion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention discloses a method and a system for monitoring a home office video supporting real-time background replacement, wherein the method comprises the following steps: a client user logs in a plug-flow client program through a computer, and the working state of the client user is updated from off-line to on-line after logging is successful; the client user clicks the working button, the monitoring module applies for the camera permission to the client user, and after the client user authorizes the camera permission, the monitoring module collects the camera video stream of the client user in real time. The invention can collect, process and analyze a plurality of monitoring video streams, perform real-time foreground segmentation and background replacement on personnel in the video, perform information summarization and visualization on all current monitoring, provide monitoring information and attendance data of the personnel in real time, solve the problem of difficult supervision of the personnel at home and in office, ensure the real-time property, privacy and safety of monitoring contents and greatly help the attendance management of the company at home and in office.
Description
Technical Field
The invention relates to the technical field of image and video processing, in particular to a method and a system for monitoring a home office video supporting real-time background replacement.
Background
At present, epidemic situations are popular all over the world, and little contact and little aggregation are integrated to meet new requirements of people on production and life. In order to respond to the national call and maintain the social security and stability, a plurality of enterprises and public institutions start to implement the policy of staffs working at home, which is particularly obvious for customer service staffs belonging to the third industry, however, the working efficiency is low, and the supervision difficulty is always the biggest pain point in the mode of working at home. At present, the supervision means of customer service personnel working at home mainly refers to 'weak monitoring' such as off-duty card punching, daily and weekly reporting, irregular spot check and reply and the like, and the method is very dependent on the experience of management personnel, consumes manpower and has little effect. In order to conveniently monitor home customer service staff and improve the efficiency of home office, a 'strong monitoring' system capable of meeting real-time video monitoring is adopted. The monitoring system supports the transmission of the pictures collected by the camera to the monitoring end for real-time playing, and is widely applied to the fields of the open kitchen system and the security monitoring. However, in consideration of the particularity of the customer service home office scene, the problem of privacy safety is inevitably brought to the direct monitoring of the customer service home office scene, so that a background replacement function is added to the monitoring system by means of the development of the artificial intelligence related technology, the situation that the customer service home environment is directly exposed in a network is avoided, and the problems of privacy and safety existing in the 'strong monitoring' system are solved.
Currently, common methods for achieving background replacement are green curtain matting, interactive matting, and matting based on deep learning. The green screen cutout uses a fixed green screen background, segmented by RGB color thresholds. The method is time-consuming and labor-consuming, has long manufacturing period and has requirements on sites and equipment. Therefore, the green screen cutout is more limited in use and poorer in applicability. Interactive matting is the computation of the opacity channel az using interactive computation to get more a priori information, common including the GrabCut algorithm and the lazyssnapping algorithm. The interactive matting algorithm needs more prior information, and the manual labeling of the foreground and the background consumes a large amount of energy, and simultaneously, the individual difference of different people's labels exists, so that the interactive matting accuracy is low, and the matting and real-time matting tasks of complex backgrounds cannot be met, so that the method is very difficult to process the matting task of a video streaming form.
There is currently no effective solution to the above problems.
Disclosure of Invention
In view of the above technical problems in the related art, the present invention provides a method and a system for monitoring a video in a home and office supporting real-time background replacement, which can overcome the above disadvantages in the prior art.
In order to achieve the technical purpose, the technical scheme of the invention is realized as follows:
a home office video monitoring method supporting real-time background replacement comprises the following steps:
s1, a client user logs in a plug-flow client program through a computer, and after the log-in is successful, the working state of the client user is updated to be online from offline;
s2, a client user clicks a working button, a monitoring module applies for the camera authority to the client user, and after the client user authorizes the camera authority, the monitoring module collects the camera video stream of the client user in real time, and meanwhile, the working state of the client user is modified to be in work from on-line;
s3, the monitoring module encodes, encapsulates and packs the acquired real-time video stream and sends video frame data to the algorithm module;
s4, the algorithm module processes the video frames in real time and returns the video frames to the monitoring module;
s5, the monitoring module pushes the processing result to a streaming media server by using an RTMP protocol;
s6, the streaming media server performs protocol conversion, converts the RTMP protocol into the WebRTC protocol, provides a Web page stream pulling capability for the monitoring platform, and meanwhile records and persists the processed real-time video stream to a database;
and S7, the system visualization module obtains data, renders the result to a UI interface, and displays the current monitoring condition to a supervisor in real time.
Further, the algorithm module in the S4 comprises preprocessing, foreground segmentation and image fusion, and the specific steps are as follows:
s41, carrying out coding and decoding preprocessing on the real-time video stream captured by the acquisition equipment;
s42, performing foreground segmentation on the cached video frame to obtain a character mask and a foreground image;
s43, carrying out image fusion on the character foreground and a preset background image to obtain a video frame subjected to background replacement;
and S44, packaging and transmitting the video frame after the background replacement.
Further, the algorithm module adopts a lightweight network and a model compression skill to reduce the computational complexity and ensure normal work under the condition of insufficient computational resources.
Further, the data in S7 includes video stream information, basic information, attendance information, and statistical information of the client user.
A home office video surveillance system supporting real-time background replacement, comprising: the system comprises a plug flow client, a streaming media server, a database and a monitoring platform end, wherein the plug flow client comprises a login module, a monitoring module and an algorithm module, the database comprises a data persistence module, and the monitoring platform comprises a service processing module and a system visualization module;
the login module is used for being responsible for a client user to log in a client program;
the monitoring module is used for acquiring a camera video of a client user computer, acquiring the latest frame image of a video stream in real time, sending the latest frame image to the algorithm module to perform background replacement processing on the video frame, and simultaneously storing related parameters to the data persistence module;
the algorithm module: the system comprises a monitoring module, a semantic segmentation network algorithm, a background image generation module, a semantic segmentation network algorithm and a real-time video frame acquisition module, wherein the monitoring module is used for acquiring a real-time video frame from the monitoring module, carrying out image preprocessing on the real-time video frame, sending the real-time video frame into the semantic segmentation network algorithm for real-time segmentation, and fusing and drawing a segmentation result and a pre-provided background image onto a latest frame image;
the streaming media server is used for receiving the video stream processed by the algorithm module, transcoding a streaming media protocol from RTMP to WebRTC, providing a Web page streaming capability for the monitoring platform, and recording and persisting the processed real-time video stream to a database;
the data persistence module is used for storing video stream data which is acquired by a camera of a client user computer and processed by an algorithm, providing data support for a monitoring platform and simultaneously providing support for the playback of subsequent monitoring videos;
the service processing module is used for realizing the user function requirement of the monitoring platform and packaging the bottom service logic into an API interface function for the calling of the visualization module;
the system visualization module is used for requesting the service processing module for the relevant information of the client user, pulling the result video stream processed by the algorithm module to be displayed to the front end, displaying the current behavior state of the client user to a supervisor in real time, and simultaneously requesting the service processing module for the working state statistical information of the client user and performing ranking display.
Further, the algorithm module comprises a real-time background replacement algorithm model.
Further, the data persistence module stores video stream data by adopting a file system and stores structured data by adopting a MySQL database.
Furthermore, the functions of the service processing module include an account management function, data integration statistics, a working state switching function and a condition retrieval function.
Further, the bottom layer service logic comprises a video stream processed by an image algorithm and an acquisition mode of the existing data.
The invention has the beneficial effects that: the invention can collect, process and analyze a plurality of monitoring video streams, perform real-time foreground segmentation and background replacement on personnel in the video, and perform information summarization and visualization on all current monitoring, can provide monitoring information and attendance data of the personnel in real time, solves the problem of difficult supervision of the personnel at home and in office, simultaneously ensures the real-time property, privacy and safety of monitoring contents, and is greatly helpful for the attendance management of the company at home and in office.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required in the embodiments will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a macroscopic overall architecture diagram of a home office video monitoring method and system supporting real-time background replacement according to an embodiment of the present invention;
fig. 2 is a flowchart of a method for monitoring a video surveillance of a home office supporting real-time background replacement according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a plug-flow client plug-flow process of a home office video monitoring method supporting real-time background replacement according to an embodiment of the present invention;
fig. 4 is a schematic diagram illustrating a change of a person state of a push streaming client in a whole life cycle of a home office video monitoring method supporting real-time background replacement according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a home office video surveillance system supporting real-time background replacement according to an embodiment of the present invention;
fig. 6 is a flow chart illustrating a business process of a monitoring platform and a database of a home office video monitoring system supporting real-time background replacement according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.
As shown in fig. 1 to 4, a method for monitoring a video in a home office supporting real-time background replacement includes the following steps:
s1, a client user logs in a plug-flow client program through a computer, and after the log-in is successful, the working state of the client user is updated to be online from offline;
s2, a client user clicks a work button, a monitoring module applies for the camera permission to the client user, and after the client user authorizes the camera permission, the monitoring module collects the camera video stream of the client user in real time, and meanwhile, the working state of the client user is modified from online to working;
s3, the monitoring module encodes, encapsulates and packs the acquired real-time video stream and sends video frame data to the algorithm module;
s4, the algorithm module processes the video frames in real time and returns the video frames to the monitoring module;
s5, the monitoring module pushes the processing result to a streaming media server by using an RTMP protocol;
s6, the streaming media server performs protocol conversion, converts the RTMP protocol into the WebRTC protocol, provides a Web page stream pulling capability for the monitoring platform, and meanwhile records and persists the processed real-time video stream to a database;
and S7, the system visualization module obtains data, renders the result to a UI (user interface), and displays the current monitoring condition to a supervisor in real time.
In the embodiment, the algorithm module in S4 includes preprocessing, foreground segmentation, and image fusion, and the specific steps are as follows:
s41, carrying out coding and decoding preprocessing on the real-time video stream captured by the acquisition equipment;
s42, performing foreground segmentation on the cached video frame to obtain a character mask and a foreground image;
s43, carrying out image fusion on the character foreground and a preset background image to obtain a video frame subjected to background replacement;
s44, the video frame after the background replacement is packaged, packed and transmitted.
In an embodiment, the algorithm module adopts a lightweight network and a model compression technique to reduce the computational complexity and ensure normal work under the condition of insufficient computational resources.
In an embodiment, the data in S7 includes video stream information, basic information, attendance information, and statistical information of the client user.
As shown in fig. 1, 5 and 6, a home office video surveillance system supporting real-time background replacement includes: the system comprises a stream pushing client, a streaming media server, a database and a monitoring platform end, wherein the stream pushing client comprises a login module, a monitoring module and an algorithm module, the database comprises a data persistence module, and the monitoring platform comprises a service processing module and a system visualization module;
the login module is used for being responsible for a client user to log in a client program;
the monitoring module is used for acquiring a camera video of a client user computer, acquiring the latest frame image of a video stream in real time, sending the latest frame image to the algorithm module to perform background replacement processing on the video frame, and simultaneously storing related parameters to the data persistence module;
the algorithm module: the system comprises a monitoring module, a semantic segmentation network algorithm, a background image generation module, a semantic segmentation network algorithm and a real-time video frame acquisition module, wherein the monitoring module is used for acquiring a real-time video frame from the monitoring module, carrying out image preprocessing on the real-time video frame, sending the real-time video frame into the semantic segmentation network algorithm for real-time segmentation, and fusing and drawing a segmentation result and a pre-provided background image onto a latest frame image;
the streaming media server is used for receiving the video stream processed by the algorithm module, transcoding a streaming media protocol from RTMP to WebRTC, providing a Web page streaming capability for the monitoring platform, and recording and persisting the processed real-time video stream to a database;
the data persistence module is used for storing video stream data which is acquired by a camera of a client user computer and processed by an algorithm, providing data support for a monitoring platform and simultaneously providing support for the playback of subsequent monitoring videos;
the service processing module is used for realizing the user function requirement of the monitoring platform and packaging the bottom service logic into an API interface function for the visual module to call;
the system visualization module is used for requesting the service processing module for the relevant information of the client user, pulling the result video stream processed by the algorithm module to be displayed to the front end, displaying the current behavior state of the client user to a supervisor in real time, and simultaneously requesting the service processing module for the working state statistical information of the client user and performing ranking display.
In an embodiment, the algorithm module comprises a real-time background replacement algorithm model.
In an embodiment, the data persistence module stores video stream data by using a file system and stores structured data by using a MySQL database.
In an embodiment, the functions of the service processing module include an account management function, a data integration and statistics function, a working state switching function, and a condition retrieval function.
In an embodiment, the underlying service logic includes a video stream processed by an image algorithm and an acquisition mode of existing data.
In order to facilitate understanding of the above-described technical aspects of the present invention, the above-described technical aspects of the present invention will be described in detail below in terms of specific usage.
When the method is used specifically, the method for monitoring the home office video supporting real-time background replacement comprises the following technical scheme:
a video real-time monitoring scheme under a home and office scene. The scheme relates to three major parts, namely a stream pushing client, a stream media server and a monitoring platform, and comprises the following specific steps:
s11: the method comprises the following steps that household office staff use a plug-flow client installed on a working computer to collect real-time video streams through a PC camera;
s12: the plug-flow client carries out algorithm processing on the acquired real-time video stream and pushes the stream to the streaming media server;
s13: the streaming media server transcodes the video stream from RTMP to WebRTC and records the video stream for persistence;
s14: and the monitoring platform pulls the video stream from the streaming media server and visualizes the video stream to a Web page by combining other information of personnel.
A real-time background replacement method in a home office video monitoring scene. The method mainly comprises the steps of preprocessing, foreground segmentation and image fusion, and specifically comprises the following steps:
s21: preprocessing a real-time video stream captured by the acquisition equipment, such as encoding and decoding;
s22: performing foreground segmentation on the cached video frame to obtain a character mask and a foreground image;
s23: and carrying out image fusion on the character foreground and a preset background image to obtain a video frame after background replacement.
S24: and encapsulating, packing and transmitting the video frame attached with the new background.
A customer service personnel home office video surveillance system supporting real-time background replacement, comprising:
a login module: the module is responsible for customer service personnel to log in a client program and is an entrance of the system.
A monitoring module: the module is used for acquiring a camera video of a customer service computer, acquiring the latest frame image of a video stream in real time, sending the latest frame image into the algorithm module to perform background replacement processing on the video frame, and meanwhile, storing related parameters into the data persistence module.
An algorithm module: this module is the core functional module of the system, where the real-time background replacement algorithm model proposed herein is deployed. The module firstly acquires a real-time video frame from a monitoring module, carries out image preprocessing on the real-time video frame, sends the real-time video frame into a semantic segmentation network algorithm for real-time segmentation, fuses a segmentation result with a pre-provided background image, and draws the segmentation result on the latest frame image.
The streaming media server: the module is an intermediary between the client and the monitoring platform and is also an intermediary between push flow and pull flow. The system is mainly responsible for receiving the video stream processed by the algorithm module, transcoding the streaming media protocol from RTMP to WebRTC, providing a Web page streaming capability for a monitoring platform, and recording and persisting the processed real-time video stream to a database.
A data persistence module: the module is used for storing video stream data which is acquired by a client service working computer camera and processed by an algorithm, providing data support for realizing the functions of the system and simultaneously providing support for the playback of subsequent possible monitoring videos. The module uses a file system to store video stream data and a MySQL database to store other structured data.
A service processing module: the module is used for realizing application functions required by users, and bottom service logic is required to be packaged into an API interface function for external calling. Based on the functional requirement analysis, the module needs to implement four main functions including: account management function, data integration statistics, working state switching and condition retrieval function. And packaging the video stream processed by the image algorithm and the acquisition mode of the existing data into an interface function for calling.
A system visualization module: the module firstly requests the relevant information of the customer service staff from the service processing module, then pulls the result video stream processed by the algorithm module to be displayed to the front end, and displays the current behavior state of the customer service staff to the supervisor in real time. And meanwhile, requesting the service working state statistical information to a service processing module for carrying out scheduling display.
As shown in fig. 1, the system macroscopically mainly comprises the following three parts, namely, a push streaming client, a streaming media server and a monitoring platform. The push flow client serves customer service staff and is mainly responsible for operations such as video flow collection, processing and pushing. The monitoring platform serves the monitoring personnel and is mainly responsible for displaying monitoring videos and other information including attendance data, managing personnel and the like. The streaming media server serves as an intermediary part of the system, can receive the real-time video stream pushed by the customer service end and supports the pull play of the monitoring end. And simultaneously, transcoding, recording and other operations are supported.
As shown in fig. 2, a work flow diagram of a home office video monitoring method supporting real-time background replacement specifically includes the following steps:
s1: customer service personnel log in a client program through a working computer, and the working state of a user is updated from off-line to on-line at the moment;
s2: the customer service personnel click the work button to start working. The monitoring module can apply for the camera permission to the user, and after the user authorizes the monitoring module can acquire customer service camera information in real time. Meanwhile, the working state can be modified from online to working;
s3: the monitoring module carries out processing such as coding, packaging and packaging on the acquired real-time video stream and sends video frame data to the algorithm module;
s4: the algorithm module processes the video frame in real time, segments the customer service personnel foreground from the original video frame, recombines the customer service personnel foreground with a pre-provided background image into a result frame, and returns the result frame to the monitoring module;
s5: the monitoring module pushes the processing result to a streaming media server by using an RTMP protocol;
s6: the streaming media server carries out protocol conversion, converts the RTMP protocol into the WebRTC protocol, and records the video stream in real time to complete persistence;
s7: the supervisor inputs a system IP access front-end display interface in a web browser, selects a certain customer service person for real-time monitoring, and the system visualization module immediately requests data from the service processing module;
s8: the service processing module requests basic information of customer service personnel from the data persistence module and requests media resources from the streaming media server;
s9: the service processing module acquires the basic information and the media resources of the customer service personnel, packs the basic information and the media resources and returns the basic information and the media resources to the system visualization module;
s10: and the system visualization module obtains data, renders the result to a UI interface, and displays the current monitoring condition to a supervisor in real time.
Fig. 3 illustrates a complete acquisition and push flow process of the push flow client according to the present invention, in which a request for a camera permission is first made to a customer service person, and if the request for the permission fails, an exception log is recorded; video capture () method initialization and video capture. And after the latest frame is read successfully, the latest frame is sent to an algorithm module for processing. And then the processed video frame is coded and transmitted, comprising the following steps:
s1: initializing AVFormatContext;
s2: initializing an io channel;
s3: finding a suitable encoder;
s4: setting encoder parameters and turning on an encoder;
s5: allocating space for the frame and setting frame related parameters;
s6: the frame is encoded and put into a packet for transmission.
Fig. 4 illustrates the working state of the customer service person throughout the lifecycle in the push streaming client. There are four working states in total, which are respectively: online state, offline state, working state and rest state, and the mutual conversion relation of the four states is given.
FIG. 5 illustrates a block diagram between systems, including: the system comprises a plug flow client, a streaming media server, a database and a monitoring platform end, wherein the plug flow client comprises a login module, a monitoring module, an algorithm module and a computer camera, the database comprises a data persistence module, and the monitoring platform comprises a service processing module and a system visualization module, and simultaneously displays the main logic relationship between the service processing module and the system visualization module.
FIG. 6 illustrates a flow chart of the business process of the monitoring platform and the database. The monitoring platform can realize five functions: obtaining all monitoring information, obtaining recent monitoring data, obtaining personal details, obtaining a customer service list and obtaining historical exceptions.
In summary, by means of the technical scheme of the invention, the invention can collect, process and analyze a plurality of monitoring video streams, perform real-time foreground segmentation and background replacement on personnel in the video, perform information summarization and visualization on all current monitoring, provide monitoring information and attendance data of the personnel in real time, solve the problem of difficult supervision of the personnel at home and during work, ensure the real-time property, privacy and safety of monitoring contents, and greatly help the attendance management of the company at home and work.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, which is intended to cover any modifications, equivalents, improvements, etc. within the spirit and scope of the present invention.
Claims (9)
1. A home office video monitoring method supporting real-time background replacement is characterized by comprising the following steps:
s1, a client user logs in a plug-flow client program through a computer, and after the client user logs in successfully, the working state of the client user is updated to be online from offline;
s2, a client user clicks a working button, a monitoring module applies for the camera authority to the client user, and after the client user authorizes the camera authority, the monitoring module collects the camera video stream of the client user in real time, and meanwhile, the working state of the client user is modified to be in work from on-line;
s3, the monitoring module encodes, encapsulates and packs the acquired real-time video stream and sends video frame data to the algorithm module;
s4, the algorithm module processes the video frames in real time and returns the video frames to the monitoring module;
s5, the monitoring module pushes the processing result to a streaming media server by using an RTMP protocol;
s6, the streaming media server performs protocol conversion, converts the RTMP protocol into the WebRTC protocol, provides a Web page stream pulling capability for the monitoring platform, and meanwhile records and persists the processed real-time video stream to a database;
and S7, the system visualization module obtains data, renders the result to a UI interface, and displays the current monitoring condition to a supervisor in real time.
2. The method for monitoring the video of the home and the office with the support of the real-time background replacement according to claim 1, wherein the algorithm module in the S4 comprises preprocessing, foreground segmentation and image fusion, and the method comprises the following specific steps:
s41, carrying out coding and decoding preprocessing on the real-time video stream captured by the acquisition equipment;
s42, performing foreground segmentation on the cached video frame to obtain a character mask and a foreground image;
s43, carrying out image fusion on the character foreground and a preset background image to obtain a video frame subjected to background replacement;
s44, the video frame after the background replacement is packaged, packed and transmitted.
3. The method of claim 2, wherein the algorithm module employs a lightweight network and model compression techniques to reduce computational complexity to ensure proper operation with insufficient computational resources.
4. The method for home office video surveillance supporting real-time background replacement as recited in claim 1, wherein the data in S7 comprises video stream information, basic information, attendance information and statistical information of client users.
5. A home office video surveillance system supporting real-time background replacement, comprising: the system comprises a plug flow client, a streaming media server, a database and a monitoring platform end, wherein the plug flow client comprises a login module, a monitoring module and an algorithm module, the database comprises a data persistence module, and the monitoring platform comprises a service processing module and a system visualization module;
the login module is used for being responsible for a client user to log in a client program;
the monitoring module is used for acquiring a camera video of a client user computer, acquiring the latest frame image of a video stream in real time, sending the latest frame image to the algorithm module to perform background replacement processing on the video frame, and simultaneously storing related parameters to the data persistence module;
the algorithm module: the system comprises a monitoring module, a semantic segmentation network algorithm, a background image generation module, a semantic segmentation network algorithm and a real-time video frame acquisition module, wherein the monitoring module is used for acquiring a real-time video frame from the monitoring module, sending the real-time video frame into the semantic segmentation network algorithm for real-time segmentation after image preprocessing, and fusing and drawing a segmentation result and a pre-provided background image onto a latest frame image;
the streaming media server is used for receiving the video stream processed by the algorithm module, transcoding a streaming media protocol from RTMP to WebRTC, providing a Web page streaming capability for the monitoring platform, and recording and persisting the processed real-time video stream to a database;
the data persistence module is used for storing video stream data which is acquired by a camera of a client user computer and processed by an algorithm, providing data support for a monitoring platform and simultaneously providing support for the playback of subsequent monitoring videos;
the service processing module is used for realizing the user function requirement of the monitoring platform and packaging the bottom service logic into an API interface function for the calling of the visualization module;
the system visualization module is used for requesting the service processing module for the relevant information of the client user, pulling the result video stream processed by the algorithm module to be displayed to the front end, displaying the current behavior state of the client user to a supervisor in real time, and simultaneously requesting the service processing module for the working state statistical information of the client user and performing ranking display.
6. The home office video surveillance system supporting real-time background replacement of claim 1, wherein the algorithm module comprises a real-time background replacement algorithm model.
7. The home office video surveillance system supporting real-time background replacement of claim 1, wherein the data persistence module stores video stream data using a file system and structured data using a MySQL database.
8. The system as claimed in claim 1, wherein the functions of the service processing module include an account management function, a data integration and statistics function, a working state switching function, and a condition retrieval function.
9. The system of claim 1, wherein the underlying business logic comprises video streaming processed by image algorithms and means for obtaining existing data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210884211.0A CN115209111A (en) | 2022-07-26 | 2022-07-26 | Home and office video monitoring method and system supporting real-time background replacement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210884211.0A CN115209111A (en) | 2022-07-26 | 2022-07-26 | Home and office video monitoring method and system supporting real-time background replacement |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115209111A true CN115209111A (en) | 2022-10-18 |
Family
ID=83584069
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210884211.0A Pending CN115209111A (en) | 2022-07-26 | 2022-07-26 | Home and office video monitoring method and system supporting real-time background replacement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115209111A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116600147A (en) * | 2022-12-29 | 2023-08-15 | 广州紫为云科技有限公司 | Method and system for remote multi-person real-time cloud group photo |
-
2022
- 2022-07-26 CN CN202210884211.0A patent/CN115209111A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116600147A (en) * | 2022-12-29 | 2023-08-15 | 广州紫为云科技有限公司 | Method and system for remote multi-person real-time cloud group photo |
| CN116600147B (en) * | 2022-12-29 | 2024-03-29 | 广州紫为云科技有限公司 | Method and system for remote multi-person real-time cloud group photo |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020211430A1 (en) | Smart city system and implementation method therefor | |
| CN109903414A (en) | One kind is based on Internet of Things monitoring office attendance control system and method | |
| CN112087523B (en) | Intelligent building management system and device based on cloud service and computer readable storage medium | |
| CN112770088A (en) | AI video linkage perception monitoring system | |
| CN115209111A (en) | Home and office video monitoring method and system supporting real-time background replacement | |
| US10719552B2 (en) | Focalized summarizations of a video stream | |
| CN108391092A (en) | Danger identifying system based on deep learning | |
| CN111223011A (en) | Food safety supervision method and system for catering enterprises based on video analysis | |
| CN114666576B (en) | Public safety edge gateway system | |
| Savva et al. | Recognizing student facial expressions: A web application | |
| CN106993163B (en) | A kind of video monitoring system based on motion image detection | |
| CN108366232A (en) | A kind of intelligent video monitoring system based on mobile terminal virtual reality technology | |
| CN112333434B (en) | Intelligent video analysis system | |
| CN114222163A (en) | Intelligent video analysis system for business hall | |
| CN102694845A (en) | Tri-network converged mobile phone anti-theft monitoring system | |
| CN106993164B (en) | A kind of video monitoring method based on motion image detection | |
| CN103581695A (en) | System and method for achieving access of mobile terminal to global eye | |
| Yi et al. | The model of face recognition in video surveillance based on cloud computing | |
| CN111314350A (en) | Image storage system, storage method, calling system and calling method | |
| CN114979580A (en) | Image recognition device and method for machine room monitoring | |
| CN114187562A (en) | Campus safety early warning system based on behavior analysis | |
| CN114124995A (en) | Internet of things perception system service platform | |
| CN112699451A (en) | Community real-time twin system based on combination of community service middlebox and data middlebox with BIM | |
| CN115578766A (en) | Agent customer service video monitoring method and system | |
| CN116170465A (en) | Holographic perception intelligent security gateway system and implementation method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |