CN115379260B - Video privacy processing method and device, storage medium and electronic device - Google Patents
Video privacy processing method and device, storage medium and electronic device Download PDFInfo
- Publication number
- CN115379260B CN115379260B CN202211001465.XA CN202211001465A CN115379260B CN 115379260 B CN115379260 B CN 115379260B CN 202211001465 A CN202211001465 A CN 202211001465A CN 115379260 B CN115379260 B CN 115379260B
- Authority
- CN
- China
- Prior art keywords
- target
- privacy
- target area
- video
- path
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title abstract description 7
- 238000012545 processing Methods 0.000 claims abstract description 90
- 230000000694 effects Effects 0.000 claims abstract description 70
- 238000000034 method Methods 0.000 claims abstract description 45
- 230000006399 behavior Effects 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 17
- 239000003550 marker Substances 0.000 claims description 14
- 230000009471 action Effects 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 12
- 230000005540 biological transmission Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000002596 correlated effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000000875 corresponding effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
Abstract
The invention provides a video privacy processing method, a video privacy processing device, a storage medium and an electronic device, wherein the method comprises the following steps: determining a target area in a first video picture; and under the condition that the behavior of the first object entering and/or leaving the target area is detected, determining a target active path of the first object in the first video picture, and carrying out privacy processing on the target active path in the first video picture to obtain a second video picture which does not contain the target active path. The method and the device solve the problems of insufficient shielding of the privacy information in the video and lower accuracy of shielding of the privacy information in the prior art, and achieve the effect of improving the accuracy of shielding of the privacy information in the video.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to a method and apparatus for processing privacy of video, a storage medium, and an electronic apparatus.
Background
Video monitoring is increasingly permeated into the life of people at present, such as on roads, various public places, or in home environments and the like, and is widely applied to a plurality of occasions with intuitiveness, accuracy, timeliness and rich information content. In the field of video surveillance, in order to block a privacy area or other areas that cannot be monitored, a block frame is usually disposed on a video surveillance image for blocking.
In the prior art, when a privacy area is shielded by a network camera IPC, a shielding frame is usually set in a picture of a video image manually by a user, and the shielding frame is set to be black or white or mosaic in a live picture, so that the shielding effect is achieved. In some typical surveillance video frames, the identified privacy information is subject to a coding occlusion process. Therefore, the worry of privacy information leakage of most people can be reduced, and daily public monitoring safety can be ensured. The purpose of the privacy zone covering is to protect the information in the zone, but the privacy covering in the zone actually covers the action and behavior of the moving object in the zone, and in a specific scene, the target object moving in the zone needs to be subjected to the shielding treatment of the non-privacy zone even if the target object is not in the privacy shielding zone. However, in the prior art, only the privacy covering area is shielded by the video, which results in insufficient shielding of the privacy information in the video and lower accuracy of shielding of the privacy information.
In view of the above problems in the related art, no effective solution has been proposed at present.
Disclosure of Invention
The embodiment of the invention provides a video privacy processing method, a video privacy processing device, a storage medium and an electronic device, which at least solve the problems of insufficient shielding of privacy information in videos and lower accuracy of shielding of the privacy information in the related technologies.
According to an embodiment of the present invention, there is provided a method for privacy processing of video, including: determining a target area in a first video picture, wherein the target area comprises an area pre-configured as a privacy type; and under the condition that the behavior of the first object entering and/or leaving the target area is detected, determining a target active path of the first object in the first video picture, and carrying out privacy processing on the target active path in the first video picture to obtain a second video picture which does not contain the target active path.
Optionally, in case the detecting that the first object has a behavior of entering and/or exiting the target area, determining a target activity path of the first object in the first video frame is preceded by the method further comprising: acquiring a target early warning frame in the first video frame, wherein the video frame selected by the target early warning frame comprises the video frame of the target area; marking a privacy processing pre-mark for the first object under the condition that the first object enters the target early warning frame, wherein the privacy processing pre-mark is used for marking the behavior of the first object to enter the target area; modifying the first object's mark from the privacy handling pre-mark to a privacy mark in case the first object has been marked with the privacy handling pre-mark and the first object is detected to enter the target area, wherein the privacy mark is used for identifying that the first object has occurred to enter the target area; in the event that the first object has been marked with the privacy processing pre-marker and it is detected that the first object has not entered the target area and has left the target pre-warning frame, the privacy processing pre-marker for the first object is deleted after a first time interval in which the first object is determined to leave the target pre-warning frame.
Optionally, the privacy processing of the target activity path in the first video frame includes: and under the condition that the first object exists in the first video frame included in the first video picture, replacing the first video frame with a target video frame shot before the first video frame, wherein the target video frame is a video frame without any object marked with the privacy mark.
Optionally, after modifying the tag of the first object from the privacy handling pre-tag to a privacy tag in the event that the first object is detected to enter the target area, the method further comprises: and analyzing all path information of the first object in the first video picture, and determining the target activity path comprising all path information.
Optionally, the method further comprises: adding an object entering the first video picture to an object feature list in case that the object is detected to exist, wherein the object feature list is used for recording information of all objects identified in the first video picture; detecting whether a second object which is not marked with the privacy processing pre-mark or the privacy mark exists in the object feature list at intervals of a preset second time interval; and deleting the information of the second object recorded in the object feature list.
Optionally, the determining, in the case that the behavior of the first object entering and/or exiting the target area is detected, a target activity path of the first object in the first video frame includes: in the case that the first object is detected to have a behavior of leaving the target area and a first active path of the stored first object before entering the target area, which is generated in the first video picture, a second active path of the first object after leaving the target area is associated with the first active path, so as to obtain the target active path comprising the first active path and the second active path; determining the second activity path of the first object after leaving the target area as the target activity path in the case that the first object is detected to have the action of leaving the target area and no stored first activity path of the first object before entering the target area, which is generated in the first video picture; and determining a first active path of the first object before entering the target area, which is generated in the first video picture, as the target active path under the condition that the first object is detected to have the action of entering the target area.
Optionally, the method further comprises: returning the second video picture to the user with the first authority under the condition that a video picture viewing request initiated by the user with the first authority is received, wherein the first authority is the authority which does not have the function of viewing the target activity path; and/or returning the first video picture to the user with the second authority under the condition that a video picture viewing request initiated by the user with the second authority is received, wherein the second authority is the authority for viewing the target activity path.
According to another embodiment of the present invention, there is provided a privacy processing apparatus of video, including: a determining module configured to determine a target area in a first video picture, wherein the target area includes an area configured in advance as a privacy type; and the processing module is used for determining a target active path of the first object in the first video picture under the condition that the first object is detected to enter and/or leave the target area, and carrying out privacy processing on the target active path in the first video picture to obtain a second video picture which does not contain the target active path.
According to a further embodiment of the invention, there is also provided a storage medium having stored therein a computer program, wherein the computer program is arranged to perform the steps of any of the method embodiments described above when run.
According to a further embodiment of the invention, there is also provided an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
According to the method and the device, the target area in the first video picture is firstly determined, the target activity path of the first object in the first video picture is determined under the condition that the first object is detected to enter and/or leave the target area, privacy processing is carried out on the target activity path in the first video picture to obtain the second video picture which does not comprise the target activity path, the target activity path of the first object is determined under the condition that the first object is detected to enter and/or leave the target area, privacy processing is carried out on the target activity path in the first video to obtain the second video which does not comprise the target activity path, and privacy processing on the target activity path is achieved, so that the problems that privacy information in video is blocked insufficiently and privacy information is blocked with lower accuracy in the prior art are solved, and the effect of improving the accuracy of privacy information blocking in video is achieved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
fig. 1 is a block diagram of a mobile terminal hardware structure of a video privacy processing method according to an embodiment of the present application;
FIG. 2 is a flow chart of a method for privacy processing of video in accordance with an embodiment of the present application;
FIG. 3 is a schematic diagram of an alternative privacy zone in video in accordance with an embodiment of the present application;
FIG. 4 is a schematic diagram of an alternate in-video early warning dashed box according to an embodiment of the present application;
FIG. 5 is a schematic diagram of an alternative in-video target activity path in accordance with an embodiment of the application;
FIG. 6 is a schematic diagram of an alternative in-video target activity path according to an embodiment of the application;
FIG. 7 is a schematic diagram of an alternative in-video target activity path according to an embodiment of the application;
fig. 8 is a block diagram of a structure of a video privacy processing apparatus according to an embodiment of the present application.
Detailed Description
The application will be described in detail hereinafter with reference to the drawings in conjunction with embodiments. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.
The method according to the first embodiment of the present application may be implemented in a mobile terminal, a computer terminal or a similar computing device. Taking the mobile terminal as an example, fig. 1 is a block diagram of a hardware structure of the mobile terminal of a video privacy processing method according to an embodiment of the present application. As shown in fig. 1, the mobile terminal may include one or more (only one is shown in fig. 1) processors 102 (the processors 102 may include, but are not limited to, a microprocessor MCU or a processing device such as a programmable logic device FPGA) and a memory 104 for storing data, and optionally, a transmission device 106 for communication functions and an input-output device 108. It will be appreciated by those skilled in the art that the structure shown in fig. 1 is merely illustrative and not limiting of the structure of the mobile terminal described above. For example, the mobile terminal may also include more or fewer components than shown in fig. 1, or have a different configuration than shown in fig. 1.
The memory 104 may be used to store a computer program, for example, a software program of application software and a module, such as a computer program corresponding to a video privacy processing method in an embodiment of the present invention, and the processor 102 executes the computer program stored in the memory 104 to perform various functional applications and data processing, that is, implement the above-mentioned method. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, which may be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, simply referred to as NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is configured to communicate with the internet wirelessly.
In this embodiment, a method for processing privacy of a video running on the mobile terminal is provided, and fig. 2 is a flowchart of a method for processing privacy of a video according to an embodiment of the present invention, as shown in fig. 2, where the flowchart includes the following steps:
step S202, determining a target area in a first video picture, wherein the target area comprises an area which is preconfigured as a privacy type;
the first video frame may be a video frame captured by the network camera IPC. The target region may be a privacy region in the first video picture that needs to be occluded.
Step S204, under the condition that the behavior of a first object entering and/or leaving the target area is detected, determining a target active path of the first object in the first video picture, and performing privacy processing on the target active path in the first video picture to obtain a second video picture which does not contain the target active path;
the first object may be a person or other objects that may be moved, and the target area may be a privacy area that needs to be blocked, for example, may be a privacy area as in fig. 3, where, when the first object is detected to enter and/or leave the privacy area, a target activity path of the first object in the whole video frame is determined, and privacy processing is performed on the target activity path in the first video frame to obtain a second video frame that does not include the target activity path, where, the privacy processing may be any manner of eliminating the target activity path in the first video frame, and a manner of privacy processing is not limited.
Through the steps, the target area in the first video picture is firstly determined, the target activity path of the first object in the first video picture is determined under the condition that the first object is detected to enter and/or leave the target area, privacy processing is carried out on the target activity path in the first video picture to obtain the second video picture which does not comprise the target activity path, the target activity path of the first object is determined under the condition that the first object is detected to enter and/or leave the target area, privacy processing is carried out on the target activity path in the first video to obtain the second video which does not comprise the target activity path, and privacy processing on the target activity path is realized, so that the problems of insufficient privacy information shielding in videos and lower accuracy of privacy information shielding in the prior art can be solved, and the effect of improving the accuracy of privacy information shielding in videos is achieved.
Alternatively, the execution subject of the above steps may be a terminal or the like, but is not limited thereto.
Optionally, in case a first object is detected to have a behavior of entering and/or exiting the target area, determining a target activity path of the first object in the first video frame is preceded by the method further comprising: acquiring a target early warning frame in the first video frame, wherein the video frame selected by the target early warning frame comprises the video frame of the target area; marking a privacy processing pre-mark for the first object under the condition that the first object enters the target early warning frame, wherein the privacy processing pre-mark is used for marking the behavior of the first object to enter the target area; modifying the first object's mark from the privacy handling pre-mark to a privacy mark in case the first object has been marked with the privacy handling pre-mark and the first object is detected to enter the target area, wherein the privacy mark is used for identifying that the first object has occurred to enter the target area; in the event that the first object has been marked with the privacy processing pre-marker and it is detected that the first object has not entered the target area and has left the target pre-warning frame, the privacy processing pre-marker for the first object is deleted after a first time interval in which the first object is determined to leave the target pre-warning frame.
As an alternative implementation manner, a dashed box may be drawn on the target area inside the system, and as a target early warning box, for example, an early warning dashed box as shown in fig. 4 may be used to indicate an early warning operation when an external object enters the shielding area. It should be noted that, the user viewing the video does not see the target early warning frame. After a target early warning frame drawn in advance in a first video picture is obtained, whether an object enters the target early warning frame is detected, a privacy processing pre-mark is marked on a first object under the condition that the first object enters the target early warning frame, feature recognition and analysis are started on the first object, and the mark of the first object is modified from the privacy processing pre-mark to a privacy mark under the condition that the first object is marked with the privacy processing pre-mark and the first object enters a target area is detected.
In the case that the first object has been marked with the privacy processing pre-marking, there may be a case that the first object gradually leaves the target pre-warning frame without entering the target area, at this time, a first time interval may be set, for example, may be 10 seconds, 20 seconds or 30 seconds, where the first time interval is not limited, and assuming that the first time interval is 30 seconds, the privacy processing pre-marking of the first object may be deleted after the first object is detected to have not entered the target area and has left the target pre-warning frame for 30 seconds, and it should be noted that after the first object leaves the target pre-warning frame, there may be a case of returning to enter the target pre-warning frame or enter the target area, so that a first time interval may be set, and the privacy processing marking of the first object and the feature identification information of the first object may be retained, so that in the case that the first object has a behavior of returning to enter the target area, the first object may be directly modified from the privacy processing pre-marking to the privacy processing pre-marking.
Optionally, privacy processing of the target activity path in the first video frame includes: and under the condition that the first object exists in the first video frame included in the first video picture, replacing the first video frame with a target video frame shot before the first video frame, wherein the target video frame is a video frame without any object marked with the privacy mark.
As an alternative embodiment, in the case that the first object is detected to have the behavior of entering and/or leaving the target area, it is indicated that the first object is an object that is active in the privacy area and needs to be subjected to privacy processing, where a target active path of the first object in the first video frame needs to be determined. When it is detected that the first object to be privacy-processed exists in the first video frame included in the first video frame, the first video frame needs to be replaced by the target video frame shot before the first video frame, for example, the first video frame may be replaced by a video frame before one second or a video frame before two seconds, it should be noted that, assuming that the video frame before one second has other objects to be privacy-processed and marked with privacy marks, the previous video frame that is closest to the object to be privacy-processed and does not exist on the mark is continuously searched for to be replaced, and the time position of the selected replaced video frame is not limited.
Optionally, in a case where the first object is detected to enter the target area, after modifying the tag of the first object from the privacy processing pre-tag to a privacy tag, the method further includes: and analyzing all path information of the first object in the first video picture, and determining the target activity path comprising all path information.
As an optional implementation manner, in the case that the first object is detected to enter the target area, modifying the mark of the first object from the privacy processing pre-mark to the privacy mark, which indicates that the first object is an object needing privacy processing, at this time, all path information of the first object in the first video frame needs to be analyzed, and a target activity path including path information of all the first objects is determined.
Optionally, the method further comprises: adding an object entering the first video picture to an object feature list in case that the object is detected to exist, wherein the object feature list is used for recording information of all objects identified in the first video picture; detecting whether a second object which is not marked with the privacy processing pre-mark or the privacy mark exists in the object feature list at intervals of a preset second time interval; and deleting the information of the second object recorded in the object feature list.
As an alternative embodiment, the preset second time interval may be 10 seconds, 30 seconds or 2 minutes, where the preset second time interval is not limited. Assuming that the preset second time interval is 30 seconds, adding the object to an object feature list when detecting that the object entering the first video picture exists, detecting whether a second object which is not marked with the privacy processing pre-mark or the privacy mark exists in the object feature list every 30 seconds, deleting information of the second object recorded in the object feature list when the second object exists, and cleaning the object information which does not need to be subjected to the privacy processing at regular time, so that occupation of a storage memory can be reduced.
Optionally, in case a first object is detected to have a behavior of entering and/or exiting the target area, determining a target activity path of the first object in the first video frame, the method comprising: in the case that the first object is detected to have a behavior of leaving the target area and a first active path of the stored first object before entering the target area, which is generated in the first video picture, a second active path of the first object after leaving the target area is associated with the first active path, so as to obtain the target active path comprising the first active path and the second active path; determining the second activity path of the first object after leaving the target area as the target activity path in the case that the first object is detected to have the action of leaving the target area and no stored first activity path of the first object before entering the target area, which is generated in the first video picture; and determining a first active path of the first object before entering the target area, which is generated in the first video picture, as the target active path under the condition that the first object is detected to have the action of entering the target area.
As an alternative embodiment, there are three cases where the first object that needs to perform privacy processing in the first video frame is the first object that enters and leaves the target area, the first object that leaves only the target area (in this case, the object may be an object that is always in the target area or may be an object that is missed when entering the target area), the third object that enters only the target area, and the following describes the manner of acquiring the target activity path of the first object in the above three cases:
scene one: when the behavior of the first object leaving the target area is detected, whether the related information of the first object exists or not is needed to be checked in the stored object information, and under the condition that the stored first active path of the first object before entering the target area generated in the first video picture exists, a second active path of the first object after leaving the target area is stored in a correlated mode with the first active path, so that a complete target active path comprising the first active path and the second active path is obtained.
Scene II: when the first object is detected to have the action of leaving the target area, whether the related information of the first object exists or not needs to be checked in the stored object information, and when the first active path generated by the stored first object in the first video picture before entering the target area is determined to be absent, only the second active path after the first object leaves the target area needs to be determined to be the target active path.
Scene III: in the case that the first object is detected to have the action of entering the target area, the first active path of the first object before entering the target area, which is generated in the first video picture, is determined as a target active path.
Optionally, the method further comprises: returning the second video picture to the user with the first authority under the condition that a video picture viewing request initiated by the user with the first authority is received, wherein the first authority is the authority which does not have the function of viewing the target activity path; and/or the number of the groups of groups,
and returning the first video picture to the user with the second authority under the condition that a video picture viewing request initiated by the user with the second authority is received, wherein the second authority is the authority with the function of viewing the target activity path.
As an alternative embodiment, the rights of the user may include a first right and a second right, where the first right may indicate that the user does not have the right to view the target activity path, and the second right may indicate that the user does have the right to view the target activity path. Returning a second video picture which does not contain the target activity path to the user with the first authority under the condition that a video picture viewing request initiated by the user with the first authority is received, so as to realize privacy processing of the target activity path of the first object; under the condition that a video picture viewing request initiated by a user with the second authority is received, returning the original first video picture which is not subjected to privacy treatment to the user with the second authority, and facilitating management and viewing of the user with the second authority.
As an alternative embodiment, the privacy zone mask is actually an occlusion of the active behavior of the target object within the zone. Shown in fig. 3 is a privacy zone in a first video picture. Therefore, privacy protection in a non-shielding area is also necessary for the target object entering the area, and privacy, such as identity, dressing, activity path in the non-shielding area, and the like, of the target object in the privacy area is fully ensured. The method for realizing the privacy of the movable target object comprises the following steps:
and step 1, carrying out privacy area picture frames on the area to represent privacy shielding areas.
And 2, drawing a dotted line frame for the area in the system to represent early warning operation when the external target object enters the shielding area. The preprocessing dotted line is not visible to the user, and as shown in fig. 4, an optional schematic diagram of the early warning dotted line box in the video according to an embodiment of the present invention is shown.
And 3, when the target enters the picture, starting tracking snapshot of the target object, and selecting the best face picture for characteristic analysis. If the face cannot be captured, capturing the target, performing feature analysis, recording the moving track of the target, and recording the moving track into an object feature list. When the target approaches the dotted line box, the target is represented with a high probability to enter the area, and privacy processing is started on the target. The target object is pre-marked for privacy processing.
And 4, if the target does not enter the privacy zone and gradually gets away from the zone, canceling the privacy treatment pre-marking of the object.
And step 5, if the target enters the privacy zone, changing the preprocessing mark into the target privacy mark of the zone. And analyzing all the moving paths of the target in the picture, and outputting a moving path diagram of the target object. A target object activity path diagram as shown in fig. 5.
And step 6, if the target subsequently leaves the privacy zone, identifying the object characteristics and searching the object characteristic list in the step 3. If the object is not in the object feature list, the object may enter the area through a path of the object, query other devices in the system, and if the other devices do not have the object, the object may have operations such as changing the object in the privacy area. The target object is directly added to the object feature list in step 3 and recorded as a target only. The object is then privacy-masked. If there are multiple privacy zones, the object privacy coverage is performed all the way from one zone to another. The object feature list maintains target objects that enter into privacy zone activity.
And 7, starting the target entering area, and setting the live broadcast and video playback permission of the object. The user of unauthorized authority cannot view the target object. That is, the target object may be privacy-processed or hidden-processed when the user views the video. The privacy process is a mosaic filling or blurring process for the occlusion region. In order to achieve the aim of fully hiding, the video index of the target object is marked as a privacy object, and the playing flow can replace frame data of the object at a certain position with video before one second, so that the moving path of the target object in a non-shielding area is replaced by frames without the object in the last second. If there are other targets in the previous second, the nearest non-target frame is found to replace. Thus, the video viewed by an unauthorized user is free of any target, and the target object can be fully enabled to achieve the effect of privacy treatment. As shown in fig. 4. An unauthorized user sees a non-targeted picture. The recording process does not modify the original target recording data, but only when the user downloads or views playback, the recording process module sends processed data to the requesting user. An unauthorized user is unable to obtain the original data of the target object in the non-occluded area.
Step 8, if there are multiple shielding areas, when the target object is from one area to another, the user with authorized authority sees the effect as shown in fig. 6. An object not authorized for the right cannot view the privacy object as shown in fig. 7.
And 9, a target object privacy processing module. When the user checks or downloads the video, the user checks whether the privacy object list has the target object or not, and if so, the user combines the target activity path in the time period in the step 5. The user processes the original video with the preset privacy authority when downloading the data. In the non-privacy zone, only users authorized to view the privacy object may view. If the user has no authority, the non-privacy area sees a picture without a target object.
From the description of the above embodiments, it will be clear to a person skilled in the art that the method according to the above embodiments may be implemented by means of software plus the necessary general hardware platform, but of course also by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The embodiment also provides a device for privacy processing of video, which is used for implementing the above embodiment and the preferred implementation, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
Fig. 8 is a block diagram of a privacy processing apparatus of a video according to an embodiment of the present invention, as shown in fig. 8, the apparatus includes a determining module 82 configured to determine a target area in a first video frame, wherein the target area includes an area configured as a privacy type in advance; the processing module 84 is configured to determine a target active path of the first object in the first video frame when the first object is detected to have a behavior of entering and/or exiting the target area, and perform privacy processing on the target active path in the first video frame, so as to obtain a second video frame that does not include the target active path.
Optionally, the apparatus further includes: the first acquisition module is used for acquiring a target early warning frame in the first video frame before determining a target activity path of a first object in the first video frame under the condition that the first object is detected to enter and/or leave the target area, wherein the video frame selected by the target early warning frame comprises the video frame in which the target area is positioned; the marking module is used for marking the first object with a privacy processing pre-marking when the first object enters the target early warning frame, wherein the privacy processing pre-marking is used for marking the first object to enter the target area; a modifying module, configured to modify, when the first object has been marked with the privacy processing pre-marker and it is detected that the first object enters the target area, the marking of the first object from the privacy processing pre-marker to a privacy marker, where the privacy marker is used to identify that the first object has occurred to enter the target area; and the first deleting module is used for deleting the privacy processing pre-mark of the first object after determining a first time interval when the first object leaves the target early warning frame under the condition that the first object is marked with the privacy processing pre-mark and the first object is detected not to enter the target area and leaves the target early warning frame.
The processing module 84 includes: and the replacing module is used for replacing the first video frame with a target video frame shot before the first video frame under the condition that the first object exists in the first video frame included in the first video picture, wherein the target video frame is a video frame without any object marked with the privacy mark.
The device is further used for analyzing all path information of the first object in the first video picture after the mark of the first object is modified from the privacy processing pre-mark to the privacy mark under the condition that the first object is detected to enter the target area, and determining the target activity path comprising all path information.
The device further comprises: an adding module, configured to add an object entering the first video frame to an object feature list when the object is detected to exist, where the object feature list is used to record information of all objects identified in the first video frame; the detection module is used for detecting whether a second object which is not marked with the privacy processing pre-mark or the privacy mark exists in the object feature list at intervals of a preset second time interval; and the second deleting module is used for deleting the information of the second object recorded in the object feature list when the second object exists.
The processing module 84 further includes: the first detection unit is used for storing a second active path of the first object after leaving the target area and the first active path in a correlated way under the condition that the first object is detected to leave the target area and a stored first active path of the first object before entering the target area, which is generated in the first video picture, is stored, so that the target active path comprising the first active path and the second active path is obtained; a second detection unit configured to determine, as the target active path, the second active path after the first object leaves the target area, in a case where it is detected that the first object has a behavior of leaving the target area, and there is no first active path of the first object that has been stored before entering the target area, the first active path being generated in the first video frame; and a third detection unit, configured to determine, as the target active path, a first active path generated by the first object in the first video frame before entering the target area, in a case where it is detected that the first object has a behavior of entering the target area.
The device further comprises: the method comprises the steps of carrying out a first treatment on the surface of the The first receiving module is used for returning the second video picture to the user with the first authority under the condition that a video picture viewing request initiated by the user with the first authority is received, wherein the first authority is the authority which does not have the authority to view the target activity path; and/or the second receiving module is used for returning the first video picture to the user with the second authority under the condition that the video picture viewing request initiated by the user with the second authority is received, wherein the second authority is the authority for viewing the target activity path.
It should be noted that each of the above modules may be implemented by software or hardware, and for the latter, it may be implemented by, but not limited to: the modules are all located in the same processor; alternatively, the above modules may be located in different processors in any combination.
An embodiment of the invention also provides a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the method embodiments described above when run.
Alternatively, in the present embodiment, the above-described storage medium may be configured to store a computer program for performing the steps of:
S1, determining a target area in a first video picture, wherein the target area comprises an area which is preconfigured as a privacy type;
s2, under the condition that the behavior of the first object entering and/or leaving the target area is detected, determining a target active path of the first object in the first video picture, and performing privacy processing on the target active path in the first video picture to obtain a second video picture which does not contain the target active path.
Alternatively, in the present embodiment, the storage medium may include, but is not limited to: a usb disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing a computer program.
An embodiment of the invention also provides an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, where the transmission device is connected to the processor, and the input/output device is connected to the processor.
Alternatively, in the present embodiment, the above-described processor may be configured to execute the following steps by a computer program:
s1, determining a target area in a first video picture, wherein the target area comprises an area which is preconfigured as a privacy type;
s2, under the condition that the behavior of the first object entering and/or leaving the target area is detected, determining a target active path of the first object in the first video picture, and performing privacy processing on the target active path in the first video picture to obtain a second video picture which does not contain the target active path.
Alternatively, specific examples in this embodiment may refer to examples described in the foregoing embodiments and optional implementations, and this embodiment is not described herein.
It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a memory device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. A method for privacy processing of video, comprising:
determining a target area in a first video picture, wherein the target area comprises an area pre-configured as a privacy type;
under the condition that the behavior of a first object entering and/or leaving the target area is detected, determining a target active path of the first object in the first video picture, and carrying out privacy processing on the target active path in the first video picture to obtain a second video picture which does not contain the target active path;
wherein in case the detecting that a first object has an action of entering and/or leaving the target area, determining a target activity path of the first object in the first video frame is preceded by the method further comprising: acquiring a target early warning frame in the first video frame, wherein the video frame selected by the target early warning frame comprises the video frame of the target area; marking a privacy processing pre-mark for the first object under the condition that the first object enters the target early warning frame, wherein the privacy processing pre-mark is used for marking the behavior of the first object to enter the target area; modifying the first object's mark from the privacy handling pre-mark to a privacy mark in case the first object has been marked with the privacy handling pre-mark and the first object is detected to enter the target area, wherein the privacy mark is used for identifying that the first object has occurred to enter the target area; in the event that the first object has been marked with the privacy processing pre-marker and it is detected that the first object has not entered the target area and has left the target pre-warning frame, the privacy processing pre-marker for the first object is deleted after a first time interval in which the first object is determined to leave the target pre-warning frame.
2. The method of claim 1, wherein privacy processing the target activity path in the first video picture comprises:
and under the condition that the first object exists in the first video frame included in the first video picture, replacing the first video frame with a target video frame shot before the first video frame, wherein the target video frame is a video frame without any object marked with the privacy mark.
3. The method of claim 1, wherein upon the detecting that the first object enters the target area, modifying the tag of the first object from the privacy handling pre-tag to a privacy tag, the method further comprises:
and analyzing all path information of the first object in the first video picture, and determining the target activity path comprising all path information.
4. The method according to claim 1, wherein the method further comprises:
adding an object entering the first video picture to an object feature list in case that the object is detected to exist, wherein the object feature list is used for recording information of all objects identified in the first video picture;
Detecting whether a second object which is not marked with the privacy processing pre-mark or the privacy mark exists in the object feature list at intervals of a preset second time interval;
and deleting the information of the second object recorded in the object feature list when the second object exists.
5. The method of claim 1, wherein the determining a target activity path of the first object in the first video frame if a first object is detected to have an activity to enter and/or leave the target area comprises:
in the case that the first object is detected to have a behavior of leaving the target area and a first active path of the stored first object before entering the target area, which is generated in the first video picture, a second active path of the first object after leaving the target area is associated with the first active path, so as to obtain the target active path comprising the first active path and the second active path;
determining the second activity path of the first object after leaving the target area as the target activity path in the case that the first object is detected to have the action of leaving the target area and no stored first activity path of the first object before entering the target area, which is generated in the first video picture;
And determining a first active path of the first object before entering the target area, which is generated in the first video picture, as the target active path under the condition that the first object is detected to have the action of entering the target area.
6. The method according to claim 1, wherein the method further comprises:
returning the second video picture to the user with the first authority under the condition that a video picture viewing request initiated by the user with the first authority is received, wherein the first authority is the authority which does not have the function of viewing the target activity path; and/or the number of the groups of groups,
and returning the first video picture to the user with the second authority under the condition that a video picture viewing request initiated by the user with the second authority is received, wherein the second authority is the authority with the function of viewing the target activity path.
7. A privacy processing apparatus for video, comprising:
a determining module configured to determine a target area in a first video picture, wherein the target area includes an area configured in advance as a privacy type;
the processing module is used for determining a target active path of a first object in the first video picture under the condition that the first object is detected to enter and/or leave the target area, and carrying out privacy processing on the target active path in the first video picture to obtain a second video picture which does not contain the target active path;
Wherein the apparatus further comprises: the first acquisition module is used for acquiring a target early warning frame in the first video frame before determining a target activity path of a first object in the first video frame under the condition that the first object is detected to enter and/or leave the target area, wherein the video frame selected by the target early warning frame comprises the video frame in which the target area is positioned; the marking module is used for marking the first object with a privacy processing pre-marking when the first object enters the target early warning frame, wherein the privacy processing pre-marking is used for marking the first object to enter the target area; a modifying module, configured to modify, when the first object has been marked with the privacy processing pre-marker and it is detected that the first object enters the target area, the marking of the first object from the privacy processing pre-marker to a privacy marker, where the privacy marker is used to identify that the first object has occurred to enter the target area; and the first deleting module is used for deleting the privacy processing pre-mark of the first object after determining a first time interval when the first object leaves the target early warning frame under the condition that the first object is marked with the privacy processing pre-mark and the first object is detected not to enter the target area and leaves the target early warning frame.
8. A storage medium having stored therein a computer program, wherein the program is executable by a terminal device or a computer to perform the method of any of claims 1 to 6.
9. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to run the computer program to perform the method of any of the claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211001465.XA CN115379260B (en) | 2022-08-19 | 2022-08-19 | Video privacy processing method and device, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211001465.XA CN115379260B (en) | 2022-08-19 | 2022-08-19 | Video privacy processing method and device, storage medium and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115379260A CN115379260A (en) | 2022-11-22 |
| CN115379260B true CN115379260B (en) | 2023-11-03 |
Family
ID=84064861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211001465.XA Active CN115379260B (en) | 2022-08-19 | 2022-08-19 | Video privacy processing method and device, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115379260B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008027464A2 (en) * | 2006-08-30 | 2008-03-06 | The Directv Group, Inc. | Mosaic channel video stream with interactive services |
| CN103747217A (en) * | 2014-01-26 | 2014-04-23 | 国家电网公司 | Video monitoring method and device |
| CN105007395A (en) * | 2015-07-22 | 2015-10-28 | 深圳市万姓宗祠网络科技股份有限公司 | Privacy processing method for continuously recording video |
| CN106228218A (en) * | 2016-08-15 | 2016-12-14 | 深圳市校联宝科技有限公司 | The intelligent control method of a kind of destination object based on movement and system |
| CN108234918A (en) * | 2016-12-22 | 2018-06-29 | Tcl集团股份有限公司 | The exploration of indoor unmanned plane with privacy aware and communication architecture method and system |
| CN110363172A (en) * | 2019-07-22 | 2019-10-22 | 曲靖正则软件开发有限公司 | A kind of method for processing video frequency, device, electronic equipment and readable storage medium storing program for executing |
| CN110942027A (en) * | 2019-11-26 | 2020-03-31 | 浙江大华技术股份有限公司 | Method and device for determining occlusion strategy, storage medium and electronic device |
| CN112037127A (en) * | 2020-07-27 | 2020-12-04 | 浙江大华技术股份有限公司 | Privacy shielding method and device for video monitoring, storage medium and electronic device |
| CN112906047A (en) * | 2021-02-04 | 2021-06-04 | 陕西师范大学 | Image privacy information protection system and method based on deep learning |
| CN113890991A (en) * | 2021-09-08 | 2022-01-04 | 浙江大华技术股份有限公司 | Privacy protection method and device applied to high-altitude parabolic detection |
| CN113887445A (en) * | 2021-10-08 | 2022-01-04 | 山东可信云信息技术研究院 | Method and system for identifying standing and loitering behaviors in video |
| CN114025173A (en) * | 2021-11-17 | 2022-02-08 | 浙江大华技术股份有限公司 | Image processing method, terminal and computer readable storage medium |
| CN114339367A (en) * | 2021-12-29 | 2022-04-12 | 杭州海康威视数字技术股份有限公司 | Video frame processing method, device and equipment |
| CN114742856A (en) * | 2022-04-08 | 2022-07-12 | 北京字跳网络技术有限公司 | Video processing method, device, equipment and medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070291985A1 (en) * | 2006-06-20 | 2007-12-20 | Nils Oliver Krahnstoever | Intelligent railyard monitoring system |
-
2022
- 2022-08-19 CN CN202211001465.XA patent/CN115379260B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008027464A2 (en) * | 2006-08-30 | 2008-03-06 | The Directv Group, Inc. | Mosaic channel video stream with interactive services |
| CN103747217A (en) * | 2014-01-26 | 2014-04-23 | 国家电网公司 | Video monitoring method and device |
| CN105007395A (en) * | 2015-07-22 | 2015-10-28 | 深圳市万姓宗祠网络科技股份有限公司 | Privacy processing method for continuously recording video |
| CN106228218A (en) * | 2016-08-15 | 2016-12-14 | 深圳市校联宝科技有限公司 | The intelligent control method of a kind of destination object based on movement and system |
| CN108234918A (en) * | 2016-12-22 | 2018-06-29 | Tcl集团股份有限公司 | The exploration of indoor unmanned plane with privacy aware and communication architecture method and system |
| CN110363172A (en) * | 2019-07-22 | 2019-10-22 | 曲靖正则软件开发有限公司 | A kind of method for processing video frequency, device, electronic equipment and readable storage medium storing program for executing |
| CN110942027A (en) * | 2019-11-26 | 2020-03-31 | 浙江大华技术股份有限公司 | Method and device for determining occlusion strategy, storage medium and electronic device |
| CN112037127A (en) * | 2020-07-27 | 2020-12-04 | 浙江大华技术股份有限公司 | Privacy shielding method and device for video monitoring, storage medium and electronic device |
| CN112906047A (en) * | 2021-02-04 | 2021-06-04 | 陕西师范大学 | Image privacy information protection system and method based on deep learning |
| CN113890991A (en) * | 2021-09-08 | 2022-01-04 | 浙江大华技术股份有限公司 | Privacy protection method and device applied to high-altitude parabolic detection |
| CN113887445A (en) * | 2021-10-08 | 2022-01-04 | 山东可信云信息技术研究院 | Method and system for identifying standing and loitering behaviors in video |
| CN114025173A (en) * | 2021-11-17 | 2022-02-08 | 浙江大华技术股份有限公司 | Image processing method, terminal and computer readable storage medium |
| CN114339367A (en) * | 2021-12-29 | 2022-04-12 | 杭州海康威视数字技术股份有限公司 | Video frame processing method, device and equipment |
| CN114742856A (en) * | 2022-04-08 | 2022-07-12 | 北京字跳网络技术有限公司 | Video processing method, device, equipment and medium |
Non-Patent Citations (3)
| Title |
|---|
| People detection and tracking in high resolution panoramic video mosaic;R. Patil;2004 IEEE/RSJ International Conference on Intelligent Robots and Systems;全文 * |
| 博物馆监控视频中慢速移动稀疏目标异常轨迹检测;刘速;《科学与技术工程》;第18卷(第22期);全文 * |
| 海事场景的视频监控系统;洪晶;中国优秀硕士学位论文全文数据库;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115379260A (en) | 2022-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10037451B2 (en) | Methods, computer program products, and systems for providing automated video tracking via radio frequency identification | |
| CN111010547A (en) | Target object tracking method and device, storage medium and electronic device | |
| TW201722136A (en) | Security system and method | |
| CN110705477A (en) | Behavior analysis method and apparatus, electronic device, and computer storage medium | |
| US11250251B2 (en) | Method for identifying potential associates of at least one target person, and an identification device | |
| KR20160008267A (en) | User's behavior analysis system on the network based video surveillance system | |
| CN111291682A (en) | Method and device for determining target object, storage medium and electronic device | |
| CN113378616A (en) | Video analysis method, video analysis management method and related equipment | |
| CN110705476A (en) | Data analysis method and device, electronic equipment and computer storage medium | |
| US20150242980A1 (en) | Processes to Enable Individuals to Opt Out (or be Opted Out) of Various Facial Recognition and other Schemes and Enable Businesses and other Entities to Comply with Such Decisions | |
| CN111400745A (en) | Picture management method and device and electronic equipment | |
| WO2018031106A1 (en) | Methods and systems of updating motion models for object trackers in video analytics | |
| CN115379260B (en) | Video privacy processing method and device, storage medium and electronic device | |
| CN104935478A (en) | Intelligent terminal depth perception method and system thereof | |
| KR101695655B1 (en) | Method and apparatus for analyzing video and image | |
| WO2020050003A1 (en) | Method, identification device and non-transitory computer readable medium for multi-layer potential associates discovery | |
| US11044277B2 (en) | System and method for identifying imaging devices | |
| CN111432286A (en) | Video processing method, device and system | |
| CN107633180B (en) | Data query method and system of public security system | |
| CN111243289A (en) | Target vehicle tracking method and device, storage medium and electronic device | |
| CN113727070B (en) | Equipment resource management method and device, electronic equipment and storage medium | |
| JP7276516B2 (en) | Method, device and program | |
| EP3065024A1 (en) | Wearable camera apparatus with selectable processing of image content | |
| CN114038043A (en) | Personnel detection method based on intelligent lock equipment, intelligent lock equipment and storage medium | |
| CN116012237A (en) | Method and device for determining image shielding area, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |