Background
In recent years, the level of urbanization in China is continuously improved, the scale of urban communities is continuously enlarged, people, vehicles and objects are highly concentrated, the security situation is gradually complicated, and the work of social security management, illegal criminal activities attack and the like is heavily stressed. In a public security case with large human input, the manual investigation is like a sea fishing needle in the face of massive image data, so that the time and labor are wasted, the efficiency is low, and the cost is high. Under the circumstances, how to improve the social security management level and attack the illegal criminal activities by using advanced technological means becomes a problem to be solved urgently at present.
The intelligent security is also called intelligent security. The intelligent security technology mainly comprises an artificial intelligence algorithm, high-performance calculation, distributed calculation and storage, large-scale operation and maintenance and the like. The data sharing of the intelligent security system based on big data is very important for realizing high-performance intelligent security, and the key difficulty is how to meet the dual requirements on performance and precision in data sharing.
Disclosure of Invention
In order to solve the problems in the related art, the application provides a data sharing method of the intelligent security system based on big data.
According to the embodiment of the application, a data sharing method of an intelligent security system based on big data is provided, and the method is characterized by comprising the following steps:
receiving monitoring data uploaded by each security monitoring device;
partitioning the monitoring data into blocks;
deleting the repeated data blocks;
establishing an index for the deduplicated data blocks;
and utilizing the index to provide retrieval service for the monitoring data.
Preferably, the blocking the monitoring data includes:
creating a block flag with a bit operation;
matching the block mark with the monitoring data;
and partitioning the matched position.
Preferably, creating the block flag with the bit operation comprises:
constructing a mark character string S ═ [ b1, b2, b3, … bm ], wherein bi is a binary string of each character in the mark character string, and the number of binary digits is n;
the data fingerprint f (Si) is set to be the combination of the i-th bits of b1, b2, b3, … bm, where i takes the 1-n-th bits of the binary string.
Preferably, matching the block flag with the data comprises:
a division into a jth data block BBj of the monitored data;
step 1: skipping k characters to the right from the Bj-1 end mark;
step 2: aligning the current bit to S left;
and step 3: fetching m characters to the left from the last position of the data aligned with S;
and 4, step 4: taking the ith bit from the m characters to form Xn, and comparing Xn with f (Si) one by one, wherein i is from 1 to n.
Preferably, matching the block flag with the data further comprises:
step 6: if i from 1 to n, all satisfy Xn ═ f (si), the last bit in the data that is currently aligned with S is marked as the end bit of Mj.
Preferably, matching the block flag with the data further comprises:
and 7: if the two characters are different, continuing to skip k characters to the right, and returning to the step 2;
and 8: returning to step 2 for L times, the last bit in the data currently aligned with S is marked as the end bit of Bj.
Preferably, matching the block flag with the monitoring data comprises:
entering the division of the jth data block Bj of the monitoring data;
step 1: skipping k characters to the right from the Bj-1 end mark;
step 2: aligning the current bit to S left;
and step 3: taking r characters from the first position of the data after being aligned with S to the right, wherein r is far smaller than m;
and 4, step 4: taking the ith component Xir from the r characters taken, and then
Step 4.1: shifting the head position of Xir to the right to obtain r positions f (sir), and comparing Xir with f (sir) one by one, wherein i is from 1 to n;
step 4.2: if not, executing SHLf (Sir),1, and then returning to the step 4.1;
step 4.3: if the step 4.1 is returned to m-r times, returning to the step 2;
step 4.3: if i from 1 to n, all satisfy Xir ═ f (sir), then step 5 is entered.
Preferably, matching the block flag with the monitoring data further comprises:
and 5: fetching m characters to the left from the last position of the data aligned with S;
step 6: taking the ith bit from the m characters to form Xi, and comparing Xi with f (Si) one by one, wherein i is from 1 to n.
Preferably, matching the block flag with the monitoring data further comprises:
and 7: if i from 1 to n, all satisfy Xi ═ f (si), the last bit in the data that is currently aligned with S is marked as the end bit of Bj.
Preferably, matching the block flag with the monitoring data further comprises:
and 8: if the two characters are different, continuing to skip k characters to the right, and returning to the step 2;
and step 9: returning to step 2 for L times, the last bit in the data currently aligned with S is marked as the end bit of Bj.
The technical scheme provided by the embodiment of the application can have the following beneficial effects: the invention provides an efficient data sharing method of an intelligent security system based on big data, which is used for creating indexes for monitoring data through retrieval and matching, thereby efficiently providing services for future data retrieval and meeting the dual requirements on performance and precision during data sharing.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
The following disclosure provides many different embodiments, or examples, for implementing different features of the application. In order to simplify the disclosure of the present application, specific example components and arrangements are described below. Of course, they are merely examples and are not intended to limit the present application. Further, the present application may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. Further, examples of various specific processes and materials are provided herein, but one of ordinary skill in the art may recognize the applicability of other processes and/or the use of other materials. In addition, the structure of a first feature described below as "on" a second feature may include embodiments in which the first and second features are formed in direct contact, and may also include embodiments in which additional features are formed between the first and second features, such that the first and second features may not be in direct contact.
In the description of the present application, it should be noted that, unless otherwise specified and limited, the terms "mounted," "connected," and "connected" are to be interpreted broadly, and may be, for example, a mechanical connection or an electrical connection, a communication between two elements, a direct connection, or an indirect connection via an intermediate medium, and specific meanings of the terms may be understood by those skilled in the art according to specific situations.
Fig. 1 is a flowchart illustrating a data sharing method for a smart security system based on big data according to an exemplary embodiment. Referring to fig. 1, it includes:
step S10, receiving monitoring data uploaded by each security monitoring device;
step S20, establishing an index for the monitoring data;
step S30, providing a search service for the monitoring data.
The embodiment provides an efficient data sharing method of the intelligent security system based on big data, and the index is created for the monitored data, so that the service is efficiently provided for future data retrieval, and the dual requirements on performance and precision during data sharing are met.
Preferably, the indexing of the monitoring data comprises:
determining a background and a target in the monitoring data;
performing background matching on the determined background in a pre-established background library, and performing target matching on the determined target in a pre-established target library;
and creating an index of the monitoring data by using the matching result.
The embodiment notes that the monitoring data has the difference between the background and the target, and divides the monitoring data, thereby remarkably reducing the calculation amount of matching retrieval.
For example, video surveillance is a still picture most of the time except when there is occasional human activity. Preferably, the still image is determined as the background and the moving image is determined as the object by analyzing the still image and the moving image.
For example, in audio monitoring, ambient noise is most of the time, except for the accidental occurrence of human voice or the impact sound of a particular object. Preferably, the ambient noise is determined as the background and the human voice or the specific object collision sound is determined as the target by analyzing the human voice or the specific object collision sound.
In the preferred embodiment, a background library and a target library are also created in advance, so that retrieval and matching of the background and the target after the monitoring data division can be realized. Through the pre-creation of the database and the division into the background library and the target library, the efficiency of searching and matching is remarkably improved. Preferably, for audio monitoring, only the matching target is retrieved, i.e. only the human voice or the specific object collision sound is indexed, and the environmental noise data is discarded, so that the efficiency of retrieving the matching can be further improved.
Preferably, performing background matching comprises:
extracting a characteristic combination X from the determined background;
searching whether a characteristic combination Y with the difference from X smaller than a preset threshold exists from a background library;
if so, determining that Y matches X; if not, a number is created for X and X is added to the background pool.
Preferably, performing the target matching comprises:
extracting a characteristic combination X from the determined target;
searching whether a characteristic combination Y with the difference from X smaller than a preset threshold exists from a target library;
if so, determining that Y matches X; if not, a number is created for X and X is added to the target repository.
Preferably, the extracting the feature combination X includes:
creating
Xi is the ith local feature in the background or object and N is the number of local features.
Preferably, the monitored data is an image,
Xi=(ai,si,ρi,vi);
wherein ai refers to the coordinate of the local feature in the image, si refers to the size of the local feature, ρ i refers to the saliency value of the local feature, and vi refers to the feature vector of the local feature.
Preferably, the searching whether the feature combination Y with the difference from X smaller than the preset threshold exists comprises:
step 1: calculating Ω ═ d (vi (Xi), vi (Yi)), where vi (Xi) refers to the feature vector of Xi, vi (Yi) refers to the feature vector of Yi, and d () refers to the euler distance function;
step 2, judging whether omega is smaller than a preset threshold value G1;
step 3, if the difference value is smaller than the preset threshold value G2, converting Xi according to ai and si of Yi, and judging whether the difference value of rho i of Xi and rho i of Yi is smaller than the preset threshold value G2 after conversion;
and 4, if the value is less than the preset value, considering that the Xi is matched with the Yi.
Preferably, the monitoring data is audio,
Xi=(ti,li,fi,ρi,vi);
wherein ti refers to a time point of the local feature in the audio, li refers to a duration of the local feature, fi refers to a spectrum description of the local feature, ρ i refers to a saliency value of the local feature, and vi refers to a feature vector of the local feature.
Preferably, the searching whether the feature combination Y with the difference from X smaller than the preset threshold exists comprises:
step 1: calculating Ω ═ d (vi (Xi), vi (Yi)), where vi (Xi) refers to the feature vector of Xi, vi (Yi) refers to the feature vector of Yi, and d () refers to the euler distance function;
step 2, judging whether omega is smaller than a preset threshold value G1;
step 3, if the difference value is smaller than the preset threshold value G2, converting Xi according to ti and li of Yi, and judging whether the difference value of rho i of Xi and rho i of Yi is smaller than the preset threshold value G2 after conversion;
step 4, if the fi of the Xi is smaller than the fi of the Yi, judging whether the fi of the Xi is similar to the fi of the Yi;
and 5, if the two are similar, considering that Xi and Yi are matched.
The preferable embodiment provides an original scheme of feature matching, different schemes are designed for images and sound respectively, and the inventor carries out a great deal of practice on the scheme, so that double requirements on performance and precision during data sharing are well met in the data sharing of the intelligent security system based on big data.
Preferably, the creating an index of the monitoring data using the matching result includes:
and if the number of the matches of the Xi and the Yi is larger than a preset threshold value G3, considering that the X and the Y are matched, and representing the background or the target corresponding to the X by the number of the background or the target corresponding to the Y when the index is created for the monitoring data.
Preferably, the background number and the target number should be reserved separately when creating the index.
In practice, when a user provides a picture, for example, an identification card photo of a person, according to the above embodiment, the system determines the portrait in the identification card photo as a target, converts the portrait into a search number, and searches the index established in the system by using the search number (an efficient algorithm may be adopted, and only the target number in the index is matched), so as to find all matched monitoring data.
In practice, when a user provides a picture, for example, a warehouse picture, according to the above embodiment, the system determines the picture as a background, converts the picture into a search number, and searches the index established in the system by using the search number (an efficient algorithm may be adopted, and only the background number in the index is matched), so as to find all matched monitoring data.
The embodiment of the invention adopts a scheme of feature matching to provide efficient data sharing for the intelligent security system, and the following embodiment adopts a scheme of data block matching to provide efficient data sharing for the intelligent security system.
Fig. 2 is a flowchart illustrating a big data-based smart security system data sharing method according to another exemplary embodiment, including:
step S10, receiving monitoring data uploaded by each security monitoring device;
step S20, blocking the monitoring data;
step S30, deleting the repeated data blocks;
step S40, establishing an index for the deduplicated data block;
and step S50, utilizing the index to provide retrieval service for the monitoring data.
The embodiment provides an efficient data sharing method of the intelligent security system based on big data, and the index is created for the monitored data, so that the service is efficiently provided for future data retrieval, and the dual requirements on performance and precision during data sharing are met.
In addition, the intelligent security system is connected with a large number of monitoring devices and needs to collect various audio and video data, so that the data uploaded by each monitoring device in each time period form a file.
In the embodiment, the data uploaded by each monitoring device is stored in a blocking and deduplication manner, so that the same data block of a new file is not saved repeatedly, and only one pointer is reserved to point to the repeated data block. The efficient storage mode also solves the problem that the storage space of the intelligent security system based on big data in the prior art is insufficient.
Preferably, the blocking the monitoring data includes:
creating a block flag with a bit operation;
matching the block mark with the monitoring data;
and partitioning the matched position.
The CPU of the computer is characterized by high bit execution efficiency and low system overhead. The preferred embodiment adopts bit operation to create the block mark, thereby improving the comparison efficiency and being beneficial to processing the monitoring data of the intelligent security system with huge data volume.
Preferably, creating the block flag with the bit operation comprises:
constructing a mark character string S ═ [ b1, b2, b3, … bm ], wherein bi is a binary string of each character in the mark character string, and the number of binary digits is n;
the data fingerprint f (Si) is set to be the combination of the i-th bits of b1, b2, b3, … bm, where i takes the 1-n-th bits of the binary string.
Preferably, matching the block flag with the monitoring data comprises:
entering the division of the jth data block Bj of the monitoring data;
step 1: skipping k characters to the right from the Bj-1 end mark;
step 2: aligning the current bit to S left;
and step 3: fetching m characters to the left from the last position of the data aligned with S;
and 4, step 4: taking the ith bit from the m characters to form Xn, and comparing Xn with f (Si) one by one, wherein i is from 1 to n.
And 5: if i from 1 to n, all satisfy Xn ═ f (si), the last bit in the data that is currently aligned with S is marked as the end bit of Bj.
Step 6: if the two characters are different, continuing to skip k characters to the right, and returning to the step 2;
and 7: returning to step 2 for L times, the last bit in the data currently aligned with S is marked as the end bit of Bj.
In the above comparison method, the comparison based on the flag character string S is realized, the data blocking of the file can be completed very efficiently, and the method is particularly suitable for executing the preferred embodiment by using the CPU.
Preferably, matching the block flag with the monitoring data comprises:
entering the division of the jth data block Bj of the monitoring data;
step 1: skipping k characters to the right from the Bj-1 end mark;
step 2: aligning the current bit to S left;
and step 3: taking r characters from the first position of the data after being aligned with S to the right, wherein r is far smaller than m;
and 4, step 4: taking the ith component Xir from the r characters taken, and then
Step 4.1: shifting the head position of Xir to the right to obtain r positions f (sir), and comparing Xir with f (sir) one by one, wherein i is from 1 to n;
step 4.2: if not, executing SHLf (Sir),1, and then returning to the step 4.1;
step 4.3: if the step 4.1 is returned to m-r times, returning to the step 2;
step 4.3: if i from 1 to n, all satisfy Xir ═ f (sir), then step 5 is entered.
And 5: fetching m characters to the left from the last position of the data aligned with S;
step 6: taking the ith bit from the m characters to form Xi, and comparing Xi with f (Si) one by one, wherein i is from 1 to n.
And 7: if i from 1 to n, all satisfy Xi ═ f (si), the last bit in the data that is currently aligned with S is marked as the end bit of Bj.
And 8: if the two characters are different, continuing to skip k characters to the right, and returning to the step 2;
and step 9: returning to step 2 for L times, the last bit in the data currently aligned with S is marked as the end bit of Bj.
In the comparison method, the characteristic comparison is carried out by taking the data volume of which r is far less than m, so that the comparison volume is further obviously reduced, the data blocking of the file can be completed with high efficiency, and the method is particularly suitable for the extremely huge data volume of the intelligent security system based on big data.
The preferred L-k-210 is the amount of data that applicants set empirically, with a good balance of space and efficiency.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.