CN114338587B - Multimedia data processing method and device, electronic equipment and storage medium - Google Patents
Multimedia data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114338587B CN114338587B CN202111603582.9A CN202111603582A CN114338587B CN 114338587 B CN114338587 B CN 114338587B CN 202111603582 A CN202111603582 A CN 202111603582A CN 114338587 B CN114338587 B CN 114338587B
- Authority
- CN
- China
- Prior art keywords
- multimedia data
- abnormal
- data
- feature
- reference abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims description 15
- 230000002159 abnormal effect Effects 0.000 claims abstract description 233
- 238000012545 processing Methods 0.000 claims abstract description 48
- 238000000034 method Methods 0.000 claims abstract description 36
- 230000008569 process Effects 0.000 claims abstract description 16
- 238000001514 detection method Methods 0.000 claims description 40
- 230000005856 abnormality Effects 0.000 claims description 18
- 238000013136 deep learning model Methods 0.000 claims description 17
- 238000012549 training Methods 0.000 claims description 16
- 230000006399 behavior Effects 0.000 claims description 14
- 238000012790 confirmation Methods 0.000 claims description 14
- 238000012216 screening Methods 0.000 abstract description 8
- 238000004891 communication Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 6
- 230000002547 anomalous effect Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 238000013145 classification model Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 2
- 238000012550 audit Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000005252 bulbus oculi Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure relates to a multimedia data processing system, method, apparatus, electronic device and storage medium, including: determining multimedia data meeting preset abnormal conditions in the multimedia data set as reference abnormal multimedia data; determining data characteristics of reference abnormal multimedia data to obtain reference abnormal characteristics; determining feature similarity between the reference abnormal feature and data features of each multimedia data except the reference abnormal multimedia data in the multimedia data set; and determining the multimedia data with the feature similarity larger than the similarity threshold as newly added reference abnormal multimedia data. Therefore, after one piece of multimedia data is determined to be the reference abnormal multimedia data, other multimedia data with higher characteristic similarity can be quickly queried, more reference abnormal multimedia data can be determined, the screening process is efficient and quick, and further, pushing of the multimedia data can be quickly adjusted, and the content quality of the multimedia data is improved.
Description
Technical Field
The present disclosure relates to the field of multimedia data processing, and in particular, to a multimedia data processing system, a method, an apparatus, an electronic device, and a storage medium.
Background
With the development of the internet in recent years, multimedia data on various multimedia data platforms is rapidly increased, and the content quality of the multimedia data on the multimedia data platforms is a very important index for users. In some situations, some users uploading multimedia data upload some multimedia data containing abnormal factors in order to attract eyeballs, and at this time, the multimedia data platform is required to process the multimedia data uploaded by the users, and the multimedia data containing abnormal factors is subjected to processing such as shielding or deleting.
In the prior art, the abnormal multimedia data are processed in a deep learning mode, a large amount of sample multimedia data are trained to obtain a classification model, and part of the sample multimedia data comprise the abnormal multimedia data acquired in advance, so that the obtained classification model can classify any multimedia data and judge whether the multimedia data are the abnormal multimedia data.
However, various abnormal factors in the multimedia data are layered endlessly and frequently replaced at present, the classification model obtained by training in advance can only be trained according to known abnormal factors, the timeliness is low, the recognition accuracy of the multimedia data containing novel abnormal factors is low, the content quality of the multimedia data is reduced, and bad experience is brought to users.
Disclosure of Invention
The disclosure provides a multimedia data processing method, a device, an electronic device and a storage medium, so as to at least solve the problems that a classification model obtained by pre-training in the related technology can only be trained according to known abnormal factors, the timeliness is low, the recognition accuracy of multimedia data containing novel abnormal factors is low, the content quality of the multimedia data is reduced, and bad experience is brought to users. The technical scheme of the present disclosure is as follows:
according to a first aspect of an embodiment of the present disclosure, there is provided a multimedia data processing method, including:
determining multimedia data meeting preset abnormal conditions in the multimedia data set as reference abnormal multimedia data;
determining the data characteristics of the reference abnormal multimedia data to obtain reference abnormal characteristics;
determining feature similarity between the reference abnormal feature and data features of each multimedia data in the multimedia data set except the reference abnormal multimedia data;
and determining the multimedia data with the feature similarity larger than a similarity threshold as newly added reference abnormal multimedia data.
Optionally, the method further comprises:
Performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained through pre-training to obtain an anomaly detection result of the multimedia data, wherein the anomaly detection result represents the anomaly probability of the multimedia data;
and determining the multimedia data with the abnormal probability larger than a probability threshold as abnormal multimedia data.
Optionally, the performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained by training in advance to obtain an anomaly detection result of the multimedia data includes:
performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained through pre-training to obtain data characteristics of the multimedia data, and storing the data characteristics of the multimedia data into a preset storage space;
determining an abnormality detection result of the multimedia data according to the data characteristics;
the determining the feature similarity between the reference abnormal feature and the data feature of each multimedia data except the reference abnormal multimedia data in the multimedia data set comprises:
And determining the feature similarity between the reference abnormal feature and the data features of all multimedia data except the reference abnormal multimedia data in the preset storage space.
Optionally, the method further comprises:
performing preset processing on the reference abnormal multimedia data and the abnormal multimedia data; the preset process is used for reducing the probability that the reference abnormal multimedia data and the abnormal multimedia data are recommended.
Optionally, the determining, as the reference abnormal multimedia data, the multimedia data satisfying the preset abnormal condition in the multimedia data set includes:
and acquiring operation behavior data corresponding to each piece of multimedia data in the multimedia data set, and taking the multimedia data of which the operation behavior data meets the preset abnormal condition as reference abnormal multimedia data.
Optionally, the determining, as the reference abnormal multimedia data, the multimedia data satisfying the preset abnormal condition in the multimedia data set includes:
pushing the multimedia data meeting the preset abnormal condition in the multimedia data set to a target terminal;
responding to the confirmation operation information of the target terminal on any multimedia data meeting the preset abnormal condition, and taking any multimedia data meeting the preset abnormal condition as reference abnormal multimedia data;
The determining the multimedia data with the feature similarity greater than the similarity threshold as the newly added reference abnormal multimedia data comprises the following steps:
pushing the multimedia data with the feature similarity larger than a similarity threshold to the target terminal;
and responding to the confirmation operation information of the target terminal on any multimedia data with the feature similarity larger than a similarity threshold, and taking any multimedia data with the feature similarity larger than the similarity threshold as newly added reference abnormal multimedia data.
According to a second aspect of embodiments of the present disclosure, there is provided a multimedia data processing apparatus comprising:
a first determination unit configured to perform determination of multimedia data satisfying a preset abnormal condition in the multimedia data set as reference abnormal multimedia data;
a second determining unit configured to perform determining a data characteristic of the reference abnormal multimedia data, resulting in a reference abnormal characteristic;
a third determination unit configured to perform determination of feature similarity between the reference abnormal feature and data features of each of the multimedia data other than the reference abnormal multimedia data in the multimedia data set;
And a fourth determining unit configured to perform determination of the multimedia data whose feature similarity is greater than a similarity threshold as newly added reference abnormal multimedia data.
Optionally, the apparatus further includes:
the detection unit is configured to perform abnormality detection on the multimedia data in the multimedia data set by using a deep learning model obtained through training in advance, so as to obtain an abnormality detection result of the multimedia data, wherein the abnormality detection result represents the abnormality probability of the multimedia data; and determining the multimedia data with the abnormal probability larger than a probability threshold as abnormal multimedia data.
Optionally, the detection unit is configured to perform:
performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained through pre-training to obtain data characteristics of the multimedia data, and storing the data characteristics of the multimedia data into a preset storage space;
determining an abnormality detection result of the multimedia data according to the data characteristics;
the third determination unit is configured to perform:
and determining the feature similarity between the reference abnormal feature and the data features of all multimedia data except the reference abnormal multimedia data in the preset storage space.
Optionally, the apparatus further includes:
a processing unit configured to perform preset processing on the reference abnormal multimedia data and the abnormal multimedia data; the preset process is used for reducing the probability that the reference abnormal multimedia data and the abnormal multimedia data are recommended.
Optionally, the first determining unit is configured to perform:
and acquiring operation behavior data corresponding to each piece of multimedia data in the multimedia data set, and taking the multimedia data of which the operation behavior data meets the preset abnormal condition as reference abnormal multimedia data.
Optionally, the first determining unit is configured to perform:
pushing the multimedia data meeting the preset abnormal condition in the multimedia data set to a target terminal;
responding to the confirmation operation information of the target terminal on any multimedia data meeting the preset abnormal condition, and taking any multimedia data meeting the preset abnormal condition as reference abnormal multimedia data;
the fourth determination unit is configured to perform:
pushing the multimedia data with the feature similarity larger than a similarity threshold to the target terminal;
and responding to the confirmation operation information of the target terminal on any multimedia data with the feature similarity larger than a similarity threshold, and taking any multimedia data with the feature similarity larger than the similarity threshold as newly added reference abnormal multimedia data.
According to a third aspect of embodiments of the present disclosure, there is provided a multimedia data processing electronic device comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the multimedia data processing method of the first item above.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer readable storage medium, which when executed by a processor of a multimedia data processing electronic device, enables the multimedia data processing electronic device to perform the multimedia data processing method of the first item above.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising a computer program/instruction which, when executed by a processor, implements the multimedia data processing method of the first aspect described above.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
determining multimedia data meeting preset abnormal conditions in the multimedia data set as reference abnormal multimedia data; determining data characteristics of reference abnormal multimedia data to obtain reference abnormal characteristics; determining feature similarity between the reference abnormal feature and data features of each multimedia data except the reference abnormal multimedia data in the multimedia data set; and determining the multimedia data with the feature similarity larger than the similarity threshold as newly added reference abnormal multimedia data.
Therefore, after one piece of multimedia data is determined to be the reference abnormal multimedia data, other multimedia data with higher characteristic similarity can be quickly queried according to the data characteristics of the multimedia data, more reference abnormal multimedia data can be determined, the screening process of the reference abnormal multimedia data is more efficient and quick, and further, the pushing of the multimedia data can be quickly adjusted according to the reference abnormal multimedia data, the content quality of the multimedia data is improved, and better experience is brought to users.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure and do not constitute an undue limitation on the disclosure.
Fig. 1 is a flowchart illustrating a multimedia data processing method according to an exemplary embodiment.
Fig. 2 is a schematic diagram illustrating a scheme of a multimedia data processing method according to an exemplary embodiment.
Fig. 3 is a block diagram illustrating a multimedia data processing apparatus according to an exemplary embodiment.
Fig. 4 is a block diagram of an electronic device for multimedia data processing, according to an example embodiment.
Fig. 5 is a block diagram illustrating an apparatus for multimedia data processing according to an exemplary embodiment.
Detailed Description
In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the foregoing figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the disclosure described herein may be capable of operation in sequences other than those illustrated or described herein. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.
Fig. 1 is a flowchart illustrating a multimedia data processing method according to an exemplary embodiment, and as shown in fig. 1, the multimedia data processing method includes:
In step S11, multimedia data satisfying a preset abnormal condition in the multimedia data set is determined as reference abnormal multimedia data.
In the present disclosure, the multimedia data may be video data, audio data or image data, and the like, and is not particularly limited. The multimedia data set is a set of multimedia data to be screened, and the multimedia data set may be stored in a database, where some reference abnormal multimedia data may be included in the multimedia data set, and the reference abnormal multimedia data is multimedia data unsuitable for propagation or pushing, for example, multimedia data including non-compliance information, or multimedia data that easily causes complaints, and so on.
In one implementation manner, the anomaly detection can be performed on the multimedia data in the multimedia data set by using a deep learning model obtained through training in advance to obtain an anomaly detection result of the multimedia data, wherein the anomaly detection result represents the anomaly probability of the multimedia data; then, multimedia data having an anomaly probability greater than a probability threshold is determined as anomalous multimedia data.
The abnormal multimedia data and the reference abnormal multimedia data may be multimedia data with different abnormal factors, and in general, the abnormal reasons of the abnormal multimedia data are more common or have higher occurrence frequency, so that the abnormality detection accuracy of the trained deep learning model is higher, and therefore, a part of abnormal multimedia data can be screened out from the multimedia data set through the pre-trained deep learning model, thereby improving the processing efficiency of the multimedia data.
The deep learning model may construct a reasonable anomaly tag in advance according to priori knowledge of the anomaly multimedia data, collect corresponding training data, and train the collected training data, where after anomaly detection is performed on the multimedia data in the multimedia data set, the higher the anomaly probability of the multimedia data, the more likely the multimedia data is the anomaly multimedia data.
The deep learning model can be a multi-modal model, the multi-modal model is utilized to analyze the multimedia data, the data characteristics of the obtained multimedia data are discrete numerical vectors obtained by model learning, no corresponding physical meaning exists, and the whole characteristic vector is sparse characteristic representation of the multimedia data.
In this step, the reference abnormal multimedia data may be determined according to analysis of operation behavior data of the multimedia data, for example, operation behavior data corresponding to any one of the multimedia data may be obtained, and the multimedia data whose operation behavior data satisfies a preset abnormal condition is used as the reference abnormal multimedia data. For example, if a certain multimedia data brings about a larger amount of consumption in a shorter time, the multimedia data may be considered to be abnormal, and it is highly likely that the abnormal multimedia data is referred to.
In addition, in one implementation manner, after determining the reference abnormal multimedia data, the target terminal may audit the reference abnormal multimedia data, for example, the multimedia data meeting the preset abnormal condition in the multimedia data set may be pushed to the target terminal, and then any multimedia data meeting the preset abnormal condition is used as the reference abnormal multimedia data in response to the confirmation operation information of the target terminal on any multimedia data meeting the preset abnormal condition. Therefore, through auditing, the accuracy of the confirmed reference abnormal multimedia data is higher, and the possibility of false screening is reduced.
In step S12, data characteristics of the reference abnormal multimedia data are determined, resulting in reference abnormal characteristics.
In this step, after the reference abnormal multimedia data is determined, further feature recognition may be performed on the determined reference abnormal multimedia data, and the data features of the reference abnormal multimedia data, that is, the reference abnormal data features, may be extracted.
It will be appreciated that multimedia data having similar data characteristics will likely include similar content, and therefore, multimedia data similar to reference anomalous multimedia data may be further screened from the multimedia data collection using the reference anomalous characteristics, and these multimedia data will likely also have anomalous factors similar to the reference anomalous multimedia data.
In step S13, feature similarities between the reference abnormal feature and data features of each multimedia data other than the reference abnormal multimedia data in the multimedia data set are determined.
The feature similarity may be represented by a feature distance, and in general, the feature distance is inversely proportional to the feature similarity, that is, the larger the feature distance between the data features of the two multimedia data is, the smaller the feature similarity between the two multimedia data is, whereas the smaller the feature distance between the data features of the two multimedia data is, the larger the feature similarity between the two multimedia data is. The feature distance may be a euclidean distance, a cosine distance, or the like, and is not particularly limited.
In one implementation manner, for multimedia data in the multimedia data set, anomaly detection is performed by using a deep learning model obtained by training in advance, and the anomaly detection is often dependent on data features of the multimedia data, so that the data features of the multimedia data in the multimedia data set can be stored in the anomaly detection process, and the method is directly invoked when feature similarity is calculated in the step. Therefore, the data characteristics are calculated only once for each multimedia data, so that the calculation amount can be reduced, the retrieval efficiency is improved, and the processing of the multimedia data is more time-efficient.
Specifically, in the anomaly detection process, firstly, performing anomaly detection on multimedia data in a multimedia data set by using a deep learning model obtained through training in advance to obtain data characteristics of the multimedia data, and storing the data characteristics of the multimedia data into a preset storage space; then, determining an abnormal detection result of the multimedia data according to the data characteristics; further, in this step, the feature similarity between the reference abnormal feature and the data feature of each multimedia data other than the reference abnormal multimedia data in the preset storage space may be determined.
In step S14, multimedia data having a feature similarity greater than the similarity threshold is determined as newly added reference abnormal multimedia data.
In this step, the similarity threshold may be a specific value, for example, when the feature distance between the data feature of any multimedia data and the reference abnormal data feature is smaller than 0.1, the feature similarity between the data feature of the multimedia data and the reference abnormal data feature is considered to be larger than the similarity threshold. Alternatively, the similarity threshold may be a range threshold, for example, according to the ranking of feature similarity, the top N pieces of multimedia data are used as reference abnormal multimedia data, where N is any positive integer, and the like, and is not limited in particular
In one implementation, after determining the multimedia data with the feature similarity greater than the similarity threshold, the target terminal may audit the multimedia data with the feature similarity greater than the similarity threshold.
For example, the multimedia data with the feature similarity greater than the similarity threshold may be pushed to the target terminal, and then, in response to the confirmation operation information of the target terminal on the multimedia data with the feature similarity greater than the similarity threshold, the multimedia data with the feature similarity greater than the similarity threshold may be used as the newly added reference abnormal multimedia data. Therefore, the accuracy of the newly added reference abnormal multimedia data is higher through auditing, and the possibility of error screening is further reduced.
In one implementation manner, after the reference abnormal multimedia data is determined, the reference abnormal multimedia data may be subjected to preset processing, and at the same time, after the abnormal multimedia data is determined from the multimedia data set, the abnormal multimedia data may be subjected to the same preset processing, where the preset processing is used to reduce the probability that the reference abnormal multimedia data and the abnormal multimedia data are recommended, so as to improve the content quality of the multimedia data pushed to the target terminal.
For example, the preset processing may be a process of reducing the weight of the reference abnormal multimedia data and the abnormal multimedia data or filtering the reference abnormal multimedia data, so as to reduce the frequency of pushing the reference abnormal multimedia data to the target terminal, or the preset processing may also be a process of shielding or deleting the reference abnormal multimedia data from the multimedia data set, or the like, which is not limited in particular.
Fig. 2 is a schematic diagram of a scheme of a multimedia data processing method, in which multimedia data is video data, and a multimedia data set is a search video library storing a plurality of videos.
On one hand, the video in the searched video library can be subjected to anomaly detection through a pre-trained deep learning model, namely a multi-mode model, so that the anomaly probability of the video is obtained, and the video with the anomaly probability higher than the probability threshold can be identified as the anomaly video, namely the anomaly multimedia data. Further, a preset operation can be performed on these videos, reducing the possibility of pushing abnormal videos to the target terminal. In addition, a feature retrieval system can be constructed for subsequent collisions with the data features of the reference anomaly video based on the data features of the video output by the multimodal model.
On the other hand, the video meeting the preset abnormal condition can be determined from the search video library according to the operation behavior data of each video, then the target terminal is used for checking the video meeting the preset abnormal condition, the reference abnormal video, namely the reference abnormal multimedia data, is determined based on the confirmation operation information of the target terminal, after the reference abnormal multimedia data is determined, the preset operation is carried out on the videos, and the possibility of pushing the reference abnormal video to the target terminal is reduced. And meanwhile, determining the reference abnormal characteristics of the reference abnormal video, performing collision of the data characteristics in a characteristic retrieval system, determining the characteristic similarity between the reference abnormal characteristics and the data characteristics of each video, and taking the video with the characteristic similarity larger than a similarity threshold as a new reference abnormal video.
In addition, for videos for which the target terminal does not return confirmation operation information or videos with feature similarity not greater than a similarity threshold, the videos can be considered to be not abnormal videos or abnormal videos, the videos are transmitted from the data processing system, and the videos can be pushed normally without performing preset operations on the videos.
From the above, it can be seen that, according to the technical scheme provided by the embodiment of the disclosure, after one piece of multimedia data is determined as the reference abnormal multimedia data, other multimedia data with higher feature similarity can be quickly queried according to the data feature of the multimedia data, more reference abnormal multimedia data is determined, the screening process of the reference abnormal multimedia data is more efficient and quick, and further, the pushing of the multimedia data can be quickly adjusted according to the reference abnormal multimedia data, so that the content quality of the multimedia data is improved, and better experience is brought to the user.
Fig. 3 is a block diagram of a multimedia data processing apparatus according to an exemplary embodiment, the apparatus comprising:
a first determining unit 201 configured to perform determination of multimedia data satisfying a preset abnormal condition in a multimedia data set as reference abnormal multimedia data;
a second determining unit 202 configured to perform determining a data characteristic of the reference abnormal multimedia data, resulting in a reference abnormal characteristic;
a third determining unit 203 configured to perform determining a feature similarity between the reference abnormal feature and a data feature of each of the multimedia data other than the reference abnormal multimedia data in the multimedia data set;
A fourth determining unit 204 configured to perform determination of the multimedia data with the feature similarity larger than a similarity threshold as newly added reference abnormal multimedia data.
In one implementation, the apparatus further comprises:
the detection unit is configured to perform abnormality detection on the multimedia data in the multimedia data set by using a deep learning model obtained through training in advance, so as to obtain an abnormality detection result of the multimedia data, wherein the abnormality detection result represents the abnormality probability of the multimedia data; and determining the multimedia data with the abnormal probability larger than a probability threshold as abnormal multimedia data.
In one implementation, the detection unit is configured to perform:
performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained through pre-training to obtain data characteristics of the multimedia data, and storing the data characteristics of the multimedia data into a preset storage space;
determining an abnormality detection result of the multimedia data according to the data characteristics;
the third determining unit 203 is configured to perform:
and determining the feature similarity between the reference abnormal feature and the data features of all multimedia data except the reference abnormal multimedia data in the preset storage space.
In one implementation, the apparatus further comprises:
a processing unit configured to perform preset processing on the reference abnormal multimedia data and the abnormal multimedia data; the preset process is used for reducing the probability that the reference abnormal multimedia data and the abnormal multimedia data are recommended.
In an implementation manner, the first determining unit 201 is configured to perform:
and acquiring operation behavior data corresponding to each piece of multimedia data in the multimedia data set, and taking the multimedia data of which the operation behavior data meets the preset abnormal condition as reference abnormal multimedia data.
In an implementation manner, the first determining unit 201 is configured to perform:
pushing the multimedia data meeting the preset abnormal condition in the multimedia data set to a target terminal;
responding to the confirmation operation information of the target terminal on any multimedia data meeting the preset abnormal condition, and taking any multimedia data meeting the preset abnormal condition as reference abnormal multimedia data;
the fourth determining unit 204 is configured to perform:
pushing the multimedia data with the feature similarity larger than a similarity threshold to the target terminal;
And responding to the confirmation operation information of the target terminal on any multimedia data with the feature similarity larger than a similarity threshold, and taking any multimedia data with the feature similarity larger than the similarity threshold as newly added reference abnormal multimedia data.
From the above, in the scheme provided by the embodiment of the disclosure, after one piece of multimedia data is determined as the reference abnormal multimedia data, other multimedia data with higher feature similarity can be quickly queried according to the data feature of the multimedia data, more reference abnormal multimedia data is determined, the screening process of the reference abnormal multimedia data is more efficient and quick, and further, the pushing of the multimedia data can be quickly adjusted according to the reference abnormal multimedia data, so that the content quality of the multimedia data is improved, and better experience is brought to a user.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
Fig. 4 is a block diagram of an electronic device for multimedia data processing, according to an example embodiment.
In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory, comprising instructions executable by a processor of an electronic device to perform the above-described method. Alternatively, the computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
In an exemplary embodiment, a computer program product is also provided which, when run on a computer, causes the computer to carry out the method of multimedia data processing described above.
From the above, in the scheme provided by the embodiment of the disclosure, after one piece of multimedia data is determined as the reference abnormal multimedia data, other multimedia data with higher feature similarity can be quickly queried according to the data feature of the multimedia data, more reference abnormal multimedia data is determined, the screening process of the reference abnormal multimedia data is more efficient and quick, and further, the pushing of the multimedia data can be quickly adjusted according to the reference abnormal multimedia data, so that the content quality of the multimedia data is improved, and better experience is brought to a user.
Fig. 5 is a block diagram illustrating an apparatus 800 for multimedia data processing according to an exemplary embodiment.
For example, apparatus 800 may be a mobile phone, computer, digital broadcast electronic device, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or the like.
Referring to fig. 5, apparatus 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.
The processing component 802 generally controls overall operation of the apparatus 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interactions between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operations at the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
Power supply component 807 provides power to the various components of device 800. Power supply component 807 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for device 800.
The multimedia component 808 includes a screen between the device 800 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. The front camera and/or the rear camera may receive external multimedia data when the device 800 is in an operational mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 further includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.
The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of the apparatus 800. For example, the sensor assembly 814 may detect an on/off state of the device 800, a relative positioning of the components, such as a display and keypad of the apparatus 800, the sensor assembly 814 may also detect a change in position of the apparatus 800 or one component of the apparatus 800, the presence or absence of user contact with the apparatus 800, an orientation or acceleration/deceleration of the apparatus 800, and a change in temperature of the apparatus 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 816 is configured to facilitate communication between the apparatus 800 and other devices, either in a wired or wireless manner. The device 800 may access a wireless network based on a communication standard, such as WiFi, an operator network (e.g., 2G, 3G, 4G, or 5G), or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic elements for executing the methods described in the first and second aspects.
In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 804 including instructions executable by processor 820 of apparatus 800 to perform the above-described method. Alternatively, for example, the storage medium may be a non-transitory computer-readable storage medium, which may be, for example, ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
In an exemplary embodiment, a computer program product comprising instructions which, when run on a computer, cause the computer to perform the multimedia data processing method of the first described in the above embodiments is also provided.
From the above, in the scheme provided by the embodiment of the disclosure, after one piece of multimedia data is determined as the reference abnormal multimedia data, other multimedia data with higher feature similarity can be quickly queried according to the data feature of the multimedia data, more reference abnormal multimedia data is determined, the screening process of the reference abnormal multimedia data is more efficient and quick, and further, the pushing of the multimedia data can be quickly adjusted according to the reference abnormal multimedia data, so that the content quality of the multimedia data is improved, and better experience is brought to a user.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A method of multimedia data processing, the method comprising:
determining multimedia data meeting preset abnormal conditions in the multimedia data set as reference abnormal multimedia data;
determining the data characteristics of the reference abnormal multimedia data to obtain reference abnormal characteristics;
determining feature similarity between the reference abnormal feature and data features of each multimedia data in the multimedia data set except the reference abnormal multimedia data;
determining the multimedia data with the feature similarity larger than a similarity threshold as newly added reference abnormal multimedia data;
the method further comprises the steps of:
performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained through pre-training to obtain data characteristics of the multimedia data, and storing the data characteristics of the multimedia data into a preset storage space;
Determining an abnormality detection result of the multimedia data according to the data characteristics, wherein the abnormality detection result represents the abnormality probability of the multimedia data;
determining the multimedia data with the abnormal probability larger than a probability threshold as abnormal multimedia data;
the determining the feature similarity between the reference abnormal feature and the data feature of each multimedia data except the reference abnormal multimedia data in the multimedia data set comprises:
determining feature similarity between the reference abnormal feature and the data features of each multimedia data except the reference abnormal multimedia data in the preset storage space through a feature retrieval system; the feature retrieval system is constructed from the data features of the multimedia data.
2. The method of multimedia data processing according to claim 1, wherein the method further comprises:
performing preset processing on the reference abnormal multimedia data and the abnormal multimedia data; the preset process is used for reducing the probability that the reference abnormal multimedia data and the abnormal multimedia data are recommended.
3. The method according to claim 1, wherein determining multimedia data satisfying a preset abnormal condition in the multimedia data set as reference abnormal multimedia data comprises:
And acquiring operation behavior data corresponding to each piece of multimedia data in the multimedia data set, and taking the multimedia data of which the operation behavior data meets the preset abnormal condition as reference abnormal multimedia data.
4. The method according to claim 1, wherein determining multimedia data satisfying a preset abnormal condition in the multimedia data set as reference abnormal multimedia data comprises:
pushing the multimedia data meeting the preset abnormal condition in the multimedia data set to a target terminal;
responding to the confirmation operation information of the target terminal on any multimedia data meeting the preset abnormal condition, and taking any multimedia data meeting the preset abnormal condition as reference abnormal multimedia data;
the determining the multimedia data with the feature similarity greater than the similarity threshold as the newly added reference abnormal multimedia data comprises the following steps:
pushing the multimedia data with the feature similarity larger than a similarity threshold to the target terminal;
and responding to the confirmation operation information of the target terminal on any multimedia data with the feature similarity larger than a similarity threshold, and taking any multimedia data with the feature similarity larger than the similarity threshold as newly added reference abnormal multimedia data.
5. A multimedia data processing apparatus, comprising:
a first determination unit configured to perform determination of multimedia data satisfying a preset abnormal condition in the multimedia data set as reference abnormal multimedia data;
a second determining unit configured to perform determining a data characteristic of the reference abnormal multimedia data, resulting in a reference abnormal characteristic;
a third determination unit configured to perform determination of feature similarity between the reference abnormal feature and data features of each of the multimedia data other than the reference abnormal multimedia data in the multimedia data set;
a fourth determination unit configured to perform determination of the multimedia data whose feature similarity is greater than a similarity threshold as newly added reference abnormal multimedia data;
the apparatus further comprises:
the detection unit is configured to perform anomaly detection on the multimedia data in the multimedia data set by utilizing a deep learning model obtained through pre-training, obtain the data characteristics of the multimedia data, and store the data characteristics of the multimedia data into a preset storage space; determining an abnormality detection result of the multimedia data according to the data characteristics, wherein the abnormality detection result represents the abnormality probability of the multimedia data; determining the multimedia data with the abnormal probability larger than a probability threshold as abnormal multimedia data;
The third determination unit is configured to perform:
determining feature similarity between the reference abnormal feature and the data features of each multimedia data except the reference abnormal multimedia data in the preset storage space through a feature retrieval system; the feature retrieval system is constructed from the data features of the multimedia data.
6. The multimedia data processing apparatus according to claim 5, wherein the apparatus further comprises:
a processing unit configured to perform preset processing on the reference abnormal multimedia data and the abnormal multimedia data; the preset process is used for reducing the probability that the reference abnormal multimedia data and the abnormal multimedia data are recommended.
7. The multimedia data processing apparatus of claim 5, wherein the first determination unit is configured to perform:
and acquiring operation behavior data corresponding to each piece of multimedia data in the multimedia data set, and taking the multimedia data of which the operation behavior data meets the preset abnormal condition as reference abnormal multimedia data.
8. The multimedia data processing apparatus of claim 5, wherein the first determination unit is configured to perform:
Pushing the multimedia data meeting the preset abnormal condition in the multimedia data set to a target terminal;
responding to the confirmation operation information of the target terminal on any multimedia data meeting the preset abnormal condition, and taking any multimedia data meeting the preset abnormal condition as reference abnormal multimedia data;
the fourth determination unit is configured to perform:
pushing the multimedia data with the feature similarity larger than a similarity threshold to the target terminal;
and responding to the confirmation operation information of the target terminal on any multimedia data with the feature similarity larger than a similarity threshold, and taking any multimedia data with the feature similarity larger than the similarity threshold as newly added reference abnormal multimedia data.
9. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the multimedia data processing method of any of claims 1 to 4.
10. A computer readable storage medium, characterized in that instructions in the computer readable storage medium, when executed by a processor of a multimedia data processing electronic device, enable the multimedia data processing electronic device to perform the multimedia data processing method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111603582.9A CN114338587B (en) | 2021-12-24 | 2021-12-24 | Multimedia data processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111603582.9A CN114338587B (en) | 2021-12-24 | 2021-12-24 | Multimedia data processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114338587A CN114338587A (en) | 2022-04-12 |
| CN114338587B true CN114338587B (en) | 2024-03-12 |
Family
ID=81012704
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111603582.9A Active CN114338587B (en) | 2021-12-24 | 2021-12-24 | Multimedia data processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114338587B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111047049A (en) * | 2019-12-05 | 2020-04-21 | 北京小米移动软件有限公司 | Method, apparatus and medium for processing multimedia data based on machine learning model |
| CN111258435A (en) * | 2020-01-15 | 2020-06-09 | 北京达佳互联信息技术有限公司 | Multimedia resource commenting method and device, electronic equipment and storage medium |
| CN111858973A (en) * | 2020-07-30 | 2020-10-30 | 北京达佳互联信息技术有限公司 | Multimedia event information detection method, device, server and storage medium |
| CN111949808A (en) * | 2019-04-29 | 2020-11-17 | 北京达佳互联信息技术有限公司 | Multimedia content similarity determination method and device, electronic equipment and storage medium |
| CN112597864A (en) * | 2020-12-16 | 2021-04-02 | 佳都新太科技股份有限公司 | Monitoring video abnormity detection method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102583582B1 (en) * | 2020-02-24 | 2023-09-27 | 주식회사 마키나락스 | Method for generating anoamlous data |
| US11188798B1 (en) * | 2020-05-21 | 2021-11-30 | International Business Machines Corporation | Detecting artificial multimedia content using deep network response analysis |
-
2021
- 2021-12-24 CN CN202111603582.9A patent/CN114338587B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111949808A (en) * | 2019-04-29 | 2020-11-17 | 北京达佳互联信息技术有限公司 | Multimedia content similarity determination method and device, electronic equipment and storage medium |
| CN111047049A (en) * | 2019-12-05 | 2020-04-21 | 北京小米移动软件有限公司 | Method, apparatus and medium for processing multimedia data based on machine learning model |
| CN111258435A (en) * | 2020-01-15 | 2020-06-09 | 北京达佳互联信息技术有限公司 | Multimedia resource commenting method and device, electronic equipment and storage medium |
| CN111858973A (en) * | 2020-07-30 | 2020-10-30 | 北京达佳互联信息技术有限公司 | Multimedia event information detection method, device, server and storage medium |
| CN112597864A (en) * | 2020-12-16 | 2021-04-02 | 佳都新太科技股份有限公司 | Monitoring video abnormity detection method and device |
Non-Patent Citations (1)
| Title |
|---|
| 基于深度信念网络和线性单分类SVM的高维异常检测;李昊奇;应娜;郭春生;王金华;;电信科学(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114338587A (en) | 2022-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109446994B (en) | Gesture key point detection method and device, electronic equipment and storage medium | |
| EP3767488A1 (en) | Method and device for processing untagged data, and storage medium | |
| CN110781323A (en) | Method and device for determining label of multimedia resource, electronic equipment and storage medium | |
| CN112148923B (en) | Method for ordering search results, method, device and equipment for generating ordering model | |
| CN111046927B (en) | Method and device for processing annotation data, electronic equipment and storage medium | |
| CN111813932B (en) | Text data processing method, text data classifying device and readable storage medium | |
| CN113779257A (en) | Method, device, equipment, medium and product for analyzing text classification model | |
| CN110738267B (en) | Image classification method, device, electronic equipment and storage medium | |
| CN112884040A (en) | Training sample data optimization method and system, storage medium and electronic equipment | |
| CN105551047A (en) | Picture content detecting method and device | |
| CN112328809A (en) | Entity classification method, device and computer readable storage medium | |
| CN109145151B (en) | Video emotion classification acquisition method and device | |
| CN114338587B (en) | Multimedia data processing method and device, electronic equipment and storage medium | |
| CN107301188B (en) | Method for acquiring user interest and electronic equipment | |
| CN112149653B (en) | Information processing method, information processing device, electronic equipment and storage medium | |
| CN114547421A (en) | Search processing method and device, electronic equipment and storage medium | |
| CN111368161B (en) | Search intention recognition method, intention recognition model training method and device | |
| CN111382367B (en) | Search result ordering method and device | |
| CN112036507B (en) | Training method and device of image recognition model, storage medium and electronic equipment | |
| CN111428806B (en) | Image tag determining method and device, electronic equipment and storage medium | |
| CN115484471B (en) | Method and device for recommending anchor | |
| CN115225702B (en) | Information pushing method and device, electronic equipment and storage medium | |
| CN114722238B (en) | Video recommendation method and device, electronic equipment, storage medium and program product | |
| CN111782761B (en) | Comment information determining method and device, electronic equipment and storage medium | |
| CN110147472B (en) | Detection method and device for cheating sites and detection device for cheating sites |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |