CN114338587A - Multimedia data processing method and device, electronic equipment and storage medium - Google Patents
Multimedia data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114338587A CN114338587A CN202111603582.9A CN202111603582A CN114338587A CN 114338587 A CN114338587 A CN 114338587A CN 202111603582 A CN202111603582 A CN 202111603582A CN 114338587 A CN114338587 A CN 114338587A
- Authority
- CN
- China
- Prior art keywords
- multimedia data
- abnormal
- data
- determining
- reference abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims description 16
- 230000002159 abnormal effect Effects 0.000 claims abstract description 226
- 238000012545 processing Methods 0.000 claims abstract description 46
- 238000000034 method Methods 0.000 claims abstract description 25
- 238000001514 detection method Methods 0.000 claims description 40
- 238000013136 deep learning model Methods 0.000 claims description 18
- 238000012549 training Methods 0.000 claims description 18
- 230000006399 behavior Effects 0.000 claims description 12
- 238000012790 confirmation Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 abstract description 6
- 238000012216 screening Methods 0.000 abstract description 4
- 238000004891 communication Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 238000013145 classification model Methods 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 2
- 230000002547 anomalous effect Effects 0.000 description 2
- 238000012550 audit Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000013585 weight reducing agent Substances 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The present disclosure relates to a multimedia data processing system, method, apparatus, electronic device and storage medium, including: determining multimedia data meeting preset abnormal conditions in the multimedia data set as reference abnormal multimedia data; determining data characteristics of the reference abnormal multimedia data to obtain reference abnormal characteristics; determining feature similarity between the reference abnormal features and data features of the multimedia data in the multimedia data set except the reference abnormal multimedia data; and determining the multimedia data with the characteristic similarity larger than the similarity threshold as the newly added reference abnormal multimedia data. Therefore, after one multimedia data is determined as the reference abnormal multimedia data, other multimedia data with higher feature similarity can be quickly inquired, more reference abnormal multimedia data are determined, the screening process is more efficient and quick, further, the pushing of the multimedia data can be quickly adjusted, and the content quality of the multimedia data is improved.
Description
Technical Field
The present disclosure relates to the field of multimedia data processing, and in particular, to a multimedia data processing system, method, apparatus, electronic device, and storage medium.
Background
With the development of internet in recent years, multimedia data on most media data platforms are rapidly increasing, and for users, the content quality of the multimedia data on the multimedia data platforms is a very important index. In some scenarios, some users uploading multimedia data may upload some multimedia data including abnormal factors in order to catch eyes, and at this time, the multimedia data platform is required to process the multimedia data uploaded by the users, and the multimedia data including the abnormal factors are subjected to processing such as shielding or deleting.
In the prior art, a deep learning manner is usually adopted to process the abnormal multimedia data, a large amount of sample multimedia data is trained to obtain a classification model, and the sample multimedia data comprises part of the abnormal multimedia data acquired in advance, so that the obtained classification model can classify any multimedia data and judge whether the multimedia data is the abnormal multimedia data.
However, various abnormal factors in the multimedia data are present in a large number and are frequently replaced, the classification model obtained by pre-training can only be trained according to the known abnormal factors, timeliness is low, recognition accuracy for the multimedia data containing the novel abnormal factors is low, content quality of the multimedia data is reduced, and bad experience is brought to users.
Disclosure of Invention
The present disclosure provides a multimedia data processing method, an apparatus, an electronic device, and a storage medium, so as to solve at least the problems that a classification model obtained by pre-training in the related art can only be trained according to known abnormal factors, timeliness is low, identification accuracy for multimedia data including novel abnormal factors is low, content quality of the multimedia data is reduced, and bad experience is brought to a user. The technical scheme of the disclosure is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided a multimedia data processing method, including:
determining multimedia data meeting preset abnormal conditions in the multimedia data set as reference abnormal multimedia data;
determining the data characteristics of the reference abnormal multimedia data to obtain reference abnormal characteristics;
determining feature similarity between the reference abnormal feature and data features of multimedia data in the multimedia data set except the reference abnormal multimedia data;
and determining the multimedia data with the characteristic similarity larger than a similarity threshold as newly added reference abnormal multimedia data.
Optionally, the method further includes:
performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained by pre-training to obtain an anomaly detection result of the multimedia data, wherein the anomaly detection result represents the anomaly probability of the multimedia data;
and determining the multimedia data with the abnormal probability larger than a probability threshold value as abnormal multimedia data.
Optionally, the performing anomaly detection on the multimedia data in the multimedia data set by using the deep learning model obtained by pre-training to obtain an anomaly detection result of the multimedia data includes:
performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained by pre-training to obtain data characteristics of the multimedia data, and storing the data characteristics of the multimedia data into a preset storage space;
determining an abnormal detection result of the multimedia data according to the data characteristics;
the determining the feature similarity between the reference abnormal feature and the data feature of each multimedia data in the multimedia data set except the reference abnormal multimedia data comprises:
and determining the feature similarity between the reference abnormal feature and the data features of the multimedia data in the preset storage space except the reference abnormal multimedia data.
Optionally, the method further includes:
presetting the reference abnormal multimedia data and the abnormal multimedia data; the preset processing is used for reducing the reference abnormal multimedia data and the probability that the abnormal multimedia data is recommended.
Optionally, the determining the multimedia data meeting the preset abnormal condition in the multimedia data set as reference abnormal multimedia data includes:
acquiring operation behavior data corresponding to each multimedia data in a multimedia data set, and taking the multimedia data of which the operation behavior data meets preset abnormal conditions as reference abnormal multimedia data.
Optionally, the determining the multimedia data meeting the preset abnormal condition in the multimedia data set as reference abnormal multimedia data includes:
pushing the multimedia data meeting the preset abnormal conditions in the multimedia data set to a target terminal;
responding to the confirmation operation information of any multimedia data meeting the preset abnormal conditions by the target terminal, and taking any multimedia data meeting the preset abnormal conditions as reference abnormal multimedia data;
the determining the multimedia data with the feature similarity larger than the similarity threshold as the newly added reference abnormal multimedia data includes:
pushing the multimedia data with the characteristic similarity larger than a similarity threshold to the target terminal;
and responding to the confirmation operation information of the target terminal on any multimedia data with the characteristic similarity larger than the similarity threshold, and taking the multimedia data with the characteristic similarity larger than the similarity threshold as newly added reference abnormal multimedia data.
According to a second aspect of the embodiments of the present disclosure, there is provided a multimedia data processing apparatus including:
the first determining unit is configured to determine multimedia data meeting preset abnormal conditions in the multimedia data set as reference abnormal multimedia data;
a second determining unit configured to perform determining a data characteristic of the reference abnormal multimedia data, resulting in a reference abnormal characteristic;
a third determining unit configured to perform determining a feature similarity between the reference abnormal feature and a data feature of each multimedia data in the multimedia data set other than the reference abnormal multimedia data;
a fourth determining unit configured to perform determination of the multimedia data of which the feature similarity is greater than a similarity threshold as newly added reference abnormal multimedia data.
Optionally, the apparatus further comprises:
the detection unit is configured to perform anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained through pre-training to obtain an anomaly detection result of the multimedia data, wherein the anomaly detection result represents the anomaly probability of the multimedia data; and determining the multimedia data with the abnormal probability larger than a probability threshold value as abnormal multimedia data.
Optionally, the detecting unit is configured to perform:
performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained by pre-training to obtain data characteristics of the multimedia data, and storing the data characteristics of the multimedia data into a preset storage space;
determining an abnormal detection result of the multimedia data according to the data characteristics;
the third determination unit is configured to perform:
and determining the feature similarity between the reference abnormal feature and the data features of the multimedia data in the preset storage space except the reference abnormal multimedia data.
Optionally, the apparatus further comprises:
a processing unit configured to perform preset processing on the reference abnormal multimedia data and abnormal multimedia data; the preset processing is used for reducing the reference abnormal multimedia data and the probability that the abnormal multimedia data is recommended.
Optionally, the first determining unit is configured to perform:
acquiring operation behavior data corresponding to each multimedia data in a multimedia data set, and taking the multimedia data of which the operation behavior data meets preset abnormal conditions as reference abnormal multimedia data.
Optionally, the first determining unit is configured to perform:
pushing the multimedia data meeting the preset abnormal conditions in the multimedia data set to a target terminal;
responding to the confirmation operation information of any multimedia data meeting the preset abnormal conditions by the target terminal, and taking any multimedia data meeting the preset abnormal conditions as reference abnormal multimedia data;
the fourth determination unit configured to perform:
pushing the multimedia data with the characteristic similarity larger than a similarity threshold to the target terminal;
and responding to the confirmation operation information of the target terminal on any multimedia data with the characteristic similarity larger than the similarity threshold, and taking the multimedia data with the characteristic similarity larger than the similarity threshold as newly added reference abnormal multimedia data.
According to a third aspect of the embodiments of the present disclosure, there is provided a multimedia data processing electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the multimedia data processing method of the first item.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, wherein instructions, when executed by a processor of a multimedia data processing electronic device, enable the multimedia data processing electronic device to perform the multimedia data processing method of the first item.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising computer programs/instructions which, when executed by a processor, implement the multimedia data processing method of the first item described above.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
determining multimedia data meeting preset abnormal conditions in the multimedia data set as reference abnormal multimedia data; determining data characteristics of the reference abnormal multimedia data to obtain reference abnormal characteristics; determining feature similarity between the reference abnormal features and data features of the multimedia data in the multimedia data set except the reference abnormal multimedia data; and determining the multimedia data with the characteristic similarity larger than the similarity threshold as the newly added reference abnormal multimedia data.
Therefore, after one multimedia data is determined as the reference abnormal multimedia data, other multimedia data with higher similarity to the characteristics of the multimedia data can be quickly inquired according to the data characteristics of the multimedia data, more reference abnormal multimedia data are determined, the screening process of the reference abnormal multimedia data is more efficient and quick, and then the pushing of the multimedia data can be quickly adjusted according to the reference abnormal multimedia data, the content quality of the multimedia data is improved, and better experience is brought to a user.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
Fig. 1 is a flow chart illustrating a multimedia data processing method according to an exemplary embodiment.
Fig. 2 is a schematic diagram illustrating a multimedia data processing method according to an exemplary embodiment.
Fig. 3 is a block diagram illustrating a multimedia data processing apparatus according to an example embodiment.
FIG. 4 is a block diagram illustrating an electronic device for multimedia data processing in accordance with an exemplary embodiment.
Fig. 5 is a block diagram illustrating an apparatus for multimedia data processing according to an example embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Fig. 1 is a flowchart illustrating a multimedia data processing method according to an exemplary embodiment, as shown in fig. 1, the multimedia data processing method including:
in step S11, multimedia data satisfying a predetermined abnormal condition in the multimedia data set is determined as reference abnormal multimedia data.
In the present disclosure, the multimedia data may be video data, audio data or image data, and the like, and is not limited specifically. The multimedia data set is a set of multimedia data that needs to be filtered, and the multimedia data set may be stored in a database, where the multimedia data set may include some reference abnormal multimedia data, and the reference abnormal multimedia data is multimedia data that is not suitable for dissemination or pushing, such as multimedia data that includes non-compliance information or multimedia data that easily causes complaints, and so on.
In one implementation mode, a deep learning model obtained through pre-training can be used for carrying out anomaly detection on multimedia data in a multimedia data set to obtain an anomaly detection result of the multimedia data, and the anomaly detection result represents the anomaly probability of the multimedia data; then, the multimedia data with the abnormal probability larger than the probability threshold is determined as abnormal multimedia data.
The abnormal multimedia data and the reference abnormal multimedia data may be multimedia data with different abnormal factors, and generally, the abnormal reason of the abnormal multimedia data is common or the occurrence frequency is high, so that the abnormal detection accuracy of the deep learning model obtained by training is high, and therefore, a part of abnormal multimedia data can be screened out from the multimedia data set through the deep learning model trained in advance, and the processing efficiency of the multimedia data is improved.
The deep learning model can construct a reasonable abnormal label in advance according to the priori knowledge of the abnormal multimedia data, collect corresponding training data, and then train through the collected training data, wherein after the multimedia data in the multimedia data set is subjected to abnormal detection, the higher the abnormal probability of the multimedia data is, the more likely the multimedia data is to be abnormal multimedia data.
The deep learning model can be a multi-mode model, the multi-mode model is utilized to analyze the multimedia data, the data features of the obtained multimedia data are discrete numerical vectors obtained by model learning, corresponding physical meanings do not exist, and the whole feature vector is sparse feature representation of the multimedia data.
In this step, the reference abnormal multimedia data may be determined according to analysis of operation behavior data of the multimedia data, for example, operation behavior data corresponding to any multimedia data may be obtained, and the multimedia data whose operation behavior data satisfies a preset abnormal condition is used as the reference abnormal multimedia data. For example, if a certain multimedia data is consumed in a relatively short time, the multimedia data is considered to be abnormal, and it is highly possible to refer to the abnormal multimedia data.
In addition, in an implementation manner, after the reference abnormal multimedia data is determined, the target terminal may perform auditing on the reference abnormal multimedia data, for example, the multimedia data that satisfies the preset abnormal condition in the multimedia data set may be pushed to the target terminal, and then, in response to the confirmation operation information of the target terminal on any multimedia data that satisfies the preset abnormal condition, any multimedia data that satisfies the preset abnormal condition is taken as the reference abnormal multimedia data. Therefore, through auditing, the accuracy of the confirmed reference abnormal multimedia data is higher, and the possibility of error screening is reduced.
In step S12, the data characteristics of the reference abnormal multimedia data are determined, resulting in reference abnormal characteristics.
In this step, after the reference abnormal multimedia data is determined, the determined reference abnormal multimedia data may be further subjected to feature recognition, and a data feature of the reference abnormal multimedia data, that is, a reference abnormal data feature, may be extracted.
It is understood that multimedia data with similar data characteristics are likely to include similar content, and therefore, with the reference abnormal characteristics, multimedia data similar to the reference abnormal multimedia data, which is also likely to have abnormal factors similar to the reference abnormal multimedia data, can be further screened from the multimedia data set.
In step S13, a feature similarity between the reference abnormal feature and a data feature of each multimedia data in the multimedia data set other than the reference abnormal multimedia data is determined.
The feature similarity may be embodied by a feature distance, and generally, the feature distance is inversely proportional to the feature similarity, that is, the greater the feature distance between the data features of the two multimedia data is, the smaller the feature similarity between the two multimedia data is, and conversely, the smaller the feature distance between the data features of the two multimedia data is, the greater the feature similarity between the two multimedia data is. The characteristic distance may be an euclidean distance, a cosine distance, or the like, and is not particularly limited.
In one implementation, for multimedia data in a multimedia data set, anomaly detection is performed by using a deep learning model obtained through pre-training, and the anomaly detection often depends on data characteristics of the multimedia data, so that the data characteristics of the multimedia data in the multimedia data set can be stored in the anomaly detection process, and are directly called when the feature similarity is calculated in the step. Therefore, the data characteristics are calculated for each multimedia data only once, the calculation amount can be reduced, the retrieval efficiency is improved, and the processing of the multimedia data is more time-efficient.
Specifically, in the anomaly detection process, firstly, carrying out anomaly detection on multimedia data in a multimedia data set by using a deep learning model obtained by pre-training to obtain data characteristics of the multimedia data, and storing the data characteristics of the multimedia data into a preset storage space; then, according to the data characteristics, determining an abnormal detection result of the multimedia data; furthermore, in this step, a feature similarity between the reference abnormal feature and a data feature of each multimedia data in the preset storage space except the reference abnormal multimedia data may be determined.
In step S14, the multimedia data with the feature similarity greater than the similarity threshold is determined as the newly added reference abnormal multimedia data.
In this step, the similarity threshold may be a specific value, for example, when the feature distance between the data feature of any multimedia data and the reference abnormal data feature is less than 0.1, it is considered that the feature similarity between the data feature of the multimedia data and the reference abnormal data feature is greater than the similarity threshold. Or, the similarity threshold may also be a range threshold, for example, according to the ranking of the feature similarity, the top N multimedia data are used as reference abnormal multimedia data, where the value of N is any positive integer, and the like, which is not particularly limited
In one implementation, after determining the multimedia data with the feature similarity greater than the similarity threshold, the target terminal may audit the multimedia data with the feature similarity greater than the similarity threshold.
For example, the multimedia data with the feature similarity greater than the similarity threshold may be pushed to the target terminal, and then, in response to the confirmation operation information of the target terminal on the multimedia data with the feature similarity greater than the similarity threshold, the multimedia data with the feature similarity greater than the similarity threshold may be used as the newly added reference abnormal multimedia data. Therefore, through the audit, the accuracy of the newly added reference abnormal multimedia data is higher, and the possibility of error screening is further reduced.
In one implementation, after determining the reference abnormal multimedia data, the reference abnormal multimedia data may be subjected to preset processing, meanwhile, the multimedia data in the multimedia data set is subjected to abnormality detection, and after determining the abnormal multimedia data from the multimedia data set, the abnormal multimedia data may also be subjected to the same preset processing, where the preset processing is used to reduce the probability that the reference abnormal multimedia data and the abnormal multimedia data are recommended, so as to improve the content quality of the multimedia data that is pushed to the target terminal.
For example, the preset processing may be a weight reduction or filtering processing on the reference abnormal multimedia data and the abnormal multimedia data, so as to reduce the frequency of pushing the reference abnormal multimedia data to the target terminal, or the preset processing may also be a masking or deleting processing on the reference abnormal multimedia data from the multimedia data set, and the like, which is not limited specifically.
Fig. 2 is a schematic diagram of a multimedia data processing method, in which multimedia data is video data, and a multimedia data set is a search video library storing a plurality of videos.
On one hand, for videos in a searched video library, on the one hand, the videos in the searched video library can be subjected to anomaly detection through a pre-trained deep learning model, namely a multi-modal model, so that the anomaly probability of the videos is obtained, and the videos with the anomaly probability higher than a probability threshold can be regarded as anomalous videos, namely anomalous multimedia data. Furthermore, preset operations can be executed on the videos, and the possibility of pushing abnormal videos to the target terminal is reduced. In addition, based on the data characteristics of the video output by the multi-modal model, a characteristic retrieval system can be constructed for subsequent collision with the data characteristics of the reference abnormal video.
On the other hand, videos meeting preset abnormal conditions can be determined from a search video library according to the operation behavior data of each video, then the target terminal examines the videos meeting the preset abnormal conditions, a reference abnormal video, namely reference abnormal multimedia data, is determined based on the confirmation operation information of the target terminal, after the reference abnormal multimedia data are determined, preset operations are performed on the videos, and the possibility of pushing the reference abnormal video to the target terminal is reduced. And meanwhile, determining reference abnormal features of the reference abnormal videos, performing collision of data features in a feature retrieval system, determining feature similarity between the reference abnormal features and the data features of the videos, and taking the videos with the feature similarity larger than a similarity threshold value as new reference abnormal videos.
In addition, for videos for which the target terminal does not return confirmation operation information or videos for which the feature similarity is not greater than the similarity threshold, it can be considered that the videos are not reference abnormal videos or abnormal videos, the videos are exported from the data processing system, and the videos can be pushed normally without performing preset operations on the videos.
As can be seen from the above, according to the technical scheme provided by the embodiment of the disclosure, after one multimedia data is determined as the reference abnormal multimedia data, other multimedia data with higher similarity to the characteristic of the multimedia data can be quickly queried according to the data characteristic of the multimedia data, and more reference abnormal multimedia data can be determined.
Fig. 3 is a block diagram illustrating a multimedia data processing apparatus according to an exemplary embodiment, the apparatus comprising:
a first determining unit 201 configured to perform determining multimedia data satisfying a preset abnormal condition in the multimedia data set as reference abnormal multimedia data;
a second determining unit 202, configured to perform determining a data characteristic of the reference abnormal multimedia data, resulting in a reference abnormal characteristic;
a third determining unit 203 configured to perform determining a feature similarity between the reference abnormal feature and a data feature of each multimedia data in the multimedia data set except the reference abnormal multimedia data;
a fourth determining unit 204 configured to perform determining the multimedia data with the feature similarity greater than the similarity threshold as the newly added reference abnormal multimedia data.
In one implementation, the apparatus further includes:
the detection unit is configured to perform anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained through pre-training to obtain an anomaly detection result of the multimedia data, wherein the anomaly detection result represents the anomaly probability of the multimedia data; and determining the multimedia data with the abnormal probability larger than a probability threshold value as abnormal multimedia data.
In one implementation, the detection unit is configured to perform:
performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained by pre-training to obtain data characteristics of the multimedia data, and storing the data characteristics of the multimedia data into a preset storage space;
determining an abnormal detection result of the multimedia data according to the data characteristics;
the third determining unit 203 is configured to perform:
and determining the feature similarity between the reference abnormal feature and the data features of the multimedia data in the preset storage space except the reference abnormal multimedia data.
In one implementation, the apparatus further includes:
a processing unit configured to perform preset processing on the reference abnormal multimedia data and abnormal multimedia data; the preset processing is used for reducing the reference abnormal multimedia data and the probability that the abnormal multimedia data is recommended.
In one implementation, the first determining unit 201 is configured to perform:
acquiring operation behavior data corresponding to each multimedia data in a multimedia data set, and taking the multimedia data of which the operation behavior data meets preset abnormal conditions as reference abnormal multimedia data.
In one implementation, the first determining unit 201 is configured to perform:
pushing the multimedia data meeting the preset abnormal conditions in the multimedia data set to a target terminal;
responding to the confirmation operation information of any multimedia data meeting the preset abnormal conditions by the target terminal, and taking any multimedia data meeting the preset abnormal conditions as reference abnormal multimedia data;
the fourth determining unit 204 is configured to perform:
pushing the multimedia data with the characteristic similarity larger than a similarity threshold to the target terminal;
and responding to the confirmation operation information of the target terminal on any multimedia data with the characteristic similarity larger than the similarity threshold, and taking the multimedia data with the characteristic similarity larger than the similarity threshold as newly added reference abnormal multimedia data.
As can be seen from the above, in the scheme provided in the embodiment of the present disclosure, after a multimedia data is determined as a reference abnormal multimedia data, other multimedia data with a higher similarity to the characteristic of the multimedia data can be quickly queried according to the data characteristic of the multimedia data, and more reference abnormal multimedia data can be determined.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
FIG. 4 is a block diagram illustrating an electronic device for multimedia data processing in accordance with an exemplary embodiment.
In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as a memory comprising instructions, executable by a processor of an electronic device to perform the above-described method is also provided. Alternatively, the computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, there is also provided a computer program product which, when run on a computer, causes the computer to implement the method of multimedia data processing described above.
As can be seen from the above, in the scheme provided in the embodiment of the present disclosure, after a multimedia data is determined as a reference abnormal multimedia data, other multimedia data with a higher similarity to the characteristic of the multimedia data can be quickly queried according to the data characteristic of the multimedia data, and more reference abnormal multimedia data can be determined.
Fig. 5 is a block diagram illustrating an apparatus 800 for multimedia data processing according to an example embodiment.
For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast electronic device, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 5, the apparatus 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.
The processing component 802 generally controls overall operation of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operation at the device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
Power supply components 807 provide power to the various components of device 800. The power components 807 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the apparatus 800.
The multimedia component 808 includes a screen that provides an output interface between the device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data when the device 800 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the apparatus 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the device 800. For example, the sensor assembly 814 may detect the open/closed state of the device 800, the relative positioning of the components, such as a display and keypad of the apparatus 800, the sensor assembly 814 may also detect a change in position of the apparatus 800 or a component of the apparatus 800, the presence or absence of user contact with the apparatus 800, orientation or acceleration/deceleration of the apparatus 800, and a change in temperature of the apparatus 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 816 is configured to facilitate communications between the apparatus 800 and other devices in a wired or wireless manner. The apparatus 800 may access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G, or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the methods of the first and second aspects.
In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 820 of the device 800 to perform the above-described method is also provided. Alternatively, for example, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the multimedia data processing method described in the first of the above embodiments.
As can be seen from the above, in the scheme provided in the embodiment of the present disclosure, after a multimedia data is determined as a reference abnormal multimedia data, other multimedia data with a higher similarity to the characteristic of the multimedia data can be quickly queried according to the data characteristic of the multimedia data, and more reference abnormal multimedia data can be determined.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A method of multimedia data processing, the method comprising:
determining multimedia data meeting preset abnormal conditions in the multimedia data set as reference abnormal multimedia data;
determining the data characteristics of the reference abnormal multimedia data to obtain reference abnormal characteristics;
determining feature similarity between the reference abnormal feature and data features of multimedia data in the multimedia data set except the reference abnormal multimedia data;
and determining the multimedia data with the characteristic similarity larger than a similarity threshold as newly added reference abnormal multimedia data.
2. The method of claim 1, further comprising:
performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained by pre-training to obtain an anomaly detection result of the multimedia data, wherein the anomaly detection result represents the anomaly probability of the multimedia data;
and determining the multimedia data with the abnormal probability larger than a probability threshold value as abnormal multimedia data.
3. The method according to claim 2, wherein the performing anomaly detection on the multimedia data in the multimedia data set by using the deep learning model obtained by pre-training to obtain an anomaly detection result of the multimedia data comprises:
performing anomaly detection on the multimedia data in the multimedia data set by using a deep learning model obtained by pre-training to obtain data characteristics of the multimedia data, and storing the data characteristics of the multimedia data into a preset storage space;
determining an abnormal detection result of the multimedia data according to the data characteristics;
the determining the feature similarity between the reference abnormal feature and the data feature of each multimedia data in the multimedia data set except the reference abnormal multimedia data comprises:
and determining the feature similarity between the reference abnormal feature and the data features of the multimedia data in the preset storage space except the reference abnormal multimedia data.
4. The method of claim 2, further comprising:
presetting the reference abnormal multimedia data and the abnormal multimedia data; the preset processing is used for reducing the reference abnormal multimedia data and the probability that the abnormal multimedia data is recommended.
5. The method as claimed in claim 1, wherein the determining the multimedia data satisfying the predetermined abnormal condition in the multimedia data set as the reference abnormal multimedia data comprises:
acquiring operation behavior data corresponding to each multimedia data in a multimedia data set, and taking the multimedia data of which the operation behavior data meets preset abnormal conditions as reference abnormal multimedia data.
6. The method as claimed in claim 1, wherein the determining the multimedia data satisfying the predetermined abnormal condition in the multimedia data set as the reference abnormal multimedia data comprises:
pushing the multimedia data meeting the preset abnormal conditions in the multimedia data set to a target terminal;
responding to the confirmation operation information of any multimedia data meeting the preset abnormal conditions by the target terminal, and taking any multimedia data meeting the preset abnormal conditions as reference abnormal multimedia data;
the determining the multimedia data with the feature similarity larger than the similarity threshold as the newly added reference abnormal multimedia data includes:
pushing the multimedia data with the characteristic similarity larger than a similarity threshold to the target terminal;
and responding to the confirmation operation information of the target terminal on any multimedia data with the characteristic similarity larger than the similarity threshold, and taking the multimedia data with the characteristic similarity larger than the similarity threshold as newly added reference abnormal multimedia data.
7. A multimedia data processing system apparatus, comprising:
the first determining unit is configured to determine multimedia data meeting preset abnormal conditions in the multimedia data set as reference abnormal multimedia data;
a second determining unit configured to perform determining a data characteristic of the reference abnormal multimedia data, resulting in a reference abnormal characteristic;
a third determining unit configured to perform determining a feature similarity between the reference abnormal feature and a data feature of each multimedia data in the multimedia data set other than the reference abnormal multimedia data;
a fourth determining unit configured to perform determination of the multimedia data of which the feature similarity is greater than a similarity threshold as newly added reference abnormal multimedia data.
8. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the multimedia data processing method of any of claims 1 to 6.
9. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of a multimedia data processing electronic device, enable the multimedia data processing electronic device to perform the multimedia data processing method of any of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the multimedia data processing method of any of claims 1 to 6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111603582.9A CN114338587B (en) | 2021-12-24 | 2021-12-24 | Multimedia data processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111603582.9A CN114338587B (en) | 2021-12-24 | 2021-12-24 | Multimedia data processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114338587A true CN114338587A (en) | 2022-04-12 |
| CN114338587B CN114338587B (en) | 2024-03-12 |
Family
ID=81012704
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111603582.9A Active CN114338587B (en) | 2021-12-24 | 2021-12-24 | Multimedia data processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114338587B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111047049A (en) * | 2019-12-05 | 2020-04-21 | 北京小米移动软件有限公司 | Method, apparatus and medium for processing multimedia data based on machine learning model |
| CN111258435A (en) * | 2020-01-15 | 2020-06-09 | 北京达佳互联信息技术有限公司 | Multimedia resource commenting method and device, electronic equipment and storage medium |
| CN111858973A (en) * | 2020-07-30 | 2020-10-30 | 北京达佳互联信息技术有限公司 | Multimedia event information detection method, device, server and storage medium |
| CN111949808A (en) * | 2019-04-29 | 2020-11-17 | 北京达佳互联信息技术有限公司 | Multimedia content similarity determination method and device, electronic equipment and storage medium |
| CN112597864A (en) * | 2020-12-16 | 2021-04-02 | 佳都新太科技股份有限公司 | Monitoring video abnormity detection method and device |
| US20210264209A1 (en) * | 2020-02-24 | 2021-08-26 | MakinaRocks Co., Ltd. | Method for generating anomalous data |
| US20210365714A1 (en) * | 2020-05-21 | 2021-11-25 | International Business Machines Corporation | Detecting Artificial Multimedia Content Using Deep Network Response Analysis |
-
2021
- 2021-12-24 CN CN202111603582.9A patent/CN114338587B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111949808A (en) * | 2019-04-29 | 2020-11-17 | 北京达佳互联信息技术有限公司 | Multimedia content similarity determination method and device, electronic equipment and storage medium |
| CN111047049A (en) * | 2019-12-05 | 2020-04-21 | 北京小米移动软件有限公司 | Method, apparatus and medium for processing multimedia data based on machine learning model |
| CN111258435A (en) * | 2020-01-15 | 2020-06-09 | 北京达佳互联信息技术有限公司 | Multimedia resource commenting method and device, electronic equipment and storage medium |
| US20210218696A1 (en) * | 2020-01-15 | 2021-07-15 | Beijing Dajia Internet Information Technology Co., Ltd. | Method and device for commenting on multimedia resource |
| US20210264209A1 (en) * | 2020-02-24 | 2021-08-26 | MakinaRocks Co., Ltd. | Method for generating anomalous data |
| US20210365714A1 (en) * | 2020-05-21 | 2021-11-25 | International Business Machines Corporation | Detecting Artificial Multimedia Content Using Deep Network Response Analysis |
| CN111858973A (en) * | 2020-07-30 | 2020-10-30 | 北京达佳互联信息技术有限公司 | Multimedia event information detection method, device, server and storage medium |
| CN112597864A (en) * | 2020-12-16 | 2021-04-02 | 佳都新太科技股份有限公司 | Monitoring video abnormity detection method and device |
Non-Patent Citations (1)
| Title |
|---|
| 李昊奇;应娜;郭春生;王金华;: "基于深度信念网络和线性单分类SVM的高维异常检测", 电信科学, no. 01 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114338587B (en) | 2024-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109446994B (en) | Gesture key point detection method and device, electronic equipment and storage medium | |
| CN108038102B (en) | Method and device for recommending expression image, terminal and storage medium | |
| CN111539443A (en) | Image recognition model training method and device and storage medium | |
| EP3767488A1 (en) | Method and device for processing untagged data, and storage medium | |
| CN110781323A (en) | Method and device for determining label of multimedia resource, electronic equipment and storage medium | |
| CN113676671B (en) | Video editing method, device, electronic equipment and storage medium | |
| CN111428032B (en) | Content quality evaluation method and device, electronic equipment and storage medium | |
| CN111046927A (en) | Method and device for processing labeled data, electronic equipment and storage medium | |
| CN112948704A (en) | Model training method and device for information recommendation, electronic equipment and medium | |
| CN113779257A (en) | Method, device, equipment, medium and product for analyzing text classification model | |
| CN111813932B (en) | Text data processing method, text data classifying device and readable storage medium | |
| CN110738267B (en) | Image classification method, device, electronic equipment and storage medium | |
| CN105551047A (en) | Picture content detecting method and device | |
| CN111125388B (en) | Method, device and equipment for detecting multimedia resources and storage medium | |
| CN110941727B (en) | Resource recommendation method and device, electronic equipment and storage medium | |
| CN112328809A (en) | Entity classification method, device and computer readable storage medium | |
| CN111859097B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN107301188B (en) | Method for acquiring user interest and electronic equipment | |
| CN114338587B (en) | Multimedia data processing method and device, electronic equipment and storage medium | |
| CN114547421A (en) | Search processing method and device, electronic equipment and storage medium | |
| CN112884040A (en) | Training sample data optimization method and system, storage medium and electronic equipment | |
| CN109145151B (en) | Video emotion classification acquisition method and device | |
| CN113761275A (en) | Video preview moving picture generation method, device and equipment and readable storage medium | |
| CN115225702B (en) | Information pushing method and device, electronic equipment and storage medium | |
| CN111428806B (en) | Image tag determining method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |