CN111460182A

CN111460182A - Multimedia evaluation method and device

Info

Publication number: CN111460182A
Application number: CN202010306932.4A
Authority: CN
Inventors: 涂畅
Original assignee: Alipay Hangzhou Information Technology Co Ltd
Current assignee: Alipay Hangzhou Information Technology Co Ltd
Priority date: 2020-04-17
Filing date: 2020-04-17
Publication date: 2020-07-28

Abstract

The present specification provides a multimedia evaluation method and apparatus, wherein the multimedia evaluation method comprises: acquiring an image set formed by image frames acquired by an image acquisition module aiming at a visible area of multimedia data; inputting each image frame contained in the image set into an identification model for user identification to obtain the identified watching data of the user in the image frame; determining an evaluation result for the multimedia data based on the viewing data and attribute data of the multimedia data in various attribute dimensions.

Description

Multimedia evaluation method and device

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to a multimedia evaluation method and apparatus.

Background

With the development of internet technology, multimedia data recommendation modes are more and more diversified, and in order to make recommendation effects better, a multimedia data delivery party generally delivers a large amount of multimedia data in places where users often appear; for example, the advertisement is pushed through the advertisement space in the subway station, the railway station or the market; although the effect of pushing multimedia data is achieved at this time, the number of users that can be achieved in effect is not ideal, and since the multimedia data is a large expense both in terms of manufacturing cost and pushing cost for the delivering party of the multimedia data, if the final delivering effect does not achieve the intended purpose, a great loss will be caused for the delivering party, so a scheme capable of detecting the delivering effect of the multimedia data is urgently needed.

Disclosure of Invention

In view of this, the embodiments of the present disclosure provide a multimedia evaluation method. The present specification also relates to a multimedia evaluation apparatus, a computing device, and a computer-readable storage medium to solve the technical problems of the prior art.

According to a first aspect of embodiments of the present specification, there is provided a multimedia evaluation method including:

acquiring an image set formed by image frames acquired by an image acquisition module aiming at a visible area of multimedia data;

inputting each image frame contained in the image set into an identification model for user identification to obtain the identified watching data of the user in the image frame;

determining an evaluation result for the multimedia data based on the viewing data and attribute data of the multimedia data in various attribute dimensions.

Optionally, the identifying model performs user identification, including:

identifying a face region in each image frame;

determining a watching user set consisting of watching users according to the face angles of the users in the face area;

and determining the watching duration corresponding to the watching user based on the image frame to which the watching user belongs, and counting the number of users of the watching users in the watching user set.

Optionally, the determining an evaluation result for the multimedia data based on the viewing data and the attribute data of the multimedia data in each attribute dimension includes:

acquiring equipment data of multimedia playing equipment for playing the multimedia data;

determining the attribute data of the multimedia data in each attribute dimension according to the equipment data;

performing primary evaluation on the multimedia data based on the attribute data to determine a first evaluation result, and performing secondary evaluation on the multimedia data based on the watching duration and the number of users to determine a second evaluation result;

and integrating the first evaluation result and the second evaluation result, and determining the evaluation result aiming at the multimedia data according to the integrated result.

Optionally, the identifying model performs user identification, including:

identifying a face region in each image frame;

determining a watching user set consisting of watching users and an unviewed user set consisting of unviewed users according to the face angles of the users in the face area;

and counting the number of watching users of the watching user set and the number of non-watching users of the non-watching user set.

and evaluating the multimedia data based on the attribute data, the number of the watching users and the number of the non-watching users to obtain the evaluation result.

Optionally, before the step of acquiring an image set composed of image frames captured by the image capturing module for a visible region of multimedia data is executed, the method further includes:

and displaying the multimedia data according to a preset time period.

Optionally, after the step of determining an evaluation result for the multimedia data based on the viewing data and the attribute data of the multimedia data in each attribute dimension is executed, the method further includes:

detecting whether the multimedia data meet a replacement condition according to the evaluation result;

if so, selecting target multimedia data from a preset multimedia database for displaying; and the multimedia data contained in the multimedia database is prestored by the delivering party to which the multimedia data belongs.

sending the evaluation result to a delivery party delivering the multimedia data;

and under the condition that a replacement request aiming at the multimedia data uploading by the releasing party is received, replacing and displaying the multimedia data according to target multimedia data carried in the replacement request.

Optionally, the multimedia evaluation method is applied to a multimedia playing device, and correspondingly, the image acquisition module is configured in the multimedia playing device.

Optionally, the training process of the recognition model includes:

acquiring a training image set, and carrying out face labeling on a user on training images contained in the training image set;

and taking the marked training image and the training image as training samples, and training the recognition model to be trained to obtain the recognition model.

Optionally, the attribute dimension includes at least one of:

a location dimension, a traffic dimension, a price dimension, a multimedia type dimension;

accordingly, the attribute data includes at least one of:

location data, traffic data, price data, multimedia type data.

According to a second aspect of embodiments of the present specification, there is provided a multimedia evaluation apparatus including:

the acquisition module is configured to acquire an image set formed by image frames acquired by the image acquisition module aiming at a visible area of the multimedia data;

the identification module is configured to input each image frame contained in the image set into an identification model for user identification, and obtain the identified watching data of a user in the image frame;

a determination module configured to determine an evaluation result for the multimedia data based on the viewing data and attribute data of the multimedia data in respective attribute dimensions.

According to a third aspect of embodiments herein, there is provided a computing device comprising:

a memory and a processor;

the memory is to store computer-executable instructions, and the processor is to execute the computer-executable instructions to:

According to a fourth aspect of embodiments herein, there is provided a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the multimedia evaluation method.

In the multimedia evaluation method provided in an embodiment of the present specification, after the multimedia data is delivered, in order to evaluate the delivery effect of the multimedia data, by obtaining an image set composed of image frames collected by an image collection module for a visible area of the multimedia data, and inputting each image frame into a recognition model for user recognition, obtaining viewing data of a user in the identified image frame, and then determining an evaluation result for the multimedia data based on the viewing data and attribute data of the multimedia data in each attribute dimension, the delivery effect of the multimedia data is evaluated by combining the viewing data and the attribute data of the user, the delivery effect of the multimedia data can be more intuitively reflected, and the accuracy of the evaluation of the multimedia data is effectively improved, therefore, the multimedia data can be effectively adjusted by the multimedia data releasing party according to the evaluation result.

Drawings

Fig. 1 is a flowchart of a multimedia evaluation method provided in an embodiment of the present specification;

FIG. 2 is a flowchart of a multimedia evaluation method applied in an advertisement evaluation scenario according to an embodiment of the present disclosure;

FIG. 3 is a schematic structural diagram of a multimedia evaluation apparatus according to an embodiment of the present disclosure;

fig. 4 is a block diagram of a computing device according to an embodiment of the present disclosure.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, as those skilled in the art will be able to make and use the present disclosure without departing from the spirit and scope of the present disclosure.

The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

In the present specification, a multimedia evaluation method is provided, and the present specification relates to a multimedia evaluation apparatus, a computing device, and a computer-readable storage medium, which are described in detail in the following embodiments one by one.

Fig. 1 is a flowchart illustrating a multimedia evaluation method according to an embodiment of the present disclosure, which specifically includes the following steps:

step 102, an image set composed of image frames acquired by an image acquisition module aiming at a visible area of multimedia data is acquired.

In practical application, after a multimedia delivery party delivers multimedia data to a certain multimedia playing device, in order to attract more users through the multimedia data, the multimedia delivery party selects to display and play in a specific time period or selects to deliver the multimedia data on a specific date, and the attracted users are in direct proportion to the delivery effect of the multimedia data, so that the delivery effect of the multimedia data needs to be accurately evaluated, the user conversion rate brought by the multimedia data can be further determined, and the user reaching rate of the multimedia data played by the multimedia playing device is improved.

In order to accurately evaluate the multimedia data delivery effect, the multimedia evaluation method provided in this embodiment obtains an image set composed of image frames acquired by an image acquisition module for a visual area of the multimedia data, inputs each image frame into an identification model for user identification, obtains viewing data of a user in the identified image frame, and then determines an evaluation result for the multimedia data based on the viewing data and attribute data of the multimedia data in each attribute dimension, so that the multimedia data delivery effect can be evaluated by combining the viewing data and the attribute data of the user, the multimedia data delivery effect can be more intuitively reflected, the accuracy of the multimedia data evaluation is effectively improved, and a delivery party of the multimedia data can effectively adjust the multimedia data according to the evaluation result, the experience effect of the throwing party is further improved.

In specific implementation, the multimedia evaluation method provided by this embodiment can implement the evaluation of multimedia data in an off-line state, and the multimedia evaluation method is applied to a multimedia playing device, and accordingly, the image acquisition module is configured in the multimedia playing device, and in addition, the multimedia playing device is also configured with modules such as a processor, a memory, and a database, so as to implement the evaluation of multimedia data; the multimedia playing device specifically refers to a device capable of playing or displaying multimedia data, such as a billboard, a display, and the like, capable of displaying or playing multimedia data; the image acquisition module is specifically a camera configured on the multimedia playing device, and the camera is aligned with a visible area of the multimedia playing device, such as a camera configured on a billboard or a display; the multimedia data specifically refers to picture data, video data or text data, such as picture advertisements, text advertisements or video advertisements.

Based on this, the embodiment describes the multimedia evaluation method with the multimedia data as an advertisement, and the multimedia playing device is a billboard for playing the advertisement; it should be noted that the advertisement related in this embodiment may be a video advertisement, a picture advertisement or a text advertisement, and correspondingly, the billboard is a device capable of displaying or playing the video advertisement, the picture advertisement or the text advertisement.

Specifically, the visible region of the multimedia data specifically refers to a region in which the multimedia data can be viewed, the image set is composed of at least one image frame acquired by an image acquisition module, the image frame specifically refers to an image acquired by the image acquisition module at a preset time interval for the visible region, where the preset time interval may be 10 seconds, 20 seconds, or 60 seconds, and the like, and the specific time interval may be set according to an actual application scene, which is not limited herein;

further, in order to realize more accurate evaluation of the advertisement delivery effect, at the moment, image frame collection is carried out on the advertisement visual area through a preset time interval to form an image set, and then all image frames contained in the image set are processed, so that the advertisement delivery effect is accurately evaluated through combination of user watching data in the image frames and attribute data of the advertisement.

In addition, before acquiring an image set composed of image frames, the multimedia data also needs to be played by a multimedia playing device, and in order to avoid causing excessive resource waste, a playing time period may be preset to achieve an effect of saving resources, in one or more embodiments of this embodiment, specific implementation manners are as follows:

and displaying the multimedia data according to a preset time period.

In practical application, before the multimedia data is displayed, a display time period needs to be set according to practical conditions, and then the multimedia data is displayed according to a preset time period when the multimedia data is displayed, wherein the preset time period can be set according to practical application scenes, for example, a billboard is installed in a subway station, and the display time period of the advertisement can be set to be 5 every day: 00 to 23: 00, and the billboard can be closed at other time, thus saving resources and prolonging the service life of the billboard.

And 104, inputting each image frame contained in the image set into an identification model for user identification, and obtaining the viewing data of the user in the identified image frame.

Specifically, under the condition that the image set composed of the image frames is obtained, if the number of the image frames acquired by the image acquisition module at the time meets the evaluation condition for evaluating the multimedia data, the image frames composing the image set are input into an identification model for user identification, and the viewing data of the user in the image frames identified by the identification model is obtained; the viewing data may be a viewing duration of a user, a number of viewing users, a number of non-viewing users, or a total number of users corresponding to the image set.

Wherein, the training process of the recognition model comprises the following steps: acquiring a training image set, and carrying out face labeling on a user on training images contained in the training image set; and taking the marked training image and the training image as training samples, and training the recognition model to be trained to obtain the recognition model.

Furthermore, in the subsequent multimedia data evaluation process, the first aspect can be realized by combining attribute data according to the watching duration of the watching users and the number of the users of the watching users; the second aspect can be implemented by combining attribute data according to the number of viewing users and the number of non-viewing users;

based on this, if the evaluation of the multimedia data needs to be implemented through the first aspect, the viewing duration and the number of users viewing the users related to the first aspect need to be identified through the identification model, and in one or more embodiments of this embodiment, specific implementation manners are as follows:

identifying a face region in each image frame;

In practical application, the face region specifically refers to a region corresponding to a face of a user in each image frame, the face angle specifically refers to an angle existing between the face contained in the image frame and a multimedia playing device displaying multimedia data, the watching user specifically refers to a user watching the multimedia data, and the watching duration specifically refers to a time length of each watching user watching the multimedia data;

based on the above, after each image frame is input into the recognition model, the recognition model firstly recognizes a face region in each image frame, wherein the face region comprises a region corresponding to a face of a user watching multimedia data and a region corresponding to a face of a user not watching multimedia data, secondly selects watching users according to face angles of the users in the face region, forms a watching user set, and finally determines watching duration of each watching user based on the image frame to which the watching users belong, and simultaneously counts the number of the users watching users in the watching user set for subsequent multimedia data putting effect evaluation.

For example, a company puts an advertisement for a certain product under the flag on a billboard numbered 301 in a subway station a, and a camera terminal is arranged on the billboard 301, so that the effect of putting the advertisement can be evaluated, the camera terminal is aligned to a visible area of the billboard 301, and is set to collect one frame of image every 1 minute, and the time interval for collecting the image is 5: 00 to 23: 00;

based on this, 1140 image frames are collected after one day of collection, the advertisement is evaluated for effectiveness, 1140 image frames are input into a recognition model for user recognition to determine the viewing data of the user, and the specific recognition process of the recognition model comprises: 11400 face regions are identified in 1140 image frames, then face angles of all users in the 11400 personal face regions are detected, it is determined that 400 face angles in the 11400 personal face regions meet the standard for watching the billboard 301 according to the detection result, at this time, 400 watching users are determined, and then the watching duration of each watching user is determined according to the image frames to which the 400 watching users belong, so as to be used for evaluating the subsequent advertisement putting effect.

In practical application, the watching duration of the watching user is determined through the image frame to which the watching user belongs, specifically, the image frame to which the watching user belongs is firstly extracted, the image frame appearing for the first time of the watching user and the image frame appearing for the last time of the watching user are determined, secondly, the number of the image frames existing between the image frame appearing for the first time of the watching user and the image frame appearing for the last time is counted, and finally, the product of the time interval and the number of the image frames of each frame of image collected by the image collecting module can be used for determining the watching duration of the watching user.

In addition, the watching duration of the watching user may not be determined through the continuous image frames (the watching user has a fast moving speed, and the image acquisition module only acquires one frame of image corresponding to the watching user), so that the watching duration of the watching user may be determined to be 0 second; or the time interval for collecting the image frames by the image collecting module can be shortened appropriately, so that the watching time length of the watching user can be determined on the continuous multiframe images.

In conclusion, the number of users of the watching users and the watching duration of the watching users are identified through the identification model for subsequent evaluation of the multimedia data, the multimedia data launching effect can be intuitively reflected according to the watching data of the watching users, and the evaluation accuracy of the multimedia data evaluation is further improved.

Furthermore, if the evaluation of the multimedia data by the second aspect is required, the number of viewing users and the number of non-viewing users involved in the second aspect are required to be identified by the identification model, and in one or more embodiments of this embodiment, specific implementation manners are as follows:

identifying a face region in each image frame;

In practical application, after each image frame is input into the recognition model, the recognition model firstly recognizes a face region in each image frame, secondly selects watching users to form a watching user set and non-watching users to form a non-watching user set according to the face angle of the users in the face region, and finally counts the number of the watching users of the watching user set and the number of the non-watching users of the non-watching user set so as to be used for evaluating the subsequent multimedia data release effect.

Along with the above example, after the collection of image frames of one day, a total of 1140 image frames are collected, at this time, the advertisement will be evaluated for placement effect, and 1140 image frames are input into the recognition model for user recognition to determine the viewing data of the user, wherein the specific recognition process of the recognition model comprises: 11400 face regions are identified in 1140 image frames, then face angles of all users in the 11400 personal face regions are detected, it is determined that the face angles in the 400 personal face regions in the 11400 personal face regions meet the standard for watching the advertising board 301 according to the detection result, at this time, it is determined that 400 watching users form a watching user set, the face angles in the residual 11000 personal face regions do not meet the standard for watching the advertising board 301, it is determined that 11000 non-watching users form a non-watching user set, and then the advertisement delivery effect is evaluated according to the watching user set and the non-watching user set.

In conclusion, the number of watching users of the watching users and the number of non-watching users of the non-watching users are identified by the identification model for subsequent evaluation of the multimedia data, so that the evaluation accuracy of the multimedia data is further improved.

In addition, the recognition results of the recognition models used in the first aspect and the second aspect are different, so that different manners are adopted for training in the training process; the recognition model used in the first aspect will be trained by: acquiring a training image set, sequencing training images contained in the training image set according to an extraction sequence, then carrying out face labeling on the sequenced training images for watching users, adding a time length label to the face of the same watching user in the training images, forming a training sample by the training images and the labeled training images added with the time length label, and training a recognition model to be trained, so as to obtain the recognition model used in the first aspect;

the recognition model used in the second aspect will be trained by: acquiring a training image set, carrying out user face labeling on training images contained in the training image set, adding a watching user label to a watching user, adding a non-watching user label to a non-watching user, then forming a training sample by adding the watching user label and the training image added with the non-watching user label through the training images, and training a recognition model to be trained, so that the recognition model used in the second aspect can be obtained; namely: based on the first aspect and the second aspect, the recognition models are trained in different ways during the training process, so that the recognition results of the recognition model used in the first aspect and the recognition model used in the second aspect are different.

Step 106, determining an evaluation result for the multimedia data based on the viewing data and the attribute data of the multimedia data in each attribute dimension.

Specifically, on the basis of the above-mentioned viewing data of the user output by the recognition model, further, an evaluation result for the multimedia data is determined based on the viewing data and attribute data of the multimedia data in each attribute dimension.

Based thereon, the attribute dimension includes at least one of: a location dimension, a traffic dimension, a price dimension, a multimedia type dimension; correspondingly, the attribute data of the location dimension is location data, the attribute data of the traffic dimension is traffic data, the attribute data of the price dimension is price data, and the attribute data of the multimedia type dimension is multimedia type data.

The position data specifically refers to the position of a multimedia playing device displaying the multimedia data; the traffic data specifically refers to the passenger flow of the position where the multimedia playing equipment for displaying the multimedia data is located; the price data specifically refers to the lease price of the multimedia playing equipment displaying the multimedia data; the multimedia type data specifically refers to a type of displaying the multimedia data.

In the specific implementation process, the advertisement putting effect is not only influenced by the advertisement content, but also influenced by the positions of the advertising boards for displaying the advertisements, the price of the rental advertising boards, the passenger flow of the positions of the advertising boards and the like, so that the advertisement putting effect is evaluated by combining the number of watching users, the advertisement putting effect can be evaluated by combining data of other dimensions, and the result can be accurately determined and evaluated.

In practical application, the evaluation result may represent the advertisement delivery effect through a statistical graph or a comparison graph; in addition, transverse comparison can be carried out by obtaining the advertisement putting effect of the same type of advertisement and evaluating the advertisement putting effect, and the evaluation result of the evaluation advertisement is reflected according to the comparison result.

As described above, in the first aspect, on the basis that the viewing data of the user includes the viewing duration and the number of users of the viewing user through the identification model, further, in order to evaluate the multimedia data more accurately, the multimedia data is evaluated in combination with the attribute data of the multimedia data, in one or more embodiments of this embodiment, the specific implementation manner is as follows:

Specifically, the device data specifically refers to data related to the multimedia playing device, where the device data may include position data of a position where the multimedia playing device is located, data of a type of multimedia data that can be played by the multimedia playing device, passenger flow volume data of the position where the multimedia playing device is located, and the like;

based on this, after the device data of the multimedia playing device is obtained, the attribute data of the multimedia data in each attribute dimension can be determined, then, the multimedia data is primarily evaluated based on the attribute data, a first evaluation result is determined, meanwhile, the multimedia data is secondarily evaluated based on the viewing duration and the number of users, a second evaluation result is determined, and finally, the evaluation result for the multimedia data can be determined by integrating the first evaluation result and the second evaluation result.

Following the above example, in the case of determining the viewing users who view the billboard 301 and the viewing durations of 400 viewing users, the advertisements will be evaluated in combination with the attribute data of the advertisements, and the device data of the billboard 301 is first obtained, and the location data is determined: subway station, flow data: 15000 persons/day, price data: 178/day, billboard type data: determining attribute data of the advertisement according to the equipment data of the billboard; secondly, carrying out primary evaluation on the advertisement according to the attribute data, determining that a first evaluation result of the advertisement is high quality, and simultaneously carrying out secondary evaluation on the advertisement according to the number of watching users 400 and the watching duration of each watching user, and determining that a second evaluation result of the advertisement is good; and finally integrating the first evaluation result and the second evaluation result to determine that the evaluation result aiming at the advertisement is high quality, and determining that the advertisement putting effect is good.

In summary, in the process of evaluating the multimedia data, the multimedia data is firstly evaluated primarily according to the attribute data, secondly evaluated secondarily according to the viewing duration and the number of users, and finally a final evaluation result of the multimedia data is determined by integrating a primary evaluation result and a secondary evaluation result, so that the multimedia data can be evaluated more accurately, and a subsequent delivery party delivering the multimedia data can conveniently adjust the multimedia data according to the evaluation result.

As described above, in the second aspect, on the basis of identifying the number of viewing users and the number of non-viewing users through the identification model, further, in order to evaluate the multimedia data more accurately, the multimedia data is evaluated in combination with attribute data of the multimedia data, in one or more embodiments of this embodiment, specific implementation manners are as follows:

Specifically, after the device data of the multimedia playing device is acquired, the attribute data of the multimedia data in each attribute dimension can be determined, and then the multimedia data is evaluated in combination with the attribute data, the number of watching users and the number of unviewed users, that is, the evaluation result for the multimedia data can be determined.

Following the above example, in the case where it is determined that the number of viewing users viewing the billboard 301 is 400 and the number of non-viewing users is 11000, the advertisement will be evaluated in combination with the attribute data of the advertisement, and first the device data of the billboard 301 is obtained, and the position data is determined: subway station, flow data: 15000 persons/day, price data: 178/day, billboard type data: determining attribute data of the advertisement according to the equipment data of the billboard; and at the moment, the advertisement is evaluated by combining all attribute data, the number of watching users and the number of non-watching users, and the advertisement is determined to have good delivery effect if the evaluation result of the advertisement is determined to be high quality.

In summary, in the process of evaluating the multimedia data, the multimedia data is evaluated by combining the number of watching users, the number of non-watching users and the attribute data of the multimedia data, so that the multimedia data can be evaluated more accurately, and a throwing party who subsequently throws the multimedia data can conveniently adjust the multimedia data according to the evaluation result.

In practical applications, the evaluation result is only a result of evaluating the current multimedia data by the multimedia playing device, and is used for reflecting the multimedia data launching effect, and the measurement standard for the multimedia data launching effect depends on a multimedia data launching party, and different launching parties may have different measurement standards, so that the multimedia data is subsequently adjusted according to the evaluation result and is determined according to the replacement condition set by the launching party.

Further, after determining the evaluation result for the multimedia data, in order to complete the replacement of the multimedia data in the offline state according to the evaluation result, in one or more embodiments of this embodiment, a specific implementation manner is as follows:

if so, selecting target multimedia data from a preset multimedia database for displaying; the multimedia data contained in the multimedia database is prestored by a delivery party to which the multimedia data belongs;

if not, no processing is carried out.

Specifically, the replacement condition is preset by a delivering party delivering multimedia data, specifically, whether the multimedia data needs to be replaced is detected according to an evaluation result, and the replacement condition may be whether the number of watching users watching the multimedia data reaches a threshold value or not, or whether a user ratio of the watching users watching the multimedia data in one day to a user passing through a visible area of the multimedia data in one day reaches the threshold value or not, so as to determine whether the multimedia data needs to be replaced; it should be noted that the replacement condition is preset by the multimedia data delivery party;

on the basis, if the detection result is yes, the target multimedia data can be selected from a preset multimedia database for displaying, wherein the multimedia data contained in the multimedia database is prestored by the releasing party to which the multimedia data belongs, and the releasing effect of the multimedia data at the moment cannot reach the expected standard of the releasing party; if the detection result is negative, the multimedia data launching effect can reach the expected standard of the launching party at the moment, and then no processing is needed.

In practical applications, the selection mode for selecting the target multimedia data in the multimedia database may be selected according to an order set by a delivery party, or when it is determined that the multimedia data meets the replacement condition, other types of multimedia data different from the type of the multimedia data may be selected as the target multimedia data.

Along the use example, when the number of users watching an advertisement displayed on the billboard 301 in one day is determined to be 400, whether the advertisement displayed on the billboard 301 meets the replacement condition is detected, the replacement condition set by a company to which the advertisement belongs is to detect whether the number of users watching the advertisement in one day exceeds 500, the comparison result shows that the advertisement meets the replacement condition and the placement effect of the advertisement does not meet the placement effect standard of the company, and other advertisement documents aiming at the commodity are selected from a database configured in the billboard 301 for display, wherein other advertisements existing in the database are prestored by the company.

In summary, even when the multimedia playing device is in the offline state, the evaluation of the multimedia data can still be completed, and the multimedia data can be replaced according to the evaluation result, and the target multimedia data to be replaced is pre-stored in the multimedia database by the delivering party, so that the multimedia data can be replaced in the offline state, and the multimedia data with better delivering effect can be displayed through the multimedia playing device without excessive operation of the delivering party.

In addition, after determining the evaluation result for the multimedia data, the multimedia data may be replaced according to the requirement of the delivering party, and in one or more embodiments of this embodiment, specific implementation manners are as follows:

Specifically, after the evaluation result for the multimedia data is determined, the evaluation result may be sent to a delivery party delivering the multimedia data, and when a change request for uploading the multimedia data by the delivery party is received, it indicates that the delivery effect of the multimedia data does not meet the delivery effect standard of the delivery party, and the multimedia data is replaced and displayed according to the target multimedia data carried in the change request.

The multimedia evaluation method provided by the embodiment obtains an image set composed of image frames acquired by an image acquisition module for the visible area of the multimedia data, inputting each image frame into an identification model for user identification to obtain the viewing data of the user in the identified image frame, then determining the evaluation result aiming at the multimedia data based on the viewing data and the attribute data of the multimedia data in each attribute dimension, realizing the evaluation of the multimedia data delivery effect by combining the viewing data and the attribute data of the user, can more intuitively reflect the putting effect of the multimedia data, effectively improve the accuracy of the multimedia data evaluation, therefore, the multimedia data releasing party can effectively adjust the multimedia data according to the evaluation result, and the experience effect of the releasing party is further improved.

The multimedia evaluation method provided in the present specification is further described below with reference to fig. 2, taking an application of the multimedia evaluation method in an advertisement evaluation scenario as an example. Fig. 2 shows a processing flow chart of a multimedia evaluation method applied in an advertisement evaluation scenario provided in an embodiment of the present specification, and specifically includes the following steps:

step 202, displaying the advertisement to be evaluated according to a preset time period.

And step 204, acquiring an image set formed by image frames acquired by the image acquisition module aiming at the visual area of the advertisement to be evaluated.

Specifically, the billboard displaying the advertisement to be evaluated is provided with an image acquisition module for acquiring image frames, a processing module for identifying users, and a database for storing more advertisements.

Step 206, inputting each image frame contained in the image set into a recognition model for user recognition.

At step 208, the number of viewing users and the number of non-viewing users in the identified image frame are obtained.

Step 210, position data, type data and price data of the billboard displaying the advertisement to be evaluated are obtained.

Specifically, the position data specifically refers to the position of the billboard, the type data specifically refers to the type of the billboard displaying the advertisement, and the price data specifically refers to the price of the rental billboard.

Step 212, determining attribute data of the advertisement to be evaluated according to the position data, the type data and the price data.

Step 214, evaluating the advertisement to be evaluated based on the attribute data, the number of the watching users and the number of the non-watching users to obtain an evaluation result.

Step 216, detecting whether the advertisement to be evaluated meets the advertisement replacement condition according to the evaluation result; if yes, go to step 218; if not, no processing is carried out.

And step 218, selecting a target advertisement from the database, replacing and displaying the advertisement to be evaluated.

Specifically, detecting whether the advertisement to be evaluated meets the advertisement replacement condition specifically means detecting whether the ratio of the number of users watching the advertisement to the number of users passing through the visible area of the billboard in one day is greater than or equal to a preset threshold value, if so, indicating that the delivery effect of the advertisement to be evaluated does not meet the measurement standard of an advertisement delivery party, and selecting a target advertisement in a database to replace the advertisement to be evaluated and display the target advertisement; wherein the advertisements stored in the database are pre-stored by the sponsor.

The multimedia evaluation method provided by the embodiment realizes evaluation of the advertisement to be evaluated through the number of watching users, the number of non-watching users and the attribute data of the advertisement, can more visually reflect the delivery effect of the advertisement to be evaluated, effectively improves the accuracy of evaluation of the advertisement to be evaluated, further realizes replacement of the advertisement to be evaluated according to the evaluation result, can complete the process from evaluation of the advertisement to be evaluated to replacement of the target advertisement in an off-line state, does not need too many operations of a delivery party, can display the target advertisement with better delivery effect through the billboard, and effectively improves the experience effect of the delivery party.

Corresponding to the above method embodiment, the present specification further provides an embodiment of a multimedia evaluation apparatus, and fig. 3 shows a schematic structural diagram of a multimedia evaluation apparatus provided in an embodiment of the present specification. As shown in fig. 3, the apparatus includes:

an obtaining module 302 configured to obtain an image set composed of image frames collected by the image collecting module for a visible region of the multimedia data;

an identification module 304, configured to input each image frame contained in the image set into an identification model for user identification, and obtain the identified viewing data of the user in the image frame;

a determining module 306 configured to determine an evaluation result for the multimedia data based on the viewing data and attribute data of the multimedia data in various attribute dimensions.

In an optional embodiment, the recognition model performs user recognition, including:

identifying a face region in each image frame;

In an optional embodiment, the determining module 306 includes:

a first acquisition unit configured to acquire device data of a multimedia playback device that plays the multimedia data;

a first determining unit configured to determine the attribute data of the multimedia data in each attribute dimension according to the device data;

a first evaluation unit configured to perform a primary evaluation of the multimedia data based on the attribute data, determine a first evaluation result, and perform a secondary evaluation of the multimedia data based on the viewing duration and the number of users, determine a second evaluation result;

an integration unit configured to integrate the first evaluation result and the second evaluation result, the evaluation result for the multimedia data being determined according to an integration result.

identifying a face region in each image frame;

In an optional embodiment, the determining module 306 includes:

a second acquisition unit configured to acquire device data of a multimedia playback device that plays the multimedia data;

a second determining unit configured to determine the attribute data of the multimedia data in each attribute dimension according to the device data;

a second evaluation unit configured to evaluate the multimedia data based on the attribute data, the number of viewing users, and the number of unviewed users, to obtain the evaluation result.

In an optional embodiment, the multimedia evaluation apparatus further includes:

the first presentation module is configured to present the multimedia data according to a preset time period.

a detection module configured to detect whether the multimedia data meets a replacement condition according to the evaluation result;

if yes, operating a second display module;

the second display module is configured to select target multimedia data from a preset multimedia database for display; and the multimedia data contained in the multimedia database is prestored by the delivering party to which the multimedia data belongs.

a sending module configured to send the evaluation result to a delivering party delivering the multimedia data;

and the replacing module is configured to replace and display the multimedia data according to target multimedia data carried in the replacing request under the condition that the replacing request aiming at the uploading of the multimedia data by the releasing party is received.

In an optional embodiment, the multimedia evaluation apparatus is applied to a multimedia playing device, and accordingly, the image capturing module is configured on the multimedia playing device.

In an alternative embodiment, the training process of the recognition model includes:

In an optional embodiment, the attribute dimension includes at least one of:

accordingly, the attribute data includes at least one of:

location data, traffic data, price data, multimedia type data.

The multimedia evaluation device provided by the embodiment acquires an image set composed of image frames acquired by the image acquisition module for the visible area of the multimedia data, inputting each image frame into an identification model for user identification to obtain the viewing data of the user in the identified image frame, then determining the evaluation result aiming at the multimedia data based on the viewing data and the attribute data of the multimedia data in each attribute dimension, realizing the evaluation of the multimedia data delivery effect by combining the viewing data and the attribute data of the user, can more intuitively reflect the putting effect of the multimedia data, effectively improve the accuracy of the multimedia data evaluation, therefore, the multimedia data releasing party can effectively adjust the multimedia data according to the evaluation result, and the experience effect of the releasing party is further improved.

The foregoing is a schematic diagram of a multimedia evaluation apparatus according to the embodiment. It should be noted that the technical solution of the multimedia evaluation apparatus belongs to the same concept as the technical solution of the multimedia evaluation method described above, and details that are not described in detail in the technical solution of the multimedia evaluation apparatus can be referred to the description of the technical solution of the multimedia evaluation method described above.

FIG. 4 illustrates a block diagram of a computing device 400 provided according to an embodiment of the present description. The components of the computing device 400 include, but are not limited to, a memory 410 and a processor 420. Processor 420 is coupled to memory 410 via bus 430 and database 450 is used to store data.

The computing device 400 also includes AN access device 440, the access device 440 enabling the computing device 400 to communicate via one or more networks 460. examples of such networks include a Public Switched Telephone Network (PSTN), a local area network (L AN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet the access device 440 may include one or more of any type of network interface, e.g., a Network Interface Card (NIC), wired or wireless, such as AN IEEE802.11 wireless local area network (W L AN) wireless interface, a Global microwave Internet Access (Wi-MAX) interface, AN Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so forth.

In one embodiment of the present description, the above-described components of computing device 400, as well as other components not shown in FIG. 4, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 4 is for purposes of example only and is not limiting as to the scope of the present description. Those skilled in the art may add or replace other components as desired.

Computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smartphone), wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 400 may also be a mobile or stationary server.

Wherein processor 420 is configured to execute the following computer-executable instructions:

Optionally, the identifying model performs user identification, including:

identifying a face region in each image frame;

Optionally, the identifying model performs user identification, including:

identifying a face region in each image frame;

and displaying the multimedia data according to a preset time period.

Optionally, the training process of the recognition model includes:

Optionally, the attribute dimension includes at least one of:

accordingly, the attribute data includes at least one of:

location data, traffic data, price data, multimedia type data.

The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the multimedia evaluation method described above belong to the same concept, and details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the multimedia evaluation method described above.

An embodiment of the present specification also provides a computer readable storage medium storing computer instructions that, when executed by a processor, are operable to:

Optionally, the identifying model performs user identification, including:

identifying a face region in each image frame;

Optionally, the identifying model performs user identification, including:

identifying a face region in each image frame;

and displaying the multimedia data according to a preset time period.

Optionally, the training process of the recognition model includes:

Optionally, the attribute dimension includes at least one of:

accordingly, the attribute data includes at least one of:

location data, traffic data, price data, multimedia type data.

The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the above multimedia evaluation method, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the above multimedia evaluation method.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.

It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present disclosure is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present disclosure. Further, those skilled in the art should also appreciate that the embodiments described in this specification are preferred embodiments and that acts and modules referred to are not necessarily required for this description.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

The preferred embodiments of the present specification disclosed above are intended only to aid in the description of the specification. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the specification and its practical application, to thereby enable others skilled in the art to best understand the specification and its practical application. The specification is limited only by the claims and their full scope and equivalents.

Claims

1. A multimedia assessment method, comprising:

2. The multimedia assessment method of claim 1, the recognition model performing user recognition, comprising:

identifying a face region in each image frame;

3. The multimedia evaluation method of claim 2, the determining an evaluation result for the multimedia data based on the viewing data and attribute data of the multimedia data in respective attribute dimensions, comprising:

4. The multimedia assessment method of claim 1, the recognition model performing user recognition, comprising:

identifying a face region in each image frame;

5. The multimedia assessment method of claim 4, the determining an assessment result for the multimedia data based on the viewing data and attribute data of the multimedia data in various attribute dimensions, comprising:

6. The multimedia assessment method of claim 1, wherein the acquiring an image set of image frames captured by the image capture module for the viewable area of the multimedia data further comprises:

and displaying the multimedia data according to a preset time period.

7. The multimedia assessment method of claim 6, after the step of determining the assessment result for the multimedia data based on the viewing data and the attribute data of the multimedia data in the respective attribute dimensions is performed, further comprising:

8. The multimedia evaluation method of claim 1, further comprising, after the step of determining the evaluation result for the multimedia data based on the viewing data and the attribute data of the multimedia data in the respective attribute dimensions is performed:

9. The multimedia evaluation method of claim 1, wherein the multimedia evaluation method is applied to a multimedia player, and accordingly, the image capture module is configured on the multimedia player.

10. The multimedia assessment method of claim 1, the training process of the recognition model comprising:

11. The multimedia assessment method of claim 1, the attribute dimension comprising at least one of:

accordingly, the attribute data includes at least one of:

location data, traffic data, price data, multimedia type data.

12. A multimedia evaluation apparatus comprising:

13. A computing device, comprising:

a memory and a processor;

14. A computer readable storage medium storing computer instructions which, when executed by a processor, carry out the steps of the multimedia assessment method of any one of claims 1 to 11.