WO2018228037A1 - 媒体数据处理方法、装置及存储介质 - Google Patents
媒体数据处理方法、装置及存储介质 Download PDFInfo
- Publication number
- WO2018228037A1 WO2018228037A1 PCT/CN2018/082424 CN2018082424W WO2018228037A1 WO 2018228037 A1 WO2018228037 A1 WO 2018228037A1 CN 2018082424 W CN2018082424 W CN 2018082424W WO 2018228037 A1 WO2018228037 A1 WO 2018228037A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- media content
- image
- similarity
- page
- data
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 12
- 230000004044 response Effects 0.000 claims abstract description 56
- 238000012790 confirmation Methods 0.000 claims abstract description 15
- 238000000034 method Methods 0.000 claims description 41
- 238000012545 processing Methods 0.000 claims description 18
- 238000012360 testing method Methods 0.000 claims description 15
- 238000012549 training Methods 0.000 claims description 10
- 238000001454 recorded image Methods 0.000 claims description 6
- 230000000977 initiatory effect Effects 0.000 claims description 3
- 238000004088 simulation Methods 0.000 abstract description 6
- 230000014509 gene expression Effects 0.000 description 32
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 10
- 230000008921 facial expression Effects 0.000 description 9
- 230000009471 action Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 210000004709 eyebrow Anatomy 0.000 description 4
- 238000011524 similarity measure Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 230000007935 neutral effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
Definitions
- the present application relates to the field of Internet technologies, and in particular, to a media data processing method, apparatus, and storage medium.
- the application example provides a media data processing method, which is applied to a terminal device, and the method includes: sending an imitation request to an application server, in response to selecting the first media content in the first page and operating on the first control, where The imitation request carries an identifier of the first media content; in response to an acknowledgment response from the application server for the impersonation request, an image acquisition device is activated; and image data acquired according to the image acquisition device is acquired Generating the second media content; sending the data of the second media content to the application server; receiving and displaying the similarity of the second media content sent by the application server, where the similarity is passed The second media content is obtained by performing image similarity matching with the first media content.
- the application example further provides a media data processing method, which is applied to a server device, and the method includes: receiving an imitation request sent by an application client, and acquiring the first according to the identifier of the first media content carried in the imitation request. Data of the media content; sending a confirmation response to the imitation request to the application client to enable the application client to launch an image collection device; receiving data of the second media content sent by the application client; The second media content is generated according to the image data collected by the image collection device; and the second media content and the first media are determined according to the data of the first media content and the second media content Performing image similarity matching on the content, determining a similarity of the second media content, and sending the similarity of the second media content to the application client, so that the application client displays the second media content Similarity.
- the application example further provides a media data processing apparatus, the apparatus comprising a processor and a memory, wherein the memory stores computer readable instructions, the instructions may cause the processor to: respond to the first page Selecting a media content and operating the first control to send an imitation request to the application server, wherein the imitation request carries an identifier of the first media content; in response to the imitation request from the application server Confirming the response, starting the image acquisition device; acquiring the second media content generated according to the image data collected by the image collection device; transmitting the data of the second media content to the application server; and receiving and displaying the application server The similarity of the sent second media content, wherein the similarity is obtained by performing image similarity matching on the second media content and the first media content.
- the application example further provides a media data processing apparatus, the device comprising a processor and a memory, wherein the memory stores computer readable instructions, the instructions may enable the processor to: receive an imitation request sent by an application client Obtaining data of the first media content according to the identifier of the first media content carried in the imitation request; sending a confirmation response to the imitation request to the application client, so that the application client starts the image a collecting device; receiving data of the second media content sent by the application client, the second media content being generated according to the image data collected by the image collecting device; according to the first media content and the Data of the second media content, performing image similarity matching on the second media content and the first media content, determining a similarity of the second media content; and transmitting the similarity of the second media content To the application client, so that the application client displays the similarity of the second media content.
- the application examples also provide a storage medium storing computer readable instructions that enable at least one processor to perform the above method.
- an image similarity between the second media content and the first media content can be obtained, thereby providing a new application function for the application client, and improving performance of the application client.
- FIG. 1 is a schematic diagram of a system architecture involved in an example of the present application.
- FIG. 3 is a schematic diagram of a user interface involved in an example of the present application.
- FIG. 4 is a schematic diagram of another user interface involved in an example of the present application.
- FIG. 5 is a schematic diagram of still another user interface involved in the example of the present application.
- FIG. 6 is a schematic diagram of still another user interface involved in the example of the present application.
- FIG. 7 is a flow chart of a method of another example of the present application.
- FIG. 8 is a message interaction diagram of an example media data processing method of the present application.
- FIG. 9 is a structural diagram of a device according to an example of the present application.
- FIG. 10 is a structural diagram of a device of another example of the present application.
- FIG. 11 is a structural diagram of a device of an example of the present application.
- the present application proposes a media data processing method that can be applied to the system architecture shown in FIG. 1.
- the system architecture includes an application client 101 and an application server 102, and the application client 101 and the application server 102 communicate via the Internet 103.
- the application client 101 can be a terminal device such as a mobile phone, a PAD (tablet), a laptop portable computer, or a desktop computer.
- the application server 102 can be a single server device, or a server cluster consisting of several servers, or a cloud computing service center.
- the user accesses the application server 102 using an application client 101 (such as a video APP or an APP dedicated to image emulation), such as watching online video, etc.
- an application client 101 such as a video APP or an APP dedicated to image emulation
- the application server 102 may be a web server that provides various Internet services, such as a portal server.
- the APP dedicated to image emulation may be a video or picture emulation APP, an emoticon APP, and the like.
- the imitation function in the application client 101 can be triggered to perform image emulation, and the application client 101 responds to the trigger operation to the application server 102.
- Sending an imitation request wherein the imitation request carries an identifier of the first media content.
- the application server 102 acquires the first media content according to the first media content identifier in response to the imitation request, and then sends an acknowledgement response to the imitation request to the application client 101.
- the application client 101 activates the image capture device in response to the confirmation response to the imitation request described above.
- the application client 101 acquires the second media content generated according to the image data collected by the image collection device, and sends the data of the second media content to the application server 102; the application server 102 uses the second media content Image similarity matching is performed with the first media content, the similarity of the second media content is obtained, and the similarity is sent to the application client 101, and the application client 101 receives and displays the similarity.
- the similarity of the obtained second media content may be a scoring value (for example, may be displayed as “99 points” on the application client, etc.), and the score may reflect the image and the second media content.
- the imitation function in the application client 101 may be used for the imitation of the character expression in the image, for example, the emoticon of the character, the animal, and the like in a movie drama
- the second media content obtained by the application client 101 may be A picture or video (abbreviated as an imitation image) of a character's expression in a picture or video (ie, a first piece of media content, which may be referred to as an image being imitated), and can be obtained from the application server 102.
- the similarity score between the image and the simulated image is simulated, so that the user can know how similar the currently emulated character expression is.
- the media data processing method proposed by the example of the present application is applicable to the application client 101. As shown in FIG. 2, the method includes the following steps:
- Step 201 Send an imitation request to the application server in response to the selection of the first media content in the first page and the operation on the first control, wherein the imitation request carries the identifier of the first media content.
- the first media content includes videos, pictures, and the like, such as funny or exaggerated small videos and pictures.
- the first media content ie, the picture or video that the user wants to imitate
- the local video or picture at this time, the application client 101 may select the first media content obtained by the interception or the first media content in the local album, and the imitation request sent by the application client may carry the first The data of the media content and its identity.
- the first media content may also be provided by the application server 102.
- the application client 101 may browse information of each first media content provided in the page (including: cover image, title) At the same time, the application client 101 can select a first media content from the page, and the imitation request sent by the application client 101 may not carry the data of the first media content, but only carry the first The identity of the media content.
- the first media content provided by the application server 102 may be uploaded and published to the application server 102 by the party providing the impersonation application service, or may be uploaded and published to the application server by other application users through other application clients. 102.
- the application client 101 can upload and publish it as the User Generated Content (UGC) to the application server 102. It can be used as the first media content for various users to imitate.
- UGC User Generated Content
- the first control can be a "mock show” control (button) 302 as shown in FIG. 3, or a "mock show” control 402 as shown in FIG.
- FIG. 3 is a schematic diagram of a user interface according to an example of the present application. In the page shown in FIG.
- the area shown by 301 is playing a video
- the small video shown by 303 is The current user or other user intercepts the relevant part of the currently playing video, or the small video provided for the video app or video server, provides a small video displayed in the 303 area shown on the current page, and clicks the "More" control 304 to switch To the page shown in Figure 4, more small videos are displayed on the page (the page can have scroll bars and up and down arrow buttons to allow the user to view more small videos or stills).
- the application client 101 can also directly display the page shown in FIG. 4 without going through the operation of the page shown in FIG. 3, that is, as the first page in step 201.
- the small video is intercepted by a user, and other users can interact with the small video, such as like, comment, and imitate the small video (ie, trigger the "Imitation Show” control 402).
- other users can also interact with the stills (ie, pictures) intercepted by a certain user. For example, clicking on the label "stills” in the page shown in FIG. 4 can display a number of still photos, from which the user can Select the stills photo you want to imitate.
- the small video to be simulated can be selected by a control (button) 305 as shown in FIG. 3 or a control 403 as shown in FIG. That is, the "Imitation Show” control can be located in the page shown in FIG. 3, after the user operates the "Imitation Show” control 302, select the small video or stills to be imitated; or the "Imitation Show” control is located in the page shown in FIG. Imitate for a selected small video or still.
- Step 202 Initiate an image capture device in response to a confirmation response from the application server for the impersonation request.
- the confirmation response is an instruction that the application server 102 informs that the application client 101 can start emulating, and the application client 101 activates an image collection device in the terminal device in which it is located in response to the instruction (also referred to as Camera device, such as the camera of the smartphone on which it is located).
- an image collection device in the terminal device in which it is located in response to the instruction (also referred to as Camera device, such as the camera of the smartphone on which it is located).
- the application client 101 is a video app, and after receiving the imitation instruction sent by the application server 102, the video app switches from the page shown in FIG. 3 or the page shown in FIG. 4 to the page shown in FIG. 5, that is, the video.
- the app enters the page that begins to mimic.
- FIG. 5 is an image emulation interface according to an example of the present application.
- the area shown by 501 includes a user avatar and a nickname for performing image emulation; 502 is an image selected by the user to be imitated, such as a small video, such as a user selected graph. 3 or the small video 1 in the page of FIG. 4, when the video app switches the page to FIG. 5, the small video 1 is imported into the page shown in FIG. 5.
- the application client 101 launches the image capture device, and then begins collecting data of the second media content through the image capture device.
- the camera device in the device is first requested. If the camera device in the device is successfully connected, the test image collected by the camera device is attempted, and the application client 101 can pass the test image. The test image discriminates whether the camera device is working normally. After that, the user is prompted with time to start the image simulation, and the application client 101 starts collecting the data of the second media content through the camera device; if the camera device in the device cannot be successfully connected , then try to connect again, can be repeated multiple times.
- the video app switches the page shown in FIG. 3 or the page shown in FIG. 4 to the page shown in FIG. 5, first requests a smart device, such as a camera device in the mobile phone, such as a camera, if the camera in the mobile phone can be successfully connected.
- a smart device such as a camera device in the mobile phone, such as a camera
- the user is counted down to the time prompt, such as 507, after the countdown is over, the user can start the image simulation by operating the control 503, in the process of the image simulation, the video app automatically saves the data of the simulated image (ie, the second media content).
- Control 505 can be a "Re-record” control
- control 506 can be a "Cancel” control.
- the countdown prompt ends, it is also used to display the playback time progress of the small video in the 502 area.
- Step 203 Acquire second media content generated according to image data collected by the image collection device.
- the acquiring the second media content generated according to the image data collected by the image collection device comprises: displaying a second page; displaying image collection of the image collection device in the second page a real-time screen; in response to an operation of the second control in the second page, starting to record image data collected by the image capture device; when the recording end condition is satisfied, stopping recording the image data and generating the second Media content.
- the second page is shown in FIG. 5, and the image capturing real-time image of the image capturing device (such as a camera) is displayed in the area shown by 504.
- the control 503 ie, the second control
- the video app Starting to record the image data collected by the image capturing device; when the playing time progress of the small video (ie, the first media content) in the 502 area ends or the user clicks the end button, the video app stops recording the image data and generates The imitation video (i.e., the second media content) shown at 604 in FIG.
- the method further includes: in response to an operation of the second control in the second page, initiating display of the first media content in the second page; wherein the recording ends The condition includes that the first media content is displayed.
- the video app starts playing the small video 1 (ie, the first media content) in the 502 area, when At the end of the playback time of the small video 1, the video app recording ends.
- the method further includes: responsive to the operation of the third control in the second page, stopping recording the image data and deleting the recorded image data.
- the video app stops recording the image data and deletes the recorded image data.
- the method further includes: in response to operation of the fourth control in the second page, stopping recording the image data, deleting the recorded image data, and restarting recording by the image capture device Image data to.
- the fourth control is the “Re-recording” control 505 in FIG. 5.
- the video app stops recording the image data, deletes the recorded image data, and restarts in response to the above operation. Recording image data collected by the image acquisition device.
- Step 204 Send data of the second media content to the application server.
- the sending, by the application client 101, the data of the second media content that is completed after the data of the second media content is collected is sent to the application server.
- the application client 101 arranges the first media content and the first media content side by side in the page, so that the user compares the two.
- the video app switches the page as shown in FIG. 5 to the page as shown in FIG. 6, and displays the imitation in the area shown at 604.
- An image (such as an imitation video)
- the simulated image is arranged side by side with the small video 1 shown in 602, and the user can play the small video 1 and the simulated video simultaneously by clicking the small video 1 and the imitation video, so that the user can facilitate the two
- the imitation effect is determined, and the time progress of the small video 1 is displayed in the area indicated by 607.
- the application client 101 saves the collected data of the second media content, and sends the data of the second media content to the application server 102, so that the application server 102 is according to the first The media content processes the second media content.
- Step 205 Receive and display the similarity of the second media content sent by the application server, where the similarity is obtained by performing image similarity matching on the second media content and the first media content. .
- the method further includes: displaying the identification of the second media content in a second page after acquiring the second media content; displaying the second media content in the second page The similarity; displaying the second media content in response to an operation of identifying the second media content.
- the identifier of the simulated video recorded by the video app (eg, the name of the simulated video, the simulated video of the small video 1) is displayed in the area indicated by 604, and the user can operate the logo and play.
- the simulated video, the simulated video is previewed, and the score (ie, similarity) of the simulated image is displayed in the 601 area, as shown in the figure of 93 points.
- the method further includes: in response to operation of the fifth control in the second page, publishing the second media content to the application server to cause the application server to be the first The second media content generates a link address.
- the page shown in FIG. 6 is also provided with a "Publish" control 603 (ie, a fifth control) for the user to publish the simulated image to the application server 102, after which the user and other users access the corresponding page of the application server. , you can see the information of this imitation image, and click to play.
- the application server 102 can give a sharing option in the page or the new page described in FIG. 6, and the user can share the simulated image into the social relationship chain by operating the sharing option. Friends or groups (such as Weibo, WeChat, friends or groups in QQ, etc.), or share imitation images to space, Weibo, friends, and so on.
- the video app publishes the recorded user imitation video (ie, the second media content) to the video server, and the video server generates a link address for the simulated video, so that other users access the public page of the video app (eg, Dynamic "), you can see this link address in the public page, click to play the imitation video.
- the video app publishes the recorded user imitation video (ie, the second media content) to the video server, and the video server generates a link address for the simulated video, so that other users access the public page of the video app (eg, Dynamic "), you can see this link address in the public page, click to play the imitation video.
- the method further includes saving the second media content locally and generating a local link address in response to operation of the sixth control in the second page.
- the sixth control may be the “save” control 606 in FIG. 6.
- the video app saves the recorded user imitation video (ie, the second media content) locally and generates a local link address.
- the link address is displayed, and the link address can be clicked to play the video; as shown in FIG. 6, the page also provides a "re-record” control 605, which can make the user not Record again when you are satisfied with the currently recorded imitation image.
- an image similarity between the second media content and the first media content can be obtained, thereby providing a new application function for the application client, improving performance of the application client, and further
- the user's use of the client's fun, entertainment and social attributes is improved, and the application client's display effect is improved, thereby further improving the user experience.
- the present application further provides a media data processing method, which is applied to the application server 102. As shown in FIG. 7, the method includes the following steps:
- Step 701 Receive an imitation request sent by the application client, and acquire data of the first media content according to the identifier of the first media content carried in the imitation request.
- Step 702 Send a confirmation response to the imitation request to the application client, so that the application client starts the image collection device.
- the method further includes: presetting a training set, wherein the training set includes N sets of similar images, N being an integer greater than 1; using the preset formula template, using the training set training to obtain A formula for calculating an image distance; wherein the formula includes weights of image features, and a weight represents a contribution of a feature in an image to image distance.
- the method further includes: presetting a test sample, wherein the test sample includes M images, M is an integer greater than 1; respectively calculating M in the test sample using the formula for calculating an image distance The distance between the image and the M images of the image in the first media content; establishing a correspondence between the image distance and the similarity score according to the M image distances.
- the application server 102 sends the confirmation response to the imitation request to the application client 101
- the data of the first media content is acquired according to the identifier of the first media content, and the image distance and the similarity score are established.
- the correspondence between the correspondences is as follows:
- the similarity measure of any two similar facial expression images I, T, facial expression image I and facial expression image T may specifically adopt facial expression similarity measurement method based on facial expression geometric features, similar distance (or image) of facial expression image I and expression image T Distance) is:
- w i ⁇ 0 1, 2, ..., p + q
- the features are shape features (such as eyebrows, eyes, nose and mouth, etc.) or deformation features (such as the degree of stretching and contraction of the mouth, the opening and closing of the eyes and mouth, the upward and downward pressure of the eyebrows, etc.);
- ⁇ ⁇ 0 is the weight effect
- the coefficient; d i (I, T) is expressed as the difference of the i-th feature in the expression image I and the expression image T, and is specifically expressed by the formula (2):
- N sets of similar facial expression images (I n , T n ) are selected as a training set, which may be in the BU-3DFE (Binhamton University 3D Facial Expression) database provided by Binghamton University.
- BU-3DFE Woodton University 3D Facial Expression
- similar expressions require the type of expression (the category is divided into neutral, happy, disgusted, angry, fearful, sad, and annoyed in the BU-3DFE database) and the strength of the expression (the intensity is divided into 4 in the BU-3DFE database).
- the levels, intensity 1, intensity 2, intensity 3, and intensity 4) are the same.
- the database contains 100 testers, each of which has a neutral and 6 basic expressions, and each basic expression has 4 intensities.
- w i reflects the contribution of the i-th feature to the expression similar distance.
- shape features of the eyebrows, eyes, and mouth are more significant for the expression of the expression, the cheeks and chin are second, and the nose often cannot reflect the expression changes;
- the deformation range of each region is also different, that is, the deformation characteristics of each region have different contributions to the expression similar distance.
- the deformation of the mouth region is larger than that of the eye region, and the deformation of the eyebrow region is not as good as the mouth and cheek.
- the correspondence between the image distance and the similarity score is established.
- the first media content in the BU-3DFE database are respectively solved by the equation (1).
- the M image distances of the middle image and the correspondence between the image distance and the similarity score are established, for example, the above M image distances are divided into 100 intervals, wherein each interval includes several image distances, each An interval correspondingly corresponds to one of the scores of 1 to 100, for example, an image distance falling at the 86th interval corresponds to 86 points.
- the data of the first media content and the data of the second media content each comprise a plurality of frame images for Each frame image in the first media content data is processed as follows: a test sample is set in advance, wherein the test sample includes M images, M is an integer greater than 1; and the calculation is respectively performed by using the formula for calculating an image distance And measuring an image distance between the M images in the sample and the frame image in the first media content; establishing a correspondence between the image distance and the similarity score according to the M image distances.
- the application server after receiving the first media content (such as a small video) sent by the application client, the application server is based on N sets of similar facial expression images in the BU-3DFE database (I n , T n ) the training set obtains w i , and then for each frame of the small video, using formula (1) to calculate the image distance of the M images in the test sample and the frame image in the first media content, respectively, according to The M image distances establish a correspondence between the image distance and the similarity score.
- the first media content such as a small video
- the application server is based on N sets of similar facial expression images in the BU-3DFE database (I n , T n ) the training set obtains w i , and then for each frame of the small video, using formula (1) to calculate the image distance of the M images in the test sample and the frame image in the first media content, respectively, according to The M image distances establish a correspondence between the image distance and the similarity score.
- Step 703 Receive data of the second media content sent by the application client.
- the second media content is generated according to image data collected by the image collection device.
- Step 704 Perform image similarity matching on the second media content and the first media content according to the data of the first media content and the second media content, and determine a similarity of the second media content. .
- the image similarity matching between the second media content and the first media content, determining the similarity of the second media content includes: according to the first media content and the second Data of the media content, using the formula for calculating an image distance, calculating an image distance between the image of the second media content and the image of the first media content; determining the second according to the correspondence A similarity score corresponding to an image distance between an image of the media content and an image of the first media content.
- an average or weighted average of similarity scores of all frame images of the second media content may be determined as a similarity score of the second media content.
- the application server 102 may collect continuous expression actions in the small video, and collect corresponding expression actions in the second media content, and the continuous expressions Each frame of the action performs a similarity measure, respectively; or when the first media content is a still, the application server 102 performs a similarity measure on the second media content and the first media content.
- Step 705 Send the similarity of the second media content to the application client, so that the application client displays the similarity of the second media content.
- FIG. 8 shows a message interaction diagram of an example of the present application.
- the video app corresponds to the application client above, and may include an emulation module capable of implementing the foregoing method;
- the video server corresponds to the foregoing application server,
- a similar matching module for implementing the above method is included. As shown in Figure 8, the following steps are included:
- Step 801 In response to the user's operation, the operation may be for the user to click the video app to play the “Imitation Show” button in a certain video page, as shown in FIG. 3, or the user clicks on the video app to display multiple small video pages.
- the "Imitation Show” button as shown in FIG. 4, the video app sends an imitation request to the video server, the imitation request carrying the small video (or the identifier of the small video) to be imitated by the user through the control 305 or the control 403, wherein
- the small video is a part of a video that is intercepted by other users in a video that is viewed by the user or a video app or a video server, and may be a few seconds or tens of seconds.
- Step 802 The video server saves the small video in response to the imitation request, and after saving, sends a confirmation response to the imitation request to the video app to notify the video app that the emulation stage can enter the emulation stage.
- the similar matching module in the video server establishes a correspondence between the image distance and the similarity score based on the data in the BU-3DFE database provided by Binghamton University.
- Step 803 After receiving the confirmation response for the imitation request, the video app switches the interface of the video app to the emulation interface, as shown in FIG. 5, and starts the camera to read the camera information.
- Step 804 After the video app successfully reads the camera information, the user prompts the time (such as a countdown), as shown in 507, to prepare the user to start the imitation.
- the time such as a countdown
- Step 805 The video app collects the image data simulated by the user through the camera, and automatically saves the image data, records the simulated video according to the image data, and sends the simulated video to the video server.
- Step 806 The similarity matching module in the video server performs image similarity matching on the small video and the user simulated video to obtain a similarity score.
- the image distance of each frame of the emoticon video and the corresponding frame emoticon image in the small video may be calculated by using formula (1), and the frame emoticon image and the first media are determined according to the correspondence relationship.
- the small video and the simulated image comprise at least one continuous expression action, that is, including a plurality of consecutive frames
- the final score of the continuous expression action in the simulated image may be obtained by the similarity degree of each frame of the continuous expression action.
- a weighted average of the values is obtained.
- Step 807 Send the similarity score of the simulated video to the video app, so that the video app displays the similarity score.
- Step 808 In response to the “Publish” operation, the simulated video is posted to the application server of the video app, so that other users of the video app can also browse to the simulated video, and the further video app can also provide a “Share” option.
- the simulated video is posted to the application server of the video app, so that other users of the video app can also browse to the simulated video, and the further video app can also provide a “Share” option.
- apps such as friends or groups in social applications such as Weibo, WeChat, QQ, etc. shared to the current video app users, or to Weibo, QQ space or WeChat friends circle; or
- return to step 803 or in response to "Save", save the simulated video locally for later viewing by the user, and the like.
- the example of the present application further provides a media data processing apparatus 900.
- the apparatus 900 includes:
- the sending module 901 in response to the selecting of the first media content in the first page and the operation of the first control, sending an imitation request to the application server, where the imitation request carries the identifier of the first media content; Data of the second media content is sent to the application server.
- the launch module 902 in response to the acknowledgment response from the application server for the impersonation request, launches the image capture device.
- the obtaining module 903 is configured to acquire second media content generated according to the image data collected by the image capturing device.
- the displaying module 904 receives and displays the similarity of the second media content sent by the application server, where the similarity is performed by performing image similarity matching on the second media content and the first media content. get.
- the presentation module displays a second page; displaying an image capture real-time screen of the image capture device in the second page; responsive to operation of a second control in the second page,
- the obtaining module 903 starts recording image data collected by the image capturing device; when the recording end condition is satisfied, the obtaining module 903 stops recording the image data and generates the second media content.
- the display module further displays the identifier of the second media content in the second page; and further displays in the second page.
- the similarity of the second media content the display module displays the second media content in response to an operation of identifying the second media content.
- the present application further provides a media data processing apparatus 1000.
- the apparatus 1000 includes:
- the receiving module 1001 receives an imitation request sent by the application client, acquires data of the first media content according to the identifier of the first media content carried in the imitation request, and receives the second media content sent by the application client.
- the data ;
- the sending module 1002 sends a confirmation response to the imitation request to the application client, so that the application client starts the image collection device, and sends the similarity of the second media content to the application client. So that the application client displays the similarity of the second media content;
- the matching module 1003 performs image similarity matching on the second media content and the first media content according to the data of the first media content and the second media content, and determines the similarity of the second media content. degree.
- media data processing method and the media data processing apparatus in the various examples of the present application and the modules therein may be integrated into one processing unit, or each module may exist physically separately, or may be two or more devices or The modules are integrated in one unit.
- the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
- the media data processing apparatus described above can be executed in various computing devices that can perform user information processing based on the Internet and loaded into the memory of the computing device.
- FIG. 11 is a diagram showing the composition of a computing device in which a media data processing device is located.
- the computing device includes one or more processors (CPUs) 1102, communication modules 1104, memory 1106, user interfaces 1110, and a communication bus 1108 for interconnecting these components.
- processors CPUs
- communication modules 1104, memory 1106, user interfaces 1110, and a communication bus 1108 for interconnecting these components.
- the processor 1102 can receive and transmit data through the communication module 1104 to effect network communication and/or local communication.
- User interface 1110 includes one or more output devices 1112 that include one or more speakers and/or one or more visual displays.
- User interface 1110 also includes one or more input devices 1114 including, for example, a keyboard, a mouse, a voice command input unit or loudspeaker, a touch screen display, a touch sensitive tablet, a gesture capture camera or other input button or control, and the like.
- Memory 1106 can be a high speed random access memory such as DRAM, SRAM, DDR RAM, or other random access solid state storage device; or nonvolatile memory such as one or more disk storage devices, optical disk storage devices, flash memory devices, Or other non-volatile solid-state storage devices.
- DRAM dynamic random access memory
- SRAM static random access memory
- DDR RAM dynamic random access memory
- nonvolatile memory such as one or more disk storage devices, optical disk storage devices, flash memory devices, Or other non-volatile solid-state storage devices.
- the memory 1106 stores a set of instructions executable by the processor 1102, including:
- An operating system 1116 including a program for processing various basic system services and for performing hardware related tasks
- the application 1118 includes various programs for implementing media data processing that enable the processing flow in the various examples described above, such as the device 900 shown in FIG. 9 and/or the device 1000 shown in FIG.
- device 900 can include various modules 901-904 as shown in FIG. 9, and each module 901-904 can store machine-executable instructions.
- the processor 1102 can implement the functions of the above-described modules 901 to 904 by executing the machine executable instructions in the modules 901 to 904 in the memory 1106.
- device 1000 can include modules 1001 - 1003 shown in FIG. 10, and each module 1001 - 1003 can store machine executable instructions.
- the processor 1102 can implement the functions of the modules 1001 to 1003 described above by executing the machine executable instructions in the modules 1001 to 1003 in the memory 1106.
- each of the examples of the present application can be implemented by a data processing program executed by a data processing device such as a computer.
- the data processing program constitutes the present invention.
- a data processing program usually stored in a storage medium is executed by directly reading a program out of a storage medium or by installing or copying the program to a storage device (such as a hard disk and or a memory) of the data processing device. Therefore, such a storage medium also constitutes the present invention.
- the storage medium can use any type of recording method, such as paper storage medium (such as paper tape, etc.), magnetic storage medium (such as floppy disk, hard disk, flash memory, etc.), optical storage medium (such as CD-ROM, etc.), magneto-optical storage medium ( Such as MO, etc.).
- the present application examples also disclose a non-volatile storage medium in which is stored a data processing program for performing any of the above-described methods of the present application.
- the method steps described in this application can be implemented by a data processing program, and can also be implemented by hardware, for example, by logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers, and embedded control. And so on.
- ASICs application specific integrated circuits
- programmable logic controllers programmable logic controllers
- embedded control embedded control
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Processing Or Creating Images (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
本申请实例公开了一种媒体数据处理方法，包括：响应于对第一页面中第一媒体内容的选择和对第一控件的操作，向应用服务器发送模仿请求，其中，所述模仿请求携带所述第一媒体内容的标识；响应于来自所述应用服务器的对于所述模仿请求的确认响应，启动图像采集装置；获取根据所述图像采集装置采集到的图像数据而生成的第二媒体内容；将所述第二媒体内容的数据发送至所述应用服务器；接收并展示所述应用服务器发送的所述第二媒体内容的相似度，其中，所述相似度通过对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配而得到。本申请实例还公开了相应的装置及存储介质。
Description
本申请要求于2017年06月16日提交中国专利局、申请号为201710457646.6、发明名称为“媒体数据处理方法、装置及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
本申请涉及互联网技术领域，尤其涉及一种媒体数据处理方法、装置及存储介质。
背景
目前，用户在网站上或者在应用程序(APP)中进行社交活动时，可以对其感兴趣的视频或图片进行模仿，例如，对热门剧中的某个表情及搞笑的视频节目中的某个片段进行模仿，通常用户使用摄像装置将其模仿的内容进行记录，并将其发布到互联网上以供其他用户评论及观赏。
技术内容
本申请实例提供了一种媒体数据处理方法，应用于终端设备，该方法包括：响应于对第一页面中第一媒体内容的选择和对第一控件的操作，向应用服务器发送模仿请求，其中，所述模仿请求携带所述第一媒体内容的标识；响应于来自所述应用服务器的对于所述模仿请求的确认响应，启动图像采集装置；获取根据所述图像采集装置采集到的图像数据而生成的第二媒体内容；将所述第二媒体内容的数据发送至所述应用服务器；接收并展示所述应用服务器发送的所述第二媒体内容的相似度，其中，所述相似度通过对所述第二媒体内容与所述第一媒体内容进 行图像相似度匹配而得到。
本申请实例还提供了一种媒体数据处理方法，应用于服务器设备，该方法包括：接收应用客户端发送的模仿请求，根据所述模仿请求中携带的第一媒体内容的标识获取所述第一媒体内容的数据；向所述应用客户端发送对于所述模仿请求的确认响应，以使所述应用客户端启动图像采集装置；接收所述应用客户端发送的第二媒体内容的数据；所述第二媒体内容是根据所述图像采集装置采集到的图像数据而生成的；根据所述第一媒体内容和所述第二媒体内容的数据，对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配，确定所述第二媒体内容的相似度；将所述第二媒体内容的相似度发送至所述应用客户端，以使所述应用客户端展示所述第二媒体内容的相似度。
本申请实例还提供了一种媒体数据处理装置，该装置包括处理器和存储器，所述存储器中存储有计算机可读指令，所述指令可以使所述处理器：响应于对第一页面中第一媒体内容的选择和对第一控件的操作，向应用服务器发送模仿请求，其中，所述模仿请求携带所述第一媒体内容的标识；响应于来自所述应用服务器的对于所述模仿请求的确认响应，启动图像采集装置；获取根据所述图像采集装置采集到的图像数据而生成的第二媒体内容；将第二媒体内容的数据发送至所述应用服务器；及接收并展示所述应用服务器发送的所述第二媒体内容的相似度，其中，所述相似度通过对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配而得到。
本申请实例还提供了一种媒体数据处理装置，该装置包括处理器和存储器，所述存储器中存储有计算机可读指令，所述指令可以使所述处理器：接收应用客户端发送的模仿请求，根据所述模仿请求中携带的第一媒体内容的标识获取所述第一媒体内容的数据；向所述应用客户端发 送对于所述模仿请求的确认响应，以使所述应用客户端启动图像采集装置；接收所述应用客户端发送的第二媒体内容的数据，所述第二媒体内容是根据所述图像采集装置采集到的图像数据而生成的；根据所述第一媒体内容和所述第二媒体内容的数据，对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配，确定所述第二媒体内容的相似度；及将所述第二媒体内容的相似度发送至所述应用客户端，以使所述应用客户端展示所述第二媒体内容的相似度。
本申请实例还提供了一种存储介质，存储有计算机可读指令，可以使至少一个处理器执行上述方法。
采用上述技术方案，能够得到所述第二媒体内容与所述第一媒体内容之间的图像相似度，从而为所述应用客户端提供了新的应用功能，改善了应用客户端的性能。
附图简要说明
为了更清楚地说明本申请实例中的技术方案，下面将对实例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实例涉及的一种系统构架示意图；
图2是本申请一实例的方法流程图；
图3是本申请实例涉及的一种用户界面示意图；
图4是本申请实例涉及的另一种用户界面示意图；
图5是本申请实例涉及的又一种用户界面示意图；
图6是本申请实例涉及的又一种用户界面示意图；
图7是本申请另一实例的方法流程图；
图8是本申请实例媒体数据处理方法的消息交互图；
图9是本申请一实例的装置结构图；
图10是本申请另一实例的装置结构图；及
图11是本申请一实例的设备结构图。
实施方式
下面将结合本申请实例中的附图，对本申请实例中的技术方案进行清楚、完整地描述，显然，所描述的实例仅是本申请一部分实例，而不是全部的实例。基于本申请中的实例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实例，都属于本申请保护的范围。
本申请提出一种媒体数据处理方法，该方法可应用于图1所示的系统构架中。如图1所示，该系统构架包括：应用客户端101和应用服务器102，应用客户端101和应用服务器102通过互联网103进行通信。其中，应用客户端101可以为手机、PAD(平板电脑)、膝上型便携计算机、台式计算机等终端设备。应用服务器102可以为单个服务器设备，或由若干台服务器组成的服务器集群，或者是一个云计算服务中心。
用户使用应用客户端101(如视频APP或专门用于图像模仿的APP)访问应用服务器102，比如：观看在线视频等，应用服务器102可以是提供各种互联网服务的网站服务器，比如：门户网站服务器、提供在线视频播放服务的服务器、社交平台的服务器及专门用于图像模仿的应用服务器等等。这里，专门用于图像模仿的APP可以为视频或图片模仿APP、表情模仿APP等等。
当一用户正使用应用客户端101访问应用服务器102中的媒体内容(如视频)时，可以触发应用客户端101中的模仿功能进行图像模仿，应用客户端101响应于触发操作，向应用服务器102发送模仿请求，其 中，所述模仿请求携带所述第一媒体内容的标识。应用服务器102响应于上述模仿请求，根据所述第一媒体内容标识获取所述第一媒体内容，之后，向应用客户端101发送对于所述模仿请求的确认响应。应用客户端101响应于上述对于所述模仿请求的确认响应，启动图像采集装置。应用客户端101获取根据所述图像采集装置采集到的图像数据而生成的第二媒体内容，并将所述第二媒体内容的数据发送至应用服务器102；应用服务器102将所述第二媒体内容与所述第一媒体内容进行图像相似度匹配，得到所述第二媒体内容的相似度，并将所述相似度发送至应用客户端101，应用客户端101接收并展示所述相似度。
在一些实例中，所得到的第二媒体内容的相似度可以是一个打分值(比如：可以在应用客户端显示为“99分”等)，这个分值可以体现第二媒体内容中图像与第一媒体内容图像之间的相似度。在实际应用场景中，应用客户端101中的模仿功能可能用于图像中角色表情的模仿，比如：某个影视剧中人物、动物等的表情模仿，应用客户端101获得的第二媒体内容可以是现场拍摄的用户模仿某个图片或者视频(即一种第一媒体内容，可简称被模仿的图像)中某个角色表情的图片或者视频(简称模仿图像)，并能从应用服务器102获得此模仿图像与被模仿的图像之间的相似度分值，进而用户可以知道当前模仿的角色表情的相似度有多高。
在一些实例中，本申请实例提出的媒体数据处理方法可应用于应用客户端101，如图2所示，该方法包括以下步骤：
步骤201：响应于对第一页面中第一媒体内容的选择和对第一控件的操作，向应用服务器发送模仿请求，其中，所述模仿请求携带所述第一媒体内容的标识。
其中，所述第一媒体内容包括视频、图片等，如搞笑或夸张的小视 频和图片。这里，所述第一媒体内容(即用户要模仿的图片或视频)可以为用户从应用客户端101(如视频app)展示的视频或相册中截取的图片或视频，也可以是从本地相册选择的本地视频或图片，此时，应用客户端101可从页面中选择其截取获得的第一媒体内容或本地相册中的第一媒体内容，其所发出的所述模仿请求可携带所述第一媒体内容的数据及其标识。所述第一媒体内容也可为应用服务器102提供的，应用客户端101访问应用服务器102提供的页面时，可以浏览到页面中提供的各个第一媒体内容的信息(可包括：封面图片、标题、链接等)，此时，应用客户端101可从页面中选择一个第一媒体内容，其所发出的所述模仿请求可以不携带所述第一媒体内容的数据，而仅携带所述第一媒体内容的标识。
在一些实例中，应用服务器102提供的第一媒体内容，可以是提供此模仿应用服务的一方上传并发布到应用服务器102的，也可以是其它个人用户通过其它应用客户端上传并发布到应用服务器102的。相应的，应用客户端101在从正在展示的某个视频或相册中截取到图片或视频后，可以把它作为用户生成内容(User Generated Content，UGC)上传并发布到应用服务器102，这种UGC可以作为供各种用户模仿的第一媒体内容。
例如，所述第一控件可以为如图3所示的“模仿秀”控件(按钮)302，或如图4所示的“模仿秀”控件402。图3为本申请实例涉及的一种用户界面示意图，在如图3所示的页面(即步骤201中的第一页面)中，301所示的区域正在播放视频，303所示的小视频为当前用户或其他用户从当前播放视频截取的相关部分，或者为视频app或视频服务器提供的小视频，提供在当前页面所示的303区域展示部分小视频，点击“更多”控件304后，切换至如图4所示的页面，该页面中展示更多的 小视频(该页面可以具有滚动条及上下箭头按钮，使得用户查看更多的小视频或者剧照)。应用客户端101也可不经过图3所示页面的操作而直接展示图4所示的页面，即作为步骤201中的第一页面。如401所示，该小视频为某一用户截取，其他用户可对该小视频进行互动，如点赞、评论以及模仿该小视频(即触发“模仿秀”控件402)等。如图4所示，其他用户还可以对某一用户截取的剧照(即图片)进行互动，比如：点击了图4所示页面中的标签“剧照”，则可以展示若干剧照照片，用户可以从中选择要模仿的剧照照片。所述要模仿的小视频可以通过如图3所示的控件(按钮)305，或如图4所示的控件403进行选定。即“模仿秀”控件可以位于图3所示的页面中，用户操作“模仿秀”控件302后，选择其要模仿的小视频或剧照；或者“模仿秀”控件位于图4所示的页面中，针对其选定的某一个小视频或剧照进行模仿。
步骤202：响应于来自所述应用服务器的对于所述模仿请求的确认响应，启动图像采集装置。
这里，所述确认响应为所述应用服务器102告知所述应用客户端101可以开始模仿的指令，所述应用客户端101响应于此指令启动自身所在终端设备中的图像采集装置(也可称为摄像装置，如自身所在智能手机的摄像头)。
例如，应用客户端101为视频app，视频app接收应用服务器102发送的模仿指令后，从如图3所示的页面或如图4所示的页面切换至如图5所示的页面，即视频app进入开始模仿的页面。图5为本申请实例涉及的一种图像模仿的界面，501所示的区域包括进行图像模仿的用户头像及昵称；502为用户选定的要模仿的图像，如小视频，如用户选中了图3或图4页面中的小视频1，则在视频app切换页面至图5时，小视频1导入到图5所示的页面中。
在一些实例中，在所述将当前页面切换至模仿页面之后，所述应用客户端101启动图像采集装置，则开始通过图像采集装置采集第二媒体内容的数据。
这里，应用客户端101将当前页面切换至模仿页面后，首先请求设备中的摄像装置，若成功连接设备中的摄像装置，则尝试读取摄像装置采集的测试图像，应用客户端101可以通过该测试图像辨别摄像装置是否工作正常，之后，给用户时间提示以便用户开始进行图像模仿，应用客户端101则开始通过摄像装置采集所述第二媒体内容的数据；若不能成功连接设备中的摄像装置，则再次尝试连接，可重复多次。
例如，视频app将图3或如图4所示的页面切换至如图5所示的页面后，首先请求智能设备，如手机中的摄像装置，如摄像头，若能成功连接该手机中的摄像头，则给用户倒计时时间提示，如507，倒计时结束后，用户可以通过操作控件503进行开始图像模仿，在该图像模仿的过程中，视频app自动保存模仿图像(即第二媒体内容)的数据，形成用户模仿图像，若用户操作控件505(指示重录)或控件506(指示取消)，则视频app放弃保存所述模仿图像，即删除已录制的模仿图像。控件505可以为“重录”控件，控件506可以为“取消”控件。在507所示的区域，在倒计时提示结束后，还用于显示502区域中小视频的播放时间进度。
步骤203：获取根据所述图像采集装置采集到的图像数据而生成的第二媒体内容。
在一些实例中，所述获取根据所述图像采集装置采集到的图像数据而生成的第二媒体内容，包括：展示第二页面；在所述第二页面中展示所述图像采集装置的图像采集实时画面；响应于对所述第二页面中第二控件的操作，开始录制所述图像采集装置采集到的图像数据；当录制结 束条件满足时，停止录制所述图像数据并生成所述第二媒体内容。
例如，所述第二页面如图5所示，在504所示的区域展示所述图像采集装置(如摄像头)的图像采集实时画面，当用户操作控件503(即第二控件)时，视频app开始录制所述图像采集装置采集到的图像数据；当502区域中的小视频(即第一媒体内容)的播放时间进度结束或者用户点击结束按钮时，视频app停止录制所述图像数据并生成如图6中604所示的模仿视频(即第二媒体内容)。
在一些实例中，所述方法进一步包括：响应于对所述第二页面中所述第二控件的操作，开始在所述第二页面中展示所述第一媒体内容；其中，所述录制结束条件包括所述第一媒体内容展示完毕。
例如，在如图5所示的第二页面中，当用户操作控件503时(即视频app开始录制模仿视频)，视频app开始播放502区域中的小视频1(即第一媒体内容)，当小视频1的播放时间进度结束时，视频app录制结束。
在一些实例中，所述方法进一步包括：响应于对所述第二页面中第三控件的操作，停止录制所述图像数据并删除已录制的图像数据。
例如，在如图5所示的第二页面中，当用户操作控件506时，视频app停止录制所述图像数据并删除已录制的图像数据。
在一些实例中，所述方法进一步包括：响应于对所述第二页面中第四控件的操作，停止录制所述图像数据，删除已录制的图像数据，并重新开始录制所述图像采集装置采集到的图像数据。
例如，所述第四控件为图5中的“重录”控件505，当用户操作控件505后，视频app响应于上述操作，停止录制所述图像数据，删除已录制的图像数据，并重新开始录制所述图像采集装置采集到的图像数据。
步骤204：将所述第二媒体内容的数据发送至所述应用服务器。
其中，所述将所述第二媒体内容的数据发送至所述应用服务器，可以是采集所述第二媒体内容的数据结束之后，应用客户端101将采集完成的所述第二媒体内容的数据发送至所述应用服务器，同时，应用客户端101将所述第一媒体内容和所述第一媒体内容并列布置在页面中，以便用户对两者进行比较。
例如，在用户模仿结束后(如小视频1的播放时间进度结束)，视频app将如图5所示的页面切换至如图6所示的页面，并在604所示的区域展示所述模仿图像(如模仿视频)，所述模仿图像与602所示的小视频1并列布置，用户点击小视频1和模仿视频，就可同时播放小视频1和模仿视频，这样可以便于用户对两者进行比较，确定模仿效果，同时在607所示的区域展示小视频1的时间进度。
这里，应用客户端101将采集完成的所述第二媒体内容的数据进行保存，并将所述第二媒体内容的数据发送至所述应用服务器102，以便所述应用服务器102根据所述第一媒体内容对所述第二媒体内容进行处理。
步骤205：接收并展示所述应用服务器发送的所述第二媒体内容的相似度，其中，所述相似度通过对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配而得到。
在一些实例中，所述方法进一步包括：在获取所述第二媒体内容后，在第二页面中展示所述第二媒体内容的标识；在所述第二页面中展示所述第二媒体内容的所述相似度；响应于对所述第二媒体内容的标识的操作，展示所述第二媒体内容。
例如，在如图6所示的页面中，视频app录制的模仿视频的标识(例如模仿视频的名称，小视频1的模仿视频)显示在604所示的区域，用户可以操作所述标识，播放所述模仿视频，对该模仿视频进行预览，在 601区域展示所述模仿图像的得分(即相似度)，如图所示的93分。
在一些实例中，所述方法进一步包括：响应于对所述第二页面中第五控件的操作，将所述第二媒体内容发布到所述应用服务器，以使所述应用服务器为所述第二媒体内容生成链接地址。
例如，图6所示的页面还提供有“发布”控件603(即第五控件)以供用户将所述模仿图像发布到应用服务器102，之后，当用户和其它用户访问应用服务器的相应页面时，即可看到此模仿图像的信息，并可点击播放。当模仿图像成功发布到应用服务器102之后，应用服务器102可在图6所述的页面或者新的页面中给出分享选项，用户通过操作该分享选项，可以将模仿图像分享给其社交关系链中的好友或群组(如微博、微信、QQ中的好友或群组等)，或者将模仿图像分享到空间、微博、朋友圈等等。当用户操作控件603后，视频app将录制的用户模仿视频(即第二媒体内容)发布到视频服务器，视频服务器为所述模仿视频生成链接地址，以便其他用户访问视频app的公开页面(如“动态”)时，可以在公开页面中看到这个链接地址，点击播放所述模仿视频。
在一些实例中，所述方法进一步包括：响应于对所述第二页面中第六控件的操作，将所述第二媒体内容保存在本地并生成本地的链接地址。
例如，所述第六控件可以为图6中的“保存”控件606，当用户操作控件606后，视频app将录制的用户模仿视频(即第二媒体内容)保存在本地并生成本地的链接地址，之后，当用户访问本地作品页面时，会展示此链接地址，点击该链接地址可以播放视频；如图6所示的页面中还提供有“重录”控件605，该控件可以使得用户在不满意当前录制的所述模仿图像时，再次录制。
通过采用上述技术方案，可以得到所述第二媒体内容与所述第一媒 体内容之间的图像相似度，从而为所述应用客户端提供了新的应用功能，改善应用客户端的性能，进一步的提高了用户使用应客户端的趣味、娱乐性以及社交属性，提高该应用客户端的展示效果，从而进一步提高用户的体验效果。
在一些实例中，本申请实例还提出一种媒体数据处理方法，该方法应用于应用服务器102，如图7所示，该方法包括以下步骤：
步骤701：接收应用客户端发送的模仿请求，根据所述模仿请求中携带的第一媒体内容的标识获取所述第一媒体内容的数据。
步骤702：向所述应用客户端发送对于所述模仿请求的确认响应，以使所述应用客户端启动图像采集装置。
在一些实例中，所述方法进一步包括：预先设置训练集，其中，训练集包括N组相似的图像，N为大于1的整数；利用预设的公式模板，采用所述训练集训练得到用于计算图像距离的公式；其中，所述公式包括各图像特征的权重，一个权重表示一个图像中一个特征对图像距离的贡献度。
在一些实例中，所述方法进一步包括：预先设置测试样本，其中，测试样本包括M个图像，M为大于1的整数；利用所述用于计算图像距离的公式分别计算所述测试样本中M个图像与所述第一媒体内容中图像的M个图像距离；根据所述M个图像距离，建立图像距离与相似度分值之间的对应关系。
具体的，应用服务器102向应用客户端101发送对于所述模仿请求的确认响应后，根据所述第一媒体内容的标识获取所述第一媒体内容的数据，并建立图像距离与相似度分值之间的对应关系，所述对应关系求解过程如下：
首先，求解权重w
i。任意两个相似的表情图像I，T，表情图像I和 表情图像T的相似性度量具体可以采用基于面部表情几何特征的表情相似性度量方法，表情图像I和表情图像T的相似距离(或图像距离)为：
其中，w
i≥0(i＝1,2,…，p+q)表示第i个特征的权重，且满足
该特征为形状特征(例如眉、眼、鼻和嘴等)或形变特征(例如嘴角拉伸和收缩程度、眼睛和嘴巴开合大小、眉毛上扬和下压等)；δ≥0为权值作用系数；d
i(I,T)表示为表情图像I和表情图像T中第i个特征的差异，且具体以公式(2)表示：
其中，
和
为表情图像I和表情图像T的形状特征集合，
和
为表情图像I和表情图像T的形变特征集合，p表示表情区域数目，q表示需要度量的形变特征数目。
为确定权重w
i，筛选N组相似的表情图像(I
n,T
n)作为训练集，该训练集可以为由宾厄姆顿大学提供的BU-3DFE(Binhamton University 3D Facial Expression)数据库中的数据，相似的表情要求表情的类别(BU-3DFE数据库中将类别分为中性、高兴、厌恶、生气、恐惧、伤心和惊讶)相同及表情的强度(BU-3DFE数据库中将强度分为4个级别，强度1、强度2、强度3及强度4)相同，该数据库包含100个测试者，每个测试者均具有中性及6类基本表情，且每个基本表情具有4个强度。 为达到与人无关的表情相似性度量，随机选择50个人表情图像并将任意两人同类别同强度的表情作为一组，即将产生的
组相似表情作为训练集，即
并将w
i的求解过程转化为相似距离目标函数J
w最小化，即
令
则式(3)可简化为
由式(4)可知，δ＝0为无特征加权，δ＝1为线性特征加权，即为线性规划问题，可由单纯形方法(单纯形方法是一种直接、快速的搜索最小值方法，其优点是对目标函数的解析性没有要求，收敛速度快，适用面较广)求解其最优w
i，但实验表明其度量结果不佳。通过实验比较，本文选取权值作用系数δ＝2，故式(4)最小化问题转化为二次规划问题并可利用拉格朗日条件极值法求解J
w极值，引入拉格朗日函数：
对上式w
i和λ分别求偏导数，并令两个偏导数等于0，可得
解上述方程组可得
其中，
w
i反映第i个特征对表情相似距离的贡献度，一般地，眉毛、眼睛、嘴巴这些区域的形状特征对表情的表达效果较显著，脸颊下巴次之，而鼻子往往不能体现表情变化；同时各个区域的形变幅度也不尽相同，即各个区域的形变特征对表情相似距离的贡献度也不同，嘴部区域的形变幅度大于眼睛区域，眉毛区域的形变幅度也不及嘴巴和脸颊。
其次，建立图像距离与相似度分值之间的对应关系。利用式(7)求得w
i后，通过式(1)分别求解BU-3DFE数据库中M(M为余下的50个人的50×24＝1200幅表情图像)个样本与所述第一媒体内容中表情图像的M个图像距离，并建立图像距离与相似度分值之间的对应关系，例如将上述M个图像距离划分为100个间隔，其中，每一个间隔中包括若干个图像距离，每一间隔相应地分别对应1～100分中的某一个分值，如落在第86个间隔的某个图像距离对应86分。
在一些实例中，当所述第一媒体内容和所述第二媒体内容包括视频时，所述第一媒体内容的数据和所述第二媒体内容的数据均包括多个帧图像，针对所述第一媒体内容数据中的每一帧图像进行如下处理：预先设置测试样本，其中，测试样本包括M个图像，M为大于1的整数；利用所述用于计算图像距离的公式分别计算所述测试样本中M个图像与所述第一媒体内容中该帧图像的图像距离；根据所述M个图像距离，建立图像距离与相似度分值之间的对应关系。
具体的，在本申请实例中，所述应用服务器接收应用客户端发送的所述第一媒体内容(如小视频)后，基于BU-3DFE数据库中的N组相似的表情图像(I
n,T
n)训练集求得w
i，然后对小视频中的每一帧表情图像，利用公式(1)分别计算测试样本中M个图像与所述第一媒体内容中该帧图像的图像距离，根据所述M个图像距离，建立图像距离与相似度分值之间的对应关系。
步骤703：接收所述应用客户端发送的第二媒体内容的数据。所述第二媒体内容是根据所述图像采集装置采集到的图像数据而生成的。
步骤704：根据所述第一媒体内容和所述第二媒体内容的数据，对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配，确定所述第二媒体内容的相似度。
在一些实例中，所述对第二媒体内容与所述第一媒体内容进行图像相似度匹配，确定所述第二媒体内容的相似度，包括：根据所述第一媒体内容和所述第二媒体内容的数据，利用所述用于计算图像距离的公式计算所述第二媒体内容的图像与所述第一媒体内容的图像之间的图像距离；根据所述对应关系，确定所述第二媒体内容的图像与所述第一媒体内容的图像之间的图像距离对应的相似度分值。
在一些实例中，当所述第一媒体内容和所述第二媒体内容包括视频时，所述第一媒体内容的数据和所述第二媒体内容的数据均包括多个帧图像，所述对第二媒体内容与所述第一媒体内容进行图像相似度匹配，确定所述第二媒体内容的相似度，包括：针对所述第二媒体内容数据中的每一帧图像进行如下处理：利用所述用于计算图像距离的公式计算所述第二媒体内容中该帧图像与所述第一媒体内容中相应帧图像之间的图像距离；根据所述对应关系，确定所述第二媒体内容中该帧图像与所述第一媒体内容中相应帧图像之间的图像距离对应的相似度分值；最 后，根据所述第二媒体内容的所有帧图像的相似度分值，确定所述第二媒体内容的相似度分值。
这里，例如可以将所述第二媒体内容的所有帧图像的相似度分值的平均值或加权平均值，确定为所述第二媒体内容的相似度分值。
例如，所述第一媒体内容为小视频时，应用服务器102可以对小视频中连续的表情动作进行采集，以及对所述第二媒体内容中相应的表情动作进行采集，将所述连续的表情动作中的每一帧图像分别进行相似性度量；或者所述第一媒体内容为剧照时，应用服务器102对所述第二媒体内容与所述第一媒体内容进行相似性度量。
步骤705：将所述第二媒体内容的相似度发送至所述应用客户端，以使所述应用客户端展示所述第二媒体内容的相似度。
图8示出了本申请一实例的消息交互图，在此实例中，视频app对应于上文的应用客户端，可包括能够实现上述方法的模仿模块；视频服务器对应于前述的应用服务器，可包括用于实现上述方法的相似匹配模块。如图8所示，包括如下步骤：
步骤801：响应于用户的操作，所述操作可以为用户点击视频app播放某一视频页面中的“模仿秀”按钮，如图3所示，或用户点击视频app展示多个小视频页面中的“模仿秀”按钮，如图4所示，视频app向视频服务器发送模仿请求，所述模仿请求携带用户通过控件305或控件403选中的要模仿的小视频(或小视频的标识)，其中，所述小视频为其他用户在其观看的视频中截取的一部分或视频app或视频服务器提供的一段视频，时长可以为几秒或几十秒。
步骤802：视频服务器响应于上述模仿请求，将上述小视频保存，保存完毕后，向视频app发送对于所述模仿请求的确认响应，以告知视频app可以进入模仿阶段。同时，视频服务器中的相似匹配模块基于由 宾厄姆顿大学提供的BU-3DFE数据库中的数据，建立图像距离与相似度分值之间的对应关系。
步骤803：视频app接收上述对于所述模仿请求的确认响应后，将视频app的界面切换至模仿界面，如图5所示，并启动摄像头，读取摄像头信息。
步骤804：视频app成功读取到摄像头信息后，给用户时间提示(如倒计时)，如507所示，让用户准备开始模仿。
步骤805：视频app通过摄像头采集用户模仿的图像数据，并自动保存所述图像数据，根据所述图像数据录制模仿视频，将所述模仿视频发送至视频服务器。
步骤806：视频服务器中的相似度匹配模块将所述小视频与所述用户模仿视频进行图像相似度匹配，得到相似度分值。
在一些实例中，可利用公式(1)计算模仿视频中每一帧表情图像与小视频中相应帧表情图像的图像距离，并根据所述对应关系，确定该帧表情图像与所述第一媒体内容中相应帧图像之间的图像距离对应的相似度分值。因所述小视频和模仿图像包含至少一个连续的表情动作，即包含多个连续帧，该模仿图像中连续的表情动作的最终得分可以通过该连续表情动作的每一帧表情图像的相似度分值的加权平均得到。
步骤807：将所述模仿视频的相似度分值发送至视频app，以使得所述视频app展示该相似度分值。
步骤808：响应于“发布”操作，将所述模仿视频发布至本视频app的应用服务器，使得视频app的其它用户也可以浏览到此模仿视频，进一步的视频app还可以提供“分享”选项，以将该模仿视频分享到其他app，如分享给当前视频app用户的诸如微博、微信、QQ等社交应用中的好友或群组，或者分享到微博、QQ空间或微信朋友圈等；或响应于 “重录”控件，返回至步骤803；或响应于“保存”，将所述模仿视频保存在本地，以供用户后续进行查看等。
基于上述实例提供的方法，本申请实例还提出了一种媒体数据处理装置900，如图9所示，所述装置900包括：
发送模块901，响应于对第一页面中第一媒体内容的选择和对第一控件的操作，向应用服务器发送模仿请求，其中，所述模仿请求携带所述第一媒体内容的标识；及将第二媒体内容的数据发送至所述应用服务器。
启动模块902，响应于来自所述应用服务器的对于所述模仿请求的确认响应，启动图像采集装置。
获取模块903，获取根据所述图像采集装置采集到的图像数据而生成的第二媒体内容。
展示模块904，接收并展示所述应用服务器发送的所述第二媒体内容的相似度，其中，所述相似度通过对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配而得到。
在一些实例中，所述展示模块展示第二页面；在所述第二页面中展示所述图像采集装置的图像采集实时画面；响应于对所述第二页面中第二控件的操作，所述获取模块903开始录制所述图像采集装置采集到的图像数据；当录制结束条件满足时，所述获取模块903停止录制所述图像数据并生成所述第二媒体内容。
在一些实例中，在所述获取模块903获取所述第二媒体内容后，所述展示模块进一步在第二页面中展示所述第二媒体内容的标识；及在所述第二页面中进一步展示所述第二媒体内容的所述相似度；响应于对所述第二媒体内容的标识的操作，所述展示模块展示所述第二媒体内容。
在一些实例中，本申请实例还提供了一种媒体数据处理装置1000，如图10所示，该装置1000包括：
接收模块1001，接收应用客户端发送的模仿请求，根据所述模仿请求中携带的第一媒体内容的标识获取所述第一媒体内容的数据；及接收所述应用客户端发送的第二媒体内容的数据；
发送模块1002，向所述应用客户端发送对于所述模仿请求的确认响应，以使所述应用客户端启动图像采集装置；及将所述第二媒体内容的相似度发送至所述应用客户端，以使所述应用客户端展示所述第二媒体内容的相似度；
匹配模块1003，根据所述第一媒体内容和所述第二媒体内容的数据，对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配，确定所述第二媒体内容的相似度。
上述各模块功能的具体实现原理在前文已有描述，这里不再赘述。
另外，在本申请各个实例中的媒体数据处理方法和媒体数据处理装置以及其中的各模块可以集成在一个处理单元中，也可以是各个模块单独物理存在，也可以两个或两个以上装置或模块集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
在一实例中，上述的媒体数据处理装置可运行在各种可基于互联网而进行用户信息处理的计算设备中，并加载在该计算设备的存储器中。
图11示出了媒体数据处理装置所在的计算设备的组成结构图。如图11所示，该计算设备包括一个或者多个处理器(CPU)1102、通信模块1104、存储器1106、用户接口1110，以及用于互联这些组件的通信总线1108。
处理器1102可通过通信模块1104接收和发送数据以实现网络通信 和/或本地通信。
用户接口1110包括一个或多个输出设备1112，其包括一个或多个扬声器和/或一个或多个可视化显示器。用户接口1110也包括一个或多个输入设备1114，其包括诸如，键盘，鼠标，声音命令输入单元或扩音器，触屏显示器，触敏输入板，姿势捕获摄像机或其他输入按钮或控件等。
存储器1106可以是高速随机存取存储器，诸如DRAM、SRAM、DDR RAM、或其他随机存取固态存储设备；或者非易失性存储器，诸如一个或多个磁盘存储设备、光盘存储设备、闪存设备，或其他非易失性固态存储设备。
存储器1106存储处理器1102可执行的指令集，包括：
操作系统1116，包括用于处理各种基本系统服务和用于执行硬件相关任务的程序；
应用1118，包括用于实现媒体数据处理的各种程序，这种程序能够实现上述各实例中的处理流程，比如可以包括图9所示的装置900和/或图10所示的装置1000。
在一些实例中，装置900可包括图9所示的各模块901～904，各模块901～904可以存储有机器可执行指令。处理器1102通过执行存储器1106中各模块901～904中的机器可执行指令，进而能够实现上述各模块901～904的功能。
在一些实例中，装置1000可包括图10所示的各模块1001～1003，各模块1001～1003可以存储有机器可执行指令。处理器1102通过执行存储器1106中各模块1001～1003中的机器可执行指令，进而能够实现上述各模块1001～1003的功能。
另外，本申请的每一个实例可以通过由数据处理设备如计算机执行 的数据处理程序来实现。显然，数据处理程序构成了本发明。此外，通常存储在一个存储介质中的数据处理程序通过直接将程序读取出存储介质或者通过将程序安装或复制到数据处理设备的存储设备(如硬盘和或内存)中执行。因此，这样的存储介质也构成了本发明。存储介质可以使用任何类型的记录方式，例如纸张存储介质(如纸带等)、磁存储介质(如软盘、硬盘、闪存等)、光存储介质(如CD-ROM等)、磁光存储介质(如MO等)等。
因此，本申请实例还公开了一种非易失性存储介质，其中存储有数据处理程序，该数据处理程序用于执行本申请上述方法的任何一种实例。
另外，本申请所述的方法步骤除了可以用数据处理程序来实现，还可以由硬件来实现，例如，可以由逻辑门、开关、专用集成电路(ASIC)、可编程逻辑控制器和嵌微控制器等来实现。因此这种可以实现本申请所述方法的硬件也可以构成本申请。
以上所述仅为本申请的较佳实例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。
Claims (16)
- 一种媒体数据处理方法，应用于终端设备，该方法包括：响应于对第一页面中第一媒体内容的选择和对第一控件的操作，向应用服务器发送模仿请求，其中，所述模仿请求携带所述第一媒体内容的标识；响应于来自所述应用服务器的对于所述模仿请求的确认响应，启动图像采集装置；获取根据所述图像采集装置采集到的图像数据而生成的第二媒体内容；将所述第二媒体内容的数据发送至所述应用服务器；接收并展示所述应用服务器发送的所述第二媒体内容的相似度，其中，所述相似度通过对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配而得到。
- 根据权利要求1所述的方法，其中，所述获取根据所述图像采集装置采集到的图像数据而生成的第二媒体内容，包括：展示第二页面；在所述第二页面中展示所述图像采集装置的图像采集实时画面；响应于对所述第二页面中第二控件的操作，开始录制所述图像采集装置采集到的图像数据；当录制结束条件满足时，停止录制所述图像数据并生成所述第二媒体内容。
- 根据权利要求2所述的方法，进一步包括：响应于对所述第二页面中所述第二控件的操作，开始在所述第二页面中展示所述第一媒体内容；其中，所述录制结束条件包括所述第一媒体内容展示完毕。
- 根据权利要求2或3所述的方法，进一步包括：响应于对所述第二页面中第三控件的操作，停止录制所述图像数据并删除已录制的图像数据。
- 根据权利要求2或3所述的方法，进一步包括：响应于对所述第二页面中第四控件的操作，停止录制所述图像数据，删除已录制的图像数据，并重新开始录制所述图像采集装置采集到的图像数据。
- 根据权利要求1所述的方法，进一步包括：在获取所述第二媒体内容后，在第二页面中展示所述第二媒体内容的标识；在所述第二页面中展示所述第二媒体内容的所述相似度；响应于对所述第二媒体内容的标识的操作，展示所述第二媒体内容。
- 根据权利要求6所述的方法，进一步包括：响应于对所述第二页面中第五控件的操作，将所述第二媒体内容发布到所述应用服务器，以使所述应用服务器为所述第二媒体内容生成链接地址。
- 一种媒体数据处理方法，应用于服务器设备，该方法包括：接收应用客户端发送的模仿请求，根据所述模仿请求中携带的第一媒体内容的标识获取所述第一媒体内容的数据；向所述应用客户端发送对于所述模仿请求的确认响应，以使所述应用客户端启动图像采集装置；接收所述应用客户端发送的第二媒体内容的数据；所述第二媒体内容是根据所述图像采集装置采集到的图像数据而生成的；根据所述第一媒体内容和所述第二媒体内容的数据，对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配，确定所述第二媒体内容的相似度；将所述第二媒体内容的相似度发送至所述应用客户端，以使所述应用客户端展示所述第二媒体内容的相似度。
- 根据权利要求8所述的方法，进一步包括：预先设置训练集，其中，训练集包括N组相似的图像，N为大于1的整数；利用预设的公式模板，采用所述训练集训练得到用于计算图像距离的公式；其中，所述公式包括各图像特征的权重，一个权重表示一个图像中一个特征对图像距离的贡献度。
- 根据权利要求9所述的方法，进一步包括：预先设置测试样本，其中，测试样本包括M个图像，M为大于1的整数；利用所述用于计算图像距离的公式分别计算所述测试样本中M个图像与所述第一媒体内容中图像的M个图像距离；根据所述M个图像距离，建立图像距离与相似度分值之间的对应关系；其中，所述对第二媒体内容与所述第一媒体内容进行图像相似度匹配，确定所述第二媒体内容的相似度，包括：根据所述第一媒体内容的数据和所述第二媒体内容的数据，利用所述用于计算图像距离的公式计算所述第二媒体内容的图像与所述第一媒体内容的图像之间的图像距离；根据所述对应关系，确定所述第二媒体内容的图像与所述第一媒体内容的图像之间的图像距离对应的相似度分值。
- 根据权利要求9所述的方法，其中，当所述第一媒体内容和所述第二媒体内容包括视频时，所述第一媒体内容的数据和所述第二媒体内容的数据均包括多个帧图像，针对所述第一媒体内容数据中的每一帧图像进行如下处理：预先设置测试样本，其中，测试样本包括M个图像，M为大于1的整数；利用所述用于计算图像距离的公式分别计算所述测试样本中M个图像与所述第一媒体内容中该帧图像的图像距离；根据M个所述图像距离，建立图像距离与相似度分值之间的对应关系；其中，所述对第二媒体内容与所述第一媒体内容进行图像相似度匹配，确定所述第二媒体内容的相似度，包括：针对所述第二媒体内容数据中的每一帧图像进行如下处理：利用所述用于计算图像距离的公式计算所述第二媒体内容中该帧图像与所述第一媒体内容中相应帧图像之间的图像距离；根据所述对应关系，确定所述第二媒体内容中该帧图像与所述第一媒体内容中相应帧图像之间的图像距离对应的相似度分值；根据所述第二媒体内容的所有帧图像的相似度分值，确定所述第二媒体内容的相似度分值。
- 一种媒体数据处理装置，该装置包括处理器和存储器，所述存储器中存储有计算机可读指令，所述指令可以使所述处理器：响应于对第一页面中第一媒体内容的选择和对第一控件的操作，向应用服务器发送模仿请求，其中，所述模仿请求携带所述第一媒体内容的标识；响应于来自所述应用服务器的对于所述模仿请求的确认响应，启动图像采集装置；获取根据所述图像采集装置采集到的图像数据而生成的第二媒体内容；将第二媒体内容的数据发送至所述应用服务器；及接收并展示所述应用服务器发送的所述第二媒体内容的相似度，其中，所述相似度通过对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配而得到。
- 根据权利要求12所述的装置，其中，所述计算机可读指令可以使所述处理器：展示第二页面；在所述第二页面中展示所述图像采集装置的图像采集实时画面；响应于对所述第二页面中第二控件的操作，开始录制所述图像采集装置采集到的图像数据；当录制结束条件满足时，停止录制所述图像数据并生成所述第二媒体内容。
- 根据权利要求12所述的装置，其中，所述计算机可读指令可以使所述处理器：在获取所述第二媒体内容后，在第二页面中展示所述第二媒体内容的标识；在所述第二页面中展示所述第二媒体内容的所述相似度；响应于对所述第二媒体内容的标识的操作，展示所述第二媒体内容。
- 一种媒体数据处理装置，该装置包括处理器和存储器，所述存储器中存储有计算机可读指令，所述指令可以使所述处理器：接收应用客户端发送的模仿请求，根据所述模仿请求中携带的第一媒体内容的标识获取所述第一媒体内容的数据；向所述应用客户端发送对于所述模仿请求的确认响应，以使所述应用客户端启动图像采集装置；接收所述应用客户端发送的第二媒体内容的数据，所述第二媒体内容是根据所述图像采集装置采集到的图像数据而生成的；根据所述第一媒体内容和所述第二媒体内容的数据，对所述第二媒体内容与所述第一媒体内容进行图像相似度匹配，确定所述第二媒体内容的相似度；将所述第二媒体内容的相似度发送至所述应用客户端，以使所述应用客户端展示所述第二媒体内容的相似度。
- 一种存储介质，存储有计算机可读指令，可以使至少一个处理器执行如权利要求1至11任一项所述的方法。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710457646.6 | 2017-06-16 | ||
| CN201710457646.6A CN107257338B (zh) | 2017-06-16 | 2017-06-16 | 媒体数据处理方法、装置及存储介质 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2018228037A1 true WO2018228037A1 (zh) | 2018-12-20 |
Family
ID=60023149
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2018/082424 WO2018228037A1 (zh) | 2017-06-16 | 2018-04-10 | 媒体数据处理方法、装置及存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107257338B (zh) |
| WO (1) | WO2018228037A1 (zh) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110209871A (zh) * | 2019-06-17 | 2019-09-06 | 广州酷狗计算机科技有限公司 | 歌曲评论发布方法及装置 |
| CN111460343A (zh) * | 2019-01-21 | 2020-07-28 | 阿里巴巴集团控股有限公司 | 展示方法、装置、终端设备及计算机存储介质 |
| CN111626253A (zh) * | 2020-06-02 | 2020-09-04 | 上海商汤智能科技有限公司 | 表情检测的方法、装置、电子设备及存储介质 |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107257338B (zh) * | 2017-06-16 | 2018-09-28 | 腾讯科技（深圳）有限公司 | 媒体数据处理方法、装置及存储介质 |
| CN107820132B (zh) * | 2017-11-21 | 2019-12-06 | 广州华多网络科技有限公司 | 直播互动方法、装置及系统 |
| CN108197915A (zh) * | 2017-12-22 | 2018-06-22 | 广州爱美互动网络科技有限公司 | 一种虚拟红包信息处理方法 |
| CN108256835A (zh) * | 2018-01-10 | 2018-07-06 | 百度在线网络技术（北京）有限公司 | 电子红包的实现方法、装置及服务器 |
| CN108419036B (zh) * | 2018-05-07 | 2019-08-30 | 北京微播视界科技有限公司 | 一种视频拍摄方法、装置和终端设备 |
| CN108762500A (zh) * | 2018-05-23 | 2018-11-06 | 深圳市创艺工业技术有限公司 | 一种智能机器人 |
| CN109361954B (zh) * | 2018-11-02 | 2021-03-26 | 腾讯科技（深圳）有限公司 | 视频资源的录制方法、装置、存储介质及电子装置 |
| CN110879967B (zh) * | 2019-10-16 | 2023-02-17 | 厦门美柚股份有限公司 | 视频内容重复判断方法及装置 |
| CN113050800B (zh) | 2021-03-31 | 2023-06-23 | 联想(北京)有限公司 | 一种电子设备及处理方法 |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140361974A1 (en) * | 2013-06-05 | 2014-12-11 | Wenlong Li | Karaoke avatar animation based on facial motion data |
| CN105898133A (zh) * | 2015-08-19 | 2016-08-24 | 乐视网信息技术（北京）股份有限公司 | 一种视频拍摄方法及装置 |
| CN106205633A (zh) * | 2016-07-06 | 2016-12-07 | 李彦芝 | 一种模仿、表演练习打分系统 |
| CN107257338A (zh) * | 2017-06-16 | 2017-10-17 | 腾讯科技（深圳）有限公司 | 媒体数据处理方法、装置及存储介质 |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012167568A1 (zh) * | 2011-11-23 | 2012-12-13 | 华为技术有限公司 | 视频广告播放方法、设备和系统 |
| EP2954459A4 (en) * | 2013-02-08 | 2016-10-19 | Emotient Inc | HARVESTING MACHINE LEARNING TRAINING DATA FOR EXPRESSION RECOGNITION |
| CN103996029B (zh) * | 2014-05-23 | 2017-12-05 | 安庆师范学院 | 表情相似度度量方法及装置 |
| CN104133851B (zh) * | 2014-07-07 | 2018-09-04 | 小米科技有限责任公司 | 音频相似度的检测方法和检测装置、电子设备 |
| CN104299612B (zh) * | 2014-11-10 | 2017-11-07 | 科大讯飞股份有限公司 | 模仿音相似度的检测方法和装置 |
| CN106034240A (zh) * | 2015-03-13 | 2016-10-19 | 小米科技有限责任公司 | 视频检测方法及装置 |
| CN104810025B (zh) * | 2015-03-31 | 2018-04-20 | 天翼爱音乐文化科技有限公司 | 音频相似度检测方法及装置 |
| CN104882147A (zh) * | 2015-06-05 | 2015-09-02 | 福建星网视易信息系统有限公司 | 一种演唱评分显示方法、装置及系统 |
| CN106126617B (zh) * | 2016-06-22 | 2018-11-23 | 腾讯科技（深圳）有限公司 | 一种视频检测方法及服务器 |
| CN106228143A (zh) * | 2016-08-02 | 2016-12-14 | 王国兴 | 一种教学视频与摄像头视频运动对比评分的方法 |
| CN106531185B (zh) * | 2016-11-01 | 2019-12-13 | 云知声（上海）智能科技有限公司 | 基于语音相似度的语音评测方法及系统 |
-
2017
- 2017-06-16 CN CN201710457646.6A patent/CN107257338B/zh active Active
-
2018
- 2018-04-10 WO PCT/CN2018/082424 patent/WO2018228037A1/zh active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140361974A1 (en) * | 2013-06-05 | 2014-12-11 | Wenlong Li | Karaoke avatar animation based on facial motion data |
| CN105898133A (zh) * | 2015-08-19 | 2016-08-24 | 乐视网信息技术（北京）股份有限公司 | 一种视频拍摄方法及装置 |
| CN106205633A (zh) * | 2016-07-06 | 2016-12-07 | 李彦芝 | 一种模仿、表演练习打分系统 |
| CN107257338A (zh) * | 2017-06-16 | 2017-10-17 | 腾讯科技（深圳）有限公司 | 媒体数据处理方法、装置及存储介质 |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111460343A (zh) * | 2019-01-21 | 2020-07-28 | 阿里巴巴集团控股有限公司 | 展示方法、装置、终端设备及计算机存储介质 |
| CN111460343B (zh) * | 2019-01-21 | 2023-04-28 | 阿里巴巴集团控股有限公司 | 展示方法、装置、终端设备及计算机存储介质 |
| CN110209871A (zh) * | 2019-06-17 | 2019-09-06 | 广州酷狗计算机科技有限公司 | 歌曲评论发布方法及装置 |
| CN110209871B (zh) * | 2019-06-17 | 2024-04-12 | 广州酷狗计算机科技有限公司 | 歌曲评论发布方法及装置 |
| CN111626253A (zh) * | 2020-06-02 | 2020-09-04 | 上海商汤智能科技有限公司 | 表情检测的方法、装置、电子设备及存储介质 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107257338A (zh) | 2017-10-17 |
| CN107257338B (zh) | 2018-09-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2018228037A1 (zh) | 媒体数据处理方法、装置及存储介质 | |
| KR102264613B1 (ko) | 메시지 파라미터에 의한 메시지들의 라우팅 | |
| US20190237106A1 (en) | Gallery of videos set to an audio time line | |
| JP6200483B2 (ja) | 画像処理システム、画像処理方法、および画像処理プログラム | |
| KR20190084278A (ko) | 이미지들을 공유하기 위한 자동 제안들 | |
| US11934643B2 (en) | Analyzing augmented reality content item usage data | |
| EP4164760A1 (en) | Game result overlay system | |
| US20230022826A1 (en) | Media content discard notification system | |
| EP4222961A1 (en) | Method, system and computer-readable storage medium for image animation | |
| WO2019227633A1 (zh) | 建立用户画像和建立状态信息分析模型的方法和装置 | |
| JP2021005768A (ja) | コンピュータプログラム、情報処理方法および動画配信システム | |
| US20230214875A1 (en) | Content-based incentive program within messaging system | |
| WO2020078157A1 (zh) | 约跑方法、装置及计算机可读存储介质 | |
| JP7206741B2 (ja) | 健康状態判定システム、健康状態判定装置、サーバ、健康状態判定方法、及びプログラム | |
| CN111666793A (zh) | 视频处理方法、视频处理装置和电子设备 | |
| US11693539B2 (en) | Messaging system for resurfacing content items | |
| JP7064159B1 (ja) | 情報処理方法、プログラム、及び情報処理装置 | |
| US20240212264A1 (en) | Program, information processing method, and information processing system | |
| WO2023245488A1 (en) | Double camera streams | |
| KR102174569B1 (ko) | 증강현실 기반 정보 제공 방법 | |
| US20230195856A1 (en) | Method for media creation, sharing, and communication and associated system | |
| CN111104854A (zh) | 评价信息处理方法、装置、电子设备及图像处理方法 | |
| WO2023192608A1 (en) | Smart media overlay selection for a messaging system | |
| CN114764706A (zh) | 电子支付处理方法、装置及电子设备 | |
| CN117412184A (zh) | 数字人照片拍摄的方法、装置、存储介质及电子设备 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18818540 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 18818540 Country of ref document: EP Kind code of ref document: A1 |