WO2013189317A1

WO2013189317A1 - Human face information-based multimedia interaction method, device and terminal

Info

Publication number: WO2013189317A1
Application number: PCT/CN2013/080766
Authority: WO
Inventors: 金骏; 孙奥
Original assignee: 中兴通讯股份有限公司
Priority date: 2012-12-28
Filing date: 2013-08-02
Publication date: 2013-12-27
Also published as: CN103076879A

Abstract

Disclosed are a biological feature information-based multimedia interaction method, device and terminal. The method comprises: associating biological feature information about a contact person with the contact information about the contact person in advance in a terminal and storing same; extracting biological feature information from the current multimedia data, and matching the extracted biological feature information and the previously stored biological feature information; and if it is determined that matching is successful, retrieving the contact information about the contact person corresponding to the biological feature information for interaction. Compared with the previous interaction mode, the mode of the present invention is simple and effective, the interface is friendly, and the efficiency is higher, thereby greatly improving the system friendliness.

Description

Multimedia interaction method and device based on face information and terminal

The present invention relates to the field of communications technologies, and in particular, to a multimedia interaction method and apparatus based on face information and a terminal. Background technique

Human-Computer Interaction Techniques are technologies that enable people to talk to computers in an efficient manner through computer input and output devices.

Human-computer interaction technology emerged with the advent of computers. From the initial manual operation stage to the human-computer interaction technology represented by virtual reality technology, human-computer interaction technology has experienced many stages. The history of human-computer interaction has shown that it is gradually moving from a computer-centric complex interaction to a simple, natural interaction with people. The current multimedia-based, multi-channel human-computer interaction technology has achieved a lot of research results, such as text recognition, gesture recognition, speech recognition, speech synthesis and many other research results and products.

With the development of electronic technology, the popularity of mobile terminal devices such as mobile phones, MP4s, tablets, digital cameras, digital video cameras, etc. is increasing. People use the multimedia recording function of these devices to record a large amount of multimedia data, including photos and sound. And silent video recordings and more. In the recorded multimedia files, the majority of the content in the content is involved. With the development of the times, advances in technology, and improvement in people's living standards, there are more and more similar multimedia materials owned by individuals. These multimedia materials exist as long-term records in life and work or in memory. When people browse these materials, they can only passively obtain information from the data, and cannot further carry out the content on the data (mainly refers to the characters in the materials). Interaction, if the user needs to communicate further with the person recorded on the multimedia material, he or she must manually obtain another contact information, such as an address book, a memo, etc., to obtain the contact information of the contact, and then proceed with the object. Step contact.

The disadvantages of the above methods include: The interaction process is unfriendly, and the manual operation in the initial stage of human-computer interaction development is basically the same, and the efficiency is low. When the time is long, the information of the characters on the multimedia data cannot be recalled, and further interaction will be failure. Summary of the invention

In view of the above analysis, the embodiments of the present invention are directed to a multimedia interaction method and device based on face information, and a terminal, which are used to solve the problem of unfriendly human interaction in the prior art and low efficiency in the interaction process. The problem of interaction failure.

The purpose of the embodiments of the present invention is mainly achieved by the following technical solutions:

A multimedia interaction method based on biometric information, comprising:

Correlating and storing the biometric information of the contact with the contact information of the contact in the terminal;

The biometric information is extracted in the current multimedia material, and the extracted biometric information is matched with the previously stored biometric information. If the matching is determined to be successful, the contact information of the contact corresponding to the biometric information is called to interact.

If the matching fails, the biometric information is further matched to the network social media related to the end user or the terminal is directly requested to perform storage processing.

After the authorization, the network social media related to the end user is accessed, and the extracted biometric information is further matched with the biometric information of the contact stored in the social media, and if it is determined that the further matching is successful, the call is made. The contact information of the contact corresponding to the biometric information is exchanged; if the further matching fails, the terminal is requested to perform an edit storage process.

The step of the requesting terminal performing the edit storage process includes:

Requesting the terminal to store the biometric information and the corresponding known contact information, and associate the biometric information with the known contact information; if the biometric information pair is not currently known The contact information is stored, and only the biometric information is stored as a reserve.

Wherein, when the matching is successful and the contact information of the contact corresponding to the biometric information is called, the contact information is modified as needed.

The current multimedia material is multimedia material existing in the terminal, or is a multimedia material captured by the terminal in real time.

The biometric information is face information.

A multimedia interaction device based on biometric information, comprising:

The associated storage module is configured to associate and store the biometric information of the contact with the contact information of the contact in the terminal;

Matching the interaction module, extracting the biometric information in the current multimedia material, and matching the extracted biometric information with the previously stored biometric information, and if it is determined that the matching is successful, calling the contact corresponding to the biometric information Contact information to interact.

The matching interaction module is further configured to: if the matching fails, further matching the biometric information to the network social media related to the end user or requesting the associated storage module to perform editing storage processing.

A terminal comprising at least the biometric information based multimedia interaction device of claim 8 or 9.

The beneficial effects of the embodiments of the present invention are as follows:

The embodiment of the invention further excavates the information in the image, and the method is simple, effective, friendly, and efficient compared with the previous interaction mode, thereby greatly improving the friendliness of the system. DRAWINGS

1 is a schematic flow chart of a first method embodiment of the present invention;

2 is a schematic flow chart of a second method embodiment of the present invention;

3 is a schematic structural diagram of an apparatus according to an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of a terminal according to an embodiment of the present invention. detailed description

The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings in which FIG. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. It should be noted that, in the case of no conflict, the features in the embodiments and the embodiments in the present application may be arbitrarily combined with each other.

First, the method embodiments of the present invention will be described in detail with reference to FIGS. 1 and 2.

First method embodiment:

As shown in FIG. 1 , FIG. 1 is a schematic flowchart of a first method embodiment of the present invention, which may specifically include:

Step 101: Associate the biometric information of the contact with the contact information of the contact in the terminal, and store the data after the association, where the storage location may be the terminal local or the server;

Step 102: Extract biometric information in the current multimedia material, and match the extracted biometric information with the previously stored biometric information, and if it is determined that the matching is successful, call the contact information of the contact corresponding to the biometric information. Interact.

The first method embodiment of the present invention automatically detects the biometric information in the multimedia data, and extracts the identified biometric information to match the previously stored biometric information, thereby calling up the contact information of the corresponding contact. Automatically interact with multimedia materials.

Second method embodiment:

As shown in FIG. 2, FIG. 2 is a schematic flowchart of a second method embodiment of the present invention, which may specifically include the following steps:

Step 200: Collect biometric information of the contact; the collection manner may be multiple, for example, may be collected by the terminal in real time, or may be downloaded to the terminal through a computer or a network; Step 201: Associate the biometric information of the contact with the contact information of the contact in the terminal, and the storage location may be the terminal local or the server; specifically, the user collects through the collection device or the Internet or the computer. The biometric information of the contact obtained by importing and the like is associated with the contact information of the contact, and the contact information may be edited and modified according to the actual situation, for example, adding new contact information or modifying the existing contact information, etc. After the association, the contact information includes: one or more of the contact name, the title, the unit, the phone number, the email address, the QQ number, the Kaixin account, and the like; when the user associates, the terminal may be based on the terminal The original contact information function software, for example, associates the biometric information of the contact with the corresponding phone number in the address book software; or establishes a total contact information database, and associates the biometric information of the contact with all or Part of the contact information Line association; wherein, the user can establish a contact information database by using various methods, including: (1) The user can establish a database according to the currently owned, already marked multimedia data set at a certain time; (2) adopting The method of function 1 incrementally establishes the database; (3) the user marks the personal information while browsing the data, and updates the marked data to the database in real time; if, if desired, with the person on the multimedia material The interaction triggers the terminal to extract the biometric information of the contact in the current multimedia material, and searches and matches the extracted biometric information with the previously stored biometric information;

Step 203: determining whether the matching is successful, if yes, performing step 204; otherwise, performing the step

207;

Step 204: Display the matching result, that is, the contact information of the contact, in an interactive manner on the user interface, for example, prompting the terminal user to send a message to the person, or sending an email to the person;

Step 205: Determine whether the terminal user needs to modify the contact information of the contact, if not modified, go to step 206, otherwise go to step 209; Step 206: The user operates according to the prompt, and enters a corresponding software function module, such as a short message processing software module;

Step 207: As a preferred embodiment of the present invention, when the biometric information of the contact is not matched to the contact information corresponding to the contact, the user may further search for the social media related to the end user after being authorized by the user. , further matching the biometric information in the social media according to the biometric information of the contact;

It should be noted that, when the biometric information of the contact is not matched to the contact information corresponding to the contact, the terminal may be directly requested to perform storage, that is, step 209 is performed;

Step 208: Determine whether the further matching is successful, if yes, go to step 204; otherwise, go to step 209;

Step 209: The terminal is requested to perform storage; specifically, the user is asked whether to store the biometric information of the contact and associate with the corresponding contact information. If yes, the requesting terminal stores the biometric information and the currently known contact information. And associating the biometric information with the contact information; if the contact information corresponding to the biometric information is not currently known, the biometric information may be selected to be stored as a reserve for later editing and use;

Step 210: The user performs the multimedia material reading offline after completing the recording. If there is still the next biometric information to be matched, the process proceeds to step 200, and the above process is repeatedly performed.

In order to facilitate the understanding of the method in the embodiment of the present invention, the face information is taken as an example, and the use process includes but is not limited to these scenarios.

Use scenario one:

User A uses the terminal to browse the local picture. When the user is browsing the picture with the image information of the person, the user may want to interact with the person in the photo. At this time, the face detection algorithm is used to detect the face of the image, and then the detected face is identified and matched based on the detection. The sample library user A for face recognition can be authorized and configured. For example, the user A authorization program can access the local contact database. In this case, the program matches the avatar information in the local contact of the user according to the detected face information, and performs recognition of the character in the current image. If the recognition is successful, a prompt box will pop up on the current image browsing interface, giving the user A some options that can be operated, such as calling a person in the photo, or sending a message to the person in the short message. If the search fails, there will be corresponding prompts on the interface, such as whether to complete the contact information, or further request the user's authorization to determine whether the search scope can be expanded. For example, user A can authorize the program to access the homepage of the website. The character recognition is further performed in the picture of the user's buddy list and the avatar information. If the identification is successful, the interface may be prompted whether to further interact with the task, for example, displaying the status update of the task for a recent period of time.

Use scenario 2:

When User A uses the handheld terminal for image or video capture, the currently acquired images can be analyzed in real time. Similar to the previous usage scenario, the face is first detected and the detected face information is identified. If the identification fails, the user is prompted to manually mark the multimedia information material so as to conveniently interact with the multimedia material in the future. The user information marked by the user is stored as a sample of the local person sample, which is also a library of image retrieval samples that are defaulted by the user when performing image interaction in the future.

The method according to the embodiment of the present invention is described in detail with reference to the accompanying drawings 1 and 2 and the two specific application scenarios. The apparatus according to the embodiment of the present invention will be described in detail below with reference to FIG.

As shown in FIG. 3, FIG. 3 is a schematic structural diagram of an apparatus according to an embodiment of the present invention, which may include: an associated storage module and a matching interaction module, where

The associated storage module is mainly responsible for pre-associating and storing the biometric information of the contact with the contact information of the contact in the terminal, and the storage location may be the terminal local or the server; the matching interaction module is mainly responsible for extracting the current multimedia data. Biometric information, and Matching the extracted biometric information with the previously stored biometric information, and if it is determined that the matching is successful, calling out the contact information of the contact corresponding to the biometric information to interact; if the matching fails, the biometric information is Further matching with the network social media related to the end user or requesting the associated storage module to perform editing storage processing.

It should be noted that the specific implementation process of the device in the embodiment of the present invention has been described in detail in the above method, and therefore will not be described herein.

Finally, the terminal according to the embodiment of the present invention is described in detail with reference to FIG.

As shown in FIG. 4, FIG. 4 is a schematic structural diagram of a terminal according to an embodiment of the present invention. The terminal includes at least the foregoing apparatus according to the embodiment of the present invention, where the apparatus includes: an associated storage module and a matching interaction module, where

The associated storage module is mainly responsible for pre-associating and storing the biometric information of the contact with the contact information of the contact in the terminal, and the storage location may be the terminal local or the server; the matching interaction module is mainly responsible for extracting the current multimedia data. Biometric information, and the extracted biometric information is matched with the previously stored biometric information. If it is determined that the matching is successful, the contact information of the contact corresponding to the biometric information is called to interact; if the matching fails, then The biometric information is further matched to the network social media related to the end user or the associated storage module is requested to perform an edit storage process.

In summary, the embodiment of the present invention provides a multimedia interaction method and device based on face information, and a terminal. Compared with the prior art, the embodiment of the present invention further mines information in an image, and some need manual intervention. The things are handled automatically, and the user only needs to make a selection based on the output information of the system. Compared with the previous interaction mode, this method is simple, effective, friendly, and efficient, which greatly enhances the friendliness of the system.

The above is only the preferred embodiment of the present invention and is not intended to limit the scope of the present invention.

Claims

claims

1. A multimedia interaction method based on biometric information, including:

Pre-associate and store the contact's biometric information with the contact's contact information in the terminal;

Extract biometric information from the current multimedia material, and match the extracted biometric information with previously stored biometric information. If the match is determined to be successful, the contact information of the contact corresponding to the biometric information is retrieved for interaction.

2. The method according to claim 1, wherein if the matching fails, further matching is performed on the network social media related to the terminal user based on the biometric information or the terminal is directly requested for storage processing.

3. The method according to claim 2, wherein, after being authorized to access the online social media related to the terminal user, the extracted biometric information is compared with the biometric information of the contact stored in the online social media. Further matching, if it is determined that the further matching is successful, the contact information of the contact corresponding to the biometric information is called up for interaction; if the further matching fails, the terminal is requested to perform editing and storage processing.

4. The method according to claim 2 or 3, wherein the step of requesting the terminal to perform editing and storage processing includes:

Request the terminal to store the biometric information and the corresponding known contact information, and associate the biometric information with the known contact information; if the contact information corresponding to the biometric information is not currently known, only store the biometric information information as a reserve.

5. The method according to claim 1, wherein when the matching is successful and the contact information of the contact corresponding to the biometric information is called, the contact information is modified as needed.

6. The method according to claim 1, wherein the current multimedia material is multimedia material existing in the terminal, or multimedia material captured by the terminal in real time.

7. The method according to claim 1, 2, 3 or 5, wherein the biometric information for face information.

8. A multimedia interactive device based on biometric information, including:

The associated storage module is configured to associate and store the biometric information of the contact with the contact information of the contact in advance in the terminal;

The matching interaction module is used to extract biometric information from the current multimedia material, and match the extracted biometric information with previously stored biometric information. If the match is determined to be successful, the contact information corresponding to the biometric information is called up. Contact information to interact.

9. The device according to claim 8, wherein the matching interaction module is further configured to, if the matching fails, further match the biometric information to the network social media related to the terminal user or request the The associated storage module performs editing and storage processing.

10. A terminal, at least including the multimedia interaction device based on biometric information according to claim 8 or 9.