WO2019061281A1

WO2019061281A1 - Mobile communication terminal and human facial recognition based communication method thereof

Info

Publication number: WO2019061281A1
Application number: PCT/CN2017/104348
Authority: WO
Inventors: 吴锡昌
Original assignee: 深圳传音通讯有限公司
Priority date: 2017-09-29
Filing date: 2017-09-29
Publication date: 2019-04-04
Also published as: CN111418196A

Abstract

A mobile communication terminal and a human facial recognition based communication method thereof. The communication method comprises: when viewing a photo album interface, a mobile communication terminal detecting whether to trigger a predefined operation for a video and/or a photo in the photo album; and when the predefined operation is detected, popping up at least one piece of contact information about a target object, so as to perform instant communication through the contact information. By means of this method, a mobile communication terminal of the present application can efficiently merge a photo album function and a communication function together, can realize quick switching and usage and improve the interactive implementation of both functions, and can merge a photo album of the mobile communication terminal with multiple communication applications to a great extent. In the present application, upon a photo album interface, a communication interface can be entered for communication without quitting the photo album application; the operational flow is simple, convenience is brought for a user, and the usage experience of the user is improved.

Description

Mobile communication terminal and communication method based on face recognition

[0001] The present application relates to the field of communications technologies, and in particular, to a communication method based on face recognition, and to a mobile communication terminal.

Background technique

[0002] In the prior art, in addition to the complete telephone function, the mobile communication terminal usually has a large-capacity business card management function, an outgoing power management function, a telephone firewall function for preventing telephone harassment, and an internal business card of the enterprise group telephone business card. Management functions, as well as many functions of assisting the office, such as: scheduling, notes, calendar, calculator and other functions.

[0003] At the same time, mobile communication terminals have the ability to access the Internet via the Internet and have strong multimedia functions. Photo browsing, web browsing, audio and video playback, e-books, electronic photo frames, etc. can be performed on the mobile communication terminal. The functions of the same mobile communication terminal in auxiliary office, auxiliary marketing, entertainment, etc. have also been greatly enhanced.

[0004] With the rapid development of various functions of the mobile communication terminal, the contact avatar can be customized in the phone book application of the mobile communication terminal, but after the contact avatar is set, it is only displayed on the contact list interface for reference, and there is no Take advantage of other features of your contact avatar. In addition, the method for the mobile communication terminal to make a call is to find the corresponding contact from the address book, find the corresponding contact, enter the interface of the contact, and dial the dial button set by the contact interface, and the operation steps are cumbersome. .

[0005] However, with the increasing popularity of functions such as photographing and socializing of mobile communication terminals, more and more users not only use mobile communication terminals as a communication device, but also more help in communicating emotional life. The current mobile communication terminal address book only provides the function of viewing the contact number, address, company name and other attributes. The above functions are difficult for the user to associate with more stories, bits and past related to the contact. When the user is in the process of viewing the contact, and suddenly wants to view the image information of the story carried by the contact, it is necessary to exit the address book and browse through the gallery, which is very inconvenient for the user. In addition, when the user sees a contact in the browsing album, if he wants to contact the contact, he or she needs to quit the album, and then find the contact in the address book, and the communication can be performed after multiple operations. This is to a certain extent It affects the user's experience and brings great inconvenience to users.

technical problem

[0006] In the prior art, the two functions of the photo album and the communication of the mobile communication terminal are difficult to implement, and the interaction between the two forms an unfriendly experience for the user, and each of the mobile communication terminal systems The degree of integration of the application is not high. For example, when the user creates a new contact A, the photo C in the photo album is used for editing and saving, and the contact information of the contact QQ/WeChat is entered, and when the user browses to the photo C, and wants to communicate with the A, it is required. Exit the photo album application, enter the communication interface and enter the contact information for communication. The whole operation process is complicated, the operation path is cumbersome, the experience is not good, and the user is greatly inconvenient and affects the user experience.

Problem solution

Technical solution

[0007] The purpose of the present application is to provide a mobile communication terminal and a communication method based on the face recognition, which can help the mobile communication terminal to switch to another function in different interfaces, thereby facilitating communication by the user. Simple operation and improved user experience.

Based on the above technical problem, the present application provides a communication method based on face recognition, wherein the communication method includes:

[0009] After browsing the album interface, the mobile communication terminal detects whether a predefined operation of the video and/or photo in the album is triggered;

[0010] after detecting the predefined operation, performing face recognition on the video and/or photo to obtain a target object;

[0011] determining whether the target object is a contact pre-stored in a local address book;

[0012] After determining that the target object is a contact pre-stored in the local address book, at least one contact manner of the target object is popped up to perform instant communication by using the contact method.

[0013] The communication method further includes: the mobile communication terminal detects the edited contact, obtains the specified video and/or photo from the album according to the editing operation, and sets the avatar of the contact;

[0014] storing the contact and its corresponding at least one contact in a local address book, and triggering an addition of a tag and an entry in the specified video and/or photo to detect the specified After the video and/or photo performs the predefined operation, at least one contact manner corresponding to the contact is popped up. [0015] The communication method further includes: the mobile communication terminal detects the edited contact, and captures the specified video and/or photo according to the shooting operation and sets the avatar of the contact;

[0016] performing face recognition on the specified video and/or photo to obtain a target face;

[0017] performing face recognition and matching in the video and/or photo of the album according to the target face;

[0018] after the target face matches the video and/or photo face of the album successfully, storing the contact and the corresponding at least one contact manner in the local address book, and triggering in the designation The video and/or photo is tagged and entered to eject at least one contact corresponding to the contact upon detecting the predefined operation of the specified video and/or photo.

[0019] wherein the predefined operations include long press, rotation, multi-touch, and click on a face region.

[0020] wherein, the contact method comprises a mobile phone number, a fixed phone number, an email address, a short number, a QQ message, a QQ phone, a WeChat message, a WeChat phone, and a personal network homepage; The step of successfully matching the video and/or the photo face, and storing the contact and the corresponding at least one contact in the local address book, further includes:

[0021] After the target face and the photo and/or photo face of the album are successfully matched, a pop-up window presents the target face and the video and/or photo of the album to the user;

[0022] after the user confirms that the matching fails, performing face recognition and matching in the video and/or photo of the album according to the target face; after the user confirms that the matching is successful, the contact and the corresponding at least A contact is stored in the local address book.

[0023] Based on the above technical problem, the present application provides a mobile communication terminal, wherein the mobile communication terminal includes a processor, and the processor is configured to execute communication program data based on face recognition, and the implemented steps include:

[0024] In the browse album interface, detecting whether a predefined operation of the video and/or photo in the album is triggered; [0025] after detecting the predefined operation, performing a face on the video and/or photo Identify to get the target object;

Determining whether the target object is a contact pre-stored in a local address book;

[0027] After determining that the target object is a contact pre-stored in the local address book, at least one contact manner of the target object is popped up to perform instant communication by using the contact method.

[0028] wherein, the processor is configured to execute communication program data based on face recognition, and the implemented steps are further included Includes:

[0029] after the editing of the contact is detected, the specified video and/or photo is obtained from the album according to the editing operation and set as the avatar of the contact;

[0030] storing the contact and its corresponding at least one contact in a local address book, and triggering an addition of a tag and an entry in the specified video and/or photo to detect the specified After the video and/or photo performs the predefined operation, at least one contact manner corresponding to the contact is popped up.

[0031] wherein, the processor is configured to execute communication program data based on face recognition, and the implemented steps further include:

[0032] After detecting the edited contact, the specified video and/or photo is taken according to the shooting operation and set as the avatar of the contact;

[0033] performing face recognition on the specified video and/or photo to obtain a target face;

[0034] performing face recognition and matching in the video and/or photo of the album according to the target face;

[0035] after the target face and the photo and/or photo face of the album match successfully, storing the contact and the corresponding at least one contact information in the local address book, and triggering in the designation The video and/or photo is tagged and entered to eject at least one contact corresponding to the contact upon detecting the predefined operation of the specified video and/or photo.

[0036] wherein the predefined operations include long press, rotation, multi-touch, and click on a face region.

[0037] wherein the contact information includes a mobile phone number, a fixed phone number, an email address, a short number, and a Q

Q information, QQ phone, WeChat information, WeChat phone and personal network home page.

Advantageous effects of the invention

Beneficial effect

[0038] The mobile communication terminal of the present application and the communication method based on the face recognition, after browsing the album interface, the mobile communication terminal detects whether a predefined operation of the video and/or photo in the album is triggered, and when the Pre-defined operation 弹出, popping up at least one contact manner of the target object to perform instant communication through the contact method. In this way, the mobile communication terminal of the present application can fuse the two functions of the photo album and the communication well, can realize the fast switching use, improve the interaction realization between the two, and can implement the mobile communication terminal. Albums are highly integrated with multiple communication applications. In the photo album interface, the application no longer needs to exit the photo album application, and can enter the communication interface for communication, the operation flow Simple, user-friendly, and improved user experience.

Brief description of the drawing

DRAWINGS

1 is a flow chart of an embodiment of a communication method based on face recognition according to the present application.

2 is a block diagram of a block diagram of an embodiment of a mobile communication terminal according to the present application.

Embodiments of the invention

[0041] The embodiments of the present application are described below by way of specific embodiments, and those skilled in the art can readily appreciate other advantages and functions of the present application from the disclosure of the present disclosure.

[0042] In the following description, the drawings describe several embodiments of the present application. It is to be understood that other embodiments may be utilized, and structural, structural, electrical and operational changes may be made without departing from the spirit and scope of the disclosure. The following detailed description is not to be considered as limiting, and the scope of the embodiments of the present invention is defined by the appended claims. The terminology used herein is for the purpose of describing particular embodiments, and is not intended to

[0043] Although the terms first, second, etc. are used herein to describe various modules in some instances, these modules should not be limited by these terms. These terms are only used to distinguish one element from another.

In addition, as used herein, the singular forms "", "," And "comprising" indicates the presence of the described features, steps, operations, components, components, items, categories, and / or groups, but does not exclude one or more other features, steps, operations, components, components, items, categories, And/or the presence, presence, or addition of a group. The terms "or" and "and/or" are used herein to be interpreted as inclusive or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C" . An exception to this definition occurs only if the combination of components, functions, steps, or operations is inherently mutually exclusive in some way.

[0045] As used herein, the term "if" may be interpreted to mean "when..." or "once..." or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if a condition or event is stated" depends on the context, optionally interpreted as meaning "once If yes, then "or" in response to determining "or" once the [Stated Condition or Event] is detected or "in response to detecting the [Stated Condition or Event]".

Please refer to FIG. 1. FIG. 1 is a flowchart of an embodiment of a communication method based on face recognition according to the present application.

[0047] In the embodiment, the communication method based on face recognition includes but is not limited to the following steps.

[0048] Step S101: After browsing the album interface, the mobile communication terminal detects whether the video in the album is triggered.

/ or predefined actions for photos.

[0049] It should be noted that the predefined operations of the present embodiment may include long press, rotation, multi-touch, and click on a face area. For example, long press can be long press for more than 3 seconds, 5 seconds, etc., rotation can be used to rotate the video and / or photo at a certain angle (such as 30 °, 45 ° or 90 °), multi-touch can A touch action that occurs when two fingers and three fingers are equivalent to a video and/or photo. In addition, clicking on the face area allows the user to click on the face of the video and/or photo while browsing.

, which is defined as a predefined operation.

[0050] It should be noted that the video may be a short video or a long video, and if there are multiple faces, the user may click on the specified face for subsequent recognition, and the photo may be taken, or downloaded. Local picture of the mobile communication terminal.

[0051] Step S102, after detecting the predefined operation, performing face recognition on the video and/or photo to obtain a target object.

[0052] In the face recognition mode of step S102 in this embodiment, the video and/or the photo may be preprocessed, the corner points in the preprocessed picture are extracted, and the corner points are filtered and combined to obtain a connected area of the corner points. And then, extracting a centroid in the connected region of the corner point, matching the centroid with the face template, calculating a matching probability of the centroid and the face template, and positioning the region formed by the centroid whose matching probability is greater than or equal to a predetermined value For the candidate face region, feature extraction of the eye, mouth, and the like is performed on the candidate face region, and when at least one feature is extracted, it is determined that the face is recognized.

[0053] Step S103: Determine whether the target object is a contact pre-stored in the local address book.

[0054] It should be noted that the manner in which the local address book contacts are pre-stored in this embodiment may include at least the following two types.

[0055] In a first mode, the mobile communication terminal detects the edited contact, obtains the specified video and/or photo from the album according to the editing operation, and sets the avatar as the contact; and the at least the contact and the corresponding One The contact information is stored in the local address book, and triggers adding a mark and an entry in the specified video and/or photo to pop up after detecting the predefined operation on the specified video and/or photo. At least one contact manner corresponding to the contact.

[0056] In the second mode, the mobile communication terminal detects the edit contact 吋, takes a specified video and/or photo according to the shooting operation, and sets it as the avatar of the contact; performs a face on the specified video and/or photo Identifying to obtain a target face; performing face recognition and matching in the video and/or photo of the album according to the target face; and successfully matching the video and/or photo face of the target face with the album, Storing the contact and its corresponding at least one contact in a local address book and triggering an addition of a tag and entry in the specified video and/or photo to detect the specified video and/or Or the photo performs the predefined operation, and at least one contact manner corresponding to the contact is popped up.

[0057] In addition, the step of storing the contact and the corresponding at least one contact manner in the local address book after the matching of the target face and the video and/or photo face of the album is successful, The method may further include: after the target face is successfully matched with the video and/or photo face of the album, the pop-up window prompts the user and the video and/or photo of the album to the user; After the failure, the face recognition and matching are performed in the video and/or photo of the album according to the target face; after the user confirms that the matching is successful, the contact and the corresponding at least one contact manner are stored locally. In the address book.

[0058] For example, the mobile communication terminal edits the contact 吋, selects a video and/or photo of the face feature from the album or the captured video and/or photo, and then performs local processing, and the processed video and/or Or the photo is set to the contact avatar. After saving the avatar, the mobile communication terminal extracts the feature identification information in the avatar, traverses all the videos and/or photos of the album according to the feature identification information, and searches for all videos and/or photos that match the feature identification information; All matching videos and/or photos are aggregated into a collection of contacts, added to the video and/or photo properties of the contact, to complete the matching video and/or photo with the contact Interrelated.

[0059] Step S104: After determining that the target object is a contact pre-stored in the local address book, at least one contact mode of the target object is popped up to perform instant communication by using the contact mode.

[0060] In this embodiment, the contact information may include a mobile phone number, a fixed phone number, an email address, a short number, a QQ message, a QQ phone, a WeChat message, a WeChat phone, and a personal network homepage.

[0061] Please refer to FIG. 2, which is a block diagram of an embodiment of a mobile communication terminal according to the present application. [0062] The mobile communication terminal in this embodiment may include a processor 21 and a memory 22, and the memory 22 may be configured to store communication program data based on face recognition, data of an album, and other application data, and the processing The device 21 is configured to execute the face recognition based communication program data, and the implementing step may include: detecting, by browsing the album interface, whether to trigger a predefined operation on the video and/or the photo in the album; Defining an operation, performing face recognition on the video and/or photo to obtain a target object; determining whether the target object is a contact pre-stored in a local address book; determining that the target object is pre-stored in a local address book The contact port 弹出, popping up at least one contact manner of the target object to perform instant communication through the contact method.

[0063] The predefined operations of the present embodiment may include long press, rotation, multi-touch, and click on a face area. For example, long press can be long press for more than 3 seconds, 5 seconds, etc., rotation can be used to rotate the video and / or photo at a certain angle (such as 30 °, 45 ° or 90 °), multi-touch can A touch action that occurs when two fingers and three fingers are equivalent to a video and/or photo. In addition, clicking on the face area allows the user to click on the face of the video and/or photo to view the defined action.

[0064] The video of the embodiment may be a short video or a long video, and if there are multiple faces in the video, the user may click on the specified face to perform subsequent recognition, and the photo may be taken, or downloaded. A picture to the mobile communication terminal.

[0065] It should be noted that, the processor 21 is configured to execute the communication program data based on the face recognition, and the implemented step may further include: after detecting the edited contact, obtaining the specified from the album according to the editing operation. Video and/or photo and set as the avatar of the contact; storing the contact and its corresponding at least one contact in the local address book, and triggering an addition of the tag in the specified video and/or photo And an entry, to perform the predefined operation on the specified video and/or photo, popping up at least one contact manner corresponding to the contact.

[0066] In addition, the processor 21 is configured to execute the communication program data based on the face recognition, and the implemented step may further include: after detecting the edited contact, shooting the specified video and/or photo according to the shooting operation. Setting an avatar of the contact; performing face recognition on the specified video and/or photo to obtain a target face; performing face recognition on the video and/or photo of the album according to the target face and Matching; after the target face and the photo and/or photo face of the album match successfully, the contact and the pair thereof At least one contact should be stored in the local address book and triggering the addition of a tag and entry in the specified video and/or photo to detect the predefined video and/or photo of the specified video After operation, at least one contact method corresponding to the contact is popped up.

[0067] wherein the contact information may include a mobile phone number, a fixed phone number, an email address, a short number, a QQ message, a QQ phone, a WeChat message, a WeChat phone, and a personal network homepage.

The foregoing is a part of the implementation of the present application. In a preferred embodiment, the application may further simplify the operation process, and may specifically include the following: After browsing the album interface, the mobile communication terminal detects whether the video in the album is triggered. And/or a predefined operation of the photo; after detecting the predefined operation, popping up at least one contact manner of the target object to perform instant communication by the contact method.

[0069] The following description will be made in conjunction with specific application examples of the present application.

[0070] The mobile communication terminal receives a predefined operation generated by the user triggering a face of the video and/or the photo.

[0071] displaying at least one contact mode after receiving the predefined operation, such as a list, displaying a mobile phone number in the list, a fixed phone number, an email address, a short number, a QQ message, a QQ phone, a WeChat message, and a WeChat Telephone and personal network homepage for users to choose. For example, when the video and/or photo of the selected face is "Zhang San", "Zhang San, number 137****; zhangsan@xxx.com; QQ: 2798xxxx" is displayed.

[0072] The mobile communication terminal receives a selection instruction generated by selecting one of the contact methods, and performs communication by using the selected contact method, for example, making a call.

[0073] In addition, in other embodiments, a face of any one of the videos and/or photos in the album may be triggered to be marked and imported, and the contact represented by the face is bound to all contacts thereof, and then After the user browses the album, the user only needs to click on the face of the video and/or photo to directly communicate.

Industrial applicability

[0074] The mobile communication terminal of the present application and the communication method based on the face recognition, after browsing the album interface, the mobile communication terminal detects whether a predefined operation of the video and/or photo in the album is triggered, and when the Pre-defined operation 弹出, popping up at least one contact manner of the target object to perform instant communication through the contact method. In this way, the mobile communication terminal of the present application can fuse the two functions of the album and the communication well, and can realize the fast switching use and improve the interaction between the two. In a current manner, the album of the mobile communication terminal can be highly integrated with a plurality of communication applications. In the photo album interface, the application no longer needs to exit the photo album application, and can enter the communication interface for communication, and the operation flow is simple, which brings convenience to the user and improves the user experience.

In addition, the above-described embodiments are merely illustrative of the principles of the present application and its effects, and are not intended to limit the present application. Any of the above-described embodiments may be modified or altered without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and scope of the invention are still to be covered by the claims.

Claims

Claim

[Claim 1] A communication method based on face recognition, wherein the communication method includes:

After browsing the album interface, the mobile communication terminal detects whether a predefined operation of the video and/or photo in the album is triggered;

After detecting the predefined operation, performing face recognition on the video and/or photo to obtain a target object;

After determining that the target object is a contact pre-stored in the local address book, at least one contact manner of the target object is popped up to perform instant communication through the contact method.

[Claim 2] The communication method according to claim 1, wherein the communication method further includes:

After detecting that the contact is edited by the mobile communication terminal, the specified video and/or photo is obtained from the album according to the editing operation and set as the avatar of the contact;

Storing the contact and its corresponding at least one contact in a local address book and triggering an addition of a tag and entry in the specified video and/or photo to detect the specified video and/or Or the photo performs the predefined operation, and at least one contact manner corresponding to the contact is popped up.

[Claim 3] The communication method according to claim 1, wherein the communication method further includes:

After detecting that the contact is edited by the mobile communication terminal, the specified video and/or photo is taken according to the shooting operation and set as the avatar of the contact;

Performing face recognition on the specified video and/or photo to obtain a target face; performing face recognition and matching in the video and/or photo of the album according to the target face; After the video and/or photo face of the album is successfully matched, the contact and its corresponding at least one contact are stored in the local address book, and triggering the addition of a mark and an entry in the specified video and/or photo And, after detecting the predefined operation on the specified video and/or photo, popping up at least one contact manner corresponding to the contact.

[Claim 4] The communication method according to claim 1, wherein the predefined operations include long press, rotation, multi-touch, and click on a face region.

[Claim 5] The communication method according to claim 3, wherein the contact method includes a mobile phone number, a fixed telephone number, an email address, a short number, a QQ message, a QQ phone, a WeChat message, a WeChat phone, and a personal network. The step of storing the contact and the corresponding at least one contact information in the local address book after the matching of the target face and the video and/or the photo face of the album, the method further includes:

After the target face matches the video and/or photo face of the album successfully, a pop-up window prompts the user and the video and/or photo of the album to the user;

After the user confirms that the matching fails, the face recognition and matching are performed in the video and/or photo of the album according to the target face; after the user confirms that the matching is successful, the contact and the corresponding at least one of the contacts are contacted. The method is stored in the local address book.

[Claim 6] A mobile communication terminal, wherein the mobile communication terminal includes a processor, and the processor is configured to execute communication program data based on face recognition, and the implemented steps include: detecting an interface of the album, detecting Whether to trigger a predefined operation on the video and/or photo in the album;

[Claim 7] The mobile communication terminal according to claim 6, wherein the processor is configured to execute the communication program data based on the face recognition, and the implementing step further comprises: after detecting the edited contact, according to the editing The operation obtains the specified video and/or photo from the album and sets it as the avatar of the contact;

[Claim 8] The mobile communication terminal according to claim 6, wherein the processor is configured to perform based on Face recognition communication program data, the implementation steps include:

After detecting the edited contact, the specified video and/or photo is taken according to the shooting operation and set as the avatar of the contact;

[Claim 9] The mobile communication terminal according to claim 6, wherein the predefined operation includes long pressing

, rotate, multi-touch, and click on the face area.

[Claim 10] The mobile communication terminal according to claim 8, wherein the contact method comprises a mobile phone number, a fixed telephone number, an email address, a short number, a QQ message, a QQ phone, a WeChat message, a WeChat phone, and an individual. Network home page.