WO2019085625A1 - 表情图片推荐方法及设备 - Google Patents

表情图片推荐方法及设备 Download PDF

Info

Publication number
WO2019085625A1
WO2019085625A1 PCT/CN2018/103180 CN2018103180W WO2019085625A1 WO 2019085625 A1 WO2019085625 A1 WO 2019085625A1 CN 2018103180 W CN2018103180 W CN 2018103180W WO 2019085625 A1 WO2019085625 A1 WO 2019085625A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
image
user image
feature information
emoticon
Prior art date
Application number
PCT/CN2018/103180
Other languages
English (en)
French (fr)
Inventor
胡晨鹏
Original Assignee
上海掌门科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海掌门科技有限公司 filed Critical 上海掌门科技有限公司
Publication of WO2019085625A1 publication Critical patent/WO2019085625A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9535—Search customisation based on user profiles and personalisation

Definitions

  • the present application relates to the field of information technology, and in particular, to an expression picture recommendation method and device.
  • instant messaging software can provide various types of emoticons for users to download and use. Users can browse or text search for emoticons through the portal provided by instant messaging software, such as an emoticon mall, and download them.
  • An object of the present application is to provide an emoticon recommendation scheme.
  • some embodiments of the present application provide a method for recommending an emoticon of a service device, the method comprising: acquiring a user image uploaded by a user equipment; and acquiring feature information of the user image according to the user image. Matching emoticons; sending the emoticons to the user device.
  • Some embodiments of the present application further provide an emoticon picture recommendation method on a user equipment side, the method comprising: acquiring a user image, and transmitting the user image to a service device; receiving the service device in response to the user image feedback An emoticon picture; the emoticon is presented to the user.
  • Some embodiments of the present application also provide an apparatus including a memory for storing computer program instructions and a processor for executing computer program instructions that, when executed by the processor, trigger the The device performs the emoticon picture recommendation method of the foregoing user equipment or service device end.
  • Some embodiments of the present application also provide a computer readable medium having stored thereon computer program instructions executable by a processor to implement an emoticon picture recommendation method of the aforementioned user equipment or service device side.
  • the user equipment acquires a user image, and sends the user image to the service device, where the service device acquires an emoticon image that matches the feature information of the user image according to the user image.
  • the emoticon image is fed back to the user equipment and presented to the user for selection.
  • the user image the user can express the emoticon image content that is difficult to describe by the image with his own image, so that the user can more easily obtain the emoticon image that he or she wishes to obtain.
  • the user image contains some behaviors or expressions of the user, it can reflect the needs expressed by the user part. Therefore, when the user uses the image as a reference when recommending the expression image to the user, it can satisfy some personalized needs of the user, thereby improving The recommended flexibility and user experience are better.
  • FIG. 1 is a schematic diagram of a system for implementing an emoticon recommendation according to some embodiments of the present application
  • FIG. 2 is a flow chart of interaction between a user equipment and a service device when implementing an emoticon recommendation according to some embodiments of the present application
  • FIG. 3 is a schematic flowchart of a solution of some embodiments of the present application applied to instant messaging software
  • FIG. 4 is a schematic diagram of an apparatus for implementing an emoticon picture recommendation according to some embodiments of the present disclosure
  • the devices of the terminal and the service network each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
  • processors CPUs
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • the memory may include non-persistent memory, random access memory (RAM), and/or non-volatile memory in a computer readable medium, such as read only memory (ROM) or flash memory.
  • RAM random access memory
  • ROM read only memory
  • Memory is an example of a computer readable medium.
  • Computer readable media includes both permanent and non-persistent, removable and non-removable media, and information storage can be implemented by any method or technology.
  • the information can be computer readable instructions, data structures, modules of programs, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory. (ROM), EEPROM, flash memory or other memory technology, compact disc (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette A tape, tape storage or other magnetic storage device or any other non-transportable medium can be used to store information that can be accessed by a computing device.
  • PRAM phase change memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • RAM random access memory
  • ROM read only memory
  • EEPROM electrically erasable programmable read only memory
  • CD-ROM compact disc
  • DVD digital versatile disc
  • FIG. 1 is a schematic diagram of a system for implementing an emoticon recommendation according to some embodiments of the present application.
  • the system includes a user equipment 110 and a service device 120.
  • the user equipment 110 and the service device 120 The interaction process between them is shown in Figure 2, including the following processing steps:
  • the user equipment acquires a user image.
  • the user equipment may be various types of electronic devices such as a mobile phone, a tablet computer, a computer, and a wearable device. Such devices may capture and acquire user images through a built-in or external camera device.
  • the user image obtained by the user device may be an image of the user's facial expression or an image of the user's physical motion, and may be set according to different implementation scenarios.
  • the emoticon software recommendation function can be opened, and the front camera of the mobile phone is activated to capture the user image. Since the user generally takes a mobile terminal such as a mobile phone in the hand and is not convenient to take a physical motion, in this scenario, the user image can preferentially select an image of the user's facial expression.
  • the computer is connected to an external camera that can take a full-body image, and then the image of the user's body motion can be conveniently obtained at this time, then in this scenario, the user The image can also preferentially select an image of the user's limb movement.
  • the user equipment Since the user image acquired by the user equipment at this time is used for sending to the service device to match the emoticon picture, there is no need to present to the user in the user equipment, that is, the user equipment can prompt the user that the camera device has been activated only by marking or text, instead of The screen captured by the camera at this time is not displayed in the interface.
  • the user equipment may directly use the image captured by the camera as a user image; in some embodiments, the user device may compress the image captured by the camera as a user image; in some embodiments, the user The user image acquired by the device includes a plurality of key frames including the user image extracted by the user equipment from the image captured by the camera device, for example, several key frames in the whole process of the user making various expressions or several of the user's entire actions. Keyframes, etc.
  • the user equipment can adopt the following processing manners: first, acquiring a continuous image including the user image, for example, capturing a video about the complete process of the user making an expression by the camera; and then extracting multiple from the continuous image. a plurality of key frames containing user images, which may be several key pictures in a process of the user making a certain expression; finally, multiple key frames containing the user image are sent to the service device, so that several The user image contained in the key frame is matched with the emoticon image. These extracted key frames can help the service device to improve the accuracy of matching based on the user image.
  • Step S202 The user equipment sends the user image to the service device, so that the service device can match the user image to the appropriate emoticon image, thereby implementing the emoticon image recommendation.
  • Step S203 The service device acquires a user image uploaded by the user equipment.
  • Step S204 The service device acquires an emoticon image that matches the feature information of the user image according to the user image.
  • the service device may include, but is not limited to, an implementation such as a network host, a single network server, a plurality of network server sets, or a cloud computing-based computer set.
  • the cloud is composed of a large number of host or network servers based on Cloud Computing, which is a kind of distributed computing, a super virtual computer composed of a group of loosely coupled computers.
  • the user image received by the service device can include a plurality of key frames including the user image; in some embodiments, the user image received by the service device does not include a plurality of key frames including the user image, here In this case, the service device can perform an operation of extracting key frames from the received user image.
  • the matching rule that meets the requirements of the application scenario is pre-configured, so that after the user image is acquired, the appropriate emoticon image can be accurately found for the user to use.
  • the matching of the emoticons may be performed in the following manner:
  • the feature information of the user image is determined, and the feature information may be any specific information that can represent the user image, such as color features, texture features, shape features, and the like, and the image feature values can be The corresponding dimension describes what is contained in the user's image. Therefore, the matching of the emoticon image can make the matching result have a specific relationship with the content of the user image, thereby better satisfying the user's personalized requirement for the emoticon image and improving the user experience.
  • the user image may be identified by a depth learning engine to determine feature information of the user image.
  • the depth learning engine takes the image data of the user image as an input value and the feature information of the user image as an output value, and can set various types of determination conditions in advance as a decision basis of each hidden layer in the deep learning, thereby accurately The feature information of the user image is recognized.
  • the emoticon image matching the feature information of the user image is determined according to a preset matching rule.
  • the preset matching rule may be based on the feature information of the user image and the feature information of the candidate picture to obtain a matching result, and determine an expression picture that matches the feature information of the user image based on the matching result.
  • the matching process may be: if the similarity between the feature information of the user image and the feature information of the candidate picture exceeds a threshold, the candidate picture is determined as an expression picture that matches the feature information of the user image.
  • the threshold may be set according to actual requirements. When the threshold is set low, more expression images may be matched and recommended to the user, and the user may be provided with more choices, and the higher the threshold is set, the matching result is obtained. It will decrease, but the correlation between the matching emoticons and the user images will also be higher.
  • the matching process may be: sorting based on the similarity between the feature information of the user image and the feature information of the candidate image, and selecting the candidate image of the N bits before the sorting as the emoticon image matching the feature information of the user image, N It is a preset natural number.
  • the service device may maintain an expression database specifically for storing candidate pictures.
  • the candidate pictures in the database may be obtained by the service device from the Internet, for example, using a web crawler to periodically obtain various types of expression pictures. After the feature information is identified, it is stored together with the feature information in the expression database of the service device. Therefore, in some embodiments of the present application, when the service device determines the emoticon image that matches the feature information of the user image, the feature information of the obtained user image may be used as a retrieval condition, and the similarity is retrieved in the expression database. Emoticon picture.
  • the preset matching rule may be that the tag information is first obtained based on the user image feature information, and then matched with the tag information of the candidate image to obtain a matching result.
  • the feature information of the user image may be a brief description of the content of the image.
  • the feature information of the user image may determine that the content in the user image is an expression about the user laughing, and the tag information may be “laughing” or “happy”. "Wait.
  • the tag information of the candidate picture may be in the same manner; after the feature information is identified by the service device, the tag information is determined according to the feature information, or may be set by the related user of the emoticon, such as an expression.
  • the creator of the image can insert tag information into the emoticon file when making the emoticon.
  • the service device may determine: the tag information of the user image according to the feature information of the user image; The tag information of the image and the tag information of the candidate picture are in a preset relationship, and the candidate picture is determined as an emoticon image that matches the feature information of the user image.
  • the service device maintains the feature information in the expression database without saving the feature image, but directly saves the tag information of the emoticon image to facilitate the matching.
  • Step S205 The service device sends the matched emoticon image to the user equipment.
  • Step S206 The user equipment receives an emoticon picture that is sent by the service device in response to the user image feedback.
  • Step S207 the user equipment presents the emoticon image to the user.
  • the user requests the service device to recommend an emoticon picture when using the instant messaging software for chatting.
  • the user device may present the emoticon image to the user when the user presents the emoticon image.
  • the pop-up bubble is displayed. If there are multiple emoticons recommended by the service device, the user can view other emoticons by sliding the pictures in the bubble.
  • the service device may first send a thumbnail of the emoticon image, so that the user device may first present the thumbnail to the user, and when the user selects a thumbnail of one of the emoticons, the user device may obtain the selection information (eg, the user Clicking on the information corresponding to a thumbnail), and then requesting the complete data of the emoticon image from the service device according to the selection information. Therefore, when there are many recommended emoticons, the amount of data exchanged between the user equipment and the service device can be reduced, and reception delay, failure, and the like can be avoided, and the user experience is improved.
  • the selection information eg, the user Clicking on the information corresponding to a thumbnail
  • FIG. 3 is a schematic flowchart of a solution of some embodiments of the present application applied to instant messaging software, wherein the instant messaging software uses C/S (Client/Server, Client/Server) when providing services to users.
  • Software architecture The client includes software and runs on the user equipment, such as various types of terminal devices used by the user, such as mobile phones, tablets, computers, etc.; the server includes software and runs on the service device, for example, can run on the application server, the cloud Etc., to support the various functions of the client.
  • the expression image recommendation function needs to be supported by the server.
  • the specific interaction process is as follows:
  • Step S301 the user A starts the client of the instant messaging software, enters the chat interface, and the chat interface may include an option for the expression recommendation.
  • Step S302 after the user selects the expression recommendation option, the client starts the camera to acquire a continuous image including the user image.
  • the client starts the camera to acquire a continuous image including the user image.
  • Step S303 the client extracts a plurality of key frames including the user image from the continuous image, and uploads to the server.
  • Step S304 the depth learning engine of the server performs recognition processing on the user image, and extracts feature values of the image.
  • Step S305 the server searches in the expression database according to the extracted image feature values, and selects an expression image similar to the facial expression or the whole body motion of the user A, and uses the search result as a search result to recommend to the client.
  • step S306 the server returns the search result to the client.
  • Step S307 the client decodes the search result returned by the server.
  • Step S308 the decoded result is displayed near the input box of the chat interface of the client, and the user is prompted to use the expression image during the chat.
  • the user equipment acquires a user image, and sends the user image to the service device, where the service device acquires the feature information of the user image according to the user image.
  • the emoticon image is fed back to the user device and presented to the user for selection.
  • the search is mainly based on text information actively input by the user.
  • the above embodiment of the present application breaks this inertial thinking, and the user is recommended to express the expression by capturing the user image in real time. Therefore, the user can express the expression image of the character that is difficult to describe by the image related to the user, so that the user is very convenient. Get the emoticon image you want to get.
  • the user image contains some behaviors or expressions of the user, it can reflect the needs expressed by the user part. Therefore, when the user uses the image as a reference when recommending the expression image to the user, it can satisfy some personalized needs of the user, thereby improving The recommended flexibility and user experience are better.
  • a portion of the present application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide a method and/or technical solution in accordance with the present application.
  • the program instructions for invoking the method of the present application may be stored in a fixed or removable recording medium, and/or transmitted by a data stream in a broadcast or other signal bearing medium, and/or stored in a program according to the program.
  • the instruction runs in the working memory of the computer device.
  • some embodiments in accordance with the present application include an apparatus as shown in FIG.
  • the apparatus comprising one or more memories 410 storing computer readable instructions and a processor 420 for executing computer readable instructions, wherein When the computer readable instructions are executed by the processor, the apparatus is caused to perform methods and/or technical solutions based on the various embodiments of the foregoing application.
  • some embodiments of the present application also provide a computer readable medium having stored thereon computer program instructions executable by a processor to implement the methods of the foregoing various embodiments of the present application and / or technical solutions.
  • the present application can be implemented in software and/or a combination of software and hardware, for example, using an application specific integrated circuit (ASIC), a general purpose computer, or any other similar hardware device.
  • the software program of the present application can be executed by a processor to implement the above steps or functions.
  • the software programs (including related data structures) of the present application can be stored in a computer readable recording medium such as a RAM memory, a magnetic or optical drive or a floppy disk and the like.
  • some of the steps or functions of the present application may be implemented in hardware, for example, as a circuit that cooperates with a processor to perform various steps or functions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Information Transfer Between Computers (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请提供了一种表情图片推荐方案，该方案中用户设备获取用户图像，并将所述用户图像发送至服务设备，由服务设备根据所述用户图像，获取与所述用户图像的特征信息匹配的表情图片，在匹配到符合匹配规则的表情图片之后，将表情图片反馈给用户设备，呈现给用户来进行选择，由于用户图像中会包含用户的一些行为或表情，因此能够反映出用户部分表达的需求，因此在向用户推荐表情图片时以用户图像作为参考，能够满足用户的一些个性化需求，从而提高了推荐的灵活性，用户体验较好。

Description

表情图片推荐方法及设备 技术领域
本申请涉及信息技术领域，尤其涉及一种表情图片推荐方法及设备。
背景技术
随着互联网的发展，即时通信已成为人们日常生活中不可缺少的网络沟通方式。随着人们对即时通信软件的使用越来越频繁，即时通信工具推出了越来越多满足不同用户需求的功能。目前，在聊天过程中，当用户想表达自身当前的感受或者心情时，除了通过文字直接描述之外，也会通过诸如特殊符号、表情图片等来协助表达。因此，在使用即时通讯软件时，用户会希望获得各类丰富多彩的表情图片。
为了丰富用户可使用的表情，即时通信软件可以提供各类表情图片供用户下载使用。用户可通过即时通信软件提供的入口，如表情商城等，来浏览或文本搜索表情图片，并进行下载。
申请内容
本申请的一个目的是提供一种表情图片推荐方案。
为实现上述目的，本申请的一些实施例提供了一种服务设备端的表情图片推荐方法，该方法包括：获取用户设备上传的用户图像；根据所述用户图像，获取与所述用户图像的特征信息匹配的表情图片；将所述表情图片发送至所述用户设备。
本申请的一些实施例还提供了一种用户设备端的表情图片推荐方法，该方法包括：获取用户图像，并将所述用户图像发送至服务设备；接收所述服务设备响应于所述用户图像反馈的表情图片；向用户呈现所述表情图片。
本申请的一些实施例还提供了一种设备，该设备包括用于存储计算机程序指令的存储器和用于执行计算机程序指令的处理器，当该计算机程序指令被该处理器执行时，触发所述设备执行前述用户设备或者服务设备端的表情图片推荐方法。
本申请的一些实施例还提供了一种计算机可读介质，其上存储有计算机程序指令，所述计算机可读指令可被处理器执行以实现前述用户设备或者服务设备端的表情图片推荐方法。
本申请的一些实施例提供的方案中，用户设备获取用户图像，并将所述用户图像发送至服务设备，由服务设备根据所述用户图像，获取与所述用户图像的特征信息匹配的表情图片，在匹配到符合匹配规则的表情图片之后，将表情图片反馈给用户设备，呈现给用户来进行选择。通过用户图像，用户可以以与其自身有关的图像来表达文字难以描述的表情图片需求，以使用户更为方便地获得其希望获得的表情图片。此外，由于用户图像中会包含用户的一些行为或表情，因此能够反映出用户部分表达的需求，因此在向用户推荐表情图片时以用户图像作为参考，能够满足用户的一些个性化需求，从而提高了推荐的灵活性，用户体验较好。
附图说明
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本申请的其它特征、目的和优点将会变得更明显：
图1为本申请的一些实施例提供的一种实现表情图片推荐的系统的示意图；
图2为本申请一些实施例在实现表情图片推荐时用户设备和服务设备之间的交互流程图；
图3为本申请的一些实施例的方案应用于即时通信软件时的流程示意图；
图4为本申请的一些实施例提供的一种实现表情图片推荐的设备的示意图；
附图中相同或相似的附图标记代表相同或相似的部件。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
在本申请一个典型的配置中，终端、服务网络的设备均包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体，可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。
图1示出了本申请的一些实施例提供的一种实现表情图片推荐的系统的示意图，该系统包括了用户设备110和服务设备120，在实现表情图片推荐时，用户设备110和服务设备120之间的交互流程如图2所示，包括以下处理步骤：
步骤S201，用户设备获取用户图像。在实际场景中，所述用户设备可以是手机、平板电脑、计算机、可穿戴设备等各类电子设备，此类设备可以通过内置或者外接的摄像装置来拍摄并获取用户图像。用户设备 获取的用户图像可以是用户脸部表情的图像，也可以是用户肢体动作的图像，可以根据不同的实现场景来设定。
例如，用户在使用手机中安装的即时通信软件与其它用户聊天时，若需要获取表情图片，可以打开即时通信软件的表情图片推荐功能，此时手机的前置摄像头启动，拍摄用户图像。由于用户一般会将手机等移动终端拿在手中，不便于拍摄肢体动作，则在该场景中，用户图像可以优先选择用户脸部表情的图像。若用户在使用计算机上安装的即时通信软件与其它用户聊天时，该计算机连接有一个可以拍摄全身图像的外接摄像头，则此时可以方便的获取用户肢体动作的图像，那么在该场景中，用户图像也可以优先选择用户肢体动作的图像。
由于用户设备在此时获取的用户图像是用于发送给服务设备来匹配表情图片，因此无需在用户设备中呈现给用户，即用户设备可以仅仅通过标记或者文字提示用户摄像装置已经启动，而不在界面中不显示此时摄像装置所拍摄到的画面。
在一些实施例中，用户设备可直接将摄像装置拍摄的图像作为用户图像；在一些实施例中，用户设备可将摄像装置拍摄的图像进行压缩处理后作为用户图像；在一些实施例中，用户设备获取的用户图像包括用户设备从摄像装置拍摄的图像中提取出的包含用户图像的多个关键帧，例如用户做出各种表情整个过程中的几个关键帧或者用户整个动作中的几个关键帧等。
由此，用户设备可以采用如下处理方式：首先，获取包含用户图像的连续图像，例如通过摄像头拍摄一段关于用户做出某个表情的完整过程的视频；然后，从所述连续图像中提取多个包含用户图像的多个关键帧，这几个关键帧可以是用户做出某个表情过程中的几个关键画面；最终，将包含用户图像的多个关键帧发送至服务设备，使得通过几个关键帧中包含的用户图像进行表情图片的匹配。该等提取出的关键帧可有助于服务设备提高基于用户图像进行匹配的准确度。
步骤S202，用户设备将所述用户图像发送至服务设备，使得服务设备可以根据用户图像匹配到合适的表情图片，进而实现表情图片推荐。
步骤S203，服务设备获取用户设备上传的用户图像。
步骤S204，服务设备根据所述用户图像，获取与所述用户图像的特征信息匹配的表情图片。其中，服务设备可以包括但不限于如网络主机、单个网络服务器、多个网络服务器集或基于云计算的计算机集合等实现。在此，云由基于云计算(Cloud Computing)的大量主机或网络服务器构成，其中，云计算是分布式计算的一种，由一群松散耦合的计算机集组成的一个超级虚拟计算机。在一些实施例中，服务设备接收到的用户图像可包括包含用户图像的多个关键帧；在一些实施例中，服务设备接收到的用户图像未包括包含用户图像的多个关键帧，在此情况下，服务设备可进行从接收到的用户图像中提取关键帧的操作。
服务设备在基于用户图像进行匹配时，预先配置符合应用场景需求的匹配规则，使得在获取到用户图像之后，可以准确地查找到合适的表情图片供用户使用。在本申请的一些实施例中，可以采用如下方式进行表情图片的匹配：
首先，确定用户图像的特征信息，所述特征信息可以是任意能够表示用户图像的相关特定的信息，例如图像的色彩特征、纹理特征、形状特征等各类图像特征值，通过图像特征值能够在相应的维度描述用户图像中所包含的内容。因此，以此作为依据进行表情图片的匹配能够使得匹配结果与用户图像的内容存在特定的关联关系，从而更好的满足用户对于表情图片的个性化需求，提高用户体验。
为了获取提高特征信息时的准确度，可以通过深度学习引擎对所述用户图像进行识别，来确定所述用户图像的特征信息。所述深度学习引擎以用户图像的图像数据为输入值，以用户图像的特征信息为输出值，可以预先设置各类判定条件，作为深度学习中各个隐层(hidden layers)的决策依据，从而精准地识别出用户图像的特征信息。
在确定用户图像的特征信息之后，再根据预设的匹配规则，确定与所述用户图像的特征信息匹配的表情图片。
在本申请的一些实施例中，预设的匹配规则可以是基于用户图像的特征信息与候选图片的特征信息，来获取匹配结果，并基于匹配结果确 定与用户图像的特征信息匹配的表情图片。例如，匹配处理的过程可以是：若所述用户图像的特征信息与候选图片的特征信息的相似度超过阈值，将所述候选图片确定为与所述用户图像的特征信息匹配的表情图片。所述阈值可以根据实际需求设定，当阈值设定的低时，可以匹配得到更多的表情图片推荐给用户，给用户提供给更多的选择，而阈值设定的越高，则匹配结果会减少，但匹配出的表情图片与用户图像之间的相关度也将越高。又例如，匹配处理的过程可以是：基于用户图像的特征信息与候选图片的特征信息的相似度进行排序，并选取排序前N位的候选图片作为与用户图像的特征信息匹配的表情图片，N为预先设定的自然数。
在实际场景中，服务设备可以维护一个表情数据库，专门用于存储候选图片，该数据库中的候选图片可以由服务设备从互联网上获取，例如利用爬虫程序(web crawler)定期获取各类表情图片，并对其进行特征信息识别后，将连同特征信息一起保存于服务设备的表情数据库中。由此，在本申请的一些实施例中，服务设备在确定与所述用户图像的特征信息匹配的表情图片时，可以使用识别得到的用户图像的特征信息作为检索条件，在表情数据库中检索相似的表情图片。
在本申请的另一些实施例中，预设的匹配规则也可以是先基于用户图像特征信息得到其标签信息之后，再与候选图片的标签信息进行匹配，来获取匹配结果。用户图像的特征信息可以是对于图片内容的简要描述，例如通过用户图像的特征信息可以确定该用户图像中的内容是关于用户大笑的表情，则所述标签信息可以是“笑”、“开心”等。而候选图片的标签信息可以采用同样的表现方式；由服务设备对候选图片进行特征信息识别之后，根据其特征信息来确定其标签信息，或者也可以由该表情图片的相关用户设定，例如表情图片的制作者可以在制作该表情图片时，在表情图片文件中插入标签信息。
由此，服务设备在根据预设的匹配规则，确定与所述用户图像的特征信息匹配的表情图片时，可以是：根据用户图像的特征信息确定所述用户图像的标签信息；若所述用户图像的标签信息与候选图片的标签信 息符合预设关系，将所述候选图片确定为与所述用户图像的特征信息匹配的表情图片。相应地，在此场景中，服务设备维护表情数据库中可以不保存表情图片的特征信息，而是直接保存表情图片的标签信息以便于完成匹配。
步骤S205，服务设备将匹配得到的表情图片发送至所述用户设备。
步骤S206，用户设备接收所述服务设备响应于所述用户图像反馈的表情图片。
步骤S207，用户设备向用户呈现所述表情图片。在本申请一些实施例的典型应用场景中，用户会在使用即时通信软件进行聊天时请求服务设备推荐表情图片，为了便于用户选择，用户设备在向用户呈现表情图片时，可以在聊天界面中以弹出气泡的形式显示。若服务设备推荐的表情图片的数量有多个，用户可以通过滑动弹出气泡中的图片来查看其它的表情图片。
在实际场景中，当服务设备匹配到的数量多的表情图片时，若同时发送给用户设备，在网络环境不佳的情况下，容易造成用户设备端接收延迟或者失败等情况发生，降低用户体验。因此，服务设备可以先发送表情图片的缩略图，以便于用户设备可以先将缩略图呈现给用户，当用户选定其中某个表情图片的缩略图时，用户设备可以获取到选择信息(例如用户点击某个缩略图所对应的信息)，然后根据该选择信息去向服务设备请求这个表情图片的完整数据。由此，在推荐的表情图片较多时，可以减少用户设备和服务设备之间交互的数据量，避免发生接收延迟、失败等情况，提高了用户体验。
图3示出了本申请的一些实施例的方案应用于即时通信软件时的流程示意图，其中，该即时通信软件向用户提供服务时采用C/S(Client/Server，客户端/服务端)的软件构架。客户端包括软件，并运行于用户设备，如运行于用户使用的各类终端设备，例如手机、平板电脑、计算机等；服务端包括软件，并运行于服务设备，例如可以运行于应用服务器、云等，以为客户端的各类功能提供支持。例如本实施例中表情图片推荐功能需要由服务端提供支持，在实现该功能时，具体交互 流程如下：
步骤S301，用户A启动即时通信软件的客户端，进入聊天界面，聊天界面中可以包含表情推荐的选项
步骤S302，当用户选择该表情推荐选项后，客户端启动摄像头获取包含用户图像的连续图像。在一些实施例中，可以根据不同的应用场景选择拍摄用户全身的肢体动作或者是脸部表情。
步骤S303，客户端从连续图像中提取包含用户图像的多个关键帧，并上传到服务端。
步骤S304，服务端的深度学习引擎对用户图像进行识别处理，提取图像的特征值。
步骤S305，服务端根据提取的图像特征值在表情数据库中进行搜索，选择与用户A的脸部表情或者全身动作相似的表情图片，作为搜索结果，向客户端推荐。
步骤S306，服务端将搜索结果返回客户端。
步骤S307，客户端将服务端返回的搜索结果进行解码。
步骤S308，客户端的聊天界面的输入框附近显示解码后的结果，提示用户可以在聊天时使用这些表情图片。
综上，本申请的一些实施例提供的方案中，用户设备获取用户图像，并将所述用户图像发送至服务设备，由服务设备根据所述用户图像，获取与所述用户图像的特征信息匹配的表情图片，在匹配到符合匹配规则的表情图片之后，将表情图片反馈给用户设备，呈现给用户来进行选择。目前的现有技术中，搜索主要是基于用户主动输入的文本信息来进行的。而本申请的上述实施例打破了这种惯性思维，通过实时摄取用户图像来为用户推荐表情，因而，用户可以以与其自身有关的图像来表达文字难以描述的表情图片需求，使用户十分方便地获得其希望获得的表情图片。此外，由于用户图像中会包含用户的一些行为或表情，因此能够反映出用户部分表达的需求，因此在向用户推荐表情图片时以用户图像作为参考，能够满足用户的一些个性化需求，从而提高了推荐的灵活性，用户体验较好。
另外，本申请的一部分可被应用为计算机程序产品，例如计算机程序指令，当其被计算机执行时，通过该计算机的操作，可以调用或提供根据本申请的方法和/或技术方案。而调用本申请的方法的程序指令，可能被存储在固定的或可移动的记录介质中，和/或通过广播或其他信号承载媒体中的数据流而被传输，和/或被存储在根据程序指令运行的计算机设备的工作存储器中。在此，根据本申请的一些实施例包括一个如图4所示的设备，该设备包括存储有计算机可读指令的一个或多个存储器410和用于执行计算机可读指令的处理器420，其中，当该计算机可读指令被该处理器执行时，使得所述设备执行基于前述本申请的多个实施例的方法和/或技术方案。
此外，本申请的一些实施例还提供了一种计算机可读介质，其上存储有计算机程序指令，所述计算机可读指令可被处理器执行以实现前述本申请的多个实施例的方法和/或技术方案。
需要注意的是，本申请可在软件和/或软件与硬件的组合体中被实施，例如，可采用专用集成电路(ASIC)、通用目的计算机或任何其他类似硬件设备来实现。在一些实施例中，本申请的软件程序可以通过处理器执行以实现上文步骤或功能。同样地，本申请的软件程序(包括相关的数据结构)可以被存储到计算机可读记录介质中，例如，RAM存储器，磁或光驱动器或软磁盘及类似设备。另外，本申请的一些步骤或功能可采用硬件来实现，例如，作为与处理器配合从而执行各个步骤或功能的电路。
对于本领域技术人员而言，显然本申请不限于上述示范性实施例的细节，而且在不背离本申请的精神或基本特征的情况下，能够以其他的具体形式实现本申请。因此，无论从哪一点来看，均应将实施例看作是示范性的，而且是非限制性的，本申请的范围由所附权利要求而不是上述说明限定，因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本申请内。不应将权利要求中的任何附图标记视为限制所涉及的权利要求。此外，显然“包括”一词不排除其他单元或步骤，单数不排除复数。装置权利要求中陈述的多个单元或装置也可以由一个单元 或装置通过软件或者硬件来实现。第一，第二等词语用来表示名称，而并不表示任何特定的顺序。

Claims (11)

  1. 一种服务设备端的表情图片推荐方法，其中，该方法包括：
    获取用户设备上传的用户图像；
    根据所述用户图像，获取与所述用户图像的特征信息匹配的表情图片；
    将所述表情图片发送至所述用户设备。
  2. 根据权利要求1所述的方法，其中，获取用户设备上传的用户图像，包括：
    获取用户设备上传的包含用户图像的多个关键帧，其中，所述关键帧来自于包含用户图像的连续图像。
  3. 根据权利要求1所述的方法，其中，根据所述用户图像，获取与所述用户图像的特征信息匹配的表情图片，包括：
    确定所述用户图像的特征信息；
    根据预设的匹配规则，确定与所述用户图像的特征信息匹配的表情图片。
  4. 根据权利要求3所述的方法，其中，确定所述用户图像的特征信息，包括：
    通过深度学习引擎对所述用户图像进行识别，确定所述用户图像的特征信息。
  5. 根据权利要求3所述的方法，其中，根据预设的匹配规则，确定与所述用户图像的特征信息匹配的表情图片，包括：
    若所述用户图像的特征信息与候选图片的特征信息的相似度超过阈值，将所述候选图片确定为与所述用户图像的特征信息匹配的表情图片。
  6. 根据权利要求3所述的方法，其中，根据预设的匹配规则，确定与所述用户图像的特征信息匹配的表情图片，包括：
    根据用户图像的特征信息确定所述用户图像的标签信息；
    若所述用户图像的标签信息与候选图片的标签信息符合预设关系， 将所述候选图片确定为与所述用户图像的特征信息匹配的表情图片。
  7. 一种用户设备端的表情图片推荐方法，其中，该方法包括：
    获取用户图像，并将所述用户图像发送至服务设备；
    接收所述服务设备响应于所述用户图像反馈的表情图片；
    向用户呈现所述表情图片。
  8. 根据权利要求7所述的方法，其中，获取用户图像，并将所述用户图像发送至服务设备，包括：
    获取包含用户图像的连续图像；
    从所述连续图像中提取多个包含用户图像的多个关键帧；
    将包含用户图像的多个关键帧发送至服务设备。
  9. 根据权利要求1至8中任一项所述的方法，其中，所述用户图像包括用户脸部表情的图像和/或用户肢体动作的图像。
  10. 一种设备，该设备包括用于存储计算机程序指令的存储器和用于执行计算机程序指令的处理器，其中，当该计算机程序指令被该处理器执行时，触发所述设备执行权利要求1至9中任一项所述的方法。
  11. 一种计算机可读介质，其上存储有计算机程序指令，所述计算机可读指令可被处理器执行以实现如权利要求1至9中任一项所述的方法。
PCT/CN2018/103180 2017-10-31 2018-08-30 表情图片推荐方法及设备 WO2019085625A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711051614.2 2017-10-31
CN201711051614.2A CN107729543A (zh) 2017-10-31 2017-10-31 表情图片推荐方法及设备

Publications (1)

Publication Number Publication Date
WO2019085625A1 true WO2019085625A1 (zh) 2019-05-09

Family

ID=61203583

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/103180 WO2019085625A1 (zh) 2017-10-31 2018-08-30 表情图片推荐方法及设备

Country Status (2)

Country Link
CN (1) CN107729543A (zh)
WO (1) WO2019085625A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107729543A (zh) * 2017-10-31 2018-02-23 上海掌门科技有限公司 表情图片推荐方法及设备
CN109710790B (zh) * 2018-11-19 2020-12-11 北京达佳互联信息技术有限公司 表情搜索方法和装置、终端设备及存储介质
CN112532507B (zh) * 2019-09-17 2023-05-05 上海掌门科技有限公司 用于呈现表情图像、用于发送表情图像的方法和设备
CN112035692B (zh) * 2020-08-31 2023-11-03 百度在线网络技术（北京）有限公司 图片信息搜索方法和装置、计算机系统和可读存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104202718A (zh) * 2014-08-05 2014-12-10 百度在线网络技术（北京）有限公司 一种向用户提供信息的方法与装置
CN104616330A (zh) * 2015-02-10 2015-05-13 广州视源电子科技股份有限公司 一种图片的生成方法和装置
CN104753766A (zh) * 2015-03-02 2015-07-01 小米科技有限责任公司 表情发送方法及装置
JP2016103234A (ja) * 2014-11-28 2016-06-02 日本電信電話株式会社 映像特徴抽出装置、方法、及びプログラム
CN106503630A (zh) * 2016-10-08 2017-03-15 广东小天才科技有限公司 一种表情发送方法、设备及系统
CN106599926A (zh) * 2016-12-20 2017-04-26 上海寒武纪信息科技有限公司 一种表情图片推送方法及系统
CN107729543A (zh) * 2017-10-31 2018-02-23 上海掌门科技有限公司 表情图片推荐方法及设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104202718A (zh) * 2014-08-05 2014-12-10 百度在线网络技术（北京）有限公司 一种向用户提供信息的方法与装置
JP2016103234A (ja) * 2014-11-28 2016-06-02 日本電信電話株式会社 映像特徴抽出装置、方法、及びプログラム
CN104616330A (zh) * 2015-02-10 2015-05-13 广州视源电子科技股份有限公司 一种图片的生成方法和装置
CN104753766A (zh) * 2015-03-02 2015-07-01 小米科技有限责任公司 表情发送方法及装置
CN106503630A (zh) * 2016-10-08 2017-03-15 广东小天才科技有限公司 一种表情发送方法、设备及系统
CN106599926A (zh) * 2016-12-20 2017-04-26 上海寒武纪信息科技有限公司 一种表情图片推送方法及系统
CN107729543A (zh) * 2017-10-31 2018-02-23 上海掌门科技有限公司 表情图片推荐方法及设备

Also Published As

Publication number Publication date
CN107729543A (zh) 2018-02-23

Similar Documents

Publication Publication Date Title
EP3612926B1 (en) Parsing electronic conversations for presentation in an alternative interface
CN107251006B (zh) 具有共享兴趣的消息的图库
WO2019080637A1 (zh) 好友推荐方法及设备
US10970334B2 (en) Navigating video scenes using cognitive insights
US20190005332A1 (en) Video understanding platform
KR102574279B1 (ko) 검색/생성된 디지털 미디어 파일을 기반으로 잠재적 관련성에 대한 주제 예측
WO2019085625A1 (zh) 表情图片推荐方法及设备
WO2019144849A1 (zh) 一种为用户推送信息的方法和装置
US11769500B2 (en) Augmented reality-based translation of speech in association with travel
US20170109339A1 (en) Application program activation method, user terminal, and server
WO2019137391A1 (zh) 对视频进行分类匹配的方法、装置和挑选引擎
CN109274999A (zh) 一种视频播放控制方法、装置、设备及介质
CN112287168A (zh) 用于生成视频的方法和装置
US11100164B2 (en) Displaying videos based upon selectable inputs associated with tags
CN115516445A (zh) 对于检测到的对象对增强现实内容的基于语音的选择
WO2019214132A1 (zh) 信息处理方法、装置及设备
CN110674706B (zh) 社交方法、装置、电子设备及存储介质
JP5611155B2 (ja) コンテンツに対するタグ付けプログラム、サーバ及び端末
US11743530B2 (en) Systems and methods for improved searching and categorizing of media content items based on a destination for the media content machine learning
EP3306555A1 (en) Diversifying media search results on online social networks
JP7113000B2 (ja) 映像を生成するための方法および装置
CN112016001A (zh) 好友推荐方法、设备及计算机可读介质
JP6087704B2 (ja) コミュニケーションサービス提供装置、コミュニケーションサービス提供方法、およびプログラム
CN116863359A (zh) 目标物体的识别方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27.07.2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18873358

Country of ref document: EP

Kind code of ref document: A1