WO2015043532A1

WO2015043532A1 - Information processing method, apparatus, and system

Info

Publication number: WO2015043532A1
Application number: PCT/CN2014/087747
Authority: WO
Inventors: Hui Tang
Original assignee: Tencent Technology (Shenzhen) Company Limited
Priority date: 2013-09-30
Filing date: 2014-09-29
Publication date: 2015-04-02
Also published as: CN104144199A

Abstract

Method and device of saving displayed information in a user interface are disclosed. The method includes: displaying visual content, including human-recognizable text content, in the user interface； detecting a user's selection of a portion of the displayed visual content； based on the at least part of the displayed human-recognizable text content included the selected portion of the displayed visual content, recognizing a plurality of text blocks； extracting the information from at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks； and storing the extracted information in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks.

Description

INFORMATION PROCESSING METHOD, APPARATUS, AND SYSTEM

PRIORITY CLAIM AND RELATED APPLICATION

This application claims priority to Chinese Patent Application No. 201310465708. X, entitled “INFORMATION PROCESSING METHOD, APPARATUS, AND SYSTEM” filed on September 30, 2013, which is incorporated by reference in its entirety.

FIELD OF THE TECHNOLOGY

The present application relates to the field of computer sharing technologies, and more particularly to an information processing method, apparatus, and system.

BACKGROUND OF THE TECHNOLOGY

With the development of computer application technologies, electronic technologies, and communications technologies, people may obtain content such as a message or information sent by other users, merchants, or various network service providers. When a user is viewing related content, it may often occur that some fragments in the content need to be recorded and collected. For example, a user may want save a telephone number in a message or a piece of interesting news.

Currently, when a user selects a paragraph of text, an image, a video, or the like, the user needs to obtain related content in a manner of copying or manually inputting, and then, opens another interface to save the selected information. Such method is time consuming and involves quite a few steps of user operation.

Therefore, it is desirable to have an automatic and intelligent method so that the user device can better facilitate the user to store or share desired information.

SUMMARY

In some embodiments, a method of information processing (e.g. , selectively storing information) is performed at a device having one or more processors and memory storing instructions for execution by the one or more processors. The method includes: displaying visual content, including human-recognizable text content, in the user interface； detecting a user’s selection of a portion of the displayed visual content, the selected portion including at least part of the human-recognizable text content； based on the at least part of the displayed human-recognizable text content included the selected portion of the displayed visual content, recognizing a plurality of text blocks, each text block including information corresponding to at least one of a plurality of predetermined information categories； extracting the information from at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks； and storing the extracted information in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks.

In another aspect, a device comprises one or more processors, memory, and one or more program modules stored in the memory and configured for execution by the one or more processors. The one or more program modules include instructions for performing the method described above. In another aspect, a non-transitory computer readable storage medium having stored thereon instructions, which, when executed by a device, cause the device to perform the method described above.

BRIEF DESCRIPTION OF THE DRAWINGS

The aforementioned features and advantages of the application as well as additional features and advantages thereof will be more clearly understood hereinafter as a result of a detailed description of preferred embodiments when taken in conjunction with the drawings.

To describe the technical solutions more clearly, the following briefly introduces the accompanying drawings required for describing the embodiments or the prior art. Apparently, the accompanying drawings in the following description show merely some embodiments, and a person of ordinary skill in the art may still derive other drawings from these accompanying drawings without creative efforts.

FIG. 1 is a schematic flowchart of an information processing method in accordance with some embodiments；

FIG. 2 is a schematic flowchart of an information processing method in accordance with some embodiments ；

FIG. 3 is a schematic flowchart of an information processing method in accordance with some embodiments ；

FIG. 4 is a schematic flowchart of an information processing method in accordance with some embodiments；

FIG. 5 is a schematic flowchart of an information processing method in accordance with some embodiments；

FIG. 6 is a flowchart of a method of saving displayed information in accordance with some embodiments；

FIG. 7 is a schematic structural diagram of an information sharing and collection system in accordance with some embodiments；

FIG. 8 is a schematic structural diagram of an information processing apparatus in accordance with some embodiments；

FIG. 9 is one of specific schematic structural diagrams of a storage request module in FIG. 8；

FIG. 10 is a schematic structural diagram of a server in accordance with some embodiments；

FIG. 11 is one of specific schematic structural diagrams of a storage module in FIG. 10； and

FIG. 12 is a diagram of an example implementation of a user device 1200 in accordance with some embodiments.

Like reference numerals refer to corresponding parts throughout the several views of the drawings.

BRIEF DESCRIPTION OF DRAWINGS

Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the subject matter presented herein. But it will be apparent to one skilled in the art that the subject matter may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

The following clearly and completely describes technical solutions in the embodiments with reference to the accompanying drawings in the embodiments. Apparently, the described embodiments are merely a part of the embodiments rather than all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments without creative efforts shall fall within the protection scope.

FIG. 1 is a schematic flowchart of an information processing method according to an embodiment. The method is applicable between an intelligent terminal and an application server. Specifically, the method includes the following steps.

In accordance with some embodiments, in step S101, when detecting an operation in a share toolbar displayed for selected content in a current page, a first client device sends a storage request to a server according to a first account that is currently logged in to, wherein the storage request includes the selected content set as collection information.

In some embodiments, the first client device is an intelligent terminal such as a desktop computer, a mobile phone, a tablet computer, or an intelligent wearable device. The first client device may use a browser or a WebView to browse various content types such as a text webpage, an image, or a video.

If the current page is a WebView, before S101, the method may further include: when detecting a content selection operation in the WebView, the first client device invoking and displaying a preset share toolbar, wherein the preset share toolbar is generated after a menu controller in the WebView is pre-configured.

In the process of browsing information in the current page in the first client device, a user may select a paragraph of text, an image, a video, or the like in the current page by using an operation such as a left selection, a double click, or a touch screen sliding. In accordance with some embodiments, the share toolbar is obtained by pre-configuring a functional module such as the menu controller in the WebView. Therefore, the share toolbar is invoked and displayed after the user selects a paragraph of text, an image, or a video.

In accordance with some embodiments, in step S102, after receiving the storage request, the server stores the collection information based on the first account carried in the storage request.

The server is an application server of an instant messaging application or a social communication application, in which the first account is located. The server may determine, based on an agreement with the first client device, that an operation in response to the storage request is storing, based on the first account, the collection information carried in the request.

The server may locally search for a storage database created for the first account. If the database exists, the server directly stores the collection information carried in the storage request. And if the database does not exist, the server creates a database for the first account, and stores the collection information carried in the storage request into the created database.

In accordance with some embodiments, in step S103, when receiving a view request with the first account information, the server searches for the stored collection information according to the first account, and displays the found collection information to a terminal initiating the view request.

The user may obtain previously stored one or more pieces of collection information from the server, the first client device, or a second client device by logging in to the first account. After the first account is logged in to in the first client device or the second client device, the view request is sent to a corresponding server. The server extracts, based on the view request, one or more pieces of previous collection information from the database set for the first account, and displays the collection information to the user.

According to the method in some embodiments, after a user selects information from various pages such as a webpage, the selected information can be automatically stored, and then, the stored information is shared to another terminal or user when there is a sharing and view request.

FIG. 2 is a schematic flowchart of an information processing method in accordance with some embodiments. The method is applicable between an intelligent terminal and an application server. Specifically, the method includes the following steps.

In accordance with some embodiments, in step S201, when detecting an operation in a share toolbar displayed for selected content in a current page, a first client device sends a storage request to a server according to a first account that is currently logged in to, wherein the storage request includes the selected content set as collection information.

In S201, the sending, by a first client device, a storage request including collection information to a server includes: when there are multiple user accounts that are currently logged in to, the first client device requesting the user to select an account. These user accounts may specifically include various instant messaging application accounts, social communication application accounts, or other accounts that can uniquely identify the user in the server. These user accounts may be displayed to the user in a form of an account list, so that the user clicks and selects the first account therefrom.

In accordance with some embodiments, in step S202, after receiving the storage request, the server determines whether a database set for the first account carried in the storage request exists.

The server may create in advance a database for storing share information selected by the user for each registered user account.

In some embodiments, the server determines, after receiving the storage request of the first account, whether a database is created for the first account； if yes, performs S203； otherwise, performs S204.

In accordance with some embodiments, in step S203, the server stores the collection information included in the storage request into the database set for the first account carried in the storage request.

In accordance with some embodiments, in step S204, the server sets a database for the first account carried in the storage request, and stores the collection information included in the storage request into the set database.

In accordance with some embodiments, in step S205, when receiving a view request with the first account information, the server searches for the stored collection information according to the first account, and displays the found collection information to a terminal initiating the view request.

Specifically, in some embodiments, the step S205 may include: receiving a view request that is sent by a second client device by using the first account； searching for the stored collection information according to the first account initiating the view request, and sending the found collection information to the second client device that initiates.

Alternatively, the user may request, by using another user account and by sending the view request with the first account information, the server to display the collection information in the database corresponding to the first account to the user. In some embodiments, the step S205 may specifically include: receiving a view request that is sent by the second client device with a second user account； extracting the first account information from the view request； searching for the stored collection information according to the extracted first account information； and displaying the found collection information to the second client device.

In accordance with some embodiments, in step S206, when receiving an information sharing request with the first account information, the server searches for the stored collection information according to the first account, and sends the found collection information to a target location designated in the information sharing request.

Similarly, in some embodiments, the user may send the information sharing request to the server from any device by logging in to the first account. In some embodiments, one or more target locations are included in the information sharing request, and these target locations may be one or more accounts, and the server directly sends one or more pieces of collection information in the database created for the first account to these designated target locations. In some embodiments, the server may also first display the collection information included in the database, and then, send the selected collection information based on a selection of the user.

In addition, a collection message may be further shared to a target location, so that a sharing requirement of the user for collection information is further met.

FIG. 3 is a schematic flowchart of an information processing method in accordance with some embodiments. The method in some embodiments is applicable to a user device, which can be an intelligent terminal. Specifically, in accordance with some embodiments, the method includes the following steps.

In accordance with some embodiments, in step S301, the user device invokes and displays a share toolbar for selected content in a current page.

In the process of browsing information in the current page in the client device, a user may select a paragraph of text, an image, a video, or the like in the current page.

In accordance with some embodiments, in step S302, when an operation in the share toolbar is detected, the user device obtains the selected content, and sets the selected content as collection information.

In accordance with some embodiments, in step S303, the user device sends a storage request to a server according to a first account that is currently logged in to, wherein the storage request includes the selected content set as the collection information.

In accordance with some embodiments, the share toolbar is invoked and displayed after the user selects a paragraph of text, an image, or a video. If a sharing confirmation operation is sent after it is selected, the selected collection information may be obtained in a manner of automatically copying. The collection information is carried in the storage request and sent to the server. In some embodiments, a first client device specifically sends the storage request which carries the first account and the collection information.

Further, there may be multiple user accounts that are logged in to in the terminal. The step S303 may include: when there are multiple user accounts that are currently logged in to, requesting the user to select the first account therefrom, and sending the storage request by using the selected account.

FIG. 4 is a schematic flowchart of an information processing method in accordance with some embodiments. The method is applicable to application servers of various instant messaging applications or social communication applications. Specifically, the method includes the following steps.

In accordance with some embodiments, in step S401, after a storage request is received, the server extracts collection information included in the storage request and stores the collection information based on a first account carried in the storage request.

A first client device may send a storage request to a server according to the first account, wherein the storage request includes the collection information. After receiving the storage request, the server stores the collection information based on the first account carried in the storage request.

In accordance with some embodiments, in step S402, when a view request with the first account information is received, the server searches for the stored collection information according to the first account and displays the found collection information to a terminal initiating the view request.

In accordance with some embodiments, in step S403, when an information sharing request that is sent by using the first account is received, the server searches for the stored collection information according to the first account, and send the found collection information to a target location designated in the information sharing request.

FIG. 5 is a schematic flowchart of an information processing method in accordance with some embodiments. The method is performed among a first client device, a second client device, and a server. Specifically, in accordance with some embodiments, the method includes the following steps.

In accordance with some embodiments, in step S1, when detecting a content selection operation, a first client device displays a share toolbar, and obtains collection information.

In accordance with some embodiments, in step S2, the first client device initiates a storage request based on a first account, wherein the storage request carries the first account the content selected.

In accordance with some embodiments, in step S3, the server stores the collection information based on the first account.

In accordance with some embodiments, in step S4, the second client device sends a view request with the first account information.

In accordance with some embodiments, in step S5, the server searches for the stored collection information according to the first account.

In accordance with some embodiments, in step S6, the first terminal sends a view request with the first account information.

In accordance with some embodiments, in step S7, the server searches for the stored collection information according to the first account.

In an actual operation, the first client device, the second client device, or some other terminals may all extract corresponding collection information from the server by logging in to the first account or with the first account information.

In accordance with some embodiments, in step S8, the first terminal sends an information sharing request according to the first account. In an actual operation, a user may send the information sharing request by using any device to trigger the server to send the collection information stored by the first account to another designated server, e.g. , a server for another social network application.

In accordance with some embodiments, in step S9, the server sends the found collection information to another server designated in the information sharing request.

FIG. 6 is a flowchart of a method of saving displayed information in accordance with some embodiments. In some embodiments, the method is performed at a user device of having one or more processors and memory for storing one or more programs to be executed by the one or more processors.

In accordance with some embodiments, in step 601, the user device displays visual content, including human-recognizable text content, in the user interface.

In some embodiments, the visual content can include one or more of pictures, videos, animated images, plain text, formatted text, tables, spreadsheets, equations, etc. In some embodiments, the text is embedded in non-text content, e.g. , captions in a video. In some embodiments, a text in recognizable from non-text content, e.g. , title of a book is recognizable from a picture of a book cover. In some embodiments, the user interface is provided by a document processing program as described in FIG. 12.

In accordance with some embodiments, in step 602, the user device detects a user’s selection of a portion of the displayed visual content, the selected portion including at least part of the human-recognizable text content.

In some embodiments, there are various ways for a user to select a portion of the displayed visual content. One way would be use “ctrl” plus “a” to select all content displayed in a laptop. Other ways may include dragging a selection frame to select everything in the selection frame on a screen of a tablet. The boundary of the selection frame may fall anywhere on the display, and may partially include some items on the display. Sometime, a quick double touching from the user causes the user device to select a whole paragraph or section.

In accordance with some embodiments, in step 603, based on the at least part of the displayed human-recognizable text content included the selected portion of the displayed visual content, the user device recognizes a plurality of text blocks, each text block including information corresponding to at least one of a plurality of predetermined information categories. Sometimes, the text blocks are recognized from plain text or formatted text displayed on the screen. Sometimes, the text blocks may be recognized from displayed images, videos, or pictures, icons, etc.

In some embodiments, the step 603 includes a few sub-steps. First, in some embodiments, the user device recognizes all the text in the selected portion of content. If the visual content is a video or animated picture, the user may recognize text from the content being currently displayed. In some embodiments, recognizing text involves getting text information from the document being displayed. For example, a video may include or be associated with a caption document. The user device may retrieve text information from the caption document. In some embodiments, the user device uses text recognition technology, including Optical Character Recognition tools to recognize texts from non-text content, e.g. , a picture.

Second, in some embodiments, the user device obtains a plurality of predetermined information categories. The plurality of the predetermined information categories are set up to determine what information a user would want capture from the displayed content. “Information category” refers to the nature of the content itself. For example, information categories can include: names, titles, jobs, home addresses, telephone numbers, email addresses, organizations, article titles, authors, claims of patents, keywords of paper, equations, a stock quote, a distance, a physics variable, a medical check-up result, etc. Information categories may also include categories which are combination of other categories or combination of different object/content types. These information categories include: contact card, author information, summary of a news article, and so on. In some embodiments, the user device allows the user to create or change information categories. For example, if a user habitually saves the information of titles, authors, publication journals, publication dates of academic articles, the user can create an information category named “article summary” and instructs the user device to use the created information category to collect the desired information. “Content type” refers to whether the content is an image, a word document, an icon, an avatar, an equation, a spreadsheet, a table, an advertisement, etc.

Third, in some embodiments, the user device determines whether any text should be grouped together as a text block. In some embodiments, recognizing text blocks is based on punctuations, distances among text, display locations, etc. In some embodiments, determining text block is combined with the next step of determining whether any text including information corresponding to at least one of a plurality of predetermined information categories.

Fourth, in some embodiments, the user device determines whether any text blocks including information corresponding to at least one of a plurality of predetermined information categories. In some embodiments, recognizing text blocks is further based on whether some text conforms to the pre-stored criteria of information categories. For example, in determining whether two lines of text should be grouped together as a single text block or separately as two text blocks, the user device determines that the first line is street information, and the second line is city and state information. The user device further determines that the two lines should be combined in a single text block corresponding to an address.

In recognizing text blocks for information categories, the user device can take various cues from the meaning of the selected portion and the context of the selected portion. For an example of taking cues from meaning, a user device may recognize a selected portion: “Author: Jane Austen. ” Based on the word, “Author, ” the user device determines that the following two words are possibly the name of an author. The user device may also recognize that both “Jane” and “Austen” are initially capitalized and common English names, which further confirms that “Jane Austen” is an author. At last, a user device equipped with a big database and strong processing capability may directly recognize “Jane Austen” as the name of the famous novelist.

For an example of taking cues from context, a user device recognizes that selected content is from an outlook document, which indicates that the selected content is from an email. Since the bottom of an email is usually the name and contact information of the sender, the user device recognize the bottom of the email as contact information. The user device may further recognize a few text blocks that correspond to different categories of contact information. For another example, based on unselected or un-displayed part of the same document, the user device determines that the selected portion is part of text of a U.S. patent. Recognizing the U.S. patent helps further recognition in two ways. Firstly, the user device can narrow down the possible information categories that may correspond to the selected portion and therefore increase the accuracy of recognition. Secondly, U.S. patents have strict format requirement to organize information in an orderly and predictable ways. The user device can obtain the format of U.S. patents and recognize which part of the patent the selected portion is from.

In accordance with some embodiments, the user device sends the selected portion of the displayed visual content to a remote server and receives definitions of the plurality of text blocks from the server. In some embodiments, the server provides the interface which displays the visual content. For example, rather than recognizing the text blocks from a webpage, the user device requests the server that manages the webpage to recognize or define text blocks.

In accordance with some embodiments, in step 604, the user device changes a scope of the selected portion of the displayed visual content in accordance with a scope of at least one of the plurality of the text blocks.

Since it is common that a user does not select precisely, it is desirable for the user device to adjust the scope of selection. In some embodiments, under various circumstances, the user device may enlarge and/or decrease the scope of the selected portion of the displayed visual content.

In some embodiments, the user device enlarges the scope of the selected portion because one or more text blocks appear to be incomplete. The user device recognizes that a first text block contains incomplete information in accordance with the respective at least one of the predetermined categories corresponding to the first text block； and increases the scope of the selected portion of the visual content to include additional information needed to complete the information in the first text block. For example, a user selects “my telephone number is 123-456. ” But the user device recognizes that the complete number is 123-45678901 with “78901” not selected by the user. The user device thus enlarges the scope of the selected portion to cover the whole telephone number.

In some embodiments, the user device discards contents that do not belong to any of the text blocks from the selected portion of the visual content. For example, the user selects “My telephone number is 123-45678901. Feel free to call me for any questions. ” The user device determines that part of the selection “Feel free to call me for any questions” does not belong to any text blocks and discard it from the selected portion of the visual content. The user device also determines that the part of the selection “My telephone number is” does not belong to any text block either, although this part is useful in determining the following number as a telephone number. Therefore, the user device shrinks the selected portion from “My telephone number is 123-45678901. Feel free to call me for any questions. ” to “123-45678901. ”

In some embodiments, when the user device receives detects a user’s selection of a portion of the displayed visual content, the user device recognize and/or displays a boundary for each selected portion, including a boundary of the at least part of the displayed human-recognizable text content included the selected portion of the displayed visual content. When the user device changes a scope of the selected portion of the displayed visual content, it also changes the boundary of the selected portion of the displayed content in accordance with a scope of at least one of the plurality of the text blocks.

In accordance with some embodiments, in step 605, the user device extracts the information from at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks. In some embodiments, extracting the information includes copying the text in every text block and associating the copied text with corresponding information categories. In some embodiments, extracting the information includes recognizing the type of every piece of information according to the predetermined categories. In the above example of recognizing “Jane Austen, ” the user device recognizes that “Jane” is the first name and “Austen” is the last name. ” In another example, when the user device extracts from a text block “123-45678901” as a telephone number, the user device recognizes that the “123-45678901” is a telephone number with the “123” being the area code.

In some embodiments, after extracting the information from at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks, the user device prompts the user to confirm the extracted information. In some embodiments, the user device prompts the user to confirm the accuracy of the extracted text. In some embodiments, the user device prompts the user to confirm the one or more information categories corresponding to the text blocks. For example, after identifying a bank account number from a message, the user device displays an interface, containing “bank account number: 123456788, ” and asks the user to confirm, revise or reject.

In some embodiments, the user device sends an extraction request to a server, the extraction request includes the respective at least one of the text blocks and receives information extracted from the respective at least one of the text blocks from the server.

In accordance with some embodiments, in step 606, the user device stores the extracted information in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks.

In some embodiments, storing the extracted information in accordance with the respective at least one of the predetermined categories including identifying the storage location for the respective at least one of the predetermined categories and storing the extracted information in the respective identified storage location. The storage location can be a user accessible storage address or path (such as a specific location on a hard disk or a storage path accessible by the user) or a virtual storage location (such as a place within a particular interface that the user can view) . For an example of a virtual storage location, an email address is saved in the contact list of a user and the contact list is managed by a social network server, the user can view the email address by accessing the contact list through the social network program without knowing or caring about whether the contact list is saved in a memory, a newly added memory card, or a server.

In some embodiments, the user device sends a storage request to a server, wherein the storage quest includes the extracted information and the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks. In some embodiments, a user can later retrieve the information saved in a server by logging into an account through a second user device. This method of saving and retrieving information is described in FIGS. 1-5 and their accompanying text, and will not be repeated.

In some embodiments, the user device recognizes at least a first content object of the first content type and a second content object of the second content type based on the selected portion of the visual content； and presents each of the first content object and the second content object for selection by the user for individual storage. For example, the user selects a picture and a paragraph of text. The picture is a symbol of a school and shows the school’s name and the paragraph of text includes the mailing address of the school. The user device presents the picture and the mailing address and a user can select to store the picture, the mailing address, or both. In addition, the user device can ask the user whether to store the school’s name as text, which is another content type than the original content type as a picture. For another example, the first content object of the first content type can be a sequence of characters that forms a password. This sequence of characters can be shown as plain text in the visual content. The second content object of the second content type can be a picture of a storefront. The user device presents the text string for the password, and the picture for the store for the user to selectively save them as separate items (e.g., a password string and a picture) .

For example, the user can select a rough scope on the user interface, which can include partial text portions, and partial image portions, and/or partial other content/object types. The computer helps the user identify each object that can be separately stored, and lets the user decide which ones to save. For example, if a user has selected a portion of the user interface that includes one image, one half image, one table, two paragraphs, one half sentence. The user device adjusts the selection scope and recognizes three images, one table, and three sentences, and presents them as a list for the user choose which ones to save.

In some embodiments, the user device provides an option to save the first content object in accordance with another content type different from the first content type. For example, if the first content object of the first content type is a password string, the user device asks the user whether he/she wants to save the password as an image file. In a similar case, if the user device recognizes the store name in the picture (i. e. , the second content type) , the user device asks the user whether he/she wants to save the store name as text. In some embodiments, the user device can provide more than one option for the user to choose. For example, for the same password string, the user can choose to save it as text, as an image, or as a password in a password management app. The user device may offer three items for the user to save for the same password string, each item representing a saving format. The user can save from zero to all of the three items.

In some embodiments, some information categories allow the user to save multiple items of information with different object/content types. For example, a contact card category, may allow the user to save a picture, a URL, a company name, an address, and a phone number, which may at least partially be included in the selected portion of the user interface.

In some embodiments, the user device identifies a document that contains the human- recognizable text content, obtains identity information that is associated with the document； and matching at least part of the identity information with the extracted information from the respective at least one of the text blocks. For example, the user device receives an SMS message from a contact, the message including “my home address is …” The user device obtains the identity of the sender of the SMS message, finds that the sender is an existing contact within the contact list and saves the home address to the contact information of the existing contact. The same method can be applied to emails, social network messages, and any documents that provide identity information.

In some embodiments, instead of or in addition to storing the extracted information in accordance with the respective at least one of the predetermined categories, the user device shares the extracted information in accordance with the respective at least one of the predetermined categories. In some embodiments, similar to a user saving the extracted information in different formats or content types, the user can also instruct the user device to share the extracted information in different formats or content types. For example, when a user sends bank account information to a business partner, the user device can create an image showing the bank account information and send the image to the business partner. In some embodiments, the user device may also choose the best content type based on the nature of the information (e.g. , high security requirement or compactness of information item) for the user when sharing the information with another user.

FIG. 7 is a schematic structural diagram of an information sharing and collection system in accordance with some embodiments. The system includes: a first terminal 1 and a server 2.

The first terminal 1 is configured to send a storage request to the server according to the first account.

The server 2 is configured to store the collection information in the storage request based on the first account carried in the storage request.

The server 2 is further configured to search for the stored collection information according to the first account, and send the found collection information to a terminal initiating the view request.

The terminal initiating the view request may be the first terminal 1 or other terminals.

Specifically, for specific implementation of the first terminal 1 and the server 2, reference may be made to the foregoing descriptions in the embodiments corresponding to FIGS. 1-6.

FIG. 8 is a schematic structural diagram of an information processing apparatus in accordance with some embodiments. The apparatus in accordance with some embodiments is a first client device or a first terminal 1 in other figures. In some embodiments, the apparatus includes:

An invoking module 11, configured to invoke and display a share toolbar for selected content in a current page；

An obtaining module 12, configured to obtain the selected content, and set the selected content as collection information； and

A storage request module 13, configured to send a storage request to a server according to a first account that is currently logged in to.

Specifically, as shown in FIG. 9, the storage request module 13 may include: a request unit 131, configured to request the user to select the first account therefrom； and a storage unit 132, configured to send the storage request including the collection information to the server.

Further specifically, for specific implementation of the apparatus in accordance with some embodiments, reference may be made to the foregoing descriptions in the embodiments corresponding to FIGS. 1-6.

FIG. 10 is a schematic structural diagram of a server in accordance with some embodiments. The server may be a social network application server in some embodiments. In some embodiments, the server includes: a storage module 21, configured to extract collection information included in the storage request, and store the collection information； and a display module 22, configured to search for the stored collection information according to the first account, and send the found collection information to a terminal initiating the view request.

Further optionally, the apparatus in accordance with some embodiments may further include: a sharing module 23, configured to search for the stored collection information according to the first account, and send the found collection information to a target location designated in the information sharing request.

Specifically, as shown in FIG. 11, the storage module 21 may specifically include:

A determining unit 211, configured to determine whether a database set for the first account carried in the storage request exists；

A first storage unit 212, configured to store the collection information included in the storage request into the database set； and

A second storage unit 213, configured to set a database for the first account carried in the storage request, and store the collection information included in the storage request into the set database.

Further specifically, for specific implementation of the apparatus in accordance with some embodiments, reference may be made to the foregoing descriptions in the embodiments corresponding to other figures.

FIG. 12 is a diagram of an example implementation of a user device 1200 in accordance with some embodiments. While certain specific features are illustrated, those skilled in the art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity and so as not to obscure more pertinent aspects of the implementations disclosed herein. To that end, the user device 1200 includes one or more processing units (CPU’s ) 1202, one or more network or other communications interfaces 1208, a display 1201, memory 1206, and one or more communication buses 1204 for interconnecting these and various other components. The communication buses may include circuitry (sometimes called a chipset) that interconnects and controls communications between system components. The memory 1206 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices； and may include non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. The memory 1206 may optionally include one or more storage devices remotely located from the CPU (s) 1202. The memory 1206, including the non-volatile and volatile memory device (s) within the memory 1206, comprises a non-transitory computer readable storage medium.

In some implementations, the memory 1206 or the non-transitory computer readable storage medium of the memory 1206 stores the following programs, modules and data structures, or a subset thereof including an operating system 1216, a network communication module 1218, an information extraction program 1220 and various document processing programs 1225.

In accordance with some embodiments, the operating system 1216 includes procedures for handling various basic system services and for performing hardware dependent tasks.

In accordance with some embodiments, the network communication module 1218 facilitates communication with other devices via the one or more communication network interfaces 1208 (wired or wireless) and one or more communication networks, such as the internet, other wide area networks, local area networks, metropolitan area networks, and so on.

In accordance with some embodiments, the information extraction program 1220 is configured to recognize, extract, store and share certain information from visual content displayed in an interface. In some embodiments, the information extraction program 1220 comprises a selection module 1221, an extraction module 1222, a storage module 1223, and a sharing module 1224.

In accordance with some embodiments, the selection module 1221 is configured to detecting a user’s selection of a portion of the displayed visual content and change a scope of the selected portion of the displayed visual content.

In accordance with some embodiments, the extraction module 1222 is configured to extracting the information from the text blocks.

In accordance with some embodiments, the storage module 1223 is configured to store the extracted information in accordance with the predetermined categories.

In accordance with some embodiments, the sharing module 1224 is configured to share the extracted information in accordance with the predetermined categories. The working steps of information extraction program 1220 and its modules are illustrated in FIG. 6 and will not be repeated here. In some embodiments, at least some of the functions of the user device 1200 can be performed by a remote server at the request of the user device.

In accordance with some embodiments, the document processing programs 1225 are configured to process documents and display the documents in interfaces. In some embodiments, the document processing programs include Microsoft Word Program, social network programs, email programs, SMS message programs, PDF document processing programs, translation software, internet browsers, online payment programs, image processing programs, etc.

Although some of the various drawings illustrate a number of logical stages in a particular order, stages that are not order dependent may be reordered and other stages may be combined or broken out. While some reordering or other groupings are specifically mentioned, others will be obvious to those of ordinary skill in the art and so do not present an exhaustive list of alternatives. Moreover, it should be recognized that the stages could be implemented in hardware, firmware, software or any combination thereof.

The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the application to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the technology and its practical applications, to thereby enable others skilled in the art to best utilize the technology and various embodiments with various modifications as are suited to the particular use contemplated.

Claims

A method of saving displayed information in a user interface, comprising:

at a user device of having one or more processors and memory for storing one or more programs to be executed by the one or more processors:

displaying visual content, including human-recognizable text content, in the user interface；

detecting a user’s selection of a portion of the displayed visual content, the selected portion including at least part of the human-recognizable text content；

based on the at least part of the displayed human-recognizable text content included the selected portion of the displayed visual content, recognizing a plurality of text blocks, each text block including information corresponding to at least one of a plurality of predetermined information categories；

extracting the information from at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks； and

storing the extracted information in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks.
The method of claim 1, further comprising:

before extracting the information from the at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks, changing a scope of the selected portion of the displayed visual content in accordance with a scope of at least one of the plurality of the text blocks.
The method of claim 2, wherein changing the scope of the selected portion of the displayed visual content in accordance with the scope of at least one of the plurality of the text blocks comprises:

discarding content that does not belong to any of the text blocks from the selected portion of the visual content.
The method of claim 2, changing the scope of the selected portion of the displayed visual content in accordance with the scope of at least one of the plurality of the text blocks comprises:

recognizing that a first text block contains incomplete information in accordance with the respective at least one of the predetermined categories corresponding to the first text block； and

increasing the scope of the selected portion of the visual content to include additional information needed to complete the information in the first text block.
The method of claim 1, wherein the predetermined categories include predetermined categories of contact information.
The method of claim 5, further comprising:

identifying a document that contains the human-recognizable text content；

obtaining identity information that is associated with the document； and

matching at least part of the identity information with the extracted information from the respective at least one of the text blocks.
The method of claim 1, further comprising:

after extracting the information from at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks, prompting the user to confirm the extracted information.
The method of claim 1, wherein the displayed visual content includes multiple content types, the selected portion of the displayed visual content includes at least a first content type and a second content type, and the method further comprises:

recognizing at least a first content object of the first content type and a second content object of the second content type based on the selected portion of the visual content； and

presenting each of the first content object and the second content object for selection by the user for individual storage.
A device of saving displayed information in a user interface, comprising:

one or more processors； and

memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for:

displaying visual content, including human-recognizable text content, in the user interface；

detecting a user’s selection of a portion of the displayed visual content, the selected portion including at least part of the human-recognizable text content；

based on the at least part of the displayed human-recognizable text content included the selected portion of the displayed visual content, recognizing a plurality of text blocks, each text block including information corresponding to at least one of a plurality of predetermined information categories；

extracting the information from at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks； and

storing the extracted information in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks.
The device of claim 9, wherein the one or more program modules further include instructions for:

before extracting the information from the at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks, changing a scope of the selected portion of the displayed visual content in accordance with a scope of at least one of the plurality of the text blocks.
The device of claim 10, wherein changing the scope of the selected portion of the displayed visual content in accordance with the scope of at least one of the plurality of the text blocks comprises:

discarding content that does not belong to any of the text blocks from the selected portion of the visual content.
The device of claim 10, wherein changing the scope of the selected portion of the displayed visual content in accordance with the scope of at least one of the plurality of the text blocks comprises:

recognizing that a first text block contains incomplete information in accordance with the respective at least one of the predetermined categories corresponding to the first text block； and

increasing the scope of the selected portion of the visual content to include additional information needed to complete the information in the first text block.
The device of claim 9, wherein the predetermined categories include predetermined categories of contact information.
The device of claim 13, wherein the one or more program modules further include instructions for:

identifying a document that contains the human-recognizable text content；

obtaining identity information that is associated with the document； and

matching at least part of the identity information with the extracted information from the respective at least one of the text blocks.
The device of claim 9, wherein the one or more program modules further include instructions for:

after extracting the information from at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks, prompting the user to confirm the extracted information.
The device of claim 9, wherein the displayed visual content includes multiple content types, the selected portion of the displayed visual content includes at least a first content type and a second content type, and the one or more program modules further include instructions for:

recognizing at least a first content object of the first content type and a second content object of the second content type based on the selected portion of the visual content； and

presenting each of the first content object and the second content object for selection by the user for individual storage.
A non-transitory computer readable storage medium having instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:

displaying visual content, including human-recognizable text content, in the user interface；

detecting a user’s selection of a portion of the displayed visual content, the selected portion including at least part of the human-recognizable text content；

based on the at least part of the displayed human-recognizable text content included the selected portion of the displayed visual content, recognizing a plurality of text blocks, each text block including information corresponding to at least one of a plurality of predetermined information categories；

extracting the information from at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks； and

storing the extracted information in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks.
The device of claim 17, wherein the one or more program modules further include instructions for:

before extracting the information from the at least one of the text blocks in accordance with the respective at least one of the predetermined categories corresponding to the respective at least one of the text blocks, changing a scope of the selected portion of the displayed visual content in accordance with a scope of at least one of the plurality of the text blocks.
The device of claim 18, wherein changing the scope of the selected portion of the displayed visual content in accordance with the scope of at least one of the plurality of the text blocks comprises:

discarding content that does not belong to any of the text blocks from the selected portion of the visual content.
The device of claim 18, wherein changing the scope of the selected portion of the displayed visual content in accordance with the scope of at least one of the plurality of the text blocks comprises:

recognizing that a first text block contains incomplete information in accordance with the respective at least one of the predetermined categories corresponding to the first text block； and

increasing the scope of the selected portion of the visual content to include additional information needed to complete the information in the first text block.