CN102945074B - List and task is filled according to caught voice and audio content - Google Patents

Abstract

The invention describes and fill list and task according to caught voice and audio content.Provide by the mobile computing device of record audio automatic capturing task and list items can be come through voice or audio frequency input and fill it into electronic tasks or list surface.Voice or audio task or list items can be captured for being input in task application interface or in list creation surface interface, to be used as task items, prompting, " to be done " item, list items, schedule item, job engineering summary etc. subsequently.The voice caught or audio content can locally or remotely be transcribed, and the content of transcribing can be filled in task or list creation surface user interface, this task or list creation surface user interface can be displayed on capture device (such as mobile phone), maybe can be stored remotely and show explicitly from the multiple application on multiple different computing equipment subsequently.

Description

List and task is filled according to caught voice and audio content

Technical field

The present invention relates to and generate and fill list and task, especially generate according to audio content and fill list and task.

Background technology

Along with the appearance of computing machine and computer software, a large amount of progress helping people to manage their both work and inoperative life are made.In order to help just to attempt to carry out the people of work, family and the multi-task between the two simultaneously, develop the task of often making us shrinking of all things that electronic tasks and calendar program must to complete with standby maintenance, tracking and memory every day.Unfortunately, many people be usually reduced to into oneself hand-written a large amount of p.m.entry on the little scraps of paper, on notepad, note is first-class.See that many p.m.entrys are bonded to individual desk, computer screen, family's refrigerator etc. to remind the one or more task of individual's process to be general.Although individual can access and the electronic tasks that can make good use of or calendar program, he or she is exactly irresistible hand-written many p.m.entrys to follow the tracks of him or she and think those things that " really " is important.

Along with mobile computing device (such as, mobile phone and mobile computer) sharp increase and use, although most people possibly cannot easily obtain the little scraps of paper, notepad or note, but they just have at least one mobile computing device on hand usually, task or list are input to one or more electronic tasks list, calendar or list by this at least one mobile computing device available.But, under specific circumstances, such as, when user drives or when user advances, use this equipment being used for incoming task or list items may be not easy to obtain or improper.

For these and other, the present invention considers that item is made just.

Summary of the invention

Various embodiments of the present invention by through can recording audio mobile computing device (such as mobile phone or mobile computer) or inputted by the voice of the voice that are associated with fixing or wired computing devices or audio capture instrument (such as microphone) or audio frequency, automatic capturing be provided and task and list items be filled into electronic tasks or list surface, solving above and other problem.According to each embodiment, voice or audio task or list items can be captured for being input in task application interface or being input in list creation surface interface, to be used as task items, prompting, " to be done " item, list items, schedule item, job engineering summary etc. subsequently.The voice caught or audio content can locally or remotely be transcribed, and the content of transcribing can be filled in task or list creation surface user interface, this task or list creation surface user interface can be displayed on capture device (such as mobile phone), maybe can be stored remotely and show explicitly from the multiple application on multiple different computing equipment subsequently.In addition, if needed, the audio file comprising caught voice or audio content can be attached to the content of transcribing and processing so that allow user and the voice caught or audio file be shown transcribe the audio file that playback explicitly catches.

Can perform natural language processing to transcribed voice or audio content, and metadata can be applied to content that is transcribed and that process to strengthen the use to content.Such as, be applied to the metadata of content that is transcribed and process and can allow the efficient filtration of the content to transcribed and process, editor, search, classification or other similar operations.Use the event time line of " fuzzy " based reminding method that prompting can be allowed to be associated with task, event and activity, some of them task, event and/or activity must fixed date/time completes or occurs, but other tasks, event and/activity can or not complete or occur during loose or " fuzzy " date/time scope during specified date/time scope.

One or more embodiments of the detail are illustrated in the accompanying drawings and the description below.By reading detailed description below and with reference to the accompanying drawing be associated, other feature and advantage will become apparent.Should be understood that detailed description is below only illustrative, instead of the restriction to invention required for protection.

There is provided content of the present invention to introduce some concepts that will further describe in the following specific embodiments in simplified form.Content of the present invention is not intended to the key feature or the essential feature that identify theme required for protection, is not intended to the scope for helping to determine theme required for protection yet.

Accompanying drawing explanation

Fig. 1 is the simplified block diagram of the list creation surface user interface illustrated in the indicator screen deploy of computer monitor.

Fig. 2 illustrates the indicator screen deploy at computer monitor, the simplified block diagram of the list creation surface user interface of to fill with one or more task, event, activity or information segment.

Fig. 3 is the simplified block diagram of the list creation surface user interface of Fig. 2 that the information list perceived from selected task items is shown.

Fig. 4 is the simplified block diagram can implementing the computing architecture of various embodiments of the present invention wherein.

Fig. 5 illustrates that the simplified block diagram of surface user interface is created in the list shown explicitly with shown document.

Fig. 6 is simplified block diagram mobile computing device being shown and the list creation surface user interface in the indicator screen deploy of this mobile computing device is shown.

Fig. 7 is simplified block diagram mobile computing device being shown and the list creation surface user interface in the indicator screen deploy of this mobile computing device is shown.

Fig. 8 illustrates that its seizure voice available or audio content are to generate the simplified block diagram of the mobile computing device of task or the list items be associated.

Fig. 9 illustrates that its seizure voice available or audio content are to generate the simplified block diagram of the mobile computing device of task or the list items be associated.

Figure 10 is the simplified block diagram that the mobile computing device showing the list items obtained from caught phonetic entry is thereon shown.

Figure 11 is the simplified block diagram that the mobile computing device showing the list items obtained from caught phonetic entry is thereon shown.

Figure 12 is the simplified block diagram of the mobile computing device that displaying calendar information is thereon shown, this calendar information shows explicitly with a voice catching and process or audio frequency input.

Figure 13 illustrates for catching voice and/or audio frequency input and filling the process flow diagram of the method for electronic tasks or list interface by the information obtained from caught voice or audio recording.

Figure 14 is the simplified block diagram of the computing equipment of its enforcement various embodiments of the present invention available.

Figure 15 A and 15B is the simplified block diagram that can be used to the mobile computing device implementing various embodiments of the present invention.

Figure 16 is the simplified block diagram can implementing the distributed computing system of various embodiments of the present invention wherein.

Embodiment

As briefly described above, various embodiments of the present invention be intended to through can record audio mobile computing device (such as mobile phone or mobile computer) or inputted by the voice of the voice that are associated with fixing or wired computing devices or audio capture instrument (such as microphone) or audio frequency, automatic capturing task and list items, and fill it into electronic tasks or list surface.According to an embodiment, voice and audio frequency input can catch via the speech capture being equipped with in automobile or other vehicles or embedding and recognition system.Such as, when user drives, when user advances, task or list items can catch via voice or audio frequency input, and can catch long bulleted list and not need user sight line to be removed from main focus item.The available minimum touch to capture device (such as mobile phone) and/or the seizure performed alternately this information, and can perform the seizure of information and need not suspend between the input of each task or list items.According to an embodiment, can be pointed out by user the seizure of this information and start via voice command (such as ordering " record " etc.).In addition, about the decision of caught interested item or action can be postponed to check caught information concerning user more suitably one catch after time.

Following embodiment is with reference to each accompanying drawing.As possible, just use identical Reference numeral to indicate same or analogous element in the accompanying drawings and the description below.Although may describe embodiments of the invention, amendment, reorganization and other realizations are possible.Such as, can the element shown in accompanying drawing be replaced, adds or be revised, and by revising method described herein to the displacement of disclosed method, rearrangement or interpolation stage.Therefore, detailed description does not hereafter limit the present invention, and on the contrary, OK range of the present invention is limited by claims.

According to each embodiment, can be automatically populated in task application user interface according to the task items that recorded voice and/or audio frequency input generate.In addition, list items, such as necessary list items (such as supply, groceries list etc.), " to be done " list items, schedule list items etc., can generate according to recorded input, and can be filled on recording unit (such as mobile phone) or move to any other of its Distribution List item or fix in the list creation surface on computing equipment.The description of Fig. 1-7 is intended to generate, show and use the list be associated with any application and computing equipment to create surface and task interface.The description of Fig. 8-13 is intended to generate, show and use the list be associated with the list generated according to recorded voice and/or other audio contents and task to create surface and task interface.The description of Figure 14-16 is intended to the operating environment of its enforcement various embodiments of the present invention available.

Subsequently with reference to figure 1, the listuser interface 120 on list creation surface can be used as showing the electronic memorandum be associated, the little scraps of paper, notepad, note etc. with the one or more software application for inputting list items manually or automatically, and described list items such as user may remember on a piece of paper, notepad or other media originally to remain on the task of foremost or reminding user, event, activity or other information segments by being considered to important this information user.Except information being input to except user interface 120, user interface 120 can be used for Quick Catch from the information of the document be opened and the information that is associated with the document be opened, makes user need not leave current application in order to initiating task input user interface.In fact, list creation surface user interface (UI) 120 can be associated with various e-file (such as, electronic document, e-mail item, photo and other images, contacts, social networking information etc.).Except these content items, list creation surface can also be associated with the electronic chart of assisted user in one or more task.Such as, if user at the supermarket in, make list create surface and to be associated with positional information and to can be used for triggering the prompting he/her being needed to the thing bought.

As shown in Figure 1, can create in surperficial UI120 in list the one or more function button or the control 125,130,135,140 that are provided for editing or otherwise manipulate the information comprised in UI120.Such as, completed task that control 125 can be used for " checking ", control 130 can be used for adding extra task, event or other information, control 135 can be used for import information or for by information annotate for storing in user interface 120 or show, and other controls 140 various can be provided for editor, classification, filtration, search etc. to other type that the information comprised in user interface 120 is carried out.According to each embodiment, the metadata with being associated with selected function control can be caused to mark task or list items to the selection of the one or more functionality controls be associated with task or list items, selected function control can be used for Processing tasks or list items subsequently, as described below.

According to an embodiment, a this control can be used for arranging computing machine, there is this computing machine, list creation surface is used as " not bothering " pattern, make when " not bothering " pattern is activated, do not have new e-mail item, instant messaging communication (IM) item or other distracting contents can arrive list creation surface.Another this control 140 can allow " to doze off " pattern to current shown task application, makes to change the new task of display one into, and makes user need not determine how to process " dozing off " task.That is, user can keep task by application " dozing off " pattern.In addition, pattern of " dozing off " can be used for filtering out the only incoherent information with current task.Such as, if user will " doze off ", model application " plans moral event " to task, and if the current task of user is " redesign product ", and user receives the Email about this project from his/her supervisor, then can show the Email about task " redesign product ", and can not to the Email of user's display about " planning moral event " task.

According to an embodiment, replace using " not bothering " or " dozing off " function, list creation surface function can test example as user when be busy with on phone talk, prepare document, operate video games, and list creation surface automatically can retain " dozing off " function until user have time.In addition, user can they know they when not busy (such as, next hour, tomorrow, next week etc.) determine when they can receive prompting.

With reference now to Fig. 2, the creation of list shown in the expansion table of various list items surface user interface 120 is being shown, list items such as manually or has automatically been input to task in list creation surface, event, activity or other information segments by information capture.Such as, first entry " closedown sprinkling system " 220 describes a task, and this task can be input in list creation surface user interface 120 to remind him to process this task when user goes home by user.For another example, second entry " redesign product " 225 is shown to have the multiple subtasks 230 be associated with main task 225.The every display arrangement can specified according to one or more shown in LASUI120 show, such as based on regulation execution time, nearest at top, user first 5 of specifying etc.This display arrangement also can be applicable to below with reference to the perspective display described by Fig. 3.

Advantageously, information is input to list to create in surperficial UI120 and edit file is easy and efficiently.Such as, can with Bulleted List item is input to mode similar in word processing file information is typed in UI.That is, user can input one, selects " carriage return " key, " tab " key etc., and ensuing entry is by the next line that is placed in UI120 or next sentence, but still has the whole of the advantage of any applied metadata.Such as, LASUI can be formatted, make simple carriage return or tab select can to create in inputted list items can be useful to user hierarchy.Such as, user can input first task " plan dinner party ", follows by carriage return or tab, is next entry " reservation dining room ", is that another carriage return and entry " check menu supply " subsequently.By structured formatted to inputted item applying hierarchical, latter two can be automatically listed in below Section 1 and relative to this Section 1 indentation, with the hierarchical relational shown by creating at every.

According to each embodiment, create in surperficial UI120 once data or other information are filled into list, metadata (such as, phrase " team " 235) just can be applied to keyword, Key Term, key phrase or other informational contents, to allow structuring to the text comprised in UI120 or information, editor, filtration, search, classification or other robotizations manipulation.Such as, metadata " team " 235 can be applied to task " meet with Contoso team also assess the region that they can support us ", is by one or more tasks of being completed by given team to indicate example task.On the contrary, metadata " EricGruber " 240 is applied to task and " arranges meeting for looking back next time ", is associated with specific people to indicate this task.

According to each embodiment, can via such as below with reference to the various input mediums described by 14-16, this metadata being applied to various types of item of information and composition.Such as, can input via thump, touch-screen input, voice or the input of other audio frequency, posture input (computing equipment be wherein associated is equipped with the detection for catching and explain user's posture (such as, camera) function) input and application task and list items and the metadata applied.Such as, according to one embodiment of present invention, carry out apply metadata item by user's speech, wherein user says as this metadata item for being applied to Given task or list items described herein.

The metadata of other types many can be applied to list and create information in surperficial UI120.Such as, although " " symbol of example above for task is associated with people or group, " # " symbol can be used for marking task or other information by random metadata.For other examples, " $ " can be used for marking monetary information, and " & " symbol can be used for marking date or time.Should be appreciated that any one that can use in a large amount of this metadata type and symbol, and aforementioned only for exemplary purposes.Advantageously, the information that this metadata item 235,340 can be created in surperficial UI with list is associated, and does not input another text or data entry field or do not start any other user interface components.It is also understood that being applied to this metadata that list creates the various information in surperficial UI120 will allow to classify to the information comprised in UI120, search for, filter or the manipulation of other modes.Such as, use metadata " team " 235 can allow the classification to whole tasks, event or other information segments that will be performed by the Team Member be associated with metadata " team ", or the classification to the whole tasks, event or other information segments that are associated with the Team Member be associated with metadata " team ".

Except using the metadata that is employed to allow manipulation list to create except the information in surperficial UI120, the data in surperficial UI are created in other lists that the metadata be employed also can be used for adding or manipulating other users.Such as, according to an embodiment, if first user is created in surperficial UI in her list and is inputted or catch given information segment, and subsequently the metadata item of such as " Sarah " is applied to this information, then " Sarah " can have now the list being filled automatically her and create the information marked in surperficial UI, and makes her also can see the information marked.On the other hand, other users (such as, above Sarah in example) can select to exit this function completely, are selected to the task of particular category or a part for list items, or the date is selected to be associated with this function after a while.Therefore, if the metadata that is associated with another user is applied to this, then can arrange based on other user-selected access or privacy the application of limitation function.

Should be appreciated that almost endless and metadata that is type can be applied to the various information segments be imported in list creation surface.Such as, these metadata items of such as date, time, position, name, address, telephone number, alphanumeric, audio frequency, video etc. can be applied to one or more word, phrase, data, file etc., to allow to edit the information comprised in list creation surface, classify, search for or manipulate in the future.Such as, if metadata type " date " is applied to list and creates in surperficial UI120 the whole dates comprised, then this metadata can be used for marking the date comprised in UI120, filters, classifies or search for allow user the data comprised in UI120 based on the date.Such as, user may expect to classify to the full detail comprised in UI120 by date, checks those tasks or event of occurring or should perform in today to allow this user rapidly.

Automatically can apply the metadata of some type.Such as, date and time information can automatically be applied based on available informations such as the calendars be such as associated with task or list items.For another example, positional information, the positional information such as applied explicitly with gps system, task or list items can be applied to, this task or list items are used in the prompting generated when user is positioned at (such as, near the supermarket be associated with groceries list) near the position that is associated user.

Except metadata being applied to one or more word, phrase or other information segments, natural language processing to can be used for marking in list creation surface the information that comprises and/or metadata is applied to the information comprised in list creation surface.Such as, if the such as phrase of " Friday afternoon 6 meets in the Pizza shop of Bob " is imported in list creation surface user interface 120, whether natural language processor can be applied to this phrase and be associated with specific information or data type to any one determining in each word to resolve word.Such as, the combination of each word in example task item or multiple word can be resolved, to determine whether to comprise any specific information type.Such as, word " the Pizza shop of Bob " can be marked as enterprise name, and the time " 6 pm " can be marked as the time, and sky " Friday " can be marked as specific one day.These phrases can be resolved to one or more word by natural language processor, and the one or more word can be used for dictionary or the storage of search terms, mate with each known word (such as dining room title, time, sky etc.) for by the word through resolving.Once natural language processor by particular words or phrase identification for belonging to information type, available metadata marks those words or phrase, these words or phrase is made to can be used for search, classification, filter, edit or otherwise manipulate information, as mentioned above.

Except to except this manipulation of listed information, allow to carry out operation no less important to listed information.Such as, function button and control can be illustrated in list and create in surperficial UI, to allow the action for listed item.Such as, list contacts (such as " the Pizza shop of Bob ") and can cause listing " calling " button making the contact person listed by telephone program calling when checked or " Email ", " text " or the similar button etc. that can allow to send to contact person Email or text message when checked.According to each embodiment, this telephone program can be visited by personal number.That is, if user wants called out this personal number by navigation the various menu and heard his/her whole record, then he/her can do like this.This function may be useful for security purpose, and such as, if user cannot use his/her host computer device, then he/her can access the task and list items that have recorded.Should be appreciated that can be the action control of listed every displaying other types many.Such as, can show for the control etc. to the name listed by contacts folder interpolation and relevant information.

According to each embodiment, other actions be allowed to be associated with task or list items comprise issues those via social networks (as FACEBOOK) to other people.Therefore, can issue via one or more suitable order (comprising voice command) or via the application of metadata the task or list items that catch via one or more means (comprising speech capture) to each other user, as described herein.On the other hand, user can input private task or list items, comprises the application to encryption, password, biometric identification etc., makes to only have the user inputted or other users specified to access those.

Except natural language processing, the additive method identifying and utilize customizing messages fragment can be used.Such as, additive method can comprise resolves text or data, and resolved text or data are passed to one or more identifier module.Additive method also can comprise all data that usage data analysis comes on Analysis server, and illustrate automatically complete or other information (such as, inputting everyone of " Christmas Day " also uses " # vacation " to mark it, perhaps user by chance, you, also want to do like this).In addition, can use search, such as, input " the Pizza shop of Bob " is by by carrying out searching for and seeing that " the Pizza shop of Bob " is actually the URL(such as www.bobspizzaparlor.com having and be associated) dining room, cause detection/mark.

According to each embodiment, except metadata tag, other list items attributes comprising other forms of metadata can be applied to the list items be input in list creation surface.Such as, the list items attribute such as such as team's attribute, personal attribute, date property, time attribute, position attribution, name attribute, address properties, phone number property, alphanumeric attribute, audio attribute, video attribute can be applied to given list items.Should be appreciated that list items attribute can be extendible and customizable, such as, the position etc. of the thing on price attribute, shopping list in shop.Such as, comment column list item " team 1 meet carry out item for discussion in 2 pm " extraly can be carried out, to create modified list items " team 1 to meet in the office of Joe carry out item for discussion in 2 pm " with list items attribute individual's name (such as " Joe ").List creation surface the first metadata item " team 1 " can be associated to generate task with list items attribute " 2 pm " and/or " office of Joe " subsequently, for be " team 1 " member whole users listuser interface 120 in show.These metadata items can allow more effectively to use this task with associating of list items attribute.Such as, the task items of gained can allow list to create the information of surface retrieval about the member of " team 1 ", such as calendar information, to determine that whether member has time in " 2 pm ", and/or list creation surface can searching linkman information to determine the position of " office of Joe ".This information can be used as the item perceived from the task of gained and is automatically added to list creation surface 120.Should be appreciated that these are that metadata item can be associated with other list items attributes to strengthen the multimode example perhaps that the validity of the task items in surface is created in list.

As mentioned above, be imported into list to create in surperficial UI120 and mark according to one or more metadata type, list items attribute or divide into groups or to be labeled explicitly with natural language processor or the information of dividing into groups can utilize according to various helpful mode (comprising generation and the display of gained task) subsequently.Such as, being applied to task, event, date of activity or other information segments (hereinafter referred to as " task ") and/or time annotation or mark can be used for according to any applied metadata or list items attribute (such as, date/time, people, team etc.) manipulate the task and relevant information that comprise in (such as, edit, classify, search for or otherwise manipulate) list creation surface.In addition, the task by date and/or tempon data notes can be organized in event time line, and this task can be annotated to help user to complete or otherwise Processing tasks along official hour line further.As mentioned herein, timeline can be broadly defined as to comprise and represents any time, comprises date, time, calendar information, season, time etc.Such as, specific task may have prescribed time-limit, and such as, user may the appointment of the irremovable doctor in specific date and time.

Other tasks may need during the date/time scope of regulation (certain time such as, before Friday afternoon 6) to complete or process.According to various embodiments of the present invention, this date and/or timing information can be applied to being input in list creation surface user interface 120 of task, " ambiguity " to be applied to the timing aspect of comprising in user interface of task.Such as, if two tasks must complete the very specific moment of given one day or otherwise process, then available metadata annotates those tasks, this metadata allow user based on applied fixed date/time classifies, searches for or otherwise manipulate those.On the other hand, if other tasks one or more must complete on the same day, but can complete in any time that the given end time (such as 6 pm) is front, then can annotate those tasks by a metadata type, this metadata type allows those to move in the event time line be associated with the task that must complete on this regulation date or otherwise process, as long as the time completing or otherwise process those is no more than the last time restriction (such as 6 pm) of regulation.For another example, if user wants prune his/her lawn in the morning and go to a specific dining room at night, then list creation surface can allow the moment catching picture " morning " and " evening " in the mode identical with using papery calendar, pruning activity can be placed on the top of the frame of that day by them, and dining room title being placed on the bottom of the frame of that day, any one does not all have the specific time.According to an embodiment, subsequently, list is created the classification of the list items comprised in surperficial UI120, search or manipulation otherwise can for have fixed date/list items of time come, or can to have come for the list items with loose or fuzzy date/time or its combination has come.In order to further describe fixed date/time and loose or fuzzy date/time, see on June 7th, 2006 submit to be entitled as " EnteringandUsingTimeRanges(input and service time scope) " U.S. Patent No 11/422,837, and the U.S. Patent No 11/625 being entitled as " FlexibleElectronicCalendarIntegratingTasksandAppointment s(is electronic calendar integration servers and appointment flexibly) " that on January 23rd, 2007 submits to, 914, the two by reference entirety be incorporated into this.

List is created surperficial UI120 and can be provided to remind to complete or otherwise process list to user and create comprising in surperficial UI120 of task, and comprise with fixed date/metadata that time correlation joins and the metadata that is associated with loose or fuzzy date/time can be used for providing to user experiencing more really.Such as, to the date/time of regulation on the horizon (such as, specific appointment) prompting can be the prompting of a type, and the various promptings be associated with loose or fuzzy date/time, with with fixed date/prompting that joins of time correlation compared with, may be so not urgent a kind of dissimilar prompting.In addition, being associated with loose or fuzzy date/time of task automatically can be floated in the arrangement of given a day, until close to the end point (being such as not later than 6 pm) specified.Therefore, compared with loose or fuzzy date/time item, how can come by writing down every himself or herself these mode of reminding on the little scraps of paper, notepad, note etc. according to being closer similar to user with the prompting that is associated of date/time item of regulation.

As described in more detail below, can dispose list explicitly with multiple software application and data and create surface user interface 120, these data are associated from different software application types.Such as, list creation surface can be used for receiving information or captured information from various e-file, and e-file is word processing file, spreadsheet application document, slide presentation application lantern slide, explorer content, social media site contents, Video Applications, voice applications, electronic ink (such as writing electronically with stylus and electronic writing board), photo, e-mail item, calendar item, task items, speech-to-text file etc. from other tasks such as.In addition, for the information of list creation surface storage can be used for strengthening the function that surface is created in list by other.

Consider with the example task being transfused to or catching of phrase " Friday afternoon 6 meets in the Pizza shop of Bob " form.As mentioned above, once resolve each word in entry or phrase, identify, annotate or otherwise use metadata token, those items be labeled just can be used for strengthening the function that surface is created in list by other.Such as, enterprise name " the Pizza shop of Bob " can be passed to a software application, to determine the position in the Pizza shop of Bob.The position in the Pizza shop of Bob so can be delivered to user overall equipment (such as, phone, personal digital assistant etc.) GPS (GPS) mechanism, the time be associated with entry " Friday afternoon 6 point " can be delivered to the calendar function that user uses, and therefore can utilize any other word interested or phrase in phrase.This example of present continuation, if user leaves his or her office and using him to deploy mobile device that surperficial UI120 is created in list, as will be described below, when user is close to the position in example " the Pizza shop of Bob ", or when user is close to fixed time " Friday afternoon 6 point ", or during the two combination, prompting can be provided to this user via his mobile device, close in the time that " the Pizza shop of Bob " meets, or the position in " the Pizza shop of Bob " is close, or combination above.The use of presence data (such as being located the position of determined mobile device by signal intensity or GSP) also be can be used for list to create information in surface or task reminders relevant to other relevant informations (such as calendar item, conference location etc.).In addition, be associated if meeting is reminded with the people of specific people or particular group, then list creation surface can be applied with the associated person information obtaining this people or multiple people by inquiring contactor, and makes this information create surperficial UI120 by list can to use.

For another example, if user's input and the task of editing the specific part of given word processing file and being associated, then the metadata be associated with the mark of this particular document can be applied to and be input to list and create task in surface user interface 120.When next user opens the word processing file of specifying, remind can appear at list and create in surperficial UI120 and should be edited with the particular segment in reminding user word processing file.According to an embodiment, if this document is not yet opened, if user sees the prompting of the document that editor identifies, then creates by selective listing the document identified in surperficial UI120 and directly open the document from UI120.

Still with reference to figure 2, can show except above with reference to those function buttons described by figure 1 and the one or more function button except control and control in user interface 120.Prompting function 210 can allow user in UI120, mark given task or item of information, with arrange expectation prompt date/time.Private informing function 215 can allow user Given task or item of information to be labeled as " individual's ", makes this task or item of information can not show other users by the list creation surface user interface of other users.Should be appreciated that function 210 and 215 just can create in surperficial UI120 the multi-functional example perhaps of showing useful metadata or function to be applied to task or the item of information listed in UI120 in list.

Fig. 3 is the simplified block diagram of the list creation surface user interface 120 of Fig. 2 that the information list perceived from selected task items is shown.Should be appreciated that any individual task for being imported in list creation surface or other information segments, one or more subtask, subevent or sub-item of information can be transfused to and be associated with any inputting before of task.Such as, with reference to figure 3, various task or other information segments 315,320 are inputted explicitly with father's task " redesign product " 225 according to an embodiment, allow to start pivot table 310, to show the subtask 315,320 be associated with father's task 225 in list creation surface user interface 120 to the selection of father's task 225.If the available size of user interface 120 only provides the initial display of the task to fixed qty, event, activity or other information segments, then illustrate that " more " buttons 330 are to allow to show extra subtask below selected father's task 225.According to an embodiment, the shown subtask (such as 5 subtasks) of the maximum quantity of specifying can be shown, the focus of user to be remained in " front " several vital tasks.Should be appreciated that the subtask information can filtering, search for, classify or otherwise manipulate display in pivot table 310 as the situation of the information comprised in main user interface 120.

Fig. 4 is the simplified block diagram can implementing the computing architecture of various embodiments of the present invention wherein.With reference to figure 4, desk-top or laptop computer 405 describes user for inputting, catching or otherwise utilize any fixing computing equipment creating the data that surface is associated with list described herein.Mobile device 410 illustrates that mobile phone, personal digital assistant, the laptop computer of wireless connections, board-like or tablet computing device or user can utilize list to create any other computing equipment on surface with it in mobile environment.Distributed computing network 415 describes any applicable means allowing computing equipment 405,410 via remote server 420 and one or more application or database communication, such as, and the Internet, corporate intranet, Intranet etc. based on family.

Server 420 describes the universal computing device operated as remote server, can safeguard that the function on list creation surface follows user from an equipment 405 to another equipment 405 to mobile device 410 to allow this list to create surface thereon, or create surperficial UI120 for any other equipment used to list can be disposed as described here thereon.According to an embodiment, create the repertoire that is associated of surface and data with list to store and the user interface 120 that is associated can take list to create the form of surface applications or module 100, the computer executable instructions that surface applications or module have the abundance for performing function described herein is created in this list.List creation surface applications or module 100 and/or (described below) voice record and process application 101 can reside on single computing equipment 405 or 410, use explicitly for the data can accessed with equipment 405 and 410.Or, function surface can being created in the maintenance and operation list of remote server 420 place and the data be associated and the user interface 120 be associated thereof, as shown in Figure 4.

List 425,1015 describes the Database Lists or form of locally to access by equipment 405 or 410 or access via server 420, can safeguard manually or automatically be input in list creation surface and the information shown via the user interface 420 be associated at server 420 place.Should be appreciated that if user not in a distributed computing environment, then list 425,1015 and the storage data that are associated can be stored or be cached on local computing device 405,410.That is, according to an embodiment, each instantiation on list creation surface can cause generating the lists table 425,1015 safeguarded in a database, and this database is stored in the local on computing equipment 405,410 or with server 420 and stores explicitly.

In list 425,1015, each task, event, activity or other information segments can be assigned to and be stored in the given row of list 425,1015.Except storing each respective entry, mark is applied to the information (such as, metadata) of the annotation of each entry or other identification informations can be stored in list 425,1015 together with the data entries be associated.In addition, if information and other data are (such as, document, calendar item, Email entry) be associated, if or entry and other information are (such as, global position system location data, date/time data etc.) be associated, be then linked to different lists 425,1015 together on the row that the information identifying these associations also can be stored in list 425,1015 or from by each task, event, activity or other information segments be associated manually or being automatically input to that list creates in surface.When adding new data to list creation surface user interface, or when changing the data in list creation surface, or when the data comprised in list creation surface are associated with other information, update stored in the data in list 425,1015.According to alternative embodiment, the data that can memory row list item 425,1015 carried out according to the various different means except above-described database row and be associated.Such as, the extend markup language (XML) that list items and the data be associated can be stored as across list creation surface available or addressable multiple lists of links, form etc. represents or similar expression.

Still with reference to figure 4, the various information sources that list creation surface is available are shown.Such as, from the Information Availability of contact application or database 430 in obtaining the information be input in list creation surface.Information can be obtained similarly from calendar application 435 and the data be associated store.As will be described below, surperficial in filling list creation with the Information Availability of the content 440 be associated from e-mail applications.From various document 445, the Information Availability of such as word processing file, slide presentation documents, spreadsheet application document etc. is in surperficial to list creation by data stuffing.Ink application 450 describes the electronic pen that allows data to input for the contact such as by stylus and electronic writing board and ink is applied.Photo application/storage 455 describes photo and obtains by it and any application or the data storage that are copied to or are moved to list creation surface.Audio/video application and store 460 and describe one or more means for obtaining audio or video file, such as, the recording mechanism that operated by numeral or analog recording equipment or camera (such as by mobile phone etc. available).Content for list creation surface also can from explorer, social media website or other sources 465.Should be appreciated that from the data of any other useful source for moving or copy or otherwise input data electronically and Information Availability in use task, event or other information interested to the user interface 120 of filling list creation surface and be associated.

Although each data shown in Fig. 4 or information are illustrated as being associated with server 420, in these data and/or information source each can also with local computing device 405,410 directly related and/or be stored directly in local computing device 405,410.In addition, according to each embodiment, information creates surface from one or more source to list be not one-way communication.That is, according to each embodiment, surface and/or each task list or task list item are created in list can be linked to source, obtains task list item, and the information from task list can be back into this source from this source (such as, word processing file).Such as, if the information segment of task items form is arranged in LASUI120, then this information can be pushed back to it from source.Should be appreciated that various mechanism can be used for information to back into source.Path to source can be associated to each corresponding task list item.Can cause for selected item being back into source and/or being used for starting the selectable buttons of source document or control is illustrated in LASUI120 the selection of task list item.

Fig. 5 illustrates that the simplified block diagram of surface information input module 510 is created in list creation surface interface 120 and the list be associated with shown document.As mentioned above, information manually or is automatically input in list creation surface user interface 120 by information capture described below.As shown in Figure 5, example document 530 is illustrated to apply with exemplary textual process and is presented at explicitly in computer monitor display screens 500.According to various embodiments of the present invention, the extended version of list composition service user interface 120 can as above with reference to shown in figure 2 and 3 and described disposing, and comprises the information of any expectation about shown document 530 or information associated therewith for input.According to another embodiment, surface user interface 120 can be created with list and dispose list explicitly and create surface information input module 510, create the data of Given task in surperficial UI120 or item of information for input and annotation about list.List creation surface information input module 510 can be deployed for authored documents 530 application (such as, text processing application) a part, or input module 510 can be deployed the independent input neighboringly started with document and apply, for the data inputting and annotate about Given task or list items.

Example according to Fig. 5, list creation surface information input module 510 starts explicitly with task " redesign product ", and creates in surperficial UI120 in list and show.List creation surface information input module 510 comprises title division, for providing that data are just being transfused to mark, creates the information that Given task, event or other information segments that surface comprises be associated with list.Such as, " prepare manufacturer to advise " list shown in marked graph 5 explicitly with task and create surface information input module 510, this task is the subtask of father's task " redesign product ".The title division of list creation surface information input module 510 is comment part 520 below, for allowing user to input comment, comment on can be associated with example subtask extra task, event, activity or other information.Such as, in comment part 520, the comment of input can be the subtask that subtask " prepares manufacturer's suggestion ", or the comment of input in comment part 520 can be exactly the comment of the various aspects of the subtask that reminding user is associated.

Be provided for the content part 525 allowing other guide item (such as, document, audio file, video file or other guide type) to be associated with example task or subtask.Illustrate that in the bottom of list creation surface information input module 510 " people " for one or more people, many group people or team being associated with subtask partly such as, as described and shown with reference to figure 2 above, the team that can use explicitly with metadata tag " team " 235 divides into groups to be applied to given task or subtask.Other groupings or each individual can similarly be input to list and create one or more task in surperficial 100 and the UI120 that is associated or subtask is associated.Be to be understood that, configuration shown in list creation surface information input module 510, layout and field just for exemplary purposes, and not to creating other texts, Data Entry or the data notes field that provide in surface information input module 510 or part limits in list.

Still with reference to figure 5, on display screen 500, the document 530 of display describes any document, the word processing file, electronic form document, slide presentation documents, notepad documents, job file, calendar document etc. that such as can show on display screen 500.As shown in Figure 5, document 530 is just processed in some way by user, and user determines to create the information of surface 100 input about shown document via list creation surface information input module 510 to list.Such as, when user is editing shown document, user can remember one or more tasks that should perform and be associated with the project of reference in shown document.Therefore, by starting list creation surface user interface 120 and starting list creation surface information input module 510 subsequently, user can insert task, comment, content item, or by the part of document or document or to the task that document is associated and one or more people, how organize people or many groups of people carry out relevant, just as user may on the little scraps of paper or note these p.m.entrys hand-written or annotate to remind this user next to process those things.According to the embodiment that is replaced, when document 530 is opened, task or other information are input in LASUI120 and may cause being imported into task in UI120 or other information and be automatically associated with the document (that is, represent the metadata of the document can be applied to the task of inputting or other information).

According to an embodiment, creation or be stored remotely in list creation surface in otherwise creating or catch list creation surface that task list or other information can be stored on local computing device 405,410 or by server 420 in other creation application (such as e-mail applications and notepad appli-cation).As described herein, these task lists or other information can have been inputted or captured the information that his/her list creates in surface subsequently with user and carry out synchronous, and can be displayed in LASUI120.Such as, LASUI120 can be deployed in as other application such as explorer, e-mail, text processing application, notes application, calendar application inner or associated with it dispose, and input one of to be applied with those list items be associated and can be filled LASUI120 and be associated with the LASUI120 of other users.Such as, user can generate user to key in the bulleted list that will take dinner party in the e-mail item of inviting other people to participate in dinner party, and those lists that automatically can appear at user are subsequently created in surperficial UI120.If the LASUI synchronised of the LASUI120 of user and other users, then when other people more new e-mail time he/her can see subsequently other people be with what to party on.

In addition, can provide auto-complete function, other that wherein can input in the past based on this user or other users or catch complete new item and sublist automatically.Such as, when for travelling packing, if user keys in " packing ", then can automatically advise the whole subitems of user's past for " packing " for his/her list, like this, he/her just need not again key in or again remember them.Further, to him/her, the LASUI120 by active user shows that the packing item of other people (such as, the social networks of user) that surface is associated with active user is created in its list.This can use search to process the many existing task of active user or other users be associated.

According to an embodiment, list creation surface automatically can hide some or information, makes list can not be oversize.Such as, when in the list that a list items has been positioned at user more than two weeks or when exceeding at two weeks due date, it can automatically be hidden, and the thing that user not yet can not be completed him/her or the thing that he/her has not intended feel uneasy or evil.Or, to it, automatically expiry date can be set when being added for every, make after expiry date has been crossed that those are hidden from display.

As above with reference to shown in figure 4 and described, list creation surface 100 and the user interface 120 be associated thereof can use in fixing computing system 405, or list can be utilized explicitly to create surface with one or more mobile device 410.Advantageously, as described herein, the information that the list be stored in the list 425,1025 be associated with server 420 is created in surface can be disposed across various should being used for, and when user advances in the mobile device deploy of user.Therefore, in fact, user interface 120 that is surperficial and that be associated is created in mobile computing device (such as mobile phone, personal digital assistant, board-like or flat type computing equipment, the wireless game device etc.) deploy of user by making list in list creation surface, allows user to carry " to be done " list of electronic version when he or she leaves desktop operating environment.

According to each embodiment, listuser interface can import to fixing computing equipment 405 and mobile computing device 410 from remote server 420.When showing task in listuser interface, the instantiation at this listuser interface can be shown on the computing equipment fixed and on mobile computing device.When changing the task in the listuser interface at remote server place, pass to fixing and mobile device by changing with the form of the new instantiation at the listuser interface shown on fixing computing equipment and on mobile computing device.In addition, when changing creating in surperficial UI120 of the task of list of fixing or mobile computing device place, the list creation surface that these changes can be passed up to remote server 420 place and the data be associated store.

Fig. 6 and 7 illustrates the user interface using list to create surface and be associated in mobile environment.As shown in Figure 6, list creation surface user interface 620 describes as above, creates surperficial UI120 in the list of the mobile version of display screen 615 deploy of mobile phone 410.Just as above creating surface user interface 120 in the display screen deploy list of his or her computing machine or laptop computer referring to figs. 1 to the user described in Fig. 5, user also can dispose list creation surface user interface 620 on his or her mobile device, with the identical function utilized with can utilize in fixing computing environment.

With reference to figure 7, if user starts the list creation surface user interface 620 shown in Fig. 6, then can start " to be done " list on the display screen of the mobile device of user, to allow user to check one or more task, event, activity or other information, or allow the information outside user's amount of imports, editor's existing information or otherwise manipulate existing information.If user edits via his or her mobile device really or otherwise manipulate the information comprised in list creation surface user interface, then modified information is stored in list 425,1025 by server 420, and user is when his or her fixing computing equipment deploy list creation surface user interface 120 next time, those being carried out the information comprised wherein by the mobile device of user are changed or amendment will appear in the user interface 120 of disposing relative to other application one or more in the fixing computing environment of user.

In addition, mobile device 410 can be used for Quick Catch can by directly be exported to list creation surface information.Such as, the camera function of mobile phone can be used for taking the photo that can be automatically imported list creation surface.For another example, from mobile device 410 or from there is any mobile computing device of GPS (GPS) or other position determination systems (such as, laptop computer) gps data can be captured relative to specific position or address, and can be directed to list creation surface.

As above with reference to described by figure 1-5, task items and list items can be entered manually or otherwise be captured to be input to list creation surface, and this list creation surface can to show across multiple wired, wireless, fixing and mobile computing device (comprising the mobile computing device 410 shown in Fig. 6 and 7) across multiple application.According to various embodiments of the present invention, as described herein, task and list items can catch via voice or audio frequency input, to show in the task user interface on capture device (such as mobile phone) or on any other wired or wireless computing equipment or list creation surface user interface.

Such as, when user drives, when user advances, task or list items can catch via voice or audio frequency input, and can catch long list items and not need the sight line of user to remove from main focus item.The available minimum touch to capture device (such as mobile phone) and/or the seizure performed alternately this information, and can perform and need not suspend between each task of input or list items the seizure of information.In addition, about the decision of caught interested item or action can be postponed to check caught information concerning user more suitably one catch after the time.Such as, user may need for office on the horizon luncheon goes to local general store to buy various thing.According to various embodiments of the present invention, user can check his/her refrigerator or cupboard, and can by the thing list records required for the luncheon of example office on his/her mobile computing device (such as mobile phone).After the thing list required for record, the list of recording locally or remotely automatically can be transcribed on recording unit, and the list of transcribing can be automatically populated in task list or list creation surface, checks subsequently for user.In addition, according to each embodiment, if need, the audio file of the input of recording can be attached to or otherwise be associated with transcribe and be shown list or task items, input to allow this audio frequency of user's playback.

In addition, transcribed audio frequency input (phonetic entry of such as user record) can be processed according to natural language processing, and metadata can with to transcribe and processed voice or audio frequency input are associated, to provide the extra enhancing to caught input.Such as, according to natural language processing, the phonetic entry caught " obtains groceries thing after 5 in the afternoon " and can bring the metadata be associated with textual portions " point in afternoon 5 ", generate the electronic calendar that can use with user and apply to allow for caught input the task items be associated, and can be this task items and generate and automatically remind after reminding user in the afternoon, to obtain the groceries thing expected at 5.According to each embodiment, other sensors or detection means can be used for marking task and list items by metadata.Such as, GPS location system is used in when task or list items are recorded and marks task or list items by position data, and this position data is associated with this task or list items.For another example, clock, optical sensor, camera function etc. can be used for one or more item of context information to be associated with caught task or list items.

Once perform seizure, transcribe and processed voice and/or other audio frequency input, the task of gained or list items just can be stored in the local on capture device (such as mobile phone), maybe can be stored remotely on remote server 420, to use explicitly with other computing equipments, such as, list for being shown on the wired desktop computer 405 of user by distributed computing network 415 is created in surface and is used, as described above with reference to Figure 4.Namely, as described herein, according to recorded voice and/or other audio frequency input captures, transcribe and can use across various software application across various computing equipment according to the identical mode created for list described by surface and the content that is associated (above with reference to described by Fig. 1-7) with processing of task and/or list items.

With reference now to Fig. 8-13, according to each embodiment, voice/audio record and process application (VRPA) 101 can locally operate or can be accessed from remote server 420 by local device 410, to obtain record described herein, to transcribe and processing capacity on equipment 410.According to each embodiment, after equipment 410 catches voice or other audio recordings, this record can be stored in the local in equipment 410, or this record can be passed to remote server 420, to store as required and to be retrieved subsequently by equipment 410.

According to each embodiment, after catching given audio recording, if user thinks first, to be recorded in certain aspect be mistake or incorrect, then VRPA101 can allow again to record this audio recording.This record again can entirety carry out, or again can record multistage or the multirow of caught audio frequency.Such as, the record that VRPA can allow playback to catch, and the multiple parts again recording this record can be allowed maybe can to allow to insert extra audio recording.Once be recorded, these any other input items (such as, the item of key entry) created in surface or other suitable lists just as list show like that.That is, can resequence to them, classify and edit according to inputting identical mode with other.And the metadata being employed this input can input the identical mode of metadata with other and edits and utilize according to as described below.

Transcribing application can by VRPA102 locally at equipment 410 or remotely operate at server 420 place, automatically to transcribe caught voice and/or the input of other audio frequency.Namely, as above with reference to described by figure 1-7, after the given voice of reception and/or audio frequency input, input can be passed to Local or Remote and transcribe application so that input is transcribed into the Text Input be associated, the text input can be used for fill task list or list creation surface list, this task list or list creation surface list can be displayed on equipment 410 maybe can be displayed on other Practical computer teaching one or more display screen on.Transcribing and can be stored in the local in equipment 410 caught voice and/or audio frequency input, or transcribe and can be stored remotely at server 420, to be distributed to other computing equipments one or more that user uses as task or list items.

Or transcribing application can by VRPA101 locally at equipment 410 with remotely in this two operation of server 420.Such as, immediately, fast transcription process can locally complete, and more comprehensively and may process more accurately can strengthen processing speed and function may resident server 420 place complete.In addition, replace operating in server 420 transcribes application, application may operate on another equipment, such as, and on knee, board-like, the dull and stereotyped or desk-top computer be associated with equipment 410.

Fig. 8 illustrates that its seizure voice available and/or audio content are to generate the simplified block diagram of the mobile computing device of task or the list items be associated.That is, equipment can be used for catching user's voice content of saying, or equipment can be used for catching other audio contents, such as music, the record etc. play.In addition, equipment 410 can be used for catching the video content with the audio content be associated.The audio-frequency unit of the video content caught can be used for generating and annotating task described herein and list items.Computing equipment 410 shown in Fig. 8 describes any equipment, can catch voice or other audio contents with this equipment, to generate one or more task or list items for subsequently, as described herein.According to each embodiment, equipment 410 can comprise mobile phone, board-like and flat type computing machine, Handheld computing device, laptop computer, or equipment 410 can comprise and is coupled to the one or more wired of audio capture device (such as microphone) or fixing computing equipment, to catch voice or the input of other audio frequency.In addition, equipment 410 can comprise and present the computing equipment be associated to the display of content at wall on the surface, wherein this computing equipment is equipped with pen recorder, such as, for receiving the microphone of voice or the input of other audio frequency or being operably connected to computing equipment 410 or telephone plant associated therewith.

With reference to the equipment 410 shown in figure 8, exemplary mobile phone is shown to have the display screen 805 for displaying contents, this content such as telephone message, speech message, email message, text message and according to various embodiments of the present invention generate and display one or more task or list items.Computing equipment 410 can comprise the one or more devices for catching voice or other audio-frequency informations.Such as, the microphone 820 that can comprise for catching one or more voice and/or audio frequency input can be provided for voice that playback catches and/or audio frequency input or play the loudspeaker 825 of any other audio content provided by equipment 410.Camera 830 describes for catching photographs and video for the camera used explicitly with caught voice and/or audio frequency input.

According to each embodiment, the beginning of the beginning of voice or audio recording or photography or video content seizure has come by one or more suitable device.Can be provided for activating sound-track engraving apparatus or the hardware button 815 for activating camera according to the design of equipment 410.Such as, even if when equipment 410 is locked, hardware button 815 also can be used for catching voice and audio frequency.Voice or audio capture fast can be allowed to this use of hardware button 815.Such as, for button 815 pin action after, voice or audio capture can be completed.Pin the mutual use of type allow to avoid unexpected voice/audio record to this.According to an embodiment, Quick Catch audio frequency or voice content can be allowed to this use of hardware button 815, and other functions of the equipment 410 not requiring user to unlock or access to be associated.

In addition, the various user interface button touching activation allowing to activate record or the camera function catching voice, audio frequency or photographic content can be provided for, such as record button 810.Except hardware button 815 or user interface button 810, activate and control record or camera function by voice command.Such as, such as the order of " start record " etc. can be picked up to cause by microphone 820 beginning of audio recording.Equally, such as the voice command of " activation camera " etc. can be used for the camera function of activated equipment 410.

With reference to figure 9, once start the record of given voice or audio frequency input, user interface experience can be provided on the display of equipment 410, be recorded in ongoing vision instruction 910 and the selectable control 905 being provided for stopping record or continuation record to provide.Such as, button 915 can be used for recording the ensuing task after last item or list items " to record another ".

According to each embodiment, VRPA101 can utilize the grammer set through optimizing, to assist in response to caught phonetic entry to generate task items or list items.Such as, such as the word of " beginning ", " stopping ", " next one ", " beginning ", " end ", " task " etc. can be used for converting caught voice and/or audio frequency input to each task and/or list items.Such as, if user says groceries list " egg, next; bread; the next one, milk ", then the word " next one " be inserted between other three words can be used for other three words to be separated into three list items " egg ", " bread " and " milk " by VRPA101.

Equally, such as the sounding of the word of " task " can be used for indicating VRPA101 to follow word, term or phrase after word " task " to can be used as one or more task items to receive.Such as, once have recorded phrase " task-prepared project document before Tu. ", then word " prepared project document " and can be interpreted as the task of following after the sounding of word " task " before Tu..Should be appreciated that the process that multiple grammatical terms through optimization or phrase can be used for seizure voice and/or other audio frequency are inputted by VRPA101 is more efficient.

In addition, as below with reference to shown in figure 11, can be provided for allowing inputting explicitly with voice and/or other audio frequency or remove it with the manual incoming task in other places or list items and the hardware of metadata item be associated or user's interface keyboard 1105.Such as, user can record one or more list items, such as, from the item of groceries list, then manually inputs one or more item additionally when user-accessible hardware or user's interface keyboard.According to an embodiment, user can move to manual input from voice record as required to and fro glibly.Such as, user can be in the process recording one or more task or list items, and being next the manual input causing amount of imports extreme term except automatically stopping except writing function as required, is next extra voice and/or audio recording.

In addition, audio file (such as, comprising the .wav file of caught voice and/or audio frequency input) can locally or remotely be stored equally, and can be associated with transcribed voice and/or audio frequency input, for playback as required.Such as, to transcribing of recorded list items, such as " remember for Sarah buys birthday card and birthday cake ", can manually or automatically be shortened to be input in list creation surface list or task, to become such as task or the list items of " birthday cake and greeting card ".But the audio file comprising the input of whole audio frequency can be associated with stored task or list items.Therefore, user is playback of audio files optionally, to receive the whole audio frequency input of task for generating shortening or list items.Such as, after receiving the transcribing of given audio frequency input, the input of transcribed audio frequency manually can be shortened to the shorter phrase that can be used as task or list items by user, but user can expect playback subsequently, and whole audio frequency input thinks that this user provides the context of task or list items.

In addition, as described below, for use natural language processing and metadata application, the keyword from caught voice and/or audio frequency input can be used for generating the task or list items that are presented by equipment 410 or other equipment, as above referring to figs. 1 to described by 7.Such as, if use the keyword of such as " next one " etc. in audio frequency, then caught text can be divided into independent subtask, and the audio file be associated can be pruned only to have the audio frequency of the actual subtask be associated with subtask.In addition, can be used for catching and abandoning or otherwise dispose incorrect or wrong audio recording to the use of this natural language processing and metadata application.

After given voice and/or audio frequency input are captured and transcribe, the content transcribed can process according to natural language processing, as mentioned above, wherein content can be divided into each word, wherein each word can with each source (such as, dictionary, contacts list, calendar item etc.) compare, the metadata be associated with the term obtained with comprise transcribed content, word or phrase.This metadata can be used for generating to the application of content the task or list items that shorten or for transcribed content being associated with other application functions, can show task or the list items of this shortening for transcribed content.Such as, aforementioned phrase " is remembered for Sarah buys birthday card and birthday cake " to be resolved by natural language resolver, and keyword " is remembered ", " birthday ", " greeting card " and " cake " can be used for generating the task items or list items that shorten, such as " remembers birthday card and cake ".

In addition, if the content item of transcribing comprises the information such as such as date, time or other information of identifying by natural language processing, then this Information Availability is in more efficiently and more usefully process the content caught.Such as, if the voice caught and/or audio content comprise phrase " before Tuesday afternoon 5 July 4 ", then this Information Availability is in by identifying date that the content that catches comprises and/or the time generates task reminders or calendar reminding.Therefore, this meta-data extraction and utilization can be used for according to one or more application function (such as, task reminders, calendar reminding etc.) creation task or list items from caught voice and/or audio frequency input item.

With reference to Figure 10, in the user interface (the list creation surface such as, shown in device user interface) of equipment 410, present the bulleted list 1015 from caught voice record transcription.As mentioned above, sample list " egg ", " bread " and " milk " can be obtained from the groceries list that user records.Be provided for allowing user optionally by presented every " being grouped into list " button 1020 being grouped into the grouping of expectation.Such as, can in response to the list record of 14 different item being presented to 14 groceries items to user.A subset of 5 in groceries item can be associated with specific menu item (such as cake).If needed, subset of items can be grouped into the specific cluster be associated with illustrated menu item by user.Should be appreciated that and can store and access any list items or task items (comprising the grouping of list items and task items) according to the identical mode creating surface list 425 above with reference to the list described by Fig. 4.Or, replace using " being grouped into list " function 1020, user can via any suitable input media described herein input header above list, such as, " birthday cake of Julie ", then indentation list items below title, list items can by the subtask automatically arranging and be stored as below father's task " birthday cake of Julie ".

Still with reference to Figure 10, show the audio frequency icon 1010 on each shown task or list items side, this icon is for notifying that this task of user or list items allow from audio recording the original audio record that user's playback is associated with presented task or list items.Such as, as mentioned above, Given task or the list items of " egg " can be generated such as from long audio file, and user may expect to hear that whole audio file is to listen attentively to the context of the initial logger task of wherein user or list items.In addition, if generating from caught record of task or list items are distributed to other users, other members of such as working team, then other users may expect to hear the audio recording be associated, to obtain for one or more in task or list items being included in understanding in given bulleted list 1015 or project grouping or context.According to each embodiment, if the audio recording that playback is given, then user can add this record to add more information to this record, the task that the record content of adding can be used for generating extra task and/or list items by VRPA101 or generate before being used for revising or list items.

Except being associated with list or task items by audio file, other guide can be associated with given list or task items equally.Such as, the photo taken explicitly with given list or task items can be associated with this list or task items, and therefrom the document of the voice record of read list or task items can be associated with this list or task items, etc.According to each embodiment, the vision to the content be associated that can present according to the mode identical with audio frequency icon 1010 near shown list or task items indicates, to allow access (such as, display and editor's access) this content be associated.

As shown in figure 11, can be provided for changing before to have divided into groups " remove grouping " button of grouping of item or control 1110.Such as, the project team shown in Figure 10 can by " remove grouping " to remove the item as the grouping of specifying, and by every be put back into them before be extracted to be grouped in the general data list of given grouping.

Shown in Figure 11 for manually incoming task or listing information or for correct or otherwise amendment transcribe and the hardware of shown task or list items or user's interface keyboard 1105.Such as, if the word shown in Figure 10 " bread (bread) " is transcribed and be shown as " interrupting (break) ", then keyboard 1105 can be used for editing this entry as required.In addition, voice-operated editor can be used for edit shown by task or list items.Such as, user can input voice command " with word bread replace word interrupt " to make the calibrated word " bread " of VRPA101 replace the word transcribed mistakenly and show " to interrupt ".

As shown in figure 12, can be provided for providing and catch according to voice and/or audio frequency input, transcribe the calendar/prompting interface with the prompting of Processing tasks and list items.As shown in figure 12, the metadata be associated with the word, term or the phrase that extract from caught voice and/or audio frequency input can be used for generating the prompting and/or calendar that are associated with caught input, as mentioned above.Subsequently, various task application and/or calendar application function 1215,1220,1225 can be used explicitly with generated task items and list items, user is such as allowed to warn given list items or task items to other people, the prompting be associated with task or list items is provided, shares task or list items etc. with other users.Be to be understood that, as described herein, according to the function of various different software application described herein and as above referring to figs. 1 to described by 7 shown and described list creation surfaces, the task and/or list items that generate from caught voice and/or audio frequency input can be used.

Such as, as created the list in surface or (with reference to the figure 1-7) described by task items for being input to list above, timeline can be activated and can be used for generating the prompting that is associated with any generated task items or list items and calendar, this timeline comprise be associated with the task items generated according to recorded voice and/or audio content or list items fixed date/time and loose or fuzzy date/time.Namely, the list items generated according to recorded voice and/or the input of other audio frequency by VRPA101 and task items can mark according to one or more metadata type, list items attribute or divide into groups, or to be labeled explicitly with natural language processor as described herein or to divide into groups, and can utilize according to various helpful mode (comprising generation and the display of gained task items and list items) subsequently.Such as, being applied to task, event, date of activity or other information segments (i.e. task) and/or time annotation or mark can be used for according to any applied metadata or list items attribute (such as, date/time, people, team etc.) manipulate (such as, edit, classification, search or otherwise manipulate) list items and task and relevant information.

In addition, the task by date and/or tempon data notes can be organized in event time line, and this task can be annotated to help user to complete or otherwise Processing tasks along official hour line further.As mentioned herein, timeline can be broadly defined as to comprise and represents any time, comprises date, time, calendar information, season, time etc.Such as, specific task may have prescribed time-limit, and such as, user may the appointment of the irremovable doctor in specific date and time.

Other tasks may need during the date/time scope of regulation (certain time such as, before Friday afternoon 6) to complete or process.According to various embodiments of the present invention, this date and/or timing information can be applied to by VRPA101 via the task of voice and/or audio frequency input capture and process or list items, " ambiguity " to be applied to the timing aspect of these task items or list items.Such as, if two tasks must complete the very specific moment of given one day or otherwise process, then available metadata annotates those tasks, this metadata allow user based on applied fixed date/time classifies, searches for or otherwise manipulate those.On the other hand, if other tasks one or more or list items must complete on the same day, but can complete in any time that the given end time (such as 6 pm) is front, then can annotate those tasks or list items by a metadata type, this metadata type allows those to move in the event time line be associated with the task that must complete on the date of this regulation or otherwise process, as long as the time completing or otherwise process those is no more than the last time restriction (such as 6 pm) of regulation.According to an embodiment, subsequently, to the classification of the list items catching according to voice and/or audio frequency input and process or task items, search or manipulation otherwise can for have fixed date/list items of time come, or can to have come for the list items with loose or fuzzy date/time or its combination has come.

As above described by list creation surface 100, voice record and process application (VRPA) 101 can provide to user and remind to complete or otherwise to process the task items or list items that generate according to voice or the input of other audio frequency, and comprise with fixed date/metadata that time correlation joins and the metadata that is associated with loose or fuzzy date/time can be used for providing to user experiencing more really.Such as, to on the horizon fixed date/time (such as, specific appointment) prompting can be the prompting of a type, the various promptings be associated with loose or fuzzy date/time with fixed date/prompting that joins of time correlation compared with can be the prompting of so not urgent a kind of different type.In addition, being associated with loose or fuzzy date/time of task automatically can be floated in the arrangement of given a day, until close to the end point (being such as not later than 6 pm) specified.Therefore, compared with loose or fuzzy date/time item, how can come by writing down every himself or herself these mode of reminding on the little scraps of paper, notepad, note etc. according to being closer similar to user with the prompting that is associated of date/time item of regulation.

According to each embodiment, as mentioned above, the metadata of Given task or list items can create via voice record and apply.Such as, once task or list items are recorded, user just can mobilizing function control (such as button 810) to add metadata to specific task or list items.Such as, after the given task of selection or list items, user's selectable buttons 810, says word " some this morning 12 ", subsequently so that example sky and time are applied to selected task or list items as metadata.If user is just navigating to the calendar shown in Figure 12/prompting interface at present, then user and this interface can provide context to the grammar system of VRPA101 alternately, catch grammar system to allow this VRPA voice/audio and capture the fact that user is saying date and time information (or information etc. of location information or people) more accurately, and therefore create the explanation more accurately of record.

Described by the computing equipment of record audio coming automatic capturing task and list items and the system filled it in electronic tasks or list surface and operation framework via voice or audio frequency input, Figure 13 illustrates the process flow diagram catching voice and/or audio frequency input and fill the method on electronic tasks or list surface by the information obtained from caught voice or audio recording.Routine 1300 is starting to operate 1305 beginnings, and proceed to operation 1310, wherein user start record with generate will at capture device (such as, the mobile phone of user) go up one or more task items that show, that maybe will be shown on another computing equipment by distributed computing system 415 or list items, as described above with reference to Figure 4.Such as, user can utilize his/her mobile phone to record this user in ensuing two weeks processes for task needs the bulleted list of execution.

As mentioned above, user starts recording process by various means, the posture that the record button (i.e. touch screen interaction) that various means such as hardware button, user interface present, voice command activate, received by record computing equipment (such as, posture being moved via posture detection functions such as the motion sensor of such as camera or operation or accelerometers the application passing to and posture can be interpreted as order) and explained.In operation 1315, after user stops recording process, the audio file of the audio recording caught can be stored in the local in recording unit 410, or audio file can be stored remotely at server 420.In operation 1320, the audio file stored can be transcribed into one or more text string, this one or more text string can be used for generating one or more task and/or list items, shows in task list or list creation surface user interface, as implied above.In addition, as mentioned above, during transcription, given audio recording can be split into two or more independent files, and each file can receive metadata application, and each file can be used as independent task or list items.As mentioned above, the audio file of transcribing can locally or remotely be stored.

In operation 1325, the audio file of transcribing can process according to natural language processing, and wherein transcribed audio recording can be divided into one or more term, word or phrase.Each in the term extracted from transcribed audio file, word or phrase can contrast each provenance (such as, dictionary, contacts list, calendar item etc.) compare, whether be associated with one or more information segment (such as, other task items, calendar item, date, time, people, mark etc.) with any one determining in the word, term or the phrase that extract from transcribed audio file.

In operation 1330, extracted term, word or phrase can be applied to, with the text string utilizing this term, word, phrase explicitly or be made up of this term, word or phrase with one or more application with the metadata that in extracted term, word or phrase, any one is associated.Such as, the metadata (such as, being applied to the date or time of phrase " point in afternoon 5 ") being applied to given term can be used for phrase identification being date or time, to be inputted as prompting, task items or calendar by the text string comprising this phrase.As mentioned above, metadata can be applied to task or list items according to various input medium, and metadata can carry out record via phonetic entry, to be associated with given task or list items.

In operation 1335, comprise the term through extracting of generated task or list items, word or phrase to be marked by metadata, so that generated task or list items and other information stored (such as, contacts, calendar, other generating of task or list items etc.) are associated.Such as, metadata can be made to be applied to each item be grouped into together by " being grouped into list " function all list items be grouped into together, with by be grouped into together item in each be associated as and belong to together, until they otherwise by releasing grouping or separately.

In operation 1345, can create in surface in the user interface of capture device or at the task interface be associated with another computing equipment or list and show generated list items or task items, generated list items or task items are distributed to this another computing equipment.In operation 1350, playback can be allowed therefrom to generate the audio file of list items or task items, and if need, add extra audio content to revise audio file by additional records.In operation 1355, user is by thump input or correct shown list items or task items by the correction of voice activation.

In operation 1360, task or list items can be associated with other guide item or action.Such as, task or list items can with map, other tasks or list, each other users or can be associated with the metadata being applied to task or list items any other in perhaps action be associated.According to an embodiment, in operation 1360, as mentioned above, any time line can be activated and can be used for generating the prompting and calendar that are associated with task items or list items, this timeline comprise be associated with the task items generated according to recorded voice and/or audio content or list items fixed date/time and loose or fuzzy date/time.Such as, metadata that is that extract from given electronic tasks item or that be applied to given electronic tasks item can be used for identifying the deadline of this task items, and this given electronic tasks item generates according to caught phonetic entry.Task items can be stored subsequently together with other task items, and can create the timeline of each task items (comprising the task items generated according to caught phonetic entry).As mentioned above, according to task is completed fixed date/time and use that is loose or " fuzzy " date/time, the task items stored can relative to each other arrange, any one making in one or more task items that must perform at special time arranges according to this special time, and any one making in one or more task items that can perform at any time can be arranged at other pot lifes not comprising this special time.

According to each embodiment, after generating task or list items via voice and the input of other audio frequency, user and task or list items alternately can be tracked, when these are done to comprise tracking.Can via various suitable means, comprise Intranet, based on the Internet or the communication system based on cloud, or issue information about the user interactions with task or list items via one or more social networks to a user or other users, comprise the statistical information of the quantity that completed task or list items are shown.May be important helping user to understand in their achievement or achievement that they receive other users of these message for it to the announcement of these customer interaction informations.The method terminates in operation 1395.

Embodiment described herein and function can operate through many computing systems, comprise wired and wireless computing system, mobile computing system (as mobile phone, graphic tablet or Tablet PC, laptop computer etc.).In addition, embodiment as herein described and function can operate in distributed system, and wherein application function, storer, data storage and search and various processing capacity remotely can operate each other on the distributed computing network of such as the Internet or Intranet and so on.Various types of user interface and information can carry computing equipment display through plate or the remote display unit through being associated with one or more computing equipment is shown.Such as, various types of user interface and information can be shown with mutual on wall surface, and various types of user interface and information are projected on wall surface.With comprising alternately of the many computing systems that can be used for implementing various embodiments of the present invention: thump inputs, touch-screen inputs, voice or other audio frequency input, posture inputs (computing equipment be wherein associated is equipped with detection (as the camera) function of user's posture for catching and explain the function for controlling calculation equipment) etc.Figure 14-16 and the description be associated provide the discussion of the various operating environments wherein can implementing various embodiments of the present invention.But and the equipment discussed shown about Figure 14-16 and system are the objects for example and explanation, but not to being used to the restriction of a large amount of computing equipments configurations implementing various embodiments of the present invention as herein described.

Figure 14 is the block diagram that the exemplary physical assembly that can be used to the computing equipment 1400 implementing various embodiments of the present invention is shown.Computing equipment assembly described below can be suitable for above-mentioned computing equipment, such as computing equipment 405,410 and server and Database Systems 420.In basic configuration, computing equipment 1400 can comprise at least one processing unit 1402 and system storage 1404.Depend on configuration and the type of computing equipment, system storage 1404 can include, but not limited to volatile memory (such as, random-access memory (ram)), nonvolatile memory (such as, ROM (read-only memory) (ROM)), flash memory or any combination.System storage 1404 can comprise operating system 1405, one or more programming module 1406, and can comprise LAS application 100, VRPA101 and web browser application 1420.Such as, operating system 1405 is applicable to the operation of controlling calculation equipment 1400.In addition, embodiments of the invention can be put into practice in conjunction with shape library, other operating systems or any other application program, and are not limited to any application-specific or system.This basic configuration is illustrated by those assemblies in dotted line 1408 in fig. 14.

Computing equipment 1400 can have supplementary features or function.Such as, computing equipment 1400 also can comprise additional data storage device (removable and/or irremovable), such as such as, and disk, CD or tape.These extra storage are illustrated by removable storage 1409 and irremovable storage 1410 in fig. 14.

As mentioned above, the multiple program module and data file that comprise operating system 1405 can be stored in system storage 1404.When performing on processing unit 1402, the programming modules 1406 such as such as LAS application 100 and voice record and process application (VRPA) 101 can perform each process, comprise such as, as above each stage of one or more methods 1300.Said process is an example, and processing unit 1402 can perform other processes.Email and contact application, text processing application, spreadsheet application, database application, slide presentation application, drawing or computer assistant applications etc. can be comprised according to embodiments of the invention other programming modules spendable.

Generally speaking, according to embodiments of the invention, program module can comprise can perform the structure that particular task maybe can realize the routine of particular abstract data type, program, assembly, data structure and other types.In addition, embodiments of the invention can be put into practice by other computer system configurations, comprise portable equipment, multicomputer system, based on the system of microprocessor or programmable consumer electronics, minicomputer, mainframe computer etc.Embodiments of the invention also can be put into practice in the distributed computing environment that task is performed by the remote processing devices by communication network links wherein.In a distributed computing environment, program module can be arranged in local and remote both memory storage device.

In addition, embodiments of the invention can comprise the circuit of discrete electronic component, the encapsulation comprising logic gate or integrated electronic chip, utilize the circuit of microprocessor or put into practice on the one single chip comprising electronic component or microprocessor.Such as, SOC (system on a chip) (SOC) can be passed through and implement various embodiments of the present invention, wherein, each perhaps multicompartment shown in Figure 14 can be integrated on single integrated circuit.Such SOC equipment can comprise one or more processing unit, graphic element, communication unit, system virtualization unit and various application function, as single integrated circuit on all these is integrated into (or " being burnt to ") chip substrate.When being operated by SOC, the function about LAS application 100 and/or VRPA application 101 described herein can be operated by the application-specific logic being integrated with other assembly of computing equipment/system 1400 on single integrated circuit (chip).Embodiments of the invention also can use and can perform such as such as, AND(with), OR(or) and NOT(non-) the other technologies of logical operation put into practice, include but not limited to, machinery, optics, fluid and quantum techniques.In addition, embodiments of the invention can be put into practice in multi-purpose computer or any other circuit or system.

Such as, embodiments of the invention can be implemented as the goods of computer procedures (method), computing system or such as computer program or computer-readable medium and so on.Computer program can be computer system-readable and the computer-readable storage medium of computer program code to the instruction for performing computer procedures.

Term as used herein computer-readable medium can comprise computer-readable storage medium.Computer-readable storage medium can comprise the volatibility and non-volatile, removable and irremovable medium that realize for any method or technology that store the information such as such as computer-readable instruction, data structure, program module or other data.System storage 1404, removable storage 1409 and irremovable storage 1410 are all the examples of computer-readable storage medium (that is, storer stores).Computer-readable storage medium can comprise, but be not limited to, RAM, ROM, electricallyerasable ROM (EEROM) (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storages, tape cassete, tape, disk storage or other magnetic storage apparatus or can be used for storage information and any other medium can accessed by computing equipment 1400.Any such computer-readable storage medium can be a part for equipment 1400.Computing equipment 1400 also can have input equipment 1412, such as keyboard, mouse, pen, audio input device, touch input device etc.Also can comprise output device 1414, such as display, loudspeaker, printer etc.The said equipment is example, and can use other equipment.

Term as used herein computer-readable medium also can comprise communication media.Telecommunication media can be embodied by computer-readable instruction, data structure, program module or other data in the such as modulated message signal such as carrier wave or other transmission mechanisms, and comprises any information transmitting medium.Term " modulated message signal " can describe and to set in the mode of encoding to the information in this signal or to change the signal of one or more feature.Exemplarily unrestricted, communication media comprises such as cable network or the directly wire medium such as line connection, and the wireless medium such as such as acoustics, radio frequency (RF), infrared ray and other wireless mediums.

Figure 15 A and 15B illustrates the suitable mobile computing environment that can be used to implement various embodiments of the present invention, such as mobile phone 410, smart phone, tablet personal computer, laptop computer etc.With reference to figure 15A, show the example mobile computing device 410 for realizing each embodiment.In a basic configuration, mobile computing device 410 is the handheld computers with input element and output element.Input element can comprise and allows user information to be input to touch-screen display 1505 in mobile computing device 410 and load button 1510.Mobile computing device 410 also can in conjunction with the optional Side input element 815 allowing further user input.Optional Side input element 815 can be the manual input element of rotary switch, button or any other type.In alternative embodiments, mobile computing device 410 can in conjunction with more or less input element.Such as, in certain embodiments, display 1505 can not be touch-screen.In another alternate embodiment, mobile computing device is portable telephone system, as having the cell phone of display 1505 and load button 1510.Mobile computing device 410 also can comprise optional keypad 1105." soft " keypad that optional keypad 1105 can be physical keypad or generate on touch-screen display.

Mobile computing device 410 in conjunction with output element, as can the display 1505 of display graphics user interface (GUI).Other output elements comprise loudspeaker 825 and LED light 1520.In addition, mobile computing device 410 can comprise vibration module (not shown), this vibration module make mobile computing device 410 vibrate with by event notice to user.In another embodiment, mobile computing device 410 can in conjunction with earphone jack (not shown), for providing another means to provide output signal.

Although combine mobile computing device 410 herein to describe, but in alternative embodiments, the present invention also any amount of computer system capable of being combined is used, as in desktop environment, on knee or notebook computer system, multicomputer system, based on microprocessor or programmable consumer electronics, network PC, small-size computer, mainframe computer etc.Embodiments of the invention also can be put into practice in a distributed computing environment, and wherein task is performed by the remote processing devices of communication network links by distributed computing environment; Program can be arranged in the machine and remote memory storage device.Generally speaking, there is multiple environmental sensor, provide to user any computer system of multiple output element of notice and multiple notification event type can in conjunction with embodiments of the invention.

Figure 15 B is the block diagram of the assembly that the mobile computing devices such as all computing equipments as shown in fig. 15 of using in one embodiment are shown.That is, mobile computing device 410 can coupling system 1502 to realize some embodiment.Such as, system 1502 can be used to " smart phone " that realize running one or more application like the application class with desk-top or notebook, and these application examples are as browser, schedule, instant message and media player applications.In certain embodiments, system 1502 is integrated into computing equipment, such as integrated PDA(Personal Digital Assistant) and wireless telephone.

One or more application program 1566 can be loaded in storer 1562 and to run explicitly in operating system 1564 or with operating system 864.The example of application program comprises Phone Dialer, e-mail program, PIM(personal information management) program, word processing program, spreadsheet program, the Internet browser programs, messaging programs etc.System 1502 also comprises the non-volatile memories 1568 in storer 1562.The permanent message can not lost when non-volatile memories 1568 can be used to the system of being stored in 1502 power-off.Application 1566 can use information and information is stored in non-volatile memories 1568, the Email used as e-mail applications or other message etc.Synchronous applications (not shown) also to reside in system 1502 and is programmed to carry out alternately with resident corresponding synchronous applications on a host computer, to keep being stored in the information in non-volatile memories 1568 and the corresponding informance synchronised stored on a host computer.Should be appreciated that other application can be loaded in storer 1562 and to run on equipment 410, comprise list composition service described herein application 100 and voice record and process application 101.

System 1502 has the power supply 1570 that can be implemented as one or more battery.Power supply 1570 also can comprise external power source, as supplementary battery or the AC adapter again charged to battery or powered docking cradle.

System 1502 also can comprise the radio 1572 of the function performing transmitting and receiving radio frequency communication.Radio 1572 facilitates the wireless connections between system 1502 and " external world " by common carrier or service supplier.The transmission of dealing radio 1572 carries out under the control of operating system 1564.In other words, the communication that radio 1572 receives propagates into application program 1566 by operating system 1564, and vice versa.

Radio 1572 allows system 1502 such as by network and other computing device communication.Radio 1572 is examples for communication media.Communication media is embodied by the computer-readable instruction in the modulated message signal of such as carrier wave or other transmission mechanisms and so on, data structure, program module or other data usually, and comprises any information-delivery media.Term modulated message signal " " refers to have and is set in the mode of coded message in the signal or changes the signal of one or more feature.Exemplarily unrestricted, communication media comprises such as cable network or the direct wire medium to connect and so on of line, and the wireless medium of such as acoustics, RF, infrared and other wireless mediums and so on.Term as used herein " computer-readable medium " comprises storage medium and communication media.

This embodiment of system 1502 illustrates with the notice output device of two types: the LED1520 that can be used to provide visual notification, and the audio interface 1574 that can be used to that loudspeaker 825 provides audible notification.These equipment can be directly coupled to power supply 1570, make when activated, even if in order to save the power of battery and may closing process device 1560 and other assembly, they also retain one period of maintenance conduction time indicated by informing mechanism.LED1520 can be programmed to ad infinitum keep energising, the "on" position of this equipment of instruction until user takes action.Audio interface 1574 is for providing audible signal to user and accepting audible signal from user.Such as, except being coupled to loudspeaker 825, audio interface 1574 also can be coupled to microphone 820 and receive and can listen input, such as, be convenient to telephone relation.According to various embodiments of the present invention, microphone 820 also can serve as audio sensor to be convenient to the control to notice, as will be described below.System 1502 can comprise permission plate further and carry the operation of camera 830 to record the video interface 1576 of rest image, video flowing etc.

Mobile computing device realizes system 1502 can have supplementary features or function.Such as, other sensors or pick-up unit can with system 1502(as above such as global positioning satellite system (GPS) position system, clock system, optical sensor, camera function etc.) use explicitly, this system 1502 can be used for auxiliary catch and generation task or list items, and one or more item of context information is associated with caught task or list items.For another example, equipment also can comprise additional data storage device (moveable/or immovable), such as disk, CD or tape.This type of extra storage is illustrated by storage 1568 in Figure 15 B.Computer-readable storage medium can comprise the volatibility and non-volatile, removable and irremovable medium that realize for any method or technology that store the information such as such as computer-readable instruction, data structure, program module or other data.

That equipment 410 generates or catches and through system 1502 store data/information can be stored on equipment 410 in this locality as mentioned above, or data can be stored in can by equipment by radio 1572 or any amount of storage medium of being accessed by the wired connection between equipment 410 and the one point of computing equipment opened 405 be associated with equipment 410, this computing equipment is as the server computer in the distributed computing network 415 of such as the Internet and so on.As should be understood, this type of data/information can through equipment 410, through radio 1572 or accessed through distributed computing network 415.Similarly, these data/information can easily be transmitted between computing devices for storage according to known data/information transmission and storage means and use, and these means comprise Email and collaboration data/information sharing system.

Figure 16 illustrates the system architecture for the list items caught through voice and/or audio recording and task items being supplied to after generation via distributed computing environment each client devices.As previously mentioned, voice record and process application (VRPA) 101 can generate one or more list items or task items according to recorded voice or the input of other audio frequency and mark by metadata.The voice/audio input caught and the metadata that is associated and the list items generated and task items also can be stored in different communication ports or be stored in other storage classes.Such as, various database or such as directory service 1622 can be used, web door 1624, mailbox service 1626, instant messaging communication store other storage means such as 1628 and social network site 1630 and come memory row list item and task items and therefrom develop the information of this list items and task items.System/application 100,101 can use any one in the system of these types to develop list items and task items and to be stored in storage 1616.Server 420 can provide list items and task items to client computer.As an example, server 420 can be the web server being provided list items and task items by web.Server 420 is provided in alignment list item and task items to client computer by network 415 on web.The example that can obtain the client computer of list items and task items comprises computing equipment 405, and this computing equipment can comprise any general purpose personal computer 1400, tablet computing device 1603 and/or can comprise the mobile computing device 410 of smart phone.Any one in these equipment can obtain list items and task items from storage 1616.

Embodiments of the invention are described above see, for example the block diagram of method, system and computer program according to an embodiment of the invention and/or operational illustrations.In frame each function/action of indicating can occur by the order be different from shown in any process flow diagram.Such as, depend on involved function/action, in fact two frames illustrated continuously can perform substantially simultaneously, or these frames can perform by contrary order sometimes.

Although described specific embodiment of the present invention, also other embodiments may be there are.In addition, although embodiments of the invention are described to be associated with the data be stored in storer and other storage mediums, but read on the computer-readable medium that data also can be stored in other types or from it, such as auxiliary storage device (as hard disk, floppy disk or CD-ROM), from the carrier wave of the Internet or other forms of RAM or ROM.In addition, each step of disclosed method can be revised by any way, comprises by resequencing to each step and/or inserting or delete step, and does not deviate from the present invention.

To those skilled in the art, obviously can make each amendment or change to the present invention, and not deviate from scope of the present invention or spirit.After considering instructions and realizing the present invention disclosed herein, other embodiments of the present invention will be apparent to those skilled in the art.

Claims (9)

1., for generating a method for electronic tasks item according to the input of caught audio frequency, comprising:

Receive the first audio content item;

Described first audio content item is transcribed into the first text string representing described first audio content item;

One or more metadata item is applied to described first text string to identify one or more compositions of described first text string;

The first electronic tasks item is generated based on described first audio content item;

Display comprises the described first electronic tasks item of described first text string;

The audio file comprising described first audio content item is associated with shown described first electronic tasks item; And

The visual indicator selected of the audio file be associated is shown near described the first shown electronic tasks item.

2. the method for claim 1, is characterized in that, also comprises:

Receive the selection can selecting visual indicator described in the audio file be associated; And

Playback to the described audio file comprising described first audio content item is provided.

3. the method for claim 1, it is characterized in that, wherein one or more metadata item is applied to described first text string and comprises the one or more metadata item of application to be associated with one or more position by described first electronic tasks item with the one or more compositions identifying described first text string, described one or more position is associated with described first electronic tasks item.

4. the method for claim 1, it is characterized in that, wherein one or more metadata item is applied to described first text string and comprises the one or more metadata item of application to be associated with one or more date and time by described first electronic tasks item with the one or more compositions identifying described first text string, described one or more date and time is associated with described first electronic tasks item.

5. the method for claim 1, it is characterized in that, also comprise and the one or more composition of one or more composition of described first text string with second text string of transcribing according to the second audio content item is compared, to be associated with described second text string by described first text string.

6. method as claimed in claim 5, it is characterized in that, wherein described first text string and described second text string are associated to comprise and described electronic tasks item is divided into groups together with the second electronic tasks item generated according to described second text string, and comprise and described first and second electronic tasks items are shown as together relevant task items grouping.

7. the method for claim 1, is characterized in that, also comprises and generating the prompting completing described first electronic tasks item, and presented described prompting explicitly with described first electronic tasks item before the deadline of described first electronic tasks item.

8., for generating a method for electronic tasks item according to the input of caught audio frequency, comprising:

Receive the first audio content item;

Described first audio content item is transcribed into the first text string representing described first audio content item;

One or more metadata item is applied to a subset of the one or more words forming described first text string, to identify one or more compositions of described first text string;

Generate the first electronic tasks item, described first electronic tasks item comprises the described subset of one or more words of described first text string;

Show described first electronic tasks item; And

The audio file comprising described first audio content item is associated with the first shown electronic tasks item.

9., for generating a system for electronic tasks item according to the input of caught audio frequency, comprising:

The voice record be coupled with the computing equipment that can record is applied with processing, and described voice record and process application are used for:

Receive the first audio content item;

Described first audio content item is transcribed into the first text string representing described first audio content item;

One or more metadata item is applied to described first text string, to identify one or more compositions of described first text string;

Generate the first electronic tasks item, described first electronic tasks item comprises one or more parts of described first text string;

Show described first electronic tasks item creating in user interface with any one list be associated in multiple software application user interface, described multiple software application user interface is for showing described list creation user interface;

The audio file comprising described first audio content item is associated with the first shown electronic tasks item;

The visual indicator selected of the audio file be associated is shown near described the first shown electronic tasks item;

Receive the selection can selecting visual indicator described in the audio file be associated; And

Playback to the described audio file comprising described first audio content item is provided.