CN110455179A

CN110455179A - The device and method measured using augmented reality

Info

Publication number: CN110455179A
Application number: CN201910194257.8A
Authority: CN
Inventors: G·R·保罗; G·耶基斯; S·O·勒梅
Original assignee: Apple Inc
Current assignee: Apple Inc
Priority date: 2018-05-07
Filing date: 2019-03-14
Publication date: 2019-11-15

Abstract

This application involves the device and method for using augmented reality to measure.Electronic equipment shows that mark places user interface comprising the virtual placement user interface element for marking the position that will be placed of expression and instruction of physical space.Electronic equipment detects movement of the camera relative to physical space, and when placing first part of the user interface element corresponding to physical space, the movement starts.In response, electronic equipment is mobile places user interface element to correspond to the second part of the physical space.If can not identify that its corresponding objects in the expression of physical space in the second part of physical space can be linked to the object virtually marked, electronic equipment stops at least part that user interface element is placed in display.If having identified such object, the display for placing user interface element is kept.

Description

The device and method measured using augmented reality

Technical field

The present invention relates generally to the electronic equipments for virtual/augmented reality, including but not limited to existing using virtual/enhancing Real environment measures the electronic equipment of physical space and/or object.

Background technique

Augmented reality environment is for passing through the view for providing physical space and allowing users in physical space and therein On object superposition measurement come in the environment physical space and object to measure be useful.But use augmented reality into The conventional method of row measurement is troublesome, inefficient and be limited.In some cases, the conventional method measured using augmented reality Functionally it is restricted.In some cases, it is needed using the conventional method that augmented reality measures multiple individually defeated Enter (for example, a series of gestures and button press etc.) to realize expected results (for example, the user interface by activating multiple displays Element accesses different measurement functions).In addition, conventional method spends the longer time than the time required to, to waste energy. This later regard is especially important in battery-driven equipment.

Summary of the invention

Therefore, it is necessary to have based on improved method and the interface by being measured using virtual/augmented reality environment Calculation machine system.Such method and interface optionally supplement or replace normal for being measured using virtual/augmented reality environment Rule method.Such method and interface reduce the quantity, degree and/or property of input from the user, and generate more effective Man-machine interface.For battery operated device, such method and interface can save electricity consumption and increase between the charging of two primary cells Time.

It is reduce or eliminated and the use for being measured using virtual/augmented reality using disclosed computer system The associated drawbacks described above in family interface and other problems.In some embodiments, which includes desktop computer. In some embodiments, which is portable (for example, laptop, tablet computer or handheld device). In some embodiments, which includes personal electronic equipments (for example, wearable electronic, such as wrist-watch). In some embodiments, which has Trackpad (and/or with touch-control board communications).In some embodiments, The computer system have touch-sensitive display (also referred to as " touch screen " or " touch-screen display ") (and/or with it is touch-sensitive aobvious Show that device communicates).In some embodiments, which has graphic user interface (GUI), one or more processing Device, memory and one or more modules, storage are in memory for executing the program or instruction set of multiple functions.Some In embodiment, handed over by the gesture on stylus and/or finger contact and touch sensitive surface with GUI to User Part Mutually.In some embodiments, other than the measurement function based on augmented reality, these functions optionally include play game, Picture editting, demonstration, word processing, electrical form production, making and receiving calls, video conference, sends and receive e-mail, immediately at drawing Message communicating, body-building support, digital photography, digital video recordings, web page browsing, digital music play, record the note and/or number Video playing.Executable instruction for executing these functions, which is optionally included in, to be configured for being handled by one or more In the non-transient computer readable storage medium of device execution or other computer program products.

According to some embodiments, executed on the electronic equipment with touch-sensitive display and one or more input equipments A kind of method.This method includes showing the user interface of application program on the touch sensitive display.The user interface include one or It is multiple it is magazine at least one the visual field expression.The expression in the visual field shows with the first magnifying power, and based on by one or The variation of at least one multiple magazine Current vision data detected, the at any time expression of more New view.The visual field includes At least part of three-dimensional space.This method comprises: detecting the first touching on touch-sensitive display while showing that the visual field indicates Input is touched, and in response to detecting the first touch input, the first position in three-dimensional space is corresponded in the expression of the visual field Measurement point is added and shown at first position.This method further includes after adding measurement point and continuing to show that the visual field indicates While: when at least one one or more magazine movement, first in three-dimensional space is corresponded in the expression of the visual field The measurement point is shown at the position of position；Detection corresponds to the current location of the measurement point in the expression of the visual field on the touch sensitive display Position at the second touch input；Also, in response to detecting the second touch input, the visual field is indicated at least part of Display is amplified to the second magnifying power greater than the first magnifying power from the first magnifying power, and the amplification for the part that wherein visual field indicates is aobvious Show including the measurement point.

According to some embodiments, with touch-sensitive display, to detect and the intensity of the contact of touch-sensitive display A kind of method is executed at one or more sensors, and the electronic equipment of one or more cameras.This method is included in touch-sensitive The user interface of application program is shown on display.The user interface includes at least one the one or more magazine visual field Expression.Based on the variation by least one one or more magazine Current vision data detected, update at any time The expression in the visual field.User interface further includes the measurement point creation indicator being shown on the expression of the visual field.The visual field includes three-dimensional space Between at least part.This method includes the contact detected on touch-sensitive display, also, on continuously detection touch-sensitive display When contact, on the first position that measurement point creation indicator corresponds to the first position in three-dimensional space in the expression of the visual field When display, and meet the first standard according to determining, adds and show first corresponded in three-dimensional space in the expression of the visual field First measurement point of position, wherein the first standard includes that the intensity of contact is required to meet corresponding intensity threshold to make the first mark Satisfaction will definitely be arrived.This method further includes the expression of more New view when electronic equipment is mobile after adding the first measurement point.It should Method further include: after electronic equipment is mobile, the second of the second position in three-dimensional space is corresponded in the expression of the visual field While showing that measurement point creates indicator on setting, according to the determining second corresponded in three-dimensional space in the expression of the visual field It is shown on the second position set and meets the first standard while measurement point creates indicator: added and show in the expression of the visual field The second measurement point corresponding to the second position in three-dimensional space；And show the first of the first measurement point of connection and the second measurement point Measuring section.

According to some embodiments, executed on the electronic equipment with touch-sensitive display and one or more input equipments A kind of method.This method includes showing the user interface of application program on the touch sensitive display.The user interface include one or It is multiple it is magazine at least one the visual field expression.Based on by it is one or more it is magazine at least one detect it is current The variation of vision data, the at any time expression of more New view.User interface includes the measurement point wound being shown on the expression of the visual field Build indicator.The visual field includes at least part of three-dimensional space.This method includes, and while showing the expression in the visual field, determines Correspond to the anchor point at the position of the first position in three-dimensional space in the expression of the visual field.This method further include: with one Or at least one multiple magazine movement change measurement point creation and refer to when measurement point creation indicator is on anchor point Show the visual appearance of symbol to indicate: if touch input meets the first standard, measurement point will be added at anchor point.This method Further include: meet the first touch input of the first standard on detection touch-sensitive display, and meets the first mark in response to detecting The first quasi- touch input: according to determining that measurement point creates indicator on anchor point when the first standard meets, in the visual field Correspond at the anchor point of the first position in three-dimensional space in expression and adds and show the first measurement point；And existed according to determination Measurement point creates indicator not on anchor point when first standard meets, and the first position of anchor point is left in the expression of the visual field The first measurement point of place's addition and display.

According to some embodiments, on the electronic equipment with display, output equipment and one or more cameras Execute a kind of method.This method includes showing the user interface of application program over the display.The user interface include one or It is multiple it is magazine at least one the visual field expression.Based on by it is one or more it is magazine at least one detect it is current The variation of vision data, the at any time expression of more New view.The visual field includes the physical object in three-dimensional space.This method comprises: While showing the expression in the visual field, one or more user's inputs are detected via input equipment, on the expression in the visual field Addition corresponds to the expression of the first measurement of physical object.This method further include: the display first simultaneously on the expression in the visual field First label of the first measurement of expression and description of measurement, in which: according to first between determining electronic equipment and physical object Distance is less than first threshold distance, shows the first label with first threshold size；According to determine electronic equipment and physical object it Between first distance be greater than second threshold distance, be less than first threshold size second threshold size show the first label, Middle second threshold distance is greater than first threshold distance；And existed according to the first distance between determining electronic equipment and physical object Between first threshold distance and second threshold distance, first is shown with the size between first threshold size and second threshold size Label, the size depend on the first distance between electronic equipment and physical object.

According to some embodiments, on the electronic equipment with display, output equipment and one or more cameras Execute a kind of method.This method includes showing the user interface of application program over the display.The user interface include one or It is multiple it is magazine at least one the visual field expression.Based on by it is one or more it is magazine at least one detect it is current The variation of vision data, the at any time expression of more New view.The visual field includes the physical object in three-dimensional space.This method comprises: While showing the expression in the visual field, one or more user's inputs are detected via input equipment, on the expression in the visual field Addition corresponds to the expression of the first measurement of physical object, wherein expressions of the first measurement includes the corresponded on physical object The first end point of one position, the expression of the first measurement include the second endpoint corresponding to the second position on physical object；And The expression of first measurement includes the first line segment of connection first end point and the second endpoint.This method further include: be based in part on One measurement determines the first area adjoined in the expression of the visual field with the first line segment of the first measurement, and wherein first area corresponds to three Physics rectangular area in dimension space.This method further includes showing the instruction of first area in the user interface, wherein the instruction It is covered on the first area in the expression of the visual field.

According to some embodiments, executed on the electronic equipment with touch-sensitive display and one or more input equipments A kind of method.This method includes showing the first user interface of application program on the touch sensitive display.First user interface includes It is one or more it is magazine at least one the visual field expression.Based on by it is one or more it is magazine at least one detect Current vision data variation, the expression of more New view at any time.The visual field includes the physical object in three-dimensional space.Physics pair The expression of the measurement of elephant is superimposed upon in the expression of the visual field on the image of the physical object.This method comprises: showing the first user circle While face, the first touch input on touch-sensitive display in the expression of measurement is detected.This method further include: in response to detection The first touch input on to touch-sensitive display in the expression of measurement, mistake of the starting for the shared information about the measurement Journey.

According to some embodiments, on the electronic equipment with display, output equipment and one or more cameras Execute a kind of method.This method includes showing the user interface of application program over the display.The user interface include one or It is multiple it is magazine at least one the visual field expression.Based on by it is one or more it is magazine at least one detect it is current The variation of vision data, the at any time expression of more New view.The visual field includes at least part of three-dimensional space.This method includes inspection Survey the movement of electronic equipment, which move at least one the one or more magazine visuals field in a first direction.It should Method further include: detect electronic equipment make that the visual field moves in a first direction it is mobile while: according to electronic equipment It is mobile, the expression of more New view；The first element of one or more extended in a first direction in the expression of the identification visual field；Also, extremely It is at least partly based on and determines one or more first elements, shown in the expression of the visual field and extend in a first direction and correspond to The first guidance of one in the first element of one or more identified.

According to some embodiments, there are one or more input equipments, one or more to show equipment and one Or a kind of method is executed at the electronic equipment of multiple cameras: this method includes showing user circle via one or more display equipment Face, the user interface include the expression of physical space.This method comprises: receiving and using while showing the expression of physical space To create virtually mark first group of one or more input in the indicating of physical space.This method further include: in response to connecing First group of one or more input is received, the first virtual mark is added to the expression of physical space.First virtual mark link To a part of the expression of physical space.This method further include: the expression that the first virtual mark is added to physical space it Afterwards, second group of one or more input associated with the expression of physical space is received.This method further include: in response to receiving It is inputted with the associated second group of one or more of the expression of physical space: corresponding according to determining second group of one or more input The request virtually marked away from the first virtual mark within the threshold range is created in the expression in physical space, in physical space The virtually mark, while the virtual mark of holding first in the expression of physical space of creation second in expression；Also, according to determining the Two groups of one or more inputs are virtual outside away from the first virtual mark threshold distance corresponding to creating in the expression of physical space The request of mark creates second in the expression of physical space and virtually marks and remove the first void from the expression of physical space Quasi- mark.

According to some embodiments, there are one or more input equipments, one or more to show equipment and one Or a kind of method is executed at the electronic equipment of multiple cameras.This method includes putting via one or more display equipment display mark Set user interface.Mark places the expression that user interface includes: physical space；And user interface element is placed, instruction is rung Ying Yu detects that mark places input, and virtual mark will be placed on which of the expression of physical space.This method packet Include: while display marks and places user interface, detection it is one or more it is magazine at least one relative to physical space Movement.When the position for the first part for corresponding to physical space in the expression that placement user interface element is shown in physical space When place, at least one one or more magazine mobile beginnings.This method comprises: in response to detecting one or more phases Movement of at least one of the machine relative to physical space, will place user interface element be moved to it is right in the expression of physical space It should be different from the first part of physical space in the position of the second part of physical space, the second part, and according to one Or at least one multiple magazine movement relative to physical space, update the appearance that mark places user interface, comprising: root According to determining that electronic equipment can not identify that corresponding objects of its in the second part of physical space in the expression of physical space can be with It is linked to the object virtually marked, stops at least part that user interface element is placed in display；And according to determining equipment Its corresponding objects in the expression of physical space in the identified second part of physical space out can be linked to virtual mark The object of note keeps the display for placing user interface element.

According to some embodiments, a kind of computer system (for example, electronic equipment) include display generating unit (for example, Display, projector, head-up display etc.), one or more camera is (for example, continuously provide at least one in camera fields of view The live preview of point content and the view for optionally generating one or more picture frames including the content in capture camera fields of view The video camera of frequency output), and one or more input equipments (for example, touch sensitive surface, such as touch-sensitive remote controler, or also use Touch-screen display, mouse, control stick, stick controller and/or the one or more of tracking user for making display generating unit are special Levy the camera of the position of the hand of such as user), optional one or more attitude transducers, optional detection touch sensitive surface connect The one or more sensors of the intensity of touching, optional one or more tactile output generators, one or more processors, with And store the memory (and/or with these component communications) of one or more programs；One or more programs are configured as by one A or multiple processors execute, and one or more programs include for executing or causing to execute appointing in methods described herein The instruction of the operation of one method.According to some embodiments, computer readable storage medium is stored with instruction wherein, these refer to It enables when by including display generating unit, one or more cameras, one or more input equipments, optional one or more postures Sensor, optionally with one or more sensors to detect the intensity of contact with touch sensitive surface and one optional Or the computer system of multiple tactile output generators (and/or with these component communications) is when executing, so that the computer system It executes the operation of any method as described herein or the operation of any method described herein is performed.According to some embodiment party Case is including display generating unit, one or more cameras, one or more input equipments, optional one or more postures Sensor, optionally with the one or more sensors, one or more optional to detect the intensity of contact with touch sensitive surface A tactile output generator, memory and the one or more to execute one or more programs stored in memory Graphic user interface in the computer system of processor (and/or with these component communications) is included in any method described herein In shown one or more elements, the one or more element is updated in response to input, any as described herein Described in method.According to some embodiments, computer system includes display generating unit, one or more cameras, one A or multiple input equipments, optional one or more attitude transducers, optionally with to detect the contact with touch sensitive surface The one or more sensors of intensity, optional one or more tactile output generators and for executing as described herein Where method operation or device (and/or with these component communications) that the operation of any method described herein is performed.Root According to some embodiments, for include display generating unit, it is one or more cameras, one or more input equipments, optional One or more attitude transducers, optionally with the one or more sensors to detect the intensity of contact with touch sensitive surface And believe used in the computer system of optional one or more tactile output generator (and/or with these component communications) Breath processing unit includes operation for executing any method as described herein or the operation quilt for making any method described herein The device of execution.

Therefore, there is (and/or communicating) to show that generating unit, one or more cameras, one or more inputs are set Standby, optional one or more attitude transducers, optionally with detect the intensity of the contact with touch sensitive surface one or more The computer system of a sensor and optional one or more tactile output generators has for using virtual/augmented reality The improved method of environment measurement physical object and interface, to improve the validity of this computer system, efficiency and user Satisfaction.Such method and interface, which can be supplemented or be replaced for using virtual/augmented reality environment, measures physical object Conventional method.

Detailed description of the invention

The various embodiments in order to better understand should refer to following specific embodiment in conjunction with the following drawings, Wherein similar drawing reference numeral indicates corresponding part in all the appended drawings.

Figure 1A is the block diagram for showing the portable multifunction device with touch-sensitive display according to some embodiments.

Figure 1B is the block diagram for showing the example components for event handling according to some embodiments.

Fig. 1 C is to show the block diagram of the tactile output module according to some embodiments.

Fig. 2 shows the portable multifunction devices with touch screen according to some embodiments.

Fig. 3 A is the block diagram according to the exemplary multifunctional equipment with display and touch sensitive surface of some embodiments.

Fig. 3 B-3C is the block diagram according to the exemplary computer system of some embodiments.

Fig. 4 A shows the exemplary of the application menu on the portable multifunction device according to some embodiments User interface.

Fig. 4 B shows setting with the multi-functional of touch sensitive surface that display separates for having according to some embodiments Standby example user interface.

Fig. 4 C- Fig. 4 E shows the embodiment of the resistance to vibration threshold value according to some embodiments.

Fig. 4 F- Fig. 4 K shows one group of sample tactile output mode according to some embodiments.

Fig. 5 A-5CO is shown according to some embodiments for carrying out physical space measurement using augmented reality environment Exemplary user interface.

Fig. 6 A-6C be according to some embodiments with for using augmented reality environment to carry out answering for physical space measurement With the flow chart of the process of program interaction.

Fig. 7 A-7E is according to some embodiments for that will measure the physical space being added in augmented reality environment Show the flow chart of the process indicated.

Fig. 8 A-8C is according to some embodiments for adding at the anchor point automatically determined in augmented reality environment Add the flow chart of the process of virtual measurement point.

Fig. 9 A-9B is according to some embodiments for showing the label of physical space measurement in augmented reality environment Process flow chart.

Figure 10 A-10B is according to some embodiments for measuring the rectangle in physical space in augmented reality environment The flow chart in region and the process interacted.

Figure 11 A-11B is according to some embodiments for interacting and managing with metrical information in augmented reality environment The flow chart of the process of metrical information.

Figure 12 A-12C is drawing for providing the alignment automatically determined in augmented reality environment according to some embodiments The flow chart for the process led.

Figure 13 A-13C is according to some embodiments for removing the void previously added automatically in augmented reality environment The flow chart of the process of quasi- mark.

Figure 14 A-14D is whether be identified as according to the object of some embodiments being used to indicate in physical space can To track the flow chart of the process of its corresponding object indicated in augmented reality environment.

Specific embodiment

As described above, augmented reality environment is for the view by offer physical space and allows users in physical space With superposition measurement on physical object therein come in the environment physical space and object to measure be useful.Utilize increasing The conventional method that strong actual environment measures usually functionally is restricted.In some cases, conventional method needs more A series of a individual input (for example, gestures and button press etc.) is to realize expected results (for example, by activating multiple displays User interface element access different measurement functions).Embodiment disclosed herein provides existing using enhancing for user The intuitive manner that real environment measures with less input by using family in augmented reality environment (for example, can be executed Different operations, and/or by simplifying user interface).In addition, embodiments herein provides improved vision and tactile is anti- Feedback, provides a user about measured physical object and about the additional letter of the operation executed in augmented reality environment Breath.

System, method and GUI as described herein improve user circle carried out with virtual/augmented reality environment in many ways Face interaction.For example, by provide to the feature in physical space it is automatic detection, it is improved mark and alignment guidance (for example, with In the placement of improved measurement point and region recognition), and can be interacted with metrical information by using family and it is carried out Management, makes it easier to using the feature in augmented reality environment measurement physical space.

In the following, Figure 1A provides the description to example devices to Figure 1B, Fig. 2 and Fig. 3 A to Fig. 3 B.Fig. 4 A-4B and figure 5A-5CO shows the exemplary background and exemplary user interface that physical space measurement is carried out using augmented reality environment.Figure 6A-6C is shown for carrying out the process of method that the application program of physical space measurement interacts with augmented reality environment is used Figure.The flow chart for the method that the display that Fig. 7 A-7E shows the physical space that measurement is added in augmented reality environment indicates. Fig. 8 A- 8C shows the stream for the method being added to virtual measurement point at the anchor point automatically determined in augmented reality environment Cheng Tu.Fig. 9 A-9B shows the flow chart for the method for showing the label of physical space measurement in augmented reality environment.Figure 10A-10B shows the process of method for measuring the rectangular area in physical space in augmented reality environment and interacting Figure.Figure 11 A-11B shows the flow chart for interacting and managing the method for metrical information with metrical information in augmented reality environment. Figure 12 A-12C is shown provides the flow chart of the method for the alignment guidance automatically determined in augmented reality environment.Figure 13 A- 13C is the flow chart for removing the process virtually marked previously added automatically in augmented reality environment.Figure 14 A-14D is Whether the object being used to indicate in physical space has been identified as that its corresponding expression in augmented reality environment can be tracked The flow chart of the process of object.User interface in Fig. 5 A-5CO is for showing Fig. 6 A-6C, Fig. 7 A-7E, Fig. 8 A-8C, Fig. 9 A- 9B, Figure 10 A-10B, Figure 11 A-11B, Figure 12 A-12C, the process in Figure 13 A-13C and Figure 14 A-14D.

Example devices

Reference will now be made in detail to embodiment, the example of these embodiments is shown in the accompanying drawings.Following retouches in detail Many details are shown in stating, are fully understood in order to provide to various described embodiments.But to this field Those of ordinary skill is evident that various described embodiments can be without these specific details It is practiced.In other cases, well-known method, process, component, circuit and network are not described in detail, thus not It can unnecessarily make the various aspects of embodiment hard to understand.

Although will be further understood that in some cases, term " first ", " second " etc. are various for describing herein Element, but these elements should not be limited by these terms.These terms are only intended to an element and another element region It separates.For example, the first contact can be named as the second contact, and similarly, the second contact can be named as the first contact, and The range of various described embodiments is not departed from.First contact and the second contact are contact, but they are not same Contact, unless the context clearly.

Term used in the description to the various embodiments is intended merely to description particular implementation side herein The purpose of case, and be not intended to be limiting.Such as in the description and the appended claims in the various embodiments Used "one" is intended to also include plural form with "the" singular like that, indicates unless the context clearly. It will be further understood that term "and/or" used herein refer to and cover in associated listed project one Any and all possible combinations of a or multiple projects.It will be further understood that term " includes " (" includes ", " including ", " comprises " and/or " comprising ") it specifies to exist when using in the present specification and be stated Feature, integer, step, operations, elements, and/or components, but it is not excluded that in the presence of or add other one or more features, whole Number, step, operation, component, assembly unit and/or its grouping.

As used herein, based on context, term " if " be optionally interpreted to mean " and when ... when " (" when " or " upon ") or " in response to determination " or " in response to detecting ".Similarly, based on context, phrase is " if really It is fixed ... " or " if detecting [condition or event stated] " be optionally interpreted to refer to " when in determination ... " or " in response to determination ... " or " when detecting [condition or event stated] " or " in response to detecting [the item stated Part or event] ".

Computer system for virtual/augmented reality includes the electronic equipment for generating virtual/augmented reality environment.Herein Describe the embodiment of electronic equipment, the user interface of such equipment and the correlated process using such equipment.In some realities It applies in scheme, which is the portable communication device for also including other function such as PDA and/or music player functionality, all Such as mobile phone.The exemplary implementation scheme of portable multifunction device includes but is not limited to come from Apple Inc. (Cupertino, California's)iPodWithEquipment.It is optionally portable using other Electronic equipment, laptop computer or plate electricity such as with touch sensitive surface (for example, touch-screen display and/or touch tablet) Brain.It is to be further understood that in some embodiments, which is not portable communication device, but there is touch sensitive surface The desktop computer of (for example, touch-screen display and/or touch tablet) further includes one or more cameras or is communicated with.

In the following discussion, describe a kind of computer system comprising have display and touch sensitive surface (and/or With these component communications) electronic equipment.It will be appreciated, however, that the computer system optionally include it is one or more other Physical user-interface device, such as physical keyboard, mouse, control stick, stick controller and/or the one or more spy for tracking user Levy the camera of the position of the hand of such as user.

The equipment usually supports various application programs, such as one or more of following: game application, notes note Record application program, drawing application program, PowerPoint application program, word-processing application, spreadsheet applications, electricity Application program, videoconference application, email application, instant message application program, body-building support are talked about using journey Sequence, photo management application program, digital camera applications program, digital video camera application program, web page browsing application program, number The happy player application of word tone and/or video frequency player application program.

The physical user-interface device that the various application programs executed in equipment optionally use at least one general, it is all Such as touch sensitive surface.One or more functions of touch sensitive surface and the corresponding informance shown by the equipment are optionally for difference Application program is adjusted and/or changes, and/or is adjusted and/or changes in corresponding application programs.In this way, equipment shares Physical structure (such as touch sensitive surface) supports various answer optionally with intuitive for a user and clear user interface Use program.

The embodiment that attention is drawn to the portable device with touch-sensitive display.Figure 1A is to show basis The block diagram of the portable multifunction device 100 with touch-sensitive display system 112 of some embodiments.Touch-sensitive display system System 112 is called " touch screen " sometimes for convenient, and is called touch-sensitive display for short sometimes.Equipment 100 includes storage Device 102 (it optionally includes one or more computer readable storage mediums), Memory Controller 122, one or more at Manage unit (CPU) 120, peripheral device interface 118, RF circuit 108, voicefrequency circuit 110, loudspeaker 111, microphone 113, defeated Enter/export (I/O) subsystem 106, other inputs or control equipment 116 and outside port 124.Equipment 100 is optionally wrapped Include one or more optical sensors 164 (for example, a part as one or more cameras).Equipment 100 optionally includes One of contact strength in detection device 100 (for example, touch sensitive surface, touch-sensitive display system 112 of such as equipment 100) A or multiple intensity sensors 165.Equipment 100 optionally includes one or more for generating tactile output on the appliance 100 A tactile output generator 163 is (for example, in the touch-sensitive display system 112 or equipment 300 of touch sensitive surface such as equipment 100 Tactile output is generated on touch tablet 355).These components are carried out alternately through one or more communication bus or signal wire 103 Communication.

As used in the specification and claims, term " tactile output ", which refers to, to utilize user's by user The equipment that sense of touch detects is opposite relative to physical displacement, the component (for example, touch sensitive surface) of equipment of the previous position of equipment In the displacement relative to the mass center of equipment of physical displacement or component of another component (for example, shell) of equipment.For example, In The component and user of equipment or equipment connect to sensitive surface (for example, other parts of finger, palm or user's hand) is touched In the case where touching, the tactile output generated by physical displacement will be construed to sense of touch by user, which corresponds to equipment or set The variation of the physical features of standby component perceived.For example, the movement of touch sensitive surface (for example, touch-sensitive display or Trackpad) " pressing click " or " unclamp and click " to physical actuation button is optionally construed to by user.In some cases, user will Feel sense of touch, such as " click is pressed " or " unclamp and click ", even if being physically pressed (example by the movement of user Such as, be shifted) physical actuation button associated with touch sensitive surface when not moving.For another example, though touch sensitive surface light When slippery is unchanged, the movement of touch sensitive surface also optionally can be explained by user or be sensed as touch sensitive surface " roughness ".Though Individuation sensory perception by user is limited such explanation of touch by right user, but is known many sense organs of touch Feel is that most users are shared.Therefore, when tactile output is described as the specific sensory perception corresponding to user (for example, " pressing Lower click ", " unclamp click ", " roughness ") when, unless otherwise stated, otherwise tactile output generated correspond to equipment or The physical displacement of its component, the physical displacement will generate the sensory perception of typical (or common) user.It is defeated using tactile Touch feedback is provided a user out and enhances the operability of equipment, and keeps user-equipment interface more efficient (for example, passing through User's mistake when user being helped to provide input appropriate and reduce operation equipment/interact with equipment), thus by using family energy It reaches more rapidly and efficiently uses equipment and further reduce electricity usage and extend the battery life of equipment.

In some embodiments, tactile output mode specifies the characteristic of tactile output, such as amplitude of tactile output, touching Feel the duration of the shape of the moving wave shape of output, the frequency of tactile output, and/or tactile output.

When equipment (such as the one or more tactile output generators for generating tactile output via mobile removable mass) When generating the tactile output with different tactile output modes, tactile output can generate not in the user of gripping or touch apparatus Same sense of touch.Although the perception that the sense organ of user exports tactile based on user, most users will identify that equipment generates Tactile output waveform, frequency and amplitude variation.Therefore, waveform, frequency and amplitude can be conditioned to indicate to the user that Perform different operation.In this way, have be designed, select and/or be used for simulate given environment (e.g., including figure is special Seek peace the user interface of object, the analog physical environment with virtual boundary and virtual objects, there is physical boundary and physics pair The actual physical situation of elephant, and/or the combination of any of the above person) in Properties of Objects (such as size, material, weight, rigidity, Smoothness etc.)；Behavior (such as oscillation, displacement, acceleration, rotation, stretching, extension etc.)；And/or interaction (such as collide, adherency, repel, Attract, friction etc.) tactile output mode tactile output helpful feedback will be provided in some cases for user, subtract Few input error and the efficiency for improving operation of the user to equipment.In addition, tactile output is optionally generated as corresponding to and institute The unrelated feedback of analog physical characteristic (such as input threshold value or Object Selection).Such tactile output will be in some cases use Family provides helpful feedback, reduces input error and improves the efficiency of operation of the user to equipment.

In some embodiments, the tactile output with suitable tactile output mode is served as in the user interface or is being set The prompt of events of interest occurs behind standby middle screen.The example of events of interest includes providing in equipment or in user interface Show and can indicate the activation of (such as true or virtual push button or tumbler switch), request the success or failure operated, arrival or Boundary in user interface, into new state, between objects switch input focus, activation new model, reach or pass through Input threshold value, detection or a type of input of identification or gesture etc..In some embodiments, tactile output is provided to fill When about unless change direction or interrupt input be timely detected, otherwise what can be occurred will occur the warning of event or result Or prompt.Tactile output is also used for abundant user experience under other situations, improvement has vision or dyskinesia or other The user that accessibility needs is to the accessibility of equipment, and/or the efficiency and functionality of improvement user interface and/or equipment.Optionally Tactile is exported and is compared with audio input and/or visual user interface change by ground, this further enhances user and user circle The experience of user when face and/or equipment interaction, and be conducive to the more preferable biography of the information of state about user interface and/or equipment It is defeated, and this reduces input error and improves the efficiency of operation of the user to equipment.

Fig. 4 F- Fig. 4 H, which is provided, (such as to be modulated, put either individually or in combination, as it is or by one or more conversions Greatly, truncation etc.) in various scenes for various purposes (such as those described above and relative to discussed herein Those of described in user interface and method) form one group of sample tactile output mode of suitable tactile feedback.The control of tactile output This example of making sheet shows how one group of three waveform and eight frequencies can be used to generate the array of tactile output mode.It removes Except tactile output mode shown in this Fig, optionally by changing the yield value for tactile output mode come in amplitude Aspect is adjusted in these tactile output modes each, as shown, for example in Fig. 4 I- Fig. 4 K FullTap 80Hz, FullTap 200Hz, MiniTap 80Hz, MiniTap 200Hz, MicroTap 80Hz and MicroTap 200Hz, they are each From the variant for the gain for being shown with 1.0,0.75,0.5 and 0.25.As shown in Fig. 4 I- Fig. 4 K, change tactile output mode Gain with regard to change pattern amplitude, without change pattern frequency or change waveform shape.In some embodiments, change The frequency for becoming tactile output mode is also resulted in compared with low amplitude value, because some tactile output generators are limited to that how much power can be applied It is constrained to compared with low amplitude value in the upper frequency movement of removable mass, therefore mass to ensure acceleration required for forming waveform The power except the operating force range of tactile output generator is not needed (for example, the peak of the FullTap of 230Hz, 270Hz and 300Hz The amplitude for being worth the FullTap of Amplitude Ration 80Hz, 100Hz, 125Hz and 200Hz is low).

Fig. 4 F- Fig. 4 K shows the tactile output mode with specific waveforms.The waveform of tactile output mode indicates opposite In neutral position (such as x_zero) physical displacement to the mode of time, removable mass is generated by the mode has the touching Feel the tactile output of output mode.For example, first group of tactile output mode shown in Fig. 4 F (such as the tactile of " FullTap " Output mode) each to have include that there are two the waveforms of the oscillation of complete cycle (such as to start and end at neutral position for tool It sets and passes through the oscillation of neutral position three times).Second group of tactile output mode shown in Fig. 4 G (such as the touching of " MiniTap " Feel output mode) each to have include that the waveform of the oscillation containing a complete cycle (such as starts and ends at neutral position It sets and passes through the primary oscillation of neutral position).Third group tactile output mode shown in Fig. 4 H (such as the touching of " MicroTap " Feel output mode) each to have include that the waveform of the oscillation containing half of complete cycle (such as starts and ends at neutral position Set and be not passed through the oscillation of neutral position).The waveform of tactile output mode further includes indicating at tactile output beginning and end The starting buffering for ramping up and slowing down and end buffering of removable mass.Example waveform shown in Fig. 4 F- Fig. 4 K includes table Show the x of the minimum and maximum mobile degree of removable mass_minAnd x_maxValue.The removable biggish larger electronics of mass is set Standby, the mobile degree of the minimum and maximum of the mass can be greater or lesser.Embodiment shown in Fig. 4 F- Fig. 4 K describes in 1 dimension The movement of mass, but similar principle will also apply in bidimensional or three-dimensional move the movement of mass.

As shown in Fig. 4 F- Fig. 4 H, each tactile output mode also has a corresponding characteristic frequency, influences user from having " pitch " for the sense of touch that the tactile output of this feature frequency is felt.Continuous tactile is exported, characteristic frequency indicates that tactile is defeated The quantity for the circulation (such as circulation per second) that the removable mass of generator is completed in given time period out.For discrete tactile Output generates discrete output signal (such as recycling with 0.5,1 or 2), and the specified removable mass of characteristic frequency value needs It how soon moves to generate the tactile output with this feature frequency.It is defeated for each type of tactile as shown in Fig. 4 F- Fig. 4 H (such as being limited by respective waveforms, such as FullTap, MiniTap or MicroTap) out, higher frequency values correspond to removable The very fast movement of mass, therefore in general, correspond to the shorter tactile output deadline (for example including completion discrete tactile The time of the required loop number of output adds starting and ending buffer time).For example, characteristic frequency is the FullTap of 80Hz It takes longer to complete (such as in Fig. 4 F, 35.4ms and 28.3ms) than the FullTap that characteristic frequency is 100Hz.In addition, For given frequency, corresponding frequencies have in its waveform the output of more multicycle tactile than in identical corresponding frequencies in its wave There is the tactile less recycled output to take longer to complete in shape.For example, the MiniTap of the FullTap ratio 150Hz of 150Hz It takes longer to complete (such as 19.4ms vs.12.8ms), when the MicroTap of the MiniTap ratio 150Hz of 150Hz spends longer Between complete (such as 12.8ms vs.9.4ms).However for the tactile output mode with different frequency, this rule may not It is applicable in (for example, the tactile output with more multi-cycle but with higher frequency may be than with less circulation but with more low frequency The tactile output of rate spends shorter time quantum to complete, and vice versa).For example, FullTap and MiniTap are spent equally in 300Hz The long time (such as 9.9ms).

As shown in Fig. 4 F- Fig. 4 H, tactile output mode also has feature amplitude, influences the energy for including in haptic signal Amount or user " intensity " of appreciable sense of touch is exported by the tactile with this feature amplitude.In some embodiment party In case, the feature amplitude of tactile output mode refers to that expression moves mass when generating tactile output relative to neutral position The absolute value or normalized value of maximum displacement.In some embodiments, the feature amplitude of tactile output mode can according to (such as It is customized based on user interface situation and behavior) various conditions and/or preconfigured measurement (such as the degree based on input Amount, and/or the measurement based on user interface) it adjusts, such as the gain coefficient by fixing or being dynamically determined (such as between 0 and 1 Between value) adjust.In some embodiments, the measurement based on input (such as Strength Changes measurement or input speed degree Amount) measurement input during the input that triggering generates tactile output feature (such as the characteristic strength that contacts in pressing input Change rate or contact rate travel on touch sensitive surface).In some embodiments, based on the measurement of user interface The spy of (such as cross-border speed measurement) measurement user interface element during the user interface that triggering generates tactile output changes Sign (such as the element passes through hiding or visible borders movement speeds in the user interface).In some embodiments, tactile The feature amplitude of output mode can be modulated by " envelope ", and the peak value of adjacent circulation can have different amplitudes, wherein the above institute One of oscillography shape multiplied by the envelope parameters for changing over time (such as changing to 1 from 0) by further modifying, to generate tactile As the time gradually adjusts the amplitude of the part of tactile output when output.

Although in Fig. 4 F- Fig. 4 H in order to illustrate purpose explanation illustrate in sample tactile output mode specific frequency, Amplitude and waveform, but the tactile output mode with other frequencies, amplitude and waveform can also be used for similar purpose.For example, can make With the waveform having between 0.5 to 4 circulations.Other frequencies in 60Hz-400Hz range can also be used.

It should be appreciated that equipment 100 is only an example of portable multifunction device, and equipment 100 optionally has There are components more more or fewer than shown component, optionally combines two or more components, or optionally there is this The different configurations of a little components or arrangement.Various parts shown in Figure 1A are in hardware, software, firmware or any combination of them Implement in (including one or more signal processing circuits and/or specific integrated circuit).

Memory 102 optionally includes high-speed random access memory, and also optionally includes nonvolatile memory, Such as one or more disk storage equipments, flash memory device or other non-volatile solid state memory equipment.Equipment 100 other component (such as one or more CPU 120 and peripheral device interface 118) to the access of memory 102 optionally It is controlled by Memory Controller 122.

Peripheral device interface 118 can be used for the input peripheral of equipment and output peripheral equipment being couple to memory 102 and one or more CPU 120.It is stored in one or more of operations of processor 120 or execution memory 102 each Kind software program and/or instruction set are to execute the various functions of equipment 100 and handle data.

In some embodiments, peripheral device interface 118, one or more CPU 120 and Memory Controller 122 Optionally realized on one single chip such as chip 104.In some other embodiments, they are optionally in independent chip Upper realization.

RF (radio frequency) circuit 108 sends and receivees the RF signal of also referred to as electromagnetic signal.RF circuit 108 turns electric signal It is changed to electromagnetic signal/by electromagnetic signal and is converted to electric signal, and via electromagnetic signal and communication network and other communication equipments It is communicated.RF circuit 108 optionally includes the well known circuit for executing these functions, including but not limited to aerial system System, RF transceiver, one or more amplifiers, tuner, one or more oscillators, digital signal processor, encoding and decoding core Piece group, subscriber identity module (SIM) card, memory etc..RF circuit 108 comes and network and other alternately through wireless communication Equipment is communicated, these networks are that such as internet (also referred to as WWW (WWW)), Intranet and/or wireless network are (all Such as, cellular phone network, WLAN (LAN) and/or Metropolitan Area Network (MAN) (MAN)).The wireless communication is optionally using a variety of logical Any one of beacon standard, agreement and technology, including but not limited to global system for mobile communications (GSM), enhanced data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), High Speed Uplink Packet access (HSUPA), evolution clear data (EV-DO), HSPA, HSPA+, double unit HSPA (DC- HSPA), long term evolution (LTE), near-field communication (NFC), wideband code division Multiple access (W-CDMA), CDMA (CDMA), time division multiple acess (TDMA), bluetooth, Wireless Fidelity (Wi-Fi) are (for example, IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), internet protocol voice technology (VoIP), Wi-MAX, email protocol are (for example, internet message access association Discuss (IMAP) and/or post office protocol (POP)), instant message (for example, scalable message processing and there are agreement (XMPP), use In instant message and there is Session initiation Protocol (SIMPLE), instant message and the presence service (IMPS) using extension) and/ Or short message service (SMS) or include this document submission date also it is untapped go out communication protocol other are any appropriate Communication protocol.

Voicefrequency circuit 110, loudspeaker 111 and microphone 113 provide the audio interface between user and equipment 100.Audio Circuit 110 receives audio data from peripheral device interface 118, audio data is converted to electric signal, and electric signal transmission is arrived Loudspeaker 111.Loudspeaker 111 converts electrical signals to the audible sound wave of the mankind.Voicefrequency circuit 110 is also received by microphone 113 electric signals converted from sound wave.Voicefrequency circuit 110 converts electrical signals to audio data, and audio data is transferred to Peripheral device interface 118 is for handling.Audio data is optionally retrieved from and/or is transmitted to by peripheral device interface 118 and deposited Reservoir 102 and/or RF circuit 108.In some embodiments, voicefrequency circuit 110 further includes earphone jack (for example, in Fig. 2 212).Earphone jack provides the interface between voicefrequency circuit 110 and removable audio input/output peripheral equipment, this can The earphone or have output (for example, single head-receiver or ears ear that the audio input of removal/output peripheral equipment such as only exports Machine) and input both (for example, microphone) headset.

I/O subsystem 106 is by such as touch-sensitive display system 112 of the input/output peripheral equipment in equipment 100 and other Input or control equipment 116 and peripheral device interface 118 couple.I/O subsystem 106 optionally include display controller 156, Optical sensor controller 158, intensity sensor controller 159, tactile feedback controller 161 and for other input or control One or more input controllers 160 of control equipment.One or more of input controllers 160 are set from other inputs or control Standby 116 reception electric signal/send other described inputs for electric signal or control equipment.Other input control apparatus 116 are appointed Selection of land includes physical button (for example, pushing button, rocker buttons etc.), dial, slide switch, control stick, click wheel etc..In In some alternative embodiments, one or more input controllers 160 be optionally coupled to any one of the following terms (or It is not coupled to any one of the following terms): keyboard, infrared port, USB port, stylus, and/or pointing device such as mouse Mark.One or more buttons (for example, 208 in Fig. 2) optionally include the volume for loudspeaker 111 and/or microphone 113 The up/down button of control.One or more buttons, which optionally include, pushes button (for example, 206 in Fig. 2).

Touch-sensitive display system 112 provides the input interface and output interface between equipment and user.Display controller 156 Electric signal is received from touch-sensitive display system 112 and/or electric signal is sent to touch-sensitive display system 112.Touch-sensitive display System 112 shows visual output to user.Visual output optionally includes figure, text, icon, video and theirs is any It combines (being referred to as " figure ").In some embodiments, the visual output of some visual outputs or whole corresponds to user circle In face of as.As used herein, term " showing can indicate " refers to user's interactive graphical user interface object (for example, being configured as The graphical user interface object that the input for being led to graphical user interface object is responded).User interactive graphics user The example of interface object includes but is not limited to button, sliding block, icon, optional menu item, switch, hyperlink or other users circle Face control.

Touch-sensitive display system 112, which has, receives the touch-sensitive of input from the user based on tactile and/or tactile contact Surface, sensor or sensor group.Touch-sensitive display system 112 and display controller 156 (with it is any in memory 102 Associated module and/or instruction set are together) contact (and any movement of the contact in detection touch-sensitive display system 112 Or interrupt), and the contact that will test is converted to and is displayed on the user interface object in touch-sensitive display system 112 The interaction of (for example, one or more soft-key buttons, icon, webpage or image).In some embodiments, in touch-sensitive display system Contact point between system 112 and user corresponds to the finger or stylus of user.

Touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer displays) Technology or LED (light emitting diode) technology, but other display technologies are used in other embodiments.Touch-sensitive display system 112 and display controller 156 optionally using in the currently known or later a variety of touch-sensing technologies that will be developed appoint What technology and other proximity sensor arrays or for determine the one or more points contacted with touch-sensitive display system 112 its His element is including but not limited to capacitive, electric to detect contact and its any movement or interruption, a variety of touch-sensing technologies Resistive, infrared ray and surface acoustic wave technique.In some embodiments, using projection-type mutual capacitance detection technology, such as From Apple Inc.'s (Cupertino, California)iPodWithThe technology of middle discovery.

Touch-sensitive display system 112 is optionally with the video resolution for being more than 100dpi.In some embodiments, it touches Touching screen video resolution is more than 400dpi (for example, for 500dpi, 800dpi or bigger).User optionally uses any suitable Object or additives stylus, finger etc. are contacted with touch-sensitive display system 112.In some embodiments, by user circle Face is designed to work together with contact and gesture based on finger, since the contact area of finger on the touchscreen is larger, This may be accurate not as good as the input based on stylus.In some embodiments, equipment converts the rough input based on finger to Accurate pointer/cursor position or order are for executing the desired movement of user.

In some embodiments, in addition to a touch, equipment 100 optionally includes specific for activating or deactivating The Trackpad (not shown) of function.In some embodiments, Trackpad is the touch sensitive regions of equipment, different from touch screen, should Touch sensitive regions do not show visual output.Touch tablet is optionally the touch sensitive surface separated with touch-sensitive display system 112, either By the extension for the touch sensitive surface that touch screen is formed.

Equipment 100 further includes the electric system 162 for powering for various parts.Electric system 162 optionally includes electricity Power management system, one or more power supply (for example, battery, alternating current (AC)), recharging system, power failure detection circuit, Power converter or inverter, power supply status indicator (for example, light emitting diode (LED)) and in portable device Electric power generates, managees, and distributes any other associated component.

Equipment 100 optionally further comprising one or more optical sensor 164 (for example, as one or more cameras A part).Figure 1A shows the optical sensor coupled with the optical sensor controller 158 in I/O subsystem 106.Optics passes Sensor 164 optionally includes charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) phototransistor.One A or multiple optical sensors 164 receive the light projected by one or more lens from environment, and convert light to table The data of diagram picture.In conjunction with image-forming module 143 (being also designated as camera model), one or more optical sensors 164 are optionally Capture still image and/or video.In some embodiments, optical sensor be located at equipment 100 on equipment front On the opposite facing rear portion of touch-sensitive display system 112, so that touch screen can be used as still image and/or video image acquisition View finder.In some embodiments, another optical sensor is located on the front of equipment, to obtain the image of the user (for example, being used for self-timer, for carrying out video conference etc. when user watches other video conference participants on the touchscreen).

Equipment 100 optionally further includes one or more contact strength sensors 165.Figure 1A is shown and I/O subsystem The contact strength sensor that intensity sensor controller 159 in 106 couples.One or more contact strength sensors 165 are appointed Selection of land includes one or more piezoresistive strain instrument, capacitive force transducer, electric force snesor, piezoelectric force transducer, optics Force snesor, capacitive touch sensitive surfaces or other intensity sensors (for example, for measuring the contact on touch sensitive surface power (or Pressure) sensor).Contact strength sensor 165 receives contact strength information (for example, pressure information or pressure letter from environment Breath agency).In some embodiments, at least one contact strength sensor and touch sensitive surface are (for example, touch-sensitive display system 112) Alignment or neighbouring.In some embodiments, at least one contact strength sensor be located at equipment 100 be located at On the opposite facing rear portion of touch-sensitive display system 112 on the front of equipment 100.

Equipment 100 optionally further includes one or more proximity sensors 166.Figure 1A is shown and peripheral device interface The proximity sensor 166 of 118 couplings.Alternatively, 160 coupling of input controller in proximity sensor 166 and I/O subsystem 106 It connects.In some embodiments, when multifunctional equipment is placed near user's ear (for example, when user is making a phone call), Proximity sensor closes and disables touch-sensitive display system 112.

Equipment 100 is optionally further comprising one or more tactile output generators 163.Figure 1A is shown and I/O subsystem The tactile output generator that tactile feedback controller 161 in 106 couples.In some embodiments, one or more tactiles Output generator 163 includes one or more electroacoustic equipment such as loudspeaker or other acoustic components；And/or for energy to be turned Change into the electromechanical equipment of linear movement such as motor, solenoid, electroactive polymerizer, piezoelectric actuator, electrostatic actuator or its His tactile exports generating unit (for example, component for converting the electrical signal to the output of the tactile in equipment).Tactile output hair Raw device 163 receives touch feedback from haptic feedback module 133 and generates instruction, and generating on the appliance 100 can be by equipment 100 User feel tactile output.In some embodiments, at least one tactile output generator and touch sensitive surface (example Such as, touch-sensitive display system 112) Alignment or neighbouring, and optionally by vertically (for example, to the table of equipment 100 Face inside/outside) or laterally (for example, in plane identical with the surface of equipment 100 rearwardly and a forwardly) mobile touch sensitive surface Generate tactile output.In some embodiments, at least one tactile output generator sensor be located at equipment 100 be located at On the opposite facing rear portion of touch-sensitive display system 112 on the front of equipment 100.

Equipment 100 optionally further includes one or more for obtaining the information of the position (for example, posture) about equipment A accelerometer 167, gyroscope 168 and/or magnetometer 169 (for example, a part as Inertial Measurement Unit (IMU)).Figure 1A shows the sensor 167,168 and 169 coupled with peripheral device interface 118.Alternatively, sensor 167,168 and 169 Input controller 160 in selection of land and I/O subsystem 106 couples.In some embodiments, based on to from the one or more The analysis of accelerometer received data on touch-screen display shows information with longitudinal view or transverse views.If Standby 100 GPS (or GLONASS or other worldwide navigation systems for optionally including the information for obtaining the position about equipment 100 System) receiver (not shown).

In some embodiments, the software component being stored in memory 102 includes operating system 126, communication module (or instruction set) 128, contact/motion module (or instruction set) 130, figure module (or instruction set) 132, haptic feedback module (or instruction set) 133, text input module (or instruction set) 134, global positioning system (GPS) module (or instruction set) 135, with And application program (or instruction set) 136.In addition, in some embodiments, memory 102 stores equipment/overall situation internal state 157, if figure is shown in the 1A and Fig. 3.Equipment/overall situation internal state 157 includes one or more of the following: activity application Program state indicates which application program (if any) is currently movable；What display state indicates using journey Sequence, view or other information occupy each region of touch-sensitive display system 112；Sensor states, including from each of equipment The information that sensor and other inputs or control equipment 116 obtain；And about equipment position and/or posture position and/ Or azimuth information.

Operating system 126 (for example, iOS, Android, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS or Embedded OS such as VxWorks) it include for controlling and managing general system task (for example, memory management, storage Equipment control, power management etc.) various component softwares and/or driver, and be conducive to various hardware and software components it Between communication.

Communication module 128 promotes through one or more outside ports 124 to be communicated with other equipment, and also wraps It includes for handling by the various software components of 124 received data of RF circuit 108 and/or outside port.Outside port 124 (for example, universal serial bus (USB), firewire etc.) is suitable for being directly coupled to other equipment or indirectly via network (for example, mutually Networking, Wireless LAN etc.) coupling.In some embodiments, outside port be with Apple Inc. (Cupertino, California) someiPodThe identical or class with 30 needle connectors used in iPod equipment Like and/or compatible spininess (for example, 30 needles) connector.In some embodiments, outside port is and Apple Inc. (Cupertino, California's) is someiPod With used in iPod equipment The same or like and/or compatible Lightning connector of Lightning connector.In some embodiments, outer end Mouthful it is and USB C type connector used in some electronic equipments of Apple Inc. (Cupertino, California) Same or like and/or compatible USB C type connector.

Contact/motion module 130 optionally detects and touch-sensitive display system 112 (in conjunction with display controller 156) and its The contact of his touch-sensitive device (for example, touch tablet or physics click wheel).Contact/motion module 130 include various software components with The relevant various operations of detection are contacted with (such as passing through finger or stylus) for executing, such as to determine that whether being in contact (for example, detection finger down event), the intensity of determining contact (for example, the power or pressure of contact, or the power or pressure of contact Substitute), determine whether there is the movement of contact and track the movement across touch sensitive surface (for example, detecting one or more hands Refer to drag events), and determine contact (for example, detection finger is lifted away from event or contact disconnects) whether it has stopped.Contact/fortune Dynamic model block 130 receives contact data from touch sensitive surface.Determine that the movement of contact point optionally includes the rate (amount of determining contact point Value), speed (magnitude and direction) and/or acceleration (change in magnitude and/or direction), the movement of the contact point is by a series of Contacting data indicates.These operations are optionally applied to single-contact (for example, single abutment or stylus contact) or multiple spot Contact (for example, " multiple point touching "/more abutments) simultaneously.In some embodiments, contact/motion module 130 and display control Device 156 processed detects the contact on touch tablet.

Contact/motion module 130 optionally detects the gesture input of user.Different gestures on touch sensitive surface have difference Contact mode (for example, the different motion of detected contact, timing and/or intensity).Therefore, special optionally by detection Determine contact mode and carrys out detection gesture.For example, detection singly refers to that Flick gesture includes detection finger down event, then pressed with finger Detection finger lifts and (is lifted away from) thing at (for example, at picture mark position) the identical position (or substantially the same position) of lower event Part.For another example, detecting the finger on touch sensitive surface and gently sweeping gesture includes detection finger down event, then detects one or more hands Refer to drag events, and then detection finger lifts and (be lifted away from) event.Similarly, by detect stylus specific contact patterns come The tap of stylus is optionally detected, gently sweeps, drag and other gestures.

In some embodiments, detection finger Flick gesture depends on detection finger down event and digit up event Between time span, but detection finger down event and digit up event between finger contact strength it is unrelated.In In some embodiments, it is less than according to the time span determined between finger down event and digit up event predetermined It is worth (for example, less than 0.1,0.2,0.3,0.4 or 0.5 second), detects Flick gesture, is contacted but regardless of finger during tap strong Whether degree reaches given intensity threshold (being greater than Nominal contact detection intensity threshold value), such as light press or deep by Compressive Strength threshold Value.Therefore, finger Flick gesture can satisfy specific input standard, which does not require the characteristic strength of contact full The given intensity threshold of foot is to meet specific input standard.For clarity, the finger contact in Flick gesture usually requires to meet Nominal contact detection intensity threshold value is to detect finger down event, when being lower than the Nominal contact detection intensity threshold value, Bu Huijian Measure contact.Similar analysis is suitable for through stylus or other contact detection Flick gestures.It is able to detect in equipment touch-sensitive Hover on surface finger or stylus contact in the case where, Nominal contact detection intensity threshold value optionally not with finger or stylus Physical contact between touch sensitive surface is corresponding.

Same concept is suitable for other kinds of gesture in a similar manner.For example, based on satisfaction and can include in gesture Contact intensity it is unrelated or the one or more contacts for executing gesture is not required to reach intensity threshold so as to identified mark Standard gently sweeps gesture, kneading gesture, expansion gesture and/or long pressing gesture optionally to detect.For example, connecing based on one or more The amount of the movement of touching gently sweeps gesture to detect；Scaling gesture is detected based on two or more movements of contact towards each other；Expand Gesture of letting go contacts movement away from each other based on two or more to detect；Long pressing gesture is based on having on touch sensitive surface few It is detected in duration of the contact of amount of threshold shift.Do not require contact strength full accordingly, with respect to certain gestures criterion of identification The corresponding intensity threshold of foot means that certain gestures criterion of identification can be in gesture to meet the statement of certain gestures criterion of identification In contact be satisfied when being not up to corresponding intensity threshold, and can also reach in one or more contacts in gesture or It is satisfied in the case where more than corresponding intensity threshold.In some embodiments, it is examined in time predefined section based on determining Finger down event and digit up event are measured to detect Flick gesture, without considering that contact is during time predefined section Gesture gently is swept to detect greater than predefined magnitude above or below corresponding intensity threshold, and based on determining that contact is mobile, Even if it is also such that contact, which is higher than corresponding intensity threshold, at the end of contacting mobile.Even if being executed in the detection to gesture (for example, equipment is quickly when the intensity of contact is higher than intensity threshold in the specific implementation of the influence of the intensity of the contact of gesture Detect long pressing, or when the intensity of contact is higher, equipment can postpone the detection to tap input), as long as contacting not The standard that can satisfy identification gesture in the case where reaching certain strength threshold value, then will not require to connect to the detection of these gestures Touching reaches certain strength threshold value (for example, even if time quantum needed for identification gesture changes).

In some cases, contact strength threshold value, duration threshold and mobile threshold value carry out group with various various combinations It closes, distinguishes gestures different for two or more of identical input element or region to create heuritic approach, so that The set of richer user interaction and response is capable of providing with multiple and different interactions of identical input element.It is specific about one group Gesture identification standard does not require one or more intensity contacted to meet corresponding intensity threshold to meet certain gestures identification The statement of standard is not excluded for that other intensity related gesture criterion of identification are carried out while being assessed, and has with identification when gesture includes tool There are other gestures for the standard being satisfied when the contact higher than the intensity of respective strengths threshold value.For example, in some cases, first (it does not require the intensity of contact to meet corresponding intensity threshold to meet first gesture identification to the first gesture criterion of identification of gesture Standard) it is competed with the second gesture criterion of identification (it depends on the contact for reaching respective strengths threshold value) of second gesture.Such In competition, if the second gesture criterion of identification of second gesture is met first, gesture is not identified as optionally meeting The first gesture criterion of identification of first gesture.For example, if contact reaches corresponding before the mobile predefined amount of movement of contact Intensity threshold, then detect deep pressing gesture rather than gently sweep gesture.On the contrary, if reaching corresponding intensity threshold in contact The mobile predefined amount of movement of contact before then detects and gently sweeps gesture rather than deep pressing gesture.Even if in such cases, It is first-hand to meet that the first gesture criterion of identification of first gesture does not still require the intensity of contact to meet corresponding intensity threshold Gesture criterion of identification, because if contact keeps below corresponding intensity threshold until gesture terminates (for example, having will not increase to Light higher than the contact of the intensity of respective strengths threshold value sweeps gesture), gesture will be identified as gently sweeping hand by first gesture criterion of identification Gesture.Therefore, it is not required to which the intensity of contact meets corresponding intensity threshold to meet the knowledge of the certain gestures of certain gestures criterion of identification Other standard will (A) in some cases, ignore the contact strength (for example, for Flick gesture) relative to intensity threshold And/or (B) is in some cases, if before certain gestures criterion of identification identifies gesture corresponding with input, one group of competition Intensity related gesture criterion of identification (for example, for pressing gesture deeply) input is identified as and intensity related gesture pair It answers, is not then able to satisfy certain gestures criterion of identification (for example, for long pressing gesture), in this sense, still Depending on relative to intensity threshold contact strength (for example, for the long pressing gesture with deep pressing gesture competition identification and Speech).

In conjunction with accelerometer 167, gyroscope 168 and/or magnetometer 169, gesture module 131 is optionally detected about equipment Posture information, such as in the posture (such as rolling, flexion-extension and/or yaw) with particular reference to equipment in being.Gesture module 131 wraps Include the software component for executing various operations relevant to detection device position and detection device attitudes vibration.

Figure module 132 includes for figure to be rendered and shown in touch-sensitive display system 112 or other displays Various known software components, including for changing shown figure visual impact (for example, brightness, transparency, saturation degree, Contrast or other perceptual properties) component.As used herein, term " figure " includes can be displayed to user any right As without limitation including text, webpage, icon (such as including the user interface object of soft key), digital picture, video, moving Draw etc..

In some embodiments, figure module 132 stores the data for indicating figure ready for use.Each figure is optionally It is assigned corresponding code.Figure module 132 is used to specify one or more of figure to be shown from receptions such as application programs A code also receives coordinate data and other graphic attribute data together in the case of necessary, and then generates screen map As data, with output to display controller 156.

Haptic feedback module 133 includes for generating instruction (for example, the instruction used by tactile feedback controller 161) Various software components, tactile output generator 163 is used on the appliance 100 with the interaction in response to user and equipment 100 One or more positions generate tactile output.

The text input module 134 for being optionally the component of figure module 132 is provided in various application program (examples Such as, contact person 137, Email 140, IM 141, browser 147 and any other application program for needing text input) in Input the soft keyboard of text.

GPS module 135 determines the position of equipment and provides this information to use in various application programs (for example, mentioning It is supplied to the phone 138 for location-based dialing；It is provided to camera 143 and is used as picture/video metadata；And it is provided to For location based service such as weather desktop small routine, local Yellow Page desktop small routine and map/navigation desktop small routine Application program).

Virtually/augmented reality module 145 provides virtual and/or augmented reality logical block, the logic to application program 136 Component realizes augmented reality, and realizes virtual reality feature in some embodiments.Virtually/augmented reality module 145 promotees Into superposition of the virtual content in at least part of expression in the visual field of one or more cameras, virtual content is such as virtually used Family interface object (for example, virtual tape measure for carrying out the measurement based on augmented reality).For example, in virtual/augmented reality mould With the help of block 145, at least part of expression in the visual field of one or more cameras may include corresponding physical object, and And visual user interface object can be shown in the visual field of camera based on one or more in the augmented reality environment of display The position that determines of respective physical object at, or be shown in at least part of posture based on computer system (for example, being used for To the user of computer system show user interface display equipment posture) determination reality environment in.

Application program 136 is optionally included with lower module (or instruction set) or its subset or superset:

Contact module 137 (sometimes referred to as address list or contacts list)；

Phone module 138；

Video conference module 139；

Email client module 140；

Instant message (IM) module 141；

Body-building support module 142；

For still image and/or the camera model 143 of video image；

Image management module 144；

Browser module 147；

Calendaring module 148；

Desktop small routine module 149, optionally includes one or more of the following terms: weather desktop small routine 149-1, stock market desktop small routine 149-2, calculator desktop small routine 149-3, alarm clock desktop small routine 149-4, dictionary desktop The desktop small routine 149-6 of small routine 149-5 and other desktop small routines and user's creation for being obtained by user；

It is used to form the desktop small routine builder module 150 of the desktop small routine 149-6 of user's creation；

Search module 151；

The video and musical player module being optionally made of video player module and musical player module 152；

Notepad module 153；

Mapping module 154；And/or

Measurement module 155.

The example for the other applications 136 being optionally stored in memory 102 includes other text processing application journeys Other picture editting's application programs, drawing application program, application program, the application program for supporting JAVA, encryption, number is presented in sequence Word rights management, speech recognition and speech reproduction.

It is defeated in conjunction with touch-sensitive display system 112, display controller 156, contact module 130, figure module 132 and text Enter module 134, contact module 137 includes executable instruction for managing address list or contacts list (for example, being stored in In the application program internal state 192 of contact module 137 in reservoir 102 or memory 370), comprising: addition name arrives Address list；Name is deleted from address book；Telephone number, e-mail address, physical address or other information are associated with name； Image is associated with name；Name is sorted out and is classified；Telephone number and/or e-mail address are provided initiate and/ Or promote to pass through phone 138, video conference 139, the communication of Email 140 or instant message 141；Etc..

In conjunction with RF circuit 108, voicefrequency circuit 110, loudspeaker 111, microphone 113, touch-sensitive display system 112, display Controller 156, contact module 130, figure module 132 and text input module 134, phone module 138 include for carry out with The executable instruction of lower operation: the one or more electricity in input character string corresponding with telephone number, accessing address list 137 Words number, modification inputted telephone number, dial corresponding telephone number, conversate and when session complete when disconnection Or it hangs up.As described above, wireless communication is optionally using any one of a variety of communication standards, agreement and technology.

In conjunction with RF circuit 108, voicefrequency circuit 110, loudspeaker 111, microphone 113, touch-sensitive display system 112, display control Device 156 processed, one or more optical sensors 164, optical sensor controller 158, contact module 130, figure module 132, Text input module 134, contacts list 137 and phone module 138, video conference module 139 include coming according to user instructions Initiate, carry out and terminate the executable instruction of the video conference between user and other one or more participants.

In conjunction with RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, figure module 132 and text input module 134, email client module 140 include for creating, sending in response to user instruction, Receive and manage the executable instruction of Email.In conjunction with image management module 144, email client module 140 makes It is very easy to creation and sends the Email with the still image or video image that are shot by camera model 143.

In conjunction with RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, figure module 132 and text input module 134, instant message module 141 include the executable instruction for performing the following operation: input with i.e. When message corresponding character string, modification be previously entered character, the corresponding instant message of transmission is (for example, using for based on electricity The short message service (SMS) or multimedia messaging service (MMS) agreement of the instant message of words are based on internet using being directed to Instant message XMPP, SIMPLE, Apple push notification service (APNs) or IMPS), receive instant message, and check The received instant message of institute.In some embodiments, the instant message transferred and/or received optionally includes figure, phase Other attachments for being supported in piece, audio file, video file, and/or MMS and/or enhancing messaging service (EMS).Such as this Used in text, " instant message " is referred to message (for example, the message sent using SMS or MMS) based on phone and based on internet Message (for example, use XMPP, SIMPLE, APNs or IMPS send message) both.

In conjunction with RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, figure module 132, text input module 134, GPS module 135, mapping module 154 and video and musical player module 152, body-building branch Holding module 142 includes executable instruction for creating body-building (for example, having time, distance and/or caloric burn target)；With The communication of (in sporting equipment and smartwatch) body-building sensor；Receive workout sensor data；Calibration is for monitoring body-building Sensor；Music is selected and played for body-building；And it shows, store and transmit workout data.

In conjunction with touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensing Device controller 158, contact module 130, figure module 132 and image management module 144, camera model 143 include for carrying out The executable instruction operated below: still image or video (including video flowing) are captured and stores them in memory 102 In, the feature of modification still image or video, and/or delete still image or video from memory 102.

In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, figure module 132, text input Module 134 and camera model 143, image management module 144 include for arranging, modifying (for example, editor) or with its other party Formula manipulates, tags, deleting, showing (for example, in digital slide or photograph album) and storage still image and/or video The executable instruction of image.

In conjunction with RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, figure mould Block 132 and text input module 134, browser module 147 include according to user instructions come browse internet (including search, chain Be connected to, receive and show webpage or part thereof and be linked to the attachment and alternative document of webpage) executable instruction.

In conjunction with RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, figure mould Block 132, text input module 134, email client module 140 and browser module 147, calendaring module 148 include using According to user instructions come create, show, modify and store calendar and data associated with calendar (for example, calendar, Backlog etc.) executable instruction.

In conjunction with RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, figure mould Block 132, text input module 134 and browser module 147, desktop small routine module 149 are optionally to be downloaded and made by user Miniature applications program is (for example, weather desktop small routine 149-1, stock market desktop small routine 149-2, calculator desktop little Cheng Sequence 149-3, alarm clock desktop small routine 149-4 and dictionary desktop small routine 149-5) or by user create miniature applications program (for example, desktop small routine 149-6 of user's creation).In some embodiments, desktop small routine includes HTML (hypertext mark Remember language) file, CSS (cascading style sheets) file and JavaScript file.In some embodiments, desktop small routine Including XML (extensible markup language) file and JavaScript file (for example, Yahoo！Desktop small routine).

In conjunction with RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, figure mould Block 132, text input module 134 and browser module 147, desktop small routine builder module 150 include for creating desktop The executable instruction of small routine (for example, user's specified portions of webpage are gone in desktop small routine).

In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, figure module 132 and text Input module 134, search module 151 include for carrying out searching in searching storage 102 with one or more according to user instructions The matched text of rope condition (for example, search term that one or more user specifies), music, sound, image, video and/or its The executable instruction of his file.

In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, figure module 132, audio-frequency electric Road 110, loudspeaker 111, RF circuit 108 and browser module 147, video and musical player module 152 include allowing user Download and play back the music recorded stored with one or more file formats (such as MP3 or AAC file) and other sound The executable instruction of file, and for showing, presenting or otherwise play back video (for example, in touch-sensitive display system 112 Executable instruction above or on the external display being wirelessly connected via outside port 124).In some embodiments, if Standby 100 optionally include the function of MP3 player such as iPod (trade mark of Apple Inc.).

It is defeated in conjunction with touch-sensitive display system 112, display controller 156, contact module 130, figure module 132 and text Enter module 134, notepad module 153 includes can come create and manage notepad, backlog etc. for according to user instructions It executes instruction.

In conjunction with RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, figure mould Block 132, text input module 134, GPS module 135 and browser module 147, mapping module 154 include for according to user Instruction is to receive, show, modify and store map and data associated with map (for example, driving route；Specific location Or the data in neighbouring shop and other points of interest；With other location-based data) executable instruction.

In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, figure module 132 and virtual/ Augmented reality module 145, measurement module 155 include that user is allowed to measure physical space and/or wherein in augmented reality environment Object executable instruction, it is as described in more detail.

Above-mentioned identified each module and application correspond to for executing above-mentioned one or more functions and in this Shen Please described in one group of method (for example, computer implemented method described herein and other information processing method) Executable instruction.These modules (that is, instruction set) need not be realized with independent software program, process, or module, therefore these moulds Each subset of block is optionally combined in various embodiments or is otherwise rearranged.In some embodiments, Memory 102 optionally stores the subgroup of above-mentioned module and data structure.It is not retouched above in addition, memory 102 is optionally stored The other module and data structure stated.

In some embodiments, equipment 100 is that the operation of predefined one group of function in the equipment uniquely passes through Touch screen and/or touch tablet are performed equipment.By using touch screen and/or Trackpad as operating equipment 100 Main input control apparatus is physically entered control equipment (for example, pushing button, driver plate etc. optionally in reduction equipment 100 Deng) quantity.

Predefined one group of function is uniquely performed by touch screen and/or Trackpad and is optionally included in user circle Navigation between face.In some embodiments, touch tablet shows equipment 100 when being touched by user from equipment 100 Any user interface navigation is to main menu, home menu or root menu.In such embodiment, realized using touch sensitive surface " menu button ".In some other embodiments, menu button is that physics pushes button or other are physically entered control and set It is standby, rather than touch sensitive surface.

Figure 1B is the block diagram for showing the example components for event handling according to some embodiments.In some implementations In scheme, memory 102 (in Figure 1A) or memory 370 (Fig. 3 A) include event classifier 170 (for example, in operating system 126 In) and corresponding application program 136-1 (for example, any of aforementioned applications program 136,137 to 155,380 to 390 apply Program).

Event classifier 170 receives event information and determination for application program 136-1 that event information is delivered to and answers With the application view 191 of program 136-1.Event classifier 170 includes event monitor 171 and event dispatcher module 174.In some embodiments, application program 136-1 includes application program internal state 192, the application program internal state The one or more that indicating is movable when application program or while being carrying out shows in touch-sensitive display system 112 is currently answered Use Views.In some embodiments, equipment/overall situation internal state 157 by event classifier 170 is used to which is determined (which) application program is currently movable, and application program internal state 192 is used to determination by event classifier 170 and incites somebody to action The application view 191 that event information is delivered to.

In some embodiments, application program internal state 192 includes additional information, such as one of the following terms Or more persons: when application program 136-1 restores to execute, recoverys information to be used, instruction information are just being shown or are preparing handy In the user interface state information shown by application program 136-1, for allowing users to back to application program 136-1's The repetition for the prior actions that the state queue and user of previous state or view are taken/revocation queue.

Event monitor 171 receives event information from peripheral device interface 118.Event information includes about subevent (example Such as, as in the touch-sensitive display system 112 of a part of multi-touch gesture user touch) information.Peripheral equipment connects Mouthfuls 118 transmission its from I/O subsystem 106 or sensor such as proximity sensor 166, one or more accelerometers 167 and/or Microphone 113 (passing through voicefrequency circuit 110) received information.Peripheral device interface 118 is from the received letters of the institute of I/O subsystem 106 Breath includes the information from touch-sensitive display system 112 or touch sensitive surface.

In some embodiments, event monitor 171 sends the request to peripheral equipment at predetermined intervals and connects Mouth 118.In response, 118 transmitting event information of peripheral device interface.In other embodiments, peripheral device interface 118 is only When there are significant events (for example, receiving the input higher than predetermined noise threshold and/or receiving is more than to predefine Duration input) when ability transmitting event information.

In some embodiments, event classifier 170 further includes hit view determination module 172 and/or life event Identifier determining module 173.

When touch-sensitive display system 112 shows more than one view, hit view determination module 172 is provided for true The stator event software process where occurred in one or more views.View can be over the display by user The control and other elements seen are constituted.

The another aspect of user interface associated with application program is one group of view, otherwise referred to as applies journey herein Sequence view or user interface windows are wherein showing information and the gesture based on touch occur.Wherein detecting touch (corresponding application programs) application view optionally corresponds in the sequencing or view hierarchies structure of application program Sequencing is horizontal.For example, being optionally referred to as hit view in the floor level view for wherein detecting touch, and it is identified Event set to correctly enter is based in part on the hit view of initial touch optionally at least to determine, the initial touch is opened Primordium is in the gesture of touch.

It hits view determination module 172 and receives information relevant to the subevent of the gesture based on touch.Work as application program When with multiple views with hierarchical structure tissue, hit view determination module 172 will hit that view is identified as should be to sub- thing Minimum view in the hierarchical structure that part is handled.In most cases, hit view is to initiate subevent (to form thing The first subevent in the subevent sequence of part or potential event) in the floor level view wherein occurred.Once hitting view It is hit view determination module to be identified, hit view, which is just usually received, to be identified as hitting the targeted same touching of view with it It touches or the relevant all subevents of input source.

It is specific that life event identifier determining module 173 determines which or which view in view hierarchies structure should receive Subevent sequence.In some embodiments, life event identifier determining module 173 determines that only hit view should receive spy Stator sequence of events.In other embodiments, life event identifier determining module 173 determines the physical bit including subevent All views set are the active views participated in, and it is thus determined that all views actively participated in should all receive specific subevent sequence Column.In other embodiments, it even if touch subevent is confined to region associated with a particular figure completely, is classified Higher view in structure will still maintain view for active participation.

Event information is assigned to event recognizer (for example, event recognizer 180) by event dispatcher module 174.It is wrapping In the embodiment for including life event identifier determining module 173, event information is delivered to by living by event dispatcher module 174 Dynamic 173 definite event identifier of event recognizer determining module.In some embodiments, event dispatcher module 174 exists Event information is stored in event queue, which is retrieved by corresponding event receiver module 182.

In some embodiments, operating system 126 includes event classifier 170.Alternatively, application program 136-1 packet Include event classifier 170.In yet another embodiment, event classifier 170 is standalone module, or is stored in storage A part of another module (such as, contact/motion module 130) in device 102.

In some embodiments, application program 136-1 includes multiple button.onreleases 190 and one or more application Views 191, each of these all includes occurring in the corresponding views of the user interface of application program for handling The instruction of touch event.Each application view 191 of application program 136-1 includes one or more event recognizers 180. In general, corresponding application programs view 191 includes multiple event recognizers 180.In other embodiments, event recognizer 180 In one or more event recognizers be standalone module a part, which is that such as user interface tool packet is not ( Show) or the application program 136-1 therefrom higher level of inheritance method and other attributes object.In some embodiments, Corresponding event processing routine 190 includes one or more of the following terms: data renovator 176, object renovator 177, GUI Renovator 178, and/or from the received event data 179 of event classifier 170.Button.onrelease 190 optionally with or adjust With data renovator 176, object renovator 177 or GUI renovator 178 come more new application internal state 192.Alternatively, One or more application views in application view 191 include one or more corresponding event processing routines 190.Separately Outside, in some embodiments, one or more of data renovator 176, object renovator 177 and GUI renovator 178 wrap It includes in corresponding application programs view 191.

Corresponding event recognizer 180 receives event information (for example, event data 179) from event classifier 170, and From event information identification events.Event recognizer 180 includes Event receiver 182 and event comparator 184.In some implementations In scheme, event recognizer 180 further include metadata 183 and event transmitting instruction 188 (its optionally include subevent delivering Instruction) at least one subset.

Event receiver 182 receives the event information from event classifier 170.Event information includes about subevent example As touched or touching mobile information.According to subevent, event information further includes additional information, the position of such as subevent.When When subevent is related to the movement touched, event information optionally further includes rate and the direction of subevent.In some embodiments In, event include equipment from an orientation rotate to another orientation (for example, rotate to horizontal orientation from machine-direction oriented, or vice versa ), and event information includes the corresponding informance of the current orientation (also referred to as equipment posture) about equipment.

Event information and predefined event or subevent definition are compared by event comparator 184, and being based on should Compare to determine event or subevent, or the state of determining or update event or subevent.In some embodiments, event Comparator 184 includes that event defines 186.Event defines 186 definition (for example, predefined subevent sequence) comprising event, Such as event 1 (187-1), event 2 (187-2) and other.In some embodiments, the subevent in event 187 is for example Start including touching, touch and terminate, touch mobile, touch cancellation and multiple point touching.In one example, event 1 (187-1) Definition is the double-click on shown object.It is touched for example, double-clicking the first time including the predetermined duration on shown object (touch starts), the first time for predefining duration lift (touch terminates), predetermined duration on shown object the It is secondary touch (touch starts) and predetermined duration lift (touch terminates) for the second time.In another example, event 2 The definition of (187-2) is the dragging on shown object.For example, dragging includes the touching of the predetermined duration on shown object That touches the movement and touch of (or contact), touch in touch-sensitive display system 112 is lifted away from (touch terminates).In some realities It applies in scheme, event further includes the information for one or more associated button.onreleases 190.

In some embodiments, it includes the definition to the event for respective user interfaces object that event, which defines 187,.In In some embodiments, event comparator 184 executes hit test to determine which user interface object is associated with subevent. For example, being shown in touch-sensitive display system 112 in the application view of three user interface objects, when in touch-sensitive display When detecting touch in system 112, event comparator 184 executes hit Test to determine which in these three user interface objects One user interface object is associated with touch (subevent).If each shown object and corresponding event processing routine 190 is associated, then event comparator determines which button.onrelease 190 should be swashed using the result that the hit is tested It is living.For example, the selection of event comparator 184 button.onrelease associated with the object of subevent and triggering hit test.

It in some embodiments, further include delay voltage to the definition of corresponding event 187, which postpones event The delivering of information, until having determined that whether subevent sequence exactly corresponds to or do not correspond to the event type of event recognizer.

It, should when the determining subevent sequence of corresponding event identifier 180 does not define any event in 186 with event to be matched 180 entry event of corresponding event identifier is impossible, event fails or event terminates state, ignores after this based on touch The subsequent subevent of gesture.In this case, for hit view keep other movable event recognizers (if there is Words) continue to track and handle the subevent of the gesture based on touch of lasting progress.

In some embodiments, corresponding event identifier 180 includes having how instruction event delivery system should be held Configurable attribute, label and/or the metadata of list 183 of the row to the subevent delivering of the event recognizer of active participation.In In some embodiments, metadata 183 includes indicating how event recognizer interacts or how to interact each other and is configurable Attribute, mark and/or list.In some embodiments, metadata 183 includes whether instruction subevent is delivered to view or journey Configurable attribute, mark and/or the list of different levels in sequence hierarchical structure.

In some embodiments, when one or more specific subevents of event are identified, corresponding event identifier 180 activation button.onreleases 190 associated with event.In some embodiments, corresponding event identifier 180 will be with thing The associated event information of part is delivered to button.onrelease 190.Activation button.onrelease 190 is different from sending out subevent Send (and delaying to send) to corresponding hit view.In some embodiments, the event that event recognizer 180 is dished out and identified Associated label, and button.onrelease 190 associated with the label obtains the label and executes predefined process.

In some embodiments, event delivery instruction 188 includes delivering the event information about subevent without activating The subevent delivery instructions of button.onrelease.On the contrary, event information is delivered to and subevent sequence by subevent delivery instructions Associated button.onrelease or the view for being delivered to active participation.With subevent sequence or with the view phase of active participation Associated button.onrelease receives event information and executes predetermined process.

In some embodiments, data renovator 176 creates and updates the data used in application program 136-1. For example, data renovator 176 is updated telephone number used in contact module 137, or to video or music Video file used in player module 152 is stored.In some embodiments, object renovator 177 creation and Update the object used in application program 136-1.For example, object renovator 177 creates new user interface object or update The position of user interface object.GUI renovator 178 updates GUI.For example, GUI renovator 178 prepares display information, and will show Show that information is sent to figure module 132 to show on the touch sensitive display.

In some embodiments, button.onrelease 190 includes data renovator 176, object renovator 177 and GUI Renovator 178 or with the access authority to them.In some embodiments, data renovator 176, object renovator 177 and GUI renovator 178 is included in corresponding application programs 136-1 or the individual module of application view 191.At it In his embodiment, they are included in two or more software modules.

It should be appreciated that the above-mentioned discussion of the event handling about user's touch on touch-sensitive display is applied also for using defeated Enter user's input that equipment carrys out the other forms of operating multifunction equipment 100, not all user's input is all in touch screen Upper initiation.For example, optionally pressing or pinning the mouse movement to cooperate and mouse button down with single or multiple keyboards；Touching Contact in template is mobile, tap, dragging, rolling etc.；Stylus input；Based on to the view obtained by one or more cameras The input of frequency image analyzed in real time；The movement of equipment；Spoken command；The eyes detected are mobile；Biological characteristic input；With/ Or any combination of them is optionally used as corresponding to the input for the subevent for limiting the event to be identified.

Fig. 1 C is to show the block diagram of the tactile output module according to some embodiments.In some embodiments, I/O Subsystem 106 (such as tactile feedback controller 161 (Figure 1A) and/or other input controllers 160 (Figure 1A)) includes Fig. 1 C institute At least some of exemplary components shown.In some embodiments, peripheral device interface 118 includes example portion shown in Fig. 1 C At least some of part.

In some embodiments, tactile output module includes tactile feedback module 133.In some embodiments, it touches Sense feedback module 133 polymerize and combine on electronic equipment software application for user interface feedback (for example, to show Show the generation of the execution or event that operate in the user interface of the corresponding user's input of user interface and instruction electronic equipment Prompt and other notify the feedback that is responded) tactile output.Tactile feedback module 133 includes waveform module 123 (for mentioning For the waveform for generating tactile output), mixer 125 (for hybrid waveform, waveform) in such as different channels, compressor 127 (for reduce or the dynamic range of compressed waveform), low-pass filter 129 is (for filtering out the high-frequency signal in waveform point One or more of amount) and heat controller 181 (for adjusting waveform according to heat condition).In some embodiments, sense of touch Feedback module 133 is included in tactile feedback controller 161 (Figure 1A).In some embodiments, tactile feedback module 133 Individually unit (or independent specific implementation of tactile feedback module 133) be also included in Audio Controller (such as voicefrequency circuit 110, Figure 1A) in and for generating audio signal.In some embodiments, single tactile feedback module 133 be used to generate Audio signal and the waveform for generating tactile output.

In some embodiments, tactile feedback module 133 further includes igniter module 121 (for example, determination will generate touching Feel software application, operating system or the other software module for exporting and causing the process for generating corresponding tactile output).In In some embodiments, igniter module 121 generates the trigger that waveform is generated for causing (such as by waveform module 123) Signal.For example, igniter module 121 generates flop signal based on pre-set timing standard.In some embodiments In, igniter module 121 receives flop signal (for example, in some embodiments, touching except tactile feedback module 133 Feel feedback module 133 and receive flop signal from the hardware input processing module 146 being located at except tactile feedback module 133) simultaneously Flop signal is relayed to other component (such as waveform module 123) in tactile feedback module 133 or based on user interface Element (for example, application icon within or show and can indicate) or hardware input equipment (such as main button or strength sensitive it is defeated Enter interface, such as strength sensitive touch screen) activation and the software application of trigger action (for example, utilizing igniter module 121). In some embodiments, igniter module 121 also (such as from tactile feedback module 133, Figure 1A and Fig. 3) receive touch feedback Generate instruction.In some embodiments, igniter module 121 is in response to (or the tactile feedback module of tactile feedback module 133 Igniter module 121 in 133) (such as receive touch feedback instruction from tactile feedback module 133, Figure 1A and Fig. 3) and generate Flop signal.

Waveform module 123 (such as slave flipflop module 121) receives flop signal as input, and in response to receiving Flop signal and provide waveform for generating the output of one or more tactiles (for example, being assigned use from predefined one group The waveform selected in the waveform used for waveform module 123, such as below with reference to Fig. 4 F- Fig. 4 G wave in greater detail Shape).

Mixer 125 (such as from waveform module 123) receives waveform as input, and these waveforms are mixed. For example, when mixer 125 receives two or more waveforms (for example, in first waveform and second channel in first passage At least partly second waveform Chong Die with first waveform) when, the output of mixer 125 corresponds to the two or more waveform The sum of combined waveform.In some embodiments, mixer 125 also modifies one or more in the two or more waveform A waveform is to emphasize that specific waveforms are (such as specific by improving this relative to remaining waveform in the two or more waveform The scale of waveform and/or the scale for reducing other waveforms in these waveforms).In some cases, mixer 125 selects one Or multiple waveforms are removed from combined waveform (for example, when there is the waveform from more than three source to be requested by tactile When output generator 163 exports simultaneously, the waveform from most old source is dropped).

Mixer 127 receives waveform (such as combined waveform from mixer 125) as input, and modifies these waves Shape.In some embodiments, compressor 127 reduce these waveforms (for example, according to tactile output generator 163 (Figure 1A) or The physical specification of 357 (Fig. 3)) so that the tactile output for corresponding to these waveforms is contracted by.In some embodiments, it compresses Device 127 such as limits waveform by forcing predefined maximum amplitude for waveform.Surpass for example, compressor 127 reduces The amplitude of the waveform portion of predefined amplitude thresholds is crossed, and is kept for no more the amplitude of the waveform portion of predefined amplitude thresholds. In some embodiments, compressor 127 reduces the dynamic range of waveform.In some embodiments, compressor 127 is dynamically Reduce waveform dynamic range so that combined waveform be maintained at tactile output generator 163 performance specification (such as power and/or Removable mass displacement limitation) in.

Low-pass filter 129 receives waveform (such as from compressor 127 through compressed waveform) as input, and to waveform It is filtered (such as smoothing processing) (such as removing or reduce the high frequency component signal in waveform).Such as in some cases, Compressor 127 includes interfering tactile output to generate and/or defeated according to tactile is generated through compressed waveform in compressed waveform More than the irrelevant signal (such as high frequency component signal) of the performance specification of tactile output generator 163 when out.Low-pass filter 129 Reduce or remove such irrelevant signal in waveform.

Heat controller 181 receives waveform (such as from low-pass filter 129 through filter shape) as input, and according to The heat condition of equipment 100 is (such as based on the internal temperature detected in equipment 100, such as temperature of tactile feedback controller 161 The external temperature that degree and/or equipment 100 detect) adjust waveform.Such as in some cases, tactile feedback controller 161 Output changes according to temperature (for example, in response to receiving same waveform, tactile feedback controller 161 is controlled in tactile feedback Device 161 generates the output of the first tactile when being in the first temperature, and is in different from the first temperature in tactile feedback controller 161 Second temperature when generate the second tactile output).For example, the magnitude (or amplitude) of tactile output can change according to temperature.For The effect of temperature change is reduced, waveform is modified (for example, the amplitude of waveform is based on temperature and is increased or decreased).

In some embodiments, haptic feedback module 133 (such as igniter module 121) is couple to hardware input processing Module 146.In some embodiments, other input controllers 160 in Figure 1A include hardware input processing module 146.In In some embodiments, hardware input processing module 146 receive from hardware input equipment 175 (such as in Figure 1A other are defeated Enter or control equipment 116, such as main button or strength sensitive input interface, such as strength sensitive touch screen) input.One In a little embodiments, hardware input equipment 175 is any input equipment described herein, such as touch-sensitive display system 112 (Figure 1A), keyboard/mouse 350 (Fig. 3), Trackpad 355 (Fig. 3), other inputs control one of equipment 116 (Figure 1A) or strong The sensitive main button of degree.In some embodiments, hardware input equipment 175 is made of the main button of strength sensitive, rather than by touching Quick display system 112 (Figure 1A), keyboard/mouse 350 (Fig. 3) or Trackpad 355 (Fig. 3) are constituted.In some embodiments, In response to coming from the input of hardware input equipment 175 (for example, the main button of strength sensitive or touch screen), hardware input processing module 146, which provide one or more flop signals, meets predefined input standard to tactile feedback module 133 to indicate to have detected that User's input, such as input corresponding to main button " clicks " (such as " pressing clicks " or " release click ").In some realities It applies in scheme, tactile feedback module 133 is provided in response to corresponding to the input of main button " click " corresponding to main button " point Hit " waveform, thus simulate pressing the main button of physics tactile feedback.

In some embodiments, tactile output module includes (such as the sense of touch in Figure 1A of tactile feedback controller 161 Feedback controller 161), the generation of control tactile output.In some embodiments, tactile feedback controller 161 is couple to Multiple tactile output generators, and select one or more tactile output generators in multiple tactile output generators and incite somebody to action Waveform is sent to selected one or more tactile output generators for generating tactile output.In some embodiments In, tactile feedback controller 161 coordinates the tactile output request for corresponding to activation hardware input equipment 175 and corresponds to software thing The tactile output request (such as the tactile from tactile feedback module 133 exports request) of part, and modify two or more waves One or more waveforms in shape are to emphasize that specific waveforms (such as pass through relative to remaining waveform in two or more waveforms It improves the scale of specific waveforms and/or reduces the scale of remaining waveform in these waveforms, compared to corresponding to software event Tactile exports the tactile output that priority processing corresponds to activation hardware input equipment 175).

In some embodiments, as shown in Figure 1 C, the output of tactile feedback controller 161 is couple to the sound of equipment 100 Frequency circuit (such as voicefrequency circuit 110, Figure 1A), and audio signal is supplied to the voicefrequency circuit of equipment 100.In some embodiment party In case, tactile feedback controller 161 provides the waveform for generating tactile output and provides together for exporting with generation tactile Both audio signals of audio output.In some embodiments, tactile feedback controller 161 modify audio signal and/or (for generate tactile output) waveform make audio output it is synchronous with tactile output (such as by postpone audio signal and/or Waveform) in some embodiments, tactile feedback controller 161 includes the digital-to-analogue for digital waveform to be converted into analog signal Converter, analog signal is amplified device 185 and/or tactile output generator 163 receives.

In some embodiments, tactile output module includes amplifier 185.In some embodiments, amplifier 185 (such as from tactile feedback controller 161) waveform is received, and then enlarged waveform is sent to tactile by waveform amplification Output generator 163 (for example, any one in tactile output generator 163 (Figure 1A) or 357 (Fig. 3)).For example, amplifier 185 By received waveform be amplified to the signal level of the physical specification for meeting tactile output generator 163 and (such as be amplified to tactile Output generator 163 in order to generate tactile output and required voltage and/or electric current to be sent to tactile output generator 163 Signal generate to correspond to exported from the tactile of the received waveform of tactile feedback controller 161) and enlarged waveform is sent To tactile output generator 163.In response, tactile output generator 163 generates tactile output (such as by by removable kinoplaszm Block is in one or more dimensions relative to shifting before and after the neutral position of removable mass).

In some embodiments, tactile output module includes sensor 189, is couple to tactile output generator 163. Sensor 189 detects one or more components of tactile output generator 163 or tactile output generator 163 (such as giving birth to At tactile export one or more moving components, such as film) state or state change (such as mechanical location, physical displacement, And/or mobile).In some embodiments, sensor 189 is magnetic field sensor (such as hall effect sensor) or other positions Shifting and/or motion sensor.In some embodiments, sensor 189 is by information (such as one in tactile output generator 163 Position, displacement and/or the movement of a or multiple components) it is supplied to tactile feedback controller 161, and mentioned according to sensor 189 The information of the state about tactile output generator 163 supplied, tactile feedback controller 161 are adjusted from tactile feedback controller The waveform (for example, waveform that tactile output generator 163 is optionally sent to via amplifier 185) of 161 outputs.

Fig. 2 shows according to some embodiments with touch screen (for example, touch-sensitive display system 112 of Figure 1A) Portable multifunction device 100.Touch screen optionally shows one or more figures in user interface (UI) 200.At these In embodiment and in other embodiments for being described below, user can be by, for example, one or more fingers 202 (being not drawn on scale in figure) or one or more stylus 203 (being not drawn on scale in figure) are sold on figure Gesture selects one or more figures in these figures.In some embodiments, it interrupts as user and schemes with one or more When the contact of shape, the selection to one or more figures will occur.In some embodiments, gesture optionally include it is primary or Repeatedly (from left to right, from right to left, up and/or down) and/or tap, one or many gently sweep occur with equipment 100 The rolling (from right to left, from left to right, up and/or down) of the finger of contact.In some specific implementations or in some feelings Under condition, figure inadvertently will not be selected with pattern contact.For example, applying journey when gesture corresponding with selection is tap What is swept on sequence icon gently sweeps gesture and will not optionally select corresponding application program.

Equipment 100 optionally further includes one or more physical buttons, such as " home " button or menu button 204.Such as Preceding described, menu button 204 is optionally for navigating in one group of application program being optionally performed on the appliance 100 Any application program 136.Alternatively, in some embodiments, menu button is implemented as being displayed on touching Touch the soft key in the GUI on panel type display.

In some embodiments, equipment 100 includes touch-screen display, menu button 204 (sometimes referred to as press by main screen Button 204), pushing button 206, volume knob 208, Yong Hushen for keeping equipment power on/off and for locking device Part module (SIM) card slot 210, earphone jack 212 and docking/charging external port 124.Button 206 is pushed optionally for logical Under over-voltage the button and by the button be maintained at depressed state continue predefined time interval come to equipment carry out ON/OFF Machine；By depressing the button and discharging the button before in the past in the predefined time interval come locking device；And/or it is right Equipment is unlocked or initiates unlocking process.In some embodiments, equipment 100 also receives to be used for by microphone 113 Activate or deactivate the voice input of certain functions.Equipment 100 is also optionally included for detecting in touch-sensitive display system 112 One or more contact strength sensors 165 of the intensity of contact, and/or for for equipment 100 user generate tactile output One or more tactile output generators 163.

Fig. 3 A is the block diagram according to the exemplary multifunctional equipment with display and touch sensitive surface of some embodiments. Equipment 300 needs not be portable.In some embodiments, equipment 300 is laptop computer, desktop computer, plate electricity Brain, multimedia player device, navigation equipment, educational facilities (such as children for learning toy), game system or control equipment (example Such as, household controller or industrial controller).Equipment 300 generally include one or more processing units (CPU) 310, one or Multiple networks or other communication interfaces 360, memory 370 and one or more communication bus for interconnecting these components 320.Communication bus 320 optionally include make system unit interconnect and control system component between communication circuit (sometimes Referred to as chipset).Equipment 300 includes input/output (I/O) interface 330 with display 340, which is optionally Touch-screen display.I/O interface 330 also optionally includes keyboard and/or mouse (or other sensing equipments) 350 and Trackpad 355, for generating the tactile output generator 357 of tactile output in equipment 300 (for example, being similar to above with reference to Figure 1A institute The one or more tactile output generators 163 stated), sensor 359 is (for example, optical sensor, acceleration transducer, close Sensor, touch-sensitive sensors, and/or similar to one or more contact strengths sensor 165 above with reference to described in Figure 1A Contact strength sensor).Memory 370 include high-speed random access memory, such as DRAM, SRAM, DDR RAM or other with Machine accesses solid-state memory device；And nonvolatile memory is optionally included, such as one or more disk storage equipments, Optical disc memory apparatus, flash memory device or other non-volatile solid-state memory devices.Memory 370 optionally includes separate One or more storage equipment that CPU 310 is positioned.In some embodiments, the storage of memory 370 and portable multi-function Similar program, module and the data of program, module and the data structure stored in the memory 102 of equipment 100 (Figure 1A) Structure or their subgroup.In addition, memory 370 is optionally stored in the memory 102 of portable multifunction device 100 Appendage, module and the data structure being not present.For example, the memory 370 of equipment 300 optionally store graphics module 380, Module 382, word processing module 384, website creation module 386, disk editor module 388, and/or spreadsheet module is presented 390, and the memory 102 of portable multifunction device 100 (Figure 1A) does not store these modules optionally.

Each element in Fig. 3 A in above-mentioned identified element is optionally stored in previously mentioned memory devices One or more memory devices in.Each module in above-mentioned identified module corresponds to for executing above-mentioned function One group of instruction.Above-mentioned identified module or program (that is, instruction set) need not be implemented as independent software program, process or mould Block, therefore each subset of these modules is optionally combined in various embodiments or is otherwise rearranged.One In a little embodiments, memory 370 optionally stores the subgroup of above-mentioned module and data structure.In addition, memory 370 is optional Ground stores other module and data structure not described above.

Fig. 3 B-3C is the block diagram according to the exemplary computer system 301 of some embodiments.

In some embodiments, computer system 301 includes with lower component and/or communicating:

Input equipment (302 and/or 307, for example, touch sensitive surface, such as touch-sensitive remote controler, or also serve as display life At the touch-screen display of component, mouse, control stick, stick controller, and/or tracking user one or more features such as with The camera of the position of the hand at family；

Virtually/augmented reality logical block 303 (for example, virtual/augmented reality module 145)；

Show generating unit (304 and/or 308, for example, display, projector, head-up display etc.), for Family shows visual user interface element；

The camera (for example, 305 and/or 311) of the image in the visual field for capture device, for example, virtual for determining The image of a part of the placement of user interface element, the posture for determining equipment and/or physical environment where display camera；And

Attitude transducer (for example, 306 and/or 311), for determine equipment relative to the posture of physical environment and/or The variation of equipment posture.

In some computer systems (for example, 301-a in Fig. 3 B), input equipment 302, virtual/augmented reality logic section Part 303, display generating unit 304, camera 305 and attitude transducer 306 are all integrated into computer system (for example, figure Portable multifunction device 100 in 1A-1B or the equipment in Fig. 3 300 such as smart phone or tablet computer).

In some computer systems (for example, 301-b), in addition to integrated input equipment 302, virtual/augmented reality logic Except component 303, display generating unit 304, camera 305 and attitude transducer 306, computer system is also and and computer system Isolated optional equipment communication, individual 307 touch sensitive surface of input equipment, stick, remote controler etc. and/or individually aobvious Show generating unit 308, such as virtual reality headphone or augmented reality glasses, it is virtual right to cover on physical environment As.

In some computer systems (for example, 301-c in Fig. 3 C), input equipment 307, display generating unit 309, phase Machine 311 and/or attitude transducer 312 are separated with computer system and are communicated with computer system.In some embodiments, make With in computer system 301 and combined with other of component of computer system communication.For example, in some embodiments In, display generating unit 309, camera 311 and attitude transducer 312 are incorporated in and Computer systems or and department of computer science In the headphone of system communication.

In some embodiments, below with reference to all operations of Fig. 5 A-5CO description there is virtual/augmented reality to patrol It collects and is executed in the single calculating equipment (such as computer system 301-a below with reference to Fig. 3 B description) of component 303.However, answering Work as understanding, often by multiple and different calculating linking of devices together to execute the operation described below with reference to Fig. 5 A-5CO (for example, calculatings equipment with virtual/augmented reality logical block 303 and the standalone computing device with display 450 and/ Or the standalone computing device communication with touch sensitive surface 451).In any of these embodiments, retouched below with reference to Fig. 5 A- 5CO The calculating equipment stated is one or more calculating equipment comprising virtual/augmented reality logical block 303.In addition, it will be appreciated that In various embodiments, virtual/augmented reality logical block 303 can be divided in multiple and different modules or calculate equipment it Between；However, virtual/augmented reality logical block 303 will be mostly referred to as residing in single calculating for purpose described herein In equipment, to avoid other aspects for unnecessarily obscuring embodiment.

In some embodiments, virtual/augmented reality logical block 303 includes receiving the one or more for explaining input Module (for example, one or more button.onreleases 190, including one or more be such as more fully described above with reference to Figure 1B A object renovator 177 and one or more GUI renovator 178), and explain and input in response to these, it is inputted according to explaining The instruction for updating graphic user interface is generated, explanation input is used subsequently to update the graphic user interface on display. In some embodiments, detected (for example, passing through the contact movement module 130 in Figure 1A and Fig. 3), identification (example Such as, pass through the event recognizer 180 in Figure 1B) and/or distribution (for example, by event classifier 170 in Figure 1B) input Explanation input for updating the graphic user interface on display.In some embodiments, explain input by calculating equipment On module generate (for example, calculating equipment receives original contact input data, so as to from original contact input data identification hand Gesture).In some embodiments, explain input some or all by calculating equipment be received as explain input (e.g., including touching The calculating equipment of sensitive surfaces 451 handles original contact input data, to identify gesture from original contact input data and will to refer to Show that the information of gesture is sent to the calculating equipment including virtual/augmented reality logical block 303).

In some embodiments, display and touch sensitive surface all with the meter that includes virtual/augmented reality logical block 303 Calculation machine system (for example, 301-a in Fig. 3 B) is integrated.For example, the computer system can be with integrated display (for example, 340 in Fig. 3) and the desktop computer or laptop computer of Trackpad (for example, 355 in Fig. 3).For another example, which sets It is standby can be with touch screen (for example, 112 in Fig. 2) portable multifunction device 100 (for example, smart phone, PDA, Tablet computer etc.).

In some embodiments, touch sensitive surface and Computer systems, and display does not show with comprising virtual/enhancing The Computer systems of real logical block 303.For example, the computer system can be with integrated touch panel (for example, Fig. 3 In 355) equipment 300 (for example, desktop computer or laptop computer etc.), wherein the integrated touch panel and independent show Device (for example, computer monitor, TV etc.) is connected (passing through wired or wireless connection).For another example, which can be Portable multifunction device 100 with touch screen (for example, 112 in Fig. 2) is (for example, smart phone, PDA, tablet computer Deng), wherein the touch screen is connected with stand alone display (for example, computer monitor, TV etc.) (by wired or wireless company It connects).

In some embodiments, display and Computer systems, and touch sensitive surface does not show with comprising virtual/enhancing The Computer systems of real logical block 303.For example, the computer system can be with integrated display (for example, Fig. 3 In 340) equipment 300 (for example, desktop computer, laptop computer, TV with integrated form set-top box), the wherein collection It is connected with independent touch sensitive surface (for example, long-range Trackpad, portable multifunction device etc.) (by wired or wireless at display Connection).For another example, which can be the portable multifunction device with touch screen (for example, 112 in Fig. 2) 100 (for example, smart phone, PDA, tablet computers etc.), wherein the touch screen and independent touch sensitive surface (for example, long-range Trackpad, The portable multifunction device etc. that another touch screen is used as long-range Trackpad) it is connected (by wired or wireless connection).

In some embodiments, display and touch sensitive surface not with include virtual/augmented reality logical block 303 Computer system (for example, 301-c in Fig. 3 C) is integrated.For example, the computer system can be connection (by wired or wireless Connection) independent touch sensitive surface (for example, remote touch plate, portable multifunction device etc.) and stand alone display are arrived (for example, calculating Machine monitor, TV etc.) standalone computing device 300 (for example, set-top box, game console etc.).

In some embodiments, computer system has integrated audio system (for example, portable multifunction device 100 In voicefrequency circuit 110 and loudspeaker 111).In some embodiments, equipment is calculated and independently of the audio system for calculating equipment System communication.In some embodiments, audio system (for example, being integrated in the audio system in television unit) and individually display Device is integrated.In some embodiments, audio system (for example, stereo system) is and the computer system and display point The autonomous system opened.

It attention is drawn to the user interface (" UI ") optionally realized on portable multifunction device 100 Embodiment.

Fig. 4 A shows the exemplary of the application menu on the portable multifunction device 100 according to some embodiments User interface.Similar user interface is optionally realized in equipment 300.In some embodiments, user interface 400 includes Following elements or its subset or superset:

One or more signal strengths instruction of one or more wireless communication such as cellular signals and Wi-Fi signal Device；

Time；

Bluetooth indicator；

Battery Status Indicator；

Pallet 408 with the icon for common application program, the icon such as:

The icon 416 for being marked as " phone " of ο phone module 138, the icon 416 optionally include missed call or language The indicator 414 of the quantity of sound message；

The icon 418 for being marked as " mail " of ο email client module 140, the icon 418 optionally include not Read the indicator 410 of the quantity of Email；

The label of ο browser module 147 be browser " icon 420；And

The icon 422 for being marked as " music " of ο video and musical player module 152；And

The icon of other application, such as:

The icon 424 for being marked as " message " of ο IM module 141

The icon 426 for being marked as " calendar " of ο calendaring module 148；

The icon 428 for being marked as " photo " of ο image management module 144；

The icon 430 for being marked as " camera " of ο camera model 143；

The icon 432 for being marked as " measuring " of ο measurement module 155；

The icon 434 for being marked as " stock market " of the stock market ο desktop small routine 149-2；

The icon 436 for being marked as " map " of ο mapping module 154

The icon 438 for being marked as " weather " of ο weather desktop small routine 149-1；

The icon 440 for being marked as " clock " of ο alarm clock desktop small routine 149-4

The icon 442 for being marked as " body-building support " of ο body-building support module 142；

The label of ο notepad module 153 be notepad " icon 444；And

ο is arranged the icon 446 for being marked as " be arranged " of application program or module, icon offer to equipment 100 and its The access of the setting of various application programs 136.

It should be noted that icon label shown in Fig. 4 A is only exemplary.For example, other labels are optionally for each Kind application icon.In some embodiments, the label of corresponding application programs icon includes and the corresponding application programs icon pair The title for the application program answered.In some embodiments, the label of application-specific icon is different from and the specific application The title of the corresponding application program of program icon.

Fig. 4 B is shown with the touch sensitive surface 451 separated with display 450 (for example, plate or Trackpad in Fig. 3 A 355) the exemplary user interface in equipment (for example, equipment 300 in Fig. 3 A).Although touch-screen display 112 will be referred to Input on (being wherein combined with touch sensitive surface and display) provides subsequent many embodiments, but in some embodiments In, the input on touch sensitive surface that equipment detection is separated with display, as shown in Figure 4 B.In some embodiments, touch-sensitive Surface (for example, 451 in Fig. 4 B) has corresponding with main shaft (for example, 453 in Fig. 4 B) on display (for example, 450) Main shaft (for example, 452 in Fig. 4 B).According to these embodiments, equipment detection is corresponding with corresponding position on display The contact with touch sensitive surface 451 (for example, 460 in Fig. 4 B and 462) at position is (for example, in figure 4b, 460 correspond to 468 And 470) 462 correspond to.In this way, touch sensitive surface (for example, 451 in Fig. 4 B) and multifunctional equipment display (for example, 450 in Fig. 4 B) marquis when being separated, by equipment user's input detected on touch sensitive surface (for example, 460 Hes of contact 462 and their movement) be used to manipulate the user interface on display by the equipment.It should be appreciated that similar method is optional Ground is used for other users interface as described herein.

In addition, though mostly in reference to finger input (for example, finger contact, singly referring to that Flick gesture, finger gently sweep gesture Deng) provide following example it should be appreciated that in some embodiments, one in the input of these fingers or Multiple finger inputs are by input (for example, input or stylus based on mouse input) replacement from another input equipment.Example Such as, it gently sweeps gesture and (for example, rather than contacting) is optionally clicked by mouse, be cursor moving along the path gently swept later (for example, rather than movement of contact) substitution.For another example, Flick gesture is optionally by the position that cursor is located at Flick gesture When mouse click and (for example, instead of the detection to contact, be off detection contact later) substitution.Similarly, it is detected when simultaneously To multiple users input when, it should be appreciated that multiple computer mouses be optionally used simultaneously or mouse and finger contact Optionally it is used simultaneously.

As used herein, term " focus selector " refers to the user interface for being used to indicate that user is just interacting therewith The input element of current portions.In some specific implementations for including cursor or other positions label, cursor serves as " focus selection Device ", so that when cursor is on particular user interface element (for example, button, window, sliding block or other users interface element) When detect input (for example, pressing on touch sensitive surface (for example, touch sensitive surface 451 in Trackpad 355 or Fig. 4 B in Fig. 3 A) Pressure input) in the case where, which is adjusted according to detected input.It is including making it possible to reality Now with the touch-screen display of the user interface element on touch-screen display directly interacted (for example, touch-sensitive aobvious in Figure 1A Show the touch screen in device system 112 or Fig. 4 A) some specific implementations in, " focus is served as in the contact detected on the touchscreen Selector ", so that working as on touch-screen display in particular user interface element (for example, button, window, sliding block or other use Family interface element) position at detect input (for example, by contact pressing input) when, according to detected input come Adjust particular user interface element.In some specific implementations, focus is moved to user interface from a region of user interface Another region, the movement of the contact in correspondence movement or touch-screen display without cursor is (for example, by using system Focus is moved to another button from a button by table key or arrow key)；In these specific implementations, focus selector according to Movement of the focus between the different zones of user interface and move.The concrete form that focus selector is taken is not considered, it is burnt Point selector is usually from user's control to transmit with the desired interaction of the user of user interface (for example, by referring to equipment Show that the user of user interface it is expected the element that interacts) user interface element (or connecing on touch-screen display Touching).For example, when detecting pressing input on touch sensitive surface (for example, touch tablet or touch screen), focus selector (for example, Cursor, contact or choice box) position on the corresponding button will indicate user it is expected to activate the corresponding button that (rather than equipment is aobvious Show the other users interface element shown on device).In some embodiments, focus indicator (example is shown via display equipment Such as, cursor or selection indicator) it will be influenced by from the received input of one or more input equipments with indicative user interface Current portions.

As used in the present specification and claims, " intensity " of the contact on term touch sensitive surface refers to touching The power or pressure (power of per unit area) of contact (for example, finger contact or stylus contact) on sensitive surfaces, or refer to touching The power of contact on sensitive surfaces or the substitute (surrogate) of pressure.The intensity of contact has value range, which includes extremely Lack four different values and more typically include up to a hundred different values (for example, at least 256).The intensity of contact is optionally (or measurement) is determined using the combination of various methods and various sensors or sensor.For example, below touch sensitive surface or phase Adjacent to touch sensitive surface one or more force snesors optionally for measurement touch sensitive surface on difference at power.Some In specific implementation, the power measurement from multiple force sensors is merged (for example, weighted average or adduction), to determine estimation Contact force.Similarly, pressure of the pressure-sensitive top of stylus optionally for determining stylus on touch sensitive surface.Alternatively, it is touching The capacitor of touch sensitive surface near the size of the contact area detected on sensitive surfaces and/or its variation, contact and/or its variation And/or the resistance and/or its power for changing the contact being optionally used as on touch sensitive surface of the touch sensitive surface near contact Or the substitute of pressure.In some specific implementations, the substitute measurement of contact force or pressure, which is directly used in, to be determined whether More than intensity threshold (for example, intensity threshold is described with corresponding to the unit that substitute measures).It, will in some specific implementations The substitution measured value of contact force or pressure, which is converted to, estimates power or pressure, and use estimate power or pressure determine whether be more than Intensity threshold (for example, intensity threshold is the pressure threshold measured with pressure unit).The intensity of contact is used to input as user Attribute, so that use can indicated and/or receive for showing (for example, on the touch sensitive display) by allowing user to access user Family inputs the face on the spot of (for example, via touch-sensitive display, touch sensitive surface or physical control/machinery control such as knob or button) The optional equipment function that cannot be easily accessed originally in the limited smaller equipment of product.

In some embodiments, contact/motion module 130 determines operation using one group of one or more intensity threshold Whether (for example, determining that whether user " clicks " icon) is executed by user.In some embodiments, according to software parameter To determine at least one subset of intensity threshold (for example, intensity threshold is not by the activation threshold of specific physical actuation device Lai really Fixed, and can be conditioned in the case where not changing the physical hardware of equipment 100).For example, not changing Trackpad or touch In the case where panel type display hardware, mouse " click " threshold value of Trackpad or touch-screen display can be arranged to predefined thresholds It is a wide range of in any one threshold value.In addition, the user of equipment is provided with for adjusting one group of intensity in some specific implementations One or more intensity thresholds in threshold value are (for example, by adjusting each intensity threshold and/or by using to " intensity " parameter System-level click carry out the multiple intensity thresholds of Primary regulation) software setting.

As used in the specification and in the claims, " characteristic strength " of contact this term refers to one based on contact The feature of the contact of a or multiple intensity.In some embodiments, characteristic strength is based on multiple intensity samples.Characteristic strength is appointed Selection of land is based on (for example, after detecting contact, before detecting that contact is lifted away from, detecting relative to predefined event Before or after contact start is mobile, before detecting that contact terminates, before or after detecting that the intensity of contact increases And/or detect contact intensity reduce before or after) in the predetermined period (for example, 0.05 second, 0.1 Second, 0.2 second, 0.5 second, 1 second, 2 seconds, 5 seconds, 10 seconds) during acquire predefined quantity intensity sample or one group of intensity sample. The characteristic strength of contact is optionally based on one or more of the following terms: the maximum value of contact strength, contact strength it is equal Value, the average value of contact strength, contact strength preceding 10% at value, half maximum value of contact strength, contact strength 90% Maximum value, the value generated by the low-pass filtering contact strength in time predefined section or since time predefined etc..In In some embodiments, when determining characteristic strength using the duration of contact (for example, being the intensity contacted in characteristic strength When average value in time).In some embodiments, characteristic strength and one group of one or more intensity threshold are compared Compared with to determine whether user has executed operation.For example, the group one or more intensity threshold may include the first intensity threshold and Two intensity thresholds.In this embodiment, the contact that characteristic strength is less than the first intensity threshold leads to the first operation, characteristic strength More than the first intensity threshold but the contact that is less than the second intensity threshold leads to the second operation, and characteristic strength is more than the last the second The contact of degree threshold value causes third to operate.In some embodiments, using characteristic strength and one or more intensity thresholds it Between comparison come determine whether to execute one or more operations (such as, if execute respective selection or abandon executing corresponding behaviour Make), rather than the first operation or the second operation are executed for determining.

In some embodiments, identify a part of gesture for determining characteristic strength.For example, touch sensitive surface can connect Receipts continuously gently sweep contact, this is continuously gently swept contact from initial position transition and reaches end position (such as drag gesture), at this At end position, the intensity of contact increases.In this embodiment, characteristic strength of the contact at end position can be based only upon continuously It gently sweeps a part of contact, rather than entirely gently sweeps contact (for example, only a part for gently sweeping contact at end position).One In a little embodiments, algorithm can be smoothed in the intensity application that the forward direction for the characteristic strength for determining contact gently sweeps gesture.For example, flat Cunningization algorithm optionally includes one of the following terms or a variety of: not weighting sliding average smoothing algorithm, triangle smoothing Algorithm, median filter smoothing algorithm and/or exponential smoothing algorithm.In some cases, these smoothing algorithms are eliminated The narrow spike or recess in the intensity of contact is swept, gently to realize the purpose for determining characteristic strength.

User interface map as described herein optionally includes various intensity maps, these intensity illustrate connecing on touch sensitive surface Touching is relative to one or more intensity thresholds (for example, contact detection intensity threshold value IT₀, light press intensity threshold IT_L, deep press pressure Spend threshold value IT_D(for example, being at least initially higher than IT_L) and/or other one or more intensity thresholds (for example, comparing IT_LLow intensity Threshold value IT_H)) current intensity.The intensity map is not usually a part of shown user interface, but is provided to help Explain the figure.In some embodiments, light press intensity threshold corresponds to such intensity: equipment will be held under the intensity Row operation usually associated with the button for clicking physics mouse or touch tablet.In some embodiments, deep to press Compressive Strength threshold Value correspond to such intensity: under the intensity equipment will execute with usually with click physics mouse or Trackpad button it is related The different operation of the operation of connection.In some embodiments, when detect characteristic strength lower than light press intensity threshold (for example, And it is higher than Nominal contact detection intensity threshold value IT₀, the contact lower than Nominal contact detection intensity threshold value be no longer detected) When contact, equipment by according to movement of the contact on touch sensitive surface come moving focal point selector, without executing and flicking Compressive Strength Threshold value or the deep pressing associated operation of intensity threshold.In general, unless otherwise stated, otherwise these intensity thresholds in difference It is consistent between the user interface attached drawing of group.

In some embodiments, equipment depends on based on connecing during input the response of input detected by equipment Touch the standard of intensity.For example, being inputted for some " light press ", more than the intensity of the contact of the first intensity threshold during input The first response of triggering.In some embodiments, equipment depends on the response of the input as detected by equipment to include input The standard of the contact strength of period and time-based both criteria.For example, for some " deep to press " inputs, as long as meeting Pass through delay time between the second intensity threshold of first intensity threshold and satisfaction, the first of light press is exceeded more than during input The intensity of the contact of second intensity threshold of intensity threshold just triggers the second response.The duration of the delay time is usually less than (for example, 40ms, 100ms or 120ms, this depends on the magnitude of the second intensity threshold to 200ms (millisecond), wherein when the delay Between with the second intensity threshold increase and increase).The delay time helps to avoid unexpectedly to identify deep pressing input.For another example, right In some " deep pressing " inputs, the period that susceptibility reduction will occur after reaching the first intensity threshold.In the susceptibility During the reduced period, the second intensity threshold increases.This temporary increase of second intensity threshold, which additionally aids, avoids accident Deep pressing input.For other pressing inputs deeply, time-based standard is not dependent on to the response for detecting deep pressing input.

In some embodiments, one or more of input intensity threshold value and/or corresponding output are based on one or more A factor (such as user setting, contact movement, incoming timing, using operation, apply intensity when rate, input simultaneously number Amount, user's history, environmental factor (for example, ambient noise), focus selector position etc.) and change.Example factor is special in the U.S. It is described in benefit application 14/399,606 and 14/624,296, these U.S. Patent applications, which are incorporated by reference, is incorporated to this Text.

For example, the dynamic that Fig. 4 C shows the intensity being based in part on touch input 476 at any time and changes over time is strong Spend threshold value 480.Resistance to vibration threshold value 480 is the summation of two components: pre- since being initially detected touch input 476 The first component 474 for decaying after the delay time p1 of definition at any time and track the intensity of touch input 476 at any time Second component 478.The initial high-intensitive threshold value of first component 474 reduces unexpected triggering " deep pressing " response, still allows for simultaneously " deep pressing " response immediately is carried out in the case where touch input 476 provides sufficient intensity.The reduction of second component 478 passes through touch The gradual strength fluctuation of input and inadvertent free " deep pressing " response.In some embodiments, meet in touch input 476 When resistance to vibration threshold value 480 (for example, point 481 at) in figure 4 c, triggering " deep pressing " response.

Fig. 4 D shows another resistance to vibration threshold value 486 (for example, intensity threshold IT_D).Fig. 4 D also show two its His intensity threshold: the first intensity threshold IT_HWith the second intensity threshold IT_L.In fig. 4d, although touch input 484 is in time p2 Meet the first intensity threshold IT before_HWith the second intensity threshold IT_L, but just mentioned until passing through delay time p2 at the time 482 For response.In addition, in fig. 4d, resistance to vibration threshold value 486 decays at any time, wherein decay from the time 482 (when triggering with Intensity threshold IT_LWhen associated response) start by the time 488 after predefined delay time p1.It is such Resistance to vibration threshold value is reduced immediately in triggering and lower threshold intensity (such as the first intensity threshold IT_HOr the second intensity threshold IT_L) Associated response surprisingly triggers and resistance to vibration threshold value IT later or simultaneously_DAssociated response.

Fig. 4 E shows another resistance to vibration threshold value 492 (for example, intensity threshold IT_D).In Fig. 4 E, from touch Input 490 to after having been subjected to delay time p2, is triggered and intensity threshold IT by initial detecting_LAssociated response.Meanwhile it is dynamic State intensity threshold 492 from touch input 490 by initial detecting to when have been subjected to and decline after predefined delay time p1 Subtract.Therefore, in triggering and intensity threshold I_LThe intensity that touch input 490 is reduced after associated response, is not discharging then The intensity for increasing touch input 490 in the case where touch input 490 can trigger and intensity threshold IT_DIt is associated response (for example, At the time 494), even if when the intensity of touch input 490 is lower than another intensity threshold (for example, intensity threshold IT_L) when be also So.

Contact characteristic intensity is from lower than light press intensity threshold IT_LIntensity increase between light press intensity threshold IT_LWith Deep pressing intensity threshold IT_DBetween intensity be sometimes referred to as " light press " input.The characteristic strength of contact is from lower than deep pressing Intensity threshold IT_DIntensity increase to above deep pressing intensity threshold IT_DIntensity be sometimes referred to as " deep pressing " input.Contact is special Intensity is levied from lower than contact detection intensity threshold value IT₀Intensity increase between contact detection intensity threshold value IT₀With flicking Compressive Strength Threshold value IT_LBetween intensity be sometimes referred to as and detect contact on touch-surface.The characteristic strength of contact is examined from contact is higher than Survey intensity threshold IT₀Intensity be decreased below contact detection intensity threshold value IT₀Intensity sometimes referred to as detect contact from touching Surface is touched to be lifted away from.In some embodiments, IT₀It is zero.In some embodiments, IT₀Greater than zero in some illustrations, yin Shadow circle is oval for indicating the intensity of the contact on touch sensitive surface.In some illustrations, shadeless circle or ellipse are used for Indicate the corresponding contact on touch sensitive surface without the specified intensity accordingly contacted.

Herein in some embodiments, in response to detecting gesture or response including corresponding pressing input One or more operations are executed in detecting using corresponding pressing input performed by corresponding contact (or multiple contacts), wherein It is based at least partially on and detects that the intensity of the contact (or multiple contacts) increases to above pressing input intensity threshold value and detects It is inputted to the corresponding pressing.In some embodiments, in response to detecting that it is defeated that the intensity accordingly contacted increases to above pressing Enter intensity threshold to execute corresponding operating (for example, executing corresponding operating on " downward stroke " of corresponding pressing input).One In a little embodiments, pressing input includes that the intensity accordingly contacted increases to above pressing input intensity threshold value and the contact Intensity is decreased subsequently to lower than pressing input intensity threshold value, and in response to detecting that it is low that the intensity accordingly contacted is decreased subsequently to Threshold value is inputted in pressing to execute corresponding operating (for example, executing corresponding operating on " up stroke " of corresponding pressing input).

In some embodiments, equipment is lagged using intensity to avoid the accident input sometimes referred to as " shaken ", Middle equipment limits or selection has the lag intensity threshold of predefined relationship (for example, lag intensity with pressing input intensity threshold value Threshold value than the low X volume unit of pressing input intensity threshold value, or lag intensity threshold be pressing input intensity threshold value 75%, 90% or some rational proportion).Therefore, in some embodiments, pressing input includes that the intensity accordingly contacted increases to height It is decreased subsequently in the intensity of pressing input intensity threshold value and the contact lower than the lag for corresponding to pressing input intensity threshold value Intensity threshold, and in response to detecting that the intensity accordingly contacted is decreased subsequently to execute corresponding behaviour lower than lag intensity threshold Make (for example, executing corresponding operating on " up stroke " of corresponding pressing input).Similarly, in some embodiments, only It detects that contact strength is increased to from the intensity equal to or less than lag intensity threshold in equipment and is equal to or higher than pressing input by force Spend threshold value intensity and optionally contact strength be decreased subsequently to be equal to or less than lag intensity intensity when just detect by Pressure input, and hold in response to detecting pressing input (for example, according to environment, contact strength increases or contact strength reduces) Row corresponding operating.

In order to be easy to explain, be optionally in response to detect following situations and trigger in response to pressing input intensity threshold Be worth the description of associated pressing input or the operation executed in response to including the gesture of pressing input: the intensity of contact increases It is strong from pressing input is increased to above lower than the intensity for lagging intensity threshold to pressing input intensity threshold value, the intensity of contact is higher than Degree threshold value intensity, the intensity of contact be decreased below pressing input intensity threshold value or contact intensity be decreased below with by Press the corresponding lag intensity threshold of input intensity threshold value.In addition, describing the operations as in response to detecting that the intensity of contact subtracts It is small to lower than pressing input intensity threshold value and in the example that executes, be optionally in response to detect that the intensity of contact is decreased below Correspond to and is less than the lag intensity threshold of pressing input intensity threshold value to execute operation.As described above, in some embodiment party In case, the triggering of these operations is additionally depended on and meets time-based standard (for example, meeting the first intensity threshold and satisfaction Delay time is had been subjected between second intensity threshold).

Although in order to which the purpose explanation of illustration merely illustrates specific frequency in sample tactile output mode in Fig. 4 F- Fig. 4 K Rate, amplitude and waveform, but the tactile output mode with other frequencies, amplitude and waveform can also be used for similar purpose.For example, can Use the waveform having between 0.5 to 4 circulations.Other frequencies in 60Hz-400Hz range can also be used.

User interface and associated process

It attention is directed to include display generating unit (for example, display, projector, head-up display Deng), one or more camera is (for example, continuously provide the real-time pre- of at least part content in the visual field of at least one camera Look at and optionally generate including capture at least one camera the visual field in content one or more picture frames video it is defeated Video camera out), and one or more input equipments (for example, touch sensitive surface, such as touch-sensitive remote controler, or also serve as aobvious Show that touch-screen display, mouse, control stick, stick controller and/or the one or more features of tracking user of generating unit are all Such as the camera of the position of the hand of user), optional one or more attitude transducers, optional detection touch sensitive surface contact The one or more sensors of intensity and optional one or more tactile output generator (and/or with these component communications) Computer system (for example, electronic equipment, such as portable multifunction device 100 (Figure 1A), equipment 300 (Fig. 3 A) or calculate Machine system 301 (Fig. 3 B)) on the embodiment of user interface (" UI ") and correlated process realized.

Fig. 5 A-5CO is shown according to some embodiments for carrying out physical space measurement using augmented reality environment Exemplary user interface.User interface in these attached drawings be used to show process described below, including Fig. 6 A-6C, figure 7A-7E, Fig. 8 A-8C, Fig. 9 A-9B, Figure 10 A-10B, Figure 11 A-11B, Figure 12 A-12C, the mistake in Figure 13 A-13C and 14A-14D Journey.For the ease of explaining, by embodiment is discussed with reference to the operation executed in the equipment with touch-sensitive display system 112 In some embodiments.In such embodiment, focus selector is optionally: respective finger or stylus contact correspond to The expression point (for example, the center of gravity that accordingly contacts or with accordingly contact associated point) that contacts in finger or stylus or touch-sensitive The center of gravity of two or more detected contacts on display system 112.Optionally, however with 450 He of display Show user interface shown in the accompanying drawings together with coke on display 450 in response to working as in the equipment of individual touch sensitive surface 451 It detects the contact on touch sensitive surface 451 when point selector and executes similar operation.

Fig. 5 A is shown using the background about Fig. 5 A-5CO user interface described.In particular, Fig. 5 A shows object The view in space 5000 is managed, desk 5002 and equipment 100 are located in the physical space.Equipment 100 is held by user 5004, with logical The touch screen 112 for crossing equipment 100 checks the physical space 5000 of a part including desk 5002.In particular, touch screen 112 Show the view of augmented reality environment corresponding with physical space 5000.User 5004 uses the touch screen 112 of equipment 100 It is handed over via the user interface 5006 (for example, user interface of augmented reality measurement application program) of display with augmented reality environment Mutually.User interface 5006 includes at least one magazine (sometimes referred to as " phase of equipment 100 of one or more of equipment 100 Machine ", such as camera 305 (Fig. 3 A-3B) or camera 311 (Fig. 3 C) optionally include optical sensor 164 (Figure 1A) as one A part of a or multiple cameras) the visual field live preview.In some embodiments, camera is located at the area in equipment 100 In domain 5008.In some embodiments, equipment 100 includes preceding to camera 305-a, be located at it is adjacent with touch screen 112 and In the region 5008 with the touch screen the same side (for example, side of the user oriented 5004 in Fig. 5 A) of equipment 100.In In some embodiments, equipment 100 includes one or more cameras, be located in the subsequent region 5008 of touch screen 112 or Positioned at the side (for example, side of user 5004 away from Fig. 5 A in) opposite with touch screen 112 of equipment 100.At least one Camera is continually provided in the live preview of the content in camera fields of view, may include one or more in physical space 5000 A physical object (for example, desk 5002).

In some embodiments, user interface 5006 includes one for interacting with augmented reality environment generation user Or multiple user interface elements.For example, in fig. 5, user interface 5006 includes graticule (reticle) 5010, instruction and increasing The region of user's interaction occurs for strong actual environment.In some embodiments, graticule 5010 includes focus 5012, and instruction is used for The specified point of user's interaction.In some embodiments, user interface 5006 includes measurement addition button 5014, and being used for will be new Measurement (for example, new measurement point, new measuring section and/or new measured zone) is added to user interface 5006 (for example, such as herein more Detailed description).In some embodiments, graticule 5010 and focus 5012 are formed together measurement point creation indicator, refer to Show the position that the activation in response to measurement addition button 5014 newly measures addition.

In some embodiments, user interface 5006 includes media capture button 5016, is used to capture the matchmaker in the visual field Body, such as static image or video (optionally include corresponding audio), and corresponding to any of the physical object in the visual field Virtual measurement.In some embodiments, user interface 5006 includes revocation button 5018, is used to cancel user interface The operation (for example, reversing its execution) executed recently in 5006.In some embodiments, user interface 5006 includes reforming Button 5020, be used to reform cancelled recently in user interface 5006 operation (for example, by activation revocation button 5018 come Reverse the reversing of the operation executed recently).User interface 5006 can also include one or more virtual measurements, correspond to object One or more physical objects in space 5000 are managed, and are at least partially in response to the user interface member of user interface 5006 User on element inputs and shows.In some embodiments, user interface 5006 includes removing button 5022, is used to remove Shown in user interface 5006 virtual measurement (and optionally, also remove when remove button 5022 be activated when not with The virtual measurement shown in family interface 5006, such as corresponding to when remove button 5022 be activated when except camera fields of view The virtual measurement of physical object).

Fig. 5 B shows the equipment 100 in the first horizontal orientation, is contrasted with machine-direction oriented in Fig. 5 A.Work as equipment 100 when being in the first horizontal orientation, and as shown in Figure 5 B, one or more cameras of equipment 100 are located at the region on the left of equipment 100 It in 5008, and measures addition button 5014 and is shown in the user interface 5006 in the separate region 5008 in 100 right side of equipment, to subtract Few user 5004 provides the visual field of the camera of the live preview in user interface 5006 in left side gripping device 100 and blocking Chance, and also promote when user 5004 holds 100 right side of equipment (for example, when user 5004 is held with his/her right hand When equipment 100) user carries out interaction with augmented reality environment during one-handed performance.

Fig. 5 C shows the equipment 100 in the second horizontal orientation, which is different from the horizontal orientation (example in Fig. 5 B Such as, 180 degree is rotated from the horizontal orientation in Fig. 5 B).When equipment 100 is in the second horizontal orientation, as shown in Figure 5 C, equipment 100 one or more cameras are located in the region 5008 on 100 right side of equipment, and measure addition button 5014 and be shown in equipment 100 left sides are mentioned far from the user interface 100 in region 5008 with reducing user 5004 in right side gripping device 100 and blocking For the chance in the visual field of the camera of the live preview in user interface 5006, and also promote to hold equipment 100 in user 5004 During user carries out one-handed performance when left side (for example, when user 5004 holds equipment 100 with his/her left hand) with enhancing The interaction of actual environment.

Fig. 5 D shows an example of the error condition of augmented reality environment.Particularly, Fig. 5 D is shown when without foot Enough light for physical space 5000 when different physical objects and physical features in equipment 100 for identification camera fields of view view Figure.Equipment 100 shows error message 5024 (for example, comprising text " too dark " and " opening lamp is to detect in user interface 5006 Surface ") come indicate error condition and prompt user 5004 increase physical space 5000 in light quantity.In some embodiments, When there are error conditions, when so that equipment 100 can not identify physical object or feature in camera fields of view, equipment 100 stops aobvious Indicating line 5010 and/or focus 5012 indicate the error condition.

Fig. 5 E shows another example of the error condition of augmented reality environment.Particularly, Fig. 5 E, which shows to work as, foot (example when enough light is available but equipment 100 does not detect the surface of the physical object in physical space 5000 in camera fields of view Such as, user 5004 has responded to open lamp in error message 5024), the view of physical space 5000.Equipment 100 is in user circle Error message 5026 (for example, " moving from one side to the other equipment to detect surface " comprising text) is shown in face 5006 To indicate the error condition and user 5004 prompted to move from one side to the other equipment 100 (for example, to promote equipment 100 The image procossing of progress detects the physical surface in camera fields of view).Mobile arrow 5028 indicates user 5004 in response to mistake Message 5026 moves left and right equipment 100.In some embodiments, when equipment 100, not in live preview, focus 5012 is aobvious When showing the surface for detecting physical object at position above, error message 5026 is shown.In some embodiments, such as Mobile device 100 shows different error messages, then to detect the surface of physical object with instruction to fruit user 5004 too quickly Error condition simultaneously prompts the more slowly mobile device 100 of user 5004.

Fig. 5 F shows the physical space 5000 when equipment 100 detects the surface of physical object in physical space 5000 View.Particularly, in Fig. 5 F, equipment 100 has been detected by the top surface of desk 5002 (for example, the left side based on equipment 100 It moves right, above with reference to as described in Fig. 5 E, while focus 5012 is located in live preview on the top surface of desk 5002).Root According to surface is detected, other than showing focus 5012, equipment 100 also shows graticule 5010 in user interface 5006.One In a little embodiments, detected whenever focus 5012 is located in live preview with the physical object in physical space 5000 When on the corresponding region in surface, show graticule 5010 (for example, to indicate that focus 5012 is disposed thereon in live preview Region corresponding to the surface detected).In some embodiments, the surface that graticule 5010 is tilted to seem and detect It is coplanar, as illustrated in figure 5f, to indicate the surface detected by equipment.

Fig. 5 G shows the transformation from Fig. 5 F.In Fig. 5 G, user 5004 moves equipment 100, so that focus 5012 is fixed Position is not corresponded on the point on the surface detected in live preview (for example, focus 5012 does not reposition in live preview On the top surface of the desk 5002 detected).Therefore, equipment 100 changes the appearance of graticule 5010 (for example, by no longer Graticule 5010 is tilted, by the way that graticule is changed into filled circles, and/or by stopping display graticule), while continuing to show focus 5012。

Fig. 5 H shows the transformation from Fig. 5 G.In fig. 5h, user 5004 moves equipment 100, so that 5012 He of focus At least part of graticule 5010 is repositioned on the top surface of the desk 5002 detected in live preview.Equipment 100 detect the edge 5030 (for example, edge of the top surface detected) of desk 5002 in live preview.Edge 5030 At least partly in graticule 5010 (for example, focus 5012 edge 5030 predetermined threshold distance in, wherein predetermined threshold away from From the radius for being at most graticule 5010).Therefore, graticule 5010 and the visual appearance of focus 5012 change.Particularly, focus 5012 It has moved and has been arrived (for example, vertically downward) or " alignment " is to a point along edge 5030.In addition, reducing the ruler of graticule 5010 It is very little to indicate that focus point 5012 has snapped to the feature detected in live preview.In some embodiments, Mei Dangjiao When point 5012 is aligned to the feature detected in live preview, graticule 5010 is shown with size shown in Fig. 5 H.One In a little embodiments, when focus 5012 snaps to the edge or turning in live preview corresponding to the physical object in camera fields of view The feature detected when, the visual appearance of graticule 5010 and focus 5012 changes.In some embodiments, work as graticule 5010 and focus 5012 be located in the surface of the physical object corresponded in camera fields of view in live preview and (but be not edge or turn Angle) the feature detected on when, the visual appearance of graticule 5010 and focus 5012 is constant.In some embodiments, it ties It closes and focus 5012 is moved to edge 5030, equipment 100 optionally (for example, exporting generator 163, Figure 1A using tactile) generates Tactile output 5032 is with instruction point 5012 by the feature detected in the live preview of " alignment " to camera.

Fig. 5 I shows the transformation from Fig. 5 H.In Fig. 5 I, user 5004 moves equipment 100, so that edge 5030 Any part is not in graticule 5010.Therefore, focus 5012 is left from edge 5030 and is shown in graticule 5010 again Center (for example, because no longer there is any feature that focus 5012 should be aligned in graticule 5010).In addition, right in focus 5012 It arrives before edge 5030 together, the size of graticule 5010 increases to its size as illustrated in figure 5f again.In some embodiments In, when focus 5012 is not aligned to any feature detected in live preview, graticule 5010 is by default It is shown with size shown in Fig. 5 F and 5I.In some embodiments, whenever focus 5012 is unjustified into live preview When any feature detected, focus 5012 is shown in the center of graticule 5010 by default.In some embodiments In, as shown in fig. 5i, equipment 100 generates tactile output 5034 optionally in combination with focus 5012 is moved away from edge 5030, To indicate that focus 5012 no longer snaps to the feature detected in camera live preview.In some embodiments, tactile is defeated Out 5034 with tactile export 5032 (Fig. 5 H) at least one tactile output attribute (for example, frequency, amplitude and/or mode) not Together, so that the tactile that instruction snaps to the feature detected exports 5032 (Fig. 5 H) and provides and indicated from detecting to user 5004 The tactile left of feature export 5034 different touch feedbacks.

Fig. 5 J-5O shows the creation of the measurement to the horizontal left edge of desk 5002.Fig. 5 J is shown from Fig. 5 I's Transformation.In fig. 5j, user 5004 moves equipment 100, so that as shown in live preview, the rear upper left corner of desk 5002 In graticule 5010.Therefore, focus 5012 snaps to the anchor point in live preview corresponding to the rear upper left corner of desk 5002.In In some embodiments, as long as the rear upper left corner of desk 5002 is in graticule 5010 in live preview (for example, although equipment 100 May slightly be moved due to being not intended to the unstable of the mobile such as hand of user 5004 of user 5004), focus 5012 is maintained at On anchor point in live preview.Reduce the size of graticule 5010 to indicate alignment behavior (for example, being reduced to above with reference to Fig. 5 H Shown and description identical size).In addition, generating tactile output 5036 to indicate the alignment behavior.

Fig. 5 K-5L shows the transformation from Fig. 5 J, shows to user interface 5006 and adds measurement point.Particularly, Fig. 5 K shows Go out through touch input 5038 (for example, Flick gesture) activation measurement addition button 5014, which, which has, is higher than most Small contact detection threshold value IT₀Contact strength, as shown in intensitometer 5040.Addition button 5014, equipment are measured in response to activation (virtual) measurement point 5042 is added to user interface 5006 in the current position of focus 5012 and shown by 100.In conjunction with will survey Amount point 5042 is added to user interface 5006, and equipment 100 optionally generates tactile output 5044 to indicate the addition of measurement point. In some embodiments, tactile output 5044 exports 5032 (Fig. 5 H) with tactile and tactile exports 5034 (Fig. 5 I) at least It is different on one tactile output attribute (for example, frequency, amplitude and/or mode), so that the tactile output of instruction alignment behavior 5032 (Fig. 5 H) and tactile export 5034 (Fig. 5 I) and provide and the tactile for the addition for indicating measurement point output 5044 to user 5004 Different touch feedbacks.Fig. 5 L shows touch input 5038 and is lifted away from from measurement addition button 5014.

Fig. 5 M shows the transformation from Fig. 5 L.In Fig. 5 M, user 5004 is by equipment 100 diagonally downwards and towards the right side It is mobile, so that graticule 5010 is located on the different location in physical space 5000, as shown in live preview.Particularly, In Fig. 5 M, as shown in live preview, graticule 5010 is located on the upper left corner of desk 5002.Therefore, focus 5012 Snap to the anchor point in live preview corresponding to the upper left corner of desk 5002.Reduce the size of graticule 5010 to indicate to be aligned Behavior (for example, being reduced to the identical size above with reference to shown in Fig. 5 J and described).Tactile output 5046 is generated to indicate that this is right Justification is.In addition, measurement point 5042 continues to show on the rear upper left corner of desk 5002 (for example, measurement point in live preview 5042 is associated with the rear upper left corner of desk 5002 and show in the position in live preview, even if in live preview The position in the upper left corner changes with the movement of equipment 100 after desk 5002).

Movement in response to equipment 100 is so that graticule 5010 and focus 5012 are located in the difference in physical space 5000 On position, measuring section 5048 is shown between measurement point 5042 (measurement point added recently) and focus 5012.Measuring section 5048 display together with label 5049, and label instruction is empty corresponding to the point of measurement point 5042 and physics in physical space 5000 Between correspond to the distance between the point of focus 5012 (for example, " 3 feet ") in 5000.In the second end of addition measuring section 5048 Before point, measuring section 5048 is dynamic measuring section, is had as the first end point of measurement point 5042 and as focus 5012 Current location the second endpoint (for example, the length of measuring section 5048 and label 5049 indicate correspond to measuring section 5048 Both distances change the point in physical space 5000 to the corresponding point in the current location of focus 5012 according to equipment 100 It moves and updates).In addition, along in physical space 5000 on the moving direction of equipment 100 from the measurement point added recently i.e. The feature that measurement point 5042 extends shows virtual guidance 5050.In particular, virtually guidance 5050 is along desk 5002 in reality When preview in diagonally downwards and shown towards the horizontal left edge of right extension.It may be noted that the left edge of desk 5002 and Other edges of the top surface of desk 5002 are referred to herein as horizontal edge, because they are horizontal in physical space 5000 , even if they may be diagonal from the viewpoint of equipment 100 in live preview.

Fig. 5 N-5O shows the transformation from Fig. 5 M, shows to user interface 5006 and adds measurement point.Particularly, Fig. 5 N shows Go out through touch input 5052 (for example, Flick gesture) activation measurement addition button 5014, which, which has, is higher than most Small contact detection threshold value IT₀Contact strength, as shown in intensitometer 5040.Addition button 5014, equipment are measured in response to activation 100 in user interface 5006 current position of focus 5012 add and show (virtual) measurement point 5054 and as survey Measure the second endpoint of section 5048.It is added to user interface 5006 in conjunction with by measurement point 5040, equipment 100 optionally generates tactile Output 5056 is to indicate the addition of measurement point.In some embodiments, tactile output 5056 is and tactile output 5044 (is schemed 5K) the tactile output of same type, indicates the addition of measurement point.In some embodiments, tactile output 5056 is defeated with tactile 5044 (Fig. 5 K) are different at least one tactile output attribute (for example, frequency, amplitude and/or mode) out, so that tactile is defeated 5056 the touch feedback different from tactile 5044 (Fig. 5 K) of output is provided to user 5004 out.In some such embodiments In, tactile exports the addition that 5044 (Fig. 5 K) instruction starts the measurement point of new measuring section, and (example is completed in 5056 instruction of tactile output Such as, terminate) addition of the measurement point of measuring section.Fig. 5 O shows touch input 5052 and is lifted away from from measurement addition button 5014.In After touch input 5052 is lifted away from, equipment 100 continues to show measurement point 5042, measurement point 5054, connect in user interface 5006 Measurement point 5042 and 5054 measuring section 5048 and the (survey of the horizontal left edge corresponding to desk 5002 of label 5049 Amount).In some embodiments, as shown in Fig. 5 O, when completing measuring section, equipment 100 stops the virtual guidance of display, such as Virtual guidance 5050.

Fig. 5 P shows the transformation from Fig. 5 O.In Fig. 5 P, user 5004 moves equipment 100, so that graticule 5010 exists It is located in live preview on the side surface adjacent with measuring section 5048 of desk 5002.Equipment 100 has determined (for example, portion Ground is divided to be based on measuring section 5048), the region that graticule 5010 (or more specifically, focus 5012) is positioned on corresponds to physics In space 5000 physics rectangular area (for example, although detection zone in live preview from equipment 100 from the point of view of show as It is trapezoidal).Therefore, 100 display indicator 5058 of equipment is to indicate that detection zone corresponds to physics rectangular area.In addition, equipment 100 Continue to show the label on measurement point 5042, measurement point 5054, measuring section 5048 and the horizontal left edge of desk 5002 5049.In some embodiments, whenever the physical object corresponding to virtual measurement is (for example, the horizontal upper left side of desk 5002 Edge) it is visible in live preview when, the measurement element of a part as virtual measurement is (for example, measurement point 5042, measuring section 5054, measuring section 5048 and label 5049) it is shown in user interface 5006, until corresponding virtual measurement is removed.

Fig. 5 Q-5X shows the creation of the measurement to the vertical preceding left edge of desk 5002.Fig. 5 Q is shown from Fig. 5 P's Transformation.In Fig. 5 Q, user 5004 moves equipment 100, so that graticule 5010 is repositioned at the upper left corner of desk 5002 On, as shown in live preview.Therefore, focus 5012, which snaps in live preview, corresponds to the upper left corner of desk 5002 simultaneously And correspond to the anchor point of measurement point 5054.Reduce the size of graticule 5010 to indicate alignment behavior (for example, being reduced to above With reference to shown in Fig. 5 M and description identical size).Tactile output 5060 is generated to indicate the alignment behavior.Equipment 100 stops aobvious Show indicator 5058 (for example, because focus 5012 has snapped to the anchor point of the upper left corner corresponding to desk 5002, and It is not then displayed in live preview on the detection zone corresponding to the physics rectangular area of the side surface of desk 5002).

Fig. 5 R shows the transformation from Fig. 5 Q, shows to user interface 5006 and adds measurement point.Particularly, Fig. 5 R is shown Measurement addition button 5014 is activated by touch input 5062 (for example, Flick gesture), which has connects higher than minimum Touch detection threshold value IT₀Contact strength, as shown in intensitometer 5040.In some embodiments, as shown in Fig. 5 R, in response to swashing Measurement addition button 5014 living, equipment 100 use measurement point 5054 as the starting point of new measuring section to be added.In some implementations In scheme, addition button 5014 is measured in response to activation, equipment 100 is added and shown at position identical with measurement point 5054 Show another (virtual) measurement point, the starting point as new measuring section.It is combined into new measuring section and establishes any of the above-described starting point, equipment 100 It optionally generates tactile output 5064 and (and in some embodiments, is surveyed added by instruction with the addition for indicating measurement point Amount point starts new measuring section).

Fig. 5 S shows the transformation from Fig. 5 R.In Fig. 5 S, by equipment 100, horizontally court moves right user 5004, So that graticule 5010 is located on some position along the horizontal leading edge of desk 5002, and make the table in live preview The upper left corner of son 5002 is no longer in graticule 5010.Therefore, focus 5012 is shown in the horizontal front corresponding to desk 5002 (for example, the midpoint in the part of 5002 leading edge of desk being located in graticule 5010, is along desk on the anchor point of edge 5002 leading edges be located at graticule 5010 in part apart from the shortest point of 5010 centre distance of graticule).Graticule 5010 is with it Reduced size shows (for example, identical as the size in Fig. 5 R), to indicate that focus 5012 snaps to the detection in live preview The feature arrived.In addition, measurement point 5054 continues to be shown in live preview on the upper left corner of desk 5002, and move State measuring section 5066 is shown between measurement point 5054 and focus 5012.Label 5068 indicates to correspond in physical space 5000 Correspond to the distance between the point of focus 5012 in the point and physical space 5000 of measurement point 5054.In addition, in response to equipment 100 Move horizontally, equipment 100 display from measurement point 5054 it is horizontal-extending it is virtual guidance 5070.

In some embodiments, as shown in Fig. 5 S, tactile output is not generated to indicate alignment behavior.For example, working as table At least some parts of the horizontal leading edge of son 5002 are maintained in graticule 5010, so that focus 5012 is in the mobile phase of equipment 100 Between any time when all not left from the horizontal leading edge of desk 5002, although 5012 institute of focus when equipment 100 is mobile The specific anchor point of alignment changes, but equipment 100 is abandoned generating tactile output, so as in focus 5012 along detecting Edge it is mobile when avoid continuously generating tactile output.

Fig. 5 T shows the transformation from Fig. 5 S.In Fig. 5 T, the moving direction of equipment 100 becomes vertical from horizontal direction court Downwards.It vertically moves in response to moving direction from moving horizontally to be changed to, equipment 100 stops the virtual guidance of display (level) 5070 (Fig. 5 S), but show that (vertical) virtually guides 5072, the guidance is on the moving direction of equipment 100 from measurement point 5054 It extends vertically.Focus 5012 snaps to the anchor point of the vertical interior left edge 5074 corresponding to desk 5002, and in some realities It applies in scheme, as shown in Fig. 5 T, equipment 100 generates tactile output 5076 to indicate the alignment behavior.Dynamic measuring section 5066 The continuous updating according to the movement of equipment 100, to be shown between measurement point 5054 and the current location of focus 5012.In addition, The continuous updating according to the movement of equipment and the update of dynamic measuring section 5066 of label 5068, so that label 5068 is shown in dynamic The midpoint of measuring section 5066.

Fig. 5 U shows the transformation from Fig. 5 T, shows to user interface 5006 and adds measurement point.Particularly, Fig. 5 U is shown Measurement addition button 5014 is activated by touch input 5078 (for example, Flick gesture), which has connects higher than minimum Touch detection threshold value IT₀Contact strength, as shown in intensitometer 5040.Addition button 5014 is measured in response to activation, equipment 100 exists (virtual) measurement point 5080 is added to user interface 5006 and shown by the current position of focus 5012, and as measuring section 5066 the second endpoint, the measuring section which becomes completing is (for example, its second endpoint is measurement point 5080 and not now It is focus 5012 again) rather than dynamic measuring section.It is added to user interface 5006 in conjunction with by measurement point 5040, equipment 100 is optional Ground generates the tactile output 5082 of the addition of instruction measurement point, and (and the measurement point that optionally special instructions complete measuring section adds Add, such as exported using the tactile for exporting 5056 same types with the tactile of Fig. 5 N).In some embodiments, such as Fig. 5 U institute Show, even if equipment 100 continues to show virtual guidance such as virtual guidance 5072 (for example, until setting after completing measuring section Until standby 100 leave its current location).

Fig. 5 V shows the transformation from Fig. 5 U comprising executes " revocation " operation to reverse in user interface 5006 recently The operation of execution.Activate revocation button 5018 to reverse measurement point by touch input 5084 in particular, Fig. 5 V is shown 5080 addition (Fig. 5 U).Therefore, measurement point 5080 is removed from user interface 5006.Measuring section 5066 becomes (again) now Dynamic measuring section, the second endpoint are focuses 5012, and are updated as focus 5012 is mobile.

Fig. 5 W shows the transformation from Fig. 5 V.In Fig. 5 W, user 5004 has moved down equipment 100, so that graticule 5010 are located on the preceding lower left corner of desk 5002, as shown in live preview.Therefore, focus 5012 is snapped to corresponding to table The anchor point (and optionally snapping to virtual guidance 5072) in the preceding lower left corner of son 5002.The size that graticule 5010 is reduced with it It shows (for example, identical as the size in Fig. 5 U), to indicate that focus 5012 snaps to the feature detected in live preview.It is raw At tactile output 5086 to indicate the alignment behavior.In Fig. 5 X, equipment 100 has been moved, so that measuring section 5048 only has one It is partially shown in user interface 5006, because the rear upper left corner of desk 5002 is no longer visible in live preview, in reality When preview in only desk 5002 horizontal left edge it is a part of visible.Therefore, label 5049 is merely displayed in measuring section The midpoint of 5048 display portion, the midpoint (for example, as shown in Fig. 5 V) without being shown in entire measuring section 5048.

Fig. 5 X shows the transformation from Fig. 5 W, shows to user interface 5006 and adds measurement point.Particularly, Fig. 5 X is shown Measurement addition button 5014 is activated by touch input 5088 (for example, Flick gesture), which has connects higher than minimum Touch detection threshold value IT₀Contact strength, as shown in intensitometer 5040.Addition button 5014 is measured in response to activation, equipment 100 exists The current location of focus 5012 is added (virtual) measurement point 5090 to user interface 5006 and is shown, and as measuring section 5066 The second endpoint, the measuring section become complete measuring section.It is added to user interface 5006, equipment in conjunction with by measurement point 5090 100 optionally generate instruction measurement point addition tactiles output 5092 (and optionally instruction complete measuring section measurement point Addition, such as exported using the tactile that the tactile with the tactile of Fig. 5 N output 5056 or Fig. 5 U exports 5082 same types).

Fig. 5 Y-5AF is shown corresponding to the physics rectangular area (for example, surface of desk 5002) in physical space 5000 Measured zone creation.Particularly, Fig. 5 Y-5Z shows the creation of the measured zone corresponding to physics rectangular area, the object Reason rectangular area was once completely shown in live preview.Fig. 5 Y shows the transformation from Fig. 5 X.In Fig. 5 Y, user 5004 It is that equipment 100 is mobile from its position in Fig. 5 X, so that graticule 5010 is positioned in desk in live preview It is adjacent with measuring section 5048 and measuring section 5066 on 5002 side surface.Equipment 100 has determined graticule 5010 in live preview The region of (or more specifically, focus 5012) positioning above corresponds to the physics rectangular area (example in physical space 5000 Such as, which is based in part on measuring section 5048 and measuring section 5066 is the sides adjacent of detection zone).Therefore, equipment 100 is aobvious Show indicator 5094 to indicate that the detection zone that focus 5012 is disposed thereon in live preview corresponds to physics rectangular area.This Outside, equipment 100 continues to show measuring section 5048, corresponding label 5049, measuring section 5066, corresponding label 5068, Yi Jiduan Point 5042,5054 and 5090.Fig. 5 Y is similar to Fig. 5 P, unlike in addition to endpoint 5042 and 5054 and label 5049 (the first) except measuring section 5048, Fig. 5 Y include (second) measuring section 5066, the measuring section have endpoint 5054 and 5090 and Mark preceding 5068.

Fig. 5 Z shows the transformation from Fig. 5 Y, shows to user interface 5006 and adds measured zone.Particularly, Fig. 5 Z is shown When focus 5012 is on the detection zone that indicator 5094 indicates, pass through touch input 5096 (for example, Flick gesture) Activation measurement addition button 5014, which, which has, is higher than minimal-contact detection threshold value IT₀Contact strength, such as intensitometer Shown in 5040.Addition button 5014 is measured in response to activation, the addition of equipment 100 corresponds to the measurement of detection zone and change refers to Show the appearance of symbol 5094 to indicate that detection zone has been confirmed to be measured zone.In addition, 100 display label 5098 of equipment, refers to Show the area (for example, " 7.5 square feet ") for corresponding to confirmation region of physics rectangular area.Optionally, in some embodiment party In case, before detection zone is confirmed to be measured zone, label 5098 is display together with indicator 5094 (for example, some In embodiment, the indicator 5094 in label 5098 and Fig. 5 Y is display together).

Fig. 5 AA shows the transformation from Fig. 5 Z.In Fig. 5 AA, user 5004 moves equipment 100 vertically upward, makes Graticule 5010 is located in live preview on the horizontal left edge of desk 5002 and measuring section 5048.In particular, surveying The midpoint of section 5048 is measured in graticule 5010.In some embodiments, it as shown in Fig. 5 AA, is shown in user interface 5006 The midpoint of corresponding measuring section can be the anchor point that focus 5012 is aligned.Therefore, in Fig. 5 AA, focus 5012 is snapped to Anchor point corresponding to 5048 midpoint of measuring section (and the midpoint for also corresponding to the horizontal left edge of desk 5002).Such as preceding institute It states, reduces the size of graticule 5010 to indicate the alignment behavior.Tactile output 5100 is generated to indicate the alignment behavior.In addition, Equipment 100 continues to show confirmation area indicator in the detection zone of the physics rectangular area corresponded in physical space 5000 5094 (for example, confirmation area indicator 5094 is associated with the detection zone of physics rectangular area is corresponded to, and even if detection It when region is moved in live preview as equipment 100 is mobile, is also shown on detection zone in live preview).

Fig. 5 AB-5AE shows the creation of the measured zone corresponding to the physics rectangular area in physical space 5000, should Physics rectangular area is partially visible in live preview but once whole invisible (for example, there was only three sides in live preview (all or part) visible physics rectangular area).Fig. 5 AB shows the transformation from Fig. 5 AA.In Fig. 5 AB, user 5004 By equipment 100 horizontally towards moving right so that graticule 5010 be located in live preview it is adjacent with measuring section 5048 and On the region of partial view corresponding to 5002 top surface of desk.Particularly, the left edge of the top surface of desk 5002 be can To see, a part of the leading edge of the top surface of desk 5002 is visible, and the back edge of the top surface of desk 5002 A part is visible, and the right hand edge of the top surface of desk 5002 is sightless.Focus 5012 is from corresponding to measurement The anchor point at 5048 midpoints leaves and shows again at the center of graticule 5010.Graticule 5010 with its increase size again It has been shown that, and tactile output 5102 is removed from anchor point optionally in combination with focus 5012 and is generated.In addition, equipment 100 is really Fixed (for example, being based in part on measuring section 5048), the region pair that graticule 5010 (or more specifically, focus 5012) is positioned on Should in the physics rectangular area in physical space 5000 a part (for example, although detection zone is in live preview from equipment Rectangle is not appeared as from the point of view of 100).Therefore, 100 display indicator 5104 of equipment is to indicate that detection zone corresponds to physics square Shape region.Optionally, according to the movement of equipment 100, virtual guidance 5106 is shown.Virtual guidance 5106 horizontally extends and right It should be identified as the feature horizontally extended on the moving direction of equipment 100 in live preview.In some embodiments, In multiple features in the case where extending on the moving direction of equipment 100, it is shown in multiple virtual guidance of moving direction extension. In some embodiments, the feature for being only added to corresponding measurement point on it in user interface 5006 is shown in shifting The virtual guidance that dynamic side upwardly extends.

Fig. 5 AC-5AD is shown when graticule 510 is located on the top surface of desk 5002 in live preview, equipment 100 further move horizontally.In Fig. 5 AC, one of a part of the leading edge of the top surface of desk 5002 and back edge Divide and is visible, and the left edge and right hand edge of the top surface of desk 5002 are all invisible.In Fig. 5 AD, in addition to leading edge Except a part of a part and back edge, the right hand edge of the top surface of desk 5002 is visible now.Equipment 100 continues In live preview on the visible part of the top surface of desk 5002 display indicator 5104 and the level according to equipment 100 Mobile display horizontal imaginary guidance.

Fig. 5 AE shows the transformation from Fig. 5 AD, shows to user interface 5006 and adds measured zone.Particularly, Fig. 5 AE It shows when focus 5012 is on the detection zone that indicator 5104 indicates, by touch input 5108 (for example, tap hand Gesture) activation measurement addition button 5014, which, which has, is higher than minimal-contact detection threshold value IT₀Contact strength, it is such as strong Shown in degree meter 5040.Addition button 5014 is measured in response to activation, the addition of equipment 100 corresponds to the measurement of entire detection zone, Although detection zone only some as it can be seen that and changing the appearance of indicator 5104 to indicate that detection zone has been confirmed to be survey Measure region.In addition, 100 display label 5110 of equipment, label 5112 and label 5114.The instruction of label 5110 corresponds to (entire) Confirm the area of (entire) the physics rectangular area in region.The instruction of label 5112 corresponds to the physics rectangular area in confirmation region The length of first side (for example, right hand edge of the top surface of desk 5002).The instruction of label 5114 corresponds to the physics in confirmation region The length of second side (for example, leading edge of the top surface of desk 5002) adjacent with the first side of rectangular area.Optionally, In In some embodiments, before detection zone is confirmed to be measured zone, label 5114 and indicator 5104 display together (example Such as, in some embodiments, the indicator 5104 in label 5114 and Fig. 5 AD is display together).

Fig. 5 AF shows the transformation from Fig. 5 AE.In Fig. 5 AF, by equipment 100, horizontally court moves to left user 5004 It is dynamic, so that the left part of desk 5002 is visible in live preview.Indicator 5094, measuring section 5048 and measuring section 5066 (and their respective labels) are in user interface 5006 in the corresponding position of its individual features for corresponding to desk 5002 Display (for example, being side surface, horizontal left edge and the vertical preceding left edge of desk 5002 respectively), instant equipment 100 was previously It removes visible position in user interface 5006 from the character pair of these elements and desk 5002, and has moved back into now To the character pair of these elements and desk 5002 in user interface 5006 visible position (institute in such as Fig. 5 AF (again) Those of show).In addition, the visible portions that indicator 5104 (and its respective labels) is shown in the top surface of desk 5002 divide it On, even if true when equipment 100 shows the current sightless different piece in user interface 5006 of 5002 top surface of desk The detection zone accepted corresponding to the top surface of desk 5002.

Fig. 5 AG, which is shown, to be shown on live preview from the removing of user interface 5006 and corresponds to physical space 5000 All measurements (e.g., including the measurement corresponding to the physical object being not currently in camera fields of view).In Fig. 5 AG, response In removing button 5022 by the activation of touch input 5116, measuring section 5048 and 5066, measurement point are removed from user interface 506 5042,5054 and 5090, indicator 5094 and 5104 and all corresponding labels.

Fig. 5 AH-5AS shows the variation of the intensity based on continuous touch input continuously to create measurement.Particularly, scheme 5AH-5AM shows the creation of the measurement to the horizontal left edge of desk 5002.Fig. 5 AH is optionally shown from Fig. 5 AG's Transformation.In Fig. 5 AH, user 5004 moves equipment 100, so that as shown in live preview, in the rear left of desk 5002 Angle is in graticule 5010.Therefore, focus 5012 snaps to the anchor point in live preview corresponding to the rear upper left corner of desk 5002. Graticule 5010 is shown with its size reduced to indicate alignment behavior.Optionally, in addition generating tactile output 5118 should with instruction Alignment behavior.

Fig. 5 AI shows the transformation from Fig. 5 AH, shows to user interface 5006 and adds measurement point.Particularly, Fig. 5 AI shows Go out through touch input 5120 (for example, light press gesture) activation measurement addition button 5014, which, which has, is higher than Light press intensity threshold IT_LContact strength, as shown in intensitometer 5040.Addition button 5014, equipment are measured in response to activation (virtual) measurement point 5122 is added to user interface 5006 in the current position of focus 5012 and shown by 100.In conjunction with will survey Amount point 5122 is added to user interface 5006, and increases to flicking Compressive Strength threshold in response to the contact strength of touch input 5120 Value IT_LMore than, equipment 100 optionally generates tactile output 5124 to indicate the addition of measurement point.In some embodiments, such as Shown in Fig. 5 AI, as the contact strength of touch input 5120 increases, the size of measurement addition button 5014 reduces (for example, figure Measurement addition button 5014 in 5AI is smaller, and the contact strength of touch input 5120 is compared in Fig. 5 K in flicking pressure at this time Spend threshold value IT_LMore than, when in Fig. 5 K, the contact strength of touch input 5038 is in minimal-contact detection threshold value IT₀With flicking pressure Spend threshold value IT_LBetween).

Fig. 5 AJ shows the transformation from Fig. 5 AI, and display touch input 5120 is maintained on measurement addition button 5014, Contact strength is higher than minimal-contact detection threshold value IT₀But have already decreased to light press intensity threshold IT_LBelow.In some implementations In scheme, equipment 100 is reduced to light press intensity threshold IT in the contact strength for detecting touch input 5120_LIt is raw when following It exports at tactile (for example, not being in response to increase to light press intensity threshold IT in the contact strength of touch input 5120_LIt is above raw At tactile output 5124, as shown in Fig. 5 AI, but flicking Compressive Strength threshold is reduced in response to the contact strength of touch input 5120 Value IT_LIt is exported below to generate the tactile, as shown in Fig. 5 AJ).

Fig. 5 AK shows the transformation from Fig. 5 AJ.In Fig. 5 AK, user 5004 is by the diagonally downward simultaneously court of equipment 100 It moves right, while keeping touch input 5120 on measurement addition button 5014, contact strength is higher than minimal-contact detection threshold Value IT₀And it is lower than light press intensity threshold IT_L.Equipment 100 is moved, so that graticule 5010 is located in the front left of desk 5002 On upper angle, as shown in live preview.Therefore, focus 5012 snaps in live preview the front left for corresponding to desk 5002 The anchor point at upper angle.Graticule 5010 is shown with its size reduced to indicate the alignment behavior, and optionally, and it is defeated to generate tactile 5126 out to indicate the alignment behavior.Measurement point 5122 continue to be shown in live preview desk 5002 the rear upper left corner it On, and dynamic measuring section 5128 is shown between the current location of focus 5012 and measurement point 5122.In addition, according to equipment 100 movement shows virtual guidance 5130, wherein virtually guidance 5130 diagonally extends from measurement point 5122 and along user The horizontal left edge of desk 5002 extends in interface 5006.

Fig. 5 AL shows the transformation from Fig. 5 AK, shows to user interface 5006 and adds measurement point.In particular, Fig. 5 AL It shows when touch input 5120 is maintained on measurement addition button 5014, by increasing the contact strength of touch input 5120 It is added to and is higher than light press intensity threshold IT_L, as shown in intensitometer 5040, to activate measurement addition button 5014.In response to activation (virtual) measurement point 5132 is added to user circle in the current position of focus 5012 by measurement addition button 5014, equipment 100 Face 5006 is simultaneously shown, and the second endpoint as measuring section 5128.It is added to user interface in conjunction with by measurement point 5132 5006, and increase to light press intensity threshold IT in response to the contact strength of touch input 5120_LMore than, equipment 100 is optional Ground generates tactile output 5134 to indicate the addition of measurement point.In some embodiments, as shown in Fig. 5 AL (and as previously With reference to described in Fig. 5 AI), as the contact strength of touch input 5120 increases, the size of measurement addition button 5014 reduces.It is optional Ground, as shown in Fig. 5 AL, when completing measuring section 5128, equipment 100 stops the virtual guidance 5130 of display.

Fig. 5 AM shows the transformation from Fig. 5 AL, and display touch input 5120 is maintained on measurement addition button 5014, Contact strength is higher than minimal-contact detection threshold value IT₀But have already decreased to light press intensity threshold IT_LBelow.In some implementations In scheme, equipment 100 is reduced to light press intensity threshold IT in the contact strength for detecting touch input 5120_LIt is raw when following It is exported at tactile, as described in herein in regard to Fig. 5 AJ.

Fig. 5 AN-5AR shows the transformation from Fig. 5 A, and display uses 5120 addition of identical continuous touch input and measurement Another measuring section of section 5128 continuous (for example, with common endpoint).In Fig. 5 AN, equipment 100 has been diagonally by user 5004 Upwards and court moves right, while keeping touch input 5120 on measurement addition button 5014, and contact strength connects higher than minimum Touch detection threshold value IT₀And it is lower than light press intensity threshold IT_L.Equipment 100 is moved, so that graticule 5010 is located in desk 5002 top surfaces do not include on the region for any feature that focus 5012 has been aligned.Therefore, focus 5012 is shown in graticule 5010 center, and graticule 5010 is shown with its size increased.Measurement point 5122, measuring section 5128 and measurement point 5132 after Continue and shown on live preview, and dynamic measuring section 5136 be shown in focus 5012 current location and measurement point 5122 it Between.Because live preview does not include any feature extended on the moving direction of equipment 100, virtual guidance is not shown. For example, the movement of equipment 100 is greater than predetermined angular away from horizontal direction, therefore do not shown for horizontal-extending feature in live preview Virtual guidance；It in addition, the movement of equipment 100 is greater than predetermined angular away from vertical direction, therefore is not to be extended vertically in live preview Feature show virtual guidance.

Fig. 5 AO shows the transformation from Fig. 5 AN.In Fig. 5 AO, horizontally court moves right equipment 100 user 5004, Touch input 5120 is kept on measurement addition button 5014 simultaneously, contact strength is higher than minimal-contact detection threshold value IT₀And And it is lower than light press intensity threshold IT_L.Equipment 100 is moved, so that measurement point 5122, measuring section 5128 and measurement point 5132 It is not then displayed in user interface 5006, because of the reality that the physical features corresponding to them are no longer shown in user interface 5006 When preview in.Continue to show dynamic measuring section 5136, extend from the current location of focus 5012.However, because measurement point 5132 (another endpoint of dynamic measuring section 5136) are not then displayed in user interface 5006, so dynamic measuring section 5136 Extend to the edge (for example, towards projected position of measurement point 5132) of user interface 5006.In addition, according to the water of equipment 100 Translation is dynamic, shows virtual guidance 5138, wherein virtually guidance 5138 horizontally extends simultaneously from measurement point 5132 (projected position) And extend along the horizontal leading edge of desk 5002 in user interface 5006.

Fig. 5 AP shows the transformation from Fig. 5 AO.In Fig. 5 AP, user 5004 is by equipment 100 towards right and slightly downwards It is mobile, while touch input 5120 is kept on measurement addition button 5014, contact strength is higher than minimal-contact detection threshold value IT₀And it is lower than light press intensity threshold IT_L.It is that equipment 100 is mobile, so that corresponding to the water of desk 5002 in live preview The virtual guidance 5138 of flat leading edge is at least partly (for example, focus 5012 is in virtual guidance 5138 and in fact in graticule 5010 When preview in desk 5002 horizontal leading edge preset distance in (for example, preset distance be graticule 5010 radius)).Cause This, focus 5012 snaps to virtual guidance 5138 (for example, it is nearest to snap to center along virtual guidance 5138 from graticule 5010 Point).Graticule 5010 is shown with its size reduced, and optionally, generates tactile output 5140 to indicate the alignment behavior.After Continuous display dynamic measuring section 5136, extends to the edge of user interface 5006 (for example, direction from the current location of focus 5012 The projected position of measurement point 5132).In addition, the lasting movement according to equipment 100 in predeterminated level angle, horizontal imaginary guidance 5138 continue the leading edge for the desk 5002 being shown as in user interface 5006.

Fig. 5 AQ shows the transformation from Fig. 5 AP.In Fig. 5 AQ, horizontally court moves right equipment 100 user 5004, Touch input 5120 is kept on measurement addition button 5014 simultaneously, contact strength is higher than minimal-contact detection threshold value IT₀And And it is lower than light press intensity threshold IT_L.Move equipment 100 so that graticule 5010 be located in desk 5002 the preceding upper right corner it On, as shown in live preview.Therefore, focus 5012 snaps to the anchor in live preview corresponding to the preceding upper right corner of desk 5002 Fixed point.Graticule 5010 is shown with its size reduced to indicate alignment behavior.Continue to show dynamic measuring section 5136, from focus 5012 current location extends to the edge (for example, towards projected position of measurement point 5132) of user interface 5006.In addition, In In the sequence of Fig. 5 AN-5AQ, the label for corresponding to measuring section 5136 is updated, with reflection when equipment 100 is mobile by dynamically measuring The length variation for the measurement that section 5136 indicates.In addition, persistently being moved horizontally according to equipment 100, horizontal imaginary guidance 5138 continues It is shown as the leading edge of the desk 5002 in user interface 5006.

Fig. 5 AR-5AS shows the transformation from Fig. 5 AQ, shows to user interface 5006 and adds measurement point.In particular, figure 5AR is shown when touch input 5120 is maintained on measurement addition button 5014, by the way that the contact of touch input 5120 is strong Degree is increased above light press intensity threshold IT_L, as shown in intensitometer 5040, to activate measurement addition button 5014.In response to (virtual) measurement point 5142 is added to use in the current position of focus 5012 by activation measurement addition button 5014, equipment 100 Family interface 5006 is simultaneously shown, and the second endpoint as measuring section 5136.It is added to user interface in conjunction with by measurement point 5142 5006, and increase to light press intensity threshold IT in response to the contact strength of touch input 5120_LMore than, equipment 100 is optional Ground generates tactile output 5144 to indicate the addition of measurement point.In addition, the contact strength with touch input 5120 increases, measurement The size (optionally) for adding button 5014 reduces.Optionally, as shown in Fig. 5 AR, even if after completing measuring section 5136, Equipment 100 also continues to show virtual guide 5138.

Fig. 5 AS shows touch input 5120 and is lifted away from from measurement addition button 5014.In some embodiments, it is touching After (and before detecting the input of any subsequent touch) input 5120 is lifted away from from measurement addition button 5014, equipment 100 The further mobile new dynamic that will not result in display and extend (for example, and continuous with measuring section 5136) from measurement point 5142 Measuring section.That is, being lifted away from for continuous touch input 5120 finishes the continuous of the new measuring section based on touch input 5120 Creation.

In some embodiments, equipment to the responses of Flick gestures (it does not keep contacting with touch sensitive surface) a series of not It is same as the response to a series of pressing gestures (it keeps contacting with touch sensitive surface).For the input as discrete Flick gesture (rather than passing through the pressing gesture for the contact progress being individually consecutively detected), will not be connected using the measurement point each then added Measuring section is created continuously.It, will be the if user continuously puts down four points that is, for a series of serial Flick gestures Measuring section is created between one point and second point, and creates another measuring section between third point and the 4th point, but It is that will not create measuring section between second point and third point.

Fig. 5 AT shows the transformation from Fig. 5 AS.In Fig. 5 AT, equipment 100 is moved, so that graticule 5010 is located in table On the upper left corner of son 5002, as shown in live preview, and on measurement point 5132.Therefore, focus 5012 snaps to Corresponding to the measurement point 5132 at the anchor point of the upper left corner of desk 5002 in live preview.The ruler that graticule 5010 is reduced with it Very little display is to indicate alignment behavior.It is shown again in the live preview of user interface 5006 according to the character pair of desk 5002 Show, measuring section 5128 and measuring section 5136 in user interface 5006 they correspond to desk 5002 individual features (for example, point Wei left edge and horizontal leading edge in the level of desk 5002) corresponding position show.Measuring section 5128 is shown in its end Between point, measurement point 5122 and measurement point 5132.Measuring section 5136 is shown as from (the end of measuring section 5136 of measurement point 5132 Point) to extend to the edge of user interface 5006 (be the another of measuring section 5136 for example, towards the projected position of measurement point 5142 One endpoint, and currently invisible in user interface 5006).In addition, be located on measurement point 5132 according to graticule 5010, Show the virtual guidance 5146,5148 and 5150 extended from measurement point 5132.Virtual guidance 5146 the direction x (for example, along The horizontal leading edge of desk 5002 is horizontally) extend from measurement point 5132.Virtually guidance 5148 is in the direction y (for example, along desk 5002 vertical left front edge is vertically) extend from measurement point 5132.Virtually guidance 5150 is in the direction z (for example, along desk 5002 horizontal left edge is horizontally) extend from measurement point 5132.

Fig. 5 AU-5AX shows the transformation from Fig. 5 AT, shows to user interface 5006 and adds measuring section.Fig. 5 AU is shown Measurement addition button 5014 is activated by touch input 5152 (for example, Flick gesture), which has connects higher than minimum Touch detection threshold value IT₀Contact strength, as shown in intensitometer 5040.Addition button 5014 is measured in response to activation, equipment 100 makes It uses measurement point 5132 as the starting point of new measuring section to be added, and optionally, generates tactile output 5154 to indicate new survey Measure the beginning of section.

Fig. 5 AV shows (after touch input 5152 is lifted away from) user 5004 the hanging down along desk 5002 by equipment 100 Straight left front edge moves down.Focus 5012 snaps to the anchor point of the vertical left front edge along desk 5002, and the anchor point is not It is the preceding lower left corner of desk 5002.Graticule 5010 is shown with its size reduced to indicate alignment behavior.Moreover, being moved according to vertical Dynamic, equipment 100 continues display from the vertically extending virtual guidance 5148 of measurement point 5132, and has stopped showing virtual guidance 5146 and 5150, they do not extend vertically from measurement point 5132 (or with vertical direction predetermined angular in direction).In addition, Dynamic measuring section 5156 is shown between measurement point 5132 and the current location of focus 5012.

Fig. 5 AW, which is shown, activates measurement addition button 5014, the touch by touch input 5158 (for example, Flick gesture) Input, which has, is higher than minimal-contact detection threshold value IT₀Contact strength, as shown in intensitometer 5040.In response to activation measurement addition Button 5014, equipment 100 is added measurement point 5160 to user interface 5006 in the current location of focus 5012 and is shown, and makees For the second endpoint of measuring section 5156, which becomes the measuring section completed.It is added to user circle in conjunction with by measurement point 5160 Face 5006, equipment 100 optionally generate tactile output 5162 to indicate the addition of measurement point (for example, instruction measuring section is complete At).Fig. 5 AX shows touch input 5158 and is lifted away from from measurement addition button 5014.

Fig. 5 AY-5BE is shown to be interacted with the exemplary scaling of the augmented reality environment in user interface 5006.Specifically It says, Fig. 5 AY-5BA shows the repositioning to the scaling auxiliary of the measurement point of display.Fig. 5 AY is shown in measurement point 5160 On the touch input 5164 that detects, contact strength is higher than minimal-contact detection threshold value IT₀, as shown in intensitometer 5040.It rings Ying Yu detects that the touch input 5164 in measurement point 5160, equipment 100 amplify one including measurement point 5160 of live preview Partially (for example, part centered on measurement point 5160 of live preview).The amount of zoom of live preview is based on equipment 100 and surveys The distance between point on amount 5160 corresponding desks 5002 of point is (for example, point on desk 5002 is just before desk 5002 Above the lower left corner).For example, in Fig. 5 AY, as point distance d of the equipment 100 on the corresponding desk 5002 of measurement point 5160₁ When, live preview is amplified with zoom factor 4X.

Fig. 5 AZ shows the transformation from Fig. 5 AY, show touch input 5164 on touch screen 112 movement (for example, by The translation gesture or drag gesture that contact in touch input 5164 carries out) so that touch input 5164 is corresponding to desk On the anchor point in the 5002 preceding lower left corner.Measurement point 5160 with movement of the touch input 5164 on touch screen 112 and It is moved in user interface 5006.In some embodiments, as shown in Fig. 5 AZ, measurement point 5160 snaps to touch input 5164 The anchor point moved above.Therefore, measurement point 5160 is shown in the anchoring in the preceding lower left corner corresponding to desk 5002 At point.In conjunction with the movement of measurement point 5160, measuring section 5156 extends and its label updates accordingly (to indicate desk 5002 The length of vertical left front edge).In addition, equipment 100 optionally generates tactile output 5166 to indicate the alignment behavior.

In some embodiments, equipment 100 determines the measurement point in 5000 from camera position to physical space its it The vector of position on the surface of upper display detected.In some embodiments, equipment 100 determine determined by vector with The angle between surface detected.In some embodiments, according to determining identified vector on the surface detected In predetermined threshold angle (for example, identified angle is less than predetermined threshold angle, such as 15,18,20,25 or 30 degree), when connecing When receiving one group of one or more user input with traverse measurement point, measurement point is mobile by corresponding in user interface 5006 Along the position of the position of the vector of the determination.

Fig. 5 BA shows the transformation from Fig. 5 AZ.In some embodiments, as shown in Fig. 5 BA, in touch input 5164 After being lifted away from, equipment 100 stops the amplification (part) of display live preview and shows the live preview not scaled again.One In a little embodiments, after touch input 5164 is lifted away from, equipment 100 continues the live preview of display amplification, after detecting Until continuous input (to exit the live preview of scaling and return to the live preview of non-Zoom display).

Because equipment 100 is maintained at same position in Fig. 5 AX-5AZ, the graticule 5010 in Fig. 5 BA with Fig. 5 AX In identical size and identical position show (before the repositioning of the scaling auxiliary to measurement point 5160).Because as closed Measurement point 5160 is repositioned onto described in Fig. 5 AY-5AZ the anchoring in the preceding lower left corner corresponding to desk 5002 like that Point, so showing measurement point at the anchor point in Fig. 5 BA in the live preview of non-Zoom display, except graticule 5010 5160.Also show the measuring section 5156 and its corresponding update label of extension.

Fig. 5 BB-5BC shows another exemplary scaling interaction.Fig. 5 BB is similar to Fig. 5 AX, the difference is that in Fig. 5 BB Equipment 100 is than (as shown in the side view of user 5004, equipment 100 and desk 5002 in Fig. 5 AY) in Fig. 5 AX closer to table Sub 5002 (or more specifically, closer to points on the corresponding desk 5002 of measurement point 5160), distance d₂Less than in Fig. 5 AY Distance d₁.Accordingly, in response to the touch input 5168 detected in measurement point 5160, as shown in Fig. 5 BC, equipment 100 is amplified A part including measurement point 5160 of live preview.The amount of zoom of live preview in Fig. 5 BC is based on equipment 100 and desk Small distance d between 5002₂, therefore the amount of zoom (corresponding to zoom factor 2X) of the live preview in Fig. 5 BC is less than Fig. 5 AY In amount of zoom (correspond to zoom factor 4X).In addition, because equipment 100 in Fig. 5 BB than in Fig. 5 AX closer to desk 5002, so scale mark is shown with one foot of interval (for example, with display scale no in Fig. 5 AX along measuring section 5156 Label is opposite).In addition, because equipment 100 in Fig. 5 BB than in Fig. 5 AX closer to desk 5002, it is right in Fig. 5 BB The size of the label of measurement shown by Ying Yu is greater than the size in Fig. 5 AX.

Fig. 5 BD-5BE shows another exemplary scaling interaction.Fig. 5 BD is similar to Fig. 5 BB, the difference is that in Fig. 5 BD Equipment 100 than in Fig. 5 BB closer to desk 5002 (or more specifically, closer to the corresponding desk 5002 of measurement point 5160 On point), distance d₃Less than the distance d in Fig. 5 BB₂.Accordingly, in response to the touch input detected in measurement point 5160 5170, as shown in Fig. 5 BE, equipment 100 amplifies a part including measurement point 5160 of live preview.It is real-time pre- in Fig. 5 BE The amount of zoom look at is based on the small distance d between equipment 100 and desk 5002₃, therefore the scaling of the live preview in Fig. 5 BE Amount (corresponding to zoom factor 1.5X) is less than the amount of zoom (corresponding to zoom factor 2X) in Fig. 5 BC.In addition, because equipment 100 Than in Fig. 5 BB closer to desk 5002 in Fig. 5 BD, so scale mark is along measuring section 5156 with one inch of interval Display (for example, with scale mark in Fig. 5 BB with one foot of interval shown opposite).In addition, because equipment 100 is in Fig. 5 BD Than, closer to desk 5002, scheming so the size in Fig. 5 BD corresponding to the label of shown measurement is greater than in Fig. 5 BB Size in 5BB.In some embodiments, subtract with equipment 100 with the distance between the physical features for corresponding to measuring section It is small, become finer (for example, the distance more than first distance, does not show scale along the scale of measuring section display label Label；In the distance between first distance and second distance (shorter than first distance), scale mark is shown with one foot of interval； In the distance between second distance and third distance (shorter than second distance), scale mark is shown with one inch of interval；Short In the distance of third distance, scale mark is shown with the interval of quarter inch, and so on).

In some embodiments, the amount of zoom shown in Fig. 5 AY is maximum zoom amount, so that working as equipment 100 and desk The distance between 5002 (or points on the corresponding desk 5002 of measurement point of display) are greater than distance d shown in Fig. 5 AY₁When, it is real When preview amount of zoom still correspond to zoom factor 4X.In some embodiments, the amount of zoom shown in Fig. 5 BE is most Small amount of zoom, so that when the distance between equipment 100 and desk 5002 are (or on the corresponding desk 5002 of measurement point of display Point) it is less than distance d shown in Fig. 5 BE₃When, the amount of zoom of live preview still corresponds to zoom factor 1.5X.

Similarly, in some embodiments, the size of the label shown in Fig. 5 AY is minimum label sizes, so that working as The distance between equipment 100 and desk 5002 (or point on the corresponding desk 5002 of measurement point of display) are greater than shown in Fig. 5 AY Distance d₁When, the size of label is identical as in Fig. 5 AY.In some embodiments, the size of the label shown in Fig. 5 BE It is maximum label sizes, so that when the distance between equipment 100 and desk 5002 (or corresponding desk 5002 of measurement point of display On point) be less than Fig. 5 BE shown in distance d₃When, the size of label is identical as in Fig. 5 BE.

Fig. 5 BF-5BK shows the image that augmented reality environment is captured in user interface 5006.Fig. 5 BF optionally shows The transformation from Fig. 5 BA is gone out.In Fig. 5 BF, positioning device 100 of user 5004, so that measuring section 5128,5136 and 5156 And its corresponding endpoint and label are shown in user interface 5006, the character pair of desk 5002 is visible in live preview. Fig. 5 BG, which is shown through touch input 5172, activates media capture button 5016, and the intensity as shown in intensity map 5180 increases Shown in adding, which shows the contact strength of touch input 5172 at any time.

Fig. 5 BH is shown based on touch input 5172 in predetermined threshold time T_thIt is lifted away from before (for example, touch input 5172 It is Flick gesture), change from Fig. 5 BG.Therefore, intensity map 5180 shows the contact strength of touch input 5172 in time T_thIt It is preceding to be accordingly reduced to zero.In response in predetermined threshold time T_thBeing lifted away from for touch input 5172 is detected before, and equipment 100 is caught Obtain the image 5174 of augmented reality environment.Captured image 5174 is static image comprising the image of camera fields of view corresponds to The instantaneous snapshot of live preview, and measuring section 5128,5136 and 5156 and their corresponding endpoints and label are regarded in camera It is superimposed on wild image.In some embodiments, as shown in Fig. 5 BH, captured image 5174 does not include user interface 5006 In button/control image.

Fig. 5 BI is shown when equipment 100 is in different location (the left side table towards desk 5002 relative to desk 5002 Face) so that the different perspective views of desk 5002 are shown in live preview, captured in response to activation media capture button 5016 Image.In response to activating media capture button 5016 and touch input in predetermined threshold time T by touch input_thIt lifts before From as shown in intensity map 5180, equipment 100 captures the image 5176 of augmented reality environment.Captured image 5176 is static figure Picture comprising from equipment 100 at the position in Fig. 5 BI (for example, left-hand face towards desk 5002) desk 5002 Image.The visual angle quilt of measuring section 5128,5136 and 5156 and its corresponding endpoint and label based on the equipment 100 in Fig. 5 BI It is superimposed upon in captured image 5176 on the character pair of desk 5002.

Fig. 5 BJ shows the alternative transformation from Fig. 5 BG.Together with Fig. 5 BG, Fig. 5 BJ-5BK is shown in response to being maintained at Touch input (for example, long pressing gesture) on media capture button 5016 captures the video of augmented reality environment.Scheming In 5BJ, touch input 5172 remains above predetermined threshold time T on media capture button 5016_th, such as 5180 institute of intensity map Show.Therefore, equipment 100 captures the video of camera fields of view.Timer 5178 is shown in user interface 5006 and indicates capture The current length of video.In addition, the video of capture includes any measurement in camera fields of view (for example, measuring section 5128,5136 and 5156 and their corresponding endpoints and label be superimposed upon in the video of capture on the character pair of desk 5002).In some realities It applies in scheme, the video of capture does not include button/control image in user interface 5006.

Fig. 5 BK shows the equipment 100 when touch input 5172 is maintained on media capture button 5016 and has moved. Therefore, when equipment 100 is mobile, equipment 100 continues to capture the video of camera fields of view, as shown in the timer 5178 in Fig. 5 BK Current video length shown in, be longer than video length shown in the timer 5178 in Fig. 5 BJ.The video of capture includes working as to set Standby 100 are superimposed upon in camera fields of view the additional of the measuring section 5136 on character pair when being moved to its position as shown in Fig. 5 BK Part.

Fig. 5 BL-5BM shows display about the additional information of selected measurement and for answering the information sharing to another With the option of program, process or equipment.Fig. 5 BL shows the touch input 5182 detected on measuring section 5156 (for example, logical The Flick gesture that the contact crossed in touch input carries out), contact strength is higher than minimal-contact detection threshold value IT₀, such as intensitometer Shown in 5040.Fig. 5 BM is shown in response to detecting that the touch input 5182 on measuring section 5156, equipment 100 show measurement pipe Manage interface 5184.Measuring management interface 5184 includes the label of physical object, that is, desk 5002 corresponding to description measurement 5156. For example, measuring management interface 5184 includes the label 5186-a that classifies to desk 5002 (for example, desk 5002 is identified as " desk ").Measuring management interface 5184 further includes the label that the relationship between measurement 5156 and desk 5002 is classified 5186-b (for example, " height " that measurement 5156 is identified as to desk 5002).In some embodiments, based on equipment 100 It vertically moves and is classified as the relationship between measurement 5156 and desk 5002 " height ", while measurement 5156 is added to user circle Face 5006.

In some embodiments, in response to touch input 5182, the information about measurement 5156 is (for example, measurement 5156 Relationship, the amplitude of measurement 5156 between the classification and measurement 5156 and physical object of the corresponding physical object is such as long Degree or area, the image for measuring 5156 etc.) it is copied to the clipbook process executed on the appliance 100.In some embodiments In, measuring management interface includes that can be sent to it multiple destinations about the information of selected measurement 5156 (for example, right Should the icon 5192 (Figure 1A) in email client module 140, the icon 5194 (Figure 1A) corresponding to IM module 141, and Icon 5196 corresponding to the File Transfer Protocol between electronic equipment).In some embodiments, in response to touch input 5182 meet higher than minimal-contact detection threshold value IT₀Intensity threshold (for example, light press intensity threshold IT_L) (for example, in response to Touch input 5182 is light press or deep pressing gesture), show measuring management interface 5184.

Fig. 5 BN shows exemplary control centre's user interface including augmented reality measurement application icon 5190 5188.The activation starting augmented reality measurement application program of augmented reality measurement application icon 5190 simultaneously shows user interface 5006 (for example, with reference to as described in Fig. 5 A).

Fig. 5 BO is shown using the background about Fig. 5 BO-5C user interface described.Fig. 5 BO is similar to Fig. 5 A, wherein Fig. 5 B shows the view of the physical space 5000 including equipment 100, the difference is that physical space 5000 includes being placed with thereon The desk 5200 (rather than desk 5002) of object 5202.Object 5202 is in the visual field of the camera of equipment 100, and It is visible in the live preview of the physical space 5000 shown in user interface 5006 in equipment 100.In some embodiments, As shown in Fig. 5 BO, the graticule 5010 in user interface 5006 is tilted to seem coplanar with the top surface of desk 5200, to refer to Show the surface of current location that is having been detected by and corresponding to focus 5012.

Fig. 5 BP-5BQ shows that virtual measurement point is added to user interface 5006 according to some embodiments One mode.Fig. 5 BP shows the touch input 5204 on graticule 5010.According to some embodiments, Fig. 5 BP is shown, in response to Virtual measurement point 5206 is added to user interface 5006 in the current position of focus 5012 by touch input 5204, equipment 100 And it shows (as shown in Fig. 5 BO).Fig. 5 BQ is shown to be shown at position identical with Fig. 5 BP after touch input 5204 is lifted away from The measurement point 5206 shown.

Fig. 5 BR-5BS, which is shown, is added to the another of user interface 5006 for virtual measurement point according to some embodiments Select mode.Fig. 5 BR shows the touch input 5204 on graticule 5010.Compared with Fig. 5 BP, Fig. 5 BR is shown, in response to touching Input 5204, equipment 100 are abandoned adding virtual measurement point at the position of focus 5012 and be shown.On the contrary, in Fig. 5 BR, if Standby 100 idsplay order message 5208 with prompt user 5004 on measurement addition button 5014 tap (rather than in graticule 5010 Upper tap) Lai Tianjia measurement point.Fig. 5 BS, which is shown through touch input 5210, activates measurement addition button 5014.In response to Virtual measurement point 5206 is added to user interface 5006 in the current position of focus 5012 by touch input 5210, equipment 100 (as shown in Fig. 5 BQ).

Fig. 5 BT-5BU shows after adding measurement point 5206 (for example, in Fig. 5 BQ or Fig. 5 BS) corresponding to object The creation of 5202 measurement.In Fig. 5 BT, user 5004 moves equipment 100, so that graticule 5010 and focus 5012 are located in On different location in physical space 5000.In particular, graticule 5010 and focus 5012 are located in object in Fig. 5 BT (as shown in live preview) on 5202 edge, more than the edge where graticule 5010 in Fig. 5 BO and focus 5012 Close to equipment 100.Therefore, graticule 5010 is shown in Fig. 5 BT relative to its size in Fig. 5 BO with the size increased, and And focus 5012 is shown in Fig. 5 BT relative to its size in Fig. 5 BO with the size increased.In some embodiments, Between the position that the size that graticule 5010 is shown is shown above based on graticule 5010 in equipment 100 and physical space 5000 Distance optionally follows predefined minimum dimension (for being greater than the distance of predefined maximum distance) and predefined full-size (for being less than the distance of predefined minimum range).Similarly, in some embodiments, the size that focus 5012 is shown is based on The distance between the position that focus 5012 is shown above in equipment 100 and physical space 5000, optionally follows predefined Minimum dimension and predefined full-size.In addition, according to the movement of equipment 100 so that graticule 5010 and focus 5012 are located in object It manages on the different location in space 5000, is shown between measurement point 5206 (measurement point added recently) and focus 5012 (dynamic) measuring section 5212, is indicated by dotted line.Measuring section 5212 shows associated (dynamic) label, label instruction Some distance in physical space 5000, measuring section 5212 seem to extend in user interface 5006 over this distance.

Fig. 5 BU, which is shown through touch input 5214 (for example, Flick gesture), activates measurement addition button 5014.Response Addition button 5214 is measured in activation, the current position of focus 5012 in user interface 5006 of equipment 100 is added and shown Measurement point 5216 (as shown in Fig. 5 BT) and the second endpoint as measuring section 5212.According to the completion of measuring section 5212, measurement The outward appearance change of section 5212.In the example shown in Fig. 5 BU, measuring section 5212 becomes solid line from dotted line.Because of measurement point 5216 Be located in the edge of the object 5202 than showing measurement point 5206 on it of object 5202 closer to the edge of equipment 100 it On, so measurement point 5216 is shown relative to measurement point 5206 with the size of increase.

Fig. 5 BV-5BY shows the creation of the measurement corresponding to object 5202 according to some embodiments, leads to elder generation Preceding measurement is removed.Fig. 5 BV shows the transformation from Fig. 5 BU.In Fig. 5 BT, user 5004 moves equipment 100, so that Graticule 5010 and focus 5012 are located on position different from shown in Fig. 5 BU in physical space 5000.In particular, In In Fig. 5 BV, graticule 5010 and focus 5012 are located on the first turning of object 5202.In some embodiments, as schemed Shown in 5BV, mobile far from measuring section 5212 according to graticule 5010 and focus 5012, equipment 100 stops display and measuring section 5212 Associated label (" 17 inches ").

Fig. 5 BW shows the transformation from Fig. 5 BV, shows to user interface 5006 and adds measurement point.In particular, Fig. 5 BW It shows through touch input 5218 and activates measurement addition button 5014.In response, equipment 100 is in the current of focus 5012 Measurement point 5220 is added to user interface 5006 at position (as shown in Fig. 5 BV).In addition, according to addition far from being previously created The new measurement point of measuring section 5212, equipment 100 change the appearance of measuring section 5212 (for example, with instruction creation and measuring section 5212 The measuring section of disconnection will lead to measuring section 5212 and be removed from user interface 5006).In the example shown in Fig. 5 BW, measurement Section 5212 becomes dotted line from solid line, and the color (and/or transparency) of measuring section 5212 and its endpoint changes.

Fig. 5 BX shows the transformation from Fig. 5 BW, and display user 5004 moves equipment 100, so that graticule 5010 and focus 5012 are located on the second turning of object 5202.Therefore, measurement is shown in by (dynamic) measuring section 5222 that dotted line indicates Between 5206 (measurement points added recently) of point and focus 5012.Measuring section 5222 shows associated (dynamic) label (example Such as, " 17 inches "), which indicates some distance (for example, along object 5202) in physical space 5000, measuring section 5222 seem to extend in user interface 5006 over this distance.

Fig. 5 BY shows the transformation from Fig. 5 BX, display activate measurement addition button 5014 in response to touch input 5226 and Measurement point 5224 is added to user interface 5006.Fig. 5 BY shows measurement point 5224 in the current location of focus 5012 Second endpoint of the place's addition (as shown in figure BX) as measuring section 5222.Because measurement point 5224 is located in the ratio of object 5202 On showing the position of the object 5202 of measurement point 5220 from the farther position of equipment 100 on it, so 5224 phase of measurement point Measurement point 5220 is shown with reduced size.According to the completion of measuring section 5222, the appearance of measuring section 5222 becomes from dotted line For solid line.In addition, according to the completion of measuring section 5222, and it is disconnected according to measuring section 5222 and previously placed measuring section 5212 It opens, equipment 100 stops display measuring section 5212.In some embodiments, according to determining measuring section 5222 from measuring section 5212 At least predetermined threshold distance is (for example, any point on measuring section 5212 not predetermined thresholds of any point on measuring section 5222 In value distance), stop display measuring section 5212.

Fig. 5 BZ-5CF shows the creation of the measurement corresponding to object 5202 according to some embodiments, which connects It is connected to previous measurement, so that previous measurement continues to show.Fig. 5 BZ shows the transformation from Fig. 5 BY, and display user 5004 moves Equipment 100 is moved, so that graticule 5010 and focus 5012 are located on the third angle of object 5202.In some embodiments In, as shown in Fig. 5 BZ, even if graticule 5010 and focus 5012 have moved away from the movement of measuring section 5222, equipment 100 also continues to show Label associated with measuring section 5222 is (with Fig. 5 BV on the contrary, the figure shows when graticule 5010 and the separate measurement of focus 5012 Equipment 100 stops the embodiment for showing label associated with measuring section 5212 when Duan Yidong).

Fig. 5 CA shows the transformation from Fig. 5 BZ, and display activates measurement addition button 5014 in response to touch input 5230, Measurement point 5228 is added to user interface 5006 in the current position (as shown in Fig. 5 BZ) of focus 5012.According in user Measurement point 5228 is added at position far from the measuring section 5222 being previously created in interface 5006, equipment 100 changes measuring section 5222 appearance with the measuring section for indicating that creation is disconnected with measuring section 5222 (for example, will make measuring section 5222 from user interface It is removed in 5006).In the example shown in Fig. 5 CA, measuring section 5222 becomes dotted line, and the color of measuring section 5222 from solid line (and/or transparency) changes.

Fig. 5 CB shows the transformation from Fig. 5 BZ, and display user 5004 moves equipment 100, so that graticule 5010 and focus 5012 are remotely located from the third angle of object 5202.Therefore, measurement is shown in by (dynamic) measuring section 5232 that dotted line indicates Between point 5228 and focus 5012.Measuring section 5232 shows associated (dynamic) label, which indicates along object 5202 some distance (for example, " 8 inches "), measuring section 5222 seems to extend in user interface 5006 over this distance.

Fig. 5 CC shows the transformation from Fig. 5 CB, and display user 5004 moves equipment 100, so that in measuring section 5222 Point is in graticule 5010.In some embodiments, as shown in Fig. 5 CC, the corresponding measuring section that is shown in user interface 5006 Midpoint can be the anchor point that focus 5012 is aligned.Therefore, in Fig. 5 CC, focus 5012 is snapped to corresponding to measuring section The anchor point at 5222 midpoints.In order to indicate the alignment behavior, focus 5012 is to arrive anchor point relative to when focus 5012 is unjustified When focus 5012 size increase size show (for example, as shown in Fig. 5 CB).In some embodiments, such as Fig. 5 CC institute Show, when focus snaps to anchor point, the size of graticule 5010 will not change.In some embodiments, anchor point is snapped to When focus 5012 size be greater than focus 5012 predefined full-size, the predefined full-size be used for be based on equipment 100 The distance between focus 5012 is shown above in physical space 5000 position changes the size (example of focus 5012 Such as, as described in herein in regard to Fig. 5 BT).In addition, because focus 5012 snaps to the point along measuring section 5222, measuring section 5222 outward appearance change is to indicate that will lead to measuring section 5222 in the current position of focus 5012 addition measurement point continues to show Rather than it is removed.In particular, measuring section 5222 becomes solid line from dotted line (again), and measuring section 5222 color (and/ Or transparency) change, so that measuring section 5222 is shown again with its appearance as shown in Fig. 5 BZ (before adding measurement point 5228) Show.In addition, the length of dynamic measuring section 5232 is updated according to the movement of equipment 100, so that the continuation of measuring section 5232 is measuring It is shown between point 5228 and the current location of focus 5012.Tag update associated with measuring section 5232 is to reflect measuring section 5232 length changes (for example, " 12 inches ").

Fig. 5 CD shows the transformation from Fig. 5 CC, and display user 5004 moves equipment 100, so that graticule 5010 and focus 5012 are remotely located from the midpoint of measuring section 5222.Therefore, in Fig. 5 CD, dynamic measuring section 5232 and its respective labels are more Newly with reflect due to focus 5012 relative to measurement point 5228 movement caused by measuring section 5232 length variation.In addition, The appearance of measuring section 5222, which is updated to indicate that, will lead to measuring section 5222 in the current position of focus 5012 addition measurement point It is removed.

Fig. 5 CE shows the transformation from Fig. 5 CD, and display user 5004 moves equipment 100, so that measurement point 5224 is (such as Shown in Fig. 5 CD) in graticule 5010.In some embodiments, as shown in Fig. 5 CE, the endpoint of the measuring section previously added can To be anchor point that focus 5012 is aligned.Therefore, in Fig. 5 CE, focus 5012 snaps to the anchor corresponding to measurement point 5224 It pinpoints (as shown in Fig. 5 CD).In order to indicate the alignment behavior, focus 5012 with relative to when its it is unjustified to anchor point when The size that size increases shows (for example, as shown in Fig. 5 CD), although the size of graticule 5010 has not been changed, such as herein with reference to Fig. 5 CC It is described.In addition, the outward appearance change of measuring section 5222 is because focus 5012 snaps to the point along measuring section 5222 to indicate Focus 5012 current position addition measurement point will lead to measuring section 5222 continue to illustrate rather than it is removed.Specifically It says, measuring section 5222 becomes solid line from dotted line, and the color (and/or transparency) of measuring section 5222 changes, so that measuring section 5222 are shown again with its appearance as shown in Fig. 5 BZ.In addition, dynamic measuring section 5232 and its respective labels are updated to reflect that Due to focus 5012 relative to measurement point 5228 movement caused by measuring section 5232 length variation.

Fig. 5 CF shows the transformation from Fig. 5 CE, and display activates measurement addition button 5014 in response to touch input 5236, Measurement point 5234 is added to user interface 5006 in the current position (as shown in Fig. 5 CE) of focus 5012.According to along survey It measures and adds measurement point 5234 at the point of section 5222, measuring section 5222 continues to show.According to the completion of measuring section 5232, measuring section 5232 appearance becomes solid line from dotted line.

Fig. 5 CG-5CK is shown according to the close enough (for example, interior in predetermined threshold distance but do not connect of some embodiments It is connected to) creation of measurement that had previously measured, so that previously measurement continued to show.Fig. 5 CG shows the transformation from Fig. 5 CF, display User 5004 moves equipment 100, so that graticule 5010 and focus 5012 are remotely located from object 5202 and in desks On 5200 the first turning.In some embodiments, as shown in Fig. 5 CG, even if graticule 5010 and focus 5012 are far Mobile from measuring section 5222 and measuring section 5232, equipment 100 also continues display label associated with measuring section 5222 and 5232 (with Fig. 5 BV on the contrary, the figure shows when graticule 5010 and focus 5012 are mobile far from measuring section equipment 100 stop display with The embodiment of the associated label of measuring section 5212).

Fig. 5 CH shows the transformation from Fig. 5 CG, and display activates measurement addition button 5014 in response to touch input 5240, Measurement point 5238 is added to user interface 5006 in the current position (as shown in Fig. 5 CH) of focus 5012.According in user Measurement point 5238 is added at position far from the measuring section 5222 and 5232 being previously created in interface 5006, equipment 100, which changes, to be surveyed Measure section 5222 and 5232 appearance, with indicate creation with measuring section 5222 disconnect and away from along the measuring section any point and Any point along measuring section 5232 is more than that the measuring section of threshold distance will make measuring section 5222 and 5232 from user interface 5006 It removes.In the example shown in Fig. 5 CH, measuring section 5222 and 5232 becomes dotted line, and measuring section 5222 and 5232 from solid line And its color (and/or transparency) of endpoint changes.

Fig. 5 CI shows the transformation from Fig. 5 CH, and display user 5004 moves equipment 100, so that graticule 5010 and focus 5012 are located on the side surface of desk 5200.In some embodiments, as shown in Fig. 5 CI, graticule 5010 is tilted to see Get up coplanar with the side surface of desk 5200, to indicate table have been detected by and the current location corresponding to focus 5012 Face.It is shown between measurement point 5238 and focus 5012 by the dynamic measuring section 5242 that dotted line indicates together with associated dynamic label, The label indicates the distance of the side surface along desk 5200, and measuring section 5242 seems in the user interface over this distance Extend 5006 (for example, " 2 feet 10 inches ").

Fig. 5 CJ shows the transformation from Fig. 5 CI, and display user 5004 moves equipment 100, so that graticule 5010 and focus 5012 are located on the second turning of desk 5200.In some embodiments, as shown in Fig. 5 CJ, even if in focus 5012 Current position addition measurement point will lead to measuring section 5222 and 5232 and continue to illustrate rather than removed, 5222 He of measuring section 5232 appearance will not change (for example, because the institute generated and adding measurement point in the current position of focus 5012 Obtaining measuring section will be in the predetermined threshold distance of two measuring sections 5222 and 5232).

Fig. 5 CK shows the transformation from Fig. 5 CJ, and display activates measurement addition button 5014 in response to touch input 5246, Measurement point 5244 will be added in the current position (as shown in Fig. 5 CJ) of focus 5012.At the position in user interface 5006 Measurement point 5244 is added, so that measuring section 5242 is in the predetermined threshold distance of measuring section 5222 (for example, measuring section 5242 includes At least one point in the predetermined threshold distance of at least one point along measuring section 5222).Therefore, in addition measurement point Continue to show measuring section 5222 after 5244.In addition, measuring section 5222 becomes solid line, and measuring section 5222 from dotted line (again) Color (and/or transparency) change so that measuring section 5222 is shown again with its appearance as shown in Fig. 5 CG.Similarly, it surveys Amount section 5242 measuring section 5232 predetermined threshold distance in (for example, measuring section 5242 include along measuring section 5222 at least At least one point in the predetermined threshold distance of one point, such as endpoint measurement point 5234).Therefore, as measuring section 5222, Measuring section 5232 continues to show and show again with its appearance as shown in Fig. 5 CG.In some embodiments, as long as any work as At least one of measuring section (for example, the measuring section 5222 and 5232 previously connected) of preceding display point is in newly created measuring section In the predefined distance of (for example, measuring section 5242), then all measuring sections currently shown are still shown.

Fig. 5 CL-5CM shows the exemplary prompt condition in augmented reality environment.Fig. 5 CL, which is shown from Fig. 5 CK, to be turned Become, display user 5004 moves equipment 100, so that the visual field of camera no longer includes the display above of physical space 5000 The part (for example, left part of desk 5200, as shown in Fig. 5 CK) of measuring section 5222,5232 and 5242.Fig. 5 CL instruction Since the visual field of camera removes the time quantum having been subjected to from 5200 left part of desk less than the first predefined thresholds time quantum T_th。

Fig. 5 CM shows the transformation from Fig. 5 CL, shows since the visual field of camera is removed from 5200 left part of desk The time quantum crossed has reached the first predefined thresholds time quantum.Therefore, equipment 100 display reminding message 5248 indicates to survey Amount section 5222,5232 and 5242 will be removed from user interface 5006 quickly.In some embodiments, if in display reminding Equipment 100 is moved back to its position as shown in Fig. 5 CK in the second predefined thresholds time after message 5248, then it will be in user It shows measuring section 5222,5232 and 5242 in interface 5006 again on the character pair in physical space 5000, such as schemes Shown in 5CG.In some embodiments, if in the second predefined thresholds time since prompting message 5248 is shown Equipment 100 is moved back to its position as shown in Fig. 5 CK after amount, then will not show again measuring section in user interface 5006 5222,5232 and 5242.In some embodiments, when equipment 100 is mobile, so that physical space 5000 is shown above Show that the distance of the current part in camera fields of view of the partial distance physical space 5000 of measuring section 5222,5232 and 5242 is big When threshold distance, display reminding message 5248.

Fig. 5 CN-5CO shows the another exemplary implied condition in augmented reality environment.Fig. 5 CN, which is shown, is located at distance Equipment 100 at 5200 first distance of desk, wherein first distance is less than predefined (maximum) threshold distance D_th.Show graticule 5010 and focus 5012, indicating equipment 100 detect surface at the position that focus 5012 is shown above.Figure 5CO shows the equipment 100 being located at 5200 second distance of desk, and wherein second distance is greater than predefined (maximum) threshold value Distance D_th.Therefore, equipment 100 does not show graticule 5010 and focus point 5012, this indicating equipment 100 is not in physical space 5000 Middle focus 5012 will detect surface at the position shown above.In addition, 100 display reminding message 5250 of equipment is to indicate Equipment 100 is too far from the position that focus 5012 in physical space 5000 has been shown above, and prompts user 5004 that will set Standby 100 are moved into closer to the position.Similarly, in some embodiments, when equipment 100 is located in apart from desk 5200 Less than predefined minimum threshold distance apart from when, equipment 100 does not show that graticule 5010 and focus 5012 (are not yet examined with instruction Measure surface), and display reminding message (for example, using the text such as " shifted to farther " etc) prompts user 5004 will Equipment 100 moves further away from the position that focus 5012 will be shown above in physical space 5000.

Fig. 6 A-6C be show according to some embodiments with for carrying out physical space measurement using augmented reality environment Application program interaction method 600 flow chart.Method 600 is including touch-sensitive display (for example, touch screen 112 (Figure 1A) Or combine the display generating unit 304 (Fig. 3 B) of input equipment 302), and one or more cameras are (for example, optical sensor 164 (Figure 1A) or camera 305 (Fig. 3 B)), optionally in the one or more for the intensity for detecting contact with touch-sensitive display Sensor (for example, contact strength sensor 165, Figure 1A), and optional one or more tactile output generators (for example, Tactile output generator 163 (Figure 1A) or tactile output generator 357 (Fig. 3 A)) electronic equipment (for example, portable multi-function Equipment 100 (Figure 1A), equipment 300 (Fig. 3 A) or computer system 301 (Fig. 3 B)) at execute.Some operations in method 600 It is optionally combined and/or the sequence of some operations is optionally changed.

As described below, method 600 is provided relocates the intuitive of virtual measurement point in the measurement based on augmented reality Mode.Allow to more easily, more accurately in response to being directed toward the input of measurement point to amplify on the region for including the measurement point Ground relocates the measurement point.Method 600 reduces quantity, the journey of the input from the user when relocating virtual measurement point Degree and/or property, to create more effective man-machine interface.For battery-driven electronic equipment, allow users to more Quickly and more efficiently relocate measurement point save power and increase battery charge twice between time interval.

Electronic equipment shows (602) application program (for example, augmented reality measurement application program or packet on the touch sensitive display Include the application program of augmented reality measurement function) user interface (for example, user interface 5006, Fig. 5 AX).

The user interface include (604) it is one or more it is magazine at least one the visual field expression (for example, user circle Face 5006 includes the live preview in the visual field of the camera of equipment 100, Fig. 5 AX).The expression in the visual field shows with the first magnifying power, and And (example is updated based on the variation by least one the one or more magazine Current vision data detected at any time Such as, the expression in the visual field is from least one the one or more magazine RUNTIME VIEW).In addition, the visual field includes three-dimensional At least part (for example, including the space of physical object in physical world) in space.For example, it is no scaling (or use Zoom factor 1X) in the case where show live preview, and update with the movement of equipment 100 live preview (for example, as schemed Shown in 5AU-5AV).

While showing that the visual field indicates, electronic equipment detects the first touch input (example on (606) touch-sensitive display Such as, showing the Flick gesture on capable of indicating or pressing input, when this, which shows, to indicate to be activated, measurement point is being added to shown The visual field indicate) (for example, touch input 5158, Fig. 5 AW).

In response to detecting the first touch input, electronic equipment adds (608) simultaneously at the first position in the expression of the visual field It shows (virtual) measurement point (for example, measurement point 5160, Fig. 5 AW), which corresponds to the first position in three-dimensional space.

After adding measurement point and while continuing to show that the visual field indicates (610): when one or more magazine When at least one movement, electronic equipment corresponds to display at the position of the first position in three-dimensional space in the expression of the visual field (612) measurement point.For example, when it is one or more it is magazine at least one position and/or orientation due to electronic equipment When moving and changing, which continues to be shown in the first position corresponded in three-dimensional space in RUNTIME VIEW At position, (virtual) measurement point is initially placed into here.In some embodiments, when one or more is magazine at least One it is mobile when, shown measurement point seem to adhere to or be anchored in three-dimensional space should (virtual) measurement point be originally placed Position.

In addition measurement point and while continuing to show that the visual field indicates (610), electronic equipment is on the touch sensitive display Some position at detect (614) second touch inputs (for example, touch input 5164, Fig. 5 AY), the position correspond to the visual field (it corresponds to the first position in three-dimensional space again, and measurement point is initially placed into this for the current location of measurement point in expression In).

In response to detecting the second touch input, at least part of display that electronic equipment indicates the visual field is put from first To the second magnifying power greater than the first magnifying power, (for example, in Fig. 5 AY, live preview is to scale for the display amplification (616) of big rate Factor 4X amplification).The amplification for the part that the visual field indicates is shown including measurement point.In some embodiments, in response to measurement Gesture (such as tap, double-click, press, press and keep or kneading gesture) on point, electronic equipment amplification include measurement point Region, so that the region including measurement point is amplified to the second magnifying power from the first magnifying power.In some embodiments, amplify Region including measurement point allows users to more accurately relocate measurement point, for example, in response to the measurement for being directed toward amplification The gesture of point.

In some embodiments, one or more cameras (618) are located on electronic equipment, one with touch-sensitive display Split-phase is adjacent, and the side where one or more of cameras of the part in equipment is (for example, one or more phases seat in the plane In region 5008, Fig. 5 A).In some embodiments, one or more cameras are located on electronic equipment, with touch-sensitive display First part it is adjacent.In some embodiments, the surface of touch-sensitive display extends along x/y plane, and one or more phases Machine is adjacent with the first part of touch-sensitive display along x/y plane.In some embodiments, one or more cameras are vertical In adjacent with the first part of touch-sensitive display on the direction z of x/y plane.In some embodiments, which includes aobvious Show that first in the part of the side where one or more of cameras in the separate equipment of touch-sensitive display shows energy table Show (for example, measurement addition button 5014, Fig. 5 A), and when first show can indicate to be activated when, to shown visual field expression Add measurement point.In some embodiments, first shows and can indicate to be shown in the second part of touch-sensitive display, wherein touch-sensitive The second part of display and the first part of touch-sensitive display are different, and wherein the second part of touch-sensitive display is positioned to Far from the side where one or more of cameras in equipment.In some embodiments, which further includes root (example can be indicated by showing according to the one or more second of first orientation (for example, machine-direction oriented, as shown in Figure 5A) display of electronic equipment Such as, button 5018,5020,5022, Fig. 5 A).In some embodiments, electronic equipment detects electronic equipment and moves to second orientation Dynamic (for example, rotation) (for example, horizontal orientation being rotated to, as shown in Fig. 5 B or Fig. 5 C).In some embodiments, in response to Detect that electronic equipment is mobile to second orientation, electronic equipment updates one or more second according to the second orientation of electronic equipment Show the display that can be indicated (for example, button 5018,5020 and 5022 is moved to the different zones of user interface in Fig. 5 B or Fig. 5 C Position without considering one or more cameras), and continue one or more in the separate equipment of touch-sensitive display Display first, which is shown, in the part of side where a camera to indicate (for example, in Fig. 5 B and 5C, measurement addition 5014 quilt of button It is shown apart from region 5008).

In some embodiments, in second orientation, electronic equipment updates first and shows the display that can be indicated, so that first In the part for showing the side where the one or more of cameras that can indicate to be displayed in the separate equipment of touch-sensitive display Different location (for example, the position in the edge or turning preset distance away from touch-sensitive display, in order in electronic equipment Activation first is shown and can be indicated during one-handed performance) (for example, measurement addition button 5014).In some embodiments, first shows It can indicate in the part of the side where the one or more of cameras being limited in the separate equipment of touch-sensitive display Position, to prevent user from holding electronic equipment in such a way that the hand of user blocks the visual field.In some embodiments, work as electronics When equipment is in first orientation (for example, first horizontal orientation), one or more cameras are located at electronic equipment in this orientation In left-half, first shows and can indicate to be shown in the user interface of the right half part of electronic equipment；Also, at electronic equipment When second orientation (for example, second horizontal orientation that equipment is obtained from the first horizontal orientation rotation 180 degree), in this orientation One or more cameras are located at the right half part of electronic equipment, and first shows the use that can indicate to be shown in the left-half of electronic equipment In the interface of family (for example, as herein with reference to shown in Fig. 5 B-5C and described).

When apparatus orientation changes, automatically keep for by measurement point place on the touch sensitive display far from one Or showing at the position of multiple cameras can indicate that reduce user holds in a manner of blocking the visual field for providing the camera of RUNTIME VIEW The firmly chance of electronic equipment.Relocated automatically in this way when apparatus orientation changes show can indicate to enhance equipment can Operability, and make user-equipment interface it is more efficient (for example, by help user provide it is appropriate input and with set For user's mistake is reduced when operate/interact, the visual field is such as blocked).

In some embodiments, the user interface of application program includes that (620) are corresponding with the operation of application program One or more, which is shown, to be indicated, showing including capture can indicate (for example, showing for such as virtual shutter button can indicate, when it is swashed When living, start the capture to corresponding media are indicated with the visual field) (for example, media capture button 5016, Fig. 5 A).In some realities It applies in scheme, electronic equipment shows in capture can indicate to detect third touch input, and according to determining third touch input Meet the first media capture standard, initiates the capture to the media of the first kind (for example, video or an action shot), the media packet Include it is one or more of it is magazine at least one the visual field image sequence it is (and in some embodiments, corresponding Audio) (for example, as described in referring to Fig. 5 BJ- 5BK herein).In some embodiments, the first media capture standard includes Third touch input keeps at least first requirement of predefined thresholds time quantum and third touch input on the touch sensitive display Amount of movement on the touch sensitive display is less than the second requirement of predefined thresholds amount of movement.In some embodiments, by Capture shows the static long pressing gesture that can be indicated or pressing and keeps gesture to meet the first media capture standard.

In some embodiments, captured image sequence include show on the image (virtual) metrical information (for example, One or more (virtual) measurement points, the line between measurement point, the distance between the label of measurement point and/or measurement point) (example Such as, as described in referring to Fig. 5 BJ-5BK herein).In some embodiments, captured image sequence is not included in application program User interface shown in other show the display that can be indicated (other than the metrical information shown on image), such as catch It obtains and shows and can indicate.In other words, in some embodiments, different from capturing the institute shown in the user interface of application program Substantial screen record, equipment only captures the video in the visual field, wherein (virtual) metrical information is superimposed upon on the visual field (for example, such as Herein with reference to described in Fig. 5 BJ-5BK).

Virtual shutter button is provided or other captures show the video that can indicate to make it easy to record measurand and right Virtual measurement information shown in as on.Such video is recorded without other yuan in the user interface of records application program Element enhances the operability of equipment, and makes user-equipment interface more efficient (for example, by helping user to create display The record of object and measurement, without showing the irrelevant information shown in the user interface during record).

In some embodiments, according to third touch input the second media capture standard of satisfaction is determined, electronic equipment is sent out The capture of (622) to the media of Second Type (for example, static image) is played, which includes one or more of magazine The single image (for example, without corresponding audio) in the visual field of at least one (for example, as referred to Fig. 5 BH-5BI institute herein It states).In some embodiments, the second media capture standard stops at touch-sensitive before being included in the predefined thresholds time quantum past The amount of movement of the first requirement and the second touch input of third touch input on the touch sensitive display is detected on display to be less than The second of predefined thresholds amount of movement requires.In some embodiments, by showing the static tap hand that can be indicated in capture Gesture meets the second media capture standard.

In some embodiments, the static image of capture do not include shown in the user interface of application program other Show the display that can be indicated (other than the metrical information shown on image), such as capture, which is shown, to indicate (for example, as herein It is with reference to shown in Fig. 5 BH-5BI and described).In other words, in some embodiments, different from capturing the user in application program The static image of all the elements shown in interface, equipment only capture the static image in the visual field, wherein (virtual) metrical information is folded It is added on the visual field.

Offer, which is initiated to show the capture of the capture of static image or video, can indicate to make it easy to obtain the quiet of measurand Only image or video, and the virtual measurement information shown on object.Such static image or video are obtained, and it is same Do not include the other elements in the user interface of application program, enhances the operability of equipment, and make user-equipment circle Face is more efficient (for example, by helping user to create the static image or video of display object and measurement, without during showing record The irrelevant information shown in the user interface).In addition, showing for capture can indicate to provide additional control option (for example, capture is quiet Only image or video are specifically dependent upon the feature that the touch input on capable of indicating is shown in capture), without making user interface and volume The control of outer display is chaotic, enhances the operability of equipment, and make user-equipment interface more efficient.

In some embodiments, before the user interface of display application program, electronic equipment shows that (624) control Panel user interface comprising multiple controls (for example, control centre's user interface 5188, Fig. 5 BN), wherein this multiple control In the first control correspond to the application program (for example, augmented reality measures application icon 5190, Fig. 5 BN).In some realities It applies in scheme, the touch input (for example, Flick gesture or pressing input) of electronic equipment detection the first control of activation, and responds In the touch input for detecting the first control of activation, show the user interface of application program (for example, as herein with reference to Fig. 5 BN institute It states).The access to measurement application program is provided by control panel, can more easily control search and start application program.It reduces The quantity of input needed for searching and starting application program can enhance the operability of equipment, and make user-equipment interface It is more efficient.

In some embodiments, user interface is the first user in multiple user interfaces in (626) application program Interface.In some embodiments, the first user interface corresponds to the measurement pattern of the application program.In some embodiments In, the second user interface in this multiple user interface corresponds to leveling (levelling) mode of the application program.Some In embodiment, augmented reality measurement application program or the application program including augmented reality measurement function further include leveling function Energy.Measurement is provided in same application and levelling function to search and becomes to be more easier using these correlation functions. Multiple correlation functions are provided in same application to enhance the operability of equipment and keep user-equipment interface more efficient.

In some embodiments, electronic equipment determines corresponding with measurement point in (628) electronic equipment and three-dimensional space The distance between first position is (for example, from a camera of electronic equipment into three-dimensional space first corresponding with measurement point The distance set).In some embodiments, the distance of the first position from a camera into three-dimensional space is based on by one Or multiple magazine at least one (for example, being optionally a part of one or more cameras by depth transducer) catches The depth information obtained, and/or determined based on the parallax information between multiple and different cameras (for example, by comparing by it is multiple not It is determined with the information of camera capture).In some embodiments, according to determine determined by distance be less than first threshold away from From the amplification factor between the first magnifying power and the second magnifying power is (for example, from the first magnifying power to the magnifying power of the second magnifying power Incrementss) be the first amplification factor (for example, correspond to minimum zoom amount, will not be in electronic equipment and three-dimensional space The distance between first position is reduced to first threshold apart from following and change).In some embodiments, really according to determining Fixed distance be greater than second threshold distance, amplification factor be the second amplification factor (for example, correspond to maximum zoom amount, will not Change above as the distance between the first position in electronic equipment and three-dimensional space increases to second threshold distance).In In some embodiments, according to determining identified distance between first threshold distance and second threshold distance, amplification factor It is the corresponding amplification factor between the first amplification factor and the second amplification factor, depends on identified distance.For example, such as this Text is with reference to shown in Fig. 5 AY-5BE and described, and the amount of zoom of live preview is based on the distance between equipment 100 and desk 5002, contracting High-volume optionally there is the upper limit and/or lower limit.

More generally, in some embodiments, when electronic equipment is farther from the point-of-interest in three-dimensional space, scaling Amount increases (being optionally subjected to maximum zoom amount), and when electronic equipment is closer to point-of-interest in three-dimensional space, scaling Amount reduces (being optionally subjected to minimum zoom amount).In principle, it when electronic equipment is farther from the point-of-interest in three-dimensional space, needs Amount of zoom that will be bigger, because interested feature is more difficult to distinguish (for example, in the expression of the visual field) over a larger distance, and When electronic equipment is closer to point-of-interest in three-dimensional space, smaller amount of zoom is needed, because interested feature is more Perceived (for example, in the expression of the visual field) is easier in small distance.In some embodiments, according to one or more cameras Hardware specification (for example, scale Limit) apply the maximum amount of scaling.

Distance based on the position for corresponding to measurement point in from electronic equipment to three-dimensional space automatic to change amount of zoom and make It must be easier to relocate measurement point, because showing each distance measurement point with the magnifying power of proper level.Using taking Zoom operations certainly are executed in the different amplification factors of the distance of automatic measurement, is inputted without further user, is enhanced and set Standby operability, and make user-equipment interface it is more efficient (for example, by showing measurement point automatically in some magnification levels, Input appropriate can wherein be provided to relocate measurement point).

In some embodiments, when showing at least part of amplification display that the visual field indicates, electronic equipment detection (630) the 4th touch inputs, the 4th touch input include that the contact for detecting the current position of measurement point and detection contact exist Movement (for example, touch input 5164, Fig. 5 AZ) on touch-sensitive display.In some embodiments, the second touch input and 4th touch input is different piece by individually continuously contacting with the input of progress (for example, the second touch input is touch-down The first part of the input of contact on touch-sensitive display, the 4th input are second of input of movement for including the contact Point).In some embodiments, in response to detecting the movement of contact on the touch sensitive display, electronic equipment is according to the 4th touching The movement for touching the contact in input, mobile measurement point is (for example, such as reference measure point 5160 herein in the expression in the measurement visual field It is described, Fig. 5 AZ).Measurement point is dragged while checking the magnification region around measurement point, it can be easily with single contact Movement accurately relocate measurement point.The quantity for reducing input needed for executing repositioning operation enhances equipment Operability, and make user-equipment interface it is more efficient (for example, by help user provide it is appropriate input and with User's mistake is reduced when equipment operate/interact).

In some embodiments, the first touch input it is such show can indicate to be detected (632): when this shows When can indicate to be activated, measurement point is added in shown visual field expression.In some embodiments, this, which shows, to indicate It (fixes) at predefined position and shows in user interface.In some embodiments, initially (show in response to this in the expression of the visual field Can indicate activation and) add the measurement point position be different from this show the predefined position that can be indicated.For example, in Fig. 5 AV- In 5AW, measurement point 5160 is added to user interface 5006 at the position of the position far from measurement addition button 5014.It provides and uses It can be indicated in showing for measurement point of addition, wherein this shows the position that can indicate to be initially displayed far from added measurement point, so that holding The position of measurement point will be placed by easily seeing.Display is for add showing and can indicate a little, and wherein this shows that can indicate separate initial adds The position of these points, enhances the operability of equipment, and makes user-equipment interface more efficient (for example, by helping user Correct input for adding measurement point is provided, and reduces the user's mistake occurred when operating equipment/interact with equipment Accidentally).

It should be appreciated that the concrete order that the operation in Fig. 6 A- Fig. 6 C is described is only example, it is not intended that show The order is the but one sequence that can execute these operations.Those skilled in the art will recognize that various ways come to this Operation described in text is resequenced.Moreover, it is noted that herein in relation to other methods as described herein (for example, Method 700,800,900,1000,1100,1200,1300 is with the details of 1400) other processes of description equally with similar side Formula is suitable for the method 600 described above in relation to Fig. 6 A- Fig. 6 C.For example, input, user above with reference to the description of method 600 Interface element is (for example, measurement point, measuring section, virtual mark, physical space or the expression in the visual field, showing and can indicate, prompt, indicate Symbol, label, anchor point and/or place user interface element such as graticule and point), tactile output and intensity threshold optionally have It is retouched herein with reference to other methods as described herein (for example, method 700,800,900,1000,1100,1200,1300 and 1400) The input stated, user interface element, tactile output and intensity threshold one or more features.For brevity, herein not Repeat these details.

Fig. 7 A-7E is the display according to the physical space that measurement is added in augmented reality environment of some embodiments The flow chart of the method 700 of expression.Method 700 is including touch-sensitive display (for example, touch screen 112 (Figure 1A), or combination input The display generating unit 304 (Fig. 3 B) of equipment 302), to detect and the one or more of the intensity of the contact of touch-sensitive display Sensor (for example, contact strength sensor 165, Figure 1A), one or more cameras are (for example, optical sensor 164 (Figure 1A) Or camera 305 (Fig. 3 B)) and optional one or more tactile output generator (for example, tactile generator 163 (Figure 1A) or Tactile output generator 357 (Fig. 3 A)) electronic equipment (for example, portable multifunction device 100 (Figure 1A), equipment 300 (figure 3A) or computer system 301 (Fig. 3 B)) at execute.Some operations in method 700 are optionally combined and/or one The sequence operated a bit is optionally changed.

As described below, method 700 provide more accurately added in the measurement based on augmented reality virtual measurement point and The intuitive manner of measuring section.For with touch-sensitive display, to detect with one of the intensity of the contact of touch-sensitive display or Multiple sensors, and the electronic equipment of one or more camera, by the contact being consecutively detected on touch-sensitive display into Capable repeated presses make it easier to that electronic equipment is kept to stablize (for example, compared with repetition tap on touch-sensitive display), together The position of RUNTIME VIEW positioning measurement point of the Shi Liyong from least one camera.Method 700 is changed when addition virtual measurement The property with input from the user when measuring section is put, to create more effective man-machine interface.For battery-driven electricity Sub- equipment allows users to more acurrate and adds measurement point and measuring section more quickly and save power and increase battery two Time interval between secondary charging.

Electronic equipment shows (702) application program (for example, augmented reality measurement application program or packet on the touch sensitive display Include the application program of augmented reality measurement function) user interface.

User interface (704) include it is one or more it is magazine at least one the visual field expression.Based on by one or The variation of at least one multiple magazine Current vision data detected, the expression of more New view is (for example, the visual field at any time Expression be RUNTIME VIEW, as one or more cameras are mobile and/or with the object in the visual field of one or more cameras It manages the variation in the world and changes).The user interface include the visual field expression on show measurement point creation indicator (for example, The graticule 5010 combined with focus 5012, Fig. 5 AH are shown in the top of the live preview of the camera in user interface 5006). In addition, the visual field includes at least part (for example, including the space of physical object in physical world) of three-dimensional space (for example, object Manage space 5000).

Electronic equipment detects the contact (for example, touch input 5120, Fig. 5 AI) on (706) touch-sensitive display.

The visual field is shown at contact (708) on continuously detection touch-sensitive display, and in measurement point creation indicator In expression correspond to three-dimensional space in first position first position (710) when, according to determine meet the first standard (for example, Measurement point creates standard), wherein the first standard includes that the intensity of contact is required to meet (for example, meeting or exceeding) corresponding intensity Threshold value is to meet the first standard, and electronic equipment adds (712) in the expression of the visual field and display corresponds to three-dimensional space In first position first (virtual) measurement point (for example, measurement point 5122, Fig. 5 AI).In some embodiments, electronics is set It is standby to determine whether to meet the first standard.In some embodiments, when the intensity of contact is more than intensity threshold (for example, light press Intensity threshold IT_L, it is higher than contact detection intensity threshold value IT₀) when, the first standard of satisfaction (for example, as shown in intensitometer 5040, Fig. 5 AI).In some embodiments, full when the intensity of contact drops to intensity threshold or less (more than after intensity threshold) The first standard of foot.In some embodiments, the first standard includes claimed below: when meeting the other standards of the first standard, Contact, which is positioned in, to be shown and can indicate, when this, which shows, to indicate to be activated, measurement point is added in shown visual field expression The position that shows above measurement point creation indicator (for example, measurement addition button 5014, Fig. 5 AI).

At contact (708) on continuously detection touch-sensitive display, and after adding first (virtual) measurement point, electricity Sub- equipment updates the expression in (714) visual field with the movement of electronic equipment (for example, with the visual field of one or more cameras Variation).In some embodiments, it is moved in response to electronic equipment, first (virtual) of the electronic equipment in the expression of the visual field Survey in measurement point (for example, when the visual field of one or more cameras includes the first position in three-dimensional space) and user interface Dynamic measuring section (for example, dynamic measuring section 5128, Fig. 5 AK) is shown between amount point creation indicator.In some embodiments, The display of the dynamic measuring section changes according to the movement (for example, its visual field for changing one or more cameras) of equipment.For example, The first measurement point and visual field table when fixation position of the measurement point creation indicator in the expression of the visual field is placed in the middle, in the expression of the visual field Correspond to the distance between position of measurement point creation indicator in showing with equipment (and visual field of one or more cameras) Change relative to the first position movement in three-dimensional space, therefore, the length of dynamic measuring section becomes with the movement of equipment Change.

At contact (708) on continuously detection touch-sensitive display, indicated after electronic equipment is mobile, and in the visual field In correspond to three-dimensional space in the second position the second position on show measurement point creation indicator (716) while, root Measurement point creation instruction is shown on the second position for corresponding to the second position in three-dimensional space in the expression of the visual field according to determining Meet while symbol the first standard (718): electronic equipment adds (720) in the expression of the visual field and display corresponds in three-dimensional space The second position second (virtual) measurement point (for example, as explained herein in regard to the addition of measurement point 5132, figure 5AL).In some embodiments, the second position in the expression of the visual field is identical as the first position in the expression of the visual field, such as works as survey When fixation position of the amount point creation indicator in the expression of the visual field is placed in the middle.In some embodiments, electronic equipment determines whether Meet the first standard.In addition, electronic equipment shows the first measuring section of (722) connection the first measurement point and the second measurement point.

In some embodiments, first is shown according to satisfaction the second standard (for example, measuring section creation standard) is determined Measuring section (the first measurement point of connection and the second measurement point).In some embodiments, the second standard includes claimed below: full After the first standard of foot is with the respective instance for adding and showing corresponding measurement point, contact keeps (example on the touch sensitive display Such as, showing and can indicate, after this, which shows, to indicate to be activated, measurement point is being added to the measurement point that the shown visual field indicates and is created Build at the position that indicator is shown above), until meeting the first standard for adding and showing next measurement point Next example (for example, as herein with reference to shown in Fig. 5 AH-5AS and described).That is, the second standard is included in creation completely The requirement being kept in contact between the continuous measurement point of foot (based on intensity) first standard.In some embodiments, for the The intensity threshold of one standard is above the first intensity threshold (for example, contact detection intensity threshold value (IT₀)) the second intensity threshold (for example, light press intensity threshold IT_L), and the second standard includes that the intensity contacted while being kept in contact is maintained at first Intensity threshold or requirement higher than the first intensity threshold.In some embodiments, the second standard includes same what is be kept in contact When the strength reduction that contacts arrive or lower than the requirement of the second intensity threshold (for example, if the intensity of the first standard requirements contact is super Cross the second intensity threshold).In some embodiments, the second standard includes that the intensity of contact is required to protect while being kept in contact It holds in the second intensity threshold or is lower than the second intensity threshold (for example, if the intensity of the first standard requirements contact is lower than intensity threshold Value (more than after intensity threshold)).

More generally, in some embodiments, after adding and showing corresponding point according to the first standard of satisfaction, As long as being kept in contact on the touch sensitive display, one is added and shown according to the first standard is met for each additional measurement point A or multiple additional measurement points and their corresponding measuring sections.As long as that is, being kept in contact on the touch sensitive display, completely Annex point is added between the former point that each subsequent instance when the first standard of foot is added in (newly added) additional point and recently With additional measuring section.In other words, after adding corresponding point according to the first standard of satisfaction, electronic equipment is continuously to measure (point and section) creation mode operation, until being no longer kept in contact on the touch sensitive display.

In some embodiments, in the contact on continuously detection touch-sensitive display, and second (virtual) is being added After measurement point, as electronic equipment moves again, electronic equipment updates the expression in (724) visual field (for example, such as herein with reference to figure It is shown in 5AN-5AQ and described).In some embodiments, in the contact on continuously detection touch-sensitive display, in electronic equipment After moving again, and measurement is shown on the third place in the expression of the visual field corresponding to the third place in three-dimensional space Point creation indicator while, according to determine the visual field expression in correspond to three-dimensional space in the third place the third place it Meet the first standard while upper display measurement point creation indicator: electronic equipment adds in the expression of the visual field and display corresponds to Third (virtual) measurement point (for example, measurement point 5142, Fig. 5 AR) of the third place in three-dimensional space, and show connection second The second measuring section (for example, the measuring section 5136 completed, Fig. 5 AR) of measurement point and third measurement point.In some embodiments, The third place in the expression of the visual field is identical as the first position in the expression of the visual field, such as when measurement point creation indicator is in visual field table When fixation position in showing is placed in the middle.

In some embodiments, correspond to second of the second position in three-dimensional space in the expression of the visual field according to determining It is shown on position and is unsatisfactory for the first standard while measurement point creates indicator, electronic equipment abandons (726) to be indicated in the visual field Middle addition and display correspond to second (virtual) measurement point of the second position in three-dimensional space, and abandon display connection first First measuring section of measurement point and the second measurement point.

In some embodiments, after adding second (virtual) measurement point, as electronic equipment moves again, electronics The expression in equipment update (728) visual field.In some embodiments, after electronic equipment moves again, in the expression of the visual field While showing that measurement point creates indicator on the third place corresponding to the third place in three-dimensional space, and according to true It is scheduled on the second position for corresponding to the second position in three-dimensional space in the expression of the visual field and shows measurement point creation indicator Meet the first standard simultaneously, electronic equipment added in the expression of the visual field and show corresponding to the third place in three-dimensional space the Three (virtual) measurement points.In some embodiments, meet the second standard (for example, measuring section creation standard) according to determining, In the second standard be included in and meet the first standard for adding the second measurement point and meeting the first standard with for adding third The requirement being kept in contact between measurement point, the second measuring section of electronic equipment display connection the second measurement point and third measurement point. In some embodiments, according to second standard that is unsatisfactory for is determined, electronic equipment abandons display the second measurement point of connection and third The second measuring section (for example, as herein with reference to shown in Fig. 5 AH-5AS and described) of measurement point.In some embodiments, if Corresponding measurement point is the measurement point (for example, second measurement point) added recently and is as the measuring section (example added recently Such as, the first measuring section) measuring section endpoint, then according to determine meet the first standard (for example, measurement point creation standard) and It is unsatisfactory for the second standard, electronic equipment adds and shows additional measurement point (for example, third measurement point) and abandons corresponding Additional measuring section is shown between measurement point (showing the measurement point added recently before additional measurement point) and additional measurement point.

In some embodiments, in the contact on continuously detection touch-sensitive display, and when electronic equipment is mobile, The measurement point (for example, second measurement point) added recently and the visual field expression in correspond to measurement point creation indicator position it Between show dynamic measuring section (for example, herein with reference to operation 714 as described in dynamic measuring section), and according to determine meet second Standard and continue to show the dynamic measuring section.In some embodiments, if the measurement point added recently is as showing It is the endpoint of another measuring section of the measuring section added recently before dynamic measuring section (for example, first measuring section), then electronics is set It is standby meet the first standard with for after adding the second measurement point (for example, meeting the first standard for meeting addition the Any point before three measurement points), in response to being lifted away from for contact, stop display dynamic measuring section (for example, such as herein with reference to figure It is shown in 5AH-5AS and described).

Whether it is kept in contact according on touch-sensitive display, and then adds the measurement point with or without corresponding measuring section It is added to previous measurement point, is provided for adding measurement point without adding the additional option of measuring section (for example, user lifts contact From to indicate that next measurement point to be added is not attached to previous measurement point and section.) provide additional control option without Because additional display control makes UI confusion enhance the operability of equipment and makes user-equipment interface more efficient (for example, logical It crosses help user and correctly input is provided, and reduce the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, electronic equipment includes one or more tactile output generators.In some embodiments In, according to intensity satisfaction (for example, reach and (increase to) or be more than) the corresponding intensity threshold for determining contact, electronic equipment is generated (730) tactile output (for example, to indicate that the intensity of contact is enough to add measurement point) herein with reference to tactile (for example, such as export Described in 5124, Fig. 5 AI).In some embodiments, tactile output is generated according to the first standard of satisfaction is determined (for example, removing The intensity of contact meets outside the requirement of respective strengths threshold value, also meets other requirements of first standard).In some embodiments In, electronic equipment provides tactile output when meeting corresponding intensity threshold, to indicate that the intensity of contact is enough to add measurement point And/or instruction measurement point has been added.Improved feedback is provided to enhance the operability of equipment and make user equipment interface more Efficiently (for example, by helping user to provide correctly input, and reduce the use occurred when operating equipment/interact with equipment Family mistake).

In some embodiments that electronic equipment includes one or more tactile output generators, corresponding intensity threshold It is that (732) are higher than the first intensity threshold (for example, contact detection intensity threshold value IT₀) the second intensity threshold (for example, flicking pressure Spend threshold value IT_L).In some embodiments, the second intensity threshold is no longer met according to the intensity of determining contact and (is meeting second After intensity threshold) (for example, the strength reduction of contact arrives or lower than the second intensity threshold), and the intensity contacted meets first Intensity threshold (for example, the intensity of contact is maintained at the first intensity threshold or is higher than the first intensity threshold), electronic equipment generates touching Feel output (for example, addition to indicate measurement point) (for example, as described in herein with reference to Fig. 5 AJ).In some embodiments, when When intensity is lower than the second intensity threshold, electronic equipment provides tactile output, has added measurement point with instruction.Improved feedback is provided Enhance the operability of equipment and make user equipment interface it is more efficient (for example, by help user provide correctly input, And reduce the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, the addition of the first measurement point is executed when the intensity of contact meets corresponding intensity threshold With display (734), while the visual field expression in correspond to three-dimensional space in first position first position on show measurement Point creation indicator (for example, as herein with reference to shown in Fig. 5 AI and described).In some embodiments, mobile in electronic equipment The intensity contacted later executes the addition and display of the second measurement point when meeting corresponding intensity threshold, while in the expression of the visual field Show that measurement point creates indicator on the second position corresponding to the second position in three-dimensional space.

In some embodiments, corresponding intensity threshold is that (736) are higher than the first intensity threshold (for example, contact detection Intensity threshold IT₀) the second intensity threshold (for example, light press intensity threshold IT_L).In some embodiments, the first standard Including requiring the intensity contacted after meeting corresponding intensity threshold to drop to corresponding intensity threshold hereinafter, to meet One standard.The first measurement point is executed when in some embodiments, below the strength reduction of contact to corresponding intensity threshold Addition and display, while the visual field expression in correspond to three-dimensional space in first position first position on show measurement Point creation indicator.In some embodiments, after electronic equipment is mobile, when the strength reduction of contact is to corresponding intensity The addition and display of the second measurement point are executed when below threshold value, while corresponding to the second in three-dimensional space in the expression of the visual field Show that measurement point creates indicator on the second position set.

In some embodiments, when showing the expression in the visual field of one or more cameras, electronic equipment determines (738) Correspond to the anchor point of the corresponding position in three-dimensional space in the visual field expression of one or more cameras.In some embodiments In, as one or more cameras are mobile, when measurement point creation indicator (or its at least part) is (or attached on anchor point When closely), electronic equipment changes the visual appearance of measurement point creation indicator, meets the first standard if contacted with instruction, will The corresponding measurement point of addition (for example, as described in referring to graticule 5010 and focus 5012 in Fig. 5 A herein) at anchor point. It provides and makes it easy to be anchored by the visual feedback for adding measurement point at anchor point in the case where contact meets the first standard Measurement point is added at point.Improved feedback is provided to enhance the operability of equipment and make the more efficient (example in user equipment interface Such as, by helping user to provide correctly input, and the user's mistake occurred when operating equipment/interact with equipment is reduced).

In some embodiments, user interface includes that (740) show and can indicate that this, which shows, to indicate when being activated, and is being regarded Position addition (virtual) measurement point that measurement point creation indicator is shown above in the expression of open country is (for example, measurement addition button 5014).In some embodiments, electronic equipment detection activates this to show the touch input (for example, Flick gesture) that can be indicated, and And this shows the touch input that can be indicated in response to detecting activation, measurement point creation indicator is shown above in the expression of the visual field The position addition and display measurement point (for example, as herein with reference to shown in Fig. 5 K and described) shown.

It should be appreciated that the particular order that the operation in Fig. 7 A to Fig. 7 E is described is only exemplary, not purport Showing that the sequence is the unique order that can execute these operations.Those skilled in the art will recognize that various ways To resequence to operations described herein.Moreover, it is noted that herein in relation to other methods as described herein The details of other processes of (for example, method 600,800,900,1000,1100,1200,1300 and 1400) description is equally with class As mode be suitable for the method 700 that describes above with reference to Fig. 7 A-7E.For example, input, use above with reference to the description of method 700 Family interface element is (for example, measurement point, measuring section, virtual mark, physical space or the expression in the visual field, showing and can indicate, prompt, refer to Show symbol, label, anchor point and/or place user interface element such as graticule and point), tactile output and intensity threshold optionally have Have herein with reference to other methods as described herein (for example, method 600,800,900,1000,1100,1200,1300 and 1400) The input of description, user interface element, tactile exports and the one or more features of intensity threshold.For brevity, herein These details are not repeated.

Fig. 8 A-8C is that void is added at the anchor point automatically determined in augmented reality environment according to some embodiments The flow chart of the method 800 of quasi- measurement point.Method 800 is including touch-sensitive display (for example, touch screen 112 (Figure 1A) or combination The display generating unit 304 (Fig. 3 B) of input equipment 302), and one or more cameras are (for example, (figure of optical sensor 164 1A) or camera 305 (Fig. 3 B)), optionally in the one or more sensors for the intensity for detecting contact with touch-sensitive display (for example, contact strength sensor 165, Figure 1A), and optional one or more tactile output generators are (for example, tactile is defeated Generator 163 (Figure 1A) or tactile output generator 357 (Fig. 3 A) out) electronic equipment (for example, portable multifunction device 100 (Figure 1A), equipment 300 (Fig. 3 A) or computer system 301 (Fig. 3 B)) at execute.Some operations in method 800 are optional Ground is combined and/or the sequence of some operations is optionally changed.

As described below, method 800 provide in the measurement based on augmented reality at the anchor point automatically determined or Intuitive manner far from such anchor point addition virtual measurement point.Electronic equipment is provided in the feelings for meeting measurement point creation standard The visual feedback of measurement point will be added under condition at anchor point, this makes it easy to add measurement point at anchor point.It provides and improves Feedback enhance the operability of equipment and make user equipment interface more efficient (for example, correct by helping user to provide Input, and reduce the user's mistake occurred when operating equipment/interact with equipment).For battery-driven electronic equipment, Allow users to more rapidly and more effectively to add measurement point at the anchor point automatically determined (or far from such anchoring Point addition measurement point) save power and increase battery charge twice between time interval.

Electronic equipment shows (802) application program (for example, augmented reality measurement application program or packet on the touch sensitive display Include the application program of augmented reality measurement function) user interface.

User interface (804) include it is one or more it is magazine at least one the visual field expression.The expression base in the visual field It is updated at any time in the variation by least one the one or more magazine Current vision data detected (for example, view Wild expression is RUNTIME VIEW).In addition, user interface includes that the measurement point shown on the expression of the visual field creates indicator (example Such as, the graticule 5010 combined with focus 5012, Fig. 5 K).The visual field includes at least part of three-dimensional space (for example, physical world In include physical object space).

While showing expression (806) in the visual field, electronic equipment determines that (808) correspond to three-dimensional space in the expression of the visual field Between in first position position at anchor point.In some embodiments, electronic equipment, which determines, corresponds in three-dimensional space Multiple positions multiple anchor points, the turning of the physical object in such as three-dimensional space, along the physics pair in three-dimensional space The point at the edge of elephant etc..

When showing that the visual field indicates (806) with one or more cameras movements, and when measurement point creates indicator (or its at least part) on anchor point (near or) when, electronic equipment change (810) measurement point creation indicator view Appearance is felt, if will add at anchor point to indicate that touch input meets the first standard (for example, measurement point creation standard) Add (virtual) measurement point (for example, as described in referring to graticule 5010 and focus 5012 in Fig. 5 AK herein).

While showing expression (806) in the visual field, electronic equipment detects the satisfaction first on (812) touch-sensitive display and marks (for example, showing the Flick gesture on capable of indicating or showing the input of the pressing again on capable of indicating, this is heavy to be pressed for quasi- the first touch input Pressure input meets or is more than intensity threshold) (for example, touch input 5038, Fig. 5 K).

While showing expression (806) in the visual field, the first touch input of the first standard is met in response to detecting (814), and according to determine when meeting the first standard measurement point create indicator (or its at least part) anchor point it Upper (near or) (816), electronic equipment add at the anchor point in the expression in the visual field corresponding to the first position in three-dimensional space Adduction display first (virtual) measurement point (for example, measurement point 5042, Fig. 5 K).

While showing expression (806) in the visual field, indicator is created according to the determining measurement point when meeting the first standard (or its at least part) not on anchor point (near or) (818), electronic equipment leaves anchor point in the expression of the visual field First position (for example, not corresponded at the position of the first position in three-dimensional space in the expression of the visual field) addition and display first (virtual) measurement point.

In some embodiments, identified anchor point (820) is also the endpoint (example of the measurement expression of (current) display Such as, as described in herein in regard to the anchor point for corresponding to measurement point 5054, Fig. 5 Q), and meet the first standard in touch input In the case of at the anchor point addition measurement point will not be in formed in the expression of the visual field by multiple displays measuring section (for example, and Its associated endpoint) surround region.Measurement point is added or in response to true for example, being incorporated at determining anchor point Add measurement point at fixed anchor point and the measuring section that adds will not be formed it is polygon as being closed including added measuring section The closed polygon on the last item side of shape.It provides to add at anchor point in the case where meeting measurement point creation standard and survey The visual feedback for measuring point makes it easy to add measurement point at anchor point, including not only making the anchoring for measuring dot cycle closure At point.Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, passing through help User provides correctly input, and reduces the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, electronic equipment shows the expression of (822) first measurements on the expression in the visual field.First The expression of measurement includes the first end point corresponding to the second position in three-dimensional space, corresponding to the third place in three-dimensional space The second endpoint, and connection first end point and the second endpoint the first line segment.In addition, identified anchor point is the first line segment Midpoint (for example, the half between the second position in three dimensions of first position corresponding to anchor point and the third place Place) (for example, as herein with reference to shown in Fig. 5 AA and described).Offer will measured in the case where meeting measurement point creation standard The visual feedback that measurement point is added at the anchor point of section midpoint makes it easy to add measurement point in measuring section midpoint.It provides It is improved to feed back the operability for enhancing equipment and make user equipment interface more efficient (for example, by helping user to provide Correctly input, and reduce the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, electronic equipment includes one or more tactile output generators.In some embodiments In, at least one one or more magazine movement, and when measurement point creation indicator (or its at least part) When on anchor point (near or), electronic equipment generates the output of (824) tactile, and Binding change measurement point creates indicator Visual appearance (for example, as described in herein in regard to Fig. 5 H).In some embodiments, the feelings of the first standard are met in touch input Addition measurement point will not be in the visual field expressions of one or more cameras and form the survey by multiple displays at the anchor point under condition Measure the region that section (for example, and its associated endpoint) surrounds.For example, addition measurement point will be combined or in response to adding measurement point And add measuring section will not be formed it is more as the closure on the last item side of closed polygon including added measuring section Side shape.In some embodiments, when the physical features of the identification snapped in three-dimensional space, tactile output is generated.One It in a little embodiments, is indicated when snapping to the measurement currently shown, snaps to its corresponding measurement point (for example, endpoint) and/or right When arriving the midpoint of its measuring section together, tactile output is generated.Offer will be in anchor point in the case where meeting measurement point creation standard Both touch feedback and visual feedback of place's addition measurement point, make it easy to add measurement point at anchor point, be included in not only Make to measure at the anchor point of dot cycle closure.Improved feedback is provided to enhance the operability of equipment and make user equipment circle Face is more efficient (for example, by helping user to provide correctly input, and to reduce and occurs when operating equipment/interact with equipment User's mistake).

In some embodiments that electronic equipment includes one or more tactile output generators, electronic equipment detection (826) movement of the measurement point creation indicator far from anchor point is (for example, make measurement point creation indicator not on anchor point (or not in its vicinity)) (for example, the movement in the change visual field due to one or more cameras, while measurement point creates indicator It is maintained at the fixation position in user interface).In some embodiments, in response to detecting that measurement point creation indicator is separate The movement of anchor point, electronic equipment generate tactile output (for example, as described in herein in regard to Fig. 5 I).It provides and is meeting measurement point wound It will not be in the case where building standard at anchor point and add the touch feedback of measurement point, help to guide when placing measurement point and use Family.Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, by helping to use Family provides correctly input, and reduces the user's mistake occurred when operating equipment/interact with equipment).

It is full in response to detecting in some embodiments that electronic equipment includes one or more tactile output generators First touch input of the first standard of foot, electronic equipment add (828) first measurement points and are connected to the first measurement point without adding Measuring section (for example, the first measurement point is the first end point newly measured) and generate the first tactile output (for example, measurement start Tactile output) (for example, as described in herein in regard to measurement point 5042, Fig. 5 K).In some embodiments, electronic equipment detection is surveyed Amount point creation indicator indicated to the visual field in the second position movement, the second position correspond to three-dimensional space in second Set (for example, described herein with respect to three-dimensional space, Fig. 5 M).In some embodiments, when measurement point creation indicator exists When on the second position in the expression of the visual field, electronic equipment detects the second touch input on touch-sensitive display, meets the One standard.In some embodiments, the second touch input of the first standard is met in response to detecting, electronic equipment is in the visual field Add the second measurement point (for example, as described in herein in regard to measurement point 5054, Fig. 5 N) in the second place in expression.Some In embodiment, the second touch input of the first standard is met in response to detecting, electronic equipment is in the first measurement point and second Added between measurement point measuring section (for example, electronic equipment adds the expression of measurement by the second measurement point of addition and measuring section, The expression wherein measured includes the first measurement point, the second measurement point and measuring section) (for example, as herein in regard to 5048 institute of measuring section It states, Fig. 5 N).In some embodiments, the second touch input of the first standard is met in response to detecting, electronic equipment is raw At different from the output of the first tactile (for example, the output of the second tactile and the output of the first tactile are all at least one tactile output characteristics Such as frequency, amplitude, mode, different on the duration) the second tactile output (for example, measurement terminates tactile output) (for example, As described in herein in regard to tactile output 5056, Fig. 5 N).(first measurement point is used only) when measuring and starting and creation measures Different touch feedbacks is provided when section to be facilitated when placing measurement point by indicating them the location of in measurement process To guide user.Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, logical It crosses help user and correctly input is provided, and reduce the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, when at least one one or more magazine movement, include in the expression of visual field Region corresponding with the physical features of identification in the three-dimensional space at least one one or more magazine visual fields When, electronic equipment shows that (830) measurement point creates indicator, in addition, the expression in visual field does not include and the knowledge in three-dimensional space When the corresponding region of other physical features, electronic equipment stops display measurement point creation indicator (for example, such as reference herein Described in Fig. 5 E to Fig. 5 G).It whether include area corresponding with the physical features identified in three-dimensional space according to RUNTIME VIEW Domain shows or does not show that measurement point creates indicator, and the present or absent vision for providing the feature about automatic identification is anti- Feedback.Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, by helping to use Family provides correctly input, and reduces the user's mistake occurred when operating equipment/interact with equipment).

It should be appreciated that the concrete order that the operation in Fig. 8 A- Fig. 8 C is described is only example, it is not intended that show The order is the but one sequence that can execute these operations.Those skilled in the art will recognize that various ways come to this Operation described in text is resequenced.Moreover, it is noted that herein in relation to other methods as described herein (for example, Method 600,700,900,1000,1100,1200,1300 is with the details of 1400) other processes of description equally with similar side Formula is suitable for the method 800 described above with reference to Fig. 8 A to Fig. 8 C.For example, input, user above with reference to the description of method 800 Interface element is (for example, measurement point, measuring section, virtual mark, physical space or the expression in the visual field, showing and can indicate, prompt, indicate Symbol, label, anchor point and/or place user interface element such as graticule and point), tactile output and intensity threshold optionally have It is retouched herein with reference to other methods as described herein (for example, method 600,700,900,1000,1100,1200,1300 and 1400) The input stated, user interface element, tactile output and intensity threshold one or more features.For brevity, herein not Repeat these details.

Fig. 9 A to Fig. 9 B is to show to be shown in augmented reality environment according to some embodiments for measuring physics sky Between label method 900 flow chart.Method 900 is including display (for example, touch screen 112 (Figure 1A), display 340 (Fig. 3 A) or display generating unit 304 (Fig. 3 B)), input equipment (for example, touch screen 112 (Figure 1A), touch tablet 355 (figure 3A), input equipment 302 (Fig. 3 B), or the physical button separated with display), and one or more cameras are (for example, optics Sensor 164 (Figure 1A) or camera 305 (Fig. 3 B)), optionally with strong with the contact of the touch sensitive surface of input equipment in detecting The one or more sensors (for example, contact strength sensor 165, Figure 1A) of degree, and optional one or more tactiles are defeated The electronic equipment of generator (for example, tactile output generator 163 (Figure 1A) or tactile output generator 357 (Fig. 3 A)) out It is executed at (for example, portable multifunction device 100 (Figure 1A), equipment 300 (Fig. 3 A) or computer system 301 (Fig. 3 B)). Some operations in method 900 are optionally combined and/or the sequence of some operations is optionally changed.

As described below, method 900 is based on the distance between electronic equipment and given measurement, provides intuitive way and comes for not Same measurement provides label.In the maximum sized label of short distance offer these labels will not be become too big and blocks view The major part that open country indicates.The label that minimum dimension is provided at long range can make these labels clear and legible.And in medium spacing Relative distance of the corresponding measurement of variable size label instruction in visual field expression is provided from place.In this way at different distances It provides and enhances the operability of equipment for the label of different measurements and make user equipment interface more efficient (for example, logical It crosses help user to check when operate/interacting with equipment and using measurement label).

Electronic equipment shows (902) application program (for example, augmented reality measures application program or including increasing over the display The application program of strong reality measurement function) user interface.

User interface (904) include it is one or more it is magazine at least one the visual field expression.The expression base in the visual field It is updated at any time in the variation by least one the one or more magazine Current vision data detected (for example, view Wild expression is RUNTIME VIEW).The visual field includes the physical object in three-dimensional space (for example, space in physical world).

When showing that the visual field indicates, electronic equipment inputs (example via the one or more users of input equipment detection (906) Such as, Flick gesture or show that the pressing on capable of indicating inputs, when it is described show can indicate to be activated when, measurement point is added to shown by Visual field expression in), addition corresponds to physical object on the expression of the visual field first surveys for one or more of users input The expression of amount is (for example, correspond to the line of the display of the measurement of the length of physical object, width or other sizes, wherein shown Line be applied or overlap in the expression in the visual field).In some embodiments, electronic equipment is shown simultaneously on the expression of the visual field Show the first label of the first measurement of expression and description of (908) first measurements.In some embodiments, it is set according to determining electronics The standby first distance (for example, from camera for electronic equipment to the distance of physical object) between physical object is less than first Threshold distance (for example, lower limit distance threshold), the first label (910) are shown with first threshold size (for example, upper limit size threshold value Or largest amount, not as the distance between electronic equipment and physical object are reduced to first threshold apart from following and change).In In some embodiments, according to the first distance determined between electronic equipment and physical object be greater than second threshold distance (for example, Upper distance limit threshold value), second threshold distance is greater than first threshold distance, and the first label is to be less than the second of first threshold size Threshold size is shown (for example, lower limit size threshold value or minimal size, not as the distance between electronic equipment and physical object increase Add and be more than second threshold distance and change).In some embodiments, according to determine between electronic equipment and physical object the One distance is between first threshold distance and second threshold distance, with the size between first threshold size and second threshold size Show the first label, the size depends on the first distance between electronic equipment and physical object.Based on electronic equipment and object The variation of the tag size of the distance between reason object shows and describes herein with reference to Fig. 5 A to Fig. 5 B.

In some embodiments, when showing the label that the expression of measurement and description measure on the expression of the visual field: in electricity When sub- equipment (or camera for electronic equipment) is apart from physical object first distance, electronic equipment is indicated along measurement with first Scale shows (912) one or more first scale marks (for example, indicating aobvious with the interval of the first predefined distance along measurement Show one or more first scale marks, the interval corresponding to the in three-dimensional space first predefined physical distance).In some implementations In scheme, electronic equipment detects the movement (or movement of one or more camera) of electronic equipment, and the movement sets electronics It is standby to be moved to away from physical object second distance.In some embodiments, when electronic equipment is apart from physical object second distance, Electronic equipment is along at least part that measurement indicates with the second scale display one or more second different from the first scale Scale mark is (for example, electronic equipment indicates to show one or more second quarters with the interval of the second predefined distance along measurement Scale designation, the interval corresponding to the second predefined physical distance in three-dimensional space).Based between electronic equipment and physical object The variation of scale of display label of distance show and describe herein with reference to Fig. 5 A to Fig. 5 B.

In some embodiments, when electronic equipment is in the first predefined distance range away from physical object, along First scale mark of one or more of expression the first scale of display of measurement is (for example, distance is greater than second threshold distance or the The distance between one threshold distance and second threshold distance), wherein the first predefined distance range includes first distance.Some In embodiment, the second predefined distance range away from physical object is in (for example, first threshold distance and the in electronic equipment The distance between two threshold distances, or the distance less than first threshold distance) it is interior when, at least part indicated along measurement is aobvious Show the second scale the second scale mark of one or more (for example, if due to electronic equipment moves close to physical object and only Continue a part that display measurement indicates in the user interface), wherein the second predefined distance range includes second distance.

In one example, the movement of the electronic equipment detected keep electronic equipment mobile closer to physical object, make It obtains first distance and is greater than second distance.Although electronic equipment is away from physical object first distance, along corresponding to three-dimensional space In one foot of interval measurement expression show at certain intervals one or more scale marks.In same example, although Electronic equipment away from physical object second distance, but along correspond to three-dimensional space in one inch of interval measurement indicate with Certain intervals show one or more scale marks.Those skilled in the art are it will be recognized that work as equipment and physical object At a distance of accordingly apart from when, can indicate different predefined physical distances by the scale mark of display, such as rice, decimetre, Centimetre, millimeter, yard, foot, inch, quarter inch or any other suitable distance.

It provides as the scale mark ratio for changing from the distance for measuring equipment and changing scale automatically requires user manual It is more efficient to change scale.Operation is executed when one group of condition (for example, distance condition) obtains meeting without further User's input, which enhance the operability of equipment, and make user equipment interface more efficient (for example, by helping to use Family provides input appropriate and reduces user's mistake when operate/interact with equipment).

In some embodiments, electronic equipment detects (914) second groups of one or more user inputs, in visual field table Addition corresponds to the respective physical object in three-dimensional space (for example, the corresponding same physical object of the first measurement or three on showing Different physical objects in dimension space) second measurement expression.In some embodiments, electronic equipment indicates it in the visual field Second label of upper while the second measurement of display expression and the second measurement of description.In some embodiments, electric according to determining Second distance between sub- equipment and respective physical object is less than first threshold distance, and the second label is aobvious with first threshold size Show.In some embodiments, second threshold is greater than according to the second distance determined between electronic equipment and respective physical object Distance, the second label are shown with second threshold size.In some embodiments, according to determining electronic equipment and respective physical pair Second distance as between is between first threshold distance and second threshold distance, with first threshold size and second threshold size Between size show the second label, the size depends on the second distance between electronic equipment and respective physical object.In In some embodiments, the expression while display in the visual field change and change multiple labels of size with the visual field.

In some embodiments, the first distance between electronic equipment and the corresponding physical object of the first measurement (916) second distance being different between respective physical object corresponding to electronic equipment and the second measurement.In some embodiment party In case, the first label is with first size (for example, being based on first distance) display, and the second label is with the second size (for example, being based on Second distance) display, and first size is different from the second size.In this example, first distance is predefined apart from model first In enclosing, and second distance in the second predefined distance range (for example, the first predefined scope is one of the following terms: small In the distance of first threshold distance, greater than between the distance or first threshold distance and second threshold distance of second threshold distance Distance, and the second predefined scope is the different range in above range).In another example, first distance and second Distance is the different distance in the distance between first threshold distance and second threshold distance range, therefore, their related mark The corresponding size of label (for example, label of the measurement corresponded at first distance and second distance) is different.

It should be appreciated that the particular order that the operation in Fig. 9 A to Fig. 9 B is described is only exemplary, not purport Showing that the sequence is the unique order that can execute these operations.Those skilled in the art will recognize that various ways To resequence to operations described herein.Moreover, it is noted that herein in relation to other methods as described herein The details of other processes of (for example, method 600,700,800,1000,1100,1200,1300 and 1400) description is equally with class As mode be suitable for the method 900 that describes above with reference to Fig. 9 A to Fig. 9 B.For example, above with reference to method 900 describe input, User interface element (for example, measurement point, measuring section, virtual mark, physical space or the expression in the visual field, show can indicate, prompt, Indicator, label, anchor point and/or place user interface element such as graticule and point), tactile output and intensity threshold it is optional Ground has herein with reference to other methods as described herein (for example, 600,700,800,1000,1100,1200,1300 and of method 1400) one or more features of the input of description, user interface element, tactile output and intensity threshold.For brevity, These details are not repeated herein.

Figure 10 A to Figure 10 B is to show to be measured in physical space in augmented reality environment according to some embodiments The flow chart of rectangular area and the method 1000 interacted.Method 1000 is including display (for example, (figure of touch screen 112 1A), display 340 (Fig. 3 A) or display generating unit 304 (Fig. 3 B)), input equipment (for example, touch screen 112 (Figure 1A), touching Template 355 (Fig. 3 A), input equipment 302 (Fig. 3 B), or the physical button separated with display), and one or more cameras (for example, optical sensor 164 (Figure 1A) or camera 305 (Fig. 3 B)), optionally in the touch sensitive surface of detection and input equipment Contact intensity one or more sensors (for example, contact strength sensor 165, Figure 1A) and it is optional one or Multiple tactile output generators (for example, tactile output generator 163 (Figure 1A) or tactile output generator 357 (Fig. 3 A)) Electronic equipment (for example, portable multifunction device 100 (Figure 1A), equipment 300 (Fig. 3 A) or computer system 301 (Fig. 3 B)) Place executes.Some operations in method 1000 are optionally combined and/or the sequence of some operations is optionally changed.

As described below, method 1000 provides a kind of intuitive way to detect and indicate the rectangle adjoined with measurement.It is logical Crossing makes the rectangle detected adjoin the measurement for inputting and carrying out in response to user, and method 1000 reduces equipment and will test and indicate The risk of the rectangle of (for example, not being measured) unrelated with user, to create more efficient man-machine interface.It powers for battery Electronic equipment, can faster, more efficiently detect rectangle related to display and can save electric power and increase battery and charge twice Between time.

Electronic equipment show over the display (1002) application program (for example, augmented reality measurement application program or including Augmented reality measurement function application program) user interface.

User interface (1004) include it is one or more it is magazine at least one the visual field expression.The expression base in the visual field It is updated at any time in the variation by least one the one or more magazine Current vision data detected (for example, view Wild expression is RUNTIME VIEW).The visual field include in three-dimensional space (for example, space in physical world) physical object (or its Part).

When showing that the visual field indicates, electronic equipment inputs (example via the one or more users of input equipment detection (1006) Such as, Flick gesture or show that the pressing on capable of indicating inputs, when it is described show can indicate to be activated when, measurement point is added to shown by Visual field expression in), addition corresponds to physical object on the expression of the visual field first surveys for one or more of users input The expression of amount is (for example, the user detected inputs so that electronic equipment addition corresponds to the aobvious of the measurement at the edge of physical object The line shown, wherein shown line is applied or is overlapped in the expression of the visual field).

The expression of first measurement includes the first end point for the first position that (1008) correspond on physical object.First measurement Expression include the second endpoint corresponding to the second position on physical object.The expression of first measurement includes connection first end point With the first line segment of the second endpoint.It has shown and described herein with reference to Fig. 5 J to Fig. 5 O including two endpoints and connection two The addition that the measurement of first line segment of endpoint indicates.

Electronics portion based on the first measurement (or its expression) determine in the expression of (1010) visual field with the first measurement The first area that first line segment adjoins.First area corresponds to the physics rectangular area in three-dimensional space (for example, entire physics square Shape region or part of it).For example, physics rectangular area or part of it of the first area corresponding to physical object, and the One measurement corresponds to an edge of the physics rectangular area of physical object.In some embodiments, due to one or more At least one magazine visual angle relative to physics rectangular area, the first area in the expression of the visual field are not shown in the expression of the visual field It is shown as rectangle.In some embodiments, the image procossing (for example, using estimation of Depth) that electronic equipment is indicated based on the visual field The physical region for determining the first area corresponded in the expression of the visual field is rectangular area.In some embodiments, one or more At least one a magazine visual field includes the first part of physics rectangular area, without one including physics rectangular area Or multiple extentions.In some embodiments, check that one or more extentions of physics rectangular area need equipment Movement, at least one the mobile one or more magazine visual field of the equipment is to include one or more of physics rectangular area A extention.

Electronic equipment shows the instruction (for example, indicator 5058, Fig. 5 P) of the first area in (1012) user interface. The instruction is overlapped (for example, superposition) on the first area in the expression of the visual field.

In some embodiments, user interface includes the measurement point creation instruction that (1014) are shown on the expression of the visual field It accords with (for example, the graticule 5010 combined with focus 5012, Fig. 5 Y).In some embodiments, according to determining in one or more Camera the visual field expression in show on first area measurement point create indicator come show first area instruction (for example, It is as shown in Fig. 5 Y and described in referred to herein).In some embodiments, when measurement point creates indicator (or its at least one Point) on first area when, user's input that electronic equipment detection is carried out via input equipment is (for example, corresponding to showing energy Flick gesture on touch sensitive surface or pressing input at the position of expression, when it is described show can indicate to be activated when, will measure or Measurement point, which is added to the shown visual field, to be indicated).In some embodiments, in response to when measurement point creation indicator is first Detect that the user inputs when on region, electronic equipment changes the visual appearance of the instruction of first area, to indicate the firstth area Domain has been identified (for example, as herein with reference to shown in Fig. 5 Z and described).

In some embodiments, when measurement point creation indicator is on first area and in display first area It detects that user inputs when instruction, also, when measurement point creation indicator is on first area, and is showing the firstth area When the instruction in domain, in response to (for example, according to) detect that user inputs, change the visual appearance of the instruction of first area.One In a little embodiments, this method include show on the expression of the visual field describe first area one or more labels (for example, with The instruction that is shown with the visual appearance of variation at the same show).In some embodiments, the one or more of first area is described Label includes the label (for example, length of physics rectangular area) for indicating the length of the first side of first area, indicates the firstth area The label (for example, width of physics rectangular area) of the length of the second side in domain, and/or indicate the area of first area Label (for example, area of physics rectangular area) (for example, as herein with reference to shown in Fig. 5 Z and described).

The appearance for changing the instruction of first area, which provides electronic equipment and has been detected by user's confirmation and measure with first, to be adjoined Adjacent rectangle is correct visual feedback.Improved feedback is provided to enhance the operability of equipment and make user equipment interface It is more efficient (for example, by helping user to provide correctly input, and to reduce and occur when operating equipment/interact with equipment User's mistake).

In some embodiments, one or more user's inputs addition (1016) on the expression of the visual field corresponds to physics The expression of second measurement of object is (for example, user detected input makes electronic equipment addition correspond to the second of physical object Second display line of the measurement at edge, wherein the second display line is applied or is overlapped in the expression of the visual field) (for example, by measuring section 5066 and its corresponding endpoint indicate measurement, Fig. 5 Y).In some embodiments, the expression of the second measurement includes corresponding to The second endpoint (for example, measurement point 5054, Fig. 5 Y) of the second position on physical object.In some embodiments, it second surveys The expression of amount includes the third endpoint corresponding to the third place on physical object (for example, the third place and first position and the Two positions are different) (for example, measurement point 5090, Fig. 5 Y).In some embodiments, the expression of the second measurement includes connection the The second line segment (for example, measuring section 5066, Fig. 5 Y) of two endpoints and third endpoint.In some embodiments, it is surveyed based on first Amount and the second measurement determine the first area in the visual field expression of one or more cameras.In some embodiments, the firstth area The first line segment of the first measurement and the second line segment of the second measurement are adjoined in domain.It determines that first area reduces based on measurement twice to set The risk of the standby rectangle that will test and indicate (for example, not being measured) unrelated with user, to enhance the operability of equipment And keep user's equipment interface more efficient (for example, by helping user to provide input appropriate when operating equipment/interact with equipment And reduce user's mistake).

In some embodiments, it is the one or more it is magazine at least one the visual field include (1018) physics The first part of rectangular area, and first area corresponds to the first part of the physics rectangular area (for example, as with reference to instruction Described in region indicated by symbol 5104, Fig. 5 AB).In some embodiments, the movement of electronic equipment detection electronic equipment It is one or more it is magazine at least one the visual field movement (for example, as referred to shown in Fig. 5 AC to Fig. 5 AD herein and institute It states).In some embodiments, in response to detecting the movement for keeping the visual field mobile of electronic equipment, electronic equipment is at any time more The expression of New view is to show one or more further regions corresponding with one or more extra sections of physics rectangular area One or more instructions in domain.In some embodiments, according to determine include first area and at any time show one or The zone of convergency of multiple additional areas correspond to entire physics rectangular area, electronic equipment the visual field expression on display label, Label description corresponds to the measurement (for example, as described in reference label 5110 herein, Fig. 5 AE) of the physics rectangular area.In In some embodiments, the area (for example, area of entire physics rectangular area) of label instruction measurement.As the visual field changes, The instruction that automatic display corresponds to the additional areas of physics rectangular area provides electronic equipment and has been correctly detecting physics rectangle region The visual feedback in domain.There is provided it is improved feed back enhance equipment operability and make user equipment interface it is more efficient (for example, By helping user to provide correctly input, and reduce the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, show that description corresponds to the mark of the measurement of the physics rectangular area on the expression of the visual field After label, according to the zone of convergency is determined corresponding to entire physics rectangular area, electronic equipment detects (1020) electronic equipment Further mobile, which makes at least one the one or more magazine visual field movement so that visual field table Show including first area corresponding with the first part of physics rectangular area.In some embodiments, gathered according to determining this It closes region and corresponds to entire physics rectangular area, electronic equipment is in the first area for the first part for corresponding to physics rectangular area On show description correspond to the physics rectangular area measurement label (for example, as herein refer to Fig. 5 AF as described in).When (again) measurement for describing the physics rectangular area at the different piece of physics rectangular area is shown when showing these parts Label provides electronic equipment and has correctly detected physics rectangular area and measured correctly all with physics rectangular area The associated visual feedback in part.Improved feedback is provided to enhance the operability of equipment and make user equipment interface higher Effect (for example, by helping user to provide correctly input, and reduce the user occurred when operating equipment/interact with equipment Mistake).

It should be appreciated that the particular order that the operation in Figure 10 A to Figure 10 B is described is only exemplary, not It is intended to indicate that the sequence is the unique order that can execute these operations.Those skilled in the art will recognize that a variety of sides Formula resequences to operations described herein.Moreover, it is noted that herein in relation to its other party as described herein Method (for example, method 600,700,800,900,1100,1200,1300 and 1400) description other processes details equally with Similar mode is suitable for the method 1000 described above with reference to Figure 10 A to Figure 10 B.For example, being described above with reference to method 1000 Input, user interface element is (for example, measurement point, measuring section, virtual mark, physical space or the expression in the visual field, show can table Show, prompt, indicator, label, anchor point and/or place user interface element such as graticule and point), tactile output and intensity threshold Value optionally have herein with reference to other methods as described herein (for example, method 600,700,800,900,1100,1200, 1300 and 1400) description input, user interface element, tactile output and intensity threshold one or more features.For letter For the sake of bright, these details are not repeated herein.

Figure 11 A to Figure 11 B is to show interacting simultaneously with metrical information in augmented reality environment according to some embodiments Manage the flow chart of the method 1100 of metrical information.Method 1100 is including touch-sensitive display (for example, touch screen 112 (Figure 1A) Or combine the display generating unit 304 (Fig. 3 B) of input equipment 302), and one or more cameras are (for example, optical sensor 164 (Figure 1A) or camera 305 (Fig. 3 B)), optionally in the one or more for the intensity for detecting contact with touch-sensitive display Sensor (for example, contact strength sensor 165, Figure 1A), and optional one or more tactile output generators (for example, Tactile output generator 163 (Figure 1A) or tactile output generator 357 (Fig. 3 A)) electronic equipment (for example, portable multi-function Equipment 100 (Figure 1A), equipment 300 (Fig. 3 A) or computer system 301 (Fig. 3 B)) at execute.Some behaviour in method 1100 It is optionally combined and/or the sequence of some operations is optionally changed.

As described below, method 1100 starts and in response to detecting about the touch input indicated is measured for sharing The process of information provides the intuitive manner of the shared information about measurement.Method 1100 is in the shared information about measurement The quantity, degree and/or property of input from the user are reduced, to create more efficient man-machine interface.It is driven for battery Electronic equipment, the information for allowing users to more rapidly and more efficiently shared related measurement saves power and increases electricity Pond charge twice between data break.

Electronic equipment show on the touch sensitive display (1102) application program (for example, augmented reality measurement application program or Including augmented reality measurement function application program) the first user interface.

First user interface include (1104) it is one or more it is magazine at least one the visual field expression.The table in the visual field Show and (example is updated based on the variation by least one the one or more magazine Current vision data detected at any time Such as, the expression in the visual field is RUNTIME VIEW).The visual field includes the physics pair in three-dimensional space (for example, space in physical world) As.The expression of the measurement of physical object is superimposed upon in the expression of the visual field on the image of the physical object.

When showing the first user interface, electronic equipment detects first on (1106) touch-sensitive display on measurement indicates Touch input (for example, tap, double-click or pressing input in shown measurement) (for example, touch input 5182, Fig. 5 BL).

In response to detecting the first touch input on touch-sensitive display on measurement indicates, electronic equipment starts (1108) Process for the shared information about measurement is (for example, be sent to clipbook processing or communications applications for the information about measurement Program (for example, text messaging application program, email application, file transfer application) etc.) (for example, such as Herein with reference to described in Fig. 5 BL to Fig. 5 BM).In some embodiments, this method includes being added to metrical information to clip and paste Plate.In some embodiments, the method includes sending information to the second application program.In some embodiments, it opens Moving the process includes display second user interface, which includes can for the shared user about the information of measurement Option, such as shared table user interface.In some embodiments, second user interface includes the information of description measurement.In In some embodiments, which includes the semantic label automatically generated, and the semantic label is to the physical object for being superimposed with measurement (for example, window, wall, floor or desk) classifies.In some embodiments, which includes the semanteme automatically generated Label, the semantic label to first measurement physical object between relationship (for example, the length of physical object, width, height or Depth) classify.

In some embodiments, the process of the shared information about measurement of starting includes that (1110) are replicated about measurement Information (for example, being handled by the clipbook that the operating system of electronic equipment provides).In some embodiments, in duplication about this After the information of measurement, electronic equipment detection is to by one of the destination in the information stickup to electronic equipment about the measurement A or multiple user's inputs.In some embodiments, in response to detect to will about the information stickup of the measurement to electricity One or more users of destination in sub- equipment input, and the destination of electronic equipment on an electronic device is shown about the survey The information of amount.By enabling duplication and pasting the information of related measurement, can easily in same application and with electricity The shared measurement of other applications in sub- equipment.So that information is can be used for multiple application programs and enhances the operability of equipment simultaneously So that user equipment interface is more efficient (for example, by so that selection becomes to hold for shared or transmission measurement application program Easily).

In some embodiments, starting includes (1112) display packet for the process of the shared information about the measurement Include it is one or more can activated user interface element second user interface (for example, measuring management interface 5184, Fig. 5 BM), In it is the one or more can in activated user interface element it is corresponding can activated user interface element correspond to about the measurement The corresponding destination (for example, icon 5192,5194 and 5196, Fig. 5 BM) of information.In some embodiments, one or more Can each of activated user interface element correspond on electronic equipment corresponding application programs (in addition to the first application program it Outside) or process (for example, instant message application program (for example, icon 5194, Fig. 5 BM), email application (for example, Icon 5192, Fig. 5 BM), notes application program, File Transfer Protocol (for example, icon 5196, Fig. 5 BM) etc.).In some implementations In scheme, electronic equipment detect on touch-sensitive display in second user interface it is corresponding can on activated user interface element the Two touch inputs.In some embodiments, in response to detecting the second touch input, electronic equipment is by the letter about the measurement Breath be sent to correspond to accordingly can activated user interface element corresponding destination.

In some embodiments, sending corresponding destination for the information about measurement includes that will send information to electronics The second application program (for example, notes application program) in equipment.In some embodiments, the information about measurement is sent It include display third user interface to corresponding destination, for checking, editing before it will send information to subsequent destination And/or mark (for example, third user interface includes showing and capable of indicating, when selecting this to show can indicate, is wrapped about the information of measurement Any editor and the information marked are included from corresponding destination (for example, instant message application program or email application) It is sent to subsequent destination (for example, another electronic equipment)).In some embodiments, it sends the information about measurement to Corresponding destination includes that will send information to second via the File Transfer Protocol between electronic equipment and the second electronic equipment Electronic equipment.

For multiple destinations provide with can activated user interface element user interface (for example, band purposefully icon Shared table) for shared measurement so that being become easy with the shared measurement in these destinations.Multiple shared destination options are provided It enhances the operability of equipment and makes user equipment interface more efficient (for example, passing through so that selection is for sharing or sending The application program of measurement becomes easy).

In some embodiments, in response to detecting that the first touch on touch-sensitive display in the expression of measurement is defeated Enter, electronic equipment shows the information of (1114) about the measurement.It in some embodiments, include measurement about the information of measurement Size (for example, as shown in measuring management interface 5184, Fig. 5 BM).In some embodiments, about the information of measurement Including the semantic label classified to the relationship between measurement and physical object (for example, the length of physical object, width, height Degree or depth) (for example, as described in label 5186-b in reference measure administration interface 5184 herein, Fig. 5 BM).One In a little embodiments, the information about measurement includes that the relationship between the anchor feature of the identification to measurement and three-dimensional space is divided The label (for example, text label) of class is (for example, whether the region in the expression of the visual field corresponds to physics rectangle region parallel to the ground Domain).In some embodiments, in measurement corresponding to physics rectangular area and including linear measure longimetry, width measurement and area In the case where measurement, display length, width and area, and length and width is more highlightedly shown than the region.In addition to starting Except the process of shared information, in response to detecting that the touch input on measurement indicates provides the information about measurement, User's visual feedback of selected correctly shared measurement is provided, and user is allowed to check and use the information.It provides improved Feedback enhances the operability of equipment and makes user equipment interface more efficient (for example, correct by helping user to provide Input, and reduce the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, (1116) are executed about this according to the first touch input the first standard of satisfaction is determined The display of the information of measurement, wherein the first standard includes that the intensity of the contact in the first touch input is required to meet (for example, reaching Or be more than) corresponding intensity threshold is to meet the first standard (for example, as described in referring to Fig. 5 BL to Fig. 5 BM herein).One In a little embodiments, corresponding intensity threshold is above contact detection intensity threshold value IT₀Light press intensity threshold IT_L.In response to It detects the touch input that the measurement for meeting strength criterion indicates and the accident for reducing information about the information of measurement is provided , undesirable display.Operation is executed when meeting strength criterion to enhance the operability of equipment and make user's equipment interface More efficient (for example, appropriate input by helping user to provide when operating equipment/interact with equipment and reduce user's mistake).

In some embodiments, electronic equipment determines the classification of (1118) physical object (for example, physical object is classified For corresponding structure feature (window, wall, floor etc.) or as corresponding furniture or fixture (such as desk)).Some In embodiment, the information about measurement includes the label of the classification of indicant reason object (for example, such as reference measure pipe herein It manages described in the label 5186-a in interface 5184, Fig. 5 BM).In some embodiments, electronic equipment is based on one or more The image procossing (for example, being identified using feature) that the visual field of camera indicates classifies to physical object.Automatically to physical object Classification and marking is carried out, is inputted without further user, reduction (or elimination) the manual classification and marking physics pair of user is passed through The needs of elephant enhance the operability of equipment and keep user's equipment interface more efficient.

In some embodiments, it is based at least partially on the movement of electronic equipment during measurement in a first direction, The expression of measurement has been added to the user interface of (1120) application program.In some embodiments, electronic equipment at least portion Point ground determines the relationship between measurement and physical object based on the movement in a first direction of electronic equipment during measurement Classification (for example, the classification of measurement corresponds to length, width, height or the depth of physical object) is (for example, such as herein with reference to figure Described in 5BM).

It in some embodiments, include the classification for the relationship that instruction measures between physical object about the information of measurement Label.For example, measurement is classified as physics based on the movement of electronic equipment during measurement vertically by electronic equipment Measurement is classified as the width of physical object by the height of object, or movement based on electronic equipment during measurement in the horizontal direction Degree.In some embodiments, electronic equipment is based further on the image procossing of visual field expression between measurement and physical object Relationship classify, to determine the respective distance between the respective point on electronic equipment and physical object, the respective point pair Ying Yu along measurement respective point.For example, electronic equipment is based further on the physics pair for determining the first end point for corresponding to measurement As upper first point than on electronic equipment correspond to measurement the second endpoint second point further from (or closer) (for example, Measurement is classified as the depth of physical object by electronic equipment in a z-direction).

It is based in part on the movement of equipment during measurement to input without further user, measurement is divided automatically Class and label enhance the operability of equipment, and the need by reducing the manual classification and marking measurement of (or elimination) user It wants, keeps user's equipment interface more efficient.

It should be appreciated that the concrete order that the operation in Figure 11 A- Figure 11 B is described is only example, it is not intended that table The bright order is the but one sequence that can execute these operations.Those skilled in the art will recognize that various ways come pair Operations described herein is resequenced.Moreover, it is noted that herein in relation to other methods (example as described herein Such as, method 600,700,800,900,1000,1200,1300 is with the details of 1400) other processes of description equally with similar Mode is suitable for the method 1100 described above with reference to Figure 11 A to Figure 11 B.For example, above with reference to method 1100 describe input, User interface element (for example, measurement point, measuring section, virtual mark, physical space or the expression in the visual field, show can indicate, prompt, Indicator, label, anchor point and/or place user interface element such as graticule and point), tactile output and intensity threshold it is optional Ground has herein with reference to other methods as described herein (for example, 600,700,800,900,1000,1200,1300 and of method 1400) one or more features of the input of description, user interface element, tactile output and intensity threshold.For brevity, These details are not repeated herein.

Figure 12 A to figure 12 C is to show to provide pair automatically determined in augmented reality environment according to some embodiments The flow chart of the method 1200 of quasi- guidance.Method 1200 is including display (for example, touch screen 112 (Figure 1A), display 340 (Fig. 3 A) or display generating unit 304 (Fig. 3 B)), input equipment is (for example, touch screen 112 (Figure 1A), touch tablet 355 (Fig. 3 A) Or input equipment 302 (Fig. 3 B), or the physical button separated with display), and one or more cameras are (for example, optics passes Sensor 164 (Figure 1A) or camera 305 (Fig. 3 B)), optionally in detection and the intensity of the contact of the touch sensitive surface of input equipment One or more sensors (for example, contact strength sensor 165, Figure 1A), and optional one or more tactile output Electronic equipment (the example of generator (for example, tactile output generator 163 (Figure 1A) or tactile output generator 357 (Fig. 3 A)) Such as, portable multifunction device 100 (Figure 1A), equipment 300 (Fig. 3 A) or computer system 301 (Fig. 3 B)) at execute.Method Some operations in 1200 are optionally combined and/or the sequence of some operations is optionally changed.

As described below, method 1200 is provided provides the intuitive side of (virtual) guidance in the measurement based on augmented reality Formula, wherein guiding the moving direction extension along camera fields of view.There is provided measurement guide facilitate user be quickly and accurately positioned and Place (virtual) measurement point.Guidance is automatically provided by the moving direction along camera fields of view, method 1200 is when measuring The quantity, degree and/or property of input from the user are reduced, to create more efficient man-machine interface.It is driven for battery Electronic equipment, allow users to quickly measure and more effectively save power and increase two primary cells to charge it Between interval.

Electronic equipment show over the display (1202) application program (for example, augmented reality measurement application program or including Augmented reality measurement function application program) user interface.User interface (1204) includes that one or more is magazine extremely The expression in few one visual field；The expression in the visual field based on by it is the one or more it is magazine at least one detect work as forward sight Feel that the variation of data updates (for example, the expression in the visual field is RUNTIME VIEW) at any time；And the visual field include three-dimensional space at least A part (for example, including the space of physical object in physical world).

Electronic equipment detects the movement of (1206) electronic equipment, and the electronic equipment is along first direction (for example, horizontal or perpendicular Directly) at least one the mobile one or more magazine visual field is (alternatively, in some embodiments, substantially in first party Upwards (for example, the direction in predetermined threshold angle in a first direction, such as in a first direction 10,15,20 or 25 degree It is interior)).

When detecting movement (1208) for moving the visual field in a first direction of electronic equipment, electronic equipment is according to electronics The movement of equipment updates the expression in (1210) visual field；One or more extended in a first direction in the expression of identification (1212) visual field A first element (or feature structure) (for example, edge detected, plane detected etc.)；Also, it is based at least partially on The determination of one or more first elements, display (1214) extends in a first direction and corresponds to and known in the expression of the visual field One the first guidance in other one or more elements as reference herein is virtual (for example, guide described in 5050, figure 5M).In some embodiments, electronic equipment shows multiple guidance (for example, as herein with reference to virtual in a first direction Described in guidance 5106, Fig. 5 AB).In some embodiments, each of multiple guidance, which correspond to, extends in a first direction Corresponding recognition component (for example, respective edges of the respective physical object in the three-dimensional space extended in a first direction).Some In embodiment, electronic equipment detect electronic equipment it is mobile while show one or more guide in a first direction.

In some embodiments, the visual field includes (1216) multiple element, and the multiple element is included in except first One or more elements that side except direction upwardly extends are (for example, perpendicular to (for example, at vertical predefined thresholds angle In degree) or it is substantially perpendicular to the direction of first direction, or it is greater than the direction of predefined thresholds angle away from first direction, such as It is greater than 10 degree, 15 degree, 20 degree or 25 degree away from first direction).In some embodiments, make the visual field in detection electronic equipment Move in a first direction it is mobile when, electronic equipment abandons being shown in the guidance that the side in addition to first direction upwardly extends (for example, as described in referring to Fig. 5 AN to Fig. 5 AO and Fig. 5 AV herein).For example, moving direction of the electronic equipment based on camera is true The axis for determining physical object (for example, desk or wall) extends along the axis and guides.In some embodiments, electronics is set The standby one or more guidance of display, one or more of guidance correspond to one or more elements in the visual field, one Or it is multiple guidance extend on the moving direction of at least one camera, do not appear in addition at least one camera the direction of motion it The guidance that outer side upwardly extends.One or more guidance are provided along the moving direction in the visual field, without providing in other directions The guidance of upper extension, avoiding showing is less likely guidance relevant to ongoing measurement.Reduce the confusion in user interface Enhance the operability of equipment and make user equipment interface it is more efficient (for example, by help user provide correctly input, And reduce the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, before detecting the movement for moving the visual field along first direction of electronic equipment (1218), the first touch input on electronic equipment detection touch-sensitive display, and in response to detecting the first touch input, In Corresponding to addition at the first position in the visual field expression of the first position in three-dimensional space and show first (virtual) measurement point (for example, measurement point 5054, Fig. 5 R).The first guidance of display also corresponds to three-dimensional space based on determining one or more first elements In first position (for example, first position is the point along detected edge).In addition, the first guidance includes in the expression of the visual field First position (for example, the guidance is Chong Die with measurement point, or along first direction extend from measurement point) (for example, as herein With reference to described in virtual guidance 5070, Fig. 5 S).Offer extends from the first measurement point along the moving direction in the visual field and/or including the The guidance of one measurement point facilitates user and places the second measurement point, to measure.The visual feedback is provided and enhances equipment Operability and make user equipment interface is more efficient (for example, by helping user to provide correctly measurement input, and to subtract Few user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, the first measurement point at the first position in the expression of the visual field is (1220) visual field table Show the measurement point of middle most recent addition.There is provided measurement point extension add from most recent along the moving direction in the visual field and/or Person includes the guidance of the measurement point of most recent addition, and user is helped to place next measurement point, to add in two most recents Measurement point between measure.The visual feedback is provided to enhance the operability of equipment and make user equipment interface higher Effect (for example, by helping user to provide correctly measurement input, and reduces and occurs when operating equipment/interact with equipment User's mistake).

In some embodiments, (the example after the movement for moving the visual field in a first direction for detecting electronic equipment Such as, horizontally, as described in referring to Fig. 5 S herein), electronic equipment (1222) detects the electronics for moving the visual field in a second direction The movement of equipment (for example, vertically, as described in referring to Fig. 5 T herein).In response to detect electronic equipment in second direction The movement of dynamic visual field is moved up, electronic equipment stops the first guidance that display extends in a first direction.In some embodiments, electric Sub- equipment stops in any direction in addition to second direction or shows any draw in the predetermined threshold angle of second direction Lead (for example, as described in referring to Fig. 5 T herein).Detect electronic equipment move in a second direction the visual field it is mobile when, electricity Sub- equipment according to the expression in the mobile update visual field of electronic equipment, identify one in the visual field expression extended in a second direction or Multiple second elements, and be based at least partially on the determination of one or more second elements, display is along the in the expression of the visual field Two directions extend and corresponding to the second guidance of one of the one or more second elements identified (for example, virtual guidance 5072, Fig. 5 T).With the change of visual field moving direction, it is automatic change guidance display more likely with ongoing measurement phase The guidance (and stopping showing unlikely guidance relevant to ongoing measurement) of pass.Reduce the confusion in user interface Enhance the operability of equipment and make user equipment interface it is more efficient (for example, by help user provide correctly input, And reduce the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, corresponding identified at least one of the first element of one or more of the first guidance It is the element (for example, as described in referring to Fig. 5 S herein) at the edge that (1224) are identified as physical object in three dimensions. Along the physical object run along first direction edge automatically show guidance facilitate user place measurement point and along edge into Row measurement.Guidance is automatically provided along target edges, is inputted without further user, enhances the operability of equipment, and Keep user's equipment interface more efficient (for example, by helping user to provide measurement input appropriate and reducing operation equipment/and equipment User's mistake when interaction).

In some embodiments, corresponding identified at least one of the first element of one or more of the first guidance It is that (1226) are identified as the element of plane (for example, corresponding to the table of the physical object in three-dimensional space in three dimensions Face).Show that guidance facilitates user and places measurement point and survey along plane automatically along the plane extended in a first direction Amount.Guidance is automatically provided along plane, is inputted without further user, enhances the operability of equipment, and make user Equipment interface is more efficient (for example, by helping user to provide measurement appropriate input and reducing operation equipment/interact with equipment When user's mistake).

In some embodiments, user interface includes the measurement point creation that (1228) are shown on the visual field expression Indicator.In some embodiments, corresponding position of the electronic equipment in the expression of the visual field shows corresponding measurement point, the phase Position is answered to correspond to the corresponding position in three-dimensional space.In some embodiments, electronic equipment detection electronic equipment makes to survey The movement that amount point creation indicator moves on the corresponding measurement point in the expression of the visual field, and in response to detecting electronic equipment Movement, for measurement point creation indicator when being shown on corresponding measurement point, electronic equipment shows multiple guidance.Multiple guidance In first guidance perpendicular in multiple guidance second guidance, and it is multiple guidance corresponding measurement point intersect.Herein The display of vertical guide is described with reference to Fig. 5 AT.

In some embodiments, be based at least partially on determining respective guide indicates along the visual field extended from measurement point In element extend to show the respective guide in multiple guidance.For example, if measurement point corresponds to physics pair in three-dimensional space The angle of elephant, one or more of guidance include the edge along the physical object extended in a first direction (for example, corresponding to object Manage the edge of (level) length of object) guidance that extends from the corner of physical object along first direction (for example, along x-axis), such as Shown in the expression of the visual field.In some embodiments, one or more guidance include along the physics extended in a second direction The guidance that the edge of object extends in a second direction (for example, along y-axis) from the corner of physical object is (for example, correspond to physics pair (vertical) height of elephant), as shown in the expression of the visual field.In some embodiments, one or more guidance include along edge The edge (for example, the edge for corresponding to the depth of physical object) for the physical object that third direction extends is from the corner of physical object The guidance extended along third direction (for example, along z-axis), as shown in the expression of the visual field.

It is shown on measurement point and shows multiple vertical guides automatically in measurement point when measurement point creates indicator, helped User places other measurement point.Vertical guide is automatically provided in measurement point, inputs, enhances without further user The operability of equipment, and keep user's equipment interface more efficient (for example, by helping user to provide measurement input appropriate simultaneously Reduce user's mistake when operation equipment/interact with equipment).

In some embodiments, user interface includes that (1230) measurement point creates indicator, in the expression of the visual field It is shown on the second position.In some embodiments, in the first guidance that display extends in a first direction, electronic equipment detection The second touch input on touch-sensitive display.In response to detecting the second touch input, and according to determining the second position and the The distance between one guidance (for example, shortest distance) within the threshold range, electronic equipment should away from the second position in the first guidance It is added at the position of distance and shows second (virtual) measurement point.For example, Fig. 5 X is shown at the point in virtual guidance 5072 Add measurement point 5090.In response to detecting the second touch input, and according between the determining second position and the first guidance Not within the threshold range, electronic equipment adds in the second position and shows the second measurement point (for example, measurement point 5080 does not exist distance The within the threshold range of virtual guidance 5072, therefore measurement point 5080 is not added to the point in virtual guidance 5072).According to measurement Point creation the distance between indicator and the position of guidance add measurement point (for example, measurement point is captured in guidance automatically Some position in guidance) or measurement point is added to except guidance, help user rapidly and accurately to place measurement point.At one group It executes operation when condition (for example, distance condition) obtains meeting to input without further user, which enhance equipment Operability, and make user equipment interface more efficient (for example, inputting simultaneously by helping user to provide measurement appropriate And user's mistake is reduced when operate/interact with equipment).

It should be appreciated that the particular order that the operation in figure 12 A to figure 12 C is described is only exemplary, not It is intended to indicate that the sequence is the unique order that can execute these operations.Those skilled in the art will recognize that a variety of sides Formula resequences to operations described herein.Moreover, it is noted that herein in relation to its other party as described herein Method (for example, method 600,700,800,900,1000,1100,1300 and 1400) description other processes details equally with Similar mode is suitable for the method 1200 described above with reference to Figure 12 A- Figure 12 C.For example, described above with reference to method 1200 Input, user interface element (for example, measurement point, measuring section, virtual mark, physical space or the expression in the visual field, show can indicate, Prompt, indicator, label, anchor point and/or place user interface element such as graticule and point), tactile output and intensity threshold Optionally have herein with reference to other methods as described herein (for example, method 600,700,800,900,1000,1100,1300 With 1400) description input, user interface element, tactile output and intensity threshold one or more features.It is risen in order to concise See, these details are not repeated herein.

Figure 13 A to Figure 13 C is to show to remove the method virtually marked previously added automatically according to some embodiments 1300 flow chart.Method 1300 is including one or more input equipments (for example, touch screen 112 (Figure 1A), touch tablet 355 (Fig. 3 A) or input equipment 302 (Fig. 3 B), or the physical button separated with display), one or more display equipment (for example, Touch screen 112 (Figure 1A), display 340 (Fig. 3 A) or display generating unit 304 (Fig. 3 B)) and one or more camera (examples Such as, optical sensor 164 (Figure 1A) or camera 305 (Fig. 3 B)), it is strong with the contact of the touch sensitive surface of input equipment for detecting The optional one or more sensors (for example, contact strength sensor 165, Figure 1A) of degree, and it is optional one or more The electronics of tactile output generator (for example, tactile output generator 163 (Figure 1A) or tactile output generator 357 (Fig. 3 A)) It is held at equipment (for example, portable multifunction device 100 (Figure 1A), equipment 300 (Fig. 3 A) or computer system 301 (Fig. 3 B)) Row.Some operations in method 1300 are optionally combined and/or the sequence of some operations is optionally changed.

As described below, method 1300 provide a kind of intuitive way be automatically deleted with current virtual mark (such as when The preceding measurement based on augmented reality) not connected or relevant former virtual mark (such as previously based on the measurement of augmented reality). Be automatically deleted previous incoherent virtual mark can prevent augmented reality user interface become in the new virtual mark of creation with Early stage, virtual mark was chaotic.As described herein, method 1300 allows to be easy to carry out individually virtual mark without hand It is dynamic to delete previous virtual mark, and when new virtual mark to be added in series, it can also easily obtain a system Relevant virtual mark (for example, virtual mark of connection) is arranged virtually to mark without deleting the early stage in series.This method into The quantity, degree and/or property for reducing input from the user when the capable virtual mark based on augmented reality, to create More efficient man-machine interface.For battery powered electronic equipment, allow users to faster more efficiently carry out existing based on enhancing Real virtual mark, thus save electric power and increase battery charge twice between time.

In particular, when virtual mark is the measurement based on augmented reality, it is automatically deleted previous incoherent be based on The measurement of augmented reality can prevent measurement user interface from becoming to measure confusion with early stage when newly being measured.These are automatically deleted The measurement to the Precision Mapping of physical space may no longer be had by also removing electronic equipment.As described herein, method 1300 makes Independent measurement can be easy to carry out without deleting previous measurement manually by obtaining, and carry out a series of phases with also making it easy to Measurement (for example, measurement of connection) being closed, being measured early stage deletion in series when without adding new measurement in series.This method In the quantity, degree and/or property for carrying out reducing input from the user when the measurement based on augmented reality, to create More efficient man-machine interface.For battery powered electronic equipment, allow users to faster more efficiently carry out existing based on enhancing Real measurement, thus save electric power and increase battery charge twice between time.

Electronic equipment shows that (1302) include the user interface of the expression of physical space via one or more display equipment (for example, live preview of a part of the physical space at least one the one or more magazine visuals field).For example, User interface 5006 in Fig. 5 BS includes the expression of physical space 5000.

When showing the expression of physical space, electronic equipment receives (1304) first groups of one or more inputs in physics Virtual mark (for example, shape, line, rectangle, measurement etc.) is created in the expression in space (for example, for putting the defeated of next point Enter, be followed by movement of the electronic equipment relative to physical space, followed by another input for putting next point).

In response to receiving first group of one or more input, electronic equipment arrives the first virtual mark addition (1306) The expression of physical space.First virtual mark is linked to a part of the expression of physical space.For example, such as Fig. 5 BS to Fig. 5 BU institute Show, creation includes the virtual mark of measuring section 5212 and its associated endpoint 5206 and 5216 in user interface 5006, and It is linked to the expression of (physics) object 5202.For another example, it as shown in Fig. 5 BV to Fig. 5 BY, is created in user interface 5006 Virtual mark including measuring section 5222 and its associated endpoint 5220 and 5224, and it is linked to the table of object 5202 Show.

In some embodiments, will virtual mark be added to physical space expression include the virtual mark of creation and by its It is linked to the position in the expression of physical space, is fixed or substantially solid so that being virtually labeled in the expression of physical space It is fixed.In some embodiments, it virtually marks in the expression of " " physical space or the expression of " being added to " physical space is practical On be added in the model of physical space, and the expressed portion that ought virtually mark linked physical space separates present physics It when in the camera image in space, is plotted on the camera image of physical space, to be virtually labeled in physical space to people is a kind of In impression.In some embodiments, virtually " removal " is actually from physics sky in the expression of mark " from " physical space Between model in delete, once and it from the expression " removal " of physical space, when the linked physical space of virtual mark Expressed portion when separating in the camera image of present physical space, it is no longer plotted on the camera image of physical space, With the impression to a kind of virtual mark of people no longer in physical space.

After the first virtual mark is added to the expression of physical space, electronic equipment receives (1310) and physical space Expression associated second group of one or more input.

In response to receiving the associated second group of one or more input (1312) of expression with physical space, according to It determines that second group of one or more input corresponds to be created in the expression of physical space away from the first virtual mark within the threshold range The request (1314) virtually marked, electronic equipment creates the second virtual mark (for example, being linked in the expression of physical space The second part of the expression of physical space), while keeping first virtually to mark in the expression of physical space.In some embodiment party In case, while showing that first virtually marks and the second virtual mark.In some embodiments, threshold distance is zero (for example, Two virtually mark and must connect with the first virtual mark, to keep first virtually to mark in the virtual mark of creation second).In In some embodiments, threshold distance is greater than zero (for example, if second is virtually labeled in the preset distance virtually marked away from first It is interior, then the first virtual mark is kept when creating the second virtually mark, even if the second virtual mark is not attached to or touches first Virtual mark).For example, as shown in Fig. 5 BZ to Fig. 5 CF, because measuring section 5232 is in the threshold away from the measuring section 5222 previously added In value distance, when creating measuring section 5232/later, it is maintained first in the expression of the physical space 5000 in user interface 5006 The measuring section 5222 of preceding addition.In another example, as shown in Fig. 5 CG to Fig. 5 CK, because measuring section 5242 was away from previously adding The within the threshold range of the measuring section 5232 and 5222 added, when creating measuring section 5242/later, in user interface 5006 The measuring section 5232 and 5222 previously added is maintained in the expression of physical space 5000.

In addition, in response to receiving the associated second group of one or more input (1312) of expression with physical space, According to determine the input of second group of one or more correspond in the expression of physical space creation away from the first virtual mark threshold value away from The request (1316) virtually marked from except, electronic equipment created in the expression of physical space the second virtual mark (for example, It is linked to the second part of the expression of physical space), and remove first from the expression of physical space and virtually mark.Some In embodiment, the first virtual mark is removed in the case where no explicit request removes the first virtually mark (for example, equipment is not User's input is detected, is separated with for creating second group of input that second virtually marks, the first virtual mark is deleted in request).Example Such as, as shown in Fig. 5 BV to Fig. 5 BY, because measuring section 5222 is except the threshold distance away from the measuring section 5212 previously added, institute With when adding measuring section 5222/remove the measuring section 5212 previously added later.

In some embodiments, in physical space while holding first virtually marks in the expression of physical space In expression creation second virtually mark include (1318) correspond to the first at least part of position virtually marked at start Creation second virtually marks (for example, if the creation of the measuring section 5232 in Fig. 5 CA to Fig. 5 CF is instead to start from Fig. 5 CF In measurement point 5234 placement and terminated with the placement of the measurement point 5228 in Fig. 5 CA).When on virtual mark previous Start to automatically keep previous virtual mark when successor virtual mark so that electronic equipment be able to carry out it is a series of it is relevant virtually It marks (for example, a series of measurements being connected), when new mark is added in series, does not delete the early stage mark in series Note (for example, measurement).When one group of condition obtains meeting (for example, upper beginning successor virtual mark ought be marked virtually previous When) operation is executed without further user input, which enhance the operability of equipment, and make user equipment Interface is more efficient (for example, being used by helping user to provide input appropriate and reduce when operate/interact with equipment Family mistake).

In some embodiments, creation second virtually marks the table in physical space simultaneously in the expression of physical space Show that virtually mark includes that (1320) are corresponding to completion wound at the first at least part of position virtually marked for middle holding first Build the second virtual mark.For example, the creation of measuring section 5232 is at the end for corresponding to measuring section 5222 as shown in Fig. 5 BZ to Fig. 5 CF It is completed at the position of point, and therefore measuring section 5222 is maintained in the expression of physical space 5000.It is marked when virtual previous Terminate to automatically keep when successor virtual mark on note it is previous it is virtual mark so that electronic equipment be able to carry out it is a series of relevant A series of virtual mark (for example, measurements being connected), when new mark is added in series, does not delete the morning in series Phase mark (for example, measurement).When one group of condition obtains meeting (for example, terminating successor virtual on ought virtually marking previous When mark) operation is executed without further user input, which enhance the operability of equipment, and make user Equipment interface is more efficient (for example, by helping user to provide input appropriate and subtracting when carrying out with equipment and operating/interact Few user's mistake).

In some embodiments, defeated in response to receiving second group of one or more associated with the expression of physical space Enter (1322), according to determining that the input of second group of one or more corresponds at least one the one or more magazine visuals field Displacement be more than threshold quantity request (for example, by it is one or more it is magazine at least one visual field offset it is a certain amount of, which leads Causing the first fidelity for virtually marking the tracking linked between the expression of physical space to reduce is more than threshold quantity), electronics is set It is standby that the first virtual mark is removed from the expression of physical space.In some embodiments, first is removed in no explicit request First is removed in the case where virtual mark, and virtually (for example, electronic equipment does not detect user's input, it is virtual that request deletes first to mark Mark).The example virtually marked according to the mobile removal of camera more than threshold quantity is described with reference to Fig. 5 CL to Fig. 5 CM herein.When Automatically previous virtual mark (for example, measurement) is removed when the visual field offset of successor virtual mark is more than threshold quantity, reduces increasing Confusion in strong real user interface, and avoid tracking and show the guarantor for keeping off the former virtual mark newly virtually marked The potential problems really spent.It executes operation when one group of condition obtains meeting to input without further user, this enhancing The operability of equipment, and make user equipment interface more efficient (for example, appropriate defeated by helping user to provide Enter and reduce user's mistake when operate/interact with equipment).

In some embodiments, defeated in response to receiving second group of one or more associated with the expression of physical space Enter (1324), makes at least one the one or more magazine visuals field according to determining that the input of second group of one or more corresponds to Offset is so that first virtually marks the no longer visible request more than threshold amount of time (for example, one or more is magazine extremely The visual field offset of few one seems that the part of the physical space of link is more than threshold amount of time far from the first virtual mark), electricity Sub- equipment removes the first virtual mark from the expression of physical space.In some embodiments, it is removed in no explicit request First is removed in the case where first virtual mark, and virtually (for example, equipment does not detect user's input, it is virtual that request deletes first to mark Mark).It describes herein with reference to Fig. 5 CL to Fig. 5 CM according to the movement of camera and removes virtual mark so that virtual mark not The visible example more than threshold amount of time again.Elder generation is removed automatically when the visual field offset of successor virtual mark is more than threshold amount of time Preceding virtual mark (for example, measurement), reduces the confusion in augmented reality user interface, and avoid tracking and display not Close to the potential problems of the fidelity of the former virtual mark newly virtually marked.Operation is executed when one group of condition obtains meeting It is inputted without further user, which enhance the operability of equipment, and make user equipment interface more efficient (for example, by helping user to provide input appropriate and reducing user's mistake when operate/interact with equipment).

In some embodiments, it is in when receiving second group of one or more input and in the first virtual mark When in the expression of physical space, further input will lead to the first virtual mark from physical space for electronic equipment output (1326) The instruction being removed in expression.In some embodiments, which includes figure instruction, audio instruction and/or tactile instruction. For example, equipment 100 changes the appearance of measuring section 5212 to indicate further to input and will make to measure as shown in Fig. 5 BW to Fig. 5 BX Section 5212 is removed from the expression of the physical space 5000 in user interface 5006.In another example, such as Fig. 5 CA to figure Shown in 5CB, equipment 100 changes the appearance of measuring section 5222 to indicate further to input and measuring section 5222 will be made from user interface It is removed in the expression of physical space 5000 in 5006.Former virtual mark (for example, measurement) will be deleted by providing further input Vision, audio and/or touch feedback provide change input chance so that be unlikely to unintentionally remove former virtual mark. Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, by helping user Correctly input is provided, and reduces the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, which is the vision that (1328) are shown in the predetermined part of user interface Instruction is used for display reminding (for example, the specified prompting region in user interface, such as user interface of 5010 top of graticule Region (for example, region that error message 5248 is shown in Fig. 5 CM) in 5006).It is provided in the specified region of user interface Visual feedback for display reminding increases a possibility that user will be seen that and understand feedback/prompt.Improved feedback is provided Enhance the operability of equipment and make user equipment interface it is more efficient (for example, by help user provide correctly input, And reduce the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, which is the change of the first appearance virtually marked in the expression of (1330) physical space Change (for example, changing line color, line style, line weight, filling, opacity etc.).For example, such as Fig. 5 BW to Fig. 5 BX institute Show, changes the appearance (for example, line color and line style) of measuring section 5212.In some embodiments, when placement When graticule is far from the first virtual mark, the first cosmetic variation amplitude virtually marked increases.It is virtually marked by change first Appearance increases user and will be seen that and understand if identical input continues will to remove first providing visual feedback and virtually marks Possibility.Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, passing through side It helps user to provide correctly input, and reduces the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, the first virtual mark is the expression of (1332) measurement comprising the description (example of measurement Such as, described by measuring the text of the distance or area that measure), and the first cosmetic variation virtually marked includes from survey Description is removed in the expression of amount, while keeping at least part of the expression of measurement.For example, as shown in Fig. 5 BV, when graticule 5010 When mobile far from measuring section 5212, label associated with measuring section 5212 is removed.Keep indicate measurement lines or other Visual feedback is provided by removing label or other descriptions from measurement while shape, user is increased and sees and understand such as The identical input of fruit continues a possibility that removing measurement.Improved feedback is provided to enhance the operability of equipment and make user Equipment interface is more efficient (for example, by helping user to provide correctly input, and to be reduced and interacts in operation equipment/with equipment When user's mistake for occurring).

In some embodiments, after the cosmetic variation that display first virtually marks in the expression of physical space (1334), electronic equipment detection (for example, when creating the second virtually mark, will be used for away from the threshold distance that first virtually marks Determine whether to remove the first same threshold distance for virtually marking from display) in position from input (for example, graticule is hanged It is parked on or near the first virtual mark and is placed for point), and the input at the position in response to detecting within the threshold range It forms first virtually to mark, the cosmetic variation that reversion first virtually marks is (for example, in order to indicate if the second virtual mark includes Part at the position of the within the threshold range virtually marked away from first then will not remove first when creating the second virtually mark Virtual mark).For example, the measuring section 5222 when focus 5012 captures the point on measuring section 5222, in Fig. 5 CA to Fig. 5 CB Cosmetic variation inverted in Fig. 5 CC.After providing visual feedback to indicate to remove former virtual mark, and in user By make input closer to or former virtual mark on change its input after, provide feedback to indicate that previous void will be kept Quasi- mark facilitates consumer-oriented input.Improved feedback is provided to enhance the operability of equipment and make user equipment circle Face is more efficient (for example, by helping user to provide correctly input, and to reduce and occurs when operating equipment/interact with equipment User's mistake).

It should be appreciated that the particular order that the operation in Figure 13 A to Figure 13 C is described is only exemplary, not It is intended to indicate that the sequence is the unique order that can execute these operations.Those skilled in the art will recognize that a variety of sides Formula resequences to operations described herein.Moreover, it is noted that herein in relation to its other party as described herein Method (for example, method 600,700,800,900,1000,1100,1200 and 1400) description other processes details equally with Similar mode is suitable for the method 1300 described above with reference to Figure 13 A to Figure 13 C.For example, being described above with reference to method 1300 Input, user interface element is (for example, measurement point, measuring section, virtual mark, physical space or the expression in the visual field, show can table Show, prompt, indicator, label, anchor point and/or place user interface element such as graticule and point), tactile output and intensity threshold Value optionally have herein with reference to other methods as described herein (for example, method 600,700,800,900,1000,1100, 1200 and 1400) description input, user interface element, tactile output and intensity threshold one or more features.For letter For the sake of bright, these details are not repeated herein.

Figure 14 A to Figure 14 D be show according to the objects being used to indicate in physical space of some embodiments whether by It is identified as that the flow chart of the method 1400 of its corresponding object indicated in augmented reality environment can be tracked.Method 1400 It is including one or more input equipments (for example, touch screen 112 (Figure 1A), touch tablet 355 (Fig. 3 A) or input equipment 302 (Fig. 3 B), or the physical button separated with display), one or more display equipment is (for example, touch screen 112 (Figure 1A), display Device 340 (Fig. 3 A) or display generating unit 304 (Fig. 3 B)) and one or more camera (for example, (figure of optical sensor 164 1A) or camera 305 (Fig. 3 B)), for detect with the optional one or more of the contact strength of the touch sensitive surface of input equipment Sensor (for example, contact strength sensor 165, Figure 1A), and optional one or more tactile output generators (for example, Tactile output generator 163 (Figure 1A) or tactile output generator 357 (Fig. 3 A)) electronic equipment (for example, portable multi-function Equipment 100 (Figure 1A), equipment 300 (Fig. 3 A) or computer system 301 (Fig. 3 B)) at execute.Some operations in method 1400 It is optionally combined and/or the sequence of some operations is optionally changed.

As described below, method 1400 will virtually mark the same of (for example, virtual measurement) placement in augmented reality environment When visual feedback is provided.Improved feedback is provided to enhance the operability of equipment and make the more efficient (example in user equipment interface Such as, by helping user to provide correctly input, and the user's mistake occurred when operating equipment/interact with equipment is reduced).

Electronic equipment shows that (1402) mark places user interface via one or more display equipment.Mark places user Interface includes (1404): the expression of physical space is (for example, the physics at least one the one or more magazine visuals field The live preview of a part in space)；And user interface element (for example, placing indicator) is placed, in response to detecting Mark place input, instruction by virtually mark be placed in the expression of physical space position (for example, tap set-point button or Tap places user interface element).For example, the display of equipment 100 includes the use of the expression of physical space 5000 as shown in Fig. 5 CN The placement user interface element and focus 5012 of 5010 form of family interface 5006 and graticule, focus instruction is in response to detection The virtual mark of input is placed by the position in the expression for being placed on physical space 5000 (for example, tap measurement addition to mark Button 5014, or in some embodiments, tap graticule 5010 and/or focus 5012).

When display marks and places user interface, electronic equipment detect (1406) it is one or more it is magazine at least one Movement relative to physical space.Correspond to physical space in the expression that user interface element is shown in physical space when placing When at the position of first part, at least one one or more magazine mobile beginnings (1408).In some embodiments In, the movement include transverse shifting (for example, upwards, downwards, to the left, move right), rotation (for example, to the right, to the left, upwards, It is rotated down) or one of be moved forward or rearward or a variety of.

In response to detecting at least one one or more magazine movement relative to physical space, electronic equipment will Position of the user interface element mobile (1410) into the expression of physical space corresponding to the second part of physical space is placed, it should Second part is different from the first part of physical space, and according to it is one or more it is magazine at least one relative to physics The movement in space updates the appearance that mark places user interface, comprising: can not identify physical space according to determining electronic equipment Object in second part, corresponding objects of the object in the expression of physical space can be linked to virtual mark, stop (1412) at least part of user interface element is placed in display；And according to determining that equipment identified the of physical space Object in two parts, corresponding objects of the object in the expression of physical space can be linked to virtual mark, keep (1414) display of user interface element is placed.For example, having identified physical space according to determining equipment 100 as shown in Fig. 5 CN Object (desk 5200) in 5000 shows graticule 5010 and focus 5012, so that measurement can be added to user interface 5006 In desk 5200 expression in.In Fig. 5 CO, it is such right to be identified in physical space 5000 according to determining equipment 100 As not showing graticule 5010 and focus 5012.Offer electronic equipment can not identify the object in physical space and the object is in object Virtual mark can be linked to (for example, placing user interface element by stopping display by managing the corresponding objects in the expression in space At least part) visual feedback notify user to need to change the visual field (passing through mobile electronic device), until identifying in this way Object.Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, passing through side It helps user to provide correctly input, and reduces the user's mistake occurred when operating equipment/interact with equipment).

In some embodiments, in response to detect it is one or more it is magazine at least one relative to physical space Movement (1416): according to determining that placing user interface element is located at the position in the expression of physical space, the wherein position pair Should the first object away from one of one or more of cameras first distance in physical space, electronic equipment is aobvious with the first size Show placement user interface element；And the position in the expression of physical space is located at according to determining user interface element of placing, In the position correspond to physical world in the second object away from one of one or more of cameras second distance, electronic equipment with Second size shows the placement user interface element.In some embodiments, first distance is greater than second distance, and the One size is less than the second size.For example, corresponding to physical space 5000 when being located in live preview as shown in Fig. 5 BO and 5BT In relatively far point position on when (as shown in Fig. 5 BO), graticule 5010 and focus 5012 are than when being located in live preview Correspondingly sized display small (as shown in Fig. 5 BT) when corresponding on the position of the relatively near point in physical space 5000.Another In example, as shown in Fig. 5 CG and 5CJ, when the position being located in live preview corresponding to the relatively far point in physical space 5000 On when (as shown in Fig. 5 CG), graticule 5010 and focus 5012 are than corresponding to physical space when being located in live preview Correspondingly sized display small (as shown in Fig. 5 CJ) when on the position of the relatively near point in 5000.When placement user interface element position When the position of the given object in the RUNTIME VIEW from camera, electronic equipment is based in physical space from camera to given pair The distance of elephant places the size of user interface element to adjust.This visual feedback is provided a user about pair in physical space The information of the relative position of elephant.Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, by helping user to provide correctly input, and reduce the user's mistake occurred when operating equipment/interact with equipment Accidentally).

In some embodiments, first size of the user interface element in the expression of physical space is placed relative to object The first object is than placing second size of the user interface element in the expression of physical space relative to physical space in reason space In the second object it is bigger (1418) (for example, as described in herein with reference to Fig. 5 BT).Although in some embodiments, placing user The size of interface element based in physical space from camera to the distance of given object, but in order to avoid making the size in object When far from camera too small (or excessive when object is very close to camera), in some embodiments, which is not essence Really scaled with distance.Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, by helping user to provide correctly input, and reduce the user's mistake occurred when operating equipment/interact with equipment Accidentally).

In some embodiments, placing user interface element includes (1420) first part (for example, graticule, such as schemes Graticule 5010 in 5CC) and second part (for example, point, focus 5012 or other labels in such as Fig. 5 CC).Some In embodiment, in response to detecting at least one one or more magazine movement relative to physical space: according to true Surely it places user interface element and is located at the feature (example for corresponding to the predefined type in physical space in the expression of physical space Such as, turning, line or other features that the second part of user interface element can be aligned are placed) position, electronic equipment update The appearance of user interface element is placed, so that placing the second part of user interface element relative to placement user interface element First part is amplified；And it is located in the expression of physical space and in physical space according to determining user interface element of placing The feature (for example, turning, line or other features that the second part for placing user interface element can be aligned) of predefined type Not corresponding position, electronic equipment keep the display for placing user interface element without that will place second of user interface element Split-phase is amplified for placing the first part of user interface element.For example, as shown in Fig. 5 CC and 5CE, when focus is aligned to When anchor point (point-of-interest in the expression of the measurement or physical object such as previously added), focus 5012 is relative to graticule 5010 (and when it is unjustified to anchor point when size relative to focus 5012) be amplified.It is located at when placing user interface element Change the appearance for placing user interface element when on the feature that virtual mark point can be aligned (for example, by increasing set-point Relative to the size for placing user interface element rest part), allow to easily add virtual mark point at this feature. Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, by helping user Correctly input is provided, and reduces the user's mistake occurred when operating equipment/interact with equipment).

Place user interface element include (1422) first part (for example, graticule) and second part (for example, put or Other labels) some embodiments in, in response to detect it is one or more it is magazine at least one relative to physics sky Between movement: according to determine place user interface element be located in the expression of physical space correspond to physical space in predefining The position of the feature (for example, turning, line or other features that the second part for placing user interface element can be aligned) of type, The appearance of electronic equipment more new placement user interface element, so that the second part for placing user interface element is used relative to placement The first part of family interface element shifts；And it is located in the expression of physical space and object according to determining user interface element of placing Reason space in predefined type feature (for example, place user interface element second part can be aligned turning, line or Other features) not corresponding position, electronic equipment keeps the display for placing user interface element first without that will place user interface The second part of element is relative to the first part's displacement for placing user interface element.For example, working as focus as shown in Fig. 5 CC and 5CE When being aligned to anchor point (point-of-interest in the expression of the measurement or physical object such as previously added), focus 5012 is opposite In graticule 5010 (and when it is unjustified to anchor point when position relative to the focus 5012 in graticule 5010) be shifted.When putting It sets when user interface element is located on the feature that virtual mark point can be aligned and changes the appearance (example for placing user interface element Such as, by mobile set-point relative to the position for placing user interface element rest part, so that set-point snaps to the spy Sign), allow to easily add virtual mark point at this feature.Improved feedback is provided and enhances the operability of equipment And make user equipment interface it is more efficient (for example, by help user provide correctly input, and reduce operation equipment/ The user's mistake occurred when being interacted with equipment).

In some embodiments, in response to detect it is one or more it is magazine at least one relative to physical space Movement (1424), according to determining that equipment can not identify its in the second part of physical space in the expression of physical space Corresponding objects can be linked to the object virtually marked, and electronic equipment shows that the prompt containing information (is used for example, placing with mark Family interfacial separation or overlapping), the information instruction electronic equipment can not identify in the second part of physical space its physics sky Between expression in corresponding objects can be linked to the object virtually marked.For example, as shown in Fig. 5 CO, in response to equipment 100 Further away from the movement of desk 5200, equipment 100 shows that error message 5250 can not identify physics sky with indicating equipment 100 Between its physical space 5000 in user interface 5006 in 5000 expression in corresponding objects can be linked to virtual mark The object of (for example, virtual measurement).Display electronic equipment can not identify pair of its in physical space in the expression of physical space Reply notifies user to need to change the visual field (passing through mobile electronic device) as that can be linked to the prompt of the object virtually marked, directly To identifying such object.Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, by helping user to provide correctly input, and reduce the user's mistake occurred when operating equipment/interact with equipment Accidentally).

In some embodiments, in response to detect it is one or more it is magazine at least one relative to physical space Movement (1426), according to determining that equipment can not identify its in the second part of physical space in the expression of physical space Corresponding objects can be linked to the object virtually marked, and electronic equipment shows the prompt containing information (for example, with user circle is placed Face, which separates or overlaps, places in user interface), information instruction electronic equipment can not identify in the second part of physical space The reason of its corresponding objects in the expression of physical space can be linked to the object virtually marked.In some embodiments In, which includes the steps that description can be taken to improve the letter of the ability of the object in electronic equipment identification physical space Breath.For example, in response to equipment 100 further away from the movement of desk 5200, equipment 100 shows that mistake disappears as shown in Fig. 5 CO Breath 5250 can not identify object with indicating equipment 100 because equipment 100 need it is mobile closer to the object in the visual field of camera The such prompt of display: (1) explain why electronic equipment can not identify its in physical space in the expression of physical space Corresponding objects can be linked to the object virtually marked, and/or (2) if notify user to need to change the visual field (for example, passing through Mobile electronic device), it can help to correct such case.Improved feedback is provided to enhance the operability of equipment and to use Family equipment interface is more efficient (for example, inputting by the way that help user's offer is correct, and to be reduced and is operating equipment/and equipment friendship The user's mistake occurred when mutually).

In some embodiments, instruction electronic equipment can not identify its in the second part of physical space in physical space Expression in the information of corresponding objects the reason of being linked to the object virtually marked include (1428) one of the following Or multiple: about the instruction (for example, as shown in Fig. 5 D) for needing more light, about one or more cameras, (or electronics is set It is standby) at least one of the instruction that is moving too quickly, about needing in mobile one or more cameras (or electronic equipment) extremely Few one positions the instruction (for example, as shown in fig. 5e) on the surface in physical space, about one or more cameras (or electricity Sub- equipment) at least one of need mobile the instruction further from the object in physical space, and about one or more At least one of camera (or electronic equipment) needs mobile the instruction closer to the object in physical space (for example, as schemed Shown in 5CO).

In some embodiments, at the position of second part for corresponding to physical space in the expression in physical space When user interface element is placed in display, electronic equipment detects (1430) and places input (for example, placing light on user interface element Hitter's gesture or triggering are placed in user interface in mark places the button virtually marked).In some embodiments, in response to inspection Placement input is measured, at least part virtually marked is placed in the expression of physical space to correspond to place and use by electronic equipment At the position of family interface element.In some embodiments, the placement of the part of mark includes first point put down in measurement. In some embodiments, the placement of the part of mark includes second or the third measurement point put down in measurement.In some realities It applies in scheme, the placement of the part of mark includes that the placement of measurement is completed in the expression of physical space.For example, such as Fig. 5 BP- Shown in 5BQ, measurement point 5206 is put in touch input 5204 (for example, Flick gesture) triggering on graticule 5010 and focus 5012 It sets and corresponds in the expression of physical space 5000 at the position of focus 5012 in user interface 5006.Come in response to placing input Virtual mark point is placed at the display position for placing user interface element, is made it easy to virtual mark point location in physics Proper position in the expression in space.Improved visual feedback is provided to enhance the operability of equipment and user-is set Standby interface is more efficient (for example, by helping user to provide correctly input, and to be reduced when operating equipment/interact with equipment User's mistake of appearance).

In some embodiments, at the position of second part for corresponding to physical space in the expression in physical space When user interface element is placed in display, electronic equipment detection (1432) at the position for corresponding to placement user interface element is inputted (for example, the Flick gesture on user interface element is being placed for the electronic equipment with touch-sensitive display).In some implementations In scheme, in response to detecting input at the position for corresponding to placement user interface element, electronic equipment is placed in mark and is used Another user interface element in the interface of family nearby (or side) shows that figure instruction, figure instruction show another user The activation (for example, for touch-sensitive display, passing through the Flick gesture on another user interface element) of interface element will be led At least part virtually marked is caused to be placed at the position for corresponding in the expression of physical space and placing user interface element (for example, electronic equipment display taps button to put the instruction of next point, wherein the button is different from user interface element is placed And it is located remote from user interface element).For example, the touch input 5204 as shown in Fig. 5 BR, on graticule 5010 and focus 5012 (for example, Flick gesture) causes to show instruction message 5208 near measurement addition button 5014, measures addition button with instruction 5014 activation, which will lead to for measurement point to be placed in user interface 5006 in the expression of physical space 5000, corresponds to focus At 5012 position.In some embodiments, (rather than it is tapped when user attempts to place user interface element by tap His element, such as "+", " addition " or analogous element) come when creating virtual mark point, electronic equipment can be in "+", " addition " or class It is displayed next to a piece of news like element, indicates that this is virtual to create to tap (rather than tap places user interface element) Mark the correct element of point.Improved feedback is provided to enhance the operability of equipment and make user equipment interface more efficient (for example, by helping user to provide correctly input, and reduce the user's mistake occurred when operating equipment/interact with equipment Accidentally).

It should be appreciated that the particular order that the operation in Figure 14 A-14D is described is only exemplary, it is not intended that Show that the sequence is the unique order that can execute these operations.Those skilled in the art will recognize that various ways It resequences to operations described herein.Moreover, it is noted that herein in relation to other methods as described herein The details of other processes of (for example, method 600,700,800,900,1000,1100,1200 and 1300) description is equally with class As mode be suitable for the method 1400 that describes above with reference to Figure 14 A-14D.For example, being described above with reference to method 1400 defeated Enter, user interface element is (for example, measurement point, measuring section, virtual mark, physical space or the expression in the visual field, showing and can indicate, mention Show, indicator, label, anchor point and/or place user interface element such as graticule and point), tactile output and intensity threshold can Selection of land has herein with reference to other methods as described herein (for example, 600,700,800,900,1000,1100,1200 and of method 1300) one or more features of the input of description, user interface element, tactile output and intensity threshold.For brevity, These details are not repeated herein.

Above with reference to Fig. 6 A-6C, Fig. 7 A-7E, Fig. 8 A-8C, Fig. 9 A-9B, Figure 10 A-10B, Figure 11 A-11B, Figure 12 A- Optionally by Figure 1A, into Figure 1B, discribed component is implemented for operation described in 12C, Figure 13 A-13C and Figure 14 A-14D.Example Such as, detection operation 606,614,706,708,812,906,1006,1106,1206,1334,1430 and 1432, and receive behaviour Make 1304 and 1310 optionally to be realized by event classifier 170, event recognizer 180 and button.onrelease 190.Event point Event monitor 171 in class device 170 detects the contact on touch-sensitive display 112, and event dispatcher module 174 will Event information is transmitted to application program 136-1.The corresponding event identifier 180 of application program 136-1 is by event information and accordingly Event defines 186 and is compared, and whether determine the first contact on touch-sensitive display at first position (or the rotation of the equipment Turn whether) correspond to predefined event or subevent, such as selection to the object in user interface or the equipment are from one It is orientated the rotation of another orientation.When detecting corresponding predefined event or subevent, event recognizer 180 is activated Button.onrelease 190 associated with the detection to the event or subevent.Button.onrelease 190 is optionally used or is adjusted With data renovator 176 or object renovator 177 come more new application internal state 192.In some embodiments, at event Reason program 190 accesses corresponding GUI renovator 178 and carrys out content shown by more new application.Similarly, those skilled in the art Member can know clearly based in Figure 1A, into Figure 1B, how discribed component can realize other processes.

For illustrative purposes, the description of front is described by reference to specific embodiment.However, example above The property shown discussion is not intended to exhausted or limits the invention to disclosed precise forms.According to above teaching content, very More modifications and variations are all possible.Selection and description embodiment are to most preferably illustrate the principle of the present invention And its practical application, so as to so that others skilled in the art most preferably can be suitable for being conceived using having The described embodiment of the invention and various of the various modifications of special-purpose.

According to the method for the present invention and in the embodiment of product project listed below:

1. a kind of method, comprising:

At the electronic equipment with touch-sensitive display and one or more cameras:

The user interface of application program is shown on the touch-sensitive display, in which:

The user interface include it is one or more of it is magazine at least one the visual field expression；

The expression in the visual field is shown with the first magnifying power；

Based on the variation by least one one or more of magazine Current vision data detected, at any time Update the expression in the visual field；And

The visual field includes at least part of three-dimensional space；

While showing the expression in the visual field, the first touch input on the touch-sensitive display is detected；

In response to detecting first touch input, adds and show at the first position in the expression in the visual field Measurement point, the first position correspond to the first position in the three-dimensional space；And

After adding the measurement point and when continuing to show the expression in the visual field:

With it is one or more of it is magazine at least one movement, in the expression in the visual field with the three-dimensional space Between in the corresponding position in first position at the display measurement point；

It detects corresponding with the current location of measurement point described in the expression in the visual field on the touch-sensitive display The second touch input at position；And

In response to detecting second touch input, by at least part of display of the expression in the visual field from described First magnifying power is amplified to the second magnifying power greater than first magnifying power, wherein the part of the expression in the visual field Amplification display includes the measurement point.

2. according to method described in project 1, in which:

One or more of cameras be located at it is a part of adjacent with the touch-sensitive display on the electronic equipment, it is described The part of touch-sensitive display is close to the side where one or more of cameras in the equipment；

The user interface includes:

It is shown in the portion far from the side where one or more of cameras in the equipment of the touch-sensitive display First showing and can indicate in point, and when described first show can indicate to be activated when, add to the expression in the shown visual field Add measurement point, and

Show and can indicate according to the one or more second that the first orientation of the electronic equipment is shown；And

The described method includes:

Detect the movement of the electronic equipment to second orientation；And

In response to detecting the movement of the electronic equipment to the second orientation:

One or more of second, which are updated, according to the second orientation of the electronic equipment shows the display that can be indicated；With And

Continue the institute far from the side where one or more of cameras in the equipment in the touch-sensitive display It states and shows that described first shows and can indicate in part.

3. the method according to any one of project 1 to 2, in which:

The user interface of the application program includes one or more corresponding with the operation of the application program Showing can indicate, including capture is shown and can be indicated；And

The described method includes:

It detects the capture and shows the third touch input on capable of indicating；And

Meet the first media capture standard according to the determination third touch input, initiates catching to the media of the first kind Obtain, the media of the first kind include it is one or more of it is magazine at least one the visual field image sequence.

4. according to method described in project 3, comprising:

Meet the second media capture standard according to the determination third touch input, initiates catching to the media of Second Type Obtain, the media of the Second Type include it is one or more of it is magazine at least one the visual field single image.

5. the method according to any one of project 1 to 4, comprising:

Before the user interface for showing the application program:

Display includes the control panel user interface of multiple controls, wherein the first control in the multiple control corresponds to The application program；And

Detection activates the touch input of first control；And

The touch input that first control is activated in response to detecting, shows the user of the application program Interface.

6. the method according to any one of project 1 to 5, in which:

The user interface is the first user interface in multiple user interfaces in the application program；

First user interface corresponds to the measurement pattern of the application program；And

Second user interface in the multiple user interface corresponds to the leveling mode of the application program.

7. the method according to any one of project 1 to 6, comprising:

Determine the distance between the first position for corresponding to the measurement point in the electronic equipment and the three-dimensional space；

Wherein:

Be less than first threshold distance according to distance determined by determining, first magnifying power and second magnifying power it Between amplification factor be the first amplification factor；

According to identified distance is determined greater than second threshold distance, the amplification factor is the second amplification factor；And

According to determining identified distance between the first threshold distance and second threshold distance, the amplification The factor is the corresponding amplification factor between first amplification factor and second amplification factor, and the corresponding amplification factor takes Certainly in the identified distance.

8. the method according to any one of project 1 to 7, comprising:

When showing at least part of amplification of expression in visual field display, the 4th touch input, institute are detected Stating the 4th touch input includes the contact for detecting the current position of the measurement point and the detection contact described touch-sensitive aobvious Show the movement on device；And

In response to detecting the movement of the contact on the touch-sensitive display:

Table according to the movement of the contact in the 4th touch input, by the measurement point in the visual field Show movement.

9. the method according to any one of project 1 to 8, in which:

First touch input it is such show can indicate to be detected: when this, which shows, to indicate to be activated, will survey Amount point is added in the expression in the shown visual field；

It is described to show the predefined position that indicate to be shown in the user interface；And

Initial response shows the activation that can be indicated and adds the position of the measurement point not in described in the expression in the visual field Show the predefined position that can be indicated described in being same as.

10. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；

One or more processors；And

Memory, the memory store one or more programs, wherein one or more of programs be configured as by One or more of processors execute, and one or more of programs include the instruction for performing the following operation:

The expression in the visual field is shown with the first magnifying power；

The visual field includes at least part of three-dimensional space；

11. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of touch-sensitive display and one or more cameras executes, so that the electronics Equipment:

The expression in the visual field is shown with the first magnifying power；

The visual field includes at least part of three-dimensional space；

In response to detecting first touch input, is added at the first position in the expression of the visual field and display is surveyed Point is measured, the first position corresponds to the first position in the three-dimensional space；And

After adding the measurement point and when continuing to show that the visual field indicates:

12. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；

For showing the device of the user interface of application program on the touch-sensitive display, in which:

The expression in the visual field is shown with the first magnifying power；

The visual field includes at least part of three-dimensional space；

Showing the device enabled when the expression in the visual field, it is defeated for detecting the first touch on the touch-sensitive display Enter；

The device enabled in response to detecting first touch input, for first in the expression in the visual field Measurement point is added and shown at position, and the first position corresponds to the first position in the three-dimensional space；And

After adding the measurement point and continuing to show the device enabled when the expression in the visual field, described device Include:

The device enabled at least one one or more of magazine movement, for the table in the visual field Show and shows the measurement point at position corresponding with the first position in the three-dimensional space；

It is opposite with the current location of measurement point described in the expression in the visual field on the touch-sensitive display for detecting The device of the second touch input at the position answered；And

The device enabled in response to detecting second touch input, for by least the one of the expression in the visual field Partial display is amplified to the second magnifying power greater than first magnifying power from first magnifying power, wherein the visual field The amplification of the part indicated is shown including the measurement point.

13. a kind of for including at information used in the electronic equipment of touch-sensitive display and one or more cameras Manage device, comprising:

The expression in the visual field is shown with the first magnifying power；

The visual field includes at least part of three-dimensional space；

14. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；

One or more processors；And

Store the memory of one or more programs, wherein one or more of programs be configured as by one or Multiple processors execute, and one or more of programs include for either executing in the method according to project 1 to 9 The instruction of method.

15. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of touch-sensitive display and one or more cameras executes, so that the electronics Equipment execution method either in the method according to project 1 to 9.

16. the graphic user interface in a kind of electronic equipment, the electronic equipment includes touch-sensitive display, one or more Camera, memory and one or more processing for executing the one or more programs being stored in the memory Device, the graphic user interface include the user interface that method is shown either in the method according to project 1 to 9.

17. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；And

For either executing in the method according to project 1 to 9 device of method.

18. a kind of for including at information used in the electronic equipment of touch-sensitive display and one or more cameras Manage device, comprising:

19. a kind of method, which comprises

It is sensed with touch-sensitive display, to detect with the one or more of the intensity of the contact of the touch-sensitive display At the electronic equipment of device and one or more cameras:

Based on the variation by least one one or more of magazine Current vision data detected, at any time Update the expression in the visual field；

The user interface includes the measurement point creation indicator being shown on the expression in the visual field；And

The visual field includes at least part of three-dimensional space；

Detect the contact on the touch-sensitive display；And

In the contact being consecutively detected on the touch-sensitive display:

When the measurement point creation indicator be shown in the visual field expression in first in the three-dimensional space When setting on corresponding first position:

Meet the first standard according to determining, adds and shown corresponding in the three-dimensional space in the expression in the visual field First position the first measurement point, wherein first standard includes that the intensity of the contact is required to meet respective strengths threshold value So that first standard is met；

After adding first measurement point, the expression in the visual field described in the mobile update with the electronic equipment；With And

After the electronic equipment is mobile, when in the expression that measurement point creation indicator is shown in the visual field with When on the corresponding second position in the second position in the three-dimensional space:

According to determine when the measurement point creation indicator be shown in the expression in the visual field in the three-dimensional space The corresponding second position in the second position on when meet first standard:

The second measurement of the second position corresponded in the three-dimensional space is added and shown in the expression in the visual field Point；And

Display connects the first measuring section of first measurement point and second measurement point.

20. according to method described in project 19, comprising: in the contact being consecutively detected on the touch-sensitive display When:

After adding second measurement point, the table in the visual field described in the mobile update again with the electronic equipment Show；And

After the electronic equipment moves again, when measurement point creation indicator is shown in the visual field expression When on the third place corresponding with the third place in the three-dimensional space:

According to determine when the measurement point creation indicator be shown in the expression in the visual field in the three-dimensional space The corresponding the third place of the third place on when meet first standard:

The third measurement of the third place corresponded in the three-dimensional space is added and shown in the expression in the visual field Point；And

Display connects the second measuring section of second measurement point and the third measurement point.

21. the method according to any one of project 19 to 20, comprising:

According to determine when the measurement point creation indicator be shown in the expression in the visual field in the three-dimensional space The corresponding second position in the second position on when be unsatisfactory for first standard:

It abandons adding and showing in the expression in the visual field described in the second position corresponded in the three-dimensional space Second measurement point；And

Abandon first measuring section of display connection first measurement point and second measurement point.

22. the method according to any one of project 19 to 21, comprising:

After the electronic equipment moves again, when measurement point creation indicator is shown in the expression in the visual field In on the third place corresponding with the third place in the three-dimensional space when:

The third measurement of the third place corresponded in the three-dimensional space is added and shown in the expression in the visual field Point；

Meet the second standard according to determining, display connects the second measurement of second measurement point and the third measurement point Section, wherein second standard includes requiring meeting first standard to add second measurement point and meet described the One standard keeps the contact between the third measurement point to add；And

It is unsatisfactory for second standard according to determination, display is abandoned and connects second measurement point and the third measurement point Second measuring section.

23. the method according to any one of project 19 to 22, wherein the electronic equipment includes one or more touchings Feel output generator, and the described method includes:

Meet the respective strengths threshold value according to the intensity of the determination contact, generates tactile output.

24. the method according to any one of project 19 to 23, wherein the electronic equipment includes one or more touchings Feel output generator, the respective strengths threshold value is above the second intensity threshold of the first intensity threshold, and the method packet It includes:

Met according to the intensity that the intensity of the determination contact no longer meets second intensity threshold and the contact First intensity threshold generates tactile output.

25. the method according to any one of project 19 to 23, in which:

When the measurement point creation indicator be shown in the visual field expression in first in the three-dimensional space When setting on corresponding first position, if the intensity of the contact meets the respective strengths threshold value, described is executed The addition and the display of one measurement point；And

When the measurement point creation indicator be shown in the visual field expression in the second in the three-dimensional space When setting on the corresponding second position, if described in the intensity of the contact meets after the electronic equipment is mobile Respective strengths threshold value then executes the addition and the display of second measurement point.

26. the method according to any one of project 19 to 24, in which:

The respective strengths threshold value is above the second intensity threshold of first intensity threshold；

First standard includes that the intensity of the contact is required to drop to institute after meeting the respective strengths threshold value Respective strengths threshold value or less is stated to meet first standard；

When the measurement point creation indicator be shown in the visual field expression in first in the three-dimensional space When setting on corresponding first position, if the strength reduction of the contact to the respective strengths threshold value hereinafter, if execute The addition and the display of first measurement point；And

After the electronic equipment is mobile and when measurement point creation indicator is shown in the expression in the visual field In on the second position corresponding with the second position in the three-dimensional space when, if the strength reduction of the contact is to institute Respective strengths threshold value is stated hereinafter, then executing the addition and the display of second measurement point.

27. the method according to any one of project 19 to 26, comprising:

When showing the expression in the visual field of one or more of cameras:

Determine the corresponding positions corresponded in the three-dimensional space in the expression in the visual field of one or more of cameras The anchor point set；And

With the movement of one or more of cameras, when measurement point creation indicator is on the anchor point When, change the visual appearance of the measurement point creation indicator to indicate:, will if the contact meets first standard Corresponding measurement point is added at the anchor point.

28. the method according to any one of project 19 to 27 works as institute wherein the user interface includes showing and capable of indicating It states and shows that the position that the creation indicator of the measurement point described in the expression in the visual field is shown above adds when can indicate to be activated Add measurement point, and the method includes；

Show the touch input that can be indicated described in detection activation；And

Show the touch input that can be indicated described in activation in response to detecting, is measured described in the expression in the visual field The position addition and display measurement point that point creation indicator is shown above.

29. a kind of electronic equipment, comprising:

Touch-sensitive display；

For detecting and the one or more sensors of the intensity of the contact of the touch-sensitive display；

One or more cameras；

One or more processors；And

The visual field includes at least part of three-dimensional space；

Detect the contact on the touch-sensitive display；And

In the contact being consecutively detected on the touch-sensitive display:

30. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction when by include touch-sensitive display, to detect with one of the intensity of the contact of the touch-sensitive display or When the electronic equipment of multiple sensors and one or more cameras executes, so that the electronic equipment:

The visual field includes at least part of three-dimensional space；

Detect the contact on the touch-sensitive display；And

In the contact being consecutively detected on the touch-sensitive display:

31. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；

The visual field includes at least part of three-dimensional space；

For detecting the device of the contact on the touch-sensitive display；And

The device enabled in the contact being consecutively detected on the touch-sensitive display, described device include:

When the measurement point creation indicator be shown in the visual field expression in first in the three-dimensional space The device enabled when setting on corresponding first position, described device include:

According to the device for meeting the first standard and enabling is determined, corresponded to for adding and showing in the expression in the visual field First measurement point of the first position in the three-dimensional space, wherein first standard includes requiring the intensity of the contact Meet respective strengths threshold value so that first standard is met；

The device enabled after adding first measurement point, for described in the mobile update with the electronic equipment The expression in the visual field；And

After the electronic equipment is mobile, when in the expression that measurement point creation indicator is shown in the visual field with The device enabled when on the corresponding second position in the second position in the three-dimensional space, described device include:

According to determine when the measurement point creation indicator be shown in the expression in the visual field in the three-dimensional space The corresponding second position in the second position on when meet first standard and the device that enables:

For the second of the second position corresponded in the three-dimensional space to be added and shown in the expression in the visual field The device of measurement point；And

For showing the device for connecting the first measuring section of first measurement point and second measurement point.

32. it is a kind of for include touch-sensitive display, to detect one with the intensity of the contact of the touch-sensitive display Information processing unit used in the electronic equipment of a or multiple sensors and one or more cameras, comprising:

The visual field includes at least part of three-dimensional space；

For detecting the device of the contact on the touch-sensitive display；And

33. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；

One or more processors；And

Store the memory of one or more programs, wherein one or more of programs be configured as by one or Multiple processors execute, and one or more of programs include for executing any in the method according to project 19 to 28 The instruction of method.

34. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction when by include touch-sensitive display, to detect with one of the intensity of the contact of the touch-sensitive display or When the electronic equipment of multiple sensors and one or more camera executes so that the electronic equipment execute according to project 19 to Method either in method described in 28.

35. the graphic user interface in a kind of electronic equipment, the electronic equipment include touch-sensitive display, to detect with The one or more sensors of the intensity of the contact of the touch-sensitive display, one or more camera, memories and to hold The one or more processors for one or more programs that row is stored in the memory, the graphic user interface includes root According to the user interface that method is shown either in method described in project 19 to 28.

36. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；And

For either executing in the method according to project 19 to 28 device of method.

37. it is a kind of for include touch-sensitive display, to detect one with the intensity of the contact of the touch-sensitive display Information processing unit used in the electronic equipment of a or multiple sensors and one or more cameras, comprising:

38. a kind of method, comprising:

The visual field includes at least part of three-dimensional space；

When showing the expression in the visual field:

Determine the anchor point at the position for corresponding to the first position in the three-dimensional space in the expression in the visual field；

With at least one one or more of magazine movement, when measurement point creation indicator is in the anchor When on fixed point, change the visual appearance of the measurement point creation indicator to indicate: if the touch input meets first Standard will then add measurement point at the anchor point；

Detect the first touch input for meeting first standard on the touch-sensitive display；And

Meet first touch input of first standard in response to detecting:

According to determining that the measurement point creation indicator is on the anchor point when meeting first standard, in institute It states and adds and show the first measurement at the anchor point for corresponding to the first position in the three-dimensional space in the expression in the visual field Point；And

According to the determining measurement point creation indicator described when meeting first standard not on the anchor point, In It is left in the expression in the visual field and the first measurement point is added and shown at the first position of the anchor point.

39. according to method described in project 38, in which:

Identified anchor point is also the endpoint that shown measurement indicates；And

Adding measurement point at the anchor point in the case where touch input meets the first standard will not be in the view The region surrounded by multiple shown measuring sections is formed in wild expression.

40. according to method described in project 38, comprising:

The expression of the first measurement of display on the expression in the visual field, in which:

The expression of first measurement includes the first end point corresponding to the second position in the three-dimensional space；

The expression of first measurement includes the second endpoint corresponding to the third place in the three-dimensional space；

The expression of first measurement includes the first line segment for connecting the first end point and second endpoint；With And

Identified anchor point is the midpoint of first line segment.

41. the method according to any one of project 38 to 40, wherein the electronic equipment includes

One or more tactile output generators, and the described method includes:

With at least one one or more of magazine movement, when measurement point creation indicator is in the anchor When on fixed point:

Measurement point described in Binding change creates the visual appearance of indicator, generates tactile output；

Wherein adding measurement point at the anchor point in the case where touch input meets the first standard will not be in institute State the region for being formed in the expression in the visual field of one or more cameras and being surrounded by multiple shown measuring sections.

42. the method according to any one of project 38 to 41, wherein the electronic equipment includes one or more touchings Feel output generator, and the described method includes:

Detect the measurement point creation movement of the indicator far from the anchor point；And

In response to detecting the measurement point creation movement of the indicator far from the anchor point, it is defeated to generate tactile Out.

43. the method according to any one of project 38 to 42, wherein the electronic equipment includes one or more touchings Feel output generator, and the described method includes:

Meet first touch input of first standard in response to detecting:

Add the measuring section that first measurement point is connected to first measurement point without adding；And

Generate the output of the first tactile；

Detect movement of the measurement point creation indicator to the second position in the expression in the visual field, the second Set the second position corresponded in the three-dimensional space；

When measurement point creation indicator is on the second position in the expression in the visual field, the touching is detected The second touch input for meeting first standard on quick display；And

Meet second touch input of first standard in response to detecting:

Add the second measurement point in the second place in the expression in the visual field；

Measuring section is added between first measurement point and second measurement point；And

It generates and exports different the second tactiles outputs from first tactile.

44. the method according to any one of project 38 to 43, comprising:

With at least one one or more of magazine movement:

When the expression in the visual field includes and the institute at least one the one or more of magazine visuals field When stating the corresponding region of the physical features identified in three-dimensional space, the measurement point creation indicator is shown；And

When expression when the visual field does not include region corresponding with the physical features identified in the three-dimensional space, Stop showing that the measurement point creates indicator.

45. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；

One or more processors；And

The visual field includes at least part of three-dimensional space；

When showing the expression in the visual field:

Meet first touch input of first standard in response to detecting:

46. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of touch-sensitive display and one or more cameras executes, so that the electronics Equipment:

The visual field includes at least part of three-dimensional space；

When showing the expression in the visual field:

Meet first touch input of first standard in response to detecting:

47. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；And

The visual field includes at least part of three-dimensional space；

Showing the device enabled when the expression in the visual field, described device includes:

Correspond to the anchoring at the position of the first position in the three-dimensional space in expression for determining the visual field The device of point；

With at least one one or more of magazine movement and when measurement point creation indicator is in institute The device enabled when stating on anchor point creates the visual appearance of indicator for changing the measurement point to indicate: if institute It states touch input and meets the first standard, then will add measurement point at the anchor point；

For detecting the device for meeting the first touch input of first standard on the touch-sensitive display；And

The device enabled in response to detecting first touch input for meeting first standard, described device packet It includes:

It is opened on the anchor point according to the determining measurement point creation indicator described when meeting first standard Device, for adding at the anchor point in the expression in the visual field corresponding to the first position in the three-dimensional space Adduction the first measurement point of display；And

According to determine when meeting first standard measurement point creation indicator not on the anchor point and The device of enabling, is added at the first position for leaving the anchor point in the expression in the visual field and the first measurement of display Point.

48. a kind of for including at information used in the electronic equipment of touch-sensitive display and one or more cameras Manage device, comprising:

The visual field includes at least part of three-dimensional space；

49. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；

One or more processors；And

Store the memory of one or more programs, wherein one or more of programs be configured as by one or Multiple processors execute, and one or more of programs include for executing any in the method according to project 38 to 44 The instruction of method.

50. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of touch-sensitive display and one or more cameras executes, so that the electronics Equipment execution method either in the method according to project 38 to 44.

51. the graphic user interface in a kind of electronic equipment, the electronic equipment includes touch-sensitive display, one or more Camera, memory and one or more processing for executing the one or more programs being stored in the memory Device, the graphic user interface include the user interface that method is shown either in the method according to project 38 to 44.

52. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；And

For either executing in the method according to project 38 to 44 device of method.

53. a kind of for including at information used in the electronic equipment of touch-sensitive display and one or more cameras Manage device, comprising:

54. a kind of method, comprising:

At the electronic equipment with display, input equipment and one or more cameras:

The user interface of application program is shown on the display, in which:

The visual field includes the physical object in three-dimensional space；

While showing the expression in the visual field, one or more user's inputs, institute are detected via the input equipment It states one or more user's inputs and adds the first measurement corresponding with the physical object on the expression in the visual field It indicates；And

Show the expression and description first measurement of first measurement simultaneously on the expression in the visual field The first label, in which:

It is less than first threshold distance according to the first distance between the determination electronic equipment and the physical object, with the One threshold size shows first label；

It is greater than second threshold distance according to the first distance between the determination electronic equipment and the physical object, First label is shown to be less than the second threshold size of the first threshold size, wherein the second threshold distance is greater than The first threshold distance；And

According to the first distance between the determination electronic equipment and the physical object the first threshold away from From and the second threshold with a distance between, between the first threshold size and the second threshold size size show institute The first label is stated, the size depends on the first distance between the electronic equipment and the physical object.

55. according to method described in project 54, comprising:

It is shown described in the expression and the description measurement of the measurement simultaneously on the expression in the visual field When label:

When the electronic equipment is away from the physical object first distance, along the expression of the measurement with the first quarter Degree shows one or more first scale marks；

The electronic equipment is moved to the movement of the electronic equipment away from the physical object second distance by detection；With And

When the electronic equipment is away from the physical object second distance, along at least the one of the expression of the measurement Part shows one or more second scale marks with second scale different from first scale.

56. the method according to any one of project 54 to 55, comprising:

Second group of one or more user input is detected, second group of one or more user input is in the visual field The expression of the second measurement corresponding with the respective physical object in the three-dimensional space is added on expression；

Show the expression and description second measurement of second measurement simultaneously on the expression in the visual field The second label, in which:

It is less than the first threshold according to the second distance between the determination electronic equipment and the respective physical object Distance shows second label with the first threshold size；

It is greater than described second according to the second distance between the determination electronic equipment and the respective physical object Threshold distance shows second label with the second threshold size；And

According to the second distance between the determination electronic equipment and the respective physical object in first threshold It is worth between distance and second threshold distance, it is aobvious with the size between the first threshold size and the second threshold size Show second label, the size depend between the electronic equipment and the respective physical object described second away from From.

57. according to method described in project 56, in which:

The first distance between the electronic equipment and the corresponding physical object of first measurement is different The second distance between the corresponding respective physical object of the electronic equipment and second measurement；

First label is shown with the first size；

Second label is shown with the second size；And

Described first is of different sizes in second size.

58. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；

One or more processors；And

The user interface of application program is shown on the display, in which:

The visual field includes the physical object in three-dimensional space；

59. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of display, input equipment and one or more cameras executes, so that institute State electronic equipment:

The user interface of application program is shown on the display, in which:

The visual field includes the physical object in three-dimensional space；

It is greater than second threshold distance according to the first distance between the determination electronic equipment and the physical object, First label is shown to be less than the second threshold size of the first threshold size, and the second threshold distance is greater than described First threshold distance；And

60. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；And

For showing the device of the user interface of application program on the display, in which:

The visual field includes the physical object in three-dimensional space；

The device enabled when the expression in the visual field is being shown, for detecting one or more use via the input equipment Family input, one or more of users input add corresponding with the physical object the on the expression in the visual field The expression of one measurement；And

For showing the expression and description described first of first measurement simultaneously on the expression in the visual field The device of first label of measurement, in which:

61. a kind of for including used in the electronic equipment of display, input equipment and one or more cameras Information processing unit, comprising:

The visual field includes the physical object in three-dimensional space；

62. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；

One or more processors；And

Store the memory of one or more programs, wherein one or more of programs be configured as by one or Multiple processors execute, and one or more of programs include for executing any in the method according to project 54 to 57 The instruction of method.

63. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of display, input equipment and one or more cameras executes, so that institute State computer system execute the method according to project 54 to 57 in either method.

64. the graphic user interface in a kind of electronic equipment, the electronic equipment includes display, input equipment, one Or multiple cameras, memory and the one or more for executing the one or more programs being stored in the memory Processor, the graphic user interface include user circle that method is shown either in the method according to project 54 to 57 Face.

65. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；And

For either executing in the method according to project 54 to 57 device of method.

66. a kind of for including used in the electronic equipment of display, input equipment and one or more cameras Information processing unit, comprising:

67. a kind of method, comprising:

The user interface of application program is shown on the display, in which:

The visual field includes the physical object in three-dimensional space；

While showing the expression in the visual field, one or more user's inputs, institute are detected via the input equipment It states one or more user's inputs and adds the first measurement corresponding with the physical object on the expression in the visual field It indicates, in which:

The expression of first measurement includes the first end point corresponding to the first position on the physical object；

The expression of first measurement includes the second endpoint corresponding to the second position on the physical object；With And

The expression of first measurement includes the first line segment for connecting the first end point and second endpoint；

It is based in part on first measurement, determines the First Line in the expression in the visual field with first measurement The first area that section is adjoined, wherein the first area corresponds to the physics rectangular area in the three-dimensional space；And

The instruction of the first area is shown in the user interface, wherein the table for indicating to be covered on the visual field On the first area shown.

68. according to the method for project 67, wherein the user interface includes the survey being shown on the expression in the visual field Amount point creation indicator, and the described method includes:

When measurement point creation indicator is on the first area, via the input equipment, detection user is defeated Enter；

In response to detecting user's input when measurement point creation indicator is on the first area:

Change the visual appearance of the instruction of the first area to indicate that the first area has been identified.

69. the method according to any one of project 67 to 68, in which:

One or more of user's inputs are added corresponding with the physical object on the expression in the visual field The expression of second measurement, in which:

The expression of second measurement includes described the corresponding to the second position on the physical object Two endpoints；

The expression of second measurement includes the third endpoint corresponding to the third place on the physical object；And And

The expression of second measurement includes connecting the second line segment of second endpoint and the third endpoint；And And

The first area in the expression in the visual field of one or more of cameras is based on first measurement With it is described second measurement and determine.

70. the method according to any one of project 67 to 69, wherein one or more of magazine at least one A visual field includes the first part of the physics rectangular area, and the first area corresponds to the physics rectangle The first part in region, and the described method includes:

Detect the movement of the electronic equipment, the movement makes at least one one or more of magazine described The visual field is mobile；

In response to detecting the movement for the electronic equipment for keeping the visual field mobile, the table in the visual field is updated at any time Show one to show one or more additional areas corresponding with one or more extra sections of the physics rectangular area A or multiple instructions；And

According to the polymeric area of the determining one or more of additional areas for including the first area and showing at any time Domain corresponds to the entire physics rectangular area, the display label on the expression in the visual field, label description with it is described The corresponding measurement in physics rectangular area.

71. according to method described in project 70, comprising:

According to the determination zone of convergency correspond to the entire physics rectangular area and the expression in the visual field it After upper display describes the label of the measurement corresponding with the physics rectangular area, the electronic equipment is detected It is further mobile, the further movement make it is one or more of it is magazine at least one the visual field it is mobile to make The expression for obtaining the visual field includes the first area corresponding with the first part of the physics rectangular area；And

According to the determination zone of convergency correspond to the entire physics rectangular area, with the physics rectangular area The description survey corresponding with the physics rectangular area is shown on the corresponding first area of the first part The label of amount.

72. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；

One or more processors；And

The user interface of application program is shown on the display, in which:

The visual field includes the physical object in three-dimensional space；

73. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of display, input equipment and one or more cameras executes, so that institute State electronic equipment:

The user interface of application program is shown on the display, in which:

The visual field includes the physical object in three-dimensional space；

74. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；

The visual field includes the physical object in three-dimensional space；

The device enabled when the expression in the visual field is being shown, for detecting one or more use via the input equipment Family input, one or more of users input add corresponding with the physical object the on the expression in the visual field The expression of one measurement, in which:

For being based in part in expression of first measurement to determine the visual field and described in first measurement The device for the first area that first line segment adjoins, wherein the first area corresponds to the physics rectangle region in the three-dimensional space Domain；And

For showing the device of the instruction of the first area in the user interface, wherein the instruction is covered on institute It states on the first area in the expression in the visual field.

75. a kind of for including used in the electronic equipment of display, input equipment and one or more cameras Information processing unit, comprising:

The visual field includes the physical object in three-dimensional space；

76. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；

One or more processors；And

Store the memory of one or more programs, wherein one or more of programs be configured as by one or Multiple processors execute, and one or more of programs include for executing any in the method according to project 67 to 71 The instruction of method.

77. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of display, input equipment and one or more cameras executes, so that institute State electronic equipment execute the method according to project 67 to 71 in either method.

78. the graphic user interface in a kind of electronic equipment, the electronic equipment includes display, input equipment, one Or multiple cameras, memory and the one or more for executing the one or more programs being stored in the memory Processor, the graphic user interface include user circle that method is shown either in the method according to project 67 to 71 Face.

79. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；And

For either executing in the method according to project 67 to 71 device of method.

80. a kind of for including used in the electronic equipment of display, input equipment and one or more cameras Information processing unit, comprising:

81. a kind of method, comprising:

The first user interface of application program is shown on the touch-sensitive display, in which:

First user interface include it is one or more of it is magazine at least one the visual field expression；

The visual field includes the physical object in three-dimensional space；And

The expression of the measurement of the physical object is superimposed upon on the image of physical object described in the expression in the visual field；

While showing first user interface, the expression on the touch-sensitive display in the measurement is detected On the first touch input；And

In response to detecting first touch input on the touch-sensitive display in the expression of the measurement, Process of the starting for the shared information about the measurement.

82. according to method described in project 81, wherein the process of the starting for the shared information about the measurement The information including duplication about the measurement, and the described method includes:

After duplication is about the information of the measurement, detection is to by the information stickup about the measurement One or more users of destination on to the electronic equipment input；And

In response to detecting to by the mesh in the information stickup to the electronic equipment about the measurement Ground one or more of users input, the destination on the electronic equipment shows the institute about the measurement State information.

83. according to method described in project 81, in which:

Starting includes one or more including display for the process of the shared information about the measurement can The second user interface of activated user interface element, wherein it is one or more of can corresponding in activated user interface element can Activated user interface element corresponds to the corresponding destination of the information about the measurement；And

The described method includes:

Detect on the touch-sensitive display in the second user interface it is corresponding can be on activated user interface element Second touch input；And

In response to detecting second touch input, send the information about the measurement to described in corresponding to Accordingly can activated user interface element corresponding destination.

84. the method according to any one of project 81 to 83, comprising:

In response to detecting first touch input on the touch-sensitive display in the expression of the measurement, Show the information about the measurement.

85. according to method described in project 84, wherein meeting the first standard according to determination first touch input to hold The display of the row about the information of the measurement, wherein first standard includes requiring in first touch input The intensity of contact meet respective strengths threshold value to meet first standard.

86. the method according to any one of project 81 to 85, comprising:

Determine the classification of the physical object；

It include wherein the label for indicating the classification of the physical object about the information of the measurement.

87. the method according to any one of project 81 to 86 is wherein at least based in part on the electronic equipment and exists The expression of movement during the measurement in a first direction, the measurement has been added to the described of the application program User interface, and the described method includes:

It is based at least partially on the movement of the electronic equipment during the measurement in said first direction Determine the classification of the relationship between the measurement and the physical object；

It wherein, include the pass indicated between the measurement and the physical object about the information of the measurement The label of the classification of system.

88. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；

One or more processors；And

The visual field includes the physical object in three-dimensional space；And

89. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of touch-sensitive display and one or more cameras executes, so that the electronics Equipment:

The visual field includes the physical object in three-dimensional space；And

90. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；

For showing the device of the first user interface of application program on the touch-sensitive display, in which:

The visual field includes the physical object in three-dimensional space；And

The device enabled when showing first user interface, for detecting on the touch-sensitive display in the measurement The expression on the first touch input；And

In response to detecting first touch input on the touch-sensitive display in the expression of the measurement And the device enabled, for starting the process for being used for the shared information about the measurement.

91. a kind of for including at information used in the electronic equipment of touch-sensitive display and one or more cameras Manage device, comprising:

The visual field includes the physical object in three-dimensional space；And

92. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；

One or more processors；And

Store the memory of one or more programs, wherein one or more of programs be configured as by one or Multiple processors execute, and one or more of programs include for executing any in the method according to project 81 to 87 The instruction of method.

93. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of touch-sensitive display and one or more cameras executes, so that the electronics Equipment execution method either in the method according to project 81 to 87.

94. the graphic user interface in a kind of electronic equipment, the electronic equipment includes touch-sensitive display, one or more Camera, memory and one or more processing for executing the one or more programs being stored in the memory Device, the graphic user interface include the user interface that method is shown either in the method according to project 81 to 87.

95. a kind of electronic equipment, comprising:

Touch-sensitive display；

One or more cameras；And

For either executing in the method according to project 81 to 87 device of method.

96. a kind of for including at information used in the electronic equipment of touch-sensitive display and one or more cameras Manage device, comprising:

97. a kind of method, comprising:

The user interface of application program is shown on the display, in which:

The visual field includes at least part of three-dimensional space；

Detect the movement of the electronic equipment, the movement makes at least one one or more of magazine described It moves in a first direction in the visual field；

Detect the electronic equipment for moving the visual field in said first direction it is mobile when:

The expression in the visual field is updated according to the movement of the electronic equipment；

Identify the first element of one or more extended in the expression in the visual field along the first direction；And

It is based at least partially on the determination of one or more of first elements, institute is shown in the expression in the visual field State the first guidance that first party upwardly extends and corresponds to one in identified the first element of one or more.

98. wherein the visual field includes multiple elements, and the multiple element includes according to method described in project 97 One or more elements that side other than the first direction upwardly extends, and the described method includes:

Detection move the visual field in said first direction the electronic equipment it is mobile when, abandon being shown in The guidance that side other than the first direction upwardly extends.

99. the method according to any one of project 97 to 98, comprising:

Before the movement for detecting the electronic equipment for moving the visual field in said first direction:

Detect the first touch input on the touch-sensitive display；And

In response to detecting first touch input, adds and show at the first position in the expression in the visual field First measurement point, the first position correspond to the first position in the three-dimensional space；

Wherein:

Show that first guidance is also corresponded in the three-dimensional space based on determining one or more of first elements First position；And

First guidance includes the first position in the expression in the visual field.

100. according to method described in project 99, wherein described first at the first position in the expression in the visual field Measurement point is the measurement point that most recent is added in the expression in the visual field.

101. the method according to any one of project 97 to 100, comprising:

After the movement for detecting the electronic equipment for moving the visual field in said first direction, detection makes The movement for the electronic equipment that the visual field is moved in a second direction；

In response to detecting the movement for the electronic equipment for moving the visual field in this second direction, stop aobvious Show first guidance extended in said first direction；And

Detect the electronic equipment for moving the visual field in this second direction it is mobile when:

Identify the one or more second elements extended in the expression in the visual field along the second direction；And

It is based at least partially on the determination of one or more of second elements, institute is shown in the expression in the visual field State the second guidance that second party upwardly extends and corresponds to one in identified the second recognition element of one or more.

102. the method according to any one of project 97 to 101, wherein first guidance is corresponding identified At least one of one or more first elements are the elements for being identified as the edge of physical object in the three-dimensional space.

103. the method according to any one of project 97 to 101, wherein first guidance is corresponding identified At least one of one or more first elements are the elements for being identified as plane in the three-dimensional space.

104. the method according to any one of project 97 to 103, wherein the user interface is described including being shown in Measurement point creation indicator on the expression in the visual field, and the described method includes:

Corresponding position in the expression in the visual field shows corresponding measurement point, and the corresponding position corresponds to described three Corresponding position in dimension space；

Detection moves the measurement point creation indicator on the corresponding measurement point in the expression in the visual field The electronic equipment movement；

In response to detecting the electronics when measurement point creation indicator is shown on the corresponding measurement point The movement of equipment, shows multiple guidance, wherein the first guidance in the multiple guidance is perpendicular in the multiple guidance The second guidance, and wherein the multiple guidance is intersected in the corresponding measurement point.

105. the method according to any one of project 97 to 104, wherein the user interface is described including being shown in Measurement point creation indicator on the second position in the expression in the visual field, and the described method includes:

In the described first guidance that display extends in said first direction:

Detect the second touch input on the touch-sensitive display；

In response to detecting second touch input:

According to the distance between the determination second position and first guidance within the threshold range, draw described first It leads and the second measurement point is added and shown at the position away from the second position distance；And

According to the distance between the determination second position and first guidance not in the within the threshold range, described Add and show second measurement point in the second place.

106. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；

One or more processors；And

The user interface of application program is shown on the display, in which:

The visual field includes at least part of three-dimensional space；

107. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of display, input equipment and one or more cameras executes, so that institute State electronic equipment:

The user interface of application program is shown on the display, in which:

The visual field includes at least part of three-dimensional space；

108. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；And

The visual field includes at least part of three-dimensional space；

Move at least one the one or more of magazine visuals field in a first direction for detecting The device of the movement of the electronic equipment；

Detect the electronic equipment for moving the visual field in said first direction it is mobile when the dress that enables It sets, described device includes:

For updating the device of the expression in the visual field according to the movement of the electronic equipment；

The device of the first element of one or more extended in the expression in the visual field along the first direction for identification； And

The device for being based at least partially on the determination of one or more of first elements and enabling, in the visual field Expression in show in said first direction extend and correspond to identified the first element of one or more in one First guidance.

109. a kind of for including used in the electronic equipment of display, input equipment and one or more cameras Information processing unit, comprising:

The visual field includes at least part of three-dimensional space；

110. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；

One or more processors；And

Store the memory of one or more programs, wherein one or more of programs be configured as by one or Multiple processors execute, and one or more of programs include for executing appointing in the method according to project 97 to 105 The instruction of one method.

111. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is when by including that the electronic equipment of display, input equipment and one or more cameras executes, so that institute State electronic equipment execute the method according to project 97 to 105 in either method.

112. the graphic user interface in a kind of electronic equipment, the electronic equipment includes display, input equipment, one Or multiple cameras, memory and the one or more for executing the one or more programs being stored in the memory Processor, the graphic user interface include user circle that method is shown either in the method according to project 97 to 105 Face.

113. a kind of electronic equipment, comprising:

Display；

Input equipment；

One or more cameras；And

For either executing in the method according to project 97 to 105 device of method.

114. a kind of for including used in the electronic equipment of display, input equipment and one or more cameras Information processing unit, comprising:

115. a kind of method, comprising:

There is the electronics of one or more input equipments, one or more display equipment and one or more cameras to set Standby place:

The user interface of the expression including physical space is shown via one or more of display equipment；

When showing the expression of the physical space, receive to be created in the expression of the physical space The first group of one or more input virtually marked；

In response to receiving first group of one or more input, the first virtual mark is added to the physical space The expression, wherein the described first virtual mark is linked to a part of the expression of the physical space；

After the described first virtual mark is added to the expression of the physical space, receive empty with the physics Between the expression associated second group of one or more input；And

In response to receiving the associated second group of one or more input of the expression with the physical space:

Corresponded in the expression of the physical space according to determination second group of one or more input and is created The request virtually marked away from the described first virtual mark within the threshold range creates the in the expression of the physical space Two virtual marks, while the described first virtual mark is kept in the expression of the physical space；And

Corresponded in the expression of the physical space according to determination second group of one or more input and is created The request virtually marked outside the threshold distance is virtually marked away from described first, is created in the expression of the physical space It builds the second virtual mark and removes the described first virtual mark from the expression of the physical space.

116. according to method described in project 115, wherein creating described second in the expression of the physical space Virtual mark kept in the expression of the physical space simultaneously the described first virtual mark include: with it is described first empty Start to create the described second virtual mark at the corresponding position of at least part of quasi- mark.

117. according to method described in project 115, wherein creating described second in the expression of the physical space Virtual mark kept in the expression of the physical space simultaneously the described first virtual mark include: with it is described first empty The described second creation virtually marked is completed at the corresponding position of at least part of quasi- mark.

118. the method according to any one of project 115 to 117, comprising: empty with the physics in response to receiving Between the associated second group of one or more input of the expression, inputted according to determination second group of one or more Corresponding to by it is one or more of it is magazine at least one the visual field it is mobile be more than threshold quantity request, it is empty from the physics Between the expression in remove the described first virtual mark.

119. the method according to any one of project 115 to 118, comprising: empty with the physics in response to receiving Between the associated second group of one or more input of the expression, inputted according to determination second group of one or more Move so that described first is virtually labeled in and is more than corresponding to by least one the one or more of magazine visuals field No longer visible request in threshold amount of time removes the described first virtual mark from the expression of the physical space.

120. the method according to any one of project 115 to 119 is included in and receives described second group one or more When a input and in the described first virtual expression of the mark in the physical space, output shows further defeated Enter will lead to the instruction that the described first virtual mark is removed from the expression of the physical space.

121. according to method described in project 120, wherein the instruction is in the user interface for display reminding Predetermined portions in show visually indicate.

122. according to method described in project 120, wherein the instruction is the institute in the expression of the physical space State the first cosmetic variation virtually marked.

123. according to method described in project 122, wherein the described first virtual mark is include the description measured described The expression of measurement, and the described first cosmetic variation virtually marked includes removing institute from the expression of the measurement Description is stated, while keeping at least part of the expression of the measurement.

124. according to method described in project 122, including, described the in the expression for showing the physical space After one cosmetic variation virtually marked:

Detect the input at the position away from the described first virtual mark within the threshold range；And

In response to detecting the input at the position for virtually marking the within the threshold range away from described first, Invert the described first cosmetic variation virtually marked.

125. a kind of electronic equipment, comprising:

One or more input equipments；

One or more display equipment；

One or more cameras；

One or more processors；And

126. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is by including one or more input equipments, one or more display equipment and one or more cameras Electronic equipment execute when so that the electronic equipment:

127. a kind of electronic equipment, comprising:

One or more input equipments；

One or more display equipment；

One or more cameras；

For showing the device of the user interface of the expression including physical space via one or more of display equipment；

The device being activated when the expression of the physical space is being shown, for receiving in the physical space The expression in create virtually mark first group of one or more input；

The device being activated in response to receiving first group of one or more input, for virtually being marked first It is added to the expression of the physical space, wherein the described first virtual mark is linked to the expression of the physical space A part；

The device being activated after the described first virtual mark is added to the expression of the physical space, is used for Receive second group of one or more input associated with the expression of the physical space；And

In response to receiving the associated second group of one or more input of the expression with the physical space And the device being activated, described device include:

Corresponded in the expression of the physical space according to determination second group of one or more input and is created The device being activated away from the described first virtual request virtually marked for marking within the threshold range, in the physical space The expression in the virtual mark of creation second, while keeping in the expression of the physical space the described first virtual mark Note；And

Corresponded in the expression of the physical space according to determination second group of one or more input and is created The device for virtually marking the request virtually marked outside the threshold distance away from described first and being activated, in the physics Creation second virtually marks and removes first void from the expression of the physical space in the expression in space Quasi- mark.

128. one kind for include one or more input equipments, one or more display equipment and

Information processing unit used in the electronic equipment of one or more cameras, comprising:

129. a kind of electronic equipment, comprising:

One or more input equipments；

One or more display equipment；

One or more cameras；

One or more processors；And

Store the memory of one or more programs, wherein one or more of programs be configured as by one or Multiple processors execute, and one or more of programs include for executing appointing in the method according to project 115 to 124 The instruction of one method.

130. a kind of computer readable storage medium for storing one or more programs, one or more of programs include Instruction, described instruction is by including one or more input equipments, one or more display equipment and one or more cameras Electronic equipment execute when so that the electronic equipment execute the method according to project 115 to 124 in either method.

131. the graphic user interface in a kind of electronic equipment, the electronic equipment include one or more input equipments, One or more shows equipment, one or more camera, memories and for executing be stored in the memory one The one or more processors of a or multiple programs, the graphic user interface include the method according to project 115 to 124 In either the user interface that shows of method.

132. a kind of electronic equipment, comprising:

One or more input equipments；

One or more display equipment；

One or more cameras；And

For either executing in the method according to project 115 to 124 device of method.

133. one kind is for including one or more input equipments, one or more display equipment and one or more Information processing unit used in the electronic equipment of camera, comprising:

134. a kind of method, comprising:

There are one or more input equipments, one or more display equipment, and the electronics of one or more cameras At equipment:

Show that mark places user interface via one or more of display equipment, the mark places user interface packet It includes:

The expression of physical space；And

User interface element is placed, the placement user interface element instruction is empty in response to detecting that mark places input Quasi- mark will be placed on the position in the expression of the physical space；

When showing that the mark places user interface, detection it is one or more of it is magazine at least one relative to The movement of the physical space, wherein when right in the expression that the placement user interface element is shown in the physical space When at the position of the first part of physical space described in Ying Yu, at least one one or more of magazine movements Start；And

In response to detecting at least one one or more of magazine described shifting relative to the physical space It is dynamic, the placement user interface element is moved in the expression of the physical space the corresponding to the physical space The position of two parts, the second part are different from the first part of the physical space, and according to one or At least one multiple magazine described movement relative to the physical space update the mark and place the outer of user interface It sees, comprising:

Its in the second part of the physical space can not be identified in the object according to the determination electronic equipment The object virtually marked can be linked to by managing the corresponding objects in the expression in space, stop showing the placement user interface At least part of element；And

Identify its in the second part of the physical space in the physics according to the determination equipment Corresponding objects in the expression in space can be linked to the object virtually marked, keep the placement user interface element Display.

135. according to method described in project 134, wherein in response to detect it is one or more of it is magazine at least One movement relative to the physical space:

According to determination it is described place user interface element be located in the expression of the physical space with the physics Away from the corresponding position of the first object of one of one or more of cameras first distance in space, institute is shown with the first size State placement user interface element；And

According to determination it is described place user interface element be located in the expression of the physical space with the physics Away from the corresponding position of the second object of one of one or more of cameras second distance in space, institute is shown with the second size Placement user interface element is stated, wherein the first distance, which is greater than the second distance and first size, is less than described the Two sizes.

136. according to method described in project 135, wherein the user interface element of placing is in the institute of the physical space First size in indicating is stated relative to first object in the physical space than the placement user interface member Second size of the element in the expression of the physical space is relative to second object in the physical space It is bigger.

137. the method according to any one of project 134 to 136, wherein the placement user interface element includes the A part and second part, and the method includes, in response to detect it is one or more of it is magazine at least one The movement relative to the physical space:

It is located in the expression of the physical space according to the determination placement user interface element empty with the physics Between in predefined type the corresponding position of feature, update it is described place user interface element appearance so that described put The second part for setting user interface element is amplified relative to the first part for placing user interface element；With And

It is located in the expression of the physical space according to the determination placement user interface element empty with the physics Between in predefined type the not corresponding position of feature, keep it is described place user interface element display without being put described The second part of user interface element is set relative to the first part amplification for placing user interface element.

138. the method according to any one of project 134 to 136, wherein the placement user interface element includes the A part and second part, and the method includes, in response to detect it is one or more of it is magazine at least one The movement relative to the physical space:

It is located in the expression of the physical space according to the determination placement user interface element empty with the physics Between in predefined type the corresponding position of feature, update it is described place user interface element appearance so that described put The second part of user interface element is set relative to the first part displacement for placing user interface element；And

It is located in the expression of the physical space according to the determination placement user interface element empty with the physics Between in predefined type the not corresponding position of feature, keep it is described place user interface element display without being put described The second part of user interface element is set relative to the first part displacement for placing user interface element.

139. the method according to any one of project 134 to 138, including it is one or more of in response to detecting At least one magazine movement relative to the physical space, can not identify the physical space according to the determination equipment The second part in its corresponding objects in the expression of the physical space can be linked to and virtually mark Object, shows the prompt containing information, and the information indicates that the electronic equipment can not identify described the of the physical space Its corresponding objects in the expression of the physical space in two parts can be linked to the object virtually marked.

140. the method according to any one of project 134 to 138, including it is one or more of in response to detecting At least one magazine movement relative to the physical space, can not identify the physics according to the determination electronic equipment Its corresponding objects in the expression of the physical space in the second part in space can be linked to virtual mark The object of note, shows the prompt containing information, and the information indicates that the electronic equipment can not identify the institute of the physical space Its corresponding objects in the expression of the physical space stated in second part can be linked to the object virtually marked The reason of.

141. methods according to project 140, wherein indicating that the electronic equipment can not identify the physical space Corresponding objects of its in the second part in the expression of the physical space can be linked to the object virtually marked The reason of the information include one of the following or multiple: about the instruction for needing more light, about one or more At least one a magazine instruction for being moving too quickly, about need to move it is one or more of it is magazine at least one Position the instruction on the surface in the physical space, about it is one or more of it is magazine at least one need it is mobile Further from the instruction of the object in the physical space, and about it is one or more of it is magazine at least one need to move Dynamic the instruction closer to the object in the physical space.

142. methods according to any one of project 134 to 141, including, when the table in the physical space When showing the display placement user interface element at the position corresponding to the second part of the physical space:

Input is placed in detection；And

In response to detecting the placement input, at least part virtually marked is placed on to the institute of the physical space It states and corresponds at the position for placing user interface element in indicating.

143. methods according to any one of project 134 to 141, including, when the table in the physical space When showing the display placement user interface element at the position corresponding to the second part of the physical space:

Detection corresponds to the input at the position for placing user interface element；And

In response to detecting the input corresponded at the position for placing user interface element, in the mark Note places the figure instruction for showing that the user interface element different from one is adjacent in user interface, and the figure indicates described in instruction The activation of different user interface elements will lead at least part virtually marked and be placed on the described of the physical space Correspond at the position for placing user interface element in expression.

144. a kind of electronic equipment, comprising:

One or more input equipments；

One or more display equipment；

One or more cameras；

One or more processors；And

The expression of physical space；And

Identify its in the second part of the physical space described according to the determination electronic equipment Corresponding objects in the expression of physical space can be linked to the object virtually marked, keep the placement user interface member The display of element.

A kind of 145. computer readable storage mediums for storing one or more programs, one or more of programs include Instruction, described instruction is by including one or more input equipments, one or more display equipment and one or more cameras Electronic equipment execute when so that the electronic equipment:

The expression of physical space；And

146. a kind of electronic equipment, comprising:

One or more input equipments；

One or more display equipment；

One or more cameras；

For placing the device of user interface via one or more of display equipment display marks, the mark is placed User interface includes:

The expression of physical space；And

The device enabled when showing that the mark places user interface, it is one or more of magazine for detecting At least one movement relative to the physical space, wherein when the placement user interface element is shown in the physical space The expression in correspond to the physical space first part position at when, it is one or more of it is magazine at least One movement starts；And

In response to detecting at least one one or more of magazine described shifting relative to the physical space Dynamic and enabling device, corresponds to for the placement user interface element to be moved in the expression of the physical space The position of the second part of the physical space, the second part are different from the first part of the physical space, and And for according at least one one or more of magazine described movement relative to the physical space, described in update Mark places the appearance of user interface, comprising:

Its in the second part of the physical space can not be identified in the object according to the determination electronic equipment The device that the corresponding objects in the expression in space can be linked to the object virtually marked and enable is managed, for stopping display Described at least part for placing user interface element；And

Identify its in the second part of the physical space described according to the determination electronic equipment Corresponding objects in the expression of physical space can be linked to the object virtually marked and the device that enables, for keeping State the display for placing user interface element.

147. is a kind of for including one or more input equipments, one or more display equipment and one or more Information processing unit used in the electronic equipment of camera, comprising:

The expression of physical space；And

148. a kind of electronic equipment, comprising:

One or more input equipments；

One or more display equipment；

One or more cameras；

One or more processors；And

Store the memory of one or more programs, wherein one or more of programs be configured as by one or Multiple processors execute, and one or more of programs include for executing appointing in the method according to project 134 to 143 The instruction of one method.

A kind of 149. computer readable storage mediums for storing one or more programs, one or more of programs include Instruction, described instruction is by including one or more input equipments, one or more display equipment and one or more cameras Electronic equipment execute when so that the electronic equipment execute the method according to project 134 to 143 in either method.

Graphic user interface in 150. a kind of electronic equipment, the electronic equipment include one or more input equipments, One or more shows equipment, one or more camera, memories and for executing be stored in the memory one The one or more processors of a or multiple programs, the graphic user interface include the method according to project 134 to 143 In either the user interface that shows of method.

151. a kind of electronic equipment, comprising:

One or more input equipments；

One or more display equipment；

One or more cameras；And

For either executing in the method according to project 134 to 143 device of method.

152. it is a kind of for include one or more input equipments, one or more display equipment and

Claims

1. a kind of method, comprising:

In the electronic equipment with one or more input equipments, one or more display equipment and one or more cameras Place:

Show that mark places user interface via one or more of display equipment, the mark places user interface and includes:

The expression of physical space；And

User interface element is placed, the placement user interface element instruction is virtual to mark in response to detecting that mark places input Note will be placed on the position in the expression of the physical space；

When showing that the mark places user interface, detection it is one or more of it is magazine at least one relative to described The movement of physical space, wherein corresponding to when in the expression that the placement user interface element is shown in the physical space When at the position of the first part of the physical space, at least one one or more of magazine movements are opened Begin；And

It, will in response to detecting at least one one or more of magazine described movement relative to the physical space The placement user interface element is moved to second in the expression of the physical space corresponding to the physical space The position divided, the second part are different from the first part of the physical space, and according to one or more of At least one magazine described movement relative to the physical space updates the appearance that the mark places user interface, Include:

According to the determination electronic equipment can not identify in the second part of the physical space it is empty in the physics Between the expression in corresponding objects can be linked to the object virtually marked, stop showing the placement user interface element At least part；And

Identify its in the second part of the physical space in the physics according to the determination electronic equipment Corresponding objects in the expression in space can be linked to the object virtually marked, keep the placement user interface element Display.

2. according to the method described in claim 1, wherein in response to detect it is one or more of it is magazine at least one The movement relative to the physical space:

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element The corresponding position of the first object away from one of one or more of cameras first distance, shows the placement with the first size User interface element；And

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element The corresponding position of the second object away from one of one or more of cameras second distance, shows the placement with the second size User interface element, wherein the first distance is greater than the second distance and first size is less than described second largest It is small.

3. according to the method described in claim 2, wherein the user interface element of placing is in the table of the physical space First size shown exists relative to first object in the physical space than the placement user interface element Second size in the expression of the physical space is bigger relative to second object in the physical space.

4. method according to any one of claim 1-3, wherein the placement user interface element includes first part And second part, and the method includes, in response to detect it is one or more of it is magazine at least one relative to The movement of the physical space:

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element Predefined type the corresponding position of feature, update it is described place user interface element appearance so that placements use The second part of family interface element is amplified relative to the first part for placing user interface element；And

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element Predefined type the not corresponding position of feature, keep it is described place user interface element display without by placements use The second part of family interface element is relative to the first part amplification for placing user interface element.

5. method according to any one of claim 1-3, wherein the placement user interface element includes first part And second part, and the method includes, in response to detect it is one or more of it is magazine at least one relative to The movement of the physical space:

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element Predefined type the corresponding position of feature, update it is described place user interface element appearance so that placements use The second part of family interface element is relative to the first part displacement for placing user interface element；And

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element Predefined type the not corresponding position of feature, keep it is described place user interface element display without by placements use The second part of family interface element is relative to the first part displacement for placing user interface element.

6. method according to any one of claims 1-5, including in response to detecting in one or more of cameras At least one movement relative to the physical space, the physical space can not be identified according to the determination electronic equipment Its corresponding objects in the expression of the physical space in the second part can be linked to pair virtually marked As showing the prompt containing information, the information indicates that the electronic equipment can not identify described the second of the physical space Its corresponding objects in the expression of the physical space in part can be linked to the object virtually marked.

7. method according to any one of claims 1-5, including in response to detecting in one or more of cameras At least one movement relative to the physical space, the physical space can not be identified according to the determination electronic equipment Its corresponding objects in the expression of the physical space in the second part can be linked to pair virtually marked As showing the prompt containing information, the information indicates that the electronic equipment can not identify described the second of the physical space The reason of its corresponding objects in the expression of the physical space in part can be linked to the object virtually marked.

8. according to the method described in claim 7, wherein indicating that the electronic equipment can not identify the described of the physical space Corresponding objects of its in second part in the expression of the physical space can be linked to the original of the object virtually marked The information of cause includes one of the following or multiple: about the instruction for needing more light, about one or more of phases The instruction that at least one of machine is moving too quickly, about need to move it is one or more of it is magazine at least one determine The instruction on the surface in the physical space of position, it is fartherly mobile about at least one one or more of magazine needs Instruction from the object in the physical space, and mobile about at least one one or more of magazine needs Closer to the instruction of the object in the physical space.

9. method according to claim 1 to 8, including, when corresponding in the expression in the physical space When showing the placement user interface element at the position of the second part of the physical space:

Input is placed in detection；And

In response to detecting the placement input, at least part virtually marked is placed on to the table of the physical space Correspond at the position for placing user interface element in showing.

10. method according to claim 1 to 8, including, when the expression on pair in the physical space When showing the placement user interface element at the position of the second part of physical space described in Ying Yu:

In response to detecting the input corresponded at the position for placing user interface element, put in the mark The adjacent figure instruction of display is different from one in user interface user interface element is set, the figure instruction indicates the difference The activation of user interface element will lead to the expression that at least part virtually marked is placed on the physical space In correspond to it is described place user interface element position at.

11. a kind of computer readable storage medium for storing one or more programs, one or more of programs include instruction, Described instruction is by including one or more input equipments, one or more electronics for showing equipment and one or more cameras When equipment executes, so that the electronic equipment perform claim requires method described in any one of 1-10.

12. one kind is for including one or more input equipments, one or more display equipment and one or more cameras Electronic equipment used in information processing unit, comprising:

For placing the device of user interface via one or more of display equipment display marks, the mark places user Interface includes:

The expression of physical space；And

The device that enables when showing that the mark places user interface, for detect it is one or more of it is magazine at least One movement relative to the physical space, wherein when the placement user interface element is shown in the institute of the physical space State indicate in correspond to the physical space first part position at when, it is one or more of it is magazine at least one The movement start；And

In response to detect it is one or more of it is magazine at least one relative to the physical space the movement and The device of enabling, for the placement user interface element to be moved in the expression of the physical space corresponding to described The position of the second part of physical space, the second part are different from the first part of the physical space, and use According at least one one or more of magazine described movement relative to the physical space, the mark is updated Place the appearance of user interface, comprising:

According to the determination electronic equipment can not identify in the second part of the physical space it is empty in the physics Between the expression in corresponding objects can be linked to the object virtually marked and the device that enables, for stopping described in display Place at least part of user interface element；And

Identify its in the second part of the physical space in the physics according to the determination electronic equipment Corresponding objects in the expression in space can be linked to the object virtually marked and the device that enables, put for keeping described Set the display of user interface element.

13. information processing unit according to claim 12, including in response to detecting in one or more of cameras At least one described movement relative to the physical space and the device that enables, comprising:

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element The corresponding position of the first object away from one of one or more of cameras first distance and the device enabled, for first Size shows the placement user interface element；And

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element The corresponding position of the second object away from one of one or more of cameras second distance and the device enabled, for second Size shows the placement user interface element, wherein the first distance is greater than the second distance and first size Less than second size.

14. information processing unit according to claim 13, wherein the placement user interface element is empty in the physics Between the expression in first size relative to first object in the physical space than the placement user Second size of the interface element in the expression of the physical space is relative to described in the physical space Two objects are bigger.

15. information processing unit described in any one of 2-14 according to claim 1, wherein the placement user interface element packet First part and second part are included, and the information processing unit includes, in response to detecting one or more of cameras At least one of relative to the physical space the movement and the device that enables, comprising:

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element Predefined type the corresponding position of feature and the device that enables, for updating the appearance for placing user interface element So that the second part for placing user interface element places described the first of user interface element relative to described Part is amplified；And

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element Predefined type the not corresponding position of feature and the device that enables, for keeping the display for placing user interface element Without the second part for placing user interface element is placed described the first of user interface element relative to described Amplify part.

16. information processing unit described in any one of 2-14 according to claim 1, wherein the placement user interface element packet First part and second part are included, and the information processing unit includes, in response to detecting one or more of cameras At least one of relative to the physical space the movement and the device that enables, comprising:

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element Predefined type the corresponding position of feature and the device that enables, for updating the appearance for placing user interface element So that the second part for placing user interface element places described the first of user interface element relative to described Transposition of partial；

And

It is located in the expression of the physical space and in the physical space according to the determination placement user interface element Predefined type the not corresponding position of feature and the device that enables, for keeping the display for placing user interface element Without the second part for placing user interface element is placed described the first of user interface element relative to described Transposition of partial.

17. information processing unit described in any one of 2-16 according to claim 1, including in response to detect it is one or It is multiple it is magazine at least one relative to the physical space movement and can not be identified according to the determination electronic equipment Its corresponding objects in the expression of the physical space in the second part of the physical space can link The device that enables to the object virtually marked, for showing the prompt containing information, the information indicates the electronic equipment It can not identify its corresponding objects in the expression of the physical space in the second part of the physical space The object virtually marked can be linked to.

18. information processing unit described in any one of 2-16 according to claim 1, including in response to detect it is one or It is multiple it is magazine at least one relative to the physical space movement and can not be identified according to the determination electronic equipment Its corresponding objects in the expression of the physical space in the second part of the physical space can link The device that enables to the object virtually marked, for showing the prompt containing information, the information indicates the electronic equipment It can not identify its corresponding objects in the expression of the physical space in the second part of the physical space The reason of object virtually marked can be linked to.

19. information processing unit according to claim 18, wherein indicating that the electronic equipment can not identify the physics Its corresponding objects in the expression of the physical space in the second part in space can be linked to virtual mark The information of the reason of object of note includes one of the following or multiple: about the instruction for needing more light, about described At least one one or more magazine instruction for being moving too quickly, it is one or more of magazine about needing to move At least one instruction to position the surface in the physical space, about it is one or more of it is magazine at least one need Move further from the object in the physical space instruction, and about one or more of magazine at least one It is a to need mobile the instruction closer to the object in the physical space.

20. information processing unit described in any one of 2-19 according to claim 1, including, when the institute in the physical space It states and is opened when showing the placement user interface element at the position for corresponding to the second part of the physical space on indicating Device, comprising:

For detecting the device for placing input；And

It is described for at least part virtually marked to be placed in response to detecting the device placing input and enabling Correspond at the position for placing user interface element in the expression of physical space.

21. information processing unit described in any one of 2-19 according to claim 1, including, when the institute in the physical space It states and is opened when showing the placement user interface element at the position for corresponding to the second part of the physical space on indicating Device, comprising:

For detecting the device for corresponding to the input at the position for placing user interface element；

And

The device enabled in response to detecting the input corresponded at the position for placing user interface element, For placing the figure instruction for showing that the user interface element different from one is adjacent in user interface, the figure in the mark Indicate that the activation of the different user interface element will lead at least part virtually marked and be placed on the object It manages and corresponds at the position for placing user interface element in the expression in space.

22. a kind of electronic equipment, comprising:

One or more input equipments；

One or more display equipment；

One or more cameras；And

Information processing unit described in any one of 2-21 according to claim 1.