CN104871214B - For having the user interface of the device of augmented reality ability - Google Patents

Abstract

The present invention discloses a kind of method and apparatus for being used to show augmented reality content.Methods described can be included：Camera is controlled to scan the environment in the visual field of user；Recognized one group of surface in the environment for display user interface form according to the characteristic of the environment；The priority of one group of augmented reality content for display is arranged relative to described group of surface in environment；And described group of augmented reality content be shown on described group of surface in the display.The characteristic of the environment at least includes：The ratio of width to height of described group of surface relative to described group of augmented reality content to be shown；And/or described group of surface is relative to the backcolor of described group of augmented reality content to be shown.

Description

For having the user interface of the device of augmented reality ability

The cross reference of related application

Present application advocates application on December 18th, 2012 and entitled " for having the user of the device of augmented reality ability No. 13/718,923 U.S. of interface (User Interface for Augmented Reality Enabled Devices) " The rights and interests and priority of state's application case, the application case transfers the present assignee, and is incorporated herein by reference.

Technical field

The present invention relates to the field of augmented reality.Exactly, the present invention relates to for having the device of augmented reality ability User interface.

Background technology

Head mounted display can be configured to allow the world that user is seen before it, while still being able to read its electricity Sub- mail, obtain stock quotation etc..However, the common issue of existing apparatus, which is the existing apparatus, can cause information overload, this It is because user has to inspect real world and also have to change focus to inspect AR contents.Another problem of existing apparatus Content to be shown is not based on for the existing apparatus to enter the Free Region in the visual field of user row major order row Row and tissue.Fig. 1 illustrates the example according to the shown information of some conventional applications.As shown in fig. 1, various advertisements are placed In on indicator screen.Some advertisements (for example, 152,154 and 156) in the advertisement may stop some portions of scene Divide and be allowed to not be seen by a user.Therefore, conventional application is not yet while allowing users to interact naturally with its surrounding Effectively by augmented reality content integration into environment, so as to negatively affect the Consumer's Experience of head mounted display.

Accordingly, it would be desirable to which the method and device of the problem above of conventional solution can be solved.

The content of the invention

The present invention relates to for having the user interface of the device of augmented reality ability.Embodiments in accordance with the present invention, it is a kind of The method of display augmented reality content includes：Camera is controlled to scan the environment in the visual field of user；According to the environment Characteristic come recognize one group of surface in the environment for display user interface form；Relative to described group of table in environment Face come arrange for display one group of augmented reality content priority；And in the display show described group of augmented reality content It is shown on described group of surface.The characteristic of the environment at least includes：Described group of surface is real relative to described group of amplification to be shown The ratio of width to height held within the border；And/or described group of surface is relative to the backcolor of described group of augmented reality content to be shown.

In some embodiments, arranged relative to described group of surface in environment in one group of augmented reality for display The method of the priority of appearance includes the priority that the described group of augmented reality content for display is arranged based on the following： The area on described group of surface, the orienting of described group of surface, position of the described group of surface relative to the visual field of user and described group of table The trackability in face.

In some of the other embodiments, one group of amplification reality for display is arranged relative to described group of surface in environment The method for the priority held within the border further comprises arranging described group for display based at least one of the following The priority of augmented reality content：The ratio of width to height of described group of surface relative to described group of augmented reality content to be shown；And Backcolor of the described group of surface relative to described group of augmented reality content to be shown.

In other other embodiments, one group of amplification for display is arranged relative to described group of surface in environment The method of the real priority held within the border further comprises the history of the previously used pattern according to user in the environment to determine For one group of augmented reality content of display.It is determined that the method for one group of augmented reality content of display is included in the following extremely Few one：Working to select first group of augmented reality content for display in response to determination user；In response to determining user Second group of augmented reality content for display is just selected at home；Just it is being in business meetings to select in response to determination user For the 3rd group of augmented reality content of display；And be just in social event to select for display the 4th in response to determination user Group augmented reality content.

It should be noted that first group of augmented reality content comprising user communicate being used for of getting used to operationally using and file at The application program of reason, and second group of augmented reality content comprising user being used for of getting used to using at home communicate and entertain should Use program.First group of augmented reality content for communication and file process includes Email, web browser and office Productivity ratio (office productivity) application program, and included for the second group of augmented reality content for communicating and entertaining Facebook, Twitter, film and video game application programs.

In other other embodiments, one group of amplification for display is arranged relative to described group of surface in environment The method of the real priority held within the border further comprises arranging based at least one of the following for described in display The priority of group augmented reality content：The scale factor of described group of augmented reality content to be shown；And it is to be shown described The logical relation of group augmented reality content.

In other other embodiments, one group of amplification for display is arranged relative to described group of surface in environment The method of the real priority held within the border further comprises：One group of amplification for display is determined according to predefined user preference Real domestic appearance；And described group of augmented reality content be shown on described group of surface in the display.The predefined user Preference includes at least one of the following：Importance of the augmented reality content to user；Augmented reality content to be shown Amount；And augmented reality content inspects orientation relative to described group of surface.

In another embodiment, computer program product includes the computer that storage is performed for one or more computer systems The non-transitory media of program；The computer program product includes：Scanned for controlling camera in the visual field of user The code of environment；Recognize that one in the environment group of surface connects for display user for the characteristic according to the environment The code of mouth form；For arranging the preferential of one group of augmented reality content for supplying to show relative to described group of surface in environment The code of order；For the code being in the display shown in described group of augmented reality content on described group of surface.

In another embodiment, equipment includes at least one processor, wherein at least one described processor includes：Through with Put to control camera to scan the logic of the environment in the visual field of user；It is configured to be recognized according to the characteristic of the environment One group of surface in the environment is for showing the logic of user interface form；It is configured to relative to described in environment The logic for the priority for organizing surface to arrange one group of augmented reality content for display；And be configured to institute in the display State the logic that group augmented reality content is shown on described group of surface.

In another embodiment, for showing that the system of augmented reality content includes：Scanned for controlling camera with The device of environment in the visual field at family；Recognized for the characteristic according to the environment one group of surface in the environment with In the device of display user interface form；For arranging one group of amplification reality for display relative to described group of surface in environment The device for the priority held within the border；And for described group of augmented reality content to be shown in into described group of surface in the display Device.

Brief description of the drawings

This hair can be more clearly understood from after the detailed description of figures below reading embodiments of the invention is combined Bright features described above and advantage and its additional features and advantage.

Fig. 1 illustrates the example according to the shown information of some conventional applications.

Fig. 2 illustrates the device of tool augmented reality ability according to certain aspects of the invention.

Fig. 3 illustrates the block diagram of the device of exemplary tool augmented reality ability according to certain aspects of the invention.

Fig. 4 illustrates the device of another exemplary tool augmented reality ability according to certain aspects of the invention.

Fig. 5 illustrates the block diagram of the device of the tool augmented reality ability of Fig. 4 according to certain aspects of the invention.

Fig. 6 illustrates that the processing quality for the device for having augmented reality ability according to certain aspects of the invention connects Mouthful.

Fig. 7 illustrates the exemplary method of display augmented reality content according to certain aspects of the invention.

Embodiment

Disclose for having the embodiment of the user interface of the device of augmented reality ability.Following description is presented so that affiliated The technical staff in field can make and using the present invention.Description to specific embodiment and application is provided only as example. The various modifications and combination of example described herein will be easily by those skilled in the art it is clear that and can be not Generic principles defined herein is applied to other examples and application in the case of the spirit and scope of the present invention.Cause This, the present invention is not intended to be limited to example that is described and showing, but will meet and principle disclosed herein and spy Levy consistent widest scope.Word " exemplary " or " example " are herein to mean " serving as example, example or explanation ".This Any aspect or embodiment for being described as " exemplary " or " example " in text are not necessarily to be construed as than other side or implementation Example is preferably or favourable.

Fig. 2 illustrates the device of tool augmented reality ability according to certain aspects of the invention.As shown in Figure 2, tool amplification The device (ARD) 200 of real border ability includes shell 101, display 102, one or more loudspeakers 104 and microphone 106.Display The explainable image captured by camera 105 of device 102 (it can be touch-screen display), or any other desired user interface letter Breath.Certainly, ARD 200 can comprising may not be related to the present invention additional assemblies.

As used herein, ARD devices refer to any portable electron device, such as honeycomb fashion or other radio communications Device, PCS Personal Communications System (PCS) device, personal navigation apparatus (PND), personal information manager (PIM), personal digital assistant (PDA), on knee or other suitable mobile platforms.Mobile platform can receive wireless communication signals and/or navigation signal (for example, navigator fix signal).Term ARD is also meant to include for example by short-distance radio, infrared ray, wired connection or other companies The device for connecing and being communicated with personal navigation apparatus (PND), this is related to satellite signal receiving, assistance data reception and/or position Processing occurs unrelated at described device or at PND.Made moreover, ARD is intended to include to be trapped in posture tracking Image and all electronic installations for being able to carry out augmented reality user interface function, comprising radio communication device, are calculated Machine, laptop computer, tablet PC, smart phone, digital camera etc..

Fig. 3 illustrates the block diagram of the device of exemplary tool augmented reality ability according to certain aspects of the invention.ARD 300 Mobile platform include be used for capture environment image camera 108, described image can be indivedual photos or frame of video.ARD 300 mobile platform can also include sensor 116, and the mobile platform that the sensor 116 can be used to provide ARD 300 can be so as to Determine its position and orient the data of (that is, posture).The example bag for the sensor that can be used together with ARD 300 mobile platform Containing accelerometer, quartz transducer, gyroscope, MEMS (MEMS) sensor for being used as linear accelerometer, and magnetic force Meter.

ARD 300 mobile platform can also include user interface 110, and the user interface 110 is included being capable of display image Display 112.User interface 110 can also can be so as to entering information into ARD 300 movement comprising keypad 114 or user Other input units in platform.If desired, can be by the way that virtual keypad be integrated into the display with touch sensing Keypad 114 is excluded in 112.User interface 110 can also be comprising microphone 117 and one or more loudspeakers 118 (for example, such as Fruit mobile platform is cellular phone).Certainly, ARD 300 mobile platform can include other components unrelated to the invention.

ARD 300 mobile platform additionally comprises control unit 120, and described control unit 120 is connected to the following simultaneously Communicated with the following：Camera 108 and sensor 116, and user interface 110, together with any other desired characteristics.It can pass through One or more processors 122 and/or associated memory/memory 124 provide control unit 120.Control unit 120 is also Software 126 and hardware 128 and firmware 130 can be included.Control unit 120 includes tracking module 132, the tracking module 132 It is configured to track ARD 300 position and tracks by the position of one or more objects monitored of ARD 300.Control unit 120 further include augmented reality Subscriber Interface Module SIM 134, and the augmented reality Subscriber Interface Module SIM 134 is configured to expand Increase real border interaction to be presented on ARD 300 display 112.For clarity, tracking module 132 and augmented reality user interface Module 134 separates to be illustrated with processor 122 and/or hardware 128, but can be based in software 126 and firmware 130 Tracking module 132 and augmented reality Subscriber Interface Module SIM 134 are combined and/or are implemented on processor 122 and/or hardware by instruction In 128.

Fig. 4 illustrates the device 400 of another exemplary tool augmented reality ability according to certain aspects of the invention.Such as Fig. 4 Shown in, the device 400 of tool augmented reality ability can be wear-type device.According to aspects of the present invention, wear-type device can be through Configuration is to learn the conventional use pattern and preference of user, and the time that may be in based on user and environment are automatically adjusted It is applicable family interface window.For example, wear-type device can be configured with the previously used mould according to user in a certain environment The history of formula determines one group of augmented reality content for display, and then correspondingly shows described group of augmented reality content.

In one approach, wear-type device can be configured to work in response to determination user to select for display First group of augmented reality content.In another approach, wear-type device can be configured with response to determining user just at home To select second group of augmented reality content for display.In another method, wear-type device can be configured with response to true Determine user be just in business meetings to select the 3rd group of augmented reality content for display.In another method, wear-type Device can be configured to be just in social event in response to determination user to select the 4th group of augmented reality content for display.

According to aspects of the present invention, what first group of augmented reality content can be got used to operationally using comprising user is used for Communication and the application program of file process, and second group of augmented reality content being used for of can getting used to using at home comprising user Communication and the application program of amusement.First group of augmented reality content for communication and file process can include Email, net Page browsing device and office's productivity application, and can be included for the second group of augmented reality content for communicating and entertaining Facebook, Twitter, film and video game application programs.

Fig. 5 illustrates the block diagram of the device of the tool augmented reality ability of Fig. 4 according to certain aspects of the invention.In some realities Apply in scheme, wear-type device 400 can be operated as the part of a system, in the system, and sensor array 500 can Provide data to processor 507, the processor 507 performs the operation of various embodiments described herein and by data It is communicated to server and receives data from server.It should be noted that the processor 507 of wear-type device 400 can include more than one Processor (or multi-core processor), wherein executable whole control functions of core processor, and coprocessor performs application Program (sometimes referred to as application processor).Core processor and application processor can be configured in identical microchip package In (for example, multi-core processor) or in separated chip.Moreover, processor 507 can be with being associated with the processors of other functions It is packaged in identical microchip package, other functions are for example：Radio communication (that is, modem processor), navigation (for example, processor in gps receiver) and graphics process (for example, graphics processing unit or " GPU ").

Wear-type device 400 can be with that can include other computing devices (for example, can access personal computer and the shifting of internet Dynamic device) communication system or network service.This little personal computer and mobile device can include antenna 551, transmitter/reception Device or transceiver 552 and A/D converter 553, the A/D converter 553 is coupled to processor 507 so that the processor Data can be sent and received via cordless communication network.For example, mobile device (for example, cellular phone) can be via Cordless communication network (for example, Wi-Fi or cellular telephone data communication) accesses internet.This little cordless communication network Can be comprising being coupled to multiple base stations of gateway or be coupled to the internet switch-on server of internet.Personal computer can be with any Conventional mode is coupled to internet, such as by the wired connection reached via internet gateway (not shown) or by wireless Communication network.

Referring to Fig. 5, wear-type device 400 can include the scene sensor 500 and sound for being coupled to control system processor 507 Video sensor 505, the control system processor 507 may be configured with some module 510-550.In one embodiment, processor 507 or scene sensor 500 anatomical features identification algorithm can be applied to image to detect one or more anatomical features.With control The associated processor 507 of system can inspect detected anatomical features to recognize one or more dumb shows and will be distinguished The dumb show of knowledge is processed as input order.For example, following article is discussed in more detail, and user is executable to correspond to input The mobile dumb show of order, for example, point to virtual objects to close the virtual objects by finger.In response to picking out this reality Example dumb show, processor 507 can remove the virtual objects from display.As another example, user can be by one on hand Forefinger touch thumb and form the instruction for confirmation being presented on display of " OK " symbol or option.

Scene sensor 500 (its can comprising stereoscopic camera, orientation sensor (for example, accelerometer and electronic compass) and away from From sensor) data (for example, image) related to scene can be provided to the scene manager being implemented in processor 507 510, the scene manager 510 can be configured with interpreting three dimensional scene information.In various embodiments, scene sensor 500 Stereoscopic camera (as described below) and range sensor can be included, it, which can be included, is used for for the red of infrared camera light scene Outside line optical transmitting set.For example, in Figure 5 in embodiment described, scene sensor 500 can be included：Three-dimensional RGB (RGB) camera 503a, it is used to collect stereo-picture；And infrared camera 503b, it is configured to can be by the red of structuring Make scene imaging in the infrared ray that outside line optical transmitting set 503c is provided.As discussed below, the infrared ray transmitting of structuring Device can be configured to launch the pulse for the infrared ray that can be imaged by infrared camera 503b, wherein the time of the pixel received It is recorded and determines the distance away from image component to be calculated using the flight time.Can be by three-dimensional RGB camera 503a, infrared ray Camera 503b and RF transmitter 503c is referred to collectively as RGB-D (D is distance) camera 503.

Scene manager module 510 can scan the range measurement provided by scene sensor 500 and image, to produce figure The three-dimensional reconstruction thing of object as in, includes the distance away from stereoscopic camera and surface orientation information.In one embodiment, scene is passed Sensor 500 (and more particularly, RGB-D cameras 503) may point to the direction being aligned with the visual field of user and wear-type device 400 On.Scene sensor 500 can provide whole body three-dimensional motion capture and dumb show identification.Scene sensor 500 can have with it is red The infra-red light emitter 503c of outside line camera 503c (such as monochromatic cmos sensor) combination.Scene sensor 500 can enter one Step includes the stereoscopic camera 503a of capture three dimensional video data.Scene sensor 500 can be in ambient light, sunshine or complete darkness Middle work, and RGB-D cameras as described in this article can be included.Scene sensor 500 can include near infrared ray (NIR) pulse Light fixture and the imaging sensor with quick door operating mechanisms.Collect the pulse signal of each pixel, and the pulse Signal correspond to pulse reflect from position and can be used to calculating the distance away from the corresponding points in captured main body.

In another embodiment, other distance measurement techniques (that is, different types of distance can be used in scene sensor 500 Sensor) capture the distance of the object in image, for example, ultrasonic echolocation, radar, the triangulation of stereo-picture Deng.As discussed above, in one embodiment, scene sensor 500 can include ranging camera, flashlight detection and ranging (LIDAR) camera, flight time (ToF) camera and/or RGB-D cameras 503, ranging gate ToF sensings, RF modulations can be used in it At least one of ToF sensings, pulsed light ToF sensings and projected light solid sensing determine the distance away from object.In another reality Apply in example, scene sensor 500 can be used stereoscopic camera 503a to capture the stereo-picture of scene and include based on described image The brightness of some captured pixels determines distance.As mentioned above, for uniformity, herein generally by this little type Distance measurement sensor and one or more of technology be referred to as " range sensor ".It may be present and have different abilities and resolution ratio Multiple scene sensors are to help to draw to physical environment, position and accurate tracking user pair of the accurate tracking user in environment The visual field of environment.

Wear-type device 400 can also include audio sensor 505, such as microphone or microphone array.Audio sensor 505 make wear-type device 400 be able to record that audio, and carry out auditory localization and ambient noise suppression.Audio sensor 505 can be captureed Obtain audio and audio signal is transformed into auditory digital data.The processor associated with control system can inspect digital audio number According to and application voice recognition algorithm so that the data are transformed into the text data that can search for.Processor can also inspect produced Text data with find it is some recognized order or keyword and order or keyword will have been recognized be used as inputting a command for performing One or more tasks.For example, user can say the order of such as " grappling virtual objects " with by shown virtual objects It is anchored on selected surface.For example, user can say " closing application program " to close the application being shown on display Program.

Wear-type device 400 can also include display 540.Display 540 can show the camera in scene sensor 500 Obtain or by the wear-type device 400 or being coupled to the image that the processor of wear-type device 400 is produced.In an embodiment In, display 540 can be micro-display.Display 540 can be full masking display (fully occluded display). In another embodiment, display 540 can be semi-transparent display, its can on screen display image, user can pass through the screen The surrounding of curtain viewing environments.Display 540 can be configured with simple eye or three-dimensional (that is, eyes) configuration.Or, wear-type shows It can be the helmet mount type display device for being worn on head or the part as the helmet to show device 400, and it, which can have, is located at one Eyes are (simple eye) anterior or positioned at the anterior optical element of miniscope 540 of two eyes (that is, eyes or three-dimensional display). Or, wear-type device 400 can also be comprising two display units 540, and the display unit 540 is through miniaturization and can be following Any one or more in person：Cathode-ray tube (CRT) display, liquid crystal display (LCD), liquid crystal over silicon (LCos) display Device, Organic Light Emitting Diode (OLED) display, based on interference modulator (IMOD) element (its be simple MEMS (MEMS) device) Mirasol displays, light-guide display and Waveguide display, and exist and can be developed it is other aobvious Show device technology.In another embodiment, display 540 may include multiple micro-displays 540 with increase total resolution and increase regard It is wild.

Wear-type device 400 can also include audio output device 550, the audio output device 550 can for earphone and/or Loudspeaker (by reference numeral 550 is jointly shown as) is to export audio.Wear-type device 400 can also include one or more processing Device, one or more described processors can by control function provide to wear-type device 400 and produce image (for example following article with The user interface form that Fig. 6 is associated and describes).For example, wear-type device 400 can be comprising core processor, using place Manage device, graphics processor and navigating processor.Or, head mounted display 400 can be coupled to one point of processor opened, such as position Processor in smart phone or other mobile computing devices.Video/sound can be handled by processor or by mobile CPU Frequency is exported, and the mobile CPU connections (via wired or wireless network) arrive wear-type device 400.Wear-type device 400 can also be wrapped Block containing scene manager 510, subscriber control block 515, surface manager block 520, audio manager block 525 and Information Access block 530, above those for separated circuit module or can be implemented in processor as software module.Wear-type device 400 It can further include local storage and for wirelessly or non-wirelessly network service wirelessly or non-wirelessly to connect with other devices or region Mouthful, to receive numerical data from remote memory 555.In systems can be by reducing in device using remote memory 555 Memory chip and circuit board and wear-type device 400 is become lighter.

The scene manager block 510 of controller can receive the virtual of data and construction physical environment from scene sensor 500 Represent.For example, laser can be used to transmitting from the object reflection in room and be trapped in magazine laser, wherein will The two-way time of light is to calculate away from each object and the distance on surface in room.This little range measurement can be used to determine room In object position, size and shape, and produce scene map.After establishment map, scene manager block 510 can be by Map is attached to other produced maps to form the larger map of presumptive area.In one embodiment, can by scene and away from Server or other computing devices are transmitted into from data, the server or other computing devices can be based on from some wear-types dress Put reception image, distance and map datum (and over time when user moves around in scene) produce merge or Integrated mapping.This integrated mapping data can be made to be available for wear-type device processor to use via wireless data link.Other maps It can be the map that is scanned by the present apparatus or by other wear-type devices, or cloud service can be received from.Scene manager 510 can Surface is recognized based on the data from scene sensor 500 and the current location of user is tracked.Subscriber control block 515 can be collected User's control to system inputs (for example, voice command, dumb show and input unit (for example, keyboard, mouse)).It is real one Apply in example, subscriber control block 515 can include dumb show dictionary or may be configured to access dumb show dictionary, to interpret by field The movement for the user's body part that scape manager 510 is recognized.It should be noted that dumb show dictionary can be stored for recognizing dumb show Mobile data or pattern, dumb show can be comprising gently stabbing, pat, touching, by, guiding, style of writing, rotation, rotation, crawl and drawing Dynamic, both hands palm is spread out for moving horizontally image, drawing (for example, the method for finger drawing), with finger type forming shape (for example, " OK " Symbol) and stir, above those all can be realized or connect closely on the apparent position of the virtual objects in produced display thing The nearly apparent position is realized.Subscriber control block 515 can also recognize interdictum duplex.This can include two or more orders. For example, dumb show and sound (for example, applause) or voice control command (for example, it is made and with voice command or mouth State word combination to confirm " OK " the hand dumb show detected of operation).When identifying user's control 515, controller It will can ask to provide another sub-component for arriving wear-type device 400.

Wear-type device 400 can also include surface manager block 520.Surface manager block 520 can be based on the figure captured Measure continuously to track the position on the surface in scene as (such as by scene manager block 510 is managed) and from range sensor Put.Surface manager block 520 continuously updated can also be anchored to the virtual objects on the surface in captured image Position.Surface manager block 520 can be responsible for surface and user interface form in effect.Audio manager block 525 can provide for The control instruction of audio input and audio output.Audio manager block 525 can construction be delivered to the sound of earphone and loudspeaker 550 Frequency flows.

Information Access block 530 can provide control instruction to mediate the access to digital information.Data can be stored in wear-type On local memory storage media on device 400.The teledata that data can be also stored on accessible digital device is deposited Store up on media 555, or data can be stored on distributing high in the clouds storage memory, distributing high in the clouds storage memory can be by Wear-type device 400 is accessed.Information Access block 530 communicates with data storage device 555, and the data storage device 555 can be Memory, disk, remote memory, cloud computing resource or integrated memory 555.

Fig. 6 illustrates that the processing quality for the device for having augmented reality ability according to certain aspects of the invention connects Mouthful.As shown in Figure 6, user interface can be configured with AR contents effectively to merge real-world objects.In a demonstration In property embodiment, ARD to camera carrys out scanning circumstance using preceding.In one approach, Qualcomm Vuforia^TMComputer is regarded Feel that application program can be used to detect flat surfaces.As shown in Figure 6, the flat surfaces can be included in user top visual field, Surface in bottom visual field, left view circle, right visual field and preceding visual field, and environment can include real-world objects, such as desk 602, Window 604, oil painting 606 and lamp 608.Can be then by user interface form (for example, 610,612,614,616,618,620,622 And 624) be placed in flat surfaces for display augmented reality content (for example, Email, PowerPoint demonstration, Facebook, push away literary (tweet), film etc.).

In some embodiments, if the number of augmented reality content is likely larger than recognized user interface form Number, then ARD can be configured to carry out prioritizing to the augmented reality content of the number to be shown and wipe out. In some of the other embodiments, ARD can be configured to merge or divide detected flat surfaces, so as to form different groups User interface form matches augmented reality content to be shown.

Embodiments in accordance with the present invention, ARD can be configured to arrange detected flat surfaces based on following criterion Priority, the criterion including (but not limited to)：A) degree that the feature of user interface form can be traced；B) surface Area/size；C) position of the surface relative to the user visual field (FOV)；4) orientation and the ratio of width to height on surface；And 5) the background on surface Color.In some embodiments, if surface has many colors and with high-frequency change, then a group echo may be selected So that even if his head is moved on to another visual field by user from a visual field, (or multiple) virtual user interface form still can " glutinous " On the surface.In addition, in the situation that user is just talking with people, surface in periphery than the surface being located in center compared with It is good, so as to talk of the user interface form without interference with user and people.In some applications, big region is preferable than zonule.

In some embodiments, depending on content to be shown, the surface with vertical orientation can be for display Word For file preferably, the surface with horizontal orientation can be for display film preferably.In addition, expansion to be shown can be made Increase when real domestic appearance is matched with user interface form and the ratio of width to height of each user interface form is used as criterion.For example, it is long And narrow level user interface window can be used to show stock ticker, and with 16:9 or 4:3 substantially the ratio of width to height User interface form can be used to show film.

In other embodiments, can be when making augmented reality content to be shown be matched with user interface form by surface The backcolor of (for example, wall) is used as criterion.For example, wall with grey wallpaper than simple white wall compared with It is good, because the simple white wall of the light of other light sources of the participant reflection from window or in environment is compared, it is shown in Augmented reality content on gray face can have bigger contrast.

According to aspects of the present invention, can be then by user after the priority on surface of base map to be used as has been arranged Interface window is characterized for display.Attribute is provided to user interface form based on following criterion, the criterion is not comprising (but It is limited to)：A) importance of a certain content to user；B) amount of content to be shown；And c) content to be shown inspects orientation And the ratio of width to height.

ARD can be next based on criterion as described above and user interface form is assigned into surface.For example, will can have The higher-priority user interface form for having many contents is placed in the high priority surface with large area.ARD can be configured Whether moved in the frame captured by forward direction camera with tracking a surface (because user has moved his head, or in reality Border surface (for example, bus side) by movement in the case of).Passed based on one or more six degree of freedoms used in ARD Sensor (for example, accelerometer, magnetometer and gyroscope) makes this determination.For example, if user's head has been moved on surface Remained when dynamic static, then the surface can be wiped out, and user interface form can be reassigned.According to aspects of the present invention, In one approach, ARD can be configured the figure so that Scale invariant features transform (SIFT) technology to be used to be captured by ARD camera As the available surface come in tracking environmental.In another approach, ARD can be configured with by consistent (RANSAC) skill of grab sample Art is used for the image captured by ARD camera come the available surface in tracking environmental.In another method, ARD can be configured With the surface come using non-vision method (such as available radio frequency identification (RFID) in use environment) in tracking environmental.

According to certain aspects of the invention, the virtual objects in scene continuously can be tracked and present (such as in grappling table On face) so that or even virtual objects still can behave as fixation when user rotates her head and moves around.It is as mentioned above And, ARD (for example, wear-type device) is oriented relative to surrounding and is determined away from the distance on each surface and each surface The program of relative orientation can continuously be realized by following operation：Monitor orientation sensor (for example, accelerometer and gyroscope) And the position and visual angle to described device carry out triangulation (when described device is moved in space, by constantly referring to The displacement of the landform of three-dimensional (3D) environment scanned), therefore perform (SLAM) operation that positions and draw simultaneously.

By using method as described above, as shown in Figure 6, user interface form 610 can be described in transverse mode Size on lower ground is 120 square feet of region, and it has relatively low priority and relatively low trackability.User The size that interface window 612 can describe on vertical pattern bottom right wall is 70 square feet of region, and it has relatively medium Priority and relatively high trackability.The size that user interface form 614 can be described on vertical pattern bottom right wall is 100 square feet of region, it has relatively medium priority and relatively high trackability.User interface form 616 can The region that size under transverse mode on ceiling is 100 square feet is described, it has relatively low priority and relative Low trackability.The size that user interface form 618 can describe under transverse mode in front wall is 80 square feet of area Domain, it has relatively high priority and relatively high trackability.User interface form 620 can be described in transverse mode bottom left Size on wall is 50 square feet of region, and it has relatively high priority and relatively high trackability.User connects The size that mouth form 622 can describe on vertical pattern bottom left wall is 100 square feet of region, and it has relatively medium Priority and relatively high trackability.The size that user interface form 624 can describe on transverse mode bottom left wall is 50 The region of square feet, it has relatively medium priority and relatively high trackability.

After the flat surfaces during ARD has identified environment, ARD can be referred to AR forms by the order of its corresponding priority Task the flat surfaces.For example, the Word that user is busy with recently before his meeting on his personal computer File can have higher-priority, and can be placed on the wall in meeting room before him.According to aspects of the present invention, ARD is contemplated that the size of AR forms to ensure itself and AR content matchings to be shown.For example, if file to be shown In font may be too small and be difficult to read, then the size of AR forms can be amplified.On the other hand, if the size pair of AR forms Greatly and eyes of user may be caused uncomfortable very much for file, then the size of AR forms can be reduced.In some embodiments, ARD can be using one group of scale factor, and it can be used to stretch or reduces AR forms.In one approach, the minimum scale factor and most The vast scale factor (such as resize_min and resize_max) can be used to adjust the size of AR forms.For example, for showing Show file representative value can based on predetermined AR forms size (such as 640x480VGA) resize_min be 0.5 (50%) with Resize_max is in the scope between 3.0 (300%).For another example, treating from Netflix application programs is broadcast The resize_min and resize_max for the film put can be respectively 0.5 (50%) and 10.0 (1,000%), and this indicates that film can It is reset into the size much larger than file.

In some embodiments, based on size, AR forms can be shown in given surface, can make determination to divide Flat surfaces are to support two or more AR forms.According to aspects of the present invention, if to be shown in the first AR forms AR contents may be without using whole available surface, then the allocated surface to the first AR forms can be further split into many Individual AR forms.On the other hand, if AR contents to be shown need larger display surface, then can merge multiple AR forms with Form new AR forms.

In some of the other embodiments, the excellent of AR contents can be determined by the logical relation between AR contents to be shown The first sequentially appointment of arrangement and AR contents to AR forms.ARD can be configured with two or more AR closer to each other Multiple logically related AR contents are shown in form.For example, ARD can be configured to show explanation section using AR forms The file of theory, and the video for the demonstration for showing the scientific theory is shown using neighbouring AR forms.It is used as another reality Example, ARD can be configured to be shown in using AR forms by the paper of announcement in discussion, and be shown using neighbouring AR forms Show that author can be used for the presentation slides that the paper is presented in discussion.

By user interface as described above, user can be read in augmented reality from the user interface form set up While appearance with its surrounding it is more natural interact because this little user interface form is not only placed in user Above.Truth is that according to aspects of the present invention, the user interface form is preferably fused to the nature in environment through storing In scene.Therefore, the efficiency and productivity ratio of user can be improved using disclosed user interface.

Fig. 7 illustrates the exemplary method of display augmented reality content according to certain aspects of the invention.In block 702, Processor 122 and/or augmented reality Subscriber Interface Module SIM 134 may be configured to control camera to scan in the visual field of user Environment.In block 704, processor 122 and/or augmented reality Subscriber Interface Module SIM 134 can be configured with according to the environment Characteristic come recognize one group of surface in the environment for display user interface form.In block 706, processor 122 and/ Or augmented reality Subscriber Interface Module SIM 134 can be configured to arrange one group for display relative to described group of surface in environment The priority of augmented reality content.In block 708, processor 122 and/or augmented reality Subscriber Interface Module SIM 134 can through with Put that described group of augmented reality content is shown on described group of surface in the display.

According to aspects of the present invention, environment includes real-world objects, and wherein described group surface is included in the top of user Surface in portion's visual field, bottom visual field, left view circle, right visual field and preceding visual field, and display is head mounted display.The spy of environment Property including (but not limited to)：The ratio of width to height of described group of surface relative to described group of augmented reality content to be shown；And described group Backcolor of the surface relative to described group of augmented reality content to be shown.

In embodiments in accordance with the present invention, block 706 performed method can additionally comprise block 710,712,714,716 and Performed method in 718.In block 710, processor 122 and/or augmented reality Subscriber Interface Module SIM 134 can be configured with base The priority of the described group of augmented reality content for display is arranged in the following：The area on described group of surface, described group The orienting of surface, described group of surface are relative to the position in the visual field of user and the trackability on described group of surface.

In block 712, processor 122 and/or augmented reality Subscriber Interface Module SIM 134 can be configured with based on the following At least one of come arrange for display described group of augmented reality content priority：Described group of surface is relative to be shown Described group of augmented reality content the ratio of width to height；And described group of surface is relative to the back of the body of described group of augmented reality content to be shown Scenery is color.

In block 714, processor 122 and/or augmented reality Subscriber Interface Module SIM 134 can be configured with according to user in ring The history of previously used pattern in border come determine for display one group of augmented reality content.In some embodiments, handle Device 122 and/or augmented reality Subscriber Interface Module SIM 134 can be configured with：In response to determining that it is aobvious to select to supply that user is working The first group of augmented reality content shown；In response to determine user just at home come select for display second group of augmented reality in Hold；Just it is being in business meetings to select the 3rd group of augmented reality content for display in response to determination user；Or in response to true Determine user be just in social event to select the 4th group of augmented reality content for display.

In some embodiments, first group of augmented reality content is led to being used for of getting used to operationally using comprising user The application program of letter and file process, and second group of augmented reality content communicate being used for of getting used to using at home comprising user And the application program of amusement.In addition, first group of augmented reality content for communication and file process includes Email, webpage Browser and office's productivity application, and included for the second group of augmented reality content for communicating and entertaining Facebook, Twitter, film and video game application programs.

In block 716, processor 122 and/or augmented reality Subscriber Interface Module SIM 134 can be configured with based on the following At least one of come arrange for display described group of augmented reality content priority：Described group of augmented reality to be shown The scale factor of content；And the logical relation of described group of augmented reality content to be shown.

In block 718, processor 122 and/or augmented reality Subscriber Interface Module SIM 134 can be configured with：According to predefined User preference come determine for display one group of augmented reality content；And in the display show described group of augmented reality content In on described group of surface.It should be noted that the predefined user preference includes at least one of the following：In augmented reality Hold the importance to user；The amount of augmented reality content to be shown；And augmented reality content is relative to the inspection on described group of surface Depending on orientation.

According to aspects of the present invention, processor 122 and/or augmented reality Subscriber Interface Module SIM 134 can be configured with：Identification A group echo in environment；Make the group echo associated with described group of surface；And described group is tracked using the group echo Surface.According to certain aspects of the invention, by Fig. 3 processor 122 or augmented reality Subscriber Interface Module SIM 134 or it can pass through The function described in Fig. 7 is implemented in both combination (potentially being combined with one or more other elements).In some embodiment party In case, the function can be by processor 122, software 126, hardware 128 and firmware 130 or more each to perform this hair The combinations of the various functions of equipment described in bright is performed.In other other embodiments, the work(described in Fig. 7 It is able to can be implemented by the processor 507 combined with one or more other elements (for example, Fig. 5 element 500-550).

It should be noted that at least paragraph [0066] is provided to [0068], Fig. 3, Fig. 5, Fig. 7 and its corresponding description：For controlling Camera scans the device of the environment in the visual field of user；Recognized for the characteristic according to the environment in the environment One group of surface for show user interface form device；For being arranged relative to described group of surface in environment for aobvious The device of the priority for the one group of augmented reality content shown；And in the display showing described group of augmented reality content Device on described group of surface.At least paragraph [0066] to [0068], Fig. 3, Fig. 5, Fig. 6, Fig. 7 and its corresponding description are carried For：For the device for the priority that the described group of augmented reality content for display is arranged based on the following：Described group of table The area in face, the orienting of described group of surface, described group of surface relative to the visual field of user position and described group of surface can be with Track；And for arranging the priority of the described group of augmented reality content for display based at least one of the following Device：The ratio of width to height of described group of surface relative to described group of augmented reality content to be shown；And described group of surface relative to The backcolor of described group of augmented reality content to be shown；History for the previously used pattern according to user in the environment To determine the device of one group of augmented reality content for display；For being arranged based at least one of the following for display Described group of augmented reality content priority device：The scale factor of described group of augmented reality content to be shown；And The logical relation of described group of augmented reality content to be shown；For determining one for display according to predefined user preference The device of group augmented reality content；And for described group of augmented reality content to be shown on described group of surface in the display Device.

Method and mobile device described herein may depend on using and implemented by various devices.Citing comes Say, this little method can be implemented with hardware, firmware, software or its combination.For hardware implementation, processing unit may be implemented in It is one or more application specific integrated circuits (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable Logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, electronics dress Put, be designed to perform in other electronic units of function described herein or its combination.Herein, term " patrol by control Volume " cover by software, hardware, firmware or the implemented logic of its combination.

For firmware and/or software implementation, methods described can be by performing the module of function described herein (for example, program, function etc.) is implemented.Any machine for visibly embodying instruction can be used when implementing method described herein Device readable media.For example, software code can be stored in memory and be performed by processing unit.Memory may be implemented in In processing unit or outside processing unit.As used herein, term " memory " refers to any kind of long-term, short Phase, volatibility, non-volatile or other storage devices, and it is not restricted to any certain types of memory or any certain number Purpose memory, or the type of the media of storage memory thereon.

If with firmware and/or software implementation, then the function can be stored in as one or more instructions or code On computer-readable media.Example has the meter of computer program comprising the computer-readable media and coding that coding has data structure Calculation machine readable media.Computer-readable media can take the form of product.Computer-readable media is stored comprising physical computer Media and/or other non-transitory medias.Storage media can for can by computer access any useable medium.By example Unrestricted, this little computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other disk storage, disk storage Device or other magnetic storage devices, or can be used to storage in instruction or data structure form want program code and can be by counting Any other media of calculation machine access；As used herein, disk and CD include compact disk (CD), laser-optical disk, light The usual magnetically replicate data of disk, digital versatile disc (DVD), floppy discs and Blu-ray Disc, wherein disk, and light Disk passes through laser replicate data optically.The combination of each thing should also contain in the range of computer-readable media above.

In addition to being stored on computer-readable media, instruction and/or data are alternatively arranged as signal and are provided in include In transmission media in communication equipment.For example, communication equipment can include the receipts of the signal with indicator and data Send out device.It is described instruction and data are configured so that one or more processors implement the function of being summarized in claims.That is, lead to Letter equipment includes the transmission media with the signal for indicating the information to perform disclosed function.In the very first time, communication is set Transmission media included in standby can include the Part I of the information to perform disclosed function, and in the second time, lead to Transmission media included in letter equipment can include the Part II of the information to perform disclosed function.

Such as wireless wide area network (WWAN), WLAN (WLAN), wireless personal local area network (WPAN) etc. can be combined Various cordless communication networks implement the present invention.Term " network " and " system " is usually interchangeably used.Usually interchangeably Use term " position (position) " and " position (location) ".WWAN can be more for CDMA (CDMA) network, time-division Location (TDMA) network, frequency division multiple access (FDMA) network, OFDM (OFDMA) network, single-carrier frequency division multiple access (SC- FDMA) network, Long Term Evolution (LTE) network, WiMAX (IEEE 802.16) network, etc..Cdma network can be implemented one or more Plant radio access technologies (RAT), such as cdma2000, wideband CDMA (W-CDMA), etc..Cdma2000 comprising IS-95, IS2000 and IS-856 standards.TDMA networks can implement global system for mobile communications (GSM), digital advanced mobile phone system , or a certain other RAT (D-AMPS).GSM and W-CDMA are described in from entitled " third generation partner program " (3GPP) Association file in.Cdma2000 is described in from the entitled " text of the association of third generation partner program 2 " (3GPP2) In part.3GPP and 3GPP2 files are publicly available.WLAN can be IEEE 802.11x networks, and WPAN can for blueteeth network, IEEE 802.15x or some other type of networks.The technology may also be combined with WWAN, WLAN and/or WPAN any combinations To implement.

Mobile station is the device for referring to receive wireless communication signals and/or navigation signal, such as honeycomb fashion or other nothings Line communicator, PCS Personal Communications System (PCS) device, personal navigation apparatus (PND), personal information manager (PIM), individual number Word assistant (PDA), laptop computer or other suitable mobile devices.Term " mobile station " is also meant to include for example by short Journey is wireless, infrared ray, wired connection or other connections carry out the device that is communicated with personal navigation apparatus (PND), this and satellite-signal Reception, assistance data reception and/or position relevant treatment occur unrelated at described device or at PND.Moreover, " mobile Platform " is intended to include all devices, and comprising radio communication device, computer, laptop computer etc., it can be for example via because of spy Net, Wi-Fi or other networks come with server communication, and it is related to satellite signal receiving, assistance data reception and/or position place It is unrelated at the raw another device still associated with network at described device, at server of haircut.Any of above-mentioned each can Operative combination is also regarded as " mobile station ".

It is that " optimized ", " required " or other specify do not indicate that the present invention is only applicable to optimization to specify something System, or wherein exist " required " element system (or being attributed to other other limitations specified).This is specified only a bit Refer to specific described embodiment.Certainly, many embodiments are possible.The technology can be from different institutes in this article The agreement (comprising developing or agreement leaved for development) of the agreement of discussion is used together.

Those skilled in the art will realize that can make while still using the basic bottom mechanism of identical and method With many possible modifications and combination of disclosed embodiment.For illustrative purposes, write with reference to specific embodiment Foregoing description.However, illustrative discussions above be not intended to be restricted in detail or by this disclosure it is disclosed accurate Form.In view of teachings above, many modifications and change are possible.Select and describe the embodiment to explain the original of the present invention Reason and its practical application, and enable those skilled in the art best using the present invention and with suitable for desired spy Determine the various embodiments of the various modifications of purposes.

Claims (44)

1. a kind of method for showing augmented reality content, it includes：

The scanning of the environment in the visual field of user is received by camera；

Recognize one group of surface in the environment for display user interface form；

Surface priority is assigned to each surface in one group of surface, wherein the surface priority choosing on each surface From multiple surface priority；

Surface trackability rank is assigned to each surface in one group of surface, wherein the surface on each surface can Tracing property rank is selected from multiple surface trackability ranks；

Content prioritization is assigned to one group of augmented reality content for display relative to one group of surface in the environment In each content item, wherein the content prioritization of each content item be selected from multiple content prioritizations；And

In the display will be for aobvious based on the surface priority, the surface trackability rank and the content prioritization Each content item in the one group of augmented reality content shown is shown on the surface in one group of surface.

2. according to the method described in claim 1, wherein the environment includes real-world objects, and wherein described one group of surface Surface included in the top visual field of the user, bottom visual field, left view circle, right visual field and preceding visual field；And it is wherein described aobvious It is head mounted display to show device.

3. according to the method described in claim 1, wherein the surface priority is assigned to each in one group of surface Surface includes：

Area based on one group of surface, the orientation on one group of surface and one group of surface are relative to the user visual field Position assign the surface priority, wherein the multiple surface priority includes high surface priority, medium surface Priority and low surface priority.

4. according to the method described in claim 1, wherein relative to one group of surface in the environment that the content is excellent Each content item that first level is assigned in one group of augmented reality content for display includes：

The content prioritization is assigned based at least one of the following：One group of surface is relative to described in be shown The ratio of width to height of one group of augmented reality content；And one group of surface is relative to the back of the body of one group of augmented reality content to be shown Scenery is color.

5. according to the method described in claim 1, wherein relative to one group of surface in the environment that the content is excellent Each content item that first level is assigned in one group of augmented reality content for display includes：

The history of previously used pattern based on the user in the environment assigns the content prioritization.

6. method according to claim 5, wherein previously used pattern based on the user in the environment is gone through History assigns the content prioritization to include at least one of the following：

Working to select first group of augmented reality content for display in response to the determination user；

In response to determining that the user is just selecting second group of augmented reality content for display at home；

Just it is being in business meetings to select the 3rd group of augmented reality content for display in response to the determination user；And

Just it is being in social event to select the 4th group of augmented reality content for display in response to the determination user.

7. method according to claim 6, wherein

First group of augmented reality content communicates and file process being used for of getting used to operationally using comprising the user Application program；And

Second group of augmented reality content comprising the user get used to using at home for the application that communicates and entertain Program.

8. method according to claim 7, wherein

First group of augmented reality content for communication and file process includes Email, web browser and office Productivity application；And

Second group of augmented reality content for communicating and entertaining includes social networking website, film and video game application Program.

9. according to the method described in claim 1, wherein relative to one group of surface in the environment that the content is excellent Each content item that first level is assigned in one group of augmented reality content for display includes：

The content prioritization is assigned based at least one of the following：One group of augmented reality content to be shown Scale factor；And the logical relation of one group of augmented reality content to be shown.

10. method according to claim 9, wherein the logical relation of one group of augmented reality content to be shown Including：

The corresponding video display of file and the file.

11. method according to claim 9, wherein the logical relation of one group of augmented reality content to be shown Including：

The corresponding presentation slides of file and the file.

12. according to the method described in claim 1, wherein relative to one group of surface in the environment that the content is excellent Each content item that first level is assigned in one group of augmented reality content for display includes：

The content prioritization is assigned based on predefined user preference.

13. method according to claim 12, wherein the predefined user preference is included in the following at least One：

Importance of the augmented reality content to the user；

The amount of the augmented reality content to be shown；And

The augmented reality content inspects orientation relative to one group of surface.

14. according to the method described in claim 1, it further comprises：

Recognize the group echo in the environment；

Make a group echo associated with one group of surface；And

One group of surface is tracked using a group echo.

15. a kind of equipment for showing augmented reality content, it includes：

For the device for the scanning that the environment in the visual field of user is received by camera；

For recognizing one group of surface in the environment for the device of display user interface form；

For the device on each surface being assigned to surface priority in one group of surface, wherein the table on each surface Face priority is selected from multiple surface priority；

For the device on each surface being assigned to surface trackability rank in one group of surface, wherein each surface The surface trackability rank is selected from multiple surface trackability ranks；

For content prioritization to be assigned to one group of augmented reality for display relative to one group of surface in the environment Each content item destination device in content, wherein the content prioritization of each content item is selected from multiple content priorities Level；And

For in the display will based on the surface priority, the surface trackability rank and the content prioritization The dress on the surface in one group of surface is shown in for each content item in one group of augmented reality content of display Put.

16. equipment according to claim 15, wherein the environment includes real-world objects, and wherein described one group of table Bread is contained in the surface in the top visual field of the user, bottom visual field, left view circle, right visual field and preceding visual field；And it is wherein described Display is head mounted display.

17. equipment according to claim 15, wherein for the surface priority to be assigned in one group of surface The device on each surface include：

For the area based on one group of surface, the orientation on one group of surface and one group of surface relative to user The device of the surface priority is assigned in the position in the visual field, wherein to include high surface preferential for the multiple surface priority Level, medium surface priority and low surface priority.

18. equipment according to claim 15, wherein for inciting somebody to action described relative to one group of surface in the environment Each content item destination device that content prioritization is assigned in one group of augmented reality content for display includes：

Device for assigning the content prioritization based at least one of the following：One group of surface is relative to treating The ratio of width to height of one group of augmented reality content of display；And one group of surface is real relative to one group of amplification to be shown The backcolor held within the border.

19. equipment according to claim 15, wherein for inciting somebody to action described relative to one group of surface in the environment Each content item destination device that content prioritization is assigned in one group of augmented reality content for display further comprises：

History for the previously used pattern based on the user in the environment assigns the device of the content prioritization.

20. equipment according to claim 19, wherein for the previously used mould based on the user in the environment The history of formula assigns the device of the content prioritization to include at least one of the following：

The device of first group of augmented reality content for display is selected for being worked in response to the determination user；

For in response to determining that the user is just selecting the device of second group of augmented reality content for display at home；

For selecting the 3rd group of augmented reality content for display in response to determining that the user is just being in business meetings Device；And

For selecting the 4th group of augmented reality content for display in response to determining that the user is just being in social event Device.

21. equipment according to claim 20, wherein

First group of augmented reality content communicates and file process being used for of getting used to operationally using comprising the user Application program；And

Second group of augmented reality content comprising the user get used to using at home for the application that communicates and entertain Program.

22. equipment according to claim 21, wherein

First group of augmented reality content for communication and file process includes Email, web browser and office Productivity application；And

Second group of augmented reality content for communicating and entertaining includes social networking website, film and video game application Program.

23. equipment according to claim 15, wherein for inciting somebody to action described relative to one group of surface in the environment Each content item destination device that content prioritization is assigned in one group of augmented reality content for display includes：

Device for assigning the content prioritization based at least one of the following：One group of amplification to be shown The real scale factor held within the border；And the logical relation of one group of augmented reality content to be shown.

24. equipment according to claim 15, wherein for inciting somebody to action described relative to one group of surface in the environment Each content item destination device that content prioritization is assigned in one group of augmented reality content for display includes：

Device for assigning the content prioritization based on predefined user preference.

25. equipment according to claim 24, wherein the predefined user preference is included in the following at least One：

Importance of the augmented reality content to the user；

The amount of the augmented reality content to be shown；And

The augmented reality content inspects orientation relative to one group of surface.

26. equipment according to claim 15, it further comprises：

Device for recognizing the group echo in the environment；

For making the group echo device associated with one group of surface；And

Device for tracking one group of surface using a group echo.

27. a kind of equipment for showing augmented reality content, it includes：

At least one processor, wherein at least one described processor includes：

The logic for the scanning for being configured to camera to receive the environment in the visual field of user；

It is configured to recognize one group of surface in the environment to show the logic of user interface form；

Be configured to the logic on each surface surface priority being assigned in one group of surface, wherein the institute on each surface State surface priority and be selected from multiple surface priority；

Be configured to the logic on each surface surface trackability rank being assigned in one group of surface, wherein each table The surface trackability rank in face is selected from multiple surface trackability ranks；

It is configured to that content prioritization is assigned to one group of amplification for display relative to one group of surface in the environment The logic of each content item in real domestic appearance, wherein the content prioritization of each content item is excellent selected from multiple contents First level；And

It is configured to based on the surface priority, the surface trackability rank and the content prioritization in display It is middle to be shown in for each content item in one group of augmented reality content of display on the surface in one group of surface Logic.

28. equipment according to claim 27, wherein the environment includes real-world objects, and wherein described one group of table Bread is contained in the surface in the top visual field of the user, bottom visual field, left view circle, right visual field and preceding visual field；And it is wherein described Display is head mounted display.

29. equipment according to claim 27, wherein being configured to the surface priority being assigned to one group of table The logic on each surface in face includes：

Be configured to the area based on one group of surface, the orientation on one group of surface and one group of surface relative to The logic of the surface priority is assigned in the position in the user visual field, wherein to include high surface excellent for the multiple surface priority First level, medium surface priority and low surface priority.

30. equipment according to claim 27, wherein being configured to incite somebody to action relative to one group of surface in the environment The logic for each content item that the content prioritization is assigned in one group of augmented reality content for display includes：

It is configured to assign the logic of the content prioritization based at least one of the following：One group of surface is relative In the ratio of width to height of one group of augmented reality content to be shown；And one group of surface is relative to one group of expansion to be shown Increase the real backcolor held within the border.

31. equipment according to claim 27, wherein being configured to incite somebody to action relative to one group of surface in the environment The logic for each content item that the content prioritization is assigned in one group of augmented reality content for display includes：

The history for the previously used pattern being configured to based on the user in the environment assigns the content prioritization Logic.

32. equipment according to claim 31, wherein being configured to previously the making in the environment based on the user The logic for assigning the content prioritization with the history of pattern includes at least one of the following：

It is configured to respond to determine that the user is working to select the logic of first group of augmented reality content for display；

It is configured to respond to determine that the user just selects the logic of second group of augmented reality content for display at home；

It is configured to respond to determine that the user is just being in business meetings to select in the 3rd group of augmented reality for display The logic of appearance；And

It is configured to respond to determine that the user is just being in social event to select in the 4th group of augmented reality for display The logic of appearance.

33. equipment according to claim 32, wherein

First group of augmented reality content communicates and file process being used for of getting used to operationally using comprising the user Application program；And

Second group of augmented reality content comprising the user get used to using at home for the application that communicates and entertain Program.

34. equipment according to claim 33, wherein

First group of augmented reality content for communication and file process includes Email, web browser and office Productivity application；And

Second group of augmented reality content for communicating and entertaining includes social networking website, film and video game application Program.

35. equipment according to claim 27, wherein being configured to incite somebody to action relative to one group of surface in the environment The logic for each content item that the content prioritization is assigned in one group of augmented reality content for display includes：

It is configured to assign the logic of the content prioritization based at least one of the following：To be shown described one group The scale factor of augmented reality content；And the logical relation of one group of augmented reality content to be shown.

36. equipment according to claim 27, wherein being configured to incite somebody to action relative to one group of surface in the environment Each content item logic that the content prioritization is assigned in one group of augmented reality content for display includes：

It is configured to assign the logic of the content prioritization based on predefined user preference.

37. equipment according to claim 36, wherein the predefined user preference is included in the following at least One：

Importance of the augmented reality content to the user；

The amount of the augmented reality content to be shown；And

The augmented reality content inspects orientation relative to one group of surface.

38. equipment according to claim 27, it further comprises：

The logic for the group echo for being configured to recognize in the environment；

It is configured so that the group echo logic associated with one group of surface；And

It is configured to track the logic on one group of surface using a group echo.

39. a kind of system for showing augmented reality content, it includes：

For the device for the scanning that the environment in the visual field of user is received by camera；

For recognizing one group of surface in the environment for the device of display user interface form；

Assigned for each surface by being assigned to surface priority in one group of surface, by surface trackability rank Content prioritization is assigned to each surface in one group of surface, and relative to one group of surface in the environment For each content item in one group of augmented reality content of display, arranged relative to one group of surface in the environment For the device of the priority of one group of augmented reality content of display, wherein the surface priority on each surface is selected from multiple Surface priority, wherein the surface trackability rank on each surface is selected from multiple surface trackability ranks, wherein often The content prioritization of individual content item is selected from multiple content prioritizations；And

For in the display will based on the surface priority, the surface trackability rank and the content prioritization The dress on the surface in one group of surface is shown in for each content item in one group of augmented reality content of display Put.

40. the system according to claim 39, wherein for being arranged relative to one group of surface in the environment Include for the device of the priority of one group of augmented reality content of display：

For the area based on one group of surface, the orientation on one group of surface and one group of surface relative to user The device of the surface priority is assigned in the position in the visual field, wherein to include high surface preferential for the multiple surface priority Level, medium surface priority and low surface priority.

41. the system according to claim 39, wherein for being arranged relative to one group of surface in the environment Include for the device of the priority of one group of augmented reality content of display：

Device for assigning the content prioritization based at least one of the following：One group of surface is relative to treating The ratio of width to height of one group of augmented reality content of display；And one group of surface is real relative to one group of amplification to be shown The backcolor held within the border.

42. the system according to claim 39, wherein for being arranged relative to one group of surface in the environment Include for the device of the priority of one group of augmented reality content of display：

History for the previously used pattern according to the user in the environment is relative to described one in the environment The device for the priority for organizing surface to arrange one group of augmented reality content for display.

43. the system according to claim 39, wherein for being arranged relative to one group of surface in the environment Include for the device of the priority of one group of augmented reality content of display：

Device for assigning the content prioritization based at least one of the following：One group of amplification to be shown The real scale factor held within the border；And the logical relation of one group of augmented reality content to be shown.

44. the system according to claim 39, wherein for being arranged relative to one group of surface in the environment Include for the device of the priority of one group of augmented reality content of display：

For according to predefined user preference relative to one group of surface in the environment come arrange for display described in The device of the priority of one group of augmented reality content.