EP3238104A1

EP3238104A1 - Native application deeplink scoring

Info

Publication number: EP3238104A1
Application number: EP16741823.5A
Authority: EP
Inventors: Dong Ha Lee; Jaehyun Yeom
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2015-07-23
Filing date: 2016-07-15
Publication date: 2017-11-01
Also published as: WO2017015134A1

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving, by the data processing apparatus and from a user device, a search query including a term, scoring each deeplink of a plurality of deeplinks in a plurality of native applications for relevance to the term. Each deeplink specifies a particular environment instance of a native application and causes the native application to instantiate an instance of the specified native application in which content is displayed. Each native application is an application operable to generate environment instances for display on a user device and operates independent of a browser application. Ranking the deeplinks according to their respective relevance scores, and providing search results to the user device according to an order based, at least in part, on the ranking according to their respective relevance scores, where each search result specifies a deeplink. Effective ranking of deeplinks may provide a number of benefits to the system as a whole, not least a reduction in use of computational resources.

Description

NATIVE APPLICATION DEEPLINK SCORING

BACKGROUND

[0001] The Internet provides access to a wide variety of information. For example, digital image files, video and/or audio files, as well as web page resources for particular subjects or particular news articles, are accessible over the Internet. With respect to web page resources, many of these resources are designed to facilitate the performing of particular functions, such as banking, booking hotel reservations, shopping, etc., or to provide structured information, such as on-line encyclopedias, movie databases, etc. Search engines crawl and index these resources to facilitate searching of the resources.

[0002] Furthermore, with the advent of tablet computers and smart phones, native applications that facilitate the performance of the same functions facilitated by the use of web page resources are now being provided in large numbers. Additionally, native applications that do not have corresponding websites with similar content, such as games, are also very popular on tablet computers and smart phones. Accordingly, search engines now also facilitate searching of these native applications.

[0003] One process by which search engines gather information for native applications is by accessing "deeplinks" for native applications. A deeplink is an instruction specifying a particular environment instance of a native application and configured to cause the native application to instantiate the environment instance of the specified native application when selected at a user device. The native application generates the environment instance for display within the native application on a user device. For example, a deeplink may be a URI that specifies a particular native application, resource content that the native application is to access, and a particular user interface that should be instantiated when the native application is launched by use of the deeplink. For example, a deeplink may specify a selection menu for a game environment; or a particular selection of a song for a music application; or a particular recipe for a cooking application; and the like.

[0004] While search engines can index the content that is presented by native

applications, scoring a deeplink based on its content in a manner that is likely to meet a user's informational need can be challenging. SUMMARY

[0005] In general, one innovative aspect of the subject matter described in this specification can be embodied in methods that include the actions of receiving, by the data processing apparatus and from a user device, a search query including a term, scoring each deeplink of a plurality of deeplinks in a plurality of native applications for relevance to the term. Each deeplink specifies a particular environment instance of a native application and causes the native application to instantiate an instance of the specified native application in which content is displayed and each native application is an application operable to generate environment instances for display on a user device and operates independent of a browser application. The scoring includes determining an inter-inverse popularity score for the term based on a popularity of the term in content in each environmental instance of each native application. For each native application, determining an intra-inverse popularity score for the term based on a popularity of the term in content in each environment instance of the native application. For each deeplink determining a term score for the term based on a number of times the term appears in the content in the environment instance specified by the deeplink. Generating a relevance score for the deeplink based on the inter-inverse popularity score of the term, the intra-inverse popularity score of the native application for the deeplink, and the term score for the deeplink. Ranking the deeplinks according to their respective relevance scores and providing search results to the user device according to an order based, at least in part, on the ranking according to their respective relevance scores, where each search result specifies a deeplink. Other embodiments of this aspect include

corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices.

[0006] Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages. The present systems and methods rank deeplinks for native applications in response to a user query. The ranking of deeplinks enables a search engine to process a query to better fulfil an informational need for content within native applications.

[0007] Native applications are often associated with mobile devices and, as such, searches for content within native applications may often be performed using mobile devices. Typically, such devices have relatively small screens, on which it may be possible to display only a small number of deeplinks in response to a search query. Consequently, effective ranking of the deeplinks based on relevance to the search query may be particularly important for reducing, on average, the need for users to navigate beyond an initial page of search results. By reducing the need to navigate beyond the initial page, the use of computational resources that are associated with such an action (processing, bandwidth, screen-on time, power etc.) may also be reduced.

[0008] Ranking of deeplinks can be based, in part, on the steadiness of the content of the native application. The steadiness is a measure of the tendency of the content of the native application to change or be modified over a particular duration. The steadiness is also a signal that is indicative of the relevance of the content to the query; generally, the less steady the content, the less likely it is that the content can be relied upon for relevance. By taking into account the steadiness of the content, a system is more likely to surface content that is relevant to the query terms, thereby better fulfilling an informational need. In addition, by reducing the likelihood of surfacing content that is not relevant to the query terms (because, for instance, the content in a particular environment instance has changed since it was indexed), the use of computational resources may again be reduced. Such computational resource use may stem from selection of an alternative deeplink when content associated with an initially-selected deeplink is found not to be relevant to the query terms, and may include, for instance, processing of additional user inputs, rendering of the alternative environment instance and power usage due to additional screen-on time (which may be particularly undesirable in mobile devices). Furthermore, taking steadiness into account when ranking deeplinks may also increase the reproducibility of the content for which deeplinks are returned in response to a particular search query. Thus, the search system as a whole could be said to be more reliable.

[0009] The details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Fig. 1 is a block diagram of an example environment in which the search and the ranking of native application deep links occurs. [0011] Fig. 2 is a block diagram of an example system flow for scoring and ranking deeplinks.

[0012] Fig. 3 is a flow chart describing a process for scoring and ranking deeplinks.

[0013] Fig. 4 is a flow chart describing a process for providing a list of deeplinks according to ranked native applications.

[0014] Fig. 5 is an illustration of a user device displaying deeplink search results grouped by native applications in an interface.

[0015] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION

[0016] A system scores and ranks deeplinks of native applications. A search query typically includes at least one term. Each deeplink accesses resource content and also includes terms, and the terms and deep links are indexed by a search system. The score for a deeplink describes how relevant the resource content accessed by the deeplink is as it relates to the term. The search engine processes the query and accesses the index to score the deeplinks by various scoring techniques. For each deeplink of each native application, the scoring can take into account how relevant the term is to the content accessed by the deeplink, how relevant the term is to the entirety of content accessed by all deeplinks of the native application, and how relevant the term is to the content accessed by all deeplinks of a corpus of applications. Still, other scoring factors can also be considered.

[0017] Deeplinks can be provided, in response to a search query, on an individual basis or as a whole set of mixed search results. The set of search results can be ranked in order according to various implementations, which will be described in further detail below.

[0018] Fig. 1 is a block diagram of an example environment 100 in which triggering the search of and the ranking of native applications occur. A computer network 102, such as the Internet, connects resource publisher websites 104, application publishers 106, user devices 108 and a search engine 120.

[0019] A resource publisher website 104 includes one or more web resources 105 associated with a domain and hosted by one or more servers in one or more locations. Generally, a resource publisher website is a collection of web pages formatted in hypertext markup language (HTML) that can contain text, images, multimedia content, and programming elements. Each website 104 is maintained by a content publisher, which is an entity that controls, manages and/or owns the website 104.

[0020] A web page resource is any data that can be provided by a publisher website 104 over the network 102 and that has a resource address, e.g., a uniform resource locator (URL). Web resources may be HTML pages, images files, video files, audio files, and feed sources, to name just a few. The resources may include embedded information, e.g., meta information and hyperlinks, and/or embedded instructions, e.g., client-side scripts.

More generally, a "resource" is anything identifiable over a network, and can also include native applications.

[0021] An application publisher website 106 may also include one or more web resources 105, and also provides native applications 107. As used herein, a native application is an application that generates environment instances for display on a user device within an environment of the native application, and operates independent of a browser application on the user device. A native application is an application

specifically designed to run on a particular user device operating system and machine firmware. Native applications thus differ from browser-based applications and browser- rendered resources. The latter require all, or at least some, elements or instructions downloaded from a web server each time they are instantiated or rendered. Furthermore, browser-based applications and browser-rendered resources can be processed by all web- capable mobile devices within the browser and thus are not operating system specific.

[0022] As used in this specification, an "environment instance" is a display environment within a native application and in which is displayed content, such as text, images, and the like. An environment instance is specific to the particular native application, and the native application is specific to the particular operating system of the user device 108. An environment instance differs from a rendered web resource in that the environment instance is generated within and specific to the native application, while a web resource may be rendered in any browser for which the web page resource is compatible, and is independent of the operating system of the user device.

[0023] A user device 108 is an electronic device that is under the control of a user. A user device 108 is typically capable of requesting and receiving web page resources 105 and native applications 107 over the network 102. Example user devices 108 include personal computers, mobile communication devices, and tablet computers. [0024] To search web resources 105 and the native applications 107, the search engine 120 accesses a web index 116 and an application index 114. The web index 116 is an index of web resources 105 that has, for example, been built from crawling the publisher websites 104. The application index 114 is an index of application pages for native applications 107, and is constructed using an application data extractor and processor 110 and an indexer 112. Although shown as separate indexes, the web index 116 and the application index 114 can be combined in a single index.

[0025] The user devices 108 submit search queries to the search engine 120. In response to each query, the search engine 120 accesses the web index 116 and, optionally, the application index 114 to identify resources and applications, respectively, which are relevant to the query. Generally, a first type of search operation implementing a first search algorithm is used to search the web index 116, and a second type of search operation implementing a second, different algorithm is used to search the application index 114. The search engine 120 implements a resource scorer 130 process to score web resources, and deeplink scorer 134 process to score native applications and deeplinks. A variety of appropriate search engine algorithms can be used to implement the resource scorer 130.

[0026] In the event the native application index 114 is searched and a native application search result is to be provided to a user device, the search engine 120 utilizes a mixer 136 to insert the native application search result, which can be based on the scoring of deeplinks, into search results of other resources. The deeplink scorer 134 is described in more detail with reference to Figs. 2, 3, and 4 below. The search engine 120 utilizes a search engine front end 138, such as a web server, arranges and provides the search results to the user device 108 from which the query was received.

[0027] A web resource search result is data generated by the search engine 120 that identifies a web resource and provides information that satisfies a particular search query. A web resource search result for a resource can include a web page title, a snippet of text extracted from the resource, and a resource locator for the resource, e.g., the URL of a web page. A native application search result specifies a native application and is generated in response to a search of the application index 114. A native application search result may include a "deeplink" specifying a particular environment instance of the native application and which is configured to cause the native application to instantiate the specified environmental instance. For example, selection of a native application search result may cause the native application to launch (if installed on the user device 108) and generate an environment instance referenced in the application search result in the form of a screen shot. Alternatively, a native application search result may include a "purchase" (or "install") command that, when selected, results in a purchase (or free download) and installation of the native application on a user device.

[0028] Publishers 106 that provide native applications 107 may also provide the deeplinks 109 to the search engine 120. For example, an application publisher may provide a list of deeplinks 109 in the form of uniform resource identifiers (URIs) (or other instruction types that are specific to the native application published by the publisher). These deeplinks are deeplinks that publisher 106 desires to be crawled and indexed in the application index 114.

[0029] The deeplink scorer 134 scores and ranks deeplinks that are indexed in the application index 114 according to the deeplink's relevance to a particular term. Fig. 2 is a block diagram of an example system flow 200 for scoring and ranking deeplinks. As previously described, the application index 114 has an index of deeplinks, the native applications that contain the deeplinks, and the content referenced by each deeplink. A variety of index architectures and indexing techniques can be used.

[0030] The deeplink scorer 134 scores deeplinks according to scoring attributes 202. In some implementations, the scoring attributes include an intra-inverse popularity 204, an inter-inverse popularity 206, and a term score 208. In variations of this implementations, additional scoring attributes can include a steadiness score 210, an application link score 212, and a popularity score 214. The deeplink scorer 134 determines the score of each scoring attribute 202 and uses a combination of the scoring attributes to determine an overall relevance score for each deeplink. Generation of the respective scores and the operations related to the scores are described in more detail below.

[0031] The deeplink scorer 134 determines the score for each deeplink, ranks the deeplinks according to the respective scores of the deeplinks, and sends data describing the ranking to the search engine front end 138, which, in turn, creates a list of the ranked deeplinks. The search engine front end 138 generates search results to be displayed on the user device 108. As will be described in Figs. 3 and 4, the deeplink scorer 134 can use various combinations of the ranking attributes 202 to determine the deeplink score and to rank the deeplinks. The various combinations enable deeplinks to be scored for different purposes (e.g., to enhance diversity of native applications or deeplinks that are ranked and provided in response to a search query).

[0032] Fig. 3 is a flow chart describing a process 300 for scoring and ranking deeplinks. The search engine 120 receives a search query that includes a term (302). For example, a user may provide a search query to the search engine 120. The search query may be provided in the context of an "application only" search, in which the search is limited to a search of native applications, such as in an application store. Alternatively, the search query can be provided in the context of an informational search of multiple corpora, one of which includes the corpus of native applications.

[0033] The deeplink scorer 134 accesses the application index to generate scores for applications and deeplinks that measure the relevance of the applications and deeplinks to the query.

[0034] The deeplink scorer 134 determines an inter-inverse popularity Inter l(T) 206 for the term based on a popularity of the term in content in each environment instance of each native application (304). For example, the deeplink scorer 134 determines how many times a term appears in of the content of all deeplinks for all native applications, and then calculates the inter-inverse popularity score based on an inverse of the number of times. For example, the term "the" will have a lower score because it appears often, while the term "endgame" may have a much higher score than the term "the" since it does not appear as often as the term "the."

[0035] For each native application, the deeplink scorer 134 determines the intra-inverse popularity Intra_I(T, A) 204 for the term based on a popularity of the term in content in each environment instance of the native application (304). Similar to the inter-inverse popularity Inter l(T) 206, the intra-inverse popularity Intra_I(T, A) 204 is based on an inverse of the number of times a particular term appears in the content referenced by each deep link in a particular native application.

[0036] The deeplink scorer 134, for each deeplink, determines a term score TS(T, D) 208 for the term based on a number of times the term appears in the content in the environment instance specified by the deeplink (308). The term score TS(T, D) 208 is a measure of how often the term appears in the resource content that is accessed by the deeplink.

[0037] In some implementations, the deeplink scorer 134 may score each deeplink based on the inter-inverse popularity Inter l(T) score 206, the intra-inverse popularity Inter_I(T, A) score 204, and the term score TS(T, D) 204(310). For example, each deeplink score may be calculated according to Equation (1) below:

DLS(D, A, T) = f(Inter_I(T), Intra_I(T, A), TS(T, D)) (1)

[0038] Where:

[0039] D is the Deeplink of a particular URI;

[0040] A is a particular native application; and

[0041] T is the query term.

[0042] A variety of scoring formulas can be used. In some implementations, the relevance score (e.g., deeplink score) can be determined by taking the product of the inter-inverse popularity Intra l(T) score 206, the intra-inverse popularity Inter_I(T, A) score 204, and the term score TS(T, D) 204.

[0043] The deeplink scorer 134 ranks the deeplinks according to their respective relevance scores (312). The deeplink scorer 134 provides data describing the ranking to the search engine front end, and the search engine front end 138 provides search results to the user device according to an order based, at least in part, on the ranking (314). Alternatively, when other search results are to be provided for other resources (e.g., web pages, images, etc.), the deep link scorer provides the scores to a mixer 136, which then converts the scores of the multiple different resource scores to a common scoring reference.

[0044] In some implementations, the deeplink scorer 134 can consider other scoring attributes to score a deeplink. For example, the scoring may also include, for each native application, determining a steadiness S(T, A) 210 score that is a measure of the relative change of distribution of the term in the content of each environmental instance of the native application over a predefined amount of time. Some native applications may have dynamic content that is modified often (e.g., news applications, social media, etc.), while other native applications may have content that is relatively static (e.g., dictionary, encyclopedia, etc.). The deeplink scorer 134 may use a modified version of equation 1, which is shown below as equation 2: DLS(D, A, T) = f(Inter_I(T), Intra_I(T, A), TS(T, D), S(T, A)) (2)

[0045] To determine the steadiness score, the deeplink scorer 134 determines, for a first predefined time period, a first number of deeplinks in the native application that specify environmental instances having content that includes the term. The deeplink scorer 134 then determines, for a second predefined time period that is different from the first predefined time period, a second number of deeplinks for the native application that specify environmental instances having content that include the term. Based on a ratio of the first number of deeplinks and the second number of deeplinks, the deeplink scorer 134 determines the steadiness S(T, A) score 210. Other techniques to determine a steadiness score can also be used.

[0046] In some instances, a user may want search query results provided according to the native applications that have the most relevant deeplinks that access resource content which describes content related to the term in the search query. Thus, in some implementations, the deeplink scorer 134 can rank a set of native applications that have the most content relevant to the term. Further, the deeplink scorer 134 can provide a ranked list of deeplinks for each application that was determined to have the most content relevant to the term. This results in a set of native applications being identified for a query, and for each native application, a subset of deeplinks for that native application.

[0047] Fig. 4 is a flow chart describing a process 400 for providing a list of deeplinks according to ranked native applications. In some implementations, the process 400 determines an application link score ALS(T, A) 212 that is indicative of the number of deeplinks in the native application for which respective content includes the term, relative to total number of all deeplinks of all native applications (402). The application link score (T, A) 212 can be used as a weight for a native application according to the term in the search query. Accordingly, a native application that contains more deeplinks with the term is determined to have a higher weight than a native application that contains fewer deeplinks with the term.

[0048] The process 400 ranks the native applications according to their respective application link scores (404). Then the process 400 provides the search results for the deeplinks in subsets of search results, wherein each subset corresponding to one native application, and the subsets are provided to the user device in an order based on the ranking of the native applications (406).

[0049] For example, assume applications Al, A2 and A3 are respectively ranked 1, 2 and 3 according to their respective application link scores. A set of search results will be provided to the user device, and the search results for deep links will include a first subset for application Al, e.g., A1DL1, A1DL2, and A1DL3; a second subset for application A2, e.g., A2DL1, and A2DL2; and a third subset for application A3, e.g., A3DL1, A3DL2, and A3DL3.

[0050] The search results may be displayed according to an application hierarchy, e.g., as shown in Fig. 5, which Fig. 5 is an illustration of user device 500 displaying deeplink search results grouped by native applications in an interface 502. Here, for the search query "Chess games with opening books", three native applications - Chess App 1, Chess App 2 and Chess App 3 are identified by respective search results. Nested beneath each search results are deeplink results for each native application. The deeplink results are ranked, for example, according to either formulas (1) or (2) above.

[0051] In alternate implementations and to enhance deeplink scoring diversity, the deeplink scorer 134 can determine the popularity of a native application. For example, the scoring further comprises, for each native application, determining popularity score P(A) 214 for the native application that is a measure of popularity of the native application relative to other native applications. The deeplink scorer 134 generating a relevance score for the deeplink further comprises generating the relevance score based also on the popularity score P(A) 214 of the native application for the deeplink.

[0052] In some instances, native application popularity can be based on the number of times a native application has been downloaded. The deeplink scorer 134 may use the most popular native applications to determine how to rank relevant deeplinks. The deeplink scorer 134 may also determine the inverse popularity score to eliminate popular applications and provide a ranked deeplinks list that includes less popular native applications. To include the native application popularity score P(A) 214 as an attribute for determining the relevance score, the deeplink scorer 134, in some implementations, the relevance score (e.g., deeplink score) can be determined by calculating the product of the inter-inverse popularity Inter l(T) 206, the intra-inverse popularity Intra_I(T, A) 204, the term score TS(T, D) 208, and the popularity score P(A) 214. [0053] The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer- readable storage devices or received from other sources.

[0054] The term "data processing apparatus" encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, or combinations, of the foregoing. The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question.

[0055] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0056] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for performing actions in accordance with instructions and one or more memory devices for storing instructions and data.

[0057] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a

communication network. Examples of communication networks include a local area network ("LAN") and a wide area network ("WAN"), an inter-network (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0058] The computing system can include users and servers. A user and server are generally remote from each other and typically interact through a communication network. The relationship of user and server arises by virtue of computer programs running on the respective computers and having a user-server relationship to each other. In some embodiments, a server transmits data (e.g., an HTML page) to a user device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the user device). Data generated at the user device (e.g., a result of the user interaction) can be received from the user device at the server.

[0059] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a

subcombination.

[0060] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0061] Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

Claims

What is claimed is:

1. A computer-implemented method performed by data processing apparatus, comprising:

receiving, by the data processing apparatus and from a user device, a search query including a term;

scoring each deeplink of a plurality of deeplinks in a plurality of native applications for relevance to the term, wherein:

each deeplink specifies a particular environment instance of a native application and causes the native application to instantiate an instance of the specified native application in which content is displayed;

each native application is an application operable to generate environment instances for display on a user device and operates independent of a browser application; and

the scoring comprises:

determining an inter-inverse popularity score for the term based on a popularity of the term in content in each environmental instance of each native application;

for each native application, determining an intra-inverse popularity score for the term based on a popularity of the term in content in each environment instance of the native application; and

for each deeplink:

determining a term score for the term based on a number of times the term appears in the content in the environment instance specified by the deeplink; and

generating a relevance score for the deeplink based on the inter-inverse popularity score of the term, the intra-inverse popularity score of the native application for the deeplink, and the term score for the deeplink;

ranking the deeplinks according to their respective relevance scores; and providing search results to the user device according to an order based, at least in part, on the ranking according to their respective relevance scores, where each search result specifies a deeplink.

2. The computer-implemented method of claim 1, wherein:

the scoring further comprises, for each native application, determining a steadiness score that is a measure of the relative change of distribution of the term in the content of each environmental instance of the native application over a predefined amount of time; and

generating a relevance score for the deeplink further comprises generating the relevance score based also on the steadiness score of the native application for the deeplink.

3. The computer-implemented method of claim 2, wherein determining the steadiness score for each native application comprises:

determining, for a first predefined time period, a first number of deeplinks that specify environmental instances having content that include the term;

determining, for a second predefined time period that is different from the first predefined time period, a second number of deeplinks the specify environmental instances having content that include the term; and

determining the steadiness score based on a ratio of the first number to the second number.

4. The computer-implemented method of any preceding claim, wherein:

the scoring further comprises, for each native application, determining an application link score indicative of the number of deeplinks in the native application for which respective content includes the term relative to total number of all deeplinks of all native applications; and

generating a relevance score for the deeplink further comprises generating the relevance score based also on the application link score of the native application for the deeplink.

5. The computer-implemented method of any of claims 1 to 3, wherein:

providing search results to the user device according to an order based, at least in part, on the ranking according to their respective relevance scores comprises: ranking the native applications according to their respective application link scores;

for a set of search results to be provided to the user device:

providing the search results for the deeplinks in subsets of search results, each subset corresponding to one native application and provided to the user device in an order based on the ranking of the native applications, wherein each subset includes one or more deeplinks for the native application to which the subset

corresponds, and for each subset that includes two or more deeplinks, the deeplinks are ordered according to the respective relevance scores.

6. The computer-implemented method of any of claims 1 to 4, further comprising: the scoring further comprises, for each native application, determining popularity score for the native application that is a measure of popularity of the native application relative to other native applications; and

generating a relevance score for the deeplink further comprises generating the relevance score based also on the popularity score of the native application for the deeplink.

7. A system, comprising:

a data processing apparatus; and

software stored in non-transitory computer readable storage medium storing instructions executable by the data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising:

each native application is an application operable to generate environment instances for display on a user device and operates independent of a browser application; and the scoring comprises:

for each deeplink:

8. The system of claim 7, wherein:

9. The system of claim 8, wherein determining the steadiness score for each native application comprises:

determining, for a first predefined time period, a first number of deeplinks that specify environmental instances having content that include the term; determining, for a second predefined time period that is different from the first predefined time period, a second number of deeplinks the specify environmental instances having content that include the term; and

10. The system of any of claims 7 to 9, wherein:

11. The system of any of claims 7 to 9, wherein:

providing search results to the user device according to an order based, at least in part, on the ranking according to their respective relevance scores comprises:

ranking the native applications according to their respective application link scores;

for a set of search results to be provided to the user device:

12. The system of claims 7 to 10, further comprising: the scoring further comprises, for each native application, determining popularity score for the native application that is a measure of popularity of the native application relative to other native applications; and

13. A non-transitory computer readable storage medium storing instructions executable by a data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising:

the scoring comprises:

for each deeplink:

14. The non-transitory computer readable storage medium of claim 13, wherein: the scoring further comprises, for each native application, determining a steadiness score that is a measure of the relative change of distribution of the term in the content of each environmental instance of the native application over a predefined amount of time; and

15. The non-transitory computer readable storage medium of claim 14, wherein determining the steadiness score for each native application comprises:

16. The non-transitory computer readable storage medium of any of claim 13 to 15, wherein:

the scoring further comprises, for each native application, determining an application link score indicative of the number of deeplinks in the native application for which respective content includes the term relative to total number of all deeplinks of all native applications; and generating a relevance score for the deeplink further comprises generating the relevance score based also on the application link score of the native application for the deeplink.

17. The non-transitory computer readable storage medium of any of claims 13 to 15, wherein:

for a set of search results to be provided to the user device:

18. The non-transitory computer readable storage medium of any of claims 13 to 16, further comprising:

the scoring further comprises, for each native application, determining popularity score for the native application that is a measure of popularity of the native application relative to other native applications; and