WO2012116763A1 - Searching - Google Patents

Abstract

The present invention relates to searching, in particular guided searching through a database of stored objects. A user is presented with proposals of objects and is invited to select the objects she considers closest to the intended objective of her search. One of the presented objects that is not selected is then replaced with a different object and the user is invited to make a fresh selection. The procedure is repeated iteratively until the user is satisfied with the result. The choice of replacement object is made taking into account the users previous choices, thereby improving the efficacy of the search.

Description

SEARCHING

The present invention relates to searching, in particular to searching through a database of stored objects.

Traditionally, searching a database is carried out by inputting one or more queries. However, there can be situations in which content searching with queries can be difficult, for example if a user cannot formulate the query in an appropriate manner. Searching by comparison can provide an alternative to such traditional methods, but this not always straightforward.

According to the present invention, there is provided a method as of allowing a user to retrieve a stored object including the steps of: (i) storing a plurality of objects and similarity information, the similarity information providing an indication of the relative similarity between each object and at least one other object, each object having a presented state in which the object is presented to the user such that the object can be selected, and a non presented state in which the object is not presented to the user such that the object can be selected; (ii) placing a plurality of the stored objects in the presented state; (iii) receiving a user input indicative of the selection by the user of an object in the presented state; (iv) in response to the selection by the user of an object in the presented state, retaining the selected object in the presented state, and replacing another of the objects in the presented state with a different object in the presented state, the different object being determined using the similarity information; and, (v) iteratively repeating steps (iii) and (iv) until an object satisfying the user has been presented, wherein the similarity information is updated to take into account the user selection. Because account is taken of the user choices, further search can be carried out quicker or more efficiently. The objects may be images that presented visually, for example on a touch sensitive display, in which case the user input may be effected through the user touch that part of the display in which the object is displayed. However, the objects may be audio objects, and may be presented though an audio interface, user inputs being received through a microphone.

Preferably, the objects are displayed on a display in the presented state, and are hidden from view in the non presented state. In such a situation, the different object will preferably be displayed in the position of the object it is replacing.

According to a further aspect of the invention, there is provided searching apparatus including: a user interface for rendering a plurality of objects; a memory for storing a plurality of objects and similarity information, the similarity information providing an indication of the relative similarity between each object and at least one other object; and, a processor, the processor being arranged, in used, to retrieve, out of the stored objects, a plurality of objects from the memory, and to present the retrieved objects to a user a the user interface, the user interface being arranged such that the user can select an object from presented, wherein the processor is configured in use to: (i) in response to the selection by the user of one of the presented objects, use the similarity information to determine a different object for presentation to the user whilst retaining the selected object; (ii) present to the user the previously selected object and the newly determined object; and, (iii) iteratively repeating steps (i) and (ii) until an object satisfying the user has been presented, the processor being further configured to update the similarity information to take into account the user selection.

The apparatus may be arranged in a local fashion, for example as a hand held user device. Alternatively, the apparatus may be distributed. It may be arranged such that a user terminal is connected through a network to a server device containing a data base with the stored objects, the user terminal being arranged to render content and to accept user inputs concerning the user choice of displayed content. According to yet a further aspect of the invention, there is provided a method of allowing a user to retrieve a stored object including the steps of: (i) storing a plurality of objects and similarity information, the similarity information providing an indication of the relative similarity between each object and at least one other object;

(ii) out of the stored objects, presenting a choice of a plurality of selectable objects to a user; (iii) in response to the selection by the user of one of the presented objects, using the similarity information to determine a different object for presentation to the user whilst retaining the selected object; (iv) presenting to the user the previously selected object and the newly determined object; (v) iteratively repeating steps (iii) and (iv) until an object satisfying the user has been presented, wherein the similarity information is updated to take into account the user selection.

The invention will now be further described, by way of example only, and with reference to the following drawings in which: Figure 1 is a functional diagram of a computer device according to the present invention;

Figure 2 shows a display;

Figure 3 shows identifiers for stored objects arranged as an array.

Figure 1 shows a computer device 10 that has a memory 12, and a display 14, each of which is connected to a processor 16. A keyboard 20 having a plurality of keys (not shown) is provided to allow a user to enter instructions for causing the processor to perform various tasks. I addition, the display 14 has a touch-sensitive surface, allowing a user to select an area of the display by touching that area. Stored in the memory are a plurality of data objects 18, which, in the present example, are images. The processor includes a random number generator 17 for generating numbers or values in random or pseudo random fashion. In overview, the computer device is arranged to facilitate the searching of a given object through the data base of objects. A user's search is guided for content by displaying randomized proposals of objects. Pairs of objects are displayed and thereby proposed on a display as shown in Figure 2, which has a first frame 20 and a second frame 22, each for receiving a respective image of an object.

The user is able to select one of the displayed objects as being the one closest to the intended objective of her search. In response to a selection by the user, a new object is displayed in place of the object not selected, and the user selects again from the currently displayed objects. Thus the selection by the user is between an object previously selected by the user and the new randomized proposal.

Searching proceeds in cycles as follows: first, two objects are proposed: object o(l) and object o(2); a user selects between them the closest to her intended object, which we may denote by Sel(l); the processor then issues a randomized proposal object o(3), and the user selects between Sel(l) and object o(3), denoted by Sel(2). This procedure keeps going until the user finally terminates her search, let's say in k stages, where k is intended to be a reasonably small number. The result of the search is then Sel(k), the last selection of the user.

The images are stored together with information which can be represented as an array as indicated in Figure 3, where each object is represented by a number from 1 to 5. For each given object, referred to as a root object, there is associated therewith an ordered list of the other objects. The order of the objects in a given list is determined by a metric indicative of their relative similarly to the root object.

In Figure 3, each list and its associated root form a column, the root of a list being at the foot of the column in which that list lies. Considering the left hand side column, the root object is object number 1, whilst the ordered list associated with that object is composed of objects 3,5,2,4. Although only five objects are shown in Figure 3, in practice there may be at least 1000 or at least 10,000 objects in the memory.

The rows are numbers i=l, i=2,i=3,l=4, with objects in row 1 being more similar to the associated root object to those in row 2, etc.. The row number of an object in a given column can be seen as a ranking value R for that object in that column. That is, R gives the order of the objects in a list in terms of their similarity to the root object of that list.

One advantage of using a ranking is that it does not require a knowledge of the absolute extent of the similarity of objects in a list with the root object, but only whether the similarity of one object is greater or less than that of other objects in the list. The objects can be ordered initially on the basis of automated objective similarity tests. In the case of images, these may be performed on the basis of a colour map.

For each object o, the processor 16 is arranged to store an indication of the popularity of that object, referred to as a popularity count p(o) for the object o. In the present example, o is a numerical label that can take values from 0 to N, where N is the total number of objects. Initially, the indication of popularity may be allocated arbitrarily, or the indication may be based on externally acquired data, for example from user reviews.

The processor 16 is arranged to compute weighting factors which are to be used in selecting objects for presentation to a user. For each ordered list, each object i is given a weighing factor W, where W=p(o(j))/r(j). Here, p(o(j)) is the popularity count of object o in row i. The value r(j) is the sum of the popularity counts p(o) over the i objects most similar to the root object of the list. Thus, considering the first column, whose root is object 1, r(l) is simply p(o(4)), the popularity count for object 4. For r(2) , we have r(2) = p(o(4)) + p(o(2)), and for r(3) we have r(3) = p(o(4)) + p(o(2)) + p(o(0)) etc...The weighting factor W for object 4 is W(l)=p(o(4))/r(l), since object 4 is in the first row, and the weighting factor W for object 2 is W(2)=p(o(2))/r(2), since object 2 happens to be in the second row etc....

In this way, a set of weighting factors is obtained for each (root) object in the data base, each set containing weighting factors for rows i=l, ...,N, where N is the total number of objects in the data base.

Next, a normalization constant Z is computed by the processor 16, the normalization constant being the sum over i=l,...,N of the ratios p(o(i))/r(i) explain above.

The randomized object proposal is performed as follows: initially, in response to a user instruction indicating that a search is to begin, the processor selects two objects at random or according to a predetermined rule, for example in dependence on the popularity count: more precisely, the two objects having the highest popularity counts values may be chosen. Alternatively, a probability distribution which puts a weight on an object that is proportional to its popularity count may be used.

The selected objects are proposed to a user by processor. This is done by respectively displaying these objects in the first and second frames of the display.

Subsequently, the following further steps are carried out.

1. The user selects one of the pair of displayed images by touching the display in the area of the selecting image. It is assumed that out of the displayed pair, the selected image is the one that most closely matches the image the user wishes to find. 2. In response to the user choice, the processor 16 designates the image chosen by the user as being a root image. The processor then replaces the image not chosen with a different image. This different image is selected an image from the list associated with the root image in a random fashion, but with a weighting given by the weighting factor for each object in the list. Thus, the probability of a given object being selected is given by its weighting factor divided by the normalization constant. That is, the probability of image in row i being selected is given by W(j)/Z. The processor performs this task by using its random number generator to generate a random rank R, equal to any i=l,...,N where there are N objects, with a probability given by the ratio p(o(i))/r(i), divided by the normalization constant Z. As required, the sum of these probabilities is 1, by definition of the normalization constant Z. The image selected or proposed by the processor as next alternative to the user is the image that is the R-th closest to the current selection Sel, where R is the randomly selected rank value (here an integer).

3. The images displayed at this stage are the image previously selected by the user and the image freshly selected by the processor. The user then makes a further choice as between these two images and enters that choice as a user command by touching the display in the area of the chosen image.

Steps 2 and 3 above are repeated in a cyclic fashion, each time a new root image being designated by the processor 16 as being the last image chosen by the user. Consequently, the user is guided in an iterative fashion to navigate through the data base towards her intended target.

The search procedure is terminated when the image chosen by the user is the last image to be selected by the processor, as it is assumed that the user has found the image she was looking for, or at least is satisfied with the result of the search so far.

Referring back to Figure 3, if images 3 and 5 are initially selected by the processor, and the user chooses image 3, the processor will then designate image 3 as the root image, and select an image from the ordered list for image 3. If, say, this selection results in image 2, the processor will replace image 5 with image 2, and display an indication to the user that she should choose again. If the user chooses image 3, the processor will designate image 3 as the root, and again choose an image at random (but with weighted probability) from the list for image 3, and will display this newly selected image in place of the image for image 2, etc...When the user chooses the last image selected by the processor (rather than the one before last), the procedure is terminated.

When a search is terminated, the processor takes into account of the choices of the user in order to improve the efficiency of future searches, if any. To understand how this is done, it is helpful to assume that the search has succeeded after k steps, with successions of objects proposed by the processor being object o(0) and object o(l) initially, then object o(2), object o(k), and the corresponding objects selected by the user being Sel(l) which is either object o(0) or object o(l), Sel(2) is either Sel(l) or object o(2),etc, until finally Sel(k) is necessarily object o(k), the last proposal made by the processor matching the user's expectation.

The processor is arranged to update the popularity count p(o(k)) of the target Sel(k). In the case where the processor simply counts the number of times Sel(k) has been the successful outcome of a search over a period of time, for example in the past week, the processor increments by 1 p(Sel(k)) after this search, stores the date at which this search occurred, and decrements such counters as p(Sel(k)) by 1, one week after the date of a successful search.

In a subsequent operation performed by the processor, the list corresponding to the last selected object Sel (the target object) is re-arranged. The objects therein are rearranged according to their proximity to the search target, Sel. In other words the respective rankings of other objects with respect a given object are updated after the completion of a search. This is done to reflect the fact that the search operation has revealed how the user ranks objects with respect to this particular target object.

For example, if in the first step, Sel(l) happened to be object o(l) rather than object o(0), then this reveals that object o(l) is closer to Sel than object o(0). To take this possibility into account, the processor is arranged to compare the existing ordering of objects according to their proximity with respect to Sel (the target object) with the order implied by the user through her search inputs. That is, the processor is arranged to check whether the previous order is consistent with the choices made by the user. For example if object o(l) occurs before object o(0) in this list then the ordering of objects can be left unchanged. In the opposite situation, where object o(l) occurs before object o(0) in this list, then this is clearly inconsistent with the choice made by the user, who indicated through her choice that that object o(l) is closer to Sel than object o(0). Consequently, the processor swaps the object o(l) and object o(0) within this list. These verifications are made in respect of choice made by the user. For instance, if among Sel(l) and object o(2), Sel(l) was deemed by the user closer to Sel than object o(2), so that Sel(2) was equal to Sel(l), it is again checked whether Sel(l) appears before object o(2) in the list relative to destination Sel, and the position in the list of two objects are interchanged if this is not the case. That is, with reference to Figure 3, the row numbers of the two objects are swapped. The logic for swapping is applied in this way in respect of each step of the search. Instead of pictures, the computer device 10 may be arranged to allow a user to browse through music pieces or other forms of content. In this situation, the computer device will have a audio interface (not shown) which allows music pieces to be played or otherwise rendered. In this situation, the user will hear a piece of music, and will be invited to input an indication of whether the rendered piece is closer than the last to her target search result.

Although only two images a displayed for the user to select from, more images or other objects could be displayed, the user choosing the most similar one as in the above embodiments.

The weighting factor W can be seen as representing a probability distribution which is chosen for the following reason. One can view the candidate examples (that is, the stored objects) as organized in increasing circles around the current best object so far, from the most similar to the less similar, with in the first circle say 10 objects, in the second twice that much, in the third four times as many, etc, doubling each time the number of objects within the next circle. Then, by selecting according to the proposed probability distribution, one ensures that objects are drawn from the first circle with probability proportional to 1, from the second circle with probability proportional to ½, from the third circle with probability proportional to 1/3, etc. (k-th circle chosen with probability proportional to 1/k). This particular combination of slow decay in the probability of circle selection, and fast increase (exponentially fast) of the number of candidates within each circle, provides a particularly good trade-off between frequency of proposals of very similar objects, needed to guide the final stages of the search, with proposals of very different objects, needed to move quickly across the collection of objects in the early stages of a search.

As will be appreciated from the above description of embodiments of the invention, these pertain to searching in a database of objects through comparisons. Contrary to traditional databases, the need is reduced for users to submit queries that are subsequently matched to objects. A search for a target object is conducted in several phases. In each phase, a user is presented with a small list of database objects. The user selects from this list the object that is most similar to the target she wishes to find. A new list is then presented to the user, and the above process is repeated. When the target item is among the items presented to the user, she retrieves this item and the search terminates.

The above embodiments have several important real-life applications. A example is navigating through a database of pictures of humans, such as Flickr or Picasa, in which subjects are photographed under a variety of non-idealized conditions. For example, the pictures can be taken indoors or outdoors, from different angles or distances, while the subjects can assume different poses, be partially obscured, etc. Using automated methods to identify similarities between two subjects can be difficult in this scenario. On the other hand, a human user wishing to locate the picture of a particular person can easily compare two pictures and tell which subject is more similar to the person she has in mind, based on their sex, age, facial features, etc. However, she may not be able to quantify how great this similarity is by, e.g., assigning it a numerical value. The above example can be generalized various databases of objects, in particular those whose objects can be easily compared by humans but whose similarity is difficult to quantify in terms of magnitude.

A need for such searching can arises when database users find it difficult to state and submit explicit queries. For example, users may be unable to submit queries because they not start with a clear target in mind when initiating a search. A user surfing the web might wish to navigate the blogosphere to find an interesting topic, without knowing a priori which post she wishes to read; presenting a list of blog posts and letting the surfer chose the one she likes the best can steer her in the right direction. Alternatively, users might be unable to issue queries because they are unfamiliar with the search domain: a novice classical music listener wishing to navigate through a collection of recordings might not be able to express that she is, e.g., looking for a fugue or a sonata. She might however be able to listen to samples from two musical pieces and select the one closest to the one she has in mind.

In the above applications, the problem of content search through comparisons can be viewed as a way of determining which objects to present to the user in order to find the target object as quickly as possible. This may be particularly true under the scenario of heterogeneous demand. In practice objects in a database are indeed unlikely to be requested with the same frequency. Ideally, assuming that some objects are more likely to be targets than others, it is desirable to exploit this information to further reduce the number of queries submitted to the oracle. In summary, the embodiments involve using a database that displays pairs of objects to users. Users select the object that is closest to a target object that they have in mind, and report this to the database. The database then selects another pair of objects to present to the user and this process is repeated until the target object is found. In the background, the a processor learns from the user selections how similar are objects to each other, simply by monitoring the user selections. It also monitors how frequently each object is the target of searches by users. It then uses this information to "improve" the pairs of objects it presents to users, in order to reduce the time until the target is found.

Claims

1. A method of allowing a user to retrieve a stored object including the steps of:

(i) storing a plurality of objects and similarity information, the similarity information providing an indication of the relative similarity between each object and at least one other object, each object having a presented state in which the object is presented to the user such that the object can be selected, and a non presented state in which the object is not presented to the user such that the object can be selected;

(ii) placing a plurality of the stored objects in the presented state;

(iii) receiving a user input indicative of the selection by the user of an object in the presented state;

(iv) in response to the selection by the user of an object in the presented state, retaining the selected object in the presented state, and replacing another of the objects in the presented state with a different object in the presented state, the different object being determined using the similarity information; and,

(v) iteratively repeating steps (iii) and (iv) until an object satisfying the user has been presented, wherein the similarity information is updated to take into account the user selection.

2. A method as claimed in claim 1, wherein the similarity information is updated to take into account the user selection after the search as terminated.

3. A method as claimed in claim 1 or claim 2, wherein objects are stored with a ranking, the ranking of objects indicating their relative similarity to a given object, and wherein the ranking of objects is updated to take into account the selections of the user.

4. A method as claimed in any of the preceding claims, wherein the objects each have a popularity indicator associated therewith, the popularity indicator being indicative of the popularity of the associated object.

5. A method as claimed in claim 4, wherein the popularity indicator of an object is increased when the user selects that object.

6. A method as claimed in any of the preceding claims, wherein the search is terminated when the user selects the last object placed into the presented state.

7. A method as claimed in claim 6, wherein the popularity indicator of the last object selected by a user before a search is terminated is increased.

8. A method as claimed in any of the preceding claims, wherein the objects presented to a user are selected in dependence at least in part on a random or pseudo random variable.

9. A method as claimed in claim 8, wherein the random or pseudo random variable is weighted for each object with a weighting value dependent at least in part on a probability indicator that provides an indication of popularity for that object.

10. A method as claimed in claim 8 or claim 9, wherein the weighting value for a given object depends on one or more probability indicators of other objects.

11. A method as claimed in claim 10, wherein the weighting value for a given object is dependent, at least in part, on a ratio of a first quantity and a second quantity, the first quantity including a probability indicator for that object, and the second quantity including a sum of probability indicators.

12. A method as claimed in any of the preceding claims, wherein the objects are representative of images.

13. A method as claimed in any of the preceding claims, where a user selects an object from a pair of rendered objects.

14. A searching apparatus including:

a user interface for rendering a plurality of objects;

a memory for storing a plurality of objects and similarity information, the similarity information providing an indication of the relative similarity between each object and at least one other object;

and, a processor, the processor being arranged, in used, to retrieve, out of the stored objects, a plurality of objects from the memory, and to present the retrieved objects to a user a the user interface, the user interface being arranged such that the user can select an object from presented,

wherein the processor is configured in use to:

(i) in response to the selection by the user of one of the presented objects, use the similarity information to determine a different object for presentation to the user whilst retaining the selected object;

(ii) present to the user the previously selected object and the newly determined object; and,

(iii) iteratively repeating steps (i) and (ii) until an object satisfying the user has been presented, the processor being further configured to update the similarity information to take into account the user selection.

15. Searching apparatus as claimed in claim 14, wherein the apparatus is distributed over a network.