GB2537139A - System and method for processing and retrieving digital content - Google Patents

Abstract

A system for processing and retrieving digital content, preferably digital photographic images. The system most usefully provides a commercial professional photography solution whereby modern facial recognition and mapping techniques are applied to high-resolution digital images captured by professional photographers, for example during an event such as a wedding, and a database is created which correlates the digital identifiers or face maps which are established for each human face identified within the digital image to the image itself. This database provides a look up function whereby it is possible to extract all the digital images in which a particular individual is pictured. In practice, any individual attending the event and having appeared in one or possibly many earlier captured digital images can visit a stand or kiosk provided by the photographer which is provided with a display screen and comparatively low-resolution camera, web-cam or digital imaging device, both of which are connected to a local computer which hosts or is in communication with the database. Despite the imaging device at the kiosk being of comparatively much lower quality than the camera of the professional photographer, importantly the face of individual visiting the stand can be captured thereby in relative close-up, whereupon the computer can apply the identical facial recognition and mapping techniques used on the much higher quality digital images, and produce a digital identifier or face map for that individual which can then be used in a database query, the results thereof being all the stored, high-quality digital images (or references thereto) in which that individual appears.

Description

SYSTEM AND METHOD FOR PROCESSING AND RETRIEVING DIGITAL CONTENT

Field of the Invention

The present invention relates to a system and method for processing and retrieving digital content, such term encompassing digital photographs, video, artwork, multimedia and the like, and specifically to a system and method adapted for processing and retrieving multiple discrete instances of digital content, each depicting or containing one or more identifiable entities.

In most common applications of the invention, the digital content will be a digital image, is and the digitally recognisable entity will be a human face appearing in the image.

However, the skilled reader will be aware that modern known digital image, video and other multimedia processing and analysis techniques can be adapted to identify any desired entity appearing in the processed content, and therefore the particular nature or form of the entity is of lesser importance than the requirement that the entity appearing the digital content can be identified therein by modern software and/or hardware-driven analyses and techniques.

Furthermore, although the following description is provided with almost exclusive reference to the processing and retrieving of digital content in the form of images, specifically photographs, the invention should not be considered as limited thereto, as it will immediately understood by the skilled reader that digital content can be subjected to a great many different entity identification processes and techniques depending on the entity sought to be identified in, and commonly appearing in, that digital content.

Background to the Invention

The vast majority of commercial professional photographers, of the type commonly employed to attend and photographically capture events and celebrations, nowadays almost exclusively use digital technologies, such as digital single lens reflex (SLR) cameras and in certain cases, digital video cameras. Examples of the events they may be employed to capture are weddings, charity fund raising events, black tie dinners and the like, or they may be specifically employed by cruise ship or theme park owners, often for periods of time longer than a single day. Regardless of the event, the photographer's presence is fundamentally commercial in that he or his employer wishes to offer photographs (or other articles having photographic imagery applied thereto) for sale to the attendees of the event, either after or during the event. As will be appreciated by the skilled reader, a single photographer using a modern digital SLR camera provided having sufficient memory capacity (often many tens of gigabytes) may take many hundreds if not thousands of photographs during the course of the event. Such convenience brings with it inherent difficulties, a prime example of which is the difficulty of easily and/ro quickly correlating the digital images with the subjects, that is (most usually) the human individuals, appearing in them.

A further difficulty brought about not only by the digitisation of photography but by the digital revolution in general is that individuals have ever increasing expectations for instantaneity. Modern smart devices for instance include cameras from which captured digital images can appear on the screen of the device immediately. The on-screen image can then, almost as rapidly, be transmitted or shared with others, either directly on a peer-to-peer basis, or through social media channels. Accordingly, professional photographers are under increasing demands to provide their work to customers entirely in the digital domain, and similarly quickly. While digital photography does facilitate relatively easy and quick display of the digital images captured (they can simply be transferred en-masse from the camera memory to a computer attached to a display unit, and a simple slide show initiated thereon), individuals wishing to view images in which they appear are required to scan through a great many more images in which they do not appear. This is very off-putting to potential customers. Of course, captured digital images take the form of binary computer-readable files having time stamps, and therefore it may be possible for individuals to request or select for viewing only those digital images captured around some specific point in time, but this is a very basic form of selectivity, and in any event can still require scanning many images captured around this time. Even when a particular image in which a subject is pictured is inspected, that image may be unflattering to the subject and would therefore be instantly rejected thereby.

Solutions have of course been proposed for the above problems, as follows: Make physical prints of all the images for people to view, or simply display the images on a TV or computer screen, e.g. in a slide show, and Provide all those individuals attending the event or present for a particular photo with either - a simple numbered or otherwise identified card, or - a radio-frequency identifier (RFID) tag, and before taking any and every photograph, a note of the card or RF scan is performed, so the photographer has some means of correlating any particular photo to the individuals appearing within it.

These solutions are of course riven with drawbacks: In terms of (i) above, drawbacks include: -producing prints is a lengthy and costly exercise (especially as it is rare that all prints are subsequently sold), and they are somewhat outdated as far modern technologies, particularly social media, are concerned, - displaying all the prints (or images) utilises a lot of space and/or requires a custom built display stand, -it is also impossible to produce other products from a print (eg. products capable of personalisation with photographic imagery, such as keyrings, coasters, placemats, crockery etc.); As regards (ii) above, drawbacks include: - individuals often lose their numbered identifier card thus requiring a reversion to a manual, and thus laborious image search, if individuals wish to view images in which they have been pictured immediately after the photograph was taken, then usually a viewing/display screen will be required immediately adjacent the photographer; this is not always possible; also individuals may only wish to view images some time after the photograph was taken (e.g. after a cloakroom/bathroom visit); in this case the numbered card is no longer valid, and a manual search must again be performed; - In cases where individuals are provided with a RFID tag, this system requires the individuals to pre-register themselves with the specific RFID tag they have been given; this is a yet further complication to an already very expensive system.

Some or all the prior art systems suffer from the following commercial drawbacks: if an individual requests a copy of a particular image (i.e. wishes to buy the image), then further involved interaction is required with the photographer to take the order, or alternatively, if photographer is still taking photos, an assistant is required to effect the placement of the order, - individuals wishing to share purchased images on social media or image sharing websites (such being very useful to the photographer in terms of marketing both himself and the event or organisers thereof) are either required to login to the relevant website directly on the display computer, which the photographer may understandably be reluctant to allow, or alternatively the photographer may have to either email the image to the individual or provide a link to the image if stored in a cloud-based service, - the workflow of the order placement process is slow and laborious and requires valuable sales screen and photographer/assistant time.

In one aspect therefore, an object of the present invention is to provide a commercial digital photography system whereby digital images can be both received and processed by the system, and subsequently retrieved from or by the system for display in a selective and relevant manner as regards the individuals appearing in those images.

In a broader aspect, it is an object of the invention to provide a system and method for processing and retrieving multiple discrete instances of any form of digital content (a discrete instance encompassing, for example, both frames of digital video and digital video files comprising multiple frames alike), each discrete instance of which contains one or more identifiable entities, wherein the system and method facilitate the retrieval of those discrete instances of digital content which contain the same entity.

Summary of the Invention

According to the present invention there is provided a system for receiving, processing and storing multiple discrete instances of digital content each containing at least one identifiable entity, and for subsequently retrieving one or more of those stored selected instances of said digital content, said system comprising primary digital content capture means capable of capturing discrete instances of said digital content, entity recognition means which subjects each discrete instance of digital content originating from said primary digital content capture means to an image processing function which establishes a primary identifier for each entity recognised in said instance, said primary identifier being at least partially representative of one or more digitally or physically observable entity features by which one entity is distinguishable is from another, storage means to which each discrete instance of digital content is written entirely or substantially so, and to which is also written each and every identifier established therefor, characterised in that said system further includes secondary imaging means capable of producing secondary discrete instances of digital content which are subjected to the same or an identical image processing function so as to establish one or more secondary identifiers for the entities recognised therein, and display means, and further characterised in that said storage means said storage means further comprises lookup means which provides a link between the stored discrete instances of digital content and the set of primary identifiers established for each thereof, said secondary identifiers being submitted to said lookup means and compared with said primary identifiers therein, whereupon said lookup means returns a set of all those discrete instances of digital content whose primary identifiers substantially match said secondary identifiers, said system subsequently causing display of one or more of the discrete instances of digital content within said set on said display means.

Preferably, the digital content is digital imagery, and the discrete instances thereof are individual digital photographic images.

Most preferably, the entity capable of being recognised within any discrete digital photographic image is a human face.

Most preferably, the entity recognition means entails at least some aspect of facial recognition, and the image processing function forming part thereof includes a digital facial distillation process whereby a human face, or more particularly the various physical features, attributes, characteristics and general physiognomy is distilled down to a set of indicators which together make up the said identifier by means of which any human face may be digitally characterised and thus may be digitally, and almost if not totally uniquely, distinguished.

is Preferably, the primary digital content capture means is of relatively much higher calibre, e.g. being capable of capturing instances of digital content of much greater quality, than the secondary imaging means. For example, the primary digital content capture means may be a digital SLR camera, and the secondary imaging means may be a simple web camera ("webcam"), smartphone camera or other camera which comparatively is - of much lower cost, - is only capable of capturing or producing digital content of much lower resolution, and - is ideally suited for taking pictures containing only relatively few subjects, i.e. not possessing any great depth of field.

In a preferred embodiment, the system is capable of adapting its processing depending on the calibre, quality or some other indicator present in, or forming part of, the discrete instances of the digital content, or depending on the manner in which they are received, or from which source. Ultimately, the system is capable of processing digital content from the primary digital content capture means differently from that received from the secondary imaging means.

Most preferably the secondary imaging means is disposed proximate the display means, most preferably affixed thereto such that digital images of individuals passing casually thereby are automatically provided to the system which in turn retrieves and displays, preferably either consecutively in a slide show, or simultaneously in a montage, previously stored images in which one or more of those casual passers-by appeared.

In a most preferred embodiment, the secondary discrete instances of digital content are only transiently stored by the system, and most preferably only for as long as any entity recognised therein is also recognised in a temporally later-captured secondary discrete instance of digital content. By this technique, only images in which an individual, at that time proximate the display, appears are displayed -as soon as that individual moves away, and another individual moves into the field of view of the web-camera, the system automatically adapts and displays only images in which the second individual appears. Accordingly, and preferably, the system discards secondary discrete instances of digital content when it determines they are no longer relevant.

is In a particularly preferred embodiment, the secondary imaging means produces discrete instances of digital content intermittently, further preferably periodically.

Although the system is ideally suited for commercial photography, the inventor envisages that the system could have beneficial application, for example, as a learning or play device for children. For instance, if the entities appearing in images were animals as opposed to human faces, and a large number of pre-stored images depicted different animals in their natural habitats, it would be possible for a child or teacher or present an image of a single animal in front of the display screen and attached webcam, whereupon the system would display of a variety of different images on the display in which that animal was pictured. The play and educational value of such a system could be significant.

Preferably, said lookup means compares the secondary identifiers submitted thereto with the primary identifiers for one or more of: -identity, and -a similarity so substantial that it is more probable than not that the primary and secondary identifier so compared identify the same entity.

In a most preferred embodiment, the system further comprises a commerce function which facilitates the purchase by an individual of any of the images currently being displayed to that individual and in which, by virtue of the proximity and orientation of that individual as regards the display screen and the camera attached or proximate thereto and the system as a whole, will appear in.

In a particularly preferred embodiment, the storage means and the lookup means may be consolidated in a single database capable of storing both the discrete instances of digital content, usually one or more fields capable of containing binary large objects (BLOBs) as well as the primary identifiers established therefor.

In an alternate aspect of the invention there is provided a method of processing multiple discrete instances of digital content each including at least one digitally identifiable entity therein, said method including the steps of: -receiving a discrete instance of digital content of a first calibre, - processing said instance to identify one or more entities within said instance and create a set of primary identifiers, one for each entity so identified, said identifiers being at least partially representative of one or more digitally or physically observable features of that entity by which one entity may be distinguished from another, - storing both the digital content discrete instance and the set of primary identifiers established therefor in a correlated manner, - repeating the above for second and further digital content discrete instances being of similar or identical calibre, the method being characterised by the further steps of: - receiving a discrete instance of digital content of a second calibre which is substantially different from said first calibre, - processing said second calibre instance in identical manner to said first calibre digital content instance so as to provide a set of secondary identifiers of the same form and type as the primary identifiers, - comparing the set of secondary identifiers to the all the stored primary identifiers such that a subset of said stored primary identifiers is returned which are identical or so similar to said set of secondary identifiers that it is almost certain that the identifiers relate to the same entity, - correlating the returned subset of primary identifiers to the discrete instances of digital content of the first calibre to which they relate, and - retrieving one or more of the said discrete instances of digital content of the first calibre.

Preferably, the method includes the further step of displaying the one or more discrete instances of digital content of the first calibre so retrieved.

Preferably, the method includes the further step of recognising the particular calibre of the received discrete instance of digital content, and adapting the processing thereof accordingly.

In one embodiment, the method includes a requirement for specific user activation of the receiving of a discrete instance of digital content of a second calibre. For instance, a user may be require to activate the web-camera or the like prior to that camera capturing an image of the user activating it.

Although the captured discrete instances of digital content may be stored locally as part of the system, it is of course possible that the storage may be remote, networked, or web-or cloud-based, and indeed any storage facility may be contemplated provided that the a host computer (that on which the software providing the system or method is currently installed and executing) can access said storage, and more specifically read from and write to that storage. It should however be mentioned that certain storage options may suffer from increased latency as regards loading and displaying relevant images.

Preferably, multiple discrete instances of digital content of a second calibre are received, for example in the form frames comprised in a stream of video from any type of digital camera. In a preferred embodiment, the method includes the step of identifying one or more entities which repeatedly appear in said frames, and subjecting said repeatedly appearing entities to processing to establish an identifier for each said repeating entity.

It is preferable that the system and method include facility for adjusting a matching tolerance as regards the comparison of the secondary identifiers with the primary identifiers.

Other preferable features abovementioned in respect of the system of the present invention are equally applicable to the method.

The discrete instances of digital content of the first calibre will usually be digital images captured by a digital SLR camera, and may be transferred by any suitable communications channel, e.g. by physical cable connection between camera and computer, over the internet, wirelessly, etc. In a third aspect of the invention there is provided a computer program comprising computer program code means adapted to perform all the steps of any of the methods is above when run on a computer.

In a further aspect of the invention, there is provided a computer program when embodied on or in a computer readable medium.

The invention provides a system of great practical and commercial advantage, especially at events and functions where large numbers of individuals are in attendance, because and individual simply passing the display of the system can automatically have displayed to them some if not all of the earlier images captured by the photographer and in which they appear.

The inventor herefor has realised that facial recognition techniques can be usefully and commercially applied in circumstances where the faces of individuals may appear multiple times, for example in successive digital photographs, and in circumstances where multiple individuals' faces appear in a single photograph, e.g. a group photograph, and has made the further realisation that such techniques can be usefully employed in providing a very commercial and practical professional photography system.

In terms of specific advantages, it will be immediately understood by the skilled reader that no pre-registration is required to match the an individual with any image in which they appear. Furthermore, the system can provide an extremely simple user interface, and therefore little or no training on, or experience of, the software is required by the individuals wishing to see images in which they appear-the system operates completely autonomously. Yet further, digital photographic images are available for viewing with no practically no interaction whatsoever from the photographer or their assistant. Even further advantageously, the system is largely if not entirely language-agnostic -it is an entirely visual system.

A specific embodiment of the invention is now described by way of example and with reference to the accompanying drawings wherein.

Brief Description of the Drawings

Figure 1 provides a schematic illustration of a system according to the present invention, and Figure 2 shows a flow diagram schematically representing the various steps in a commercial or point-of-sale (POS) system based on the invention.

Detailed Description

Referring firstly to Figure 1, a system according to the present invention is indicated generally at 2. The system 2 comprises firstly a digital content capture device 4, typically a digital SLR camera commonly used by professional photographers, usually having onboard or removable memory of sufficient capacity (not shown) to store a great many high resolution digital images, possibly all those captured throughout an entire day.

Professional photographers are commonly employed to photograph events having a large number of attendees, such as weddings, gala dinners, and the like, and naturally the photographer will take a large number of photographs during the event, both of individuals and of groups. In Fig. 1, camera 4 is shown focussing on a group 6 of nine individuals 8A, 8B, 8C, 8D, 8E, 8F, 8G, 8H, 81, all of whom will thus appear within the photograph once it is taken. During the course of the event, the photographer may take a great many photographs, each of which results in the creation of a high-resolution digital image which is usually immediately written to the camera memory. In essence, the digital image file is simple a computer-readable binary file. In some cases it is possible for the digital camera to transmit the digital image wirelessly to a computer 10, but in most cases, to transfer the digital images stored on the camera memory, either the it is removed and inserted into a memory card reader device (not shown) attached to the computer, or even more simply, the photographer will merely physically connect the camera to the computer 10. Regardless of the manner in which the digital images are transferred from camera to computer, the data transfer betwixt the two is generally represented at 12.

Once the digital images are stored locally in computer 10, they are then transferred, at 14 to image processing and mapping functions 16, 18 respectively. Although the image is processing and mapping functions are shown in the Figure as discrete entities for the purposes hereof, one may form part of the other, or both functions might better be considered as a single image processing function. In any event, the aim of said functions 16, 18 is firstly, in any digital image submitted thereto for processing, to identify portions or specific sets of pixels within the entire digital image which have a high probability of depicting the face of an individual, and then secondly, to provide some form of digital identifier or "face map" for each of the faces so identified. The digital identifier is not merely some generic identifier which bears no resemblance whatsoever to the individuals face, but instead it is a digital (and therefore searchable) key or "map" which essentially characterises the features which may or may not be present in any individuals face.

To provide an exceedingly simplistic example, a simple 7 bit bitmask might provide a searchable identifier which might also represent of common characteristics of a face, for example: Blue or green eyes/Brown eyes (0/1) Eyes Closer/Further apart than average (0/1) Larger/smaller nose (0/1) Larger/smaller mouth (0/1) Larger/Smaller face size (0/1) Larger/smaller Forehead/Face ratio (0/1) Darker/lighter skin tone (0/1) Thus, even using a simple bit structure, it is possible to provide not only a computer-readable or digital identifier which in some way represents the features or characteristics of human individual faces, but also a means of searching for individuals possessing specific characteristics, albeit a simple one. In this case, simple bit-wise logic operators such as "AND", "OR", "XOR" etc may be used to search for individuals possessing specific characteristics, as will be readily understood by the skilled reader.

The example above is of course a simplistic one, and importantly it can be seen that there are only 127 (27-1) unique values. However, an identifier of only 64 bits (and having unique values of 2m-1 = 8589934591) could quite capably be unique for the every individual on the planet.

One additional aspect of the simplistic example provided above is worth mentioning-the degree of correlation between: one identifier and another which differs from the first only by a single or maybe two bits, and (ii) the similarities in the human individual faces which, appearing in one or more different digital images and which, as a result of the processing described above, produced those two similar, but different identifiers.

Continuing with the above example, a 7 bit word '1101111' is of course very similar to the 7-bit word '1111111', and the context of the facial characteristics those bits represent, the only difference is in the third bit which defines whether an individual has a "larger/smaller nose". In this case therefore, not only are the bit words similar, but it is highly likely that the faces of the individuals represented thereby are also very similar. It might be considered that the third bit in the 7-bit word has a relatively Low difference weighting -meaning that a difference in this bit is of relatively low importance as far as the physical appearance of the two different faces represented thereby. Conversely, the 7th bit in the 7-bit word ('darker/lighter skin tone') might have a much larger difference weighting, because the physical appearances of the human face represented by the 7 bit word '1111110' and that represented by '1111111' would be instantly differentiable.

The above is an important consideration for the present invention, as will become apparent from the below, but for now it is sufficient to mention that commercially available facial recognition and mapping software is available from Luxand Inc (e.g. specifically their "FacesSDK" product).

However, returning to Figure 1, after having been subjected to the image processing and mapping functions 16, 18, each and every digital image I. (separately referenced at 20) may have attributed thereto a set 22 ({a) comprising, zero, one or more digital identifiers (or "face maps"), depending on the number of human faces identified and mapped within said image. An empty set (or NULL value) will of course result if the image processing and mapping functions being unable to identify and/or recognize the presence in the image of any human face -a quite possible and common occurrence. Both digital image I. and the set of identifiers 22 are then either both stored in a database 24 (which may be reside locally within the computer 10, or remotely and externally of the computer, for example on a network drive or share, in the cloud, or even just simple on storage device directly attached to said computer 10), or only the set of identifiers is stored in said database, along with some referential information which enables the system to quickly, readily and accurately locate the particular digital image for which that set of identifiers is relevant. The skilled person will naturally understand that a great many different architectures are possible, but it is important for the invention that the set of identifiers created for any particular digital image are somehow correlated or linked to, or reference that original digital image, and/or the storage location thereof.

In this embodiment, the actual digital images k are shown in one table 26 as 11,12,13, and a further table 28, shown expanded at 30, is provided wherein a first field 32 contains an indicator value referencing a particular digital image, and a second field 34 contains a single digital identifier value. As can be seen from expanded view 30, table 26 has multiple rows 26A, 26B, 26C, 26D, 26E, 26F, 26G, 26H, 261, each of which contain the same digital image indicator 11, but which each contain a different 7-bit binary value for the digital identifier. In essence therefore, this table is the digital representation that one digital image 11 contained the faces of 9 different human individuals, specifically those at 8A 8B, 8C, 8D, 8E, 8F, 8G, 8H, 8I originally pictured in the group photograph.

Turning now to the user/customer interface aspect of the system, at some time after the said group photograph has been taken, and the digital image 11 has been processed as described above, one of the individuals, for example 8C, visits the display or sales stand, kiosk, counter or other outlet (not shown) of the photographer. This stand may be manned, but it need not be, and will comprise at least a display 40 atop which is affixed a webcam 42 or similar relatively lower quality and/or inexpensive camera or imaging device having its lens generally directed in the same direction as the screen 44 of the display 40, i.e. generally forward facing. By this arrangement, any individual, couple or small group of individuals passing by the stand will at some point in their travel, automatically be disposed within in the field of view 46 of the webcam 42, and the height of the display and thus the webcam will be such that the field of view 46 will target the faces of individuals. Alternatively, the system may be provide, on screen 44, touchable user interface elements (in such embodiment, display 40 may be a touch-sensitive display), and casually passing individuals would be invited to initiate the system by, for example pressing an on-screen "start" button or similar. Inviting individuals to interact with the system in this manner would again have the desired effect of automatically disposing the face of the individual within the field of view 42 of the webcam, if suitably mounted, orientated and positioned. Regardless of how many individual faces there are, the overriding concern is that they are relatively generally very close to the webcam when an image, multiple images or frames of digital video are captured thereby.

In the particular embodiment depicted in Figure 1, webcam 42 captures the face 8C-1 of individual 8C in relative close-up in a digital image Pi at 48. This digital image is transmitted at 50 (wirelessly in some embodiments, but usually over a physical link) to computer 10, wherein said image is subjected to exactly the same image processing and mapping functions 16, 18 to which the relatively much high resolution images I. from the high-quality digital camera were subjected. It is possible for the image processing and mapping functions to be slightly different as between their processing of images Ix and images Px, but not in any material way as far as the establishment of the digital identifiers are concerned, as it is critical to the invention that the same general algorithm be applied.

The primary difference in processing images Px is that they are stored only transiently by the system, or they may even be immediately or relatively quickly discarded -after all their sole purpose is to provide a basis for creation of a set 52 of one or more digital identifiers (but relatively few, given that only perhaps 1, 2 or 3 people might ever be in field of view 46) which are then used as the primary criterion in a query submitted to the database 24. There is certainly no need that images Px be stored in database 24.

In this embodiment, although set 52 is shown containing three identifiers (u, v, w), as only a single individual 8C is pictured by the webcam 42, only a single digital identifier will be established by the image processing and mapping functions 16, 18. The query to which the database 24 is subjected will of course contain the digital identifier, and the aim of the query is to return from the database (e.g. at 54) all those discrete images (or at least a subset thereof) for which an identical digital identifier has already been established (i.e. which exists in field 34), or those digital images for which digital identifiers have been established and which are sufficiently similar to that forming part of the query that in practice, it is probabilistically more likely than not that the similar but different digital identifiers actually represent the same face, i.e. the same human individual. Naturally, by virtue of the underlying comparison which the databases effects between the digital identifiers, any images retrieved from the database as a result of successful comparison are only those images in which the individual 8C appears.

Once a single or a set of digital images is retrieved from the database, they are displayed on screen 44 to user 8C, who, given the speed of processing, querying, and image retrieval will still be present and directly in front of said screen. The manner of display, when more than a single image is retrieved from the database, may take the form of a slide show of consecutively displayed images, or a montage, or a combination thereof.

It should be mentioned that because, in essence, the digital images captured by the high-quality digital SLR camera 4 and the webcam 42 are in essence only digital representations of an analogue real-world, and both will therefore be prone to artefacts and other uncontrollable factors such as lighting and atmosphere variations, there will inevitably be differences between the digital identifiers established for any individual appearing in one of the images Ix and the same individual pictured in an image Px.

However, it is the relative similarity (as discussed above) which is important -as long as the two digital identifiers are sufficiently similar, and the database is capapble of reating a query which retrieves digital images for which identical or sufficiently similar digital identifiers have been established, then the system will function properly.

Referring now to Figure 2 which expands on the system described above by providing a logical flow 60 to the user/system interaction, and provides an introduction to some additional commercial aspects of the invention.

Initially at 62, the system makes an enquiry of a user or small group thereof proximate is the screen 44, most probably by displaying the enquiry on-screen, as to whether that user or group would like to view their photographs. Answering negatively at 64 results in the system reverting to display of a simple slide show of the most recently captured digital images, such retrieved from the database 24 by means of a query including some element of date and time. However, in this standby mode, the webcam 42 is still active, and connected to the computer 10, which in turn is still active as far as determining whether a face appears in the field of view of the webcam. As soon as one or more faces is determined by the computer as being present in the field of view of the webcam (as indicated in figure 2 at logic element 66), then the system reverts to the display of the enquiry 62 described above. Furthermore, on this enquiry display, it may also be possible to make a language selection 68, although such would have an effect only on the text displayed on-screen as part of the user interface, and not the underlying mechanics of the system, which is entirely language agnostic.

If a user indicates a "Yes" (70) in response to the enquiry made of him at 62, then the system proceeds with the analysis described above in detail, and represented simply in Figure 2 at 72. As discussed, one or more images is displayed on-screen, as indicated at 74. As part of the commercial aspects of the invention, it is possible at this stage for the system to invite the user to order or purchase the image at 76, and if the user affirms such invitation at 78, then the system may proceed to offer the user a number of different products and/or product options, as suggested in logic units 80, 82. The user may also be present with various templating options, as suggested in Logic units 84, 86, 88.

At the end of the product and/or option selection process, the user is invited to order other digital images at 90. A positive response results in a return of program flow to logic unit 72 as shown at 92, whereas a negative response (at 94) results in one or more further invitations 96, 98, before the user is finally asked to confirm his order and provided payment details at 100. After the user confirms the order and pays therefor, a receipt is printed at 102, and a finally thank you message may be displayed on-screen at 104.

In terms of the hardware required for the system, the actual speed of operation etc., at the date hereof, a modern but largely standard desktop computer with largely conventional hardware and operating system software takes about 1s to recognise, process, and distil a single face appearing in a high-resolution, high quality digital image. A high-resolution digital image resulting from a large group photograph in which a great many individuals are depicted can be processed in only maybe 2-3 minutes, and a digitally (Largely unique) identifier can be established for each of the individuals appearing in said image. Thus, any of the individuals appearing in that photograph can later (even just a few minutes Later) attend the photographer's stand or kiosk and provided the photographer as transferred the digital images in his camera to the system, these can largely be immediately displayed and in relevant manner to that or those individuals. Furthermore, not only the just-taken group photograph in which that or those individuals appeared can be displayed, but also a selection (or all) of the other earlier-taken photographs in which they also appear. Commercially, this is incredibly appealing to both photographer and individual alike -the photographer can directly target any individual passing by his display by showing only those photos in which that individual appears, which in turn immediately satisfies the individual's intrinsic vanity and their tendency to scan photos only to establish whether they appear, and in turn whether that appearance is flattering or unflattering.

In summary therefore, the invention discloses a system and method for processing and retrieving digital content. Ideally the digital content is digital photographic images and the system and method most usefully provide a commercial professional photography solution whereby modern facial recognition and mapping techniques are applied to high-resolution digital images captured by professional photographers, for example during an event such as a wedding, and a database is created which correlates the digital identifiers or face maps which are established for each human face identified within the digital image to the image itself. This database provides a look up function whereby it is possible to extract all the digital images in which a particular individual is pictured. In practice, any individual attending the event and having appeared in one or possibly many earlier captured digital images can visit a stand or kiosk provided by the photographer which is provided with a display screen and comparatively low-resolution camera, web-cam or digital imaging device, both of which are connected to a local computer which hosts or is in communication with the database. Despite the imaging is device at the kiosk being of comparatively much lower quality than the camera of the professional photographer, importantly the face of individual visiting the stand can be captured thereby in relative close-up, whereupon the computer can apply the identical facial recognition and mapping techniques used on the much higher quality digital images, and produce a digital identifier or face map for that individual which can then be used in a database query, the results thereof being all the stored, high-quality digital images (or references thereto) in which that individual appears.

Claims (22)

1. Claims 1. A method of processing multiple discrete instances of digital content each including at least one digitally identifiable entity therein, said method including the steps of: - receiving a discrete instance of digital content of a first calibre, - processing said instance to identify one or more entities within said instance and create a set of primary identifiers, one for each entity so identified, said identifiers being at least partially representative of one or more digitally or physically observable features of that entity by which one entity may be distinguished from another, - storing both the digital content discrete instance and the set of primary identifiers established therefor in a correlated manner, - repeating the above for second and further digital content discrete instances being of similar or identical calibre, the method being characterised by the further steps of: - receiving a discrete instance of digital content of a second calibre which is substantially different from said first calibre, - processing said second calibre instance in identical manner to said first calibre digital content instance so as to provide a set of secondary identifiers of the same form and type as the primary identifiers, - comparing the set of secondary identifiers to the all the stored primary identifiers such that a subset of said stored primary identifiers is returned which are identical or so similar to said set of secondary identifiers that it is almost certain that the identifiers relate to the same entity, - correlating the returned subset of primary identifiers to the discrete instances of digital content of the first calibre to which they relate, and - retrieving one or more of the said discrete instances of digital content of the first calibre.
2. 2. The method of claim 1 including the further step of displaying the one or more discrete instances of digital content of the first calibre so retrieved.
3. 3. A method according to any preceding claim, the method includes the further step of recognising the particular calibre of the received discrete instance of digital content, and adapting the processing thereof accordingly.
4. 4. A method according to any preceding claim wherein the method includes a requirement for specific user activation of the receiving of a discrete instance of digital content of a second calibre.
5. S. A method according to any preceding claim wherein multiple discrete instances of digital content of a second calibre are received, each of which are subjected to an interim processing step wherein one or more entities are identified which repeatedly appear in said multiple discrete instance, and said one or more identified repeating entities are then subjected processing to establish an identifier for each said repeating entity.
6. 6. A method according to any preceding claim wherein a similarity tolerance, such being the extent to which the set of secondary identifiers must be similar to the stored primary identifiers with which they are compared so as to be considered as matching, can be adjusted.
7. 7. A computer program comprising computer program code means adapted to perform all the steps of any of the methods of claims 1-6 when run on a computer.
8. 8. A computer program according to claim 7 when embodied on or in a computer readable medium.
9. 9. A system for receiving, processing and storing multiple discrete instances of digital content each containing at least one identifiable entity, and for subsequently retrieving one or more of those stored selected instances of said digital content, said system comprising - primary digital content capture means capable of capturing discrete instances of said digital content, - entity recognition means which subjects each discrete instance of digital content originating from said primary digital content capture means to an image processing function which establishes a primary identifier for each entity recognised in said instance, said primary identifier being at least partially representative of one or more digitally or physically observable entity features by which one entity is distinguishable from another, -storage means to which each discrete instance of digital content is substantially or entirely written, and to which is also written each and every identifier established therefor, characterised in that said system further includes - secondary imaging means capable of producing secondary discrete instances of digital content which are subjected to the same or an identical image processing function so as to establish one or more secondary identifiers for the entities recognised therein, and - display means, and further characterised in that - said system further comprises lookup means which provides a link between the stored is discrete instances of digital content and the set of primary identifiers established for each thereof, said secondary identifiers being submitted to said lookup means and compared with said primary identifiers therein, whereupon said lookup means returns a set of all those discrete instances of digital content whose primary identifiers substantially match said secondary identifiers, said system subsequently causing display of one or more of the discrete instances of digital content within said set on said display means.
10. 10. A system according to claim 9 wherein the digital content is digital photographic imagery, and the discrete instances thereof are individual digital photographic images.
11. 11. A system according to any of claims 9-10 wherein the entity capable of being recognised within any discrete digital photographic image is a human face, and said entity recognition means subjects discrete instances of digital content to facial recognition processing.
12. 12. A system according to claim 11 wherein the image processing function includes a digital facial distillation process whereby the various physical features, attributes, characteristics and general physiognomy of any human face appearing in any discrete instance of digital content is distilled down to a set of indicators which together make up an identifier by means of which any human face may be digitally distinguished.
13. 13. A system according to any of claims 9-12 wherein the primary digital content capture means is of relatively much higher calibre than the secondary imaging means.
14. 14. A system according to any of claims 9-13 which is capable of adapting the processing performed on the discrete instances of digital content depending on one or more of: -the calibre thereof, - the quality thereof, - the presence or absence of an indicator therein or forming part thereof, - the manner in or the channel over which they are received, - the source from which they emanated.
15. 15. A system according to any of claims 9-14 wherein the secondary imaging means is disposed proximate the display means.
16. 16. A system according to claim 15 wherein the secondary imaging means is affxed to the display means.
17. 17. A system according any of claims 9-16 wherein the secondary discrete instances of digital content are only transiently stored by the system.
18. 18. A system according to claim 17 wherein the secondary discrete instances of digital content are stored by the system only for as long as any entity recognised therein is also recognised in a temporally later-captured secondary discrete instance of digital content.
19. 19. A system according any of claims 9-18 wherein the secondary imaging means produces discrete instances of digital content one of: intermittently, periodically.
20. 20. A system according to any of claims 9-19 wherein said lookup means compares the secondary identifiers submitted thereto with the primary identifiers for one or more of: - identity, and -a similarity so substantial that it is more probable than not that the primary and secondary identifier so compared identify the same entity.
21. 21. A system according to any of claims 9-20 further comprising a commerce function which facilitates the purchase by an individual of any of the images currently being displayed to that individual and in which, by virtue of the proximity and orientation of that individual as regards the display screen and the camera attached or proximate thereto and the system as a whole, will appear in.
22. 22. A system according to any of claims 9-21 wherein the storage means and the is lookup means are consolidated in a single database.Amendment to the claims have been filed as follows Claims 1. A method of processing multiple discrete instances of digital content each including at least one digitally identifiable entity therein, said method including the steps of: - receiving a discrete instance of digital content of a first calibre, - processing said instance to identify one or more entities within said instance and create a set of primary identifiers, one for each entity so identified, said identifiers being at least partially representative of one or more digitally or physically observable features of that entity by which one entity may be distinguished from another, - storing both the digital content discrete instance and the set of primary identifiers established therefor in a correlated manner, - repeating the above for second and further digital content discrete instances (r) being of similar or identical calibre, the method being characterised by the further steps of:CO- receiving a discrete instance of digital content of a second calibre which is substantially different from said first calibre, O -recognising the particular second calibre of the received discrete instance of digital content and adapting the processing thereof accordingly; - processing said second calibre instance in identical manner to said first calibre digital content instance so as to provide a set of secondary identifiers of the same form and type as the primary identifiers for one or more entities, - comparing the set of secondary identifiers to the all the stored primary identifiers such that a subset of said stored primary identifiers is returned which are identical or so similar to said set of secondary identifiers that it is almost certain that the identifiers relate to the same entity or entities, - correlating the returned subset of primary identifiers to the discrete instances of digital content of the first calibre to which they relate, and -retrieving one or more of the said discrete instances of digital content of the first calibre.2. The method of claim 1 including the further step of displaying the one or more discrete instances of digital content of the first calibre so retrieved.3. A method according to any preceding claim wherein the method includes a requirement for specific user activation of the receiving of a discrete instance of digital content of a second calibre.4. A method according to any preceding claim wherein multiple discrete instances of digital content of a second calibre are received, each of which are subjected to an interim processing step wherein one or more entities are identified which repeatedly appear in said multiple discrete instance, and said one or more identified repeating entities are then subjected processing to establish an identifier for each said repeating entity.5. A method according to any preceding claim wherein a similarity tolerance, such being the extent to which the set of secondary identifiers must be similar to the stored cr) is primary identifiers with which they are compared so as to be considered as matching, CD can be adjusted.Operform all the steps of any of the methods of claims 1-5 when run on a computer.6. A computer program comprising computer program code means adapted to 7. A computer program according to claim 6 when embodied on or in a computer readable medium.8. A system for receiving, processing and storing multiple discrete instances of digital content each containing at least one identifiable entity, and for subsequently retrieving one or more of those stored selected instances of said digital content, said system comprising -primary digital content capture means capable of capturing discrete instances of said digital content, -entity recognition means which subjects each discrete instance of digital content originating from said primary digital content capture means to an image processing function which establishes a primary identifier for each entity recognised in said instance, said primary identifier being at least partially representative of one or more digitally or physically observable entity features by which one entity is distinguishable from another, - storage means to which each discrete instance of digital content is substantially or entirely written, and to which is also written each and every identifier established therefor, characterised in that said system further includes - secondary imaging means capable of producing secondary discrete instances of digital content which are subjected to the same or an identical image processing function so as to establish one or more secondary identifiers for the entities recognised therein, and -display means, and further characterised in that - said system is capable of adapting the processing performed on the discrete instances of digital content depending on the calibre thereof; and further comprises lookup means which provides a link between the stored discrete instances of digital content and the cr) is set of primary identifiers established for each thereof, said secondary identifiers being submitted to said lookup means and compared with said primary identifiers therein,COwhereupon said lookup means returns a set of all those discrete instances of digital content whose primary identifiers substantially match said secondary identifiers, O said system subsequently causing display of one or more of the discrete instances of digital content within said set on said display means.9. A system according to claim 8 wherein the digital content is digital photographic imagery, and the discrete instances thereof are individual digital photographic images.10. A system according to any of claims 8-9 wherein the entity capable of being recognised within any discrete digital photographic image is a human face, and said entity recognition means subjects discrete instances of digital content to facial recognition processing.11. A system according to claim 10 wherein the image processing function includes a digital facial distillation process whereby the various physical features, attributes, characteristics and general physiognomy of any human face appearing in any discrete instance of digital content is distilled down to a set of indicators which together make up an identifier by means of which any human face may be digitally distinguished.12. A system according to any of claims 8-11 wherein the primary digital content capture means is of relatively much higher calibre than the secondary imaging means.13. A system according to any of claims 8-12 which is capable of adapting the processing performed on the discrete instances of digital content depending on one or more of: - the quality thereof, - the presence or absence of an indicator therein or forming part thereof, -the manner in or the channel over which they are received, - the source from which they emanated. (r) "rCD a)14. A system according to any of claims 8-13 wherein the secondary imaging means is disposed proximate the display means.15. A system according to claim 14 wherein the secondary imaging means is affixed to the display means.16. A system according any of claims 8-15 wherein the secondary discrete instances of digital content are only transiently stored by the system.17. A system according to claim 16 wherein the secondary discrete instances of digital content are stored by the system only for as long as any entity recognised therein is also recognised in a temporally later-captured secondary discrete instance of digital 25 content.18. A system according any of claims 8-17 wherein the secondary imaging means produces discrete instances of digital content one of: intermittently, periodically.19. A system according to any of claims 8-18 wherein said lookup means compares the secondary identifiers submitted thereto with the primary identifiers for one or more of: - identity, and -a similarity so substantial that it is more probable than not that the primary and secondary identifier so compared identify the same entity.20. A system according to any of claims 8-19 further comprising a commerce function which facilitates the purchase by an individual of any of the images currently being displayed to that individual and in which, by virtue of the proximity and orientation of that individual as regards the display screen and the camera attached or proximate thereto and the system as a whole, will appear in.21. A system according to any of claims 8-20 wherein the storage means and the lookup means are consolidated in a single database.CDCD a)