GB2590619A - Image management system and method - Google Patents

Abstract

An image management system and a method of operating an image management system 1 is disclosed. Image capture requests for billboard media are received that are aggregated, then split into instruction sets when a threshold condition is fulfilled, and instruction sets are then transmitted to photographer devices. Consequently, unenriched image sets are received from the photographer devices which are then converted to enriched image sets. Conversion comprises detecting the boundary of billboard media within each photographic image, and then applying image recognition routines within that boundary to generate relevant metadata. Furthermore, a clipping path is generated from the detected boundary for ease of editing.

Description

Image management system and method

Field of the invention

The invention generally relates to an image management system, and methods relating to the operation of such an image management system. More particularly, the invention relates to an image management system for receiving, processing, storing and retrieving photographic images within which billboard media are displayed.

Background to the invention

The photographic documenting of advertising on billboard media is still a largely manual and time-intensive activity. Advertisers, wishing to document their advertising campaign, request deployment of photographers to various sites at which advertising structures, such as a billboard are located. These sites are often iconic or representative of a particular context important to the advertising campaign -such as famous sites within a city. Accordingly, each billboard and advert is photographed together with its surroundings. The photographs then need to be adjusted -for example, colour corrections and other enhancements -before the best candidate photographs are chosen and sent to the advertiser.

Whilst it is possible for an advertiser to individually commission a photographer for documenting an advertising campaign, this is inefficient. Logistical benefits can be derived from aggregating request from many advertisers, and serving these requests by deploying multiple photographers, at different times and locations, to capture a series of different advertising campaigns. Further efficiencies can be realised by transmitting all photograph images to a central image management facility at which image adjustment can be carried out.

However, this approach leads to a number of technical challenges related to: * the processing and sorting of requests from advertisers; * choosing and instructing a set of photographers able to fulfil the request; * coordinating the activity relating to the photographers when deployed; and * receiving and processing a large number of images from photographers at the central image management facility.

Leading on from this, bottlenecks can result from sorting through and choosing from the best of the large number of images received at the central image management facility for further processing. Only a subset of the images can (or need to) undergo adjustment and enhancement, as this is a step that cannot be fully automated. This is due to the need for subjective decisions to be made by human designer regarding which enhancements "look best". Often, enhancements include targeted colour-correction of the billboard media itself -for example, to match a digital copy of that advertisement. Other enhancements involve compositing multiple images taken at different exposure values -for example, for the purpose of generate images having a high dynamic range. Another enhancement is to ensure pedestrian features are blurred to ensure that privacy laws are complied with -for example, by utilising long-exposure images containing motion blur.

After enhancements have been applied to the subset of images, they are transmitted to the advertisers. However, all of the images -whether enhanced or not -are stored at the central image facility within its database. Advertisers may request to see photographic images of advertising -for example, of particular past campaigns and/or at specified locations. Accordingly, a further challenge is being able to locate appropriate images from within the ever-growing image database many months or years in the future.

It is against this background that the present invention has been conceived. Summary of the invention According to a first aspect of the present invention there is provided an image management system as set out in Claim 1.

Naturally, the invention extends in a second aspect to a method. In particular, the second aspect may provide a computer-implemented method of operating an image management system suitable for receiving, processing, storing and/or retrieving photographic images within which rectangular billboard media are displayed.

It will be understood that features and advantages of different aspects of the present invention may be combined or substituted with one another where context allows. For example, the functions or processes carried out by components associated with the first aspect of the invention may be part of the method according to the second aspect.

Furthermore, features of the first or second aspect may themselves constitute further aspects of the present invention.

Additionally, further aspects of the present invention may reside in a method according to the second aspect, carried out by at least one computing device. Still further aspects of the present invention may reside in a computer program product comprising instructions which, when the program is executed by a processor of a computing system, causes the computing system to carry out the method of the second aspect of the present invention. Still further aspects of the present invention may reside in a computer-readable storage medium comprising instructions which, when executed by a computing system, cause the computing system to carry out the method of the second aspect of the present invention.

Brief description of the drawings

In order for the invention to be more readily understood, embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings in which: Figure 1 is a schematic view of the image management system according to various embodiments of the present invention, the image management system being shown in conjunction with other devices and system that are external to it, but with which the image management system interacts; Figure 2 is a flow diagram of a process utilised by the system of Figure 1 for the management of images containing billboard media; Figure 3 is an example photographic image that may be handled by the system of Figure 1; Figure 4 is an the example image of Figure 3, after the application of an edge-detection algorithm; Figure 5 is a partial view of Figure 4, enlarged in a region containing a billboard, and schematically showing the outcome of a process for generating quadrilateral-shaped candidates for the boundary of the billboard media; Figure 6 is schematic representation of the application of a transformation function f(T) on a quadrilateral-shaped image of a billboard, such as those shown in Figures 3 to 5, to convert it to a rectangular image in a manner that counteracts perspective distortion; Figure 7 is a schematic representation of the outcome of an image analysis routine on the rectangular image to detect text and predetermined images, such as logos, thereby to generate metadata; and Figure 8 is a metadata table showing example metadata tags and values generated from image analysis of the image of Figure 3, isolated to the region corresponding to the billboard media.

Specific description of the preferred embodiments

Figure 1 is a schematic view of an image management system 1 according to a first embodiment of the present invention. The image management system 1 is suitable for receiving, processing, storing and retrieving photographic images within which billboard media are displayed. The system 1 also facilitates the editing such photographic images.

The image management system 1 comprises a system processor 10, a database 11, an advertiser interface 12, an authority communicator 13, and a photographer communication interface 14. As also shown in Figure 1, the image management system 1 is configured and arranged to interact with a number of other external devices, typically via a communications network such as the Internet. Notably, these include advertiser devices 20, 21 of advertiser systems 2, authority devices 30 of private space authorities 3, and photographer devices 41, 43, 45 of photographer systems 4, with each photographer device 41, 43, 45 communicatively linking with a corresponding camera 40, 42, 44. Each camera is controllable by a photographer to capture images of billboards 5 within their surrounding environment, and moreover, the advertising media supported on them -displayed either in print or digital-display form.

Figure 1 provides a schematic and functional view of the image management system 1. This may be implemented in many different ways known in the art. The functionality of the image management system 1 may be achieved through various combinations of hardware and components -for example, the system 1 may be implemented in the form of interconnected cloud services. By extension, the data storage and processing capabilities of the image management system 1, represented by the database 11 and system processor 10 respectively, may not necessarily relate to a singular database or processing unit on a single machine. Rather, this could encompass a set of databases and processors, each dedicated to a particular set of operations, and synchronised with one another Of and where necessary) to ensure processing and data integrity and validity.

Nonetheless, when the image management system 1 is referred to herein as performing an operation, this is typically executed or otherwise coordinated or controlled by the system processor 10. Furthermore, the image management system 1 may also include at least one computer-readable storage medium comprising instructions that are carried out by the system processor 10 to perform the functions described herein. These instructions may be stored on the database.

Broadly, the structure and functionality of the image management system 1 is designed to improve the efficiency with which the photographic documenting of advertising on billboard media can be achieved, by overcoming the technical challenges identified in the background section above.

Figure 2 shows an overview of the image management process that the system 1 is configured to perform that contributes to the overcoming of these challenges. This involves a number of steps that, for clarity, are set out sequentially. However, it will be understood that under certain circumstances it is possible for steps to be carried out in a different order as in the shown sequence, or in parallel.

In step 201 of the process, the system 1 is configured to receive, via the advertiser interface 12, and store in the database 11 a plurality of image capture requests from advertiser devices 20, 21, 22. These image capture requests, or "advertiser briefs", may come in different formats from different advertiser systems 2. One popular format is a spreadsheet, for example.

Fundamentally, each request comprises an advertiser reference that identifies the advertiser. This may be based on an identifier or name within the spreadsheet, or may be inferred from the domain or address from which the request has been sent. This is utilised to later determine where to issue a response to the request that contains the appropriate edited photographic images pursuant to the request.

Additionally, each request comprises advertising site information that identifies the location of each advertising site within the request. Typically, this is where advertising structures, such as billboards, are located. The system 1 maintains an advertising site list against which the advertising site information can be cross-referenced to allow additional functions to be performed in response to the chosen locations.

Naturally, advertising structures are used by different advertisers over different advertising periods. Thus, the request from each advertise device also includes at least one time constraint that ideally informs the system 1 of the period of availability, at each advertising site location, of advertising media to be photographed. The time constraint may also specify a preferred deadline for the delivery of images that are contained within the request. Furthermore, other time constraints may be included, such as the time of day that the advertiser would prefer an image to be captured. For example, an advertiser may prefer an image to be captured during a busy morning "rush hour" period to convey the sense that the advertisement is receiving a large amount of attention -this can also be specified as a time constraint.

Furthermore, each request includes at least one campaign reference. The campaign reference identifies a set of image artefacts that are expected to be displayed by advertising media at the advertising site location as part of the advertising campaign to be captured via photograph. The set of image artefacts may include advertiser word or picture marks (e.g. logos) that subsist within the advertising media to be photographed. The set of image artefacts may be text, such as slogans, similarly included within the media. The set of image artefacts may also include the entire advert image expected to be displayed. The set of image artefacts may be sent as part of the request, such as a set of files, or in the form of references to the images (e.g. URLs) via which the set of image artefacts can be obtained.

Nonetheless, via the campaign reference, the image management system 1 is provided with access to the set of image artefacts that correspond to the specific image capture requests received from corresponding devices 20 of advertiser systems 3.

Further parameters may be formulated in the requests, such as the format (e.g. landscape or portrait) of the resulting photographic images to be captured.

Image capture requests received from multiple advertiser devices 20, 21, 22 are aggregated with one another. Advantageously, this allows efficiencies of scale and other operational benefits. For example, two different requests may include a request to capture advertising campaigns at site locations that are sufficiently geographically co-located and available over the same time period such that they can be easily visited in quick succession -e.g. over a single trip by a single photographer.

However, as image capture requests from multiple advertiser devices are unlikely to be received simultaneously, it is necessary to delay photographer instruction until a sufficient number of image capture requests have been received to realise these advantages. At the same time, it is important not to unduly delay photographer instruction to ensure sufficient time for the photographic images to be captured, edited, and sent back to the advertiser device 20, 21, 22 originating the image capture request within any deadline specified.

Accordingly, in step 203 of the process, the system 1 is configured to maintain or otherwise monitor at least one instruction threshold condition for use in determining when and how to issue instructions to photographers to ensure the image capture requests can be effectively fulfilled.

More specifically, an instruction threshold condition can be determined by the system 1 from information within the aggregated image capture requests. An example would be an approaching set of image delivery deadlines, determined from one or more of the time constraints of image capture requests.

Another example of an instruction threshold condition is a "critical mass" condition that can be derived from determining that there exists a sufficient number of requests that can be serviced by a single photographer efficiently (e.g. sufficiently co-located to minimise travel-time wastage) within a predetermined time-frame (e.g. over a day, or half-day of work).

As mentioned, the system 1 maintains an advertising site list against which the advertising site information can be cross-referenced to allow additional functions to be performed in response to the chosen locations. A relevant example is whether the location is a private or quasi-private space location that requires permission before commercial photography can take place at that site. Sites that are seemingly public, but are actually privately-controlled, such as transport stations and retail concourses, typically require such permission. It is not permissible to deploy a photographer to a location like this without securing permission, and this is another task that would otherwise be manually handled in prior art systems.

Accordingly, an additional step that the system 1 is configured to perform is to determine if any location received in an image capture request is a private space location requiring permission (and such permission has not already been obtained). In response to this determination, the system 1 issues a permission request, via the authority communicator 13 to an authority device 30 of a private space authority 3. The permission request typically includes a requested period of occupation of the private space location for photography. The system 1 is further configured to receive a permission response, via the authority communicator 13 from the authority device 30 of a private space authority 3, the permission response approving or denying the request. In response to receipt and the content of the permission response, the system 1 is ultimately configured to control if, when and how photographers are instructed. In particular, in response, the system 1 may control the instruction threshold condition, how the aggregated image capture requests are split, and/or the timing and nature of the transmission of instruction sets to photographers.

In step 203 of the process, in response to the fulfilment of the instruction threshold condition, the system 1 is configured to split the aggregated image capture requests into instruction sets (or "photographer briefs") to be sent to appropriate photographers. The database 11 also contains a photographer information list that assists with the automated splitting and choosing of appropriate photographers based on listed information -such as a photographer's availability and location.

Each instruction set that is generated by the image management system 1 contains a photography capture schedule that defines a sequence of photography waypoints, each corresponding to advertising site locations, to be visited by a photographer within an assignment period (e.g. one working day /8 hours). The system 1 is configured to calculate the assignment period to ensure that the time constraints associated with each photography waypoint can be met, and furthermore falls within a predetermined range (e.g. half or full working day -i.e. between four and eight hours). This ensures that the time constraints relating to the period of availability, at each location, of advertising media to be photographed can be met, and also that photographers are not deployed for a period that is either inefficient, or greater than a normal working period.

Moreover, the system 1 is configured to calculate a split of the aggregated image capture requests into instruction sets according to predetermined priorities. One priority is grouping together within a common instruction set, photography waypoints that can be visited in a sequence within the assignment period. Part of this split calculation performed by the system 1 involves assigning time units for photography at each photography waypoint (e.g. 30 minutes per waypoint), and also travel between each photography waypoint (dependent of distance and traffic conditions at the deployment time).

Accordingly, to maximise efficiencies the calculation typically also prioritises photography waypoints at locations that are geographically proximate to one another and orders the waypoints according to an optimised calculated route for the time of day of instruction.

In step 204 of the process shown in Figure 2, the system 1 is configured to transmit the instruction sets, generated from the splitting of the aggregated image capture requests, via the photographer communication interface 14, to corresponding photographer devices 41, 43, 45. Naturally, the system 1 also received an acknowledgement from those devices signalling receipt and/or acceptance by a photographer of the proposed brief.

The photographer devices 41, 43, 45 may take on different forms, but are primarily envisaged to be mobile telecommunication devices, such as smartphones, tablet devices, or laptops, that are communicatively connected via the internet to the photographer communication interface 14 of the system. The photographer devices 41,43, 45 are also communicatively connected to a respective digital camera 40, 42, 44 -typically via a local communication mechanism. This may be wireless (e.g. WiFi or Bluetooth) or wired, the latter naturally including data connection via interface between the photographer devices 41, 43, 45 and a memory card of a respective camera 40, 42, 44.

Thus photographic images taken by each camera 40, 42, 44 can be transmitted via the photographer devices 41, 43, 45 to the image management system 1.

Furthermore, an application running on the photographer device 41 provides a user interface and functionality such as enabling the instruction set received at the photographer device 41 to be displayed to a photographer, allowing the photographer to confirm receipt and acceptable of a brief, allowing photographers to manage and interrogate their instruction sets / briefs, and navigate between brands and site locations.

Moreover, as photographers capture photographs specified on an instruction set, they can interact with the application to mark the photograph as successfully captured, or MIA - "missing in action" -in the case that the expected advertising campaign is not present at right location at the right time. The application can also provide maps, routes and timing data that can be generated from the photography waypoints of the instruction set. This information can be sent back to the system 1 to allow real-time update of routing, instructions and other parameters thus leading to further efficiencies in the fulfilment of image capture requests.

Additionally communicated to the photographer devices 41 are permission documents, such as those requested by the system 1 from the private space authority 3, allowing such documents to be presented, if needed, at the applicable site locations to security personnel.

By following the instruction set provided by the system, photographers then capture a set of images using cameras 40, 42, 44, and these are transferred via the corresponding photographer device 41, 43, 45 to the system 1. An intermediary cloud-based storage resource may be used to facilitate synchronisation between the system database 11 and photographer devices 41, 43, 45.

In step 205 of the process, the system 1 is configured to receive via the photographer communication interface 14, from each photographer device 41, 43, 45, and store in the database 11, an unenriched image set containing photographic images of advertising media. As mentioned, each photographic image received is captured by a respective camera 40, 42, 44 of the photographer devices 41, 43, 45.

Furthermore, the system 1 is configured to receive at least one instruction-image reference from each photographer device 41, 43, 45. The instruction-image references associate the instruction set previously communicated to the photographer device 41, 43, 45 with the unenriched image set. This allows the system 1 to determine an association between the received images and image capture requests. In certain embodiments, an instruction-image reference is derived from the advertiser reference.

This also allows initial cross-checks to be performed between parameters defined in the image capture requests and the corresponding captured images. For example, cameras 40, 42, 44 typically embed metadata into images that can be used to check that the images were captured during the right time specified in an image capture request, such as a morning commuter time slot (e.g. >07:00AM <10:00AM).

It should be noted that such pre-embedded camera metadata is not particularly relevant to the purpose of maintaining an easily-searchable database 11. This is because search parameters are unrelated to metadata normally automatically generated by cameras 41, 43, 45. For example, no metadata is normally generated by a camera that allows advertisers to see photographic images of past advertising campaigns on the basis of the content of the media used in those advertising campaigns.

Additionally, as mentioned in the background section, a further step required prior to transmission to the instructing advertiser is the adjustment and enhancement of the images. This necessarily involves applying a clipping path that isolates the region of the photographed image that contains the billboard media.

An elegant solution to both these problems provided by the system 1 involves converting the unenriched images sets into enriched image sets via the automated identification of the location of the boundary of the media (and so the clipping path), and furthermore utilising this boundary for relevant metadata generation.

In particular, it is desirable to generate metadata to be appended to each image file, the metadata including keywords or parameters via which the database 11 can be later queried. Useful search terms contained within the metadata may include: * The media format of the billboard photographed (e.g. dimensions, orientation and/or proportions); * Data identifying the media owner -i.e. the owner of the billboard structure itself; * Data identifying the advertiser; and * Data identifying characteristics of the advertising campaign, such as logos, slogans and text.

One way metadata like this can be automatically generated, is via applying image recognition techniques (including text recognition) to the photographed image. However, this leads to problems associated with the particular use-case in which the invention resides: The most important, relevant metadata for the present use-case ought to be generated in respect of the content displayed by the actual billboard advertising media.

However, as shown in Figure 3, a billboard 5 and the media 50 supported by it is photographed within its surrounding, and so many other extraneous artefacts may be picked up by general image recognition techniques, and consequently lead to superfluous metadata being generated.

This is unhelpful in a subsequent search query as it increases the chances of irrelevant "false positive" results being displayed in response to the search query.

For example, if a billboard 5 is photographed in a street setting, then objects around the billboard 5 such as street signs, vehicles (which themselves may bear advertising), pedestrians etc from which metadata may be needlessly generated by an image recognition routine. Additionally, it is wasteful of computational resources to perform image recognition.

Referring to Figures 5 and 6, the solution to this problem, provided by the present embodiment of the invention, is to automatically identify the location of billboard media 50 within a photograph, automatically determine a clipping path around the boundary 52 of the media 50 supported by the billboard 5, and then apply image recognition routines within that boundary 52 or clipping path.

This excludes having to process parts of the image outside the boundary 52, increasing the relevance of the metadata generated, and also minimising the processing burden and time for carrying out the image processing technique.

Synergistically, this can also assist with image processing itself. Billboards 5 are typically planar advertising structures which, face-on, have a rectangular shape, and so when photographed, generally occupy an area of the image bounded by a four-sided shape.

Naturally, the exact form of four-sided shape depends primarily on the relative angle between the camera and the billboard, but within the photograph, the boundary 52 of the media 50 supported by the billboard 5 -and thus the overall shape of the clipping path -is generally a non-rectangular quadrilateral in shape. Once the boundary 52 or clipping path shape is determined, the image within it can be subject to more efficient image processing.

Referring to Figure 6, a geometric transformation function f(T) can be determined from the difference in shape between the boundary 52 in the shape of a non-rectangular quadrilateral, and that of a rectangle 52T. The exact proportion of the latter can be determined from a predetermined look-up list of standard media sizes or proportions, and/or from the site location list which can provide information about the format of the advertising structure.

The determined transformation function f(T) can then be applied to the delimited image of the media 50 to derive a rectangular normalised version 50T of that media image. Using the normalised version 50T reduces the adverse effects of perspective distortion on image and text recognition. For example, it is more straightforward, and less computationally-intensive to perform optical character recognition (OCR) on normalised rather than distorted text. Alternatively, the image recognition routine can be provided with a parameter derivable from the transformation function f(T) that signifies what the perspective distortion is to allow image/text recognition to function more effectively.

Alternatively, the determination of the clipping path can be informed by carrying out image recognition first. In this case, an image recognition function is provided with a predetermined artefact or set of artefacts (e.g. an advertiser-provided image, such as a logo) to locate within a photograph the artefacts being those expected to appear within the billboard media. The image recognition function then determines the location of that artefact within the photograph, thereby defining a region of the photograph that is likely to be inside the clipping path. Furthermore, a perspective distortion value calculated by the difference between the "flat" artefact and that found in the photograph can be determined, the perspective distortion value being used to control the distortion of an otherwise rectangular clipping path thereby to increase the likelihood of the clipping path following the outline of the billboard media.

In either example, the generation of the clipping path is advantageously used in a further operation that speeds up manual processing of images. In particular, the clipping path around the billboard media is saved as a mask. This mask isolates enhancement applied by a human designer to the billboard media, reducing the time needed for manual processing, thereby improving the efficiency of the image processing system as a whole.

Accordingly, and referring back to Figure 2, the system 1 is configured in step 206 of the process, to convert unenriched image sets into enriched image sets by: * detecting, within a photographic image of the unenriched image set, a boundary of a billboard media; and then: * generating from that image: o a clipping path that follows the detected boundary; and o metadata -formulated as a string -that identifies features within the detected boundary.

The system 1 is further configured in step 207 of the process, to store the metadata and clipping path in the database 11 alongside each image. The metadata may be appended to an image, forming part of each image file Depending on the file format, a clipping path may be saved as either a separate image file (e.g. a mask image) or as a separate image manipulation layer within the image. Metadata is generally embedded within the image file itself.

The metadata is typically stored as a tag-value pairs, allowing subsequent parameterisation of search queries, and categorisation of images into different categories. For example, a metadata tag "advert text string" is used to store string values directly corresponding to text recognised within the billboard media image. Another metadata tag "advert logo name" is used to store a string values that are textual representations of advertiser logos. The latter can be determined from logo image-name lookup lists prestored on the database 11 that provide text string values for predetermined logo images. Image-name lookup lists may be provided with or generated from the campaign reference originally received from an advertiser device 20, 21, 22.

Accordingly, in steps 208 and 209, the system 1 is configured to subsequently receive an image search query, formulated as a string, and then query the database 11 to retrieve a list of images from the enriched image set that have metadata matching the search string. Naturally, the system 1 is further configured to display such images of the enriched set allowing a human operator to be provided with a visual representation of the search results. The system 1 also permits operations such as opening those files for editing.

Nonetheless, an operator can either locate an image for editing via a search (steps 208 and 209), or simply via browsing the directory structure of the database 11 (i.e. directly from step 207).

Nonetheless, the system 1 is further configured in step 210 of the process, to open for editing an image of the enriched set, that is provided with an automatically-generated clipping path. As mentioned, the automatic provision of the clipping path significantly speeds up manual editing of the image. The system 1 is also configured to allow edited images to be stored in the database 11 (step 211) and sent (step 212) via the advertiser interface 12 to an advertiser device 20 as a response to the image capture request issued by the advertiser device 20. Notably, images are provided with metadata allowing matching of the image with an advertiser and/or request. For example, the advertiser reference and/or instruction-image reference may be stored with a respective image.

Prior to sending, other operations may be performed such as secondary retouching and quality-control checks. For example, if a collection of images are to be sent in response to an advertiser image capture request, the images of that collection may be compared one another and in response colour-corrected to ensure a consistent appearance across the collection. Additionally, extraneous metadata may be stripped out of the images that are sent. Images may be sent via a cloud-computing image-delivery service.

Billboard media boundary detection process A key part of converting unenriched image sets into enriched image sets involves the detecting, within each photographic image of the unenriched image set, a boundary of a billboard media.

Whilst it is within the capabilities of a person skilled in the art to implement such a process, a non-limiting example of the billboard media boundary detection process will now be provided with reference to Figure 3 to 8.

Figure 3 is an example of a colour photographic image that may be handled by the system of Figure 1 when applying a billboard media boundary detection process. A first step of the billboard media boundary detection process involves applying an edge-detection algorithm to the photograph image. This converts the colour image into a greyscale or monochrome image. This reduces the computational burden of subsequent processing of the image for the purpose of billboard media boundary detection and, in particular, highlighting edges detected within the image.

Figure 4 is an the example image of Figure 3, after the application of an edge-detection algorithm. Various edge-detection algorithms are suitable for this purpose, and as are known by those skilled in the art, such as Roberts Cross, Canny, Difference of Gaussian, Laplacian and Sobel, etc. Following the application of an edge-detection algorithm, a straight-line detection algorithm, such as a classical Hough transform can be applied. This can be used to generate coordinates of straight lines within the image, with each straight line being efficiently defined as a pair of (x, y) coordinates. Every coordinate defines an endpoint of the line, and coordinate pairs can be used to define the location, orientation and length (size) of the line. A line size value, derived from the coordinate pairs, can be determined, for example, by applying the calculation: sqrt [ (x2 -x1)2 + (y2 -y1)2] -where (x1, y1) is the first coordinate and (x2, y2) is the second coordinate of the straight line.

An initial line size threshold is used to filter out straight lines below a predetermined length. This removes from subsequent processing any lines that are unlikely to be big enough to be edges of a billboard media within the image. The initial line size threshold may be determined relative to the overall image size -(e.g. 10% of the total height + total width of the photographic image).

The process further comprises determining a group of the plurality of lines that are positioned and arranged relative to one another to define a quadrilateral shape. From this candidates for the boundary of the billboard media are generated. This is achieved by applying geometric principles inherent to convex quadrilaterals, such as determining a set of four lines where: * every line in the set is of a minimum length greater than a threshold value; * each line coordinate is at a location within the image that is common -or sufficiently close -to the coordinates of exactly one other line of the set (corner detection); and * the four angles formed between two adjoining lines of the set sum to 360 degrees.

Further "likely" geometric features are also used as ways to increase the confidence in candidates -such as every angle formed between two adjoining lines of the set are within a predetermined range (ideally 90 degrees +/-50 degrees) -this ensures that significantly distorted quadrilateral shapes are discarded as they are unlikely to be representative of photographed billboard media.

In the event that too few, or no candidates for the boundary of the billboard media are generated, the process can be partly restarted with a line size threshold that is smaller than the initial line size threshold. In the event that too many candidates are generated, then the line size threshold can be increased.

Figure 5 is a partial view of Figure 4, enlarged in a region containing a billboard, and schematically showing the outcome of a process for generating quadrilateral-shaped candidates for the boundary of the billboard media. In this case, three candidates c1, c2, c3 have been generated.

Thereafter, the billboard media boundary detection process comprises choosing one of the generated candidates.

To do this, the process can determine the set of image artefacts expected to feature in the billboard media within the photographic image. This can be achieved by determining a link between the set of image artefact, and the instruction-image reference provided with the image when sent to the system 1 from a photographer device 41, 43, 45. In particular, the link can be determined from the relationship between the instruction-image reference, the advertiser reference and/or campaign reference, and so to the corresponding set of image artefacts.

Thus, an image search can be carried out by the system 1 using said set of image artefacts (optionally modified by the transformation function) within each quadrilateral representing a candidate for the boundary of the billboard media.

If only a single quadrilateral of one of the generated candidates solely bounds any one of these corresponding set of image artefacts, then this quadrilateral is chosen from the candidates.

Alternatively, where multiple quadrilaterals contain any one of the set of identifying image artefacts -or in the case that an image search is not carried out, or does not return any results -the largest quadrilateral is chosen from the candidates unless it is determined that it frames a smaller quadrilateral, nested within it, in which case the smaller framed quadrilateral is chosen. This serves to distinguish between the outside of the frame of a billboard (i.e. c3 in Figure 5) and the boundary of the billboard media, on the inside of the frame (i.e. c2 in Figure 5).

The determination of the framing of a smaller, nested quadrilateral can be achieved via the application of geometric principles such as determining a substantially equal spacing between pairs of adjacent and parallel lines as will be apparent to a person skilled in the art.

In alternatives, the image search need not be carried out, or at least delayed until one of the candidates are chosen. The delayed image search in this case serves to generate metadata that identifies features (such as objects and text) within the detected boundary of the billboard media.

Figure 7 is a schematic representation of the outcome of an image analysis routine on the rectangular image to detect text and predetermined images, such as logos, thereby to generate such metadata. Figure 8 is a metadata table showing example metadata tags and values generated from image analysis of the image of Figure 3, mostly (but not exclusively) isolated to the region corresponding to the billboard media: Additional operations may be carried out using similar principles. For example, the billboard owner may be added as a metadata by carrying out an image or text search within or adjacent to the candidate quadrilaterals. In Figure 5, the "outdoor plus" text and logo is detected, and added as a corresponding metadata tag-value pair in the table of Figure 8. Media owners are often identified by corresponding text/logos printed on the frame of the billboard media (i.e. between two detected quadrilaterals). The system 1 is also arranged to utilise this general property of billboard structures for the purposes of populating the value of the "media owner name" metadata tag.

Accordingly, the system 1 can generate and store in the database 11, a clipping path that follows the detected boundary of an image, and metadata, formulated as a string, that identifies features within the detected boundary.

Thus, an image management system 1 has been described that is capable of efficiently receiving, processing, storing and retrieving photographic images within which billboard media are displayed.

This allows the efficient and automated photographic documenting of advertising on billboard media. Efficiency gains can be made from aggregating image capture requests, and then turning them into instruction set based on an instruction threshold condition. Furthermore, the structure and function of the system 1 allows intelligent processing and sorting of requests from advertisers so that the most appropriate photographers can be chosen, efficiently instructed, and then coordinated to deliver image sets.

Additionally, the system provides automated receipt, processing and editing of such image sets to enrich them to meet the image capture requests, and also to allow metadataenhanced storage of the image sets to allow them to be effectively located in the future using relevant keywords or text strings.

As a whole, the above-described features, configuration and operation of embodiments of the image management system 1 leads to a significant improvement over prior-known image management systems.

However, it should be noted that whilst invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the scope of the appended claims.

Claims (6)

1. CLAIMS1 An image management system suitable for receiving, processing, storing and retrieving photographic images within which rectangular billboard media are displayed, the system comprising: * a database 11 for storing images; * an advertiser interface 12 configured to communicate with a plurality of advertiser devices 20, 21, 22; * an authority communicator 13 configured to communicate with at least one authority device 30; * a photographer communication 14 interface configured to communicate with a plurality of photographer devices 41, 43, 45; and * a system processor 10 operatively coupling the database 11, advertiser interface 12, authority communicator 13, and photographer communication interface 14 to one another; the system 1 being configured to: * receive, via advertiser interface 12, and store in the database 11 a plurality of image capture requests from the advertiser devices 20, 21, 22, each received advertiser image capture request comprising: o an advertiser reference; o advertising site information that identifies the location of each advertising site; o at least one time constraint relating to the period of availability, at each advertising site location, of advertising media to be photographed; and o a campaign reference, identifying a set of image artefacts expected to be displayed by advertising media at the advertising site locations; * aggregate the image capture requests from multiple advertiser devices 20, 21, 22; * determine when an instruction threshold condition is fulfilled and in response: o split the aggregated image capture requests into instruction sets, each instruction set containing a photography capture schedule that defines a sequence of photography waypoints, each corresponding to advertising site locations, to be visited by a photographer within an assignment period; * the assignment period being calculated to: * to meet time constraints relating to the period of availability at each location of advertising media to be photographed; * fall within a predetermined range; * the split being calculated in a way that prioritises the grouping together, within a common instruction set, photography waypoints that can be visited in a sequence within the assignment period, such as photography waypoints at locations geographically proximate to one another; and o transmit instruction sets, via the photographer communication interface 14, to corresponding photographer devices 41, 43, 45; * receive, via the photographer communication interface, from each photographer device 41, 43, 45, and store in the database 11: o an unenriched image set containing photographic images of advertising media, each photographic image having been captured by a respective camera 40, 42, 44 of the photographer devices 41, 43, 45; and o at least one instruction-image reference that associates the instruction set previously communicated to the photographer device with the unenriched image set; * convert unenriched image sets into enriched image sets by: o detecting, within a photographic image of the unenriched image set, a boundary of a billboard media by: * determining, from the instruction-image reference, the set of image artefacts expected to appear within the photographic image; * applying an edge-detection algorithm; * generating candidates for the boundary of the billboard media by: * determining a plurality of substantially straight lines within the image that are greater in length than an initial line size threshold; * determining a group of the plurality of lines that are positioned and arranged relative to one another to define a quadrilateral shape; and * reducing the threshold size and repeating the candidate generation step if no quadrilateral shapes are defined by the plurality of lines; * determining the boundary of the billboard by choosing one of the generated candidates: * a quadrilateral of the generated candidates being chosen if it solely bounds, as determined by an image search within it, any one of the set of image artefacts expected to appear within the photographic image; and * * 2.* where multiple quadrilaterals contain any one of the set of identifying image artefacts, the largest quadrilateral is chosen unless it is determined that it frames a smaller quadrilateral, nested within it, in which case the smaller framed quadrilateral is chosen; and o generating from and storing, in the database 11, with that image: * a clipping path that follows the detected boundary; and * metadata, formulated as a string, that identifies features within the detected boundary; receive an image search query that includes a search string; and retrieve from the database, in response to the image search query, images of the enriched data sets that are stored with metadata matching the search string of the query.
2. The image management system of claim 1, wherein generating candidates for the boundary of the billboard media comprises determining a set of four lines where: * every line in the set is of a minimum length greater than a threshold value; * each line coordinate is at a location within the image that is common -or sufficiently close -to the coordinates of exactly one other line of the set, thereby to detect a corner; and * the four angles formed between two adjoining lines of the set sum to 360 degrees.
3. 3. The image management system of claim 1, further configured to determine if any location received in an image capture request is a private space location requiring permission, and in response: o issue a permission request, via the authority communicator 13 to an authority device 30 of a private space authority 3, the permission request including a requested period of occupation of the private space location; o receive a permission response, via the authority communicator 13 from the authority device 30 of a private space authority 3, the permission response approving or denying the request; and o in response to receipt and/or the content of the permission response, controlling at least one of: the instruction threshold condition, the split of the aggregated image capture requests, and the transmission of instruction sets.
4. 4. A method of operating an image management system, the method comprising: * receiving a plurality of image capture requests each comprising: o an advertiser reference; o advertising site information that identifies the location of each advertising site; o at least one time constraint relating to the period of availability, at each advertising site location, of advertising media to be photographed; and o a campaign reference, identifying a set of image artefacts expected to be displayed by advertising media at the advertising site locations; * aggregating the image capture requests; * determining when an instruction threshold condition is fulfilled and in response: o split the aggregated image capture requests into instruction sets, each instruction set containing a photography capture schedule that defines a sequence of photography waypoints, each corresponding to advertising site locations, to be visited by a photographer within an assignment period; * the assignment period being calculated to: * to meet time constraints relating to the period of availability at each location of advertising media to be photographed; * fall within a predetermined range; * the split being calculated in a way that prioritises the grouping together, within a common instruction set, photography waypoints that can be visited in a sequence within the assignment period, such as photography waypoints at locations geographically proximate to one another; and o transmit instruction sets to corresponding photographer devices 41, 43, 45; * receiving from each photographer device 41, 43, 45, and storing: o an unenriched image set containing photographic images of advertising media, each photographic image having been captured by a respective camera 40, 42, 44 of the photographer devices 41, 43, 45; and o at least one instruction-image reference that associates the instruction set previously communicated to the photographer device with the unenriched image set; * converting unenriched image sets into enriched image sets by: o detecting, within a photographic image of the unenriched image set, a boundary of a billboard media by: * determining, from the instruction-image reference, the set of image artefacts expected to appear within the photographic image; * applying an edge-detection algorithm; * generating candidates for the boundary of the billboard media by: * determining a plurality of substantially straight lines within the image that are greater in length than an initial line size threshold; * determining a group of the plurality of lines that are positioned and arranged relative to one another to define a quadrilateral shape; and * reducing the threshold size and repeating the candidate generation step if no quadrilateral shapes are defined by the plurality of lines; * determining the boundary of the billboard by choosing one of the generated candidates: * a quadrilateral of the generated candidates being chosen if it solely bounds, as determined by an image search within it, any one of the set of image artefacts expected to appear within the photographic image; and * where multiple quadrilaterals contain any one of the set of identifying image artefacts, the largest quadrilateral is chosen unless it is determined that it frames a smaller quadrilateral, nested within it, in which case the smaller framed quadrilateral is chosen; and o generating from and storing with that image: * a clipping path that follows the detected boundary; and * metadata, formulated as a string, that identifies features within the detected boundary; * receiving an image search query that includes a search string; and * retrieving from the database, in response to the image search query, images of the enriched data sets that are stored with metadata matching the search string of the query.
5. 5. A computer program product comprising instructions which, when the program is executed by a processor of a computing system, causes the computing system to carry out the method of claim 4.
6. 6. A computer-readable storage medium comprising instructions which, when executed by a computing system, cause the computing system to carry out the method of claim 4.