NZ516640A - Image presentation and scale merging system - Google Patents

Abstract

A computerised method of automated product selection, includes a)Capturing in one or more databases at least a first product image of a first product and at least a further image of a further item. b)A product selection system, which includes a server for receiving and serving images and one or more capture devices, in communication with the server. The server includes a processing unit and associated memory device for storing a computer program causing the processing unit to determine the dimensions of the first product and the further item from each captured image. b)Facilitating product selection based on the matching by the processing unit, of the dimension of both captured images along one or more reference axes in each image. The processing unit uses the dimensions and at least one reference axis of each captured image to combine the captured images in to a merged image, which is displayed to the customer using one or more terminals.

Description

516640 INTELLECTUAL PROPERTY OFFICE OF N.Z. 1 t> JAN 2002 RECEIVED PATENTS ACT 1953 COMPLETE SPECIFICATION (a) Automated Product Selection Method and System (b) I, Sally Hoban, an Australian citizen of 9 Myambert Avenue, Balwyn, Vic 3103, AUSTRALIA AND I, Dorothy Alcaide, an Australian citizen of 21 Breffna Street, Preston, Vic 3072, AUSTRALIA HEREBY declare the invention, for which we pray that a patent may be granted to us and the method by which it is to be performed, to be particularly described in and by the following statement: (c) The present invention relates generally to an automated system and method of product selection, and in particular to methods and systems that combine 40470574:RNM:ib 11 January 2002 digitised image of a product with at least one other image in order to facilitate the selection by a prospective purchaser of that product. The present invention is suitable for use in Internet-based shopping applications in which consumers are able to visualise the appearance of clothing when worn, prior to the purchase of that clothing. It will be appreciated, however, that the invention is not limited to this exemplary application.

The development in recent times of electronic interactive shopping systems, via the Internet, cable network or other communications network, has enabled consumers to select and optionally purchase a wide range of products without requiring that the consumer by physically present at a retail outlet. Prospective purchasers of clothing, homewares and other items are able to view virtual images of a range of products offered by a retailer, and arrange for purchase and delivery of selected products.

Some online shopping systems further enable consumers to view an electronic image of a product together with a further electronic image provided by the consumer. For example, at least one known online shopping system is adapted to generate an image of the consumer's body, and to superimpose an image of an article of clothing on the consumer's body image. In this way, the consumer is in this way better able to visualise the appearance of that item of clothing when worn.

However, such online shopping systems only enable customers to gain a general impression only of the size or fit of the product to be purchased. The return of merchandise due to improper sizing or fit is one of the biggest problems facing the online retail shopping industry. Inexact sizing or fit, and an inaccurate representation of the appearance of merchandise when used by the consumer, are significant problems which need to be overcome by the online retail shopping industry.

There therefore exists a need for an automated product selection method and system that alleviates or overcomes one or more disadvantages of known product selection systems.

There also exists a need for a product selection method and system that enables a customer to select merchandise using information that accurately relates 40470574:RNM:rb 11 January 2002 the merchandise itself to the environment in which the merchandise is intended to be used.

With this in mind, one aspect of the present invention provides a computerised method of product selection, including; capturing in one or more databases at least a first product image of a first product and at least a further image of a further item, determining in a processing unit, dimensions of the first product and the further item from each captured image, and facilitating product selection based upon the matching by the processing unit, 10 of the dimensions of both captured images along one or more reference axes in each image wherein the dimensions and at least one reference axis of each captured image are used by the processing unit to combine the captured images into a merged image.

In one embodiment, the dimensions of each captured image may be 15 determined by comparing an outline of each captured image to gridlines and determining points of intersection between each captured image and those gridlines.

In at least one embodiment, the method may further include the step of using the dimensions of each captured image to combine the captured images into a merged image, and displaying the merged image to a customer. The images may be 20 selectively merged according to correspondence of dimensions of the captured images.

In at least one embodiment, the method may further include the step of selectively enabling the purchase of at least the first product, based upon the matching of the dimensions of both captured images. The dimensions may be 25 compared along one or more common axes to both images.

Melbourne\004329434 %)) li U lij \Jj The first product image may represent, for example, an item of clothing. Alternatively, the first product image may represent a household or other item, such as a piece of furniture.

The further product image may be a customer image, and may represent at least a portion of a customer's body. Alternatively, the customer image may represent at least a portion of a customer's environment, such as a home or room. The images may be images of the customers own body or environment, or alternatively may be images of another person's body or environment and selected 10 by the customer to be representative of that body or environment.

The first product image may be selected from a range of product images of a same or similar product type, whereby product images of that same or similar Melbourne\004149123 - Printed 2 October 2002 (16:11) -3a- product type are of different dimensions, and selection is based upon matching of the dimensions of a product image with the customer image.

The further image may be a second product image of a second product intended for use with the first product. For example, the second product image may 5 be an image of fabric intended for use with a first piece of furniture.

Another aspect of the invention provides a product selection system including a server for receiving and serving images, the server including a processing unit and computer program code stored in an associated memory device for causing the server to carry out the above-described method. 10 The product selection system may also include a first database for storing the first product images.

The product selection system may further include a second database for storing the further images.

The product selection system may also include one or more image capture 15 devices, in communication with the server, for capturing the at least first product image and/or least one further image.

One or more terminals may also be provided for display of the first product image and further image.

The following description refers in more detail to the various features of the 20 product selection method and system of the present invention. To facilitate an understanding of the invention, reference is made in the description to the accompanying drawings where the invention is illustrated in a preferred embodiment. It is to be understood that the invention is not limited, however, to the preferred embodiment illustrated in the drawings.

In the drawings: Figure 1 is schematic diagram illustrating one embodiment of product selection system according to the present invention; Figure 2 is a flow chart illustrating the operation of the product selection system of Figure 1 in the capturing of product and other images; A . 15 JUL 2003 Melbourne\004329434 -4 - I " £i£EiyED Figures 3 is a flow chart illustrating the operation of the product selection system of Figure 1 by a consumer to select products; and Figures 4 and 5 are representations of product and further images captured by the product selection system of Figure 1.

Referring now to Figure 1, there is shown generally a product selection system 1 including a main server 2 operatively connected to a customer image database 3, a text data database 4 and a product image database 5. The main server 2 includes a processing unit and a computer program stored in an associated memory device for causing the main server 2 to operate in the manner illustrated in Figures 2 and 3, as will 10 be described later. The main server 2 is connectable to a client terminal 6 via a first communications network 7, in this case the Internet, and a modem 8.

The main server 2 is also connectable to a second communications network 9, in this case an ISDN/cable network. The product supplier maintains a supplier server 10 in communication with the ISDN/cable network 9. A database 11 of product images of 15 products made available for selection, and optionally purchase, to customers via the main server 2, are maintained in the database 11. Image capture devices such as a digital camera 12 and scanner 13 are also operably connectable to the supplier server 10. A screen 14 may be used in association with the digital camera 12 in order to capture the supplier server 10 includes a processing unit and computer program codes 20 stored in a memory device in order to manage the image capture, storage and retrieval requirements of the supplier server 10. Whilst Figure 1 illustrates only one supplier server 10, it is to be appreciated that a number of supplier servers and associated databases and image capture devices may also be in communication with the ISDN/cable network 9.

One or more retail terminals 15,16 are provided at a retail outlet. Image capture devices, such as digital cameras 17, 18, and scanners 19, 20, and text input devices, such as keyboards 21, 22 are connected to the retail terminal, 15, 16. Screens 23, 24 are provided in order to facilitate the capture of consumer images by the digital cameras 17, 18. A retail server 25 is provided at the retail outlet, and is in 30 communication with a customer image database 26 for temporarily storing the Melbourne\004329434 images captured by the devices 17 to 22 at the retail outlet. A communications network 27, in this case a Local Area Network (LAN) interconnects the retail server 25, and retail terminals 15,16. The retail server 25 includes a processing unit and computer program code maintained in a memory to manage the capture, storage and 5 retrieval of image and text information stored in the database 26.

The operation of the product selection system 1 shown in Figure 1 will now be explained with reference to Figures 2 and 3. At step 30, an image of a product that is desired to be made available for selection and possibly purchase by a customer from the client terminal 6 is captured by one of the image capture devices 10 12 or 13. In order to facilitate the subsequent removal of background and other extraneous material from the captured image, the screen 14 may be provided with a single colour surface and components of the image captured in that colour subsequently removed from the captured image at the supplier server 10. In order to enable the reproduction of a colour image, red, green and blue components of the 15 product image may be captured. At step 31, the colour image captured is converted into a grey scale image.

At step 32, a second image of the retail product in question is captured. In this case, an infra-red grid is projected onto the screen 14 and the red component only of the image capture. The combined mono colour product image and grid are 20 then processed by the supplier server 10, using programs XML, C++, Java and HTML, in order to determine selected dimensions of the product image. Examples of combined mono colour product and grid images are shown in Figure 4 by a clothing image 100 and associated grid 101, and in Figure 5 by a furniture image 102 and associated grid 103. The supplier server 10 then acts to determine the 25 locations at which the product image intersects the x and y grid lines. In this way, it is possible to encode the outline of the product image in a series of x, y grid reference points. Distances between x, y grid reference points may then be analysed by the supplier server 10 in order to determine key dimensions of the product image. By providing information to the supplier server 10 defining the spacing 30 between grid lines when projected onto the screen 14, the actual dimensions of the product corresponding to the product image in question can be determined by the 40470574:RNM:rb 11 January 2002 supplier server.

Once the x, y grid reference points have been determined, at step 33, for a selected product image, the digitised red, green, blue, grey scale and x, y grid reference points components are linked into a single digital image file, at step 34. At step 35, a private code is added to the image file in order to control the subsequent manipulation of the image file. At step 36, the image file is transmitted to the main server 2 from the suppler server 10, using XML coding. The image file is then stored in the product database 5 at step 37.

Customers are able to present themselves at the retail outlet in order to have an image of their own body or their own environment captured and maintained in the customer image database 3 for subsequent comparison with product images maintained in the product database 5. Accordingly, customers place themselves in front of an image capture device, such as one of the digital cameras 17 or 18, and a digital image of their body is taken. In order to facilitate the accurate recording of their body image, close fitting clothing may be provided for the consumer to wear prior to the capture of their image. At step 40, a first colour image of the customer is captured. A single colour may be applied to the screen 23 and the red, green, and blue components of the image then captured by an image capture device. Subsequent processing of the captured at the retailer terminal acts to remove the background and other extraneous material from the image. Alternatively, a photograph or other prerecorded electronic or non-electronic image of the customer may be provided, and captured by the scanner 19 or 20. At step 41, the captured colour image of the customer is converted into a grey scale image.

At step 42, an infra-red grid is projected onto a screen 23 or 24 and a second image of the customer is captured, this time the red component only. As described previously, the combined body image and the grid image are transmitted to the retailer server 25 and analysed, at step 43, in order to determine the x, y grid reference points at which the body image intersects the x, y grid lines. An exemplary combined image of a customer body image 104 and associated reference grid 105 image is shown in Figure 4. Once the x, y grid reference points defining the outline of the customer body image have been captured, the various components 40470574:RNM:rb 11 January 2002 of the electronic customer image, namely the red, green, blue, grey scale, and x, y grid reference points, are combined into a single digital image file at step 44. A customer private code is added to the image file at step 45, in order to enable the user to subsequently manipulate their own image. The captured customer images may be temporarily stored in the database 26 during the aforementioned manipulation of the customer image.

At step 46, the image file may be transmitted to the main server 2, via the LAN 27 and Internet 7, for storage, at step 47, in the customer image database 3. Text capture devices 21 and 22 may be used by the customer at the retail outlet to enter text, at step 50, to be associated with the customer image. For example, the customer's name, date of birth, age, mail address, email address, interests and hobbies, or other personal or demographic information, may be entered and, at step 51, transmitted to the main server 2 by the LAN 27 and Internet 7. The text data associated with the customer image in question is then stored in the text database 4 at step 52. At step 53, the main server 2 acts to create a link between the text data and the associated customer image, in order to enable subsequent retrieval of both at the same time.

Subsequently, the customer is able to connect to the main server 2 via the client terminal 6 from the convenience of their own home, office or other desired location. As seen in Figure 3, once the customer has established a connection to the main server 2 at step 60, the customer may be presented with a user interface enabling them to view product images maintained in the product database 5 according to a number of different categories 62 to 67. In the illustrated example, images of products falling within the categories of toys, magazines, automobiles, homewares/electrical, fashion/accessories and gardens may be provided.

Once a product category has been selected by the user, a particular product from within that category may be selected at step 68. Each product may be available in a range of sizes, each size having differing dimensions. The product selection system 1 facilitate the selection of an appropriate product for a customer based upon the matching of the dimensions of the captured product image and at least a further image, in this case, the customer image captured at the retail outlet. 40470574:RNM:rb 11 January 2002 The dimensions of the captured product image as determined by calculating the distance between x, y grid reference points along one or more axes are then compared with the distances between x, y grid reference points of the customer image file along those same axes. For example, axes may be selected corresponding to the hip, waist, bust, and torso dimensions of a custom image, and compared to the same dimensions of a clothing article image file. Conveniently, the axes selected may be taken along grid lines of the captured mono colour customer and product images, in order to minimise the data processing requirements of the main server 2. Accordingly, once a first product size has been selected at step 69, and the dimensions along common axes of the customer image and the product image compare, a determination is made at step 71 as to whether the dimensions of the two images match, which indicates that the corresponding article of clothing is the correct size for the consumers body. If, at step 71, it is determined that one or more of the dimensions of the product image do not match the body image, a new product size is selected at step 69, and the dimensions of that new product image are again compared to the customer's body image.

If a match between the captured product image and the customer image is found at step 71, however, the two images are merged at step 72, and displayed to the customer at the client terminal 6, at step 73. The customer is by this means able to visualise the appearance of the article of clothing or other product in relation to their own body image, and decide whether the displayed product meets their needs.

Optionally, the customer may be provided at step 74 with the option of altering their own image. If this option is selected, the customer is required to enter the customer private code, at step 75, provided to them at the time of capturing their customer image at the retail outlet. If the main server 2 determines that the customer private code entered by the customer corresponds to that encoded in the customer image file, the user is able to manipulate various portions of the customer image. For example, consumers may wish to alter sections of their own body image, in order to visualise the appearance of the article of clothing or other product image were they to increase or decrease their own body dimensions.

At step 76, following the manipulation of the customers own customer image 40470574:RNM:rb 11 January 2002 a new merged image is displayed to the customer at the client terminal 6. At step 77, the customer is provided with the option of purchasing the displayed product. The customer may be presented with a number of alternative mechanisms for purchasing and arranging for delivery of the product. Such mechanisms are well known in the art and will not be described in detail in this specification.

Upon the selection of product information at step 68, upon the display of merged display of product image at step 72, and upon purchase of a product at step 77, demographic information characterising the customer may be retrieved from the main server 2 from the text database 4 and provided to the supplier server 10 of the retailer, in order to provide valuable marketing information to that retailer.

Whilst the above described examples refers principally to the matching of clothing images with a customer's body image, it will be appreciated that the present invention may be used in many other applications. For example, the customer image provided may represent at least a portion of the customer's environment, such as the customer's home or a room within the customer's home. The product displayed to the consumer may be a household or other item, such as a piece of furniture. The images of the customer's room and piece of furniture may be merged and presented to the customer at the client terminal 6 in order that the customer is able to better appreciate the actual appearance of the piece of furniture in their room, and in order to match the dimensions of that piece of furniture within the customer's room. Accordingly, pieces of furniture that are appropriately sized to fit in desired location of a customer's room may be accurately selected by a customer.

Moreover, the product selection need not necessarily involve the displaying of the captured images to the consumer. In one or more of the embodiments of the invention, the comparison of the dimensions of the customer image and the product image along one or more common axes may be compared and an automatic selection made by the main server 2. Alternatively, the main server 2 may compare the dimensions of the two images, and advise the customer by a text based or other message that a particular product has dimensions matching their own customer image.

In a further variant of the invention, the customer need not necessarily 40470574:RNM:rb 11 January 2002 provide, or have otherwise captured, their own image. A predetermined variety of images may be maintained in either the database 26, or the database 3, and the user provided with the option of selecting at either of the retail terminals 15 or 16 an image that, in their view, most closely matches their own requirements. For example, a room that most closely resembles their own room, or the image of a model that closely resembles their own appearance may be selected, and subsequent comparisons made between the dimensions of that selected image or the dimensions of the product images maintained in the product database 5.

The product selection system 1 further enables product selection to be based on the matching of dimensions of more than one product image maintained in the product database 5, without requiring the comparison of an image maintained in the product database 5 with an image maintained in the customer image database 3. By way of example, Figure 5 illustrates an example of a fabric image 106 and corresponding grid 107. A product image file corresponding to the fabric, may be maintained in a product database 5 in addition to a product image file corresponding to the piece of furniture 102. The customer's product selection may be based upon the matching of dimensions of the fabric image with the piece of furniture image. The present invention enable dimensions of both images to be captured along one or more common axes, so that, for example, a combined image of the piece of furniture and fabric may be presented to the user in order that an assessment be made of the suitability of the first product with the second.

Finally, it is to be understood that various modifications and/or additions and various alternative applications of the invention may be made without departing from the ambit of the invention as described herein. 40470574:RNM:rb 11 January 2002

Claims (20)

The claims defining the invention are as follows:

1. A computerised method of product selection, including; capturing in one or more databases at least a first product image of a first 5 product and at least a further image of a further item, determining in a processing unit, dimensions of the first product and the further item from each captured image, and facilitating product selection based upon the matching by the processing unit, of the dimensions of both captured images along one or more reference axes 10 in each image wherein the dimensions and at least one reference axis of each captured image are used by the processing unit to combine the captured images into a merged image.

2. A method according to claim 1, wherein the dimensions of the first product 15 and the further item are determined by: comparing an outline of each captured image to gridlines superposed with the image upon image capture; and, determining points of intersection between each captured image and these gridlines. 20

3. A method according to either one of claims 1 or 2, and further including the step of: displaying the merged image to a customer.

4. A method according to claim 3, wherein the images are selectively merged 25 according to correspondence of dimensions of the first product and the further item.

5. A method according to any one of the preceding claims, and further including the step of: selectively enabling the purchase of at least the first product, based upon the 30 matching of the dimensions of both captured images. Melbourne\004329434 - -12- | ^OFRCf^OF 15 JUL 2003 RECEIVED * R n ,1 cr\ y y 4. 0

6. A method according to claim 5, wherein the dimensions are compared along one or more axes common to both images. OCT 2002 -PECEiypn Melbourne\004149123 - Printed 2 October 2002 (16:12) - 12 a - "nicllcviuaL PROPERTY OFFICE OF N.Z. 2 1 FEB 2002 RECEIVED

7. A method according to any one of the preceding claims, wherein the first product image is an image of an item of clothing.

8. A method according to anyone of claims 1 to 6, wherein the first product image is an image of a household item, such as a piece of furniture.

9. A method according to any one of the preceding claims, wherein the further product image is a customer image.

10. A method according to claim 9, wherein the customer image is an image of at least a portion of a customer's body.

11. A method according to claim 9, wherein the customer image is an image of at least a portion of a customer's environment, such as a home or room.

12. A method according to claim 9, wherein the customer image is an image of another person's body or environment and is selected by the customer to be representative of the customer's own body or environment.

13. A method according to any one of the preceding claims, wherein the first product image is selected from a range of product images of a same product type, whereby product images of that same product type are of different dimensions, and selection is based upon matching of the dimensions of a product image with the customer image.

14. A method according to claim 13 when dependant upon any one of claims 1 to 8, wherein the further image is a second product image of a second product intended for use with the first product.

15. A method according to claim 14, wherein the second product image is an image of fabric intended for use with a first piece of furniture. -13- Freehills Carter Smith Beadle Melboume\003999832 Printed 20 February 2002 (11:55)

16. A product selection system including a server for receiving and serving images, the server including a processing unit and an associated memory device for storing a computer program, the computer program causing the processing unit to carry out a method according to any one of the preceding claims. 5

17. A product selection system according to claim 16, and further including a first database for storing the first product images.

18. A product selection system according to either one of claims 16 or 17, and 10 further including a second database for storing the further images.

19. A product selection system according to any one of claims 16 to 18, and further including one or more image capture devices, in communication with the server, for capturing the at least first product image and/or least one further image. 15

20. A product selection system according to any one of claims 16 to 19, and further including one or more terminals for display of the first product image and further image. 20 DATED: 14 July 2003 Freehills Carter Smith Beadle Patent Attorneys for the Applicants: SALLY HOBAN and DOROTHY ALCAIDE TELLECTUAL PROPERTY OFRCE OF M.Z 18 JUL 2003 Melbourne\004329434 - -14-