KR101699765B1 - Method for purchasing goods using picture taken by consumer - Google Patents

Abstract

The present invention relates to a method for purchasing products using a picture taken by a consumer, comprising: a step of photographing a picture of products using a camera module mounted on a product purchasing client terminal by the product purchasing client terminal; a step of extracting characteristic information on a product on the picture by using an image processing algorithm, searching at least five products, which have characteristic information similar to the characteristic information on the extracted product, from a database, and listing the at least five products on a display window of the product purchasing client terminal according to a priority order based on a degree of similarity by the product purchasing client terminal; a step of, when one product most similar to the product on the picture among the at least five products listed on the display window is selected, accessing an image and seller providing server through a uniform resource locator (URL) linked with the selected product by the product purchase client terminal; and a step of providing a product production-moving picture regarding the selected product in a moving picture streaming type to the product purchasing client terminal and transferring a product purchase path regarding the selected product to the product purchasing client terminal by the image and seller providing server.

Description

METHOD FOR PURCHASING GOODS USING PICTURE TAKEN BY CONSUMER "

The present invention relates to a method for purchasing a product with a photograph taken by a product purchaser, and more particularly, to a method for purchasing a product by a product purchasing client terminal, And a method for purchasing a product with a photograph taken by a purchaser of a product providing a video and a purchase route.

For example, items such as backpacks, bags, worn shoes, running shoes, accessories, pants, watches, etc., which are usually worn by people on the streets when walking on the street, It is not easy to find out where to sell things.

Also, when visiting unspecified blogs or cafes on the Internet, it is not easy to find information about products or sellers of products in photos posted in blogs or cafes.

In addition, there is a problem in that, even if the brand of the object is known, it is necessary to search for the object of the brand and find the object of the object, and therefore, the search time and the search are followed.

Unlike products that are on-line, sales information of products that are encountered on the off-line can not be easily obtained, so the purchaser himself is not only frustrated but also does not need time and effort required for product search There is a problem that it can be consumed.

Accordingly, although the size of the home shopping market has been growing recently, it is becoming more and more advantageous in the home shopping market if shopping becomes more convenient by differentiated services such as technology for supporting product search in the trend of becoming a red ocean due to intense competition. Is expected to be.

On the other hand, as a prior art of the present invention, a server for digitizing product photographs in real time and a system for purchasing captured photo products using the same has been filed and disclosed in Korean Patent Laid-Open Publication No. 10-2015-0011659. And a captured picture product purchase system using the same captures and stores a screen of a broadcasting company recorded at the time of generating a shaking signal from the terminal, And a server for transmitting the product related information included in the captured and stored screen after a predetermined time has elapsed.

Korean Published Patent No. 10-2012-0083612 (July 26, 2012) Korean Patent Publication No. 10-2015-0011659 (Feb.

Accordingly, in order to solve the above-described problem, the present invention provides a video and sales shop providing server for automatically displaying a guide image of a product and an online purchase route, , The sales of popular products can be further increased. Also, it is possible to search for products that are difficult to be searched by product category or keyword, and to photograph the product buyers who can promote product sales by providing product information in real time It is an object of the present invention to provide a method of purchasing a product.

According to another aspect of the present invention, there is provided a method of purchasing a product using a photograph taken by a purchaser of a product, the method comprising: photographing a product photograph using a camera module mounted on the product purchasing client terminal; ; Wherein the product purchase client terminal extracts feature information on a product in a photograph using an image processing algorithm and then searches for five or more products having a feature information similar to feature information on the extracted product from the database, To the display window of the merchandise purchasing client terminal according to the order of priority according to the degree of similarity; When the commodity purchasing client terminal selects one commodity most similar to the commodity among the commodities of the five or more commodities listed in the display window, the commodity purchasing client terminal transmits the video and the supply destination providing server through a URL (Uniform Resource Locator) ; And the video and sales point providing server provides a product introduction video for a selected one product to a product purchase client terminal in a streaming format and selects a product purchase path for a selected one product to a product purchase client terminal And transmitting the information.

A method of purchasing a product with a photograph taken by a purchaser of a product according to the present invention, which is made up of such a procedure, is as follows. When a customer purchases a desired product with a camera mounted on the product purchasing client terminal, Through the application installed in the terminal, the video and sales server provides the guide image of the product and the online purchase route automatically to the product purchasing client terminal, thereby increasing the sales of the popular product more and the product which is difficult to search by the product classification or keyword It can be easily searched and can promote the sale of goods.

1 is a control block diagram of the present invention,
2 is a control flowchart of the present invention.
3 is a view showing a product photograph displayed on a display window of a product purchase client terminal,
4 is a view showing a state in which the video and the merchandise moving image information delivered from the server providing the selling place are displayed on the display window of the merchandise purchasing client terminal,

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

As shown in FIGS. 1 and 2, a method for purchasing a product using a photograph taken by a purchaser of a product according to the present invention is as follows. A product purchase client terminal (client) (S1) of photographing a product photograph using a module, and the product purchasing client terminal (1) extracts feature information on a product in the photograph using an image processing algorithm, and then extracts features similar to the feature information on the extracted product (S2) of retrieving five or more goods from the database and listing the five or more goods in a display window of the goods purchasing client terminal (1) according to the order of priority according to the degree of similarity, 1) is selected when one of the five or more products listed in the display window is the closest to the one in the picture (S3) through a URL (Uniform Resource Locator) linked to any one of the merchandise providing server (2), the video and sales shop providing server (2) (S4) of providing the product introduction video to the product purchasing client terminal (1) in the form of streaming video and delivering the product purchase route for the selected one product to the product purchasing client terminal (1) have.

The product purchasing client terminal 1 extracts feature information on a product in the photograph using an image processing algorithm and then retrieves five or more products having the feature information similar to the feature information on the extracted product from the database, The step S2 of listing the goods in the display window of the goods purchasing client terminal 1 according to the order of preference according to the degree of similarity includes the steps of removing the noise included in the photograph by the goods purchasing client terminal 1, A step of causing the merchandise purchasing client terminal 1 to pass a photograph through a sharpening filter to highlight a border of a product in the photograph, (1) performs a contour detection process or an edge detection process ion process to extract the contour and shape of the product in the photograph and the product purchase client terminal 1 extracts the color distributed in the product in the photograph to generate binary code information according to the distribution of the color .

In addition, the product purchasing client terminal 1 extracts feature information on a product in the photograph using an image processing algorithm, and then retrieves five or more items from the database having the feature information similar to the feature information on the extracted product The step S2 of listing the five or more goods on the display window of the merchandise purchasing client terminal 1 according to the order of priority according to the degree of similarity is characterized in that the merchandise purchasing client terminal 1 grasps the characters .

The step of the merchandise purchasing client terminal 1 recognizing the characters described in the merchandise in the photograph may include the steps of converting the photograph into a gray image by the merchandise purchasing client terminal 1, A gray image is passed through a low pass filter or a median filter to remove noise included in the gray image, and the product purchase client terminal 1 transmits a gray image A step of identifying and dividing each character included in the binarized photograph of the product purchasing client terminal 1, a step of allowing the product purchasing client terminal 1 to store each character as a character And recognizing the character in comparison with the pattern.

The step of binarizing the noise-removed gray image by the product purchasing client terminal 1 binarizes the noise image into a number '0' if the brightness of each pixel of the gray image is less than a preset threshold value, Is binarized to the number '1' if the brightness of each pixel of the pixel is larger than a set threshold value.

Binarizing the image into a number '0' if the brightness of each pixel of the gray image is less than a predetermined threshold value, and binarizing it to a number '1' if the brightness of each pixel of the gray image is greater than a predetermined threshold value The threshold value is binarized by setting a middle value between a brightness peak value of a product included in the gray image in the brightness distribution histogram of the gray image pixels and a brightness peak value of the product background of the gray image as a threshold value .

The step of recognizing and dividing each character included in the binarized photographs of the product purchase client terminal 1 is a step in which the characters included in the binary photographs of the product purchase client terminal 1 are divided into a unit block, Separating each unit block into a character area and a non-character area by using a projection profile method or a connected component analysis method; And a step of grasping and dividing each character included in the separated character region by applying a heuristic rope distance matching algorithm to the separated character region.

The product purchasing client terminal 1 divides the characters included in the binary photograph into a set of unit blocks and outputs each unit block to a projection profile method or a connected component analysis method ), The projection profile method is a method of calibrating a tilted document by showing the global characteristics of the document in a wavy contour obtained by calculating the number of black pixels on the projection axis, Element extraction, and character extraction.

Since the point where the projection outline is partially minimized is determined based on the document area division, most documents are composed of vertical or horizontal blocks. Therefore, when repeatedly applied, can do.

The connection element analysis method obtains a set of pixels by connecting all the pixels in an 8-directional or 4-directional adjacent pixel to any black or white pixel.

At this time, the picture area has a bulky connection element, while the character area has a small and constant connection element.

The analysis method of the coupling element merges each element to make a coupling element having similar characteristics. It can remove the shadow or extract the necessary area by analyzing the characteristics of the coupling element such as the species and the aspect ratio, It is possible to separate character lines located in an arbitrary direction regardless of the change.

However, there is a problem that the connection element analysis method requires a large amount of computation and a lot of empirical knowledge is used.

The merchandise purchasing client terminal 1 grasps and divides each character included in the separated character region by applying a heuristic distance matching algorithm to the separated character region, It is proposed to compare binary images by Huttenlocher in 1993 as a distance matching method widely used for comparison.

Basically, the heuristic rope distance matching algorithm is used to find out whether the existing image exists in the input image or in which part of the input image best matches with the current image. The smaller the matching distance, the more similar position can be determined.

This approach is efficient in terms of computational complexity, and has characteristics that are robust to changes in noise and shape of objects.

The heuristic rope distance matching algorithm is used to find out whether a small image exists in a large image and where a small image exists in a large image.

The step of the merchandise purchasing client terminal 1 recognizing the character by comparing each character with a character pattern stored in the database may be performed by the merchandise purchasing client terminal 1 performing the normalization process, A step of making the vertical size of the character purchase client terminal 1 into a predetermined form, and a step of correcting the slope of the character that has been subjected to the normalization process by the product purchasing client terminal 1 and applying the slant corrected character to a Bi-cubic interpolation algorithm, And a step of recognizing the character by comparing the character enlarged at a predetermined ratio with the character pattern stored in the database.

The product purchasing client terminal 1 extracts feature information on a product in the photograph using an image processing algorithm and then retrieves five or more products having the feature information similar to the feature information on the extracted product from the database, The step S2 of listing the products in the display window of the product purchasing client terminal 1 according to the priority order according to similarity is the same as the first embodiment, The product purchasing client terminal 1 performs a contour detection process or an edge detection process to extract contours and shapes of the products in the photographs extracted at the step of extracting contours and shapes of the products in the photographs Over 50% similar to the contours and shapes of each product stored in the database A step of extracting binary code information on a product color generated in the step of extracting a color distributed in a product in the photograph and generating binary code information according to the color distribution, A step of culling a product similar to 50% or more by comparing with the binary code information for each product color stored in the database, the step of extracting the contour and the shape of the extracted product in the database by the product purchase client terminal And the binary code information for the color of the generated product is compared with the binary code information for each product color stored in the database to obtain the product code 50 Over 5% of overlapping items are collated against each other at the same stage. And displaying the image on the display window of the product purchasing client terminal 1. [

The product purchasing client terminal 1 extracts feature information on a product in the photograph using an image processing algorithm and then retrieves five or more products having the feature information similar to the feature information on the extracted product from the database, The step S2 of listing the goods in the display window of the goods purchasing client terminal 1 according to the priority according to the degree of similarity is the same as the second embodiment in that the sharpening filter The product purchasing client terminal 1 performs a contour detection process or an edge detection process to extract contours and shapes of the products in the photographs extracted at the step of extracting contours and shapes of the products in the photographs Over 50% similar to the contours and shapes of each product stored in the database A step of extracting binary code information on a product color generated in the step of extracting a color distributed in a product in the photograph and generating binary code information according to the color distribution, The method comprising the steps of: culling a product similar to 50% or more of the product code stored in the database with the binary code information; A step of culling 50% or more similar items in comparison with characters included in each product stored in the database, a step of extracting contours and shapes of the products in the photographs extracted by the product purchasing client terminal 1 from each product The product that is culled at the stage of culling 50% or more of similar products as compared with the contour and shape of the product, Comparing the binary code information on the color of the obtained product with the binary code information of each product color stored in the database, and comparing the culled product in the step of sorting out 50% or more similar goods, The step of comparing the culled items with each other in the step of sorting 50% or more of the similar goods in comparison with the characters contained in the goods of the product purchase client terminal 1 and displaying the duplicated goods in the display window of the product purchase client terminal 1 .

The step of removing the noise included in the photograph by the product purchasing client terminal 1 may be performed by the product purchasing client terminal 1 performing Gaussian smoothing and median filtering sequentially on the photograph, Can be removed.

Here, the Gaussian smoothing is mainly used to alleviate noise or damage of a photograph.

The Gaussian smoothing removes the noise in the photograph so that the edge of the character stroke in the product is smoothly expressed.

The Gaussian smoothing also plays an important role in reducing the capacity of the photograph.

The median filtering (median filtering) obtains brightness values of the photo pixels within the mask size, arranges the brightness values of the photo pixels within the mask size, and sets the brightness value located at the center among the photo pixels within the mask size as the output value. It can be easily removed by filtering.

The sharpening of the photograph is based on a high-pass filter. When the photograph passes through the sharpening filter, low-frequency components included in the photograph are removed, and a delicate photograph in which the outline of the photograph is emphasized is obtained.

When comparing the original photograph with the sharpened photograph, the sharpened photograph means a delicate photograph in which the blurred original image passes through the high-pass filter and the contour is emphasized.

The step of extracting the color distributed in the product in the photograph and generating the code information according to the distribution of the color may include acquiring the color value per pixel distributed in the product in the photograph and then binarizing the color value per pixel .

A method of purchasing a product with a photograph taken by a purchaser of a product according to the present invention made up of such a procedure is as follows. When the customer purchases a client's favorite product with a camera mounted on the product purchasing client terminal 1, The image and sales place providing server 2 automatically provides the product purchasing client terminal 1 with the guide image and the online purchase route for the product through the application installed in the product purchasing client terminal 1, It is possible to easily search for a product that is difficult to search by the product classification or keyword, thereby facilitating the sales of the product.

1. Product purchase client terminal
2. Video and server providing server

Claims (6)

(S1) of photographing a commodity using a camera module mounted on the commodity purchasing client terminal (1);
The product purchasing client terminal 1 extracts feature information on a product in the photograph using an image processing algorithm and then retrieves five or more products having the feature information similar to the feature information on the extracted product from the database, (S2) listing more than one merchandise on the display window of the merchandise purchasing client terminal (1) according to the order of priority according to similarity;
When the commodity purchasing client terminal 1 selects one commodity most similar to the commodity among the commodities among the five commodities listed in the display window, the commodity purchasing client terminal 1 transmits the video and / or video data through a URL (Uniform Resource Locator) (S3) connecting to the selling place providing server (2);
The video and sales point providing server 2 provides a product introduction video for any one selected product to the product purchasing client terminal 1 in the form of streaming video and selects a purchase path for a selected one of the products (S4) of delivering the goods to the goods purchasing client terminal (1)
The product purchasing client terminal 1 extracts feature information on a product in the photograph using an image processing algorithm and then retrieves five or more products having the feature information similar to the feature information on the extracted product from the database, (S2) of listing more than one merchandise on the display window of the merchandise purchasing client terminal
A step of removing the noise included in the photograph by the product purchase client terminal (1)
The merchandise purchasing client terminal 1 passing a photograph through a sharpening filter to emphasize a rim of a product in a photograph,
The product purchasing client terminal 1 performs a contour detection process or an edge detection process on a photograph in which the edge of the product in the photograph is highlighted through the sharpening filter to determine the outline of the product in the photograph Extracting a shape,
And generating binary code information according to distribution of the color by extracting colors distributed in the product in the photograph by the product purchasing client terminal (1)
Wherein the merchandise purchasing client terminal (1) recognizes the characters described in the merchandise in the photograph,
The product purchasing client terminal 1 extracts feature information on a product in the photograph using an image processing algorithm and then retrieves five or more products having the feature information similar to the feature information on the extracted product from the database, (S2) of listing more than one merchandise on the display window of the merchandise purchasing client terminal
The product purchasing client terminal 1 performs a contour detection process or an edge detection process on a photograph in which an edge of a product in the photograph is highlighted through the sharpening filter to determine the outline Comparing the contour and shape of the product in the photograph extracted at the step of extracting the shape with the contour and shape of each product stored in the database,
The product purchasing client terminal 1 extracts the color distributed in the product in the photograph and generates the binary code information according to the distribution of the color, Culling 50% or more similar products by comparing the binary code information with the color;
Comparing the character identified in the step of recognizing the character described in the product in the photograph with the character contained in each product stored in the database,
The product purchase client terminal 1 compares the contour and shape of the extracted photograph with the contour and shape of each of the products stored in the database, Comparing the binary code information on the product color with the binary code information for each product color stored in the database, and comparing the culled product in the step of sorting out 50% or more similar products, The step of comparing the culled items with each other in the step of sorting out 50% or more similar items in comparison with the characters included in the goods, and displaying 5 or more duplicated items in the display window of the product purchase client terminal 1 Including,
The step of the merchandise purchasing client terminal (1) recognizing the characters described in the merchandise in the photograph
The product purchasing client terminal 1 converting a photo into a gray image,
The product purchasing client terminal 1 passes a gray image through a low pass filter or a median filter to remove noise included in the gray image,
A step of binarizing the noise-removed gray image of the goods purchase client terminal 1,
Identifying and dividing each character included in the binarized photograph by the product purchasing client terminal 1,
And the merchandise purchase client terminal 1 recognizes the character by comparing each character with a character pattern stored in a database,
The step of binarizing the noise-removed gray image of the product purchase client terminal 1
0 'if the brightness of each pixel of the gray image is less than a preset threshold value, and binarizes it to a number' 1 'if the brightness of each pixel of the gray image is greater than a preset threshold value,
Binarizing the image into a number '0' if the brightness of each pixel of the gray image is less than a predetermined threshold value, and binarizing it to a number '1' if the brightness of each pixel of the gray image is greater than a predetermined threshold value The threshold value is binarized using a median value between a brightness peak value of a product included in the gray image in the brightness distribution histogram of gray image pixels and a brightness peak value of the product background of the gray image as a threshold value ,
The step of the merchandise purchasing client terminal 1 grasping and dividing each character included in the binarized photograph
The product purchasing client terminal 1 divides the characters included in the binary photograph into a set of unit blocks and outputs each unit block to a projection profile method or a connected component analysis method ) Into a character region and a non-character region,
And the merchandise purchasing client terminal 1 grasping and dividing each character included in the separated character area by applying a heuristic distance matching algorithm to the separated character area,
The step of the merchandise purchase client terminal 1 comparing each character with a character pattern stored in the database to recognize the character
Performing a normalization process by the product purchasing client terminal 1 to make the horizontal and vertical sizes of the divided characters into a certain shape;
The merchandise purchasing client terminal 1 corrects the slope of the character that has undergone the normalization process and applies the slant corrected character to a Bi-cubic interpolation algorithm to enlarge the character at a predetermined rate, and
And comparing the character enlarged at a predetermined rate with the character pattern stored in the database to recognize the character,
The step of the consumer purchasing client terminal 1 removing the noise included in the photograph
The product purchasing client terminal 1 performs Gaussian smoothing and median filtering in order to remove noise,
The Gaussian smoothing removes the noise in the photograph so that the edge of the character stroke in the product is smoothly expressed,
The median filtering (median filtering) is a process of obtaining a brightness value of photo pixels within a mask size, arranging the brightness value of the photo pixels in a mask size, How to buy.
delete delete delete delete delete

Images