KR20210048121A - A image based product search service providing system using the big data and a method using the same - Google Patents

Abstract

The present invention relates to a system and method for providing an image-based product search service using big data technology. More specifically, it relates to a system and method for providing an image-based product search service using big data technology, that allows a user to search for a company that provides similar products around the user through image or video thumbnails provided in various multimedia media such as TV and the Internet, and use a conventional mobile shopping mall service based on the searched information. The system includes an orderer, a mobile device, an ordering system, a server, a database, and a product store.

Description

{A image based product search service providing system using the big data and a method using the same}

The present invention relates to a system and method for providing an image-based product search service using big data technology, and in more detail, a user provides similar products around the user through images or video thumbnails provided in various multimedia media such as TV and the Internet. The present invention relates to a system and method for providing an image-based product search service using a big data technology that searches for a company to be searched for and enables a mobile shopping mall service to be used based on the searched information.

As the spread of smartphones is activated, a change is being made from a method of delivering products by wired or wireless phone to a method of proceeding processes such as product ordering and payment using an application that specializes in delivery orders.

Specifically, when order information is entered at the orderer terminal receiving the user's order information, the affiliated store server receives the order information and delivers it to the deliverer, and delivery is expected based on the order information and the user's location information (destination information). Delivery time information is calculated based on time and the order details are delivered to the user.

In the case of a mobile application service using a conventional technology, an advertisement is performed through an application using the service and a method of receiving a cost for an order system is used. In this way, synergy effects from advertisements and promotions occur only when a large number of orderers are retained.

Usually, product orderers use product delivery for reasons of time and distance limitations, and they often search and find out beforehand and use a delivery order system.

Therefore, since the actual product advertisement effect is achieved in a desired search method, a technology to solve this problem is required.

However, in the case of a conventional system, only product information on delivery and subscription points is provided, and thus, there is a problem that it is difficult to expect a large advertising effect because such a service cannot be provided.

Korean Patent Registration No. 10-1821752

The present invention was devised to solve the above problem, and when an orderer searches for a desired product through an image, the related product stores or related product providers around the orderer are matched and provided, and the orderer's preferences are provided based on the orderer's basic information. It aims to provide information according to the requirements.

It also aims to provide information by filtering data in order to detect highly reliable data from large data or big data when searching for images.

The present invention is conceived to achieve the above object, comprising: an image search module for extracting feature information from an image uploaded by an orderer and comparing it with reference data stored in a database; A category classification module that creates a preferred product category based on the product information searched by the image search module or the frequency of products delivered by an orderer, and reconfigures an interface based on the product information; Obtain order information based on the product information selected by the orderer through the reorganized interface in the category classification module, obtain the obtained order information, the position information of the orderer, and product preparation time information of the product store to obtain delivery order information and A delivery order module for transmitting the estimated delivery time to the orderer; And a reference data collection module receiving image information and metadata information for performing comparative analysis in the image search module from the orderer or the product store and storing it in the database; image-based product search using a big data technology. It relates to a providing system and a method using the same.

The present invention relates to a system and method for providing image-based product search using big data technology. In the case of using a conventional order delivery system, the product store is in charge of only simple delivery service, so the orderer cannot match and provide the desired product. Since it is difficult to provide customized product services tailored to the product preference of the orderer, the orderer must search for the desired product in advance and go through the process of using the order delivery system. There is a problem that it cannot be performed efficiently.

However, the system according to the present invention and the method using the same search for related companies only with the image of the product desired by the orderer, and order products through this, so that the desired product can be easily ordered efficiently without prior search, and the product through the image. Through search and category filtering, affiliated stores can additionally advertise. In addition, prior data is required for image search, which has the effect of enabling prosumer marketing through the use of data provided by existing orderers or affiliated stores.

In addition, the present invention does not use various image data, but corrects and stores the image data provided from the orderer or affiliated store as filterable data, and enables search based on this, so that orderers can efficiently retrieve highly reliable data. There is an effect that can be provided.

1 shows a system for providing an image-based product search service according to an embodiment of the present invention.
2 shows an order system according to an embodiment of the present invention.
3 shows a method of providing an image-based product search service using big data technology according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings. Prior to this, terms or words used in the specification and claims should not be construed as being limited to their usual or dictionary meanings, and the inventors appropriately explain the concept of terms in order to explain their own invention in the best way. Based on the principle that it can be defined, it should be interpreted as a meaning and concept consistent with the technical idea of the present invention. In addition, unless otherwise defined in the technical terms and scientific terms used, they have the meanings commonly understood by those of ordinary skill in the art to which this invention belongs, and the gist of the present invention in the following description and accompanying drawings. Descriptions of known functions and configurations that may be unnecessarily obscure will be omitted. The drawings introduced below are provided as examples in order to sufficiently convey the spirit of the present invention to those skilled in the art. Accordingly, the present invention is not limited to the drawings presented below and may be embodied in other forms. In addition, the same reference numbers throughout the specification indicate the same elements. It should be noted that the same components in the drawings are denoted by the same reference numerals wherever possible.

1 shows a system for providing an image-based product search service according to an embodiment of the present invention.

As shown in Figure 1, the image-based product search service providing system according to the present invention,

An orderer 100 placing an order for a product; A mobile device 200 with a built-in ordering system 300 and capable of wireless communication; Built in the mobile device 200, the orderer 100 provides customized product information of the orderer 100 based on the image and delivery history of the product desired by the orderer 100 and orders it to the product store 600 through the server 400 An ordering system 300 that transmits information; A server that provides information on reference data and delivery history required by the order system 300 through the mobile device 200, receives order information from the order system 300, and delivers it to the product store 600 ( 400); A database 500 included in the server 400 including product information, image information, and orderer information; And a product store 600 for receiving order information from the server 400 and delivering the product.

The orderer 100 is a consumer who requests product delivery by searching for a desired product through the mobile device 200 or selecting a desired product from the product list, and various information such as delivered product information, image information about the product, orderer information, etc. It is a data provider that provides through the ordering system (300).

The mobile device 200 is a portable terminal capable of wireless communication, and includes an ordering system 300, and receives reference data, delivery history, product information, image information, and orderer information from the server 400 through wireless communication. Then, it is transmitted to the ordering system 300, and the order information ordered by the orderer 100 is transmitted to the server 400 through the ordering system 300.

The ordering system 300 performs an image search for an image file uploaded by the orderer 100 through the mobile device 200, and provides product information, store information, and location information based on the performed information, or Based on the delivery history of 100), a customized category is provided to the orderer 100, the interface is reconfigured so that the orderer 100 can order a product, and output to the screen of the mobile device 200, and the output information Based on the order information for the product selected by the orderer 100 is transmitted to the server 400 to perform a product order.

The server 400 includes a database 500, and provides product information, image information, orderer information, and product store information to the ordering system 300 according to the request of the ordering system 300, and the ordering system ( When order information is received from 300), it is transmitted to the product store 600.

The database 500 stores reference data such as image information for image search, metadata information on the image, delivery history information of an orderer, product information, orderer information, location information, customized category information, and product store information.

Product stores 600 are product stores subscribed to the image-based product search service providing system, providing products according to order information received from the server 400, and image information to be searched by the order system 300, It provides standard data such as product information and orderer information.

2 shows an order system 300 according to an embodiment of the present invention.

As shown in Figure 2, the order system 300 according to an embodiment of the present invention,

An image search module 310 that extracts edge, color, and vector information from the image uploaded by the orderer 100 and compares it with reference data stored in the database 500 to detect a similar image;

Based on the product information searched by the image search module 310 or the order information and delivery information of the orderer 100, product information suitable for the orderer is classified by category, and the order of the orderer based on the classified product information A category classification module 320 for reconfiguring the interface to facilitate this;

Obtains order information based on the product information selected by the orderer through an interface reconstructed in the category classification module 320, and obtains the obtained order information, the orderer's location information, and product preparation time information of the product store. The delivery order module 330 for transmitting the delivery order information and the estimated delivery time to the orderer; And

A reference data collection module 340 that receives image information and metadata information for performing comparative analysis in the image search module 310 from the orderer 100 or the product store 600 and stores it in the database 500 );

The image search module 310 extracts vector information based on color, edge, and corner through image processing technology from the image uploaded from the mobile device 200 of the orderer 100, and based on the extracted vector information. By extracting image feature information and transmitting the extracted image feature information to the server 400, an image similar to the uploaded image is detected in the database 500, and the detected image information, product information, and product Store information and matched information are transmitted from the server 400.

When the orderer performs an image search, the category classification module 320 classifies the image information, product information, product store information, and matched information transmitted from the image search module 310 by distance from the orderer, A preferred product is extracted based on the classified information and the orderer information received from the database, calculated again, and categorized to generate a preferred product category.

In addition, if the orderer does not perform an image search, the category classification module 320 calls orderer information to the database, extracts the preferred product based on the orderer information received from the database, and categorizes it, Create a category.

Based on the preferred product category created by performing the image search or the preferred product category created without performing image search, product information, product store information, and location information provided to the orderer are rearranged through the mobile device. Change the interface to be displayed.

The delivery order module 330 sends order information for the product selected by the orderer to the product store 600 through the server 400, based on the interface changed or rearranged through the category classification module 320. The mobile device transmits the order information provided to the product store and the delivery information of the product store (product preparation time, estimated delivery time calculated based on the location information of the orderer, order information ordered by the orderer, etc.). It is transmitted to the device 200.

When an orderer uploads an image of a delivered product or a recommended product, the reference data collection module 340 extracts feature information by image processing the uploaded image, and based on the tag information of the uploaded image, the reference data In addition to the tag information, it is checked whether the content is similar to the image stored in the database, and if similar, the reference data for image detection is updated.

3 shows a method of providing an image-based product search service using big data technology according to an embodiment of the present invention.

As shown in Figure 3, the method for providing an image-based product search service using big data technology according to the present invention,

A user login step in which the orderer 100 logs in through the mobile device 200 (S301);

An orderer confirmation step (S302) of checking whether an orderer logged in in the user log-in step performs an order or uploads reference data;

When the orderer does not place a product order in the orderer confirmation step, when the orderer uploads an image, the meta tag of the image is checked, and similar information is detected by comparing the image stored in the database 500 based on the uploaded image. A reference data upload step (S303) of generating filtering data to facilitate data detection when searching the image based on the detected information and the meta tag;

Filtering that image-processes the image uploaded in the reference data upload step to extract feature information, stores the extracted information in a database, and generates a key by matching the stored feature information with the generated filtering data to store filtering data Data storage step (S304);

When an orderer places an order in the orderer confirmation step, a user who extracts the existing delivery information of the orderer from the database, selects a preferred product ranking based on the frequency of the product ordered by the orderer, and checks the orderer's location information Information confirmation step (S305);

When the user information is checked in the user information verification step, the orderer receives an event call as to whether to receive a product list through image search or information on all products, and a category mode is set based on the called information. Image search deactivation step to check (S306);

A search image upload step (S307) of uploading an image of a product desired by the orderer when a category mode for performing image search is selected in the image search deactivation step;

A feature data detection step (S308) of extracting image feature points by image processing the image uploaded in the search image upload step, and checking meta tag information of the uploaded image;

Compare the feature point information detected in the feature data detection step with feature point information of reference data stored in the database to detect a similar product image, extract filtering data matched with the detected similar product image, and extract the extracted filtering data A filtering data comparison step (S309) of comparing similar product information with meta tag information to check similar product information and listing the confirmed similar product information by distance based on the location information of the orderer to create a category (S309);

If the image search is not performed in the image search deactivation step, the category detection step of creating a category that provides the product menus of the registered product stores according to the ranking according to the orderer's preferred product ranking identified in the user information verification step ( S310);

A user interface change step (S311) of changing an interface so that the orderer can select a product based on the category generated in the filtering data comparison step or the category detection step;

An order information confirmation step (S312) of performing an order at a product store of the ordered product by confirming the result of the ordering by the orderer through the interface changed in the user interface changing step (S312); And

In the step of confirming the order information, the product preparation time of the product store ordered is extracted, the delivery time is extracted by calculating the location of the product store and the distance of the orderer, and the extracted information and order details are sent to the orderer. Step (S313); consists of.

100: orderer 200: mobile device
300: order system 310: image search module
320: category classification module 330: delivery order module
340: reference data collection module
400: server
500: database 600: product store

Claims (7)

An orderer 100 placing an order for a product;
A mobile device 200 with a built-in ordering system 300 and capable of wireless communication;
Built in the mobile device 200, the orderer 100 provides customized product information of the orderer 100 based on the image and delivery history of the product desired by the orderer 100, and orders it to the product store 600 through the server 400 An ordering system 300 that transmits information;
A server that provides information on reference data and delivery history required by the order system 300 through the mobile device 200, receives order information from the order system 300, and delivers it to the product store 600 ( 400);
A database 500 included in the server 400 including product information, image information, and orderer information; And
An image-based product search service system using big data technology consisting of; a product store 600 that receives order information from the server 400 and delivers a product.
The method of claim 1,
The ordering system 300,
An image search module 310 that extracts edge, color, and vector information from the image uploaded by the orderer 100 and compares it with reference data stored in the database 500 to detect a similar image;
Based on the product information searched by the image search module 310 or the order information and delivery information of the orderer 100, product information suitable for the orderer is classified by category, and the order of the orderer based on the classified product information A category classification module 320 for reconfiguring the interface to facilitate this;
Obtains order information based on the product information selected by the orderer through an interface reconstructed in the category classification module 320, and obtains the obtained order information, the orderer's location information, and product preparation time information of the product store. The delivery order module 330 for transmitting the delivery order information and the estimated delivery time to the orderer; And
A reference data collection module 340 that receives image information and metadata information for performing comparative analysis in the image search module 310 from the orderer 100 or the product store 600 and stores it in the database 500 ); Image-based product search service system using big data technology, characterized in that consisting of.
The method of claim 2,
The image search module 310,
From the image uploaded from the mobile device 200 of the orderer 100, vector information is extracted based on color, edge, and corner through image processing technology, and image feature information is extracted based on the extracted vector information, By transmitting the extracted image feature information to the server 400, an image similar to the uploaded image is detected in the database 500, and the detected image information, product information, product store information, and matched information are displayed. An image-based product search service system using big data technology, which is transmitted from the server 400.
The method of claim 2,
The category classification module 320,
When the orderer performs an image search, the image information, product information, product store information, and matched information transmitted from the image search module 310 are classified by distance to the orderer, and from the classified information and the database Based on the received orderer information, preferred products are extracted, recalculated and categorized to create preferred product categories,
When the orderer does not perform an image search, the orderer information is called in the database, the preferred product is extracted based on the orderer information received from the database, and a preferred product category is created by categorizing it. Image-based product search service system using data technology.
The method of claim 2,
The delivery order module 330,
Based on the interface changed or rearranged through the category classification module 320, order information for the product selected by the orderer is transmitted to the product store 600 through the server 400, and provided to the product store. An image-based product search service system using big data technology, characterized in that one order information and delivery information of the product store are transmitted to the mobile device (200).
The method of claim 2,
The reference data collection module 340,
When an orderer uploads an image for a delivered product or a recommended product, the uploaded image is image-processed to extract feature information, and reference data is generated based on the tag information of the uploaded image. An image-based product search service system using big data technology, characterized in that checking whether content is similar to an image stored in a database and updating reference data for image detection if similar.
A user login step in which the orderer 100 logs in through the mobile device 200 (S301);
An orderer confirmation step (S302) of checking whether an orderer logged in in the user login step performs an order or uploads reference data;
When the orderer does not place a product order in the orderer confirmation step, when the orderer uploads an image, the meta tag of the image is checked, and similar information is detected by comparing the image stored in the database 500 based on the uploaded image. A reference data upload step (S303) of generating filtering data to facilitate data detection when searching the image based on the detected information and the meta tag;
Filtering that image-processes the image uploaded in the reference data upload step to extract feature information, stores the extracted information in a database, and generates a key by matching the stored feature information with the generated filtering data to store filtering data Data storage step (S304);
When an orderer places an order in the orderer confirmation step, a user who extracts the existing delivery information of the orderer from the database, selects a preferred product ranking based on the frequency of the product ordered by the orderer, and checks the orderer's location information Information confirmation step (S305);
When the user information is checked in the user information verification step, the orderer receives an event call on whether to receive a product list through image search or information on a product, and checks the category mode based on the called information. An image search deactivation step (S306);
A search image upload step (S307) of uploading an image of a product desired by the orderer when a category mode for performing image search is selected in the image search deactivation step;
A feature data detection step (S308) of extracting image feature points by image processing the image uploaded in the search image upload step, and checking meta tag information of the uploaded image;
Compare the feature point information detected in the feature data detection step with feature point information of reference data stored in the database to detect a similar product image, extract filtering data matched with the detected similar product image, and extract the extracted filtering data A filtering data comparison step (S309) of comparing similar product information with meta tag information to check similar product information and listing the confirmed similar product information by distance based on the location information of the orderer to create a category (S309);
If the image search is not performed in the image search deactivation step, the category detection step of creating a category that provides product menus of the registered product stores according to the ranking according to the orderer's preferred product ranking identified in the user information verification step ( S310);
A user interface change step (S311) of changing an interface so that the orderer can select a product based on the category generated in the filtering data comparison step or the category detection step;
An order information confirmation step (S312) of checking a result of the orderer placing an order through the interface changed in the user interface changing step and placing an order at a product store of the ordered product (S312); And
In the step of confirming the order information, the product preparation time of the product store ordered is extracted, the location of the product store and the distance of the orderer are calculated to extract the delivery time, and the extracted information and order details are sent to the orderer. Step (S313); Image-based product search service providing method using a big data technology consisting of.

Images