KR20190020206A - Goods sales system using sensors and method thereof - Google Patents

Abstract

The present invention relates to an article sales system using a plurality of sensors and a method thereof for recognizing a purchase behavior of an item using a sensing module located near a plurality of items displayed on a shelf and requesting payment. The system is capable of more accurately recognizing movement of the item such as whether the item is moving, opened, fixed, or returned by using two or more of photo sensors, cameras, and load cell sensors located near the plurality of items displayed on the shelf.

Description

Product sales system using a plurality of sensors and its method {GOODS SALES SYSTEM USING SENSORS AND METHOD THEREOF}

The present invention relates to a product sales system using a plurality of sensors and a method thereof, and more particularly, to recognize a purchase behavior for an item using a sensing module located near a plurality of items displayed on a display stand, and to request a payment. It is about technology to do.

Retailers, wholesalers, and distributors of other products generally maintain inventory of a variety of products that can be ordered and purchased by customers or customers. For example, if you look at a typical mart, a user (customer) enters the mart, selects a product, picks up a cart containing the product, and lines up at the cash register. Thereafter, the mart employee recognizes the barcode for each product selected by the user and performs calculation on the product.

The fourth industrial revolution, which has recently been spotlighted, includes representative keywords such as big data, the Internet of Things, artificial intelligence, and deep learning. The technology of the Fourth Industrial Revolution can have a direct impact on our real life, and technology is evolving through learning and analysis beyond being a convenient tool in our lives.

Recently, with the development of these technologies, interest in offline shopping to which the fourth industrial revolution is applied has increased.

Amazon has launched a new offline mart, Amazon Go, in response to these technological changes. Amazon Go is not much different from a grocery store that sells groceries and sundries.

However, users in Amazon Go do not need to perform any calculation after selecting a product. Amazon High recognizes the product selected by the user automatically and automatically charges the payment for the recognized product by the payment means corresponding to the user.

Accordingly, US Patent Application No. 2015/0012396, filed by Amazon, is a technology related to 'TRANSITIONING ITEMS FROM A MATERIALS HANDLING FACILITY', and discloses a method of arranging and updating a user's product list by tracking the removal or placement of a product. do.

However, U.S. Patent Application Publication No. 2015/0012396 recognizes the movement of a product by using radio frequency identification (RFID) and recognizes the movement of a user by using a camera. The product (hereinafter referred to as 'item') There was a limit to tracking removal or placement).

That is, the patent describes the use of various sensors, but the embodiment or the right for the application of the sensor is not specified, and mainly by using the module of WiFi, NFC, RFID, and camera, which may occur when purchasing a product There was a limitation that the accuracy of purchasing behavior in various situations was low.

United States Patent Application Publication No. 2015/0012396 published on January 8, 2015, "TRANSITIONING ITEMS FROM A MATERIALS HANDLING FACILITY"

An object of the present invention is to use the at least two or more of the photo sensor, the camera and the load cell sensor located in the vicinity of the plurality of items displayed on the shelves to see the movement of the items, such as whether the movement, opening, correct position or return To provide a technology that can be accurately recognized.

It is also an object of the present invention to provide a technology that can optimize the number of sensors using more photosensors than load cell sensors.

In addition, an object of the present invention is to detect a user's mobile device entering a specific space, or by using a deep learning big data to detect the user's face using the 3D face recognition module, technology that can extract the user information more accurately To provide.

An article selling system according to an embodiment of the present invention includes a display stand partitioned by a plurality of rows and a plurality of columns, a photo sensor for detecting a row position and a column position of a plurality of items displayed on the display rack, and a plurality of items. A sensor module including a load cell cell sensing the weight, an image module for obtaining an image for the plurality of items, an item recognition unit for recognizing the plurality of items using the sensor module and the image module and the plurality of items And a payment processing unit for uploading and requesting information on an item in which a movement is generated by the user to an item list corresponding to the user information.

The image module may be a camera, and the camera may acquire an image of an item on which the movement is generated and a part of the user's body based on a signal sensed by the photo sensor and the load cell sensor.

The number of the photo sensor may be larger than the load cell sensor.

The item recognizing unit may recognize an item in which the movement of a pick up or a put down according to a user's selection is generated using at least two of the sensor module and the image module.

The article selling system according to the embodiment of the present invention uses a sensor module including a photo sensor for detecting the row position and the column position of the plurality of items displayed on the display rack and a load cell sensor for sensing the weight of the plurality of items. An image module for acquiring images of the plurality of items is used in a minimum quantity.

In a method of operating a merchandise sales system for a plurality of items displayed on a display stand partitioned by a plurality of rows and a plurality of columns according to an embodiment of the present invention, the plurality of items are recognized using a sensor module and an image module. And uploading and claiming information on an item in which a movement is generated by a user among the plurality of items in an item list corresponding to user information, wherein the sensor module includes the row positions and columns of the plurality of items. A photo sensor for sensing the position and a load cell sensor for sensing the weight of the plurality of items, the image module obtains an image for the plurality of items.

According to an embodiment of the present invention, by using at least two or more of a photo sensor, a camera, and a load cell sensor positioned near a plurality of items displayed on a display stand, whether an item such as whether it is open, whether it is open, whether or not it is returned or not, etc. You can recognize movement more accurately.

In addition, according to an embodiment of the present invention, the number of sensors can be optimized by using more photo sensors than the load cell sensors.

In addition, according to an embodiment of the present invention, by detecting the user's mobile device entering a specific space or by using the deep learning big data to detect the user's face using the 3D face recognition module, it is possible to extract the user information more accurately Can be.

1A and 1B are diagrams for conceptually explaining the detailed configuration of an article sales system according to an embodiment of the present invention.
2 illustrates an example in which an article selling system according to an exemplary embodiment of the present invention is applied.
3 shows an example of a display rack including a sensing module according to an embodiment of the present invention.
4A and 4B show an example of a photosensor according to an embodiment of the present invention.
5 shows an example of a load cell sensor according to an embodiment of the present invention.
6 shows an example of a camera according to an embodiment of the present invention.
7 is a flowchart illustrating a method of operating a merchandise sales system according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the embodiments. Also, like reference numerals in the drawings denote like elements.

Also, the terminology used herein is a term used to properly express a preferred embodiment of the present invention, which may vary depending on a viewer, an operator's intention, or customs in the field to which the present invention belongs. Therefore, the definitions of the terms should be made based on the contents throughout the specification.

1A and 1B are diagrams for conceptually explaining the detailed configuration of an article sales system according to an embodiment of the present invention.

More specifically, referring to FIG. 1A, an article selling system 100 according to an embodiment of the present invention uses a sensing module located near a plurality of items (or goods, articles, etc.) displayed on a display rack to display an item. Recognize purchases and pay bills accordingly.

To this end, the article selling system 100 according to the embodiment of the present invention includes an item recognition unit 110 and the payment processing unit 120.

The item recognizing unit 110 may include at least two of a plurality of sensing modules 30 including a sensor module for sensing positions and weights of a plurality of items displayed on a display rack, and an image module for obtaining images of the plurality of items. The plurality of items are recognized using the above. According to an embodiment, the display rack may be a refrigerator and may be a kind of furniture displaying items (goods).

At this time, the sensing module 30 includes a sensor module including a photo sensor for sensing the row position and the column position of the plurality of items displayed on each of the plurality of shelves in a specific space, and a load cell sensor for sensing the weight of the plurality of items; And an image module which is a camera that acquires images for a plurality of items. At this time, the present invention is characterized in that the number of photosensors more than the load cell sensor.

Here, the specific space may be a specific space of a shopping mall, a large mart, a store, a restaurant, or another commercial facility, and represents a space for selling goods or goods.

In addition, the photo sensor may be located at each of the row position and column position of the plurality of items displayed on the display rack, and may detect an item, and may detect a body part of a user who selects the plurality of items displayed on the display rack. In this case, the photo sensor may be a photo sensor or an infrared sensor, and the body part of the user may be a hand or an arm for selecting an item. In addition, the load cell sensor may be a weight sensor.

According to an embodiment, the camera may acquire an image of the body part of the user and the item on which the movement is generated based on the signals sensed by the photo sensor and the load cell sensor. For example, the camera may be in an off or sleep mode, and a sensing signal received from a photo sensor or a load cell sensor or a signal of the merchandise sales system 100 according to an embodiment of the present invention. It is activated (on) based on the can take an image at the location. Accordingly, the present invention can minimize the image processing time by minimizing the camera using the sensor.

The item recognition unit 110 of the item selling system 100 according to an embodiment of the present invention uses at least two or more of a plurality of photo sensors, load cell sensors, and a plurality of cameras located near a plurality of items displayed on a display rack. To recognize a plurality of items. That is, the present invention can more accurately recognize the buying behavior of the item of the user through the combination of the sensor and the camera, it is possible to quickly detect the malfunction and error of the device.

For example, the item recognizing unit 110 may recognize a movement of an item of pick up or put down using at least two or more. In this case, the pick-up may be an operation in which the user picks up an item, and the foot down may be an operation in which an item is held by the user.

More specifically, the user may pick up the displayed items and put them in the shopping cart, and may put the items in the shopping cart back on the display rack. As another example, the user may pick up the items displayed on the shelves and immediately put them down, and the movement of the items may occur due to carelessness of the user. That is, the item recognition unit 110 may recognize the movement of the item according to the above-described situation through the sensing module 30.

Subsequently, the payment processing unit 120 uploads and requests information on an item in which a movement is generated by the user among a plurality of items in an article list corresponding to the user information.

For example, the payment processing unit 120 may upload the billing information for the item on which the movement occurs to the item list corresponding to the user information and request the payment. If the user leaves the specific space, the payment processing unit 120 may make a payment for the item list. You can claim In this case, the item in which the movement occurs may be an item picked up by a user.

In addition, the payment processing unit 120 is a payment means included in the mobile device 10 possessed by the user, the application (Application) installed on the mobile device 10 or an external payment server (not shown) for the goods list Payment can be requested, and payment can be made in the form of direct debit using a payment method set by the user. That is, the payment processing unit 120 uses at least one of a check card, a credit card, an account transfer, and an automatic transfer set by a user using at least one of the mobile device 10, an application, or an external payment server. Through the payment and billing of the list of items can be carried out, and the payment method and processing method is not limited because it applies the method used in the existing technology.

Referring to FIG. 1A, the article selling system 100 may further include a user detector 140 and a controller 130.

The user sensor 140 may detect a user entering a specific space and extract user information corresponding to the user.

For example, the user detector 140 may detect a mobile device 10 of a user entering a specific space by using a short range wireless communication device or a face recognition module 20 located near an entrance of a specific space, or the user The user may be identified by detecting a face. Thereafter, the user sensor 140 may extract user information corresponding to the identified user based on a user profile database stored and maintained in the database unit 150.

According to an exemplary embodiment, the user sensor 140 may identify a user by receiving user information from an application in the mobile device 10 that is woken up by a signal transmitted from a short range wireless communication device, and recognizes a face. The face of the user by the module may be acquired in 3D shape to identify the user. In addition, the user sensor 140 may identify a user entering a specific space and may detect a user who leaves the specific space.

In this case, the user profile database may include customer information using a specific space, for example, at least one or more of a user name, age, mobile phone number, address, purchase history, earning information, face image and device identification information. It may include information corresponding to the user. In addition, the user profile database may be updated in real time through the controller 130.

The controller 130 may acquire item information and payment information about the item on which the movement occurs, and control to upload the obtained payment information to the item list.

For example, the controller 130 obtains item information and payment information about an item on which movement is generated based on an inventory database stored and maintained in the database unit 150, and obtains the payment information from the user information. It may be controlled to upload to the list of items corresponding to the. More specifically, the controller 130 is based on the inventory database that is stored and maintained, the position and weight of the plurality of items received from the sensing module 30, whether the movement of the item from the image, whether it is opened or not Or, it can determine whether to return, and upload the determination result to the inventory database.

Accordingly, the controller 130 may determine the inventory of the item by updating the sales status and the inventory status of the item in the inventory database in real time.

In this case, the inventory database may include item information and payment information for a plurality of items displayed in a specific space. For example, the item's trademark, price, discount information, weight, size, category, expiration date, It may include information on at least one or more items of the receipt date, inventory date, identification information, distribution company information, manager information, precautions, the image before opening, the image after opening and up / down / left / right images. In addition, the inventory database may be updated in real time through the controller 130.

Subsequently, the controller 130 receives a detection result of a user who exits a specific space from the user sensor 140, and controls a bill for an item uploaded to an item list corresponding to user information through the payment processor 120. can do.

Referring to FIG. 1A, an article selling system 100 according to an exemplary embodiment of the present invention may include a mobile device 10, a near field communication device or a face recognition module 20, and a detection module via an interface module 160. 30 and a series of signal connection relationship, and can receive the information related to the item and the user, transmit the information accordingly, and stably store and manage the received data in the database unit 150.

Referring to FIG. 1A, although the mobile device 10, the near field communication device or the face recognition module 20, and the detection module 30 are illustrated as a single unit, this is for the purpose of briefly describing the present invention. The article selling system 100 according to the embodiment may include a plurality of mobile devices 10, a plurality of short range wireless communication devices or face recognition modules 20, and a plurality of sensing modules 30.

In this case, the user's mobile device 10 is a terminal possessed by a user such as a personal computer, a laptop computer, a smart phone, a tablet, and a wearable computer. The device may refer to any terminal device capable of communicating with the short range wireless communication device 20 in a specific space and accessing a site related to a specific space or installing and executing a service-only application. The mobile device 10 of the user may perform overall service operations such as service screen configuration, data input, data transmission, data storage, payment, and billing under the control of a web / mobile site or a dedicated application.

In addition, the mobile device 10 may be implemented with a known device that may include a microprocessor and an internal memory to perform arithmetic and storage processing. A schematic configuration of a user's mobile device 10 includes a mobile communication module (not shown) and a wireless LAN (Wireless LAN) for transmitting and receiving wireless signals with a base station, an exchange, an external mobile device, or a short range wireless communication device 20 on a mobile communication network. ), A wireless internet module (not shown) for wireless internet access, and a user's key input for operation of the mobile device 10 based on Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), etc. It may include a user input module (not shown), a display module (not shown), and the like. However, in a specific embodiment, it is not limited thereto.

In this case, the short range wireless communication device 20 may output a beacon signal including its identification information, and the beacon signal may be transmitted from a short distance such as Bluetooth, Zigbee, or Infrared. Communication method can be used.

FIG. 1B is a conceptual diagram illustrating the detailed configuration of the sensing module 30 and the communication between the sensing module 30 and the product sales system 100 according to an embodiment of the present invention.

The sensing module 30 according to the embodiment of the present invention may include a load cell sensor 31, a camera 32, and a photo sensor 33, which may be a plurality.

Referring to FIG. 1B, an article selling system 100 according to an exemplary embodiment of the present invention may include a load cell sensor 31 using a USB, a universal asynchronous receiver / transmitter (USBUART), or a Bluetooth communication scheme. ), The camera 32 and the photo sensor 33 may receive the location information, the weight information and the image information of the item, the user's motion information and the image information, and the movement of the item and the user's motion using the received information. Recognize the movement of a part of the body.

According to an embodiment, the article selling system 100 according to an embodiment of the present invention may be connected to the load cell sensor 31, the camera 32, and the photo sensor 33 by wire and wireless communication channels to transmit and receive information.

2 illustrates an example in which an article selling system according to an exemplary embodiment of the present invention is applied.

Referring to FIG. 2, the article selling system 200 according to the embodiment of the present invention includes a plurality of load cell sensors 230, a camera 240, and a photo sensor 250, 260 included in a display stand in a specific space 1. It includes.

For example, the display rack located in the specific space 1 may include a plurality of shelves, and each shelf may include a plurality of items 210 displayed for each area 220. In this case, a plurality of regions 220 for dividing the items of the shelf may be located on one shelf, and a plurality of items 210 may be displayed. However, according to an exemplary embodiment, the region 220 may not be included, and the load cell sensors 230 are stacked on the shelf, and the plurality of items 210 are displayed on the stacked load cell sensors 230. It may be.

In addition, a load cell sensor 230 for sensing the weight of the plurality of items 210 may be located at the lower end of the region 220 in which the plurality of items 210 are displayed, and each of the display racks may include a plurality of items 210. The camera 240 for acquiring an image may be located.

In addition, the photo sensor 250 of the row position for detecting the row position of the plurality of items 210 and the photo sensor 260 of the column position for detecting the column position of the plurality of items 210 may be located. .

The article selling system 200 according to the exemplary embodiment of the present invention is characterized in that the number of the photo sensors 250 and 260 is larger than that of the load cell sensor 230.

Referring to FIG. 2, the load cell sensors 230 may be disposed in a single area 220, and a plurality of cameras 240 and photo sensors 250 and 260 may be disposed in the vicinity of the plurality of items 210. . In this case, the area 220 may classify the plurality of items 210 according to a category, type, or trademark, and the number and arrangement of the areas 220 are not limited thereto.

3 shows an example of a display rack including a sensing module according to an embodiment of the present invention.

Referring to FIG. 3, the display rack 300 may include a plurality of shelves 310 and 320. For example, when the upper shelf of the display rack 300 is referred to as the first shelf 310 and the lower shelf as the second shelf 320, the first shelf 310 may include a plurality of regions 311. have.

In this case, the area 311 may be a tray or a partition for dividing the item 312 by the category, type, or trademark on the first shelf 310, and is classified by the partition wall 530 illustrated in FIG. 5 below. It can also mean space. In addition, a plurality of items 312 may be displayed in the area 311.

For example, the first shelf 310 may include two regions 311, and the regions 311 may include a plurality of items 312 having different types, categories, brands, numbers, and sizes. In addition, a load cell sensor 313 that senses the weight of the plurality of items 312 is located at the bottom of the area 311, and a plurality of cameras 316 and a photo sensor of the column position are located at the top of the plurality of items 312. 315 may be located. However, according to an embodiment, the region 331 may not be included, and the load cell sensor 313 is stacked on the first shelf 310, and the plurality of items 312 are stacked on the stacked load cell sensor 313. It may be in this form.

In this case, the camera 316 may include a left camera 316 _L , a center camera 316 _C , and a right camera 316 _R for sensing the plurality of items 312 displayed on the first shelf 310. The photo sensor 315 in the column position may include a photo sensor 315 _{1 in} a first row, a photo sensor 315 _{2 in} a second row, for sensing one row or two rows of the plurality of items 312, The third row of photosensors 315 ₃ and the fourth row of photosensors 315 ₄ may be included. However, the position and the number of the camera 316 and the photo sensor 315 in the column position are not limited thereto and may be variously applied according to an embodiment of the present invention. In addition, the number of the photo sensor 315 in the column position may vary depending on the type, category, brand, number, and size of the plurality of items 312, and the number of items detected by the photo sensor 315 in the column position and Of course, the arrangement can also be variously applied according to the embodiment.

In addition, one load cell sensor 313 is one per area 311, or one per area classified by a partition (not shown) included on the first shelf 310, and the photo sensor (314, 315) And less than the number of cameras 316. Accordingly, the article sales system according to an embodiment of the present invention can reduce the number of load cell sensors 313 and use the camera 316 and photo sensors 314 and 315 to more accurately grasp the movement of items. to provide.

Taking the second shelf 320 of FIG. 3 as an example, a plurality of items 322 may be displayed on the second shelf 320. In this case, the plurality of items 322 are displayed in two layers in the area 321, a load cell sensor 323 is located at the bottom of the area 321, and a plurality of cameras 326 at the top of the plurality of items 322. And the photo sensor 325 in the column position.

In addition, the photo sensor 324 in a row position may be positioned on the side surfaces of the plurality of items 322. At this time, including a photo sensor 324 is a photosensor of the first row to detect a plurality of items 322, the display of two layers (324 ₁₎ and a photosensor (324 ₂₎ of the second row in location can do.

That is, the photo sensors 314 and 324 in the row position and the photo sensors 315 and 325 in the column position detect the movement of the items 312 and 322 existing in the row position and the column position, and the corresponding items 312 and 322. ) May detect movement of a part of the user's body.

In addition, the article selling system according to an embodiment of the present invention, the load cell sensor 313, 323, the camera 316, 326 and the photo sensor 314, 324 located on the plurality of shelves (310, 320) of the display rack 300 , 315 and 325 may be added to different identification information, and sensing information received from the load cell sensors 313 and 323, the cameras 316 and 326, and the photo sensors 314, 324, 315 and 325, respectively. The variation of the items 312 and 322 can be recognized more quickly and accurately based on the identification information.

4A and 4B show an example of a photosensor according to an embodiment of the present invention.

More specifically, FIG. 4A illustrates an example of a sensor board including a photo sensor, and FIG. 4B illustrates an example of a display rack to which a sensor board is applied.

Referring to FIG. 4A, the sensor board 410 may include a plurality of light emitters 411 and a light receiver 412. In this case, the present invention includes a light emitting unit 411 and a light receiving unit 412 to be referred to as a photo sensor.

The light emitting unit 411 and the light receiving unit 412 may be repeatedly arranged at a predetermined interval, and the predetermined interval may be variously applied according to the size and type of the display rack or the type of the item. At this time, the type, shape and number of the light emitting unit 411 and the light receiving unit 412 shown in FIG. 4A are not limited, and the material of the sensor board 410 including the light emitting unit 411 and the light receiving unit 412, Substrate type and shape are also not limited.

Referring to FIG. 4B, the sensor board 410 may be attached to all sides of the display rack and may be connected and fixed by the harness 420.

That is, the article selling system 100 according to an embodiment of the present invention detects the movement of the item from the photo sensors 411 and 412 included in each of the plurality of sensor boards 410, and the body of the user who wants to select the item. You can detect movement for some.

According to an exemplary embodiment, each of the plurality of sensor boards 410 may include different identification information, and the product sales system 100 according to the exemplary embodiment of the present invention may sense information received from the photo sensors 411 and 412. And based on the identification information, the area where the motion is generated may be more accurately detected.

5 shows an example of a load cell sensor according to an embodiment of the present invention.

Referring to FIG. 5, the display rack 500 may be classified into a plurality of shelves 510, and a plurality of load cell sensors 520 may be disposed on the plurality of shelves 510. In this case, each shelf 510 may include a partition 530 for classifying items.

According to an exemplary embodiment, the load cell sensor 520 may be disposed on a wide area on the shelf 510, and may be disposed in plural numbers at the bottom of each of a plurality of regions (not shown) included on the shelf 510. . That is, the region 220 illustrated in FIG. 2 or the regions 313 and 323 illustrated in FIG. 3 may refer to a space classified by the partition wall 530, and according to an embodiment to which the present invention is applied. May or may not be included.

The load cell sensor 520 may detect the weight of the plurality of items displayed on the shelf 510, and the article selling system according to the embodiment of the present invention may be configured to determine the number of items displayed on the shelf 510. The weight can be recognized.

According to an embodiment, assuming that a plurality of items displayed on the shelf 510 is bottled water, when a user picks up one bottled water and puts it down after ingestion, the load cell The sensor 520 may detect a change in the weight and transmit the detected weight to the product sales system. At this time, the merchandise sales system can determine the number and movement of the item from the camera and the photo sensor to determine whether the item is opened, fixed or not according to the weight change, and can request the user for payment according to the determination result. have.

That is, the load cell sensor 520 may detect whether the item is opened or may detect whether the user ingests the item. In addition, the article sales system according to an embodiment of the present invention is characterized in that the load cell sensor 520 is disposed in a predetermined area on the shelf 510 without using the plurality of items, thereby using a smaller number.

According to an embodiment, the load cell sensor 520 may detect movement of an item by a user. For example, if a user picks up an item of bottled water displayed on the top shelf and then puts it down to the bottom shelf, the load cell sensor 520 of the top shelf detects the weight loss and load cell sensor 520 of the bottom shelf. ) Can detect an increase in weight. However, in this case, the goods sales system according to the embodiment of the present invention may not be determined to be a purchase because the user does not possess the item.

Accordingly, the load cell sensor 520 transmits the weight change to the article selling system according to the embodiment of the present invention, the article selling system according to the embodiment of the present invention uses the camera and the photo sensor to change the weight on each shelf Determines the movement of the item and whether to purchase or not, and if the item is not displayed in the correct position, the information of the item may be transmitted to the administrator managing a specific space.

6 shows an example of a camera according to an embodiment of the present invention.

Referring to FIG. 6, a plurality of cameras 610 may be disposed on a wall surface of the display rack 600, and according to an embodiment, the camera 610 may be a Microsoft 720P-class PC CAM.

Camera 610 according to an embodiment of the present invention is based on the off (off) or sleep mode (sleep mode), the sensing signal received from the photo sensor or the load cell sensor or the article sale according to the embodiment of the present invention It can be activated based on a signal from the system to capture an image at that location. Accordingly, the present invention can minimize the image processing time by minimizing the camera using the sensor.

More specifically, the article selling system according to an embodiment of the present invention can detect the movement of the item or the movement of the body part of the user from the photo sensor and the load cell sensor. When a movement is detected, the merchandise sales system according to the embodiment of the present invention activates the camera 610 at the corresponding position where the movement is detected, and the movement of the item and the body part of the user from the activated camera 610 is performed. An image of the movement may be received.

That is, the camera 610 according to an embodiment of the present invention is activated when a movement of a part of the user's body or an item is generated, thereby improving the speed of image processing and increasing the processing capacity. It can be managed efficiently.

7 is a flowchart illustrating a method of operating a merchandise sales system according to an embodiment of the present invention.

The method shown in FIG. 7 may be performed by an article selling system according to the embodiment of the present invention shown in FIG. 1A.

Referring to FIG. 7, in operation 710, user information about a user entering a specific space is extracted.

In this case, step 710 may detect a mobile device possessed by a user entering the specific space using a short range wireless communication module or identify a user by detecting a face of the user using a face recognition module, and identify the user with the identified user. Extracting user information based on a user profile database that is stored and maintained.

According to an embodiment, step 710 may identify a user by receiving user information from an application in a mobile device that is woken up by a signal transmitted from a short range wireless communication device, and may be identified by a face recognition module. The user may be identified by acquiring a face in a 3D three-dimensional shape.

In this case, the user profile database may include customer information using a specific space, for example, at least one or more of a user name, age, mobile phone number, address, purchase history, earning information, face image and device identification information. It may include information corresponding to the user.

In operation 720, a proximity of a user for selecting a plurality of items displayed on the display rack is recognized.

Step 720 may be a step of recognizing a user approaching the display rack from a photo sensor that detects the row position and the column position of the plurality of items displayed on the display rack. For example, the photo sensor may be located at each of a row position and a column position of a plurality of items displayed on a display rack, and may detect an item, and may sense a body part of a user who selects a plurality of items displayed on the display rack. In this case, the photo sensor may be a photo sensor or an infrared sensor, and the body part of the user may be a hand or an arm for selecting an item.

According to an embodiment of the present invention, when the proximity of the user is recognized by the operation 720, the method of acquiring an image of a part of the user's body may be obtained by activating an image module, which is a camera located on a display stand and obtaining an image ( Not shown) may be further included.

In step 730, the item is recognized by the user movement is generated.

Step 730 is a sensor module including a photo sensor for detecting the row position and column position of the plurality of items displayed on the display rack and a load cell sensor for sensing the weight of the plurality of items, and a camera for obtaining images for the plurality of items. Recognizing an item in which a pick up or a put down movement is generated using at least two or more of a plurality of sensing modules including an image module. In this case, the pick-up may be an operation in which the user picks up an item, and the foot down may be an operation in which an item is held by the user.

Here, the present invention is characterized in that the number of photosensors more than the load cell sensor.

According to an embodiment of the present invention, when the movement of the item is recognized by the step 730, activating an image module, which is a camera, which is located on a display stand and acquires an image, acquiring an image of the movement-produced item. It may further include (not shown).

Thereafter, in operation 740, information about the item on which the motion is generated is uploaded to the item list corresponding to the extracted user information and charged.

Step 740 is a step of obtaining item information and payment information for the item in which the movement is generated based on the inventory database that is stored and maintained, and uploading and obtaining the acquired payment information to the item list corresponding to the user information. Can be.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CDROMs, DVDs, and magneto-optical media such as floppy disks. (magnetooptical media), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

1: specific space
100, 200: goods sales system
300, 500, 600: display stand
210, 312, 322: items
220, 311, 321: zone
230, 313, 323, 520: load cell sensor
240, 316, 326, 610: camera
250, 260, 314, 324, 315, 325: photo sensor
310, 320, 510: shelves
410: sensor board
411: light emitting unit
412: light receiver
420: harness
530: bulkhead

Claims (6)

A display stand partitioned by a plurality of rows and a plurality of columns;
A sensor module including a photo sensor detecting a row position and a column position of a plurality of items displayed on the display rack and a load cell cell sensing the weight of the plurality of items;
An image module for obtaining an image for the plurality of items;
An item recognizing unit recognizing the plurality of items using the sensor module and the image module; And
Payment processing unit for uploading and claim information on the item of the movement of the plurality of items by the user to the item list corresponding to the user information
Goods sales system comprising a. The method of claim 1,
The image module
Characterized in that the camera,
And the camera acquires an image of the item on which the movement is generated and a part of the user's body based on a signal sensed by the photo sensor and the load cell sensor. The method of claim 1,
And the number of photosensors is greater than the load cell sensors. The method of claim 1,
The item recognition unit
Item sales system, using the at least two of the sensor module and the image module to recognize the item in which the movement of the pick-up (put up) or put down (put down) according to the user's selection is generated . Image module for obtaining images of the plurality of items by using a sensor module including a photo sensor for detecting the row position and column position of the plurality of items displayed on the display rack and a load cell sensor for sensing the weight of the plurality of items Item sales system, characterized in that for using a minimum quantity. A method of operating a merchandise sales system for a plurality of items displayed on a display rack partitioned by a plurality of rows and a plurality of columns, the method comprising:
Recognizing the plurality of items using a sensor module and an image module; And
Including uploading and claiming information on an item in which movement is generated by a user among the plurality of items in an item list corresponding to user information,
The sensor module includes a photo sensor for sensing the row position and the column position of the plurality of items and a load cell sensor for sensing the weight of the plurality of items,
And the image module obtains an image for the plurality of items.

Images