WO2021167482A1

WO2021167482A1 - Method and system for identifying encoded data by means of image scanning

Info

Publication number: WO2021167482A1
Application number: PCT/RU2020/000083
Authority: WO
Inventors: Дмитрий Андреевич РЕТУНСКИЙ; Руслан Ильясович ШАВАЛЕЕВ
Original assignee: ФИЛИМОНОВ, Илья Евгеньевич
Priority date: 2020-02-19
Filing date: 2020-02-19
Publication date: 2021-08-26
Also published as: RU2726227C1

Abstract

The present solution relates to methods and systems for identifying codes on objects in vending machines and self-service kiosks. In a method for identifying encoded data by means of image scanning, a general list of codes of products which have been delivered to a kiosk is obtained; a still image including a product from the kiosk is obtained from a camera, each product including a code which encodes information about the product, after a user has removed the product from the kiosk; the still image with the product code obtained in the previous step is transmitted to a computer vision algorithm which identifies the product code in the still image by obtaining coordinates of said product code; the code identified by means of the computer vision algorithm and located on the product is converted by means of a computing device into text format; the codes of products which the user has removed from the kiosk are determined by comparing the general product code list obtained in the first step up to when the user opens the door and the converted code, said codes being removed from the general product code list; and a command is sent to the server to deduct funds on the basis of the determined codes of the products removed by the user.

Description

METHOD AND SYSTEM FOR RECOGNIZING ENCODED DATA THROUGH IMAGE SCANNING

FIELD OF TECHNOLOGY

[001] This technical solution generally relates to the field of computing, and in particular to methods and systems for recognizing codes on objects in vending machines and self-service kiosks.

LEVEL OF TECHNOLOGY

[002] Currently, vending machines are popular because of their convenience, because they are ready to work at any time of the day or night. In addition, vending machines can be placed in remote locations such as train stations, hotel corridors, and the like, which are convenient for customers.

[003] At the same time, with the development of computer technology, it has become possible to solve a number of problems arising in the process of life, to facilitate, accelerate, improve the quality of the result. For example, the operation of various life support systems, human-computer interaction, the emergence of robotic systems, etc. However, it is currently not possible to provide a satisfactory result in some tasks (object recognition).

[004] Creation of devices that perform functions of recognition of various objects, in most cases, provides the ability to replace a person with a specialized machine. Thanks to this, the capabilities of complex systems that perform various information, logical, and analytical tasks are significantly expanded. It should be noted that the quality of work performed by a person in the workplace depends on many factors (qualifications, experience, conscientiousness, etc.). Automatic control of complex systems allows monitoring and providing timely service, identification of interference and automatic application of appropriate data processing methods, improves the quality of information transmission. It is also obvious that the use of automatic systems for product recognition tasks can provide performance that is impossible for a person.

ESSENCE OF THE TECHNICAL SOLUTION

[005] This technical solution is aimed at eliminating the disadvantages known from the prior art.

[006] The technical problem or technical problem solved in this technical solution is the recognition of goods inside the vending machine.

[007] The technical result is to improve the accuracy of object recognition inside the vending machine.

[008] The specified technical result is achieved by implementing a method for recognizing encoded data by scanning images, performed by at least one computing device, in which a general list of codes of goods that have been loaded into the kiosk is obtained; receive from at least one camera at least one frame containing at least one product from the kiosk, and each product contains at least one code in which information about the product is encoded after the user has withdrawn from the kiosk at least one product; at least one frame with the product code obtained in the previous step is transmitted to the computer vision algorithm, which identifies at least one product code on the frame by obtaining the coordinates of this product code; converting by means of a computing device at least one code identified by means of a computer vision algorithm located on the product into a text form; identify product codes that the user has withdrawn from the kiosk by comparing the general list of item codes obtained in the first step before the user opened the door and at least one previously converted code, and removing these codes from the general list of item codes; send a command to the server to write off funds based on certain product codes that the user has withdrawn.

[009] In some implementations of the technical solution, the code on the item is a line code or two-dimensional code.

[0010] In some embodiments of the technical solution, the code is applied to a destructible sticker.

[0011] In some implementations of the technical solution, the computer age algorithm is implemented in the OpenCV library or libdwt, or SimpleCV.

[0012] In some embodiments of the technical solution, the coordinates of the product code on the image in pixels are determined.

[0013] In some embodiments of the technical solution, the door locking unit is an electromagnetic lock, which consists of a metal plate and a magnetic sensor, and / or an electromechanical lock.

[0014] In some embodiments of the technical solution, the system further comprises a temperature readout unit, which is a set of temperature sensors for generating data on temperature readings inside the kiosk.

[0015] In some embodiments of the technical solution, the information input unit is a contactless card reader.

[0016] In some embodiments of the technical solution, the system interconnection unit is a Raspberry Pi.

BRIEF DESCRIPTION OF DRAWINGS

[0017] The features and advantages of the present technical solution will become apparent from the following detailed description and the accompanying drawings, in which: [0018] FIG. 1 shows an example of the implementation of the architecture of a code recognition system at an object in a vending machine.

[0019] FIG. 2 shows an example of the implementation of the technical solution, which shows a camera view of the product with the attached code.

[0020] FIG. 3 shows an example of an implementation according to which four cameras are located above a shelf.

[0021] FIG. 4 shows an example of implementation, which displays a top view with a camera installed above the shelf.

[0022] FIG. 5 shows an example of implementation in which the product code was found, surrounded by a frame, and below is the code that will be sent to block 104.

[0023] FIG. 6 shows an embodiment of the original images containing products with attached codes.

[0024] FIG. 7 shows an embodiment of an original image containing products with a pasted code converted to black and white.

[0025] FIG. 8 shows a variant of the implementation of the original image containing the products with the pasted code, in which the gradients of the black-to-white transition were found.

[0026] FIG. 9 shows an embodiment of the original image containing a product with a pasted code, in which the outlines are found, which are highlighted in color.

DETAILED DESCRIPTION OF THE INVENTION

[0027] Below will be discussed in detail the terms and their definitions used in the description of this technical solution.

[0028] In this invention, the system means a computer system, a computer (electronic computer), CNC (numerical control), PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given, well-defined sequence of operations (actions, instructions), centralized and distributed databases, smart contracts.

[0029] By a command processor is meant an electronic unit or an integrated circuit (microprocessor) executing machine instructions (programs) or the like. A command processor reads and executes machine instructions (programs) from one or more storage devices. The role of data storage devices can be, but are not limited to, hard disks (HDD), flash memory, ROM (read only memory), solid state drives (SSD), optical drives.

[0030] A program is a sequence of instructions for execution by a computer control device or command processing device.

[0031] Server (English server) - an electronic device that performs service functions at the request of the client, providing him with access to certain resources. For the purposes of this description, a server is contemplated that has a persistent connection to the internetwork that can transmit data to a client device. The server can process this data and transmit the processing result back to the client device.

[0032] A data exchange module is a server module that can represent a receiver of incoming signals, and a converter for further processing, and a translator for further sending.

[0033] Computing module is a server module that is a microprocessor specially adapted for complex signal processing.

[0034] A database storage module is a server module that stores data, which can be configured as a hard disk drive or flash memory, which is a semiconductor of electrically programmable memory. [0035] An electronic lock is an electromechanical lock with a remote control system (RC) and a power supply. An electromechanical lock consists of three parts: a lock, a deadbolt and mounting brackets.

[0036] In this technical solution, the concepts of "vending machine", "self-service kiosk", "kiosk" are used, however, it is obvious to any person skilled in the art that any vending equipment can be used, for example, vending machines for selling drinks, vending machines food, vending machines for the sale of non-food products, amusement machines, amusement rides, etc. [0037] The proposed device can be used to work with any commercially available vending machines, including those previously installed. At the same time, for the functioning of the proposed system, the presence in the machine of such nodes as a product selection panel, a display, a bill acceptor, a coin acceptor, a change dispenser, a bank card reader, and a receipt printer is insignificant. In fact, a vending machine or a kiosk in the proposed solution is an automatic point of issue of goods, which can only contain means for storing goods and a mechanism for issuing them, controlled by a controller, which is connected through a telecommunication network with a server via a modem and a payment converter - a device necessary for the presence of blocks for accepting coins or bills in the vending machine, to confirm the payment and authorize the issuance of goods.

[0038] In some embodiments, the presence of a transparent display case, which occurs in a conventional kiosk design, is also unimportant, since the full range of goods can be transferred directly to the user's mobile device upon purchase. The absence of a display case in the vending machine contributes to the increase of its vandal resistance.

[0039] This technical solution works as follows.

[0040] Step 110: A list of item codes that have been uploaded to the kiosk is obtained. [0041] Various goods offered for sale, for example, food, are loaded into the refrigerator of the vending machine or kiosk. Having expressed a desire to buy a particular product from a specific kiosk, the user opens (or installs) the corresponding mobile application on the mobile device or immediately opens the door of the kiosk or vending machine directly if he does not use the mobile application. A list of goods (or their codes, but not limited to) is stored in the refrigerator database in the system interaction unit 104. The user opens the door, picks up the desired product or a set of them, and then closes the door.

[0042] Items inside the kiosk or vending machine are labeled with a code as shown in FIG. 2, in which information about the product can be encoded, for example, product code, expiration date, unique product number, etc. In some implementations, the code may be line code or two-dimensional. Aztec Code, Data Matrix, MaxiCode, PDF417, QR code, Microsoft Tag, etc. can be used as a two-dimensional code. EAN can be used as a line code (EAN-8 consists of 8 digits, EAN-13 - 13 digits are used), UPC (UPC-A, UPC-E), Code56, Code128 (UPC / EAN-128), Codabar, etc. .d., not limited to. Some implementations use RFID tags to track goods.

[0043] In some embodiments, cameras are positioned above the merchandise viewing shelves and unit 100. In a specific embodiment, four cameras hang above each shelf. Two cameras are connected to one unit 100 (in a particular implementation, from 1 to 4 cameras can be connected). If there are 4 shelves in the refrigerator, then it turns out that there are 16 chambers and 8 blocks 100 in the refrigerator. The chambers and block 100 are located in the housing under the shelf, as shown in FIG. 3.

[0044] Cameras are directed to item codes as shown in FIG. 4, in such a way that the field of view of the cameras overlaps and the cameras cover the entire shelf. [0045] The camera (s) may (s) be provided as a digital camera configured to capture still images and moving images (eg, video). The camera may include a lens and one or more image sensors configured to capture and convert light into electrical signals. By way of example only, an image sensor may include a CMOS image sensor (eg, a CMOS active pixel (APS) sensor) or a CCD (charge coupled device) sensor. In general, an image sensor in a camera includes an integrated circuit having an array of pixels, in which each pixel includes a photo detector that collects light. It will be apparent to those skilled in the art that photodetectors in imaging pixels generally record the intensity of light captured through a camera lens. However, photodetectors themselves are generally incapable of detecting the wavelength of captured light and thus incapable of detecting color information.

[0046] Accordingly, the image sensor may further include a color filter array (CFA) that may overlap or overlap the pixel array of the image sensor to capture color information. The array of color filters can include an array of small color filters, each of which can overlap a corresponding pixel of the image sensor and filter the captured light by wavelength. Thus, when used together, the color filter array and photodetectors can provide wavelength and intensity information with respect to the light captured by the camera, which can represent the captured image.

[0047] In one embodiment, the color filter array may include a Bayesian color filter array that provides a filter pattern that is 50% green, 25% red, and 25% blue. So Thus, an image sensor that uses an array of Bayesian color filters can provide information regarding the intensity of light received by the camera at wavelengths of green, red, and blue, so that each pixel in the image registers only one of three colors (RGB). This information, which may be referred to as "primary image data" or data in the "primary region", may then be processed using one or more demosaicing techniques to transform the primary image data into a full color image, generally by interpolating a set of red, green, and blue for each pixel.

[0048] Step 120: receive from at least one camera at least one frame containing at least one product from the kiosk, and each product contains at least one code in which information about the product is encoded.

[0049] After the user has closed the door and picked up the necessary goods, the system interaction unit 104 sends a command to block 100. Block 100 takes photos of the shelves, finds codes on them and decodes them, and then transmits the codes of all goods in the refrigerator to system interaction unit 104.

[0050] Then, the system interaction unit 104 compares the list of codes that is stored in the database with the one received from the unit 100. When the goods are loaded into the vending machine, a list of the codes of the goods that have been uploaded to the kiosk is obtained. After closing the refrigerator, block 100 returns a list of codes that it could recognize. Then, the list of codes that was before the opening of the kiosk is compared with the list received from block 100 after it was closed. Next, the difference in values is determined, which is the codes of the goods that the user has withdrawn from the refrigerator, after which the withdrawn goods are removed from the kiosk.

[0051] After determining the codes that the user took away, the system interaction unit 104 sends a command to write off the money.

[0052] More details on image processing are disclosed below. [0053] Image processing can be understood as various stages of image processing, for example, defective pixel detection / correction, lens shading correction, image demosaicing and image sharpening, noise reduction, gamma correction, image enhancement, color space conversion, image compression, color subsampling and scaling the image, etc. In some embodiments, an image processing "pipeline" may be used to perform each of the various steps in image processing. A given pipeline can be composed of a set of components, according to each desired step, and the components can be implemented using hardware (eg, digital signal processors or ASICs) or software, or through a combination of hardware and software components.

[0054] Before proceeding, it should be noted that although various embodiments of the various image processing techniques discussed below may use Bayesian CFA, the invention described herein is not subject to limitation in this regard. Indeed, those skilled in the art will appreciate that the image processing techniques presented herein can be applied to any type of color filter array, including RGBW filters, CYGM filters, etc.

[0055] Step 130: the at least one frame with the item code obtained in the previous step is transmitted to the computer vision algorithm, which identifies at least one item code on the frame by obtaining the coordinates of this item code.

[0056] Block 100 acquires at least one camera image as shown in FIG. 6, and using, for example, the library of computer vision algorithms OpenCV or alternative libraries for image processing, such as libdwt, SimpleCV, but not limited to, processes the frame. The task of the algorithm is to determine the coordinates of the product code on the image in pixels, "cut out" the area with the code and read this code - converting it to text form.

[0057] To localize the code, transform images from a camera or cameras as follows:

• remove distortion of the lens fisheye by means of the correction function, where several parameters of the lens are set and the image is corrected (in the OpenCV library this function is called cv2.fisheye.initUndistortRectifyMap);

• convert images to black and white, as shown in FIG. 7 remove noise by, for example, using the two operations dilate and erode. The first removes rare white pixels, the second removes rare black pixels;

• Find the places of the maximum color gradient (sharp transition from white to black or vice versa), as shown in Fig. eight.

[0058] The purpose of all these transformations is to highlight the boxes around the code in the images. In various embodiments, the frame selection can be square, round, rectangular, and so on, depending on the type of barcode.

[0059] After the frames are selected, outlines are searched in the images as shown in FIG. 9. As a result, from all the found contours, contours with four angles and aspect ratios from 0.5 to 2 are obtained. This range of values is selected empirically for a specific embodiment, where the contour is a square, and it is obvious to any person skilled in the art that the ratio can be changed ... These can be rectangles that look like squares. Outlines that are too small or too large (for example, an area of 2000 to 30,000 pixels) are also clipped. Experimentally selected parameters of the area allow you to distinguish among all the contours, contours containing, for example, datamatrix codes.

[0060] Step 140: transform by the computing device at least one identified by computer vision algorithm the code located on the product, in text form.

[0061] These areas are cut out by copying the image area into an array and passed to a code-to-text algorithm that is generally known in the art. This is the standard library for reading datamatrix code.

[0062] Due to the fact that the cameras read codes only at the moment the door is closed, such a scenario is possible when the user took the product, peeled off the code and returned the code back to the refrigerator. To protect against such fraud, destructible stickers with codes are used.

[0063] Shelf weight sensors are introduced as additional protection. In this case, the sensors are connected to block 100 via an adapter. Together with the list of codes, block 100 sends the weight of the shelf to block 104. Block 104 analyzes the change in weight and change in the quantity of goods. If the data does not match, this is reported to the administrative panel. A person can connect to a refrigerator to view images from cameras and determine which goods were actually purchased.

[0064] A system for recognizing encoded data by scanning images will be described in detail below.

[0065] The door locking unit 107 (or the electrical locking system) may be an electromagnetic lock consisting of a metal plate and a magnetic sensor, or for example an electromechanical lock. This unit sends a 12V signal to the server to unlock the lock. When the signal is stopped, the lock is locked. The principle of operation of an electromagnetic lock is based on the interaction of a magnet (built into the lock body) and a counterpart (a metal plate attached to the door). Unlike an electromechanical lock, in this case, the coil is constantly powered, when the power is turned off, the contact between the magnet and the contact plate opens and the door opens. The power of the magnet must be large enough to prevent the possibility of opening the door by force. In some embodiments, electromagnetic locks for magnetic locking or magnetically mechanical locking may be used.

[0066] An input unit 108, which may be a contactless card reader. A credit card reader is a special device that recognizes data from various types of plastic cards in the following ways:

[0067] 1. Magnetic tape on the card;

[0068] 2. Chip embedded in the card;

[0069] 2. Contactless payment method.

[0070] A credit card reader, when reading from a magnetic tape or chip, has a special analysis slot through which the card is passed. The required information is then scanned automatically.

[0071] The contactless method works according to the following principle: the card is brought to the module at a distance of 30 to 250 mm. Its identifier falls under the electromagnetic radiation generated by the device and scans the individual code. The information is redirected to the controller, which happens within a fraction of a second.

[0072] The user brings the plastic card to the reader, and the card and the reader interact with the EMV protocol. The information input unit 108 can be a payment and information universal terminal, made in the form of a computer and equipped with ports for external control and monitoring, which are used to connect to the central control unit via interface communication lines an information display device, an information input device, a control unit, a reader for plastic electronic or magnetic cards, sound device, characterized in that it is equipped with a USB port.

[0073] The block for reading temperature information 106, which is a block of N temperature sensors, where N> = 0. The block polls the temperature sensors at a predetermined frequency and generates data on temperature readings. The sensor for measuring temperature is based on the dependence of electrical resistance on temperature. In an exemplary embodiment, the sensor may be a low power temperature sensor such as an LM20 from Texas Instruments ™ or a Ds18b20 from Dallas Semiconductor ™.

[0074] A networking unit 103, for example, implemented as a router for connecting all components via an Ethernet network and accessing the Internet. The router is equipped with N slots for sim cards, where N = 2 and M slots for Ethernet, where M> = 2. Duplication of sim cards is necessary for uninterrupted operation of the kiosk. The unit serves as a connecting function between the kiosk and its internal components, as well as between the kiosk and the server. The router also has a firewall to protect the kiosk from cyberattacks. In general, the aforementioned networking unit 103 may comprise a mobile radio network communication interface such as, for example, a GPRS (General Purpose Packet Radio) modem, a UMTS (Universal Mobile Communications System) modem, an HSPA (High Speed Packet Data) modem, or an LTE modem. (Long Term Development Standard), in which the above interface is connected via a mobile communication protocol to a base station to communicate with the Internet. In this case, the system may include a router, such as an ADSL router or a fiber optic router that connects the LAN to the Internet and communicates with a laptop that communicates over the Internet.

[0075] The system interaction unit 104 is a computing device that polls the temperature information reading unit 106 at regular intervals to obtain data on the temperature inside the kiosk.

[0076] An information processing unit (processor), which is a computer for pattern recognition in a video stream.

[0077] An information input / output unit 105, implemented as a touchscreen display for user interaction. Display receives signal from system interconnection unit 104, for example, via an HDMI cable and displays this signal. By means of a touch screen, the display sends a signal to the system interaction unit 104, which is a computer, data on the coordinates of the user's touch to the display. As further shown, display 105 can be configured to generate various images that can be viewed by a user. For example, during operation of a kiosk to purchase goods, display 105 may display a graphical user interface ("GUI") that allows a user to interact with an operating system and / or an application running on the kiosk. The GUI can include various layers, windows, screens, templates, or other graphical elements that can be displayed all over, or a portion, of a display device. For example, in the illustrated embodiment, the operating system GUI may include various graphical icons, each of which may correspond to different applications that may open or execute upon detection of a user's selection (eg, via keyboard or touch input). Thumbnails can be displayed in a dock or in one or more graphical window elements displayed on the screen. In some embodiments, selecting an icon may result in a hierarchical navigation process whereby selecting an icon leads to a screen or opens another graphics window that includes one or more additional icons or other GUI elements.

[0078] System Interconnection Unit 104, which is a computing device containing a set of modules, which can be either software or hardware, which are disclosed below.

[0079] In an exemplary implementation, the system comprises the following modules:

• a module for pattern recognition in a sequence of frames and counting the goods that the user has bought; • a module for performing interaction between kiosk components;

• cash module for performing settlements with the user;

• operating system;

• database;

• web server.

[0080] The interaction of the above modules and components is carried out as follows:

[0081] The user physically approaches the kiosk or any other vending machine a sufficient distance to initiate interaction, for example, one meter.

[0082] The user then unlocks the kiosk through their mobile application 200 or by touching a plastic card to the plastic card reader 108. The plastic card reader 108 sends a signal to the kiosk system interaction unit 104 to the kiosk object interaction module. This module sends a signal to the remote server 109 to check the plastic card and identify the client on it.

[0083] If the client is successfully identified, the kiosk door is unlocked, and if the determination is unsuccessful, an error is displayed, for example, on the kiosk display and asked to repeat the previous step on the kiosk display or in the user's mobile application.

[0084] The user opens the kiosk door with his hand on the kiosk door handle. [0085] When the door is opened, the signal from the door lock 107 is sent to the kiosk systems interaction unit 104 to the module for interaction between the kiosk objects.

[0086] The user closes the kiosk door, the door is locked.

[0087] The door lock 107 signals the kiosk systems interaction unit 104 to the module for interaction between the kiosk objects.

[0088] A module for interaction between kiosk objects sends a signal to a server over the Internet to obtain a list of actions to be taken upon receiving a signal. [0089] Further, this module for interaction between the objects of the kiosk sends a signal to the cash register module to perform settlements with the user for the extracted object / s.

[0090] The module for interaction between kiosk objects sends a signal to the server based on the results of the settlement with the client for the retrieved object / s.

[0091] The module for interaction between the objects of the kiosk sends a signal to the display device of the kiosk about the result of the operation.

[0092] All components of this technical solution can be connected by a bus. In some examples, a data bus includes one or more data buses. The data bus can be implemented in accordance with the Controller Area Network (CAN) bus protocol, which is defined by the International Organization for Standardization (ISO) 11898-1, the Multimedia System Data Bus (MOST) protocol, the flexible CAN data bus protocol (CAN-FD) (ISO 11898-7) and / or K-line bus protocol (ISO 9141 and ISO 14230-1) and / or Ethernet ™ protocol -ujHHbi IEEE 802.3 (2002 onwards), etc. [0093] The elements of the proposed technical solution are in a functional relationship, and their joint use leads to the creation of a new and unique technical solution. Thus, all blocks are functionally linked.

[0094] All blocks used in the system can be implemented with electronic components used to create digital integrated circuits, which is obvious to a person skilled in the art. Not limited to, microcircuits, the logic of which is determined during manufacture, or programmable logic integrated circuits (FPGA), the logic of which is set through programming, can be used. For programming, programmers and debugging environments are used that allow you to set the desired structure of a digital device in the form of a circuit diagram or a program in special hardware description languages: Verilog, VHDL, AHDL, etc. Alternatives to FPGAs can be programmable logic controllers (PLCs), basic matrix crystals (BMCs), which require a factory production process for programming; ASICs are specialized custom large integrated circuits (LSI), which are significantly more expensive for small-scale and single-piece production.

[0095] Typically, the FPGA itself consists of the following components:

• configurable logic blocks that implement the required logic function;

• programmable electronic links between configurable logic blocks;

• programmable input / output blocks providing connection of the external output of the microcircuit with the internal logic.

[0096] Blocks can also be implemented using read-only memory devices.

[0097] Thus, the implementation of all the blocks used is achieved by standard means based on the classical principles of the implementation of the foundations of computing.

[0098] As will be understood by one of ordinary skill in the art, aspects of the present technical solution may be implemented as a system, method, or computer program product. Accordingly, various aspects of the present technical solution may be implemented solely as hardware, as software (including application software, and so on), or as an embodiment combining software and hardware aspects, which may generally be referred to as a "module" , "System" or "architecture". In addition, aspects of the present technical solution may take the form of a computer program product implemented on one or more computer-readable media having computer-readable program code that is implemented thereon. [0099] Any combination of one or more computer readable media can also be used. Computer readable media storage can be, without limitation, electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, apparatus, device, or any suitable combination thereof. More specifically, examples (non-exhaustive list) of a computer-readable storage medium include: an electrical connection using one or more wires, a portable computer diskette; hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), fiber optic connection, compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any combination of the above. As used herein, a computer-readable storage medium can be any flexible storage medium that can contain or store a program for use by the system itself, device, apparatus, or in connection therewith.

[00100] Program code embedded in a computer-readable medium can be transmitted using any medium, including, without limitation, wireless, wired, fiber optic, infrared, and any other suitable network or combination of the above.

[00101] Computer program code for performing operations for the steps of the present technical solution may be written in any programming language or combinations of programming languages, including an object-oriented programming language such as Java, Smalltalk, C ++, and so on, and conventional procedural programming languages such as programming language "C" or similar programming languages. The program code can be executed on the user's computer in whole, in part, or as a separate software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer. In the latter case, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN), wide area network (WAN) or connection to an external computer (for example, via the Internet using Internet providers).

[00102] Aspects of the present technical solution have been described in detail with reference to block diagrams, schematic diagrams, and / or diagrams of methods, devices (systems), and computer program products in accordance with embodiments of the present technical solution. It will be appreciated that each block from the block diagram and / or diagrams, as well as combinations of blocks from the block diagram and / or diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other data processing device to create a procedure, such that instructions executed by a computer processor or other programmable data processing device create means to implement the functions / actions specified in block or blocks of flowchart and / or diagram.

[00103] These computer program instructions may also be stored on a computer-readable medium that can control a computer other than a programmable data processing device or devices that function in a particular way, such that the instructions stored on the computer-readable medium create a device including instructions that perform the functions / actions specified in the block diagram and / or diagram.

Claims

FORMULA

1. A method for recognizing encoded data by scanning images, performed by at least one computing device, and includes the following steps:

• receive a general list of product codes that have been uploaded to the kiosk;

• receive from at least one camera at least one frame containing at least one product from the kiosk, and each product contains at least one code in which information about the product is encoded after the user has withdrawn from the kiosk at least one product;

• at least one frame with the product code obtained at the previous step is transmitted to the computer vision algorithm, which identifies at least one product code on the frame by obtaining the coordinates of this product code;

• converting by means of a computing device at least one code, identified by means of a computer vision algorithm, located on the product, into a text form;

• determining the codes of goods that the user has withdrawn from the kiosk by comparing the general list of codes of goods obtained in the first step before the user opened the door and at least one previously converted code, and these codes are removed from the general list of codes of goods;

• send a command to the server to write off funds based on certain product codes that the user has withdrawn.

2. A method according to claim 1, characterized in that the code on the product is a linear code or a two-dimensional code.

3. A method according to claim 1, characterized in that the code is glued to a destructible sticker.

4. The method according to claim 1, characterized in that the computer age algorithm is implemented in the OpenCV library or libdwt, or SimpleCV.

5. The method according to claim 1, characterized in that the coordinates of the product code on the image are determined in pixels.

6. A system for recognizing encoded data by scanning images, comprising:

• at least one door locking unit configured to send a signal to at least one server to unlock the door lock;

• at least one block of information input, configured to receive commands from the user;

• at least one system interaction unit, configured to transmit signals between system units;

• at least one computing device capable of: obtaining a general list of codes of goods that have been loaded into the kiosk; about receiving from at least one camera at least one frame containing at least one product from the kiosk, and each product contains at least one code in which information about the product is encoded after the user has withdrawn from the kiosk at least one product; about transferring at least one frame with the product code obtained at the previous step to the computer vision algorithm, which identifies at least one product code on the frame by obtaining the coordinates of this product code; o transforming at least one identified by the algorithm computer vision of the code located on the product in text form; on determining the codes of goods that the user has withdrawn from the kiosk by comparing the general list of codes of goods obtained in the first step before the user opens the door and at least one previously converted code, and removing these codes from the general list of codes of goods; about sending a command to the server to write off funds based on certain product codes that the user has withdrawn.

7. The system according to claim 6, characterized in that the door locking unit is an electromagnetic lock, which consists of a metal plate and a magnetic sensor, and / or an electromechanical lock.

8. The system according to claim 6, characterized in that it further comprises a temperature information reading unit, which is a set of temperature sensors for generating data on temperature readings inside the kiosk.

9. The system according to claim 6, characterized in that the information input unit is a contactless card reader.

10. The system according to claim 6, characterized in that the system interaction unit is a Raspberry Pi.