SI26178A

SI26178A - Advanced package management system with the help of enriched and augmented reality technology

Info

Publication number: SI26178A
Application number: SI202100078A
Authority: SI
Inventors: Klemen Pavačič; Klemen Košir; Gregor Golobič; Miloš Urbanija
Original assignee: Margento R&D D.O.O.
Priority date: 2021-04-12
Filing date: 2021-04-12
Publication date: 2022-10-28

Abstract

Napredni sistem za upravljanje s paketniki s pomočjo tehnologije obogatene in razširjene resničnosti predstavlja zasnovo celovitega zalednega sistema, ki uporabniku s pomočjo mobilne aplikacije na pametni mobilni napravi, podprti z AR tehnologijo, omogoča enostavno in hitro izvedbo oddaje ali prejema paketa. Na tak način ima uporabnik preko mobilne aplikacije vzpostavljeno varno šifrirano dvosmerno komunikacijo z zalednim sistemom, kjer s pomočjo zaslona, kamere, trenutne lokacije, bliskavice, ARtehnologije in ostalih elementov pametne mobilne naprave izvede vsa potrebna opravila v vseh korakih izvedbe (določi velikost paketa s pomočjo kamere, določi najbližji prost in po velikosti ustrezen paketnik s pomočjo trenutne lokacije, izvede navigacijo do izbranega paketnika, izvede vizualno označbo uporabnikovega paketnika izmed množice paketnikov s pomočjo AR tehnologije, izvede vizualni prikaz vsebine določenih paketnikov s pomočjo AR tehnologije ipd.). Zaradi takšne celovite zasnove sistemapredstavlja izum pomemben korak k enostavnejši, prijaznejši, hitrejši, robustnejši in cenovno ugodnejši rešitvi vseh korakov, s katerimi se uporabniki srečujejo ob vsakokratni izvedbi te vrste storitev. Na tak način opisan izum omogoča nove dodatne možnosti pri samostojni oddaji ali prejemu paketa ter širitvi koncepta ponujanja produktov v zaprtih in zaklenjenih paketnikih.The advanced package management system using augmented and augmented reality technology represents the design of a comprehensive back-end system that enables the user to easily and quickly send or receive a package with the help of a mobile application on a smart mobile device supported by AR technology. In this way, the user has a secure encrypted two-way communication with the back-end system via the mobile application, where with the help of the screen, camera, current location, flash, AR technology and other elements of the smart mobile device, he performs all the necessary tasks in all steps of the implementation (determines the size of the package using cameras, determines the nearest free and size-appropriate package box using the current location, navigates to the selected package box, performs visual marking of the user's package box from the multitude of package boxes using AR technology, performs a visual display of the contents of certain package boxes using AR technology, etc.). Due to such a comprehensive design of the system, the invention represents an important step towards a simpler, friendlier, faster, more robust and more affordable solution to all the steps that users encounter when performing this type of service every time. The invention described in this way enables new additional possibilities in the independent delivery or reception of a package and the expansion of the concept of offering products in closed and locked package boxes.

Description

NAPREDNI SISTEM ZA UPRAVLJANJE S PAKETNIKI S POMOČJO TEHNOLOGIJE OBOGATENE IN RAZŠIRJENE RESNIČNOSTIADVANCED PACKAGE MANAGEMENT SYSTEM USING ENHANCED AND AUGMENTED REALITY TECHNOLOGY

Predmet izuma predstavlja napredni sistem za upravljanje s paketniki s pomočjo tehnologije obogatene in razširjene resničnosti (angl. »Augmented Reality«, v nadaljevanju označene kot AR), ki so z uporabo obstoječih naprav, kot so paketnik in pametne naprave s podprto AR tehnologijo (pametna mobilna naprava, pametna očala in ostale sorodne naprave) namenjeni različnim aplikativnim izvedbam enostavnejše oddaje ali prejema paketa v/iz paketnika.The subject of the invention is an advanced system for the management of parcel lockers with the help of augmented and augmented reality technology ("Augmented Reality", hereinafter referred to as AR), which, using existing devices such as parcel lockers and smart devices with supported AR technology (smart mobile device, smart glasses and other related devices) intended for various application implementations of easier sending or receiving of a package to/from the packager.

Tehnični problem, ki ga rešuje izum, predstavlja zasnovo celovitega zalednega sistema, ki uporabniku s pomočjo mobilne aplikacije na pametni mobilni napravi, podprti z AR tehnologijo, omogoča enostavno in hitro manipulacijo pri oddaji ali prejemu paketa pri izvedbi vseh uporabnikovih korakov. Na tak način ima uporabnik preko mobilne aplikacije vzpostavljeno varno šifrirano dvosmerno komunikacijo z zalednim sistemom, kjer s pomočjo zaslona, kamere, trenutne lokacije, bliskavice, AR tehnologije in ostalih elementov pametne mobilne naprave izvede vsa potrebna opravila v vseh korakih izvedbe (določi velikost paketa s pomočjo kamere, določi najbližji prost in po velikosti ustrezen paketnik s pomočjo trenutne lokacije, izvede navigacijo do izbranega paketnika, izvede vizualno označbo uporabnikovega paketnika izmed množice paketnikov s pomočjo AR tehnologije, izvede vizualni prikaz vsebine določenih paketnikov s pomočjo AR tehnologije ipd.). Zaradi takšne celovite zasnove sistema predstavlja izum pomemben korak k enostavnejši, prijaznejši, hitrejši, robustnejši in cenovno ugodnejši rešitvi vseh vmesnih korakov, s katerimi se uporabniki srečujejo ob vsakokratni izvedbi te vrste storitev. Na tak način opisan izum omogoča nove dodatne možnosti pri samostojni oddaji in/ali prejemu paketa ter širitvi koncepta novega prodajnega kanala pri ponujanju produktov v zaprtih in zaklenjenih paketnikih.The technical problem solved by the invention represents the design of a comprehensive back-end system, which, with the help of a mobile application on a smart mobile device supported by AR technology, enables the user to easily and quickly manipulate when sending or receiving a package while performing all the user's steps. In this way, the user via the mobile application has a secure encrypted two-way communication with the backend system, where with the help of the screen, camera, current location, flash, AR technology and other elements of the smart mobile device, he performs all the necessary tasks in all steps of the implementation (determines the size of the package with with the help of the camera, determine the nearest free and size-appropriate package box with the help of the current location, navigate to the selected package box, visually mark the user's package box from the multitude of box boxes using AR technology, perform a visual display of the contents of certain box boxes using AR technology, etc.). Due to such a comprehensive design of the system, the invention represents an important step towards a simpler, friendlier, faster, more robust and more affordable solution to all the intermediate steps that users encounter when performing this type of service every time. The invention described in this way enables new additional possibilities for independent delivery and/or receipt of a package and expansion of the concept of a new sales channel when offering products in closed and locked package boxes.

Opisan izum predstavlja nadgradnjo obstoječega sistema paketnikov, ki so predstavljeni v patentni prijavi s številko W02020025098A1 in naslovom 'Smart Unattended Home Delivery Box'. Omenjena patentna prijava definira paketnik kot celovit produkt, ki dostavnim službam omogoča vnos paketa točno določenega uporabnika in brez njegove prisotnosti ter nadalje uporabnikov samostojni prevzem le-tega. Ohišje paketnika je zasnovano kot kompaktno in vodotesno, ki zaradi svojih lastnosti ščiti vsebino notranjosti pred zunanjimi vremenskimi vplivi.The described invention represents an upgrade of the existing system of parcel boxes, which are presented in the patent application with the number W02020025098A1 and the title 'Smart Unattended Home Delivery Box'. The aforementioned patent application defines a package holder as a complete product that enables delivery services to enter a package of a specific user without his presence, and further, the user to pick it up independently. The case of the parcel box is designed to be compact and waterproof, which due to its properties protects the contents of the interior from external weather influences.

Znanih je nekaj podobnih rešitev od katerih pa nobena ne posega v področje uporabe pametne mobilne naprave z AR tehnologijo v segment paketnih storitev in koncepta paketnikov. Znana je rešitev po patentu s številko US9230171B2 in naslovom 'Object Outlining To Initiate A Visual Search', ki je prirejena za industrijsko rabo in omogoča iskanje predmeta na podlagi video podatkov iz kamere pritrjene na očala z AR tehnologijo. Druga znana rešitev je po patentu s številko JP2013245029A in naslovom 'Picking system', ki je prav tako prirejena za industrijsko rabo in s pomočjo očal z AR tehnologijo omogoča izboljšanje delovne učinkovitosti v proizvodnih in skladiščnih obratih. Tretja znana rešitev je predstavljena v patentu s številko JP2015124057A in naslovom 'Sorting equipment, Information display method in sorting equipment, display section of Information display mechanism in sorting equipment, Information Processing section of Information display mechanism in sorting equipment', ki se podobno kot prejšnja dva uporablja v proizvodni industriji z uporabo očal z AR tehnologijo.A few similar solutions are known, none of which interferes with the field of use of a smart mobile device with AR technology in the segment of package services and the concept of package carriers. There is a known solution under patent number US9230171B2 and entitled 'Object Outlining To Initiate A Visual Search', which is adapted for industrial use and enables the search of an object based on video data from a camera attached to glasses with AR technology. Another well-known solution is according to the patent number JP2013245029A and the title 'Picking system', which is also adapted for industrial use and with the help of glasses with AR technology enables the improvement of work efficiency in production and storage facilities. The third known solution is presented in the patent with the number JP2015124057A and the title 'Sorting equipment, Information display method in sorting equipment, display section of Information display mechanism in sorting equipment, Information Processing section of Information display mechanism in sorting equipment', which, like the previous one, two are used in the manufacturing industry using glasses with AR technology.

Skupna značilnost vseh navedenih rešitev je uporaba očal z AR tehnologijo in prikaz rezultatov na drugem zaslonu ali mediju. V primeru opisanega izuma AR očala niso nujno potrebna, saj se vso dogajanje izvede lokalno v pametni mobilni napravi serijsko opremljeni z AR tehnologijo (tako zajemanje trenutne slike, kot prikaz dodatnih vsebin na zaslonu pametne mobilne naprave v realnem času). S tem je izboljšana uporabniška izkušnja in dodatno zaščitena zasebnost uporabnika, saj se zajeta slika obdeluje sproti (v realnem času) in ne pošilja v zaledni sistem.The common feature of all the mentioned solutions is the use of glasses with AR technology and the display of results on another screen or medium. In the case of the described invention, AR glasses are not absolutely necessary, as all the action takes place locally in a smart mobile device equipped as standard with AR technology (both capturing the current image and displaying additional content on the screen of the smart mobile device in real time). This improves the user experience and additionally protects the user's privacy, since the captured image is processed on the fly (in real time) and not sent to a backend system.

Ob uporabi pojmov navidezne resničnosti (angl. »Virtual Reality«, označene kot VR) in obogatene/razširjene resničnosti (angl. »Augmented Reality«, označene kot AR), je potrebno biti previden, saj med njima obstaja bistvena razlika. VR predstavlja računalniško ustvarjeno interaktivno 3D simulacijo, ki zaznava uporabnikov položaj in gibanje ter ga obogati z dodatnimi informacijami, s pomočjo katerih uporabnik pridobi občutek lastne prisotnosti v simulaciji. Medtem ko AR predstavlja resničnost, kjer uporabnik uporablja dejansko okolje, ki je obogateno z računalniško ustvarjenimi navideznimi predmeti. Na tak način dobi uporabnik občutek, da dodatno ustvarjeni navidezni predmeti dejansko sobivajo v njegovem okolju. Za oba je značilno, da so vsi navidezni predmeti ustvarjeni v realnem času in da se izvaja usklajen prikaz dejanskega in navideznega okolja.When using the concepts of virtual reality (VR) and augmented reality (AR), it is necessary to be careful, because there is a significant difference between them. VR represents a computer-generated interactive 3D simulation that detects the user's position and movement and enriches it with additional information, with the help of which the user gains a sense of his own presence in the simulation. While AR represents reality where the user uses an actual environment that is enriched with computer generated virtual objects. In this way, the user gets the feeling that the additionally created virtual objects actually coexist in his environment. Both are characterized by the fact that all virtual objects are created in real time and that a coordinated display of the real and virtual environment is performed.

S pomočjo prednosti opisanega izuma je možno v praksi izvesti različne aplikativne scenarije:With the help of the advantages of the described invention, it is possible to implement various application scenarios in practice:

Prepoznavanje paketnika s pomočjo AR tehnologijeRecognizing the packager with the help of AR technology

Na lokacijah, kjer je postavljeno večje število paketnikov (npr. matrika 10 v širino in 3 v višino) je potrebno uporabniku pomagati, da na najlažji način poišče paketnik z njegovim paketom. Podobno velja tudi pri oddaji paketa. Kadar uporabnik želi poslati paket, mora preko mobilne aplikacije izvesti »naročilo pošiljanja«. Ob tem vnese tip paketa, s pomočjo mobilne aplikacije izmeri njegovo velikost, vnese naslov prejemnika in lokacijo paketnika iz katerega želi izvesti pošiljanje. Sistem mu avtomatsko dodeli primeren paketnik, v katerega bo vstavil svoj paket (glede na primerno velikost paketa, glede na zasedenost paketnikov, sistem upošteva logiko enakomerne uporabe vseh paketnikov v sklopu ...). Kadar pride uporabnik do lokacije paketnikov, ima enak »problem«, kot pri prevzemanju paketa - poiskati mora paketnik, ki je rezerviran za oddajo njegovega paketa. Tukaj pride v poštev opisana rešitev z mobilno aplikacijo in upoštevanjem AR tehnologije: ko se uporabnik približa sklopu paketnikov, se vklopi kamera, prepozna sklop paketnikov in na sliki barvno označi paketnik, ki je rezerviran za uporabnika (bodisi za prevzem ali za oddajo paketa). Oznaka je tako prikaže na sliki, v živo. S tem je potrebno poudariti, da za izvedbo opisanega postopka zadostuje pametna mobilna naprava z AR tehnologijo, kamero in mobilno aplikacijo.In locations where a large number of parcel lockers are placed (e.g. a matrix of 10 in width and 3 in height), it is necessary to help the user to find the parcel locker with his package in the easiest way. The same applies when submitting a package. When the user wants to send a package, he must execute a "sending order" via the mobile application. At the same time, he enters the type of package, measures its size with the help of the mobile application, enters the address of the recipient and the location of the parcel from which he wants to send it. The system automatically assigns him a suitable package holder, in which he will insert his package (given the appropriate size of the package, depending on the occupancy of the package holders, the system takes into account the logic of equal use of all package holders in the set...). When the user arrives at the location of the couriers, he has the same "problem" as when picking up the package - he has to find the courier who is reserved to drop off his package. This is where the described solution with a mobile application and consideration of AR technology comes into play: when the user approaches a set of parcel boxes, the camera turns on, recognizes the set of parcel boxes and color-codes the parcel box that is reserved for the user (either for pick-up or for delivery of the package) in the image. The label is thus displayed on the image, live. It should be emphasized that a smart mobile device with AR technology, a camera and a mobile application is sufficient to carry out the described procedure.

Uporaba paketnikov za spletne trgovceUsing packagers for online retailers

Ideja temelji na nadgradnji uporabe paketnikov, kjer bi te namestili na lokacije varnih prostorov. V te namene se lahko izkoristijo neuporabljeni prostori kot so npr. poslovalnice pošte, železniške postaje, prazni lokali v centru mesta ipd. Prostor bi bil na voljo spletnim trgovcem, ki bi paketnike polnili s popularnimi izdelki (npr. najbolj popularen prenosnik, mobilni telefon, slušalke, ovitek za telefone ipd.). V vsakem paketniku bi bil vedno le en izdelek. Kadar želi kupec kupiti enega od teh izdelkov, ima tako možnost takojšnjega prevzema - aplikacija ga usmeri do prve lokacije s paketniki, kjer je ta izdelek na voljo, da ga takoj prevzame. Nakupno izkušnjo je s tem možno izvesti na različne načine:The idea is based on upgrading the use of parcel boxes, where they would be installed at the locations of safe spaces. For these purposes, unused spaces such as, for example, post offices, railway stations, empty bars in the city center, etc. The space would be available to online retailers who would fill the parcels with popular products (e.g. the most popular laptop, mobile phone, headphones, phone case, etc.). There would always be only one product in each package. When the customer wants to buy one of these products, he has the possibility of immediate collection - the application directs him to the first location with parcel boxes where this product is available for immediate collection. The shopping experience can be carried out in different ways:

- kupec doma poišče izdelek, ga izbere in plača, ta je nato zanj avtomatsko rezerviran v paketniku. Od tukaj naprej se nadaljuje izkušnja iskanja paketnika s pomočjo AR tehnologije.- the buyer searches for the product at home, selects it and pays, it is then automatically reserved for him in the package. From here on, the experience of finding a packager with the help of AR technology continues.

- kupec pride na takšno lokacijo, kjer ga zanima, kaj vse je na voljo v pred-napolnjenih paketnikih. V mobilni aplikaciji vklopi kamero in jo usmeri v paketnike. Na vsakem paketniku se mu avtomatsko nariše slika vsebine v paketniku. V kolikor želi kupiti izdelek, ga v mobilni aplikaciji izbere, plača in takoj za tem prevzame - odpre paketnik s pomočjo mobilne aplikacije. S tem pridemo do neke vrste virtualnega prodajnega avtomata (angl. »Virtual vending machine«), le da vsebina brez mobilne aplikacije ni vidna.- the buyer comes to such a location, where he is interested in what is available in pre-filled packages. In the mobile application, turn on the camera and point it at the packagers. A picture of the content in the package is automatically drawn on each package. If he wants to buy a product, he selects it in the mobile application, pays for it and immediately picks it up - he opens the parcel box with the help of the mobile application. This leads to a kind of virtual vending machine, except that the content is not visible without the mobile application.

Celota naprednega sistema za upravljanje s paketniki zahteva vzpostavitev komunikacijske infrastrukture, sestavljene iz naslednjih gradnikov:The entire advanced package management system requires the establishment of a communication infrastructure consisting of the following components:

Zaledni sistem v oblaku (slika 1)Cloud Backend System (Figure 1)

Zaledni sistem v oblaku predstavlja centralno točko za zbiranje podatkov na enem mestu in upravljanje z njimi. Na sistemski ravni vsebuje podatkovno bazo, spletne aplikacije in komunikacijske vmesnike za podporo uporabniškim mobilnim aplikacijam. Na aplikativni ravni so nanj komunikacijsko povezane vse uporabniške aplikacije registriranih uporabnikov. Ob tem z uporabo procesne logike in algoritmov s pomočjo metod strojnega učenja in umetne inteligence uporabniku pomaga pri izvajanju vseh storitev, ki jih sistem ponuja.A cloud backend system provides a central point for collecting and managing data in one place. At the system level, it contains a database, web applications and communication interfaces to support user mobile applications. At the application level, all user applications of registered users are communicatively connected to it. At the same time, using process logic and algorithms with the help of machine learning and artificial intelligence methods, it helps the user in performing all the services that the system offers.

Uporabniška aplikacija na pametni mobilni napravi (slika 2)User application on a smart mobile device (Figure 2)

Aplikacija na pametni mobilni napravi uporabniku omogoča več različnih funkcij:The application on a smart mobile device provides the user with several different functions:

avtomatsko določanje najbližjega in najoptimalnejšega (po velikosti) prostega paketnika glede na velikost in/ali maso in/ali vsebino paketa,automatic determination of the nearest and most optimal (in terms of size) free packager according to the size and/or weight and/or content of the package,

- rezervacijo paketnika pred oddajo paketa iz seznama prostih paketnikov na posamezni lokaciji, - pomoč pri določanju velikosti paketa:- reservation of a package carrier before the delivery of a package from the list of free package carriers at an individual location, - assistance in determining the size of the package:

o z ročnim vnosom zunanjih mer paketa, o s pomočjo kamere mobilnega telefona.o by manually entering the outer dimensions of the package, o with the help of a mobile phone camera.

navigacijo uporabnika do vnaprej določenega paketnika (lokacijsko vodenje po zemljevidu), pomoč pri iskanju uporabnikovega paketnika na lokaciji skupine paketnikov, kjer uporabnik s pomočjo AR tehnologije pametne mobilne naprave (telefon, tablica, pametna očala, pametne leče ...) na zaslonu zazna svoj paketnik, saj se mu ta posebej označi in na tak način vizualno predstavi, prikaz vsebine določenih paketnikov v primeru uporabe pred-napolnjenih paketnikov z najbolj prodajanimi izdelki, kjer se ponovno s pomočjo AR tehnologije pametne mobilne naprave na zaslonu prikažejo slike vsebine posameznega paketnika, ki so uporabljeni v te namene, odklepanje paketnika in oddajo ali prevzem paketa, pregled zgodovine oddanih in prejetih paketov, vse ostale informacije in storitve, ki jih sistem ponuja.navigation of the user to a predefined parcel carrier (location guidance on the map), assistance in finding the user's parcel carrier at the location of a group of parcel carriers, where the user detects his/her location on the screen using the AR technology of a smart mobile device (phone, tablet, smart glasses, smart lenses...) package, as it is specially marked and visually presented in this way, display of the contents of certain packages in the case of using pre-filled packages with the best-selling products, where, again with the help of AR technology of a smart mobile device, images of the contents of each package are displayed on the screen, which are used for these purposes, unlocking the parcel locker and sending or receiving the package, reviewing the history of sent and received packages, all other information and services offered by the system.

Diagram poteka izvajanja opravil na pametni mobilni napravi (slika 5)Flowchart of task execution on a smart mobile device (Figure 5)

Postopek pošiljanja paketa je predstavljen v diagramu poteka 400 na sliki 5, ker posamezni koraki v mobilni aplikaciji 207 na pametni mobilni napravi 200 zajemajo sledeča opravila:The process of sending a packet is represented in the flow diagram 400 of Figure 5, because the individual steps in the mobile application 207 on the smart mobile device 200 include the following tasks:

401: uporabnik se prijavi v mobilno aplikacijo 207,401: user logs into mobile application 207,

402: izbere možnost »Ustvari nov paket«,402: Selects the "Create New Package" option,

403: vnese potrebne informacije (telefonsko številko in naziv paketa),403: enter the necessary information (phone number and name of the package),

404: izbere možnost med ročnim ali avtomatskim vnosom velikosti paketa,404: chooses between manual or automatic packet size entry,

405: izbran je postopek določanja velikosti paketa s pomočjo AR tehnologije 205,405: the process of determining the size of the package using AR technology 205 is selected,

406: uporabnik s pomočjo kamere 203 in AR modula 205 zajame velikost paketa,406: the user captures the size of the package with the help of the camera 203 and the AR module 205,

- 407: na zaslonu 202 se izpiše velikost paketa,- 407: the package size is displayed on the screen 202,

- 408: glede na trenutno lokacijo uporabnika se izpišejo vsi razpoložljivi paketniki v bližini,- 408: based on the user's current location, all available packagers in the vicinity are displayed,

- 409: uporabnik izbere najprimernejši paketnik,- 409: the user selects the most suitable packager,

- 410: izbere želeno hitrost dostave,- 410: selects the desired delivery speed,

- 411: sistem prikaže seznam storitev dostavnih podjetij s cenami in okvirnim časom dostave,- 411: the system displays a list of delivery companies' services with prices and approximate delivery time,

- 412: uporabnik izbere dostavno podjetje,- 412: the user selects a delivery company,

- 413: sledi proces plačila (sredstva na kartici se rezervirajo v višini zneska poštnine), 414: izvede se postopek preverjanja rezervacije sredstev na kreditni kartici,- 413: the payment process follows (the funds on the card are reserved in the amount of the postage), 414: the procedure for checking the reservation of funds on the credit card is carried out,

- 415: v primeru neuspešne rezervacije sredstev se uporabniku ponudi izbira druge kartice ali ponovnega poskusa rezervacije z isto kartico,- 415: in the event of an unsuccessful reservation of funds, the user is offered the choice of another card or retrying the reservation with the same card,

416: sistem rezervira paketnik za dostavo za naslednjih 24 ur,416: the system reserves a packager for delivery for the next 24 hours,

417: uporabnik prispe na lokacijo paketnikov in v mobilni aplikaciji 207 izbere shranjen paket,417: the user arrives at the package delivery location and selects a stored package in the mobile application 207,

- 418: uporabnik se odloči za način iskanja rezerviranega paketnika v sistemu paketnikov 300, 419: izbran je postopek iskanja rezerviranega paketnika s pomočjo AR tehnologije 205,- 418: the user decides on the method of searching for the reserved parcel carrier in the parcel carrier system 300, 419: the procedure for searching for the reserved parcel carrier using AR technology 205 is selected,

- 420: uporabnik s pomočjo kamere 203 in AR modula 205 poišče rezerviran paketnik,- 420: the user searches for the reserved parcel box with the help of the camera 203 and the AR module 205,

- 421; na zaslonu 202 se z dodatno barvo in grafiko označi rezerviran paketnik 301,- 421; on the screen 202, the reserved parcel box 301 is marked with additional color and graphics,

- 422: uporabnik izbere opcijo »Odpri paketnik«,- 422: the user selects the "Open packager" option,

- 423: prične se predvajati »zvočni žeton« s pomočjo zvočnika pametne mobilne naprave 200,- 423: the "sound token" starts playing using the speaker of the smart mobile device 200,

- 424: izvede se preverjanje uspešnosti odpiranja paketnika,- 424: checking the success of opening the packager is performed,

425: uporabnik vstavi paket v paketnik in v mobilni aplikaciji 207 potrdi vstavljanje,425: the user inserts the package into the package holder and confirms the insertion in the mobile application 207,

426: dostavno podjetje je avtomatsko obveščeno o novemu paketu za prevzem,426: the delivery company is automatically notified of a new package for collection,

- 427: sistem paketnik rezervira za prevzem paketa iz strani kurirja dostavnega podjetja,- 427: the system reserves the parcel picker for picking up the package from the courier of the delivery company,

- 428: kurir prevzame paket iz paketnika,- 428: the courier picks up the package from the parcel box,

429: uporabnik je avtomatsko obveščen o prevzetem paketu iz paketnika.429: the user is automatically informed about the received package from the package store.

V nadaljevanju je izum prikazan shematsko. Vključeni shematski prikazi pregledno in strnjeno prikazujejo sledeče:In the following, the invention is shown schematically. The included schematics clearly and concisely show the following:

Slika 1: Shematski prikaz komponent zalednega sistema v oblaku,Figure 1: Schematic representation of the components of the cloud backend system,

Slika 2: Shematski prikaz komponent pametne mobilne naprave, ki so potrebne in aktivne v času uporabe mobilne aplikacije,Figure 2: Schematic representation of the components of a smart mobile device, which are necessary and active during the use of the mobile application,

Slika 3: Shematski prikaz primera, kjer se na zaslonu pametne mobilne naprave s pomočjo AR tehnologije in trenutne lokacije zazna sklop paketnikov in označi uporabnikov rezerviran paketnik, Slika 4: Shematski prikaz sistema s prikazanimi medsebojnimi povezavami med pametno mobilno napravo in sistemom paketnikov,Figure 3: Schematic representation of a case where a set of parcel boxes is detected on the screen of a smart mobile device with the help of AR technology and the current location and marks the user's reserved parcel box, Figure 4: Schematic representation of the system with the shown interconnections between the smart mobile device and the parcel box system,

Slika 5: Diagram poteka izvajanja opravil v mobilni aplikaciji pri pošiljanju paketa.Figure 5: Flow chart of mobile application tasks when sending a package.

Zaledni sistem 100 predstavlja strežniško rešitev v oblaku in vsebuje vsaj sledeče sklope klasične strežniške arhitekture, navedene v nadaljevanju. Podatkovni strežnik 101 skrbi za delovanje podatkovne baze 106 v kateri so shranjeni vsi podatki celotnega sistema za upravljanje s paketniki. Spletni strežnik 102 vsebuje servisne in uporabniške spletne aplikacije 108 ter vse potrebne spletne vmesnike 107 (angl. Web Services), ki so potrebni za komunikacijo z mobilno aplikacijo 207 na pametni mobilni napravi 200. Komunikacijski vmesniki 103 skrbijo za stalno komunikacijsko povezavo z drugimi sistemi, ki so zaradi različnih razlogov aplikativno povezani s storitvijo upravljanja s paketniki. Modul za strojno učenje 104 (angl. »Machine Learning«) se uporablja za analizo podatkov in odkrivanje podobnosti v podatkovni bazi 106, podatkovno rudarjenje (angl. »Data Mining«) in učenje prepoznavanja ter obnašanja uporabnikov v posameznih situacijah glede na dane razmere. Osnovni princip modula za strojno učenje 104 predstavlja modeliranje pojavov iz množice zgodovinskih podatkov, kjer kot rezultat dobimo pravila in povezave, ki omogočajo realnejše podajanje napotkov in pomoč pri vodenju uporabnika. Modul za umetno inteligenco 109 predstavlja podporo programski opremi za analizo in prepoznavanje določenega paketnika iz sklopa paketnikov 300, kjer osnovne podatke pridobiva iz žive slike iz kamere in ostalih podrobnih informacij pridobljenih s pomočjo AR tehnologije pametne mobilne naprave.The back-end system 100 represents a server solution in the cloud and contains at least the following components of the classic server architecture listed below. The data server 101 takes care of the operation of the database 106 in which all the data of the entire package management system is stored. The web server 102 contains service and user web applications 108 and all the necessary web interfaces 107 (English Web Services), which are necessary for communication with the mobile application 207 on the smart mobile device 200. The communication interfaces 103 ensure a constant communication connection with other systems, which, for various reasons, are applicatively connected to the parcel management service. The machine learning module 104 ("Machine Learning") is used for data analysis and discovery of similarities in the database 106, data mining ("Data Mining") and learning to recognize and behave users in individual situations according to given conditions. The basic principle of the machine learning module 104 represents the modeling of phenomena from a set of historical data, where as a result we get rules and connections that enable more realistic guidance and help in guiding the user. The artificial intelligence module 109 represents support for the software for analyzing and recognizing a certain packager from the set of packagers 300, where it obtains basic data from a live image from the camera and other detailed information obtained with the help of AR technology of a smart mobile device.

Pametna mobilna naprava 200 predstavlja katerokoli napravo, ki vsebuje vsaj sledeče sklope, navedene v nadaljevanju. Komunikacijski modem 201, ki preko trenutno obstoječe brezžične tehnologije 209 (npr. GSM/3G/4G/5G omrežje, direktno IP omrežje, ...) omogoča komunikacijo s spletnimi vmesniki 107 zalednega sistema 100. Mobilna aplikacija 207 na pametni mobilni napravi 200 omogoča uporabniško interakcijo z vsemi podprtimi funkcijami vsakokratne verzije mobilne aplikacije 207. Mobilna aplikacija za lastno delovanje potrebuje dostop do kamere 203, bliskavice 204, AR modula 205 in lokacijskega modula 206 (npr. GNSS ali podoben sistem) na pametni mobilni napravi 200. Kamera 203 in bliskavica 204 služita za zajem žive slike 208 in osvetlitev okolice 208, nakar se s pomočjo AR modula 205 dejanska slika paketnikov 300 nadgradi z navideznimi elementi 301 nad enim ali več paketniki, ki se ob živi sliki prikažejo na zaslonu 202 pametne mobilne naprave 200 (npr. prikaz uporabnikovega paketnika, prikaz vsebine določenih paketnikov ipd.). S pomočjo lokacijskega modula 206 je omogočeno pridobivanje trenutne lokacije in na tak način navigacijo uporabnika do končne lokacije paketnikov 300.A smart mobile device 200 represents any device that contains at least the following components listed below. Communication modem 201, which via the currently existing wireless technology 209 (e.g. GSM/3G/4G/5G network, direct IP network, ...) enables communication with the web interfaces 107 of the back-end system 100. The mobile application 207 on the smart mobile device 200 enables user interaction with all supported functions of each version of the mobile application 207. The mobile application needs access to the camera 203, the flash 204, the AR module 205 and the location module 206 (e.g. GNSS or similar system) on the smart mobile device 200 for its own operation. The camera 203 and the flash 204 serves to capture the live image 208 and the ambient lighting 208, after which, with the help of the AR module 205, the actual image of the parcels 300 is superimposed with virtual elements 301 above one or more parcels, which are displayed on the screen 202 of the smart mobile device 200 (e.g. .display of the user's package, display of the contents of specific packages, etc.). With the help of the location module 206, it is possible to obtain the current location and in this way navigate the user to the final location of the parcel carriers 300.

Claims

PATENT CLAIMS

1. An advanced parcel management system using augmented and augmented reality technology, characterized in that it includes a back-end cloud system (100), a smart mobile device (200), and a parcel system (300).

2. An advanced package management system using augmented and augmented reality technology according to claim 1, characterized in that the back-end cloud system (100) contains at least a data server (101) for managing the database (106), a web server ( 102) with service and user web applications (108), web interfaces (107) for communication with a mobile application (207) on a smart mobile device (200), communication interfaces (103) for permanent communication connection with other systems and modules for implementing modern methods of machine learning (104), artificial intelligence (109), process logic and support algorithms (105).

3. An advanced package management system using augmented and augmented reality technology according to claim 1, characterized in that the smart mobile device (200) contains at least a communication modem of the GSM/3G/4G/5G/IP type or later (201) for communicating (209) with the web interfaces (107) of the back-end cloud system (100), a screen (202) for displaying the current image (208) obtained with the help of the camera (203) and displaying the added virtual elements (301) obtained with the help of AR technology module (205), a flash (204) for ambient lighting, a module for obtaining the current location (206) and a mobile application (207) for performing user interaction.

4. A mobile application on a smart mobile device according to claim 3, characterized in that it enables the measurement of the size of the package with the help of the camera (203), the flash (204) and the AR module (205), and thus the automatic calculation of the volumetric data of the package.

5. A mobile application on a smart mobile device according to claim 3, characterized in that it enables the automatic detection of the target parcel carrier (301) from a set of parcel carriers (300) with the help of a camera (203), a flash (204) and an AR module (205) and display enriched images on the screen (202).

6. A mobile application on a smart mobile device according to claim 3, characterized in that, with the help of the location module (206), it enables the automatic determination of the nearest location of free parcel carriers (300) that are appropriate according to the measured package size.