KR102469825B1 - Logistics picking monitoring system using image recognition based on artificial intelligence and method for processing thereof - Google Patents

Abstract

The present invention relates to a logistics picking monitoring system using image recognition based on artificial intelligence and a processing method thereof. The logistics picking monitoring system of the present invention obtains a video of a picking operation in real time using a camera of smart glasses worn by a worker, and transmits the video to the picking management server through a communication network. The picking management server recognizes and separates a section peaking image from the video transmitted from the smart glasses through artificial intelligence learning, and extracts a plurality of keyframe images from the section peaking image. The picking management server sets one representative image representing an item picking state among key frame images. The picking management server categorizes, stores, and manages various information and images for picking operations by worker, working time, and picking item. According to the present invention, by using a camera of smart glasses worn by a worker to acquire and monitor an image of a picking operation in real time, it is determined whether a work error that may occur by a worker, that is, a human error, has occurred, It is possible to manage errors in work and the work ability of workers.

Description

Logistics picking monitoring system using artificial intelligence-based image recognition and its processing method

The present invention relates to a logistics picking monitoring system, and more specifically, when a product is packaged in a product manufacturing plant, distribution company, logistics company, etc., the packaging process is automatically acquired using a camera of smart glasses worn by a worker, and Logistics picking monitoring system using artificial intelligence-based image recognition to prevent black consumers by storing and learning from artificial intelligence to identify packing mistakes by workers through image recognition and using them to resolve customer claims that occur after delivery of goods and a processing method thereof.

As part of a smart factory, industrial sites such as logistics centers are areas where the introduction of digitalization and automation is actively progressing. Out of the logistics work, shipping is the most difficult, and among them, the picking work, that is, the work of putting a plurality of products, articles, etc., in the same box is the most difficult. For example, when product is picked, a picking operation is generally performed by a worker or a picking operation is performed by introducing a picking solution.

Picking is the part where human error occurs the most because it is a task to collect the smallest unit products according to the order quantity. Even if a picking solution is introduced, the work of putting individual products in the right quantity is entirely carried out by human hands. If a mistake occurs in picking, the wrong product is delivered to the customer after shipping, and a complaint is received at the customer center, which leads to a decrease in the trust of the seller or the customer of the distribution center. In addition, too much time is required in the process of accurately confirming the customer's claim. In particular, in the case of cross-border logistics, since the delivery distance is much longer than domestic, the confirmation time for this becomes longer, resulting in more customer dissatisfaction.

In general, when a shipping claim occurs, the process of returning and confirming a product is when a product is ordered from a customer at a distribution center, the product is picked and packed in one box, shipped, and shipped. Thereafter, when the customer receives and checks the product and an error occurs, a claim is received to the customer center or distribution center. Accordingly, the customer center, distribution center, or seller checks the details of the received claim, apologizes to the customer, receives the return or exchange request, processes the return, and then checks the claim status of the product and takes follow-up measures.

In managing warehousing and warehousing in a distribution center, damage due to human error always exists. In particular, in a situation where evidence is lacking, the black consumer's persistent demand for compensation will return intact to the company. Therefore, reducing human error is the best way to maintain customer trust by preventing shipping and delivery errors. Therefore, the efficiency of picking determines the efficiency of the entire logistics system, and companies investing in logistics automation are investing the most in picking.

Korean Registered Patent Publication No. 10-1515962 (published on May 04, 2015) Korean Registered Patent Publication No. 10-1866513 (published on July 04, 2018) Korean Patent Publication No. 10-2018-0061687 (published on June 08, 2018) Korean Patent Publication No. 10-2017-0107805 (published on September 26, 2017)

An object of the present invention is to provide a logistics picking monitoring system using artificial intelligence-based image recognition and a processing method thereof for acquiring an image of a picking task using a camera and recognizing an image from the image to monitor information on the picking task. is to do

Another object of the present invention is to provide a logistics picking monitoring system using artificial intelligence-based image recognition for acquiring and monitoring images of a picking operation using a camera of smart glasses worn by a worker and a processing method thereof.

Another object of the present invention is to provide a logistics picking monitoring system using artificial intelligence-based image recognition to prevent customer claims and black consumers, and a processing method thereof.

Another object of the present invention is to provide a logistics picking monitoring system and its processing method using artificial intelligence-based image recognition for use as evidence for picking work.

In order to achieve the above objects, the logistics picking monitoring system using artificial intelligence-based image recognition of the present invention obtains an image of the picking process using a camera of smart glasses worn by a worker when picking logistics, and acquires an image of the picking process using artificial intelligence. One of its characteristics is that it is analyzed and provided so that the picking operation according to image recognition can be confirmed. The logistics picking monitoring system of the present invention can be used to prevent black consumers by checking human errors during picking operations.

According to this feature, the logistics picking monitoring system using artificial intelligence-based image recognition of the present invention recognizes a login card equipped with identification information of a worker using a camera, transmits worker information, and when packaging logistics, the camera smart glasses for recognizing and tracking an item positioned in the operator's line of sight through a smart glass to acquire a video of a picking operation and transmitting the acquired video in real time; and artificial intelligence learning using a sample image representing a normal state for a picking operation of one packing box in advance, and when worker information is transmitted from the smart glasses through a communication network, the worker is authenticated, and the smart glasses Picking job information including a video, the worker information, the working time for the picking job, and the picking item information is transmitted, and each picking section for one packing box is recognized from the video and separated into a picking section image, , Picking for one packing box by recognizing items picked in the picking section image, extracting and storing a plurality of keyframe images, and setting an image having the highest similarity to the sample image among the keyframe images as a representative image. Job information is classified and indexed at least by worker, work time, and picking item, and the picking event occurrence time is determined to determine the picking event occurrence time, the picking section image, the keyframe image, the representative image, and the It includes a picking management server that provides to monitor picking job information.

In this feature, the picking management server extracts the key frame image of the analysis target candidate from the time of occurrence of the picking event, compares each of the extracted key frame images with the sample data, and obtains a relative value according to similarity. A recognition score is calculated, a keyframe image having the highest recognition score is set as the representative image for the picking task, and the representative image is matched with the picking task information to be stored and managed.

In this feature, the picking management server provides to search for and monitor the picking section image, the key frame image, the representative image, and the picking job information in conjunction with an external logistics management system through a communication network.

According to another feature of the present invention, a processing method of a logistics picking monitoring system using image recognition based on artificial intelligence of the present invention is provided.

According to this feature, in the processing method of the logistics picking monitoring system using artificial intelligence-based image recognition of the present invention, the picking management server learns the sample image for a normal picking job in advance by artificial intelligence, and the worker information of the picking job and the picking Storing and managing product information; When packing goods, when a worker handling the picking task wears smart glasses and the camera of the smart glasses recognizes the worker's login card, the smart glasses are automatically connected to the picking management server through a communication network to perform worker authentication. The step of being logged in through; The smart glasses recognize an item at a position corresponding to the operator's line of sight, track the recognized item to acquire a video of a picking operation of one packing box, and obtain the video, the operator information, work time and Transmitting the picking operation information including the picking product information to the picking management server in real time through a communication network; The picking management server receives and stores the video transmitted from the smart glasses in real time, and recognizes the picking operation for one packing box of the corresponding item by section using image recognition technology through artificial intelligence learning from the video. separating and storing the video as a section peaking image; Recognizing and extracting a plurality of keyframe images from each of the separated section picking images by the picking management server, and setting and storing one of the keyframe images as a representative image indicating a state in which products have been picked. ; And the picking management server classifies and stores the video, the section picking video, the key frame image, and the representative image together with the picking job information by worker name, work time, and picking item, and processes to search and monitor the picking job. It includes steps to

In this feature, in the step of setting and storing the representative image, the picking management server calculates and stores a recognition score according to the similarity with the sample image for each of the keyframe images, An image with a high recognition score is set as the representative image and stored.

As described above, the logistics picking monitoring system of the present invention acquires and monitors images of the picking operation in real time using the camera of the smart glasses worn by the operator, so that work errors that may occur by the operator, that is, human errors It is possible to determine whether errors occur in the picking operation, manage the work ability of the operator, and the like.

In addition, the logistics picking monitoring system of the present invention, when a customer's claim occurs, can accurately identify and respond to the cause and responsibility of the claim, thereby preventing black consumers.

In addition, the logistics picking monitoring system of the present invention can quickly respond to customer claims by recognizing and analyzing images of picking operations using image recognition technology through artificial intelligence learning. Customer trust can be improved.

In addition, the logistics picking monitoring system of the present invention indexes image recognition and analysis by artificial intelligence learning by linking the picking work time, and manages the picking work through this, thereby securing an item recognition improvement model, and according to image processing System performance can be improved and efficient management is possible.

In addition, the logistics picking monitoring system of the present invention can be used for various logistics management, delivery management, and inventory management by providing desired information according to the picking operation in conjunction with various clients or logistics management systems.

1 is a block diagram showing the network configuration of a logistics picking monitoring system according to the present invention;
2 is a block diagram showing the configuration of the smart glasses shown in FIG. 1;
Figure 3 is a block diagram showing the configuration of the picking management server shown in Figure 1, and
Figure 4 is a flow chart showing the processing procedure of the logistics picking monitoring system according to the present invention.

Embodiments of the present invention may be modified in various forms, and the scope of the present invention should not be construed as being limited due to the examples described below. This embodiment is provided to more completely explain the present invention to those skilled in the art. Therefore, the shapes of components in the drawings are exaggerated to emphasize a clearer explanation.

In the logistics picking monitoring system of the present invention, each worker wears smart glasses during a logistics picking operation in various fields such as manufacturing, production, distribution, sales, and logistics, and uses a camera of the smart glasses to monitor the status of the worker in real time. Obtain a video of the picking operation. In the logistics picking monitoring system of the present invention, a picking management server receives a video obtained from smart glasses, recognizes and extracts an image to confirm a picking operation through artificial intelligence learning such as machine learning, and picks by worker, by work time, Classify by product, etc., and store and manage.

Therefore, according to the logistics picking monitoring system of the present invention, it is possible to manage errors in picking work, the work ability of workers, etc. by determining whether work errors that may occur by workers, that is, human errors, occur. When a claim occurs, the cause of the claim and responsibility can be accurately identified and dealt with, and through this, black consumers can be prevented.

Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS. 1 to 4 attached.

1 is a block diagram showing a network configuration of a logistics picking monitoring system according to the present invention.

Referring to FIG. 1, the logistics picking monitoring system 2 of the present invention is a camera (FIG. 2 of 110) to automatically recognize items and acquire a video about the picking operation in real time, and the acquired video and operator information, work date and time, and picking item information through the communication network (4) to the picking management server 200 transmit in real time.

Here, the picking operation is to put at least one product (or product, product) produced or manufactured in one packing box when packing logistics, or to put at least one item in one packing box according to the customer's order. means work. At this time, the picking operation may include an operation of putting the same items in the same packing box or putting different items in the same packing box or each of the different packing boxes.

In the logistics picking monitoring system 2 of the present invention, the picking management server 200 uses an artificial intelligence learning image recognition technology such as machine learning for a video transmitted from each of a plurality of smart glasses 100 to form one packing box. Recognizes the sections of the picking task for , separates, stores, and manages images of the picking task in units of packing boxes.

The logistics picking monitoring system 2 of the present invention recognizes and extracts an image in which all items have been picked, that is, an image in which a picking operation for one packing box has been completed, from stored images, and selects one of these images as a representative image of the picking operation. It is indexed and stored, and stores and manages stored videos, images, work date and time, worker information, and picking item information by classifying them by worker, work time, and picking item.

In addition, the logistics picking monitoring system 2 of the present invention is a logistics management system 300 connected to the picking management server 200 through the communication network 4, and provides various information and images according to the picking situation so that managers, customers, etc. It is provided so that a client (not shown) can monitor a picking situation or search for and check necessary information related to a picking job.

Therefore, the logistics picking monitoring system 2 of the present invention can be used to monitor and check the picking operation by searching for classified videos and images, and if a customer's claim occurs after delivery, it can be used as evidence for the claim.

To this end, the logistics picking monitoring system 2 of the present invention includes a plurality of smart glasses 100 and a picking management server 200 . The smart glasses 100 and the picking management server 200 are connected through a communication network 4 to enable mutual data communication. In addition, the logistics picking monitoring system 2 of the present invention may further include a logistics management system 300 connected to the picking management server 200 through a wired or wireless communication network.

Specifically, the communication network 4 is provided as, for example, a wired/wireless communication network, a mobile communication network, and the like. The communication network 4 connects a plurality of smart glasses 100 and the picking management server 200 to enable data communication through, for example, a wireless communication network or a mobile communication network, and data communication through, for example, a wired or wireless communication network. The picking management server 200 and the logistics management system 300 are connected to enable communication.

The smart glasses 100 is a wearable device in the form of glasses worn by each worker, and includes a camera ( 110 in FIG. 2 ) that automatically captures real-time images according to the gaze of the worker. During a picking operation, the smart glasses 100 automatically recognizes an item to be picked using the camera 110, tracks it, and captures a video of the picking operation. The smart glasses 100 transmit picking job information including worker information, work date and time, product information, etc. together with the captured video to the picking management server 200 through the communication network 4 .

Of course, such smart glasses 100 may be replaced with a CCTV camera having a camera, a webcam, and the like. In this case, a plurality of CCTV cameras, webcams, etc. are fixedly installed on one side of the work space in correspondence with each worker, and a video of the picking operation in the work space is taken and the video taken through the communication network 4 is recorded. Transmission to the picking management server (200).

The picking management server 200 is connected to each of the plurality of smart glasses 100 and the logistics management system 300 through the communication network 4 to enable mutual data communication. The picking management server 200 receives and stores videos and picking job information about the picking job from each of the plurality of smart glasses 100 through the communication network 4 . The picking management server 200 recognizes the stored video through artificial intelligence learning such as machine learning, separates and stores it into sections for the picking task of one packing box, and picks in the video of the separated section in progress or completion Working images are extracted and stored, and among the extracted images, an image for which the peaking work state can be confirmed, for example, an image having the highest similarity with sample data is set as a representative image. Here, the separated section is extracted at intervals of about 100 to 500 ms, for example, for efficient processing performance of the picking management server 200.

To this end, the picking management server 200 includes, for example, a web server that processes connections with the smart glasses 100, the logistics management system 300, and various clients (not shown), and the data transmitted from the smart glasses 100. It includes an HTTP server that processes video playback and video segmentation in real time. In this embodiment, the picking management server 200 is a programming library for real-time image processing on the HTTP server, for example, open source software OpenCV (Open source Computer Vision), an artificial learning program, that is, open source software for machine learning For example, Google's TensorFlow is provided.

The picking management server 200 processes artificial intelligence learning by using normal picking images of one packing box in advance as sample data. The picking management server 200 separates the video by section from the video transmitted from the smart glasses 100, extracts images from the separated video, and performs artificial intelligence learning using the entire working image as training data. The picking management server 200 determines a picking event occurrence time point using a programming library, and extracts a candidate image to be analyzed from the picking event occurrence time point. The picking management server 200 calculates a recognition score for each image extracted for each section through an artificial learning program. At this time, the recognition score is calculated as a relative value according to similarity compared to sample data, and an image having the highest recognition score among images is set as a representative image for the picking task. The representative image is stored and managed by matching worker information with working time and picking work information for picking product information.

The picking management server 200 classifies and indexes images transmitted from the smart glasses 100 by worker, working time, and processed article. Since it is inefficient to generate keyframe images for all sections of the picking operation, in order to improve indexing efficiency linked to the picking operation time, when a picking event occurs by recognizing the barcode of an item, candidate images to be analyzed for the occurrence of the picking event are generated. Based on the event occurrence point, a plurality of key frame images are extracted and analyzed within a unit time before and after. At this time, the unit time is sufficient time to extract an appropriate image, and is set to be changeable as a system environment value for each product field. The initial default is set to eg 3 to 5 seconds.

And the logistics management system 300 is provided, for example, a manufacturing plant, a logistics company, a distribution company, a sales company, etc. and is connected to the picking management server 200 through a communication network. In addition, the logistics management system 300 is connected to clients (not shown) such as, for example, a manager terminal, a worker terminal, and a customer terminal through a communication network. The logistics management system 300 is a system for providing various services to customers by efficiently managing management resources such as workers and facilities to support management of a warehouse or delivery center related to logistics, for example, a warehouse management system (Warehouse Management System (WMS), Order Management System (OMS), Transport Management System (TMS), or Data Acquisition System (DAS).

This logistics management system 300 interworks with the picking management server 200 to provide information about the picking task by clients, for example, worker access status, tasks for picking tasks such as picking task images by worker and time slot. It is provided so that the current status can be monitored in real time or as needed, or that managers and workers can search and check desired information.

Therefore, the logistics picking monitoring system 2 of the present invention acquires an image of the picking operation in real time when picking the logistics using the camera 110 of the smart glasses 100 worn by the worker, analyzes and extracts the image, It is determined whether a picking event situation occurs through artificial intelligence learning such as running. Therefore, the logistics picking monitoring system 2 of the present invention can be used as evidence in response to claims that may occur in the future using the images and the picking job information for the picking job.

Specifically, the configuration and functions of the logistics picking monitoring system according to the present invention will be described in detail.

FIG. 2 is a block diagram illustrating the configuration of the smart glasses shown in FIG. 1 .

Referring to FIG. 2, the smart glasses 100 of the present invention are connected to the picking management server 200 through the communication network 4, and use the camera 110 to log in the card 102 equipped with identification information of the operator. By recognizing the worker information is obtained and transmitted to the picking management server 200. Here, the login card 102 stores or has identification information (eg, a worker name, a worker ID, a worker code, etc.) of a worker capable of identifying the worker provided for each worker. Through this process, the smart glasses 100 are automatically connected to the picking management server 200, logged in, and activated to take a video according to the picking operation.

The smart glasses 100 recognizes and tracks the picking item 104 located on the operator's line of sight through the camera 110 during the picking operation, and captures and obtains a video of the picking operation. At this time, the camera 110 acquires a video by recognizing an article by artificial intelligence learning and recognizing and tracking the article according to a picking operation.

Accordingly, when an item is photographed by the camera 100, the smart glasses 100 may obtain a relatively higher recognition score than when the item is not present. To this end, the smart glasses 100 sets an area for recognizing articles and packaging boxes, and when a recognition score of a certain score or higher occurs, sets the corresponding image as a keyframe image, and represents the keyframe image with the maximum recognition score among them. Register as an image.

Specifically, the smart glasses 100 of the present invention include a camera 110, a communication unit 120, a controller 130, and a storage unit 140. In addition, although the smart glasses 100 are not shown in the drawings, typical components of smart glasses include, for example, an input unit for inputting commands, a head-mounted display for outputting virtual reality images, etc., a rechargeable battery for supplying power, and A wired interface for data communication with an external electronic device through a cable may be further included.

The camera 110 is provided with, for example, a high-resolution artificial intelligence camera capable of capturing a video of 720 pixels of 5 million pixels, and recognizes the login card 102 to activate a picking operation video recording function and automatically communicates the communication unit 120 ) Accesses the picking management server 200 through and logs in by transmitting worker information. During the picking operation, the camera 110 automatically recognizes and tracks an item to be picked and a packing box through a barcode or the like along the operator's line of sight, thereby acquiring a high-resolution video of the picking operation. To this end, the camera 110 can recognize and track accurate picking items and packing boxes by iteratively learning videos acquired using, for example, an artificial intelligence algorithm through sample data. The camera 110 transmits the obtained video and the working time of the picking job to the picking management server 200 through the communication unit 120 in real time.

The communication unit 120 is provided as a wireless communication module such as, for example, LTE, 5G, Wi-Fi, and Bluetooth, and provides picking work information including video obtained in real time from the camera 110, worker information, work time, and picking items. It is transmitted to the picking management server 200 through the communication network 4.

The storage unit 140 includes a main memory and a storage memory, and stores, for example, an operating system program and an application for a picking job, and temporarily stores picking job information and videos obtained according to a process of processing the application for a picking job.

Also, the controller 130 controls to process general operations of the smart glasses 100 . That is, the controller 130 controls functions of the camera 110, the communication unit 120, and the storage unit 140 to be organically processed.

Figure 3 is a block diagram showing the configuration of the picking management server shown in Figure 1;

Referring to FIG. 3 , the picking management server 200 of the present invention processes login authentication when worker information is transmitted from the smart glasses 100 recognizing the login card 102 . The picking management server 200 receives real-time transmission of a video and picking job information about a picking job from each of the smart glasses 100 through the communication network 4 . The picking management server 200 recognizes and extracts images for a picking task through artificial intelligence learning in advance, and stores and manages the extracted images by matching them with the picking task information.

The picking management server 200 processes worker management, task management, log management, authority management, and the like for the plurality of smart glasses 100 . Worker management registers and manages, for example, worker names and identification information of workers wearing the smart glasses 100 . Job management codes and manages product information for picking jobs. The log management determines the worker by recognizing the identification information of the worker, and stores and manages the contact time with the smart glasses 100 and the image of the picking task by time by matching each other. In addition, the authority management sets and manages the authority granted to search and view various data such as videos, images, and picking work information according to the operator or manager, or monitors the operator's access status and analysis processing status in real time.

Specifically, the picking management server 200 includes a server unit 210 and a database unit 250. The server unit 210, for example, processes a streaming server for real-time streaming of a video transmitted from the smart glasses 100, receives the video from the streaming server, and monitors a picking operation using artificial intelligence learning. , and a web server that processes clients so that they can communicate with each other.

In this embodiment, the server unit 210 includes a communication unit 212, a control unit 214, a worker management unit 216, an artificial intelligence learning unit 218, a picking monitoring unit 220 and an event determination unit 222. .

The communication unit 212 is provided as, for example, a wired/wireless communication device, and interacts with a plurality of smart glasses 100, the logistics management system 300, and a plurality of clients (not shown) through the communication network 4. It handles data communication. The communication unit 212 may further include various communication devices for data communication between a typical web server and a plurality of clients, for example, a communication module, a hub, a switch, a firewall, and a computer program such as a communication algorithm.

The worker management unit 216 stores and manages worker information including a worker name, identification information, login information, etc. for each worker of the smart glasses 100 . The worker management unit 216 sets authority for each of the smart glasses 100 .

The artificial intelligence learning unit 218 repeatedly processes artificial intelligence learning using normal picking images for a picking task as sample data in advance. The artificial intelligence learning unit 218 recognizes a section for each packing box for the picking operation in the video transmitted from the smart glasses 100, separates the video for each section, and displays images for confirming the picking operation in the video separated for each section. Recognize and extract. The artificial intelligence learning unit 218 repeatedly processes artificial intelligence learning using normal picking images as sample data using software for machine learning. The artificial intelligence learning unit 218 recognizes an item in the video transmitted from the smart glasses 100, recognizes and separates a video for each section of one packaging box, and uses the entire working image of the separated video for each section as learning data. to learn artificial intelligence. The artificial intelligence learning unit 218 determines the timing of the picking event by artificial intelligence learning in the separated videos for each section, and recognizes and extracts a plurality of keyframe images that are candidates for analysis from the timing of the picking event. The artificial intelligence learning unit 218 calculates a recognition score for each of the key frame images according to the similarity with the sample data. The artificial intelligence learning unit 216 sets an image having the highest recognition score among key frame images as a representative image for the picking operation of the corresponding product. At this time, the representative image is stored and managed by matching worker information with working time and picking work information for picking items.

The picking monitoring unit 220 displays to monitor videos transmitted from each of the smart glasses 100 and picking job information about picking jobs of each worker. The picking monitoring unit 220 stores and manages each of the transmitted videos, and searches and checks them by worker, working time, and picking item. The picking monitoring unit 220 outputs to monitor worker access status, analysis processing status, and the like by worker, working time, and picking item. When necessary information is retrieved and requested from the logistics management system 300 through the communication network 4, the picking monitoring unit 220 provides corresponding videos, images, and picking work information by worker, work time, and picking item. do.

When information related to a picking event is requested from a client such as a worker or manager or from the logistics management system 300, the event determination unit 222 determines the time of the corresponding picking event, extracts the corresponding keyframe image corresponding to the time of determination, and Videos, images, worker information, working hours, and picking items are provided so that you can check them.

The control unit 214 controls to process all operations of the picking management server 200 . That is, the control unit 214 controls the functions of the communication unit 212, the worker management unit 216, the artificial intelligence learning unit 218, the picking monitoring unit 220, and the event determination unit 222 to be organically processed. When connected from the smart glasses 100, the control unit 214 authenticates the worker, receives a moving picture according to the picking operation from the smart glasses 100 in real time, and uses image recognition technology based on artificial intelligence learning for each worker. , It controls to process image processing, classification and discrimination, monitoring, etc. for picking work by working time, by picking item, etc. In addition, the control unit 214 receives a request for information on a picking operation from an external electronic device, for example, a client or the logistics management system 300, extracts and determines the information, and controls the information to be provided to the outside through the communication network 4. do.

In addition, the database unit 250 stores and manages the picking video 254 transmitted from the smart glasses 100, the picking job information 262, and various information according to the control process of the controller 214. That is, the database unit 250 stores at least worker information 252, picking video 254, picking section video 256, key frame image 258, indexing information 260, and picking job information 262. .

In this embodiment, the database unit 250 is included in the picking management server 200, but may be provided as an independent database server. In the present invention, although not shown in the drawing, worker information 252, indexing information 260, and picking work information 262 are stored and managed in the database unit 250, and a video 254 and a picking section image 256 and key frame images 258 are stored and managed in a file storage server (not shown).

As described above, in the logistics picking monitoring system 2 of the present invention, each worker wears the smart glasses 100 during a logistics picking operation in various fields such as manufacturing, production, distribution, sales, and logistics. Using the camera 110 of the smart glasses 100, a video of a worker's picking operation is obtained in real time. In the logistics picking monitoring system 100 of the present invention, the picking management server 200 receives a video obtained from the smart glasses 100 and recognizes and extracts an image to confirm a picking operation through artificial intelligence learning such as machine learning. and store and manage by classifying by worker, working hour, and picking item.

And Figure 4 is a flow chart showing the processing procedure of the logistics picking monitoring system according to the present invention.

Referring to FIG. 4, in the logistics picking monitoring system 100 of the present invention, first, in step S400, the picking management server 200 artificially learns a sample image for a normal picking operation. At this time, the picking management server 200 stores and manages worker information and picking product information.

When the worker handling the picking task wears the smart glasses 100 in step S402 and recognizes the worker's login card 102 through the camera 110, the smart glasses 100 picks the picking management server 200 in step S404. ) is automatically accessed and logged in through operator authentication. In step S406, the smart glasses 100 recognizes an item at a position corresponding to the operator's line of sight, tracks the recognized item, and acquires a moving image of a picking operation of one packing box. At this time, the product is provided with product identification information such as a barcode or QR code, and the camera 110 recognizes the product by recognizing the product identification information. In step S408, the smart glasses 100 transmits the acquired video to the picking management server 200 in real time through the communication network 4.

In step S410, the picking management server 200 receives and stores the video transmitted from the smart glasses 100 in real time, and uses image recognition technology through artificial intelligence learning from the video transmitted in step S412 to determine one of the corresponding items. Recognizes the picking work for the packing box by section, separates and stores the video.

In step S414, the picking management server 200 recognizes, extracts, and stores a plurality of keyframe images from each of the separated videos for each section. At this time, each of the key frame images calculates a recognition score according to the similarity with the sample image in step S416, and through this, the image is indexed and stored. An image having the highest recognition score among the indexed keyframe images in step S418, that is, an image in a state in which the product has been picked is set as a representative image and stored.
In step S420, the picking management server 200 classifies and stores the stored videos, videos for each section, key frame images, and representative images by work date and time, work time, worker name, and picking item.

delete

Subsequently, when the picking management server 200 receives a request for information related to the picking operation from the logistics management system 300 in step S422, the information is searched and monitored in step S424. At this time, when the picking management server 200 receives a request for information related to a picking operation by clients (not shown) of managers or workers, it may process the information to be searched and monitored. Accordingly, in step S426, the logistics management system 300 retrieves and monitors information related to the requested picking operation from the picking management server 200.

In the above, the configuration and operation of the logistics picking monitoring system using artificial intelligence-based image recognition according to the present invention has been shown in detail and in accordance with the drawings, but this is only described by way of example, and does not deviate from the technical spirit of the present invention. Various changes and changes are possible within the scope not specified.

2: Logistics Picking Monitoring System
4 : communication network
100: smart glasses
110: camera
200: picking management server
210: server unit
250: database unit
300: Logistics management system

Claims (5)

In the logistics picking monitoring system using artificial intelligence-based image recognition:
The camera is used to recognize the login card equipped with the operator's identification information and transmit the operator information, and when packing the logistics, the camera recognizes and tracks the item located in the operator's line of sight to obtain a video of the picking operation , smart glasses for transmitting the obtained video in real time; and
Artificial intelligence learning is performed using a sample image representing a normal state of a packing box picking operation in advance, and when worker information is transmitted from the smart glasses through a communication network, the worker is authenticated, and the video is displayed from the smart glasses. And, receiving the worker information, the picking job information including the working time for the picking job and the picking product information, recognizing each picking section for one packing box from the video and separating them into a picking section image, Recognizes the items picked in the picking section image, extracts and stores a plurality of keyframe images, sets the image having the highest similarity to the sample image among the keyframe images as a representative image, and picks a single packing box. Information is classified and indexed at least by worker, work time, and picking item, and the picking event occurrence time is determined to determine the picking event occurrence time, the picking section video, the keyframe image, the representative image, and the picking Including; picking management server that provides to monitor work information;
The picking management server,
The keyframe image of the analysis target candidate is extracted from the peaking event occurrence point, each extracted keyframe image is compared with the sample image, and a recognition score is calculated with a relative value according to similarity, and the highest recognition score is obtained. setting a keyframe image as the representative image for the picking job, matching the representative image with the picking job information, storing and managing;
Logistics picking using artificial intelligence-based image recognition, characterized in that the picking section image, the key frame image, the representative image, and the picking operation information are provided to search and monitor in conjunction with an external logistics management system through a communication network. surveillance system.
delete delete In the processing method of the logistics picking monitoring system using artificial intelligence-based image recognition:
The picking management server artificial intelligence learning sample images for normal picking work in advance, and storing and managing worker information and picking product information of the picking work;
When packing goods, when a worker handling the picking task wears smart glasses and the camera of the smart glasses recognizes the worker's login card, the smart glasses are automatically connected to the picking management server through a communication network to perform worker authentication. The step of being logged in through;
The smart glasses recognize an item at a position corresponding to the operator's line of sight, track the recognized item to acquire a video of a picking operation of one packing box, and obtain the video, the operator information, work time and Transmitting the picking operation information including the picking product information to the picking management server in real time through a communication network;
The picking management server receives and stores the video transmitted from the smart glasses in real time, and recognizes the picking operation for one packing box of the corresponding item by section using image recognition technology through artificial intelligence learning from the video. separating and storing the video as a section peaking image;
Recognizing and extracting a plurality of keyframe images from each of the separated section picking images by the picking management server, and setting and storing one of the keyframe images as a representative image indicating a state in which products have been picked. ; and
The picking management server classifies and stores the video, the section picking image, the key frame image, and the representative image together with the picking work information by worker name, work time, and picking item to search for and monitor the picking work. Step; including,
The step of setting and saving the representative image,
The picking management server calculates and stores a recognition score according to similarity with the sample image for each of the keyframe images, and sets and stores an image having the highest recognition score among the keyframe images as the representative image. A processing method of a logistics picking monitoring system using artificial intelligence-based image recognition. delete

Images