KR102671443B1 - Method for fulfillment packing through image processing of invoice and server thereof - Google Patents

Abstract

One embodiment is a fulfillment packing method through image processing of an invoice by a server connected to a packing video recording device and a shipping video recording device through a wired or wireless network, in which a packing video including the process of packing a product with an attached invoice is received. steps; Specifying the invoice from the packing image; Extracting a first valid value (invoice number) from the invoice specified from the packing image; Receiving a shipping video including a process of shipping the product; Specifying the invoice from the shipping video; extracting a second valid value (invoice number) from the invoice specified from the shipping video; And it is possible to provide a fulfillment packing method through image processing of an invoice, including the step of inspecting the shipment of the product by comparing the first and second effective values.

Description

Fulfillment packing method and server through image processing of invoice {METHOD FOR FULFILLMENT PACKING THROUGH IMAGE PROCESSING OF INVOICE AND SERVER THEREOF}

This embodiment relates to a fulfillment packing service for products.

Fulfillment service is a 'logistics wholesale agency service' in which a logistics specialist receives consignment from companies wishing to sell products and takes charge of all processes such as delivery, storage, packaging, delivery, inventory management, and exchange/refund services. A logistics company selects products according to the order, packages them, and completes delivery on behalf of the seller. In other words, the entire process from when the ordered product passes through the warehouse to being delivered to the customer (warehousing, storage, product selection, packaging, delivery, exchange/refund service provision, etc.) is handled in one package.

A representative example is the FBA (fulfillment by Amazon) service provided by Amazon, the world's largest e-commerce company, which first introduced the concept of fulfillment service in 1999. In Korea, Coupang introduced this service when it launched its rocket delivery service in 2014, and eBay Korea also provides this service.

However, for fulfillment services, logistics companies must handle products in large quantities. Logistics companies must classify and store numerous items and ship products appropriate to the relevant consignee, but there are several problems in this process. First, because we handle products in large quantities, there may be cases where products are missing during shipping. If a systematic management system does not exist, the number of omissions will increase. In addition, this is not a problem that only occurs in fulfillment services, but consignees cannot know how their products are received, stored, packed, and shipped. In particular, the consignee is not aware of the packing and shipping process, so he or she cannot respond appropriately if there is a problem with product delivery.

Accordingly, the inventor of the present invention has completed the present invention after a long period of research and trial and error on a fulfillment packing service that can solve the above-mentioned problems.

Against this background, one purpose of this embodiment is to provide a fulfillment service that inspects the shipment of a product using the effective value obtained from the packing image and the effective value obtained from the shipping image.

In order to achieve the above-mentioned object, in one embodiment, in a fulfillment packing method through image processing of an invoice by a server connected to a packing video recording device and a shipping video recording device and a wired or wireless network, the product to which the invoice is attached is packed. Receiving a packing image including the process; Specifying the invoice from the packing image; Extracting a first valid value (invoice number) from the invoice specified from the packing image; Receiving a shipping video including a process of shipping the product; Specifying the invoice from the shipping video; extracting a second valid value (invoice number) from the invoice specified from the shipping video; And it provides a fulfillment packing method through image processing of an invoice, including the step of inspecting the shipment of the product by comparing the first and second effective values.

In the method, obtaining an invoice form from a courier company; and extracting and storing the effective value format of the invoice form, wherein the step of extracting the first effective value format and the step of extracting the second effective value format include using the effective value format of the invoice form. You can.

In the method, the packing video capture device may include a webcam, and the shipping video capture device may include a global shutter type camera.

Another embodiment includes a communication unit that receives a packing video including a process of packing a product with an attached invoice and a shipping video including a process of shipping the product; and specifying the invoice from the packing image, extracting a first effective value (invoice number) from the invoice specified from the packing image, specifying the invoice from the shipping image, and extracting a first valid value (invoice number) from the invoice specified from the shipping image. 2. A fulfillment packing server is provided through image processing of an invoice, including a control unit that extracts a valid value (invoice number) and inspects shipment of the product by comparing the first and second valid values.

In the server, the packing video and the shipping video are stored in a database connected to the server and a wired or wireless network through the communication unit, and the stored packing video or the stored shipping video includes the effective value of the invoice and the product. It can be printed from the consignee terminal along with the time and place of packing.

As described above, according to this embodiment, it is possible to provide the customer with a live packing video captured through an image capture device (camera) without a separate scanner.

In addition, according to this embodiment, the inefficiency of manual review can be eliminated by automating the inspection of shipments in the warehouse.

And, according to this embodiment, it is possible to actively respond to accidents that occur during delivery.

And, according to this embodiment, trust in the product to be delivered can be gained between the sender and the consignee by providing a delivery image of the product.

Figure 1 is a configuration diagram of a fulfillment packing system through image processing of an invoice according to an embodiment.
Figure 2 is an exemplary diagram of a packing process among the fulfillment packing method through image processing of an invoice according to an embodiment.
Figure 3 is an exemplary diagram of a shipping process among the fulfillment packing method through image processing of an invoice according to an embodiment.
Figure 4 is a configuration diagram of a fulfillment packing server through image processing of an invoice according to an embodiment.
Figure 5 is a flowchart for explaining the operation of a fulfillment packing server through image processing of an invoice according to an embodiment.
Figure 6 is a flowchart for explaining the operation of a fulfillment packing system through image processing of an invoice according to an embodiment.

In describing the present invention, if it is determined that related known functions may unnecessarily obscure the gist of the present invention as they are obvious to those skilled in the art, the detailed description thereof will be omitted.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, identical or corresponding components will be assigned the same drawing numbers and overlapping descriptions thereof will be omitted. Do this.

Figure 1 is a configuration diagram of a fulfillment packing system through image processing of an invoice according to an embodiment.

Referring to Figure 1, the fulfillment packing system (100, hereinafter referred to as 'system') includes a packing video capture device 111, a shipping video capture device 112, a server 120, a database 130, and a consignee terminal. It may include (140) and the consignor terminal (150).

The packing video recording device 111 can generate a packing video by capturing a video of a product being packed at a delivery company. An invoice is attached to the product, and the packing video recording device 111 can capture an image so that the invoice is included.

The invoice may contain valid values such as the invoice number or the barcode of the invoice number. Additionally, the invoice may include personal information about the consignee who will receive the product - for example, name, address, contact information, etc. - or physical information about the product - the name of the product, precautions for handling, etc. Here, the effective value of the invoice may have a certain format, which may be named the valid value format. The valid value format may include the composition, location, size, notation, etc. of the invoice. For example, the organization of an invoice may include the format or arrangement of valid values, personal information, or physical information. The location of the invoice may refer to the location on the outside of the product where the invoice is attached.

The shipping video recording device 112 can capture a video of a product being shipped from a delivery company and generate a shipping video. First, the product is packed in the packing area, then transported to the shipping area through a conveyor belt and then loaded onto a delivery vehicle. The shipping video recording device 112 can film the shipping process of such products.

When a product is located in the packing area, the packing video recording device 111 can film the packing process of the product. The server 120 may receive a packing image from the packing image recording device 111. The server 120 may perform image processing on the packing image and specify an invoice from the packing image. The server 120 may extract the first valid value from the invoice specified from the packing image. An invoice may be recognized from the image-processed packing image, and the first valid value may be a valid value extracted from this invoice. The first valid value has the form of an invoice number or a barcode of the invoice number, but may not be the valid value of the actual invoice but the valid value of the invoice extracted from the image. Therefore, the first effective value may be the same as or different from the effective value of the actual invoice.

Additionally, the server 120 can obtain and secure invoice forms from all courier companies. The server 120 can identify and classify invoice forms for each delivery company through artificial intelligence learning. For example, server 120 can identify the format - location and notation - of valid values of an invoice form. The server 120 can learn the format of the valid values of the invoice form using artificial intelligence and classify the format of the valid values of the invoice form by courier company.

Therefore, the server 120 can use the format of the valid value to extract the first valid value. For example, the server 120 can obtain and secure invoice forms from all courier companies in advance. The server 120 can extract and store the valid value format of all courier companies' invoice forms. The server 120 can extract an invoice from the packing video and obtain a valid value format - a valid value format of the first effective value - from the extracted invoice. The server 120 may compare the valid value format of the first valid value with the valid value format of all previously stored courier company invoice forms. If both valid value formats match, the server 120 recognizes the first valid value. Otherwise, the server 120 may ignore the first valid value and extract the first valid value again.

When a product is located in the shipping area, the shipping video recording device 112 can film the shipping process of the product. The server 120 may receive a factory video from the factory video recording device 112. The server 120 can perform image processing on the shipment video and specify an invoice from the shipment video. The server 120 may extract the second valid value from the invoice specified from the shipping video. The invoice may be recognized from the image-processed shipping image, and the second valid value may be a valid value extracted from this invoice. The second valid value has the form of an invoice number or a barcode of the invoice number, but may not be the valid value of the actual invoice but the valid value of the invoice extracted from the image. Therefore, the second effective value may have a difference from the effective value of the actual invoice. Additionally, since the second effective value is extracted from the shipping image, it may be the same as or different from the first effective value extracted from the packing image.

And the server 120 can inspect the shipment of the product using the first effective value and the second effective value. For example, the server 120 may determine that the packing and shipping of the product are normal if the first and second valid values are the same. Alternatively, the server 120 may count the first valid values of the plurality of products and the second valid values of the plurality of products, and if the results are the same, it may be determined that the packing and shipping of the plurality of products are normal. . This can be understood as either the packed product being delivered as is or the packed product not being missing.

Meanwhile, data and information may be stored in the database 130. For example, the database 130 may be a cloud storage. The packing video and/or the shipping video may be directly transmitted from the photographing devices 111 and 112 and stored in the database 130, or may be stored in the database 130 through the server 120. The packing video and shipping video have the valid value of the product and the packing/shipping time zone as the file name, and can be stored in the database 130.

The consignee terminal 140 may be a terminal operated by the consignee, that is, the customer receiving the product. The consignee terminal 140 can receive a packing video and/or a shipping video, and check the packing process and shipping process of the product to be received in real time.

The consignor terminal 150 can receive the results of the product delivery inspection. The server 120 can use the first and second valid values to check whether there is an omission in the shipment of the product and generate a shipment inspection result for this. The server 120 can transmit the results of this shipment inspection to the consignor terminal 150. In addition, the consignor terminal 150 can receive a packing video and/or a shipping video and check the packing process and shipping process of the product scheduled for shipping in real time.

The packing video recording device 111, the shipping video recording device 112, the server 120, the database 130, the consignee terminal 140, and the consignor terminal 150 can be connected and communicate through a wired or wireless network. A wired or wireless network refers to a connection structure that allows information to be exchanged between nodes such as terminals and servers. Examples of such networks include the Internet, LAN (local area network), and wireless LAN (local area network). , WAN (wide area network), PAN (personal area network), 3G, 4G, 5G, Wi-Fi, Bluetooth, NFC, RFID, etc., but are not limited to these. Here, the consignee terminal 140 and the consignee terminal 150 are electronic devices capable of data communication through a wired or wireless network and may include various devices such as smartphones, mobile phones, tablet PCs, desktops, and laptops. It is not limited to this.

Figure 2 is an exemplary diagram of a packing process among the fulfillment packing method through image processing of an invoice according to an embodiment.

Referring to FIG. 2, a packing process may be shown among the fulfillment packing method through image processing of an invoice according to an embodiment. In the fulfillment packing of a product, the product packing process can be filmed by the packing video recording device 111.

When the product moves and reaches the packing area, the packing video recording device 111 can film the process of the product being packed by the worker (see Packing). The packing image recording device 111 can generate a packing image and transmit it to the server 120. Here, the packing image immediately generated by the packing image recording device 111 may be named the original packing image 210 (see RAW). The server 120 may process the packing image to specify an invoice and extract a valid value from the packing image. Here, the packing image processed by the server 120 may be named the processed packing image 210' (see Processed). The server 120 may transmit the processed packing image 210' to the database 130 and store it. Then, if the consignee wants to view the packing process of his product, he can receive the processed packing video 210' from the database 130 through the consignee terminal 140. The consignee places an order while watching the processed packing video 210'. You can check whether the purchased product is properly packed.

Additionally, the packing video recording device 111 used in the packing process may include a web camera or webcam. Since the product does not move much and remains in a static state while it is being packed, a web camera can sufficiently film the product packing process and extract the invoice from the packing video through image processing. The web camera can be placed high up so that the entire product packing process is visible and the invoice is included in the packing video.

Figure 3 is an exemplary diagram of a shipping process among the fulfillment packing method through image processing of an invoice according to an embodiment.

Referring to FIG. 3, a shipping process among the fulfillment packing method through image processing of an invoice according to an embodiment may be shown. In the fulfillment packing of a product, the shipping process of the product can be filmed by the shipping video recording device 112.

When the product moves and reaches the shipping area, the shipping video recording device 112 can film the process of the product being shipped by the worker (see 'Movement after packing'). The shipping video recording device 112 can generate a shipping video and transmit it to the server 120. Here, the shipping video recording device 112 is arranged in plural numbers and can capture the process of the product moving to the 'loading waiting point'. The factory video recording device 112 can photograph multiple surfaces of the product. The shipping video recording device 112 can photograph three sides, and this is to specify the invoice by photographing the side to which the invoice is attached. The shipping video in which multiple sides are captured includes an invoice 201, and the shipping video can be transmitted to the server 120. The server 120 may process the shipment video to specify an invoice and extract a valid value from the shipment video. The server 120 can inspect the shipment of the product by comparing the valid value extracted from the shipping image and the valid value extracted from the packing image (see 'Packing data and cross-inspection'). The server 120 can determine whether the product is missing from shipment and transmit the inspection result to the consignor terminal 150 (see 'Inspection Result Notification').

Additionally, the shipping video recording device 112 used in the shipping process may include a global shutter type camera. Since the product remains in a dynamic state moving along the conveyor belt while it is being shipped, a global shutter camera may be appropriate. A global shutter-type camera can be formed in a structure suitable for photographing a moving subject.

Figure 4 is a configuration diagram of a fulfillment packing server through image processing of an invoice according to an embodiment.

Referring to FIG. 4, a configuration diagram of a fulfillment packing server through image processing of an invoice according to an embodiment is shown.

The server 120 may include a communication unit 121, a control unit 122, and a storage unit 123.

The communication unit 121 may include a module capable of both wired and wireless communication for exchanging data with a packing video recording device, a shipping video recording device, a database, a recipient terminal, and a transmitter terminal. The communication unit 121 can receive or transmit packing images and shipping images.

The control unit 122 may include at least one processor and memory and perform a fulfillment packing method through image processing of an invoice. Specifically, the control unit 122 specifies the invoice from the packing image, extracts a first valid value (invoice number) from the invoice specified from the packing image, specifies the invoice from the shipping image, and The second effective value (invoice number) is extracted from the specified invoice, and the shipment of the product can be inspected by comparing the first and second effective values.

The storage unit 123 may store data or information for performing a fulfillment packing method through image processing of an invoice.

Figure 5 is a flowchart for explaining the operation of a fulfillment packing server through image processing of an invoice according to an embodiment.

Referring to Figure 5, the operation of a server according to one embodiment is shown.

The server 120 may receive the packing image from the packing image recording device (step S501). The server 120 may perform image processing or image processing on the packing image to specify the invoice (step S502). The server 120 may extract the first valid value - the invoice number or the barcode of the invoice number - from the invoice specified from the packing image (step S503). The server 120 may receive the factory video from the factory video capture device (step S504). The server 120 can specify the invoice in the shipping video (step S505). The server 120 may extract the second valid value - the invoice number or the barcode of the invoice number - from the invoice specified from the shipping image (step S506). The server 120 may inspect the shipment through the first and second valid values (step S507).

Figure 6 is a flowchart for explaining the operation of a fulfillment packing system through image processing of an invoice according to an embodiment.

Referring to Figure 6, operation of the system according to one embodiment is shown.

The server 120 can obtain invoice forms from all courier companies (step S601). The server 120 can extract the valid value format of all invoice forms and store it internally (step S602).

The packing image recording device 111 can acquire a packing image (step S603). The packing video recording device 111 transmits the packing video to the server 120, and if there is a request from the consignee terminal 140, the server 120 can transmit the packing video to the consignee terminal 140 (step S604). . The server 120 may extract the first valid value from the packed image (step S605).

The shipping video recording device 112 can acquire a shipping video (step S606). The shipping video recording device 112 transmits the shipping video to the server 120, and if there is a request from the consignee terminal 140, the server 120 can transmit the shipping video to the consignee terminal 140 (step S607). . The server 120 may extract the second valid value from the factory image (step S608).

The server 120 may inspect the shipment using the first and second valid values (step S609). The server 120 may transmit the results of the delivery inspection to the consignor terminal 150. The results of the shipment inspection may include whether packed products were omitted from shipment.

The scope of protection of the present invention is not limited to the description and expression of the embodiments explicitly described above. In addition, it is added once again that the scope of protection of the present invention may not be limited due to changes or substitutions that are obvious in the technical field to which the present invention pertains.

111: Packing video recording device
112: Factory video recording device
120: server
130: database
140: Consignee terminal
150: Songhain terminal

Claims (5)

In the fulfillment packing method through image processing of the invoice by a server connected to a packing video recording device and a shipping video recording device and a wired or wireless network,
Receiving a packing video including the process of packing a product to which an invoice is attached;
Specifying the invoice from the packing image;
extracting a first valid value from the invoice specified from the packing image;
Receiving a shipping video including a process of shipping the product;
Specifying the invoice from the shipping video;
extracting a second effective value from the invoice specified from the shipping video; and
Comprising the step of inspecting the shipment of the product by comparing the first and second effective values.
Fulfillment packing method through image processing of invoices. According to paragraph 1 above,
Obtaining an invoice form from a courier company; and
Including the step of extracting and storing the valid value format of the invoice form,
The step of extracting the first effective value format and the step of extracting the second effective value format use the effective value format of the invoice form.
Fulfillment packing method through image processing of invoices. According to paragraph 1 above,
The packing video recording device includes a webcam,
The factory video recording device includes a global shutter type camera.
Fulfillment packing method through image processing of invoices. A communication unit that receives a packing video including a process of packing a product to which an invoice is attached and receives a shipping video including a process of shipping the product; and
Specify the invoice from the packing image, extract a first valid value from the invoice specified from the packing image, specify the invoice from the shipping image, and extract a second valid value from the invoice specified from the shipping image. and a control unit that compares the first and second valid values and inspects the shipment of the product.
Fulfillment packing server through image processing of invoices. According to clause 4 above,
The packing video and the shipping video are stored in a database connected to the server and a wired or wireless network through the communication unit,
The stored packing video or the stored shipping video is output from the consignee terminal along with the effective value of the invoice and the time and place at which the product is packed.
Fulfillment packing server through image processing of invoices.

Images