CN117541739B

CN117541739B - Warehouse map visual construction method and system based on OpenCV

Info

Publication number: CN117541739B
Application number: CN202410026035.6A
Authority: CN
Inventors: 马德鑫; 纪栋梁; 崔彪; 张丰琳; 季媛; 张明宇
Original assignee: Jinxiandai Information Industry Co ltd
Current assignee: Jinxiandai Information Industry Co ltd
Priority date: 2024-01-09
Filing date: 2024-01-09
Publication date: 2024-04-09
Anticipated expiration: 2044-01-09
Also published as: CN117541739A

Abstract

The disclosure provides a warehouse map visualization construction method and system based on OpenCV, which relate to the technical field of data warehouse and comprise the following steps: obtaining a live-action image in a warehouse and object standard data in the warehouse, constructing object models of the warehouse, a goods shelf and tools and instruments, and establishing an association relationship between the goods shelf and the tools and instruments; automatically matching an object model according to the live-action image in the warehouse, generating a warehouse map and visualizing; the method for automatically matching the object model according to the live-action image in the warehouse comprises the steps of shelf positioning and shelf type identification, wherein the method for positioning the shelf comprises the following steps: and acquiring background wall images of all the shelves in the warehouse, rapidly extracting features at the junction between all the walls of the warehouse by using an OpenCV SURF feature detection method, matching feature values by using FLANN, constructing association relations among all the walls, and realizing the positioning of the shelves.

Description

Warehouse map visual construction method and system based on OpenCV

Technical Field

The disclosure relates to the technical field of data warehouse, in particular to a warehouse map visualization construction method and system based on OpenCV.

Background

The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.

Warehouse management is also called warehouse management, and refers to effective control of activities such as receiving, dispatching, balance and the like of warehouse goods, and aims to ensure the integrity of the warehouse goods for enterprises, ensure the normal running of production and management activities, and on the basis, carry out classification record on the activity status of various goods, express the status of the warehouse goods in terms of quantity and quality in a clear chart mode, and comprehensively manage the situations such as the geographical position, departments, order attribution, warehouse dispersion degree and the like. Warehouse management is an important link in supply chain management, which is initially intended to eliminate all inefficient activities; reasonable and accurate warehousing activities can reduce the replacement and flowing of commodities, reduce the operation times, and are beneficial to reducing the warehousing operation cost by adopting mechanized and automatic warehousing operations.

Traditional warehouse management methods often rely on manpower, position of articles and identity recognition of staff, and often have the problems of untimely and inaccurate, which may lead to low management efficiency and even possibly occurrence of safety risks. The existing warehouse management method is endless, the method mostly needs to jump back and forth among a plurality of systems, and the following problems still exist:

1) The manual searching of the work task is troublesome in operation and time-consuming and labor-consuming;

2) When a new person or a warehouse is large, the position of the equipment cannot be accurately positioned, and the equipment is troublesome to get/return;

3) The states of the devices cannot be visually checked, and the states to be checked and the like need to be searched for each device independently.

Disclosure of Invention

In order to solve the problems, the present disclosure provides a warehouse map visualization construction method and system based on OpenCV, by modeling a warehouse area, performing rapid feature extraction on junctions between walls by a SURF feature detection method of OpenCV technology, performing feature value matching, establishing an association relationship between walls to map, displaying the map, and facilitating warehouse management.

According to some embodiments, the present disclosure employs the following technical solutions:

the visual warehouse map construction method based on OpenCV comprises the following steps:

obtaining a live-action image in a warehouse and object standard data in the warehouse, constructing object models of the warehouse, a goods shelf and tools and instruments, and establishing an association relationship between the goods shelf and the tools and instruments;

automatically matching an object model according to the live-action image in the warehouse, generating a warehouse map and visualizing the warehouse map, and acquiring tools and instruments according to the warehouse map by a user;

the method for automatically matching the object model according to the live-action image in the warehouse comprises the steps of shelf positioning and shelf type identification, wherein the method for positioning the shelf comprises the following steps: and acquiring background wall images of all the shelves in the warehouse, rapidly extracting features at the junction between all the walls of the warehouse by using an OpenCV SURF feature detection method, matching feature values by using FLANN, constructing association relations among all the walls, and realizing the positioning of the shelves.

warehouse map visualization construction system based on OpenCV includes:

the data acquisition module is used for acquiring live-action images in the warehouse and standard data of objects in the warehouse;

the basic module construction module is used for constructing object models of a warehouse, a goods shelf and tools and instruments and establishing association relations between the goods shelf and the tools and instruments;

the map generation module is used for automatically matching the object model according to the live-action image in the warehouse, generating a warehouse map and visualizing the warehouse map;

Compared with the prior art, the beneficial effects of the present disclosure are:

the present disclosure provides a visual construction method and system for warehouse map based on OpenCV, in which features are identified and extracted from live-action photos and 3D models in a warehouse through OpenCV technology, intelligent construction of the warehouse map is completed, quick feature extraction is performed on junctions between walls through SURF feature detection method of OpenCV technology, and feature value matching is performed by using FLANN, when the feature value matching reaches eighty percent, it is identified that two photos have association, namely, an association relationship is established between the walls, and in a task mode, an optimal line can be calculated according to the warehouse map and the system, so that a pickup route is optimized for a user, and the burden of the user is reduced. In addition, the important information of the tools and instruments in the warehouse can be visually checked on the map by one map, so that subsequent test, scrapping and other works can be conveniently arranged.

Drawings

The accompanying drawings, which are included to provide a further understanding of the disclosure, illustrate and explain the exemplary embodiments of the disclosure and together with the description serve to explain the disclosure, and do not constitute an undue limitation on the disclosure.

FIG. 1 is a flow chart of a task execution method according to an embodiment of the present disclosure;

Detailed Description

The disclosure is further described below with reference to the drawings and examples.

It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments in accordance with the present disclosure. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.

Example 1

An embodiment of the present disclosure provides a warehouse map visualization construction method based on OpenCV, including:

step one: obtaining a live-action image in a warehouse and object standard data in the warehouse, constructing object models of the warehouse, a goods shelf and tools and instruments, and establishing an association relationship between the goods shelf and the tools and instruments;

step two: automatically matching an object model according to the live-action image in the warehouse, generating a warehouse map and visualizing;

the method for automatically matching the object model according to the live-action image in the warehouse comprises the steps of shelf positioning and shelf type identification, wherein the method for positioning the shelf comprises the following steps: acquiring background wall images of all the shelves in the warehouse, rapidly extracting features at the junction between all the walls of the warehouse by using an OpenCV SURF feature detection method, matching feature values by using FLANN, constructing association relations among all the walls, and realizing the positioning of the shelves

As an example, the embodiment of the OpenCV-based warehouse map visualization construction method disclosed by the disclosure is as follows:

step 1: the method for obtaining the live-action image in the warehouse and the standard data of the object in the warehouse and constructing the object model of the warehouse, the goods shelf and the tools specifically comprises the following steps:

and (5) constructing a shelf model. There are 5 types of conventional shelves known at present, the sizes are uniform, the appearance and the style are different, and the design is mainly adapted according to the type of the safety tool. The goods shelf is respectively a ground wire goods shelf, a rod goods shelf, a conventional goods shelf, a glove shoe support frame and a ladder goods shelf. In the library, four standard live-action pictures of front, back, left and right are shot according to the conventional patterns of the five shelves, and a 3D model is manufactured by using the prior art.

And (5) constructing a tool model. According to common tools, shooting four standard live-action diagrams in front, back, left and right, and manufacturing a tool 3D model by using the prior art.

And establishing the association relation between the goods shelf and the tools. Namely, the glove shoe support frame and the glove shoe form an association relation, and when the article which does not belong to the goods shelf is placed on the goods shelf, the article can be warned and reminded.

Further, the acquired related data is communicated. Three types of data are involved: personnel data, work plan information, tool data. The personnel data comprises facial recognition information of personnel and tool information viewing rights; the task data comprises information such as a work plan, a work responsible person, a work implementation person and the like; the tool data, i.e., a history table of the tool, includes basic information such as manufacturer, warehouse-in time, and test information.

And 2) carrying out feature identification and extraction on the live-action photo and the 3D model in the warehouse through an OpenCV technology, and completing intelligent construction of a warehouse map. The concrete construction steps are as follows:

1) The method comprises the steps that a live-action image in a warehouse is obtained, the live-action image in the warehouse comprises a shelf image, a tool image and wall images of all shelf backgrounds, standard data of objects in the warehouse comprise personnel data, operation plan information, tool data and basic information of the warehouse, the basic information of the warehouse comprises a plan view of the warehouse, and the plan view of the warehouse is recorded with information of the actual length and width. Before drawing the map, the basic information of the warehouse needs to be stored, including the plan view of the warehouse, including the information of the area, the actual length and width dimensions and the like.

2) And (5) photographing and modeling. The photographer needs to stand at the center of the warehouse to shoot the pictures of all the walls of the warehouse, and the pictures need to contain all the contents in the shooting wall and part of the contents (not less than twenty percent) of a certain adjacent wall to acquire the wall images of the background of each goods shelf in the warehouse.

2. The method comprises the steps of automatically matching a model with a live-action picture, generating a map, and carrying out specific analysis on the picture by adopting an OpenCV technology, wherein the picture is totally divided into two parts, namely shelf location and shelf type identification. The method comprises the following steps:

1) And carrying out rapid feature extraction on the junction between the walls by using a SURF feature detection method of an OpenCV technology, carrying out feature value matching by using FLANN, and recognizing that two pictures are associated when the feature value matching reaches eighty percent, namely, establishing an association relationship between the AB walls. The specific implementation steps are as follows:

(1) The obtained photos are numbered, the photos with doors are numbered 1, and the other four photos are marked as numbers 2,3 and 4 at a time.

(2) And (3) making a vertical dividing line from the upper corner to the ground in the photo, dividing the dividing line into 1000 parts, and selecting a part related to the adjacent wall as a recognition main body. The top layer of the adjacent wall photo of photo 1 was compared starting from the top layer and proceeding sequentially with the top layer of photos 2,3, and 4.

(3) The feature matching method comprises the following steps:

(1) the required dependency library is imported.

(2) A FLANN matcher is created. To increase the detection speed, a match is trained before calling the match function. The training process is optimized using the flannbasedfacher to build an index tree for the descriptor, which will play a great role in matching large amounts of data (training with live-action images in the warehouse and standard data of objects in the warehouse).

(3) And loading the icon file to be queried. The gray scale image is used for loading and matching the features later.

(4) The search is traversed.

(5) Minimum number of matching points. And if the number of the feature points matched with the two pictures is larger than MIN_MATCH_COUNT, the picture searching can be considered to be on, otherwise, the picture searching is not carried out.

(6) And (5) detecting characteristic points. The key points and descriptors of the image are checked using SIFT algorithm.

(7) And (5) matching the characteristic points.

(8) Searching for the best match;

(9) calculating a homography matrix;

drawing a matching frame of the corresponding mark;

⑪ match lines are drawn.

(4) If the matching pair is 80% or more, the matching pair is identified as photo 1. And sequentially finding out photos associated with the photos 2,3 and 4 to sort, for example, associating the photos 1 with the photos 3, associating the photos 3 with the photos 2, associating the photos 2 with the photos 4, and associating the photos 4 with the photos 1.

2) In order to ensure the accuracy of identification, the SIFT algorithm is adopted to extract the characteristic value when the shelf is identified, and the characteristic value is required to be identified by dividing the shelf into two parts, namely the shelf and the tools. And (3) identifying the goods shelf, comparing the characteristic value identified by the algorithm with a standard live-action diagram in a standard library, wherein the identification degree reaches more than eighty percent, and successfully preparing the goods shelf, and marking the goods shelf as the same type.

The specific implementation steps are as follows:

(1) Identifying and acquiring a shelf inflection point according to an algorithm;

(2) According to the inflection points, connecting the inflection points in series to form a line to obtain a shelf framework;

(3) Comparing the shelf framework with an object model of the shelf, and identifying the type of the shelf; the shelf model is compared to the in-library model construction. If there are four layers of shelves, the shelf with no division in the middle is a conventional shelf.

3) Identification tool

To increase the accuracy of identifying the shelves, the identification of the tools on the shelves is supplemented, i.e., one hundred percent when the tools on the shelves match the identified shelf type. When the tools on the shelf are not matched with the type of the shelf, a message notification is sent to management personnel, and the management personnel can recognize whether the tools are in a wrong shelf recognition or misplaced position. The method comprises the following specific steps:

(1) After the type of the goods shelf is identified, the system captures a sample photo of the tools and instruments related to the type of the goods shelf in the library, and compares the sample photo with the actual goods on the goods shelf through a SIFT algorithm.

(2) The feature matching specifically comprises the following steps:

(1) and (3) detecting a degree space extremum: image locations on all scales are searched. Potential keypoints that are invariant to scale and rotation are identified by gaussian difference functions.

(2) Positioning key points: at each candidate location, the location and scale is determined by a fitting fine model. The choice of key points depends on their degree of stability.

(3) And (3) determining the key point direction: one or more directions are assigned to each keypoint location based on the direction of the gradient of the image part. All subsequent operations on the image data are transformed with respect to the direction, scale and position of the keypoints, thus guaranteeing invariance to these transformations.

(4) Key point description: gradients of the image portions are measured at selected scales within a neighborhood around each keypoint. These gradients act as descriptors of key points that allow for deformation or illumination variation of relatively large local shapes.

(3) After the identification value reaches 80%, matching is completed.

After the goods shelf identification is completed, the goods shelf is placed on a map in sequence by taking the position of the door as a base point and taking the warehouse plan view as a map range. Taking an A shelf as an example, the actual height of the A shelf is h, the width is w, and the length is l. The height in the photograph is h 'and the horizontal distance from the wall where the door is located is m'. The specific placement position on the map is the percentage of the scale m '/(h'/h) from the horizontal position of the wall where the door is located. After one side of the shelf on the map is determined, the rest of the shelf is placed.

After the goods shelf is discharged, clicking a corresponding goods shelf, screening the types of the tools and instruments which can be placed according to the types of the goods shelf, and placing the RFID of the screened tools and instruments on the goods shelf by a user to finish placing the tools and instruments.

And finishing the construction of the map according to the model.

As an embodiment, when a tool on a shelf is taken according to a task mode issued after a map is constructed, a specific implementation method is as follows:

when a user gets a tool, an optimal route for the tool can be calculated according to a work task, and a common route getting method is optimized in consideration of the fact that the safety tool is a small tool. The specific calculation steps are as follows:

at the time of map construction, linear distances ma, mb, mc between each shelf and the center point of the door, and linear distances nab (distance between AB shelves), nac (distance between AC shelves) between each shelf are calculated.

When a user enters a warehouse, the latest work task of the user is identified according to the user identity, and the tools and the quantity required to be picked up are inquired.

When tools and instruments which are difficult to be taken out, such as a ladder, a grounding wire and the like, are contained in the tools and instruments, the shelf for storing the tools and instruments is identified as the last to be taken out, and other tools and instruments are recommended according to the following two algorithms.

Taking A, B, C shelves in the work task, the required quantity of each shelf is Na, nb, nc in turn, and the weights are Ma, mb and Mc in turn as examples

When the number of tools to be picked up is less than or equal to 10, recognizing the tools as a lightweight task, calculating according to the distance of the shelf where the equipment type is located, and traversing the shortest distance; when the number of tools to be picked up is larger than 10, the weight is identified as a weight-scale task, and the weight of the tools is needed to be added as a weight value when the distance is calculated.

Illustrating: if the operating rod, the insulating glove and the safety helmet are required to be taken, three tools are one for each type and are respectively placed on three shelves of ABC. If the total number is less than 10, a first algorithm is adopted, examples of ABC, ACB, BCA, BAC, CAB, CBA are traversed in sequence, and finally the shortest distance is calculated as the optimal recommended distance. If the three types of tools are still used, but the total number of the operating rod, the insulating glove, the safety helmet and the safety helmet is more than 10, the operating rod, the insulating glove and the safety helmet is 2, and a second algorithm is adopted. Taking line ABC as an example, the relative distance values of ABC are Na (nab+nbc+mc) +nb (Mb) +nc (Mc). The shortest distance among the several routes of the route ABC, ACB, BAC, BCA, CAB, CBA is the recommended route.

As one embodiment, the constructed map is visualized, and the map comprises three map modes, a general worker can only check the task mode, and when the task mode is opened each time, the latest task is automatically called according to the identity of the worker; the manager can check the scrapping and test modes, and update the tool states at six points in the morning every day.

1) And (3) scrapping mode: the system adopts a time series data analysis mode, can find out the general trend and rule of scrapping the tools and instruments, and mainly aims at tools and instruments with fixed scrapping time, fixed test times and unqualified test by analyzing parameters such as the type, quality guarantee period, production date, test times, test report and the like of the tools and instruments. And predicting the scrapping time of the scrapped tools and instruments according to the scrapping tool information, and carrying out early warning on a warehouse map model.

2) Test mode: the system collects tool information via RFID. The test period and the test frequency are predicted for different kinds of tools and the corresponding tools test date and the use times. And outputting three types of early warning of test conditions, namely, the conditions exceeding the model prediction test time, within 7 days of the model prediction test time and outside 7 days of the model prediction test time, and carrying out red, yellow and reminding on the area where the tool is located in a warehouse map.

3) Task mode: by collecting information such as the types and the number of tools and instruments required by a historical operation plan, the complexity of tasks, the required time and the like, a decision tree model is built, and training and verification are performed. The user is enabled to match a work plan according to conditions such as face identification information and time, and map positions and the number of tools required by the plan are output according to the work plan.

Example 2

In one embodiment of the present disclosure, there is provided an OpenCV-based warehouse map visualization building system, including:

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While the specific embodiments of the present disclosure have been described above with reference to the drawings, it should be understood that the present disclosure is not limited to the embodiments, and that various modifications and changes can be made by one skilled in the art without inventive effort on the basis of the technical solutions of the present disclosure while remaining within the scope of the present disclosure.

Claims

1. The visual warehouse map construction method based on OpenCV is characterized by comprising the following steps of:

the method for automatically matching the object model according to the live-action image in the warehouse comprises the steps of shelf positioning and shelf type identification, wherein the method for positioning the shelf comprises the following steps: acquiring background wall images of all the shelves in the warehouse, rapidly extracting features at the junctions among all the walls of the warehouse by using an OpenCV SURF feature detection method, matching feature values by using FLANN, constructing association relations among all the walls, and realizing the positioning of the shelves;

the shelf type identification includes: extracting the characteristic value by adopting a SIFT algorithm, and dividing the characteristic value into two parts of a shelf and tools and instruments for identification, wherein the shelf identification is that the characteristic value is extracted by adopting the SIFT algorithm, the characteristic value is compared with a live-action image, and if the characteristic value is successfully matched, the shelf is marked as the shelf of the same type;

the shelf is identified as adopting SIFT algorithm to extract the characteristic value, comprising: identifying a goods shelf by using a SIFT algorithm, acquiring a goods shelf inflection point, connecting the goods shelf inflection point in series to form a line according to the goods shelf inflection point, acquiring a framework of the goods shelf, comparing the framework of the goods shelf with an object model of the goods shelf, and identifying the type of the goods shelf;

the tool identification method comprises the following steps: grabbing a tool image associated with the type of the goods shelf, carrying out characteristic value identification on the tool image through a SIFT algorithm, and carrying out characteristic matching on the characteristic value and actual goods on the goods shelf to complete identification of the tool;

after the goods shelf identification is finished, the goods shelf is sequentially placed on the map by taking the position of the warehouse door as a base point and taking the warehouse plan view as a map range, after the goods shelf placement is finished, the corresponding goods shelf is clicked, tools and instruments are screened according to the goods shelf type, and the placement of the tools and instruments is finished, so that a warehouse map is constructed and visualized.

2. The OpenCV-based warehouse map visualization construction method of claim 1, wherein the shelf categories include ground wire shelves, pole shelves, conventional shelves, glove shoe shelves, ladder shelves.

3. The OpenCV-based warehouse map visualization construction method of claim 1, wherein establishing an association between a shelf and a tool includes establishing an association between a glove, a shoe support, and a glove, a shoe.

4. The OpenCV-based warehouse map visualization construction method of claim 1, wherein the in-warehouse live-action image comprises a shelf image, a tool image, and each shelf background wall image, and the in-warehouse object standard data comprises personnel data, work plan information, tool data, and warehouse base information, wherein the warehouse base information comprises a plan view of the warehouse including area, actual length and width dimension information.

5. The OpenCV-based warehouse map visualization construction method of claim 1, wherein constructing an association relationship between wall surfaces, and positioning a shelf specifically comprises:

and obtaining wall face images of the background of each goods shelf in the warehouse, numbering the images, making a plurality of vertical dividing lines from the upper corners of each image to the ground, selecting a wall body identification main body, sequentially carrying out feature matching, and finding out the associated wall faces.

6. Warehouse map visualization construction system based on OpenCV, characterized by comprising: