CN117275011A

CN117275011A - Commodity identification and commodity price tag matching method, system, terminal and medium

Info

Publication number: CN117275011A
Application number: CN202311312546.6A
Authority: CN
Inventors: 李德圆; 丁明; 王杰; 徐洪亮; 许洁斌
Original assignee: Guangzhou Xuanwu Wireless Technology Co Ltd
Current assignee: Guangzhou Xuanwu Wireless Technology Co Ltd
Priority date: 2023-10-11
Filing date: 2023-10-11
Publication date: 2023-12-22
Anticipated expiration: 2043-10-11
Also published as: CN117275011B

Abstract

The invention discloses a commodity identification and commodity price matching method, a system, a terminal and a medium, comprising the following steps: carrying out commodity identification and price tag detection on the picture to be identified to obtain the position information of each commodity to be matched, and the position information and the matching direction information of each price tag to be matched; traversing each price tag to be matched, carrying out comparative analysis on the information to obtain the matching score between the current price tag to be matched and each commodity to be matched, and constructing a candidate set corresponding to the current price tag to be matched by utilizing a plurality of commodities to be matched with the matching score not equal to 0; and carrying out global self-adaptive optimization on each candidate set and the matching scores of all the commodities to be matched in each candidate set, selecting the commodity to be matched with the highest matching score from each candidate set after the optimization is finished, and analyzing to obtain the matching result of each price to be matched. The invention improves the matching accuracy of commodity price tags from multiple layers based on the local price tag-commodity association relation and global self-adaptive optimization.

Description

Commodity identification and commodity price tag matching method, system, terminal and medium

Technical Field

The invention relates to the field of commodity price tag matching, in particular to a commodity identification and commodity price tag matching method, a system, a terminal and a medium.

Background

In the fast-food area, the price of goods is one of the key factors that directly influence consumer decisions. In order to promote new products and widen markets, various large brands of merchants have a lot of promotion preferential activities and minimum price requirements. However, in the execution of the actual terminal, adverse phenomena such as price missing, price inconsistent with the commodity, non-reflected preferential price adjustment, abnormal pricing of the channel manufacturer and the like often exist. The AI price tag identification becomes an important mode for terminal price control, and the AI price tag identification automatically identifies commodities and related price information corresponding to the signboards through an artificial intelligence technology, so that conditions of sales promotion, pricing execution and the like of the terminal commodities are better reflected, and a low-efficiency working mode of manually inputting prices is changed.

In the existing commodity price tag recognition technology, characters can be generally extracted through OCR, price numbers can be extracted through combining templates, regular rules, customized price detection models and the like, but in an actual terminal, price tag patterns are various and are updated continuously, quick response to terminal changes is difficult based on methods such as templates, iteration optimization cost is high, and a general solution is difficult to form. On the other hand, in the actual store display of the price tag and the commodity, partial mismatch phenomenon exists in many cases, and generally, the method for carrying out commodity price tag association based on the shortest space distance is difficult to adapt to diversified scenes, such as a laminate shelf, a hanger and the like, meanwhile, the problem that common commodity and price tag are difficult to be strictly corresponding to each other in actual placement is solved, and in addition, wrong commodity price is generated and effective price control cannot be carried out.

Disclosure of Invention

The embodiment of the invention provides a commodity identification and commodity price tag matching method, a system, a terminal and a medium.

In order to solve the above technical problems, an embodiment of the present invention provides a method for identifying and matching a commodity price tag, including:

carrying out commodity identification and price tag detection on the picture to be identified to obtain the position information of each commodity to be matched, and the position information and the matching direction information of each price tag to be matched;

traversing each price tag to be matched, carrying out comparison analysis on the position information and the matching direction information of the current price tag to be matched and the position information of all the commodities to be matched to obtain the matching score between the current price tag to be matched and each commodity to be matched, and constructing a candidate set corresponding to the current price tag to be matched by utilizing a plurality of commodities to be matched with the matching score not equal to 0;

and carrying out global self-adaptive optimization on each candidate set and the matching scores of all the commodities to be matched in each candidate set, respectively selecting the commodities to be matched with the highest matching score from each current candidate set after the optimization is finished as the optimal matching commodity of each current candidate set, and taking the optimal matching commodity as the commodity matching result of the price to be matched corresponding to the current candidate set if the commodity category of the optimal matching commodity is a preset category, otherwise, taking the commodity matching result of the price to be matched corresponding to the current candidate set as a non-matching object.

By implementing the embodiment of the invention, commodity identification and price tag detection are carried out on the pictures to be identified to obtain the position information of each commodity to be matched, the position information and the matching direction information of each price tag to be matched, and then the position information and the matching direction information of each price tag to be matched and the position information of all commodities to be matched are compared and analyzed, so that the matching degree between each price tag to be matched and each commodity to be matched can be evaluated, the matching score between each price tag to be matched and each commodity to be matched is obtained, and the commodity to be matched with the matching degree equal to zero is screened out, namely the commodity to be matched which is not associated with the label to be matched is screened out, so that the subsequent data processing amount is reduced, and the overall commodity price tag matching efficiency is improved. In addition, global self-adaptive optimization is carried out on the candidate sets corresponding to the price to be matched and the matching scores of all the commodities to be matched in the candidate sets, feedback of the matching scores of the candidate sets and the commodities to be matched can be considered in each iteration, the matching scores of the commodities to be matched and the commodities to be matched in the candidate sets are continuously optimized in a self-adaptive mode, after optimization is finished, the commodities to be matched with the current candidate sets corresponding to the price to be matched are selected as the optimal matching commodities of the current candidate sets, the optimal matching commodities are used as the commodity matching results of the commodities to be matched corresponding to the current candidate sets if the commodity types of the optimal matching commodities are preset types, otherwise, the commodity matching results of the commodities to be matched corresponding to the current candidate sets are non-matching objects, so that each price to be matched with the commodity with the highest matching degree can be ensured, accurate matching of the commodity to be achieved, and accurate commodity information is provided for users.

As a preferred solution, the global adaptive optimization is performed on the matching scores of each candidate set and all the commodities to be matched in each candidate set, and after the optimization is finished, the commodities to be matched with the highest matching score are selected from each current candidate set respectively to be used as the optimal matching commodities of each current candidate set, if the commodity category of the optimal matching commodity is a preset category, the optimal matching commodity is used as the commodity matching result of the price to be matched corresponding to the current candidate set, otherwise, the commodity matching result of the price to be matched corresponding to the current candidate set is no matching object, specifically:

according to a global self-adaptive optimization algorithm, carrying out iterative optimization on matching scores of all goods to be matched in each candidate set and each candidate set, traversing each price to be matched in the candidate set corresponding to the current price to be matched, deleting all goods to be matched which have binding relation with the current price to be matched and have price unequal to the price corresponding to the current price to be matched, if the goods types of all goods to be matched in the candidate set corresponding to the current price to be matched are the same, adding the matching scores between all goods to be matched in the candidate set corresponding to the current price to be matched and the current price to be matched, taking the result as the matching score between any one goods to be matched in the candidate set corresponding to the current price to be matched and the current price to finish updating of the matching score, then only keeping the goods to be matched which finish updating of the matching score in the candidate set corresponding to be matched, and establishing the binding relation of the goods to be matched which finish updating of the matching score and the current price to be matched until all the goods to be matched are not matched are changed;

After finishing iteration, selecting the commodities to be matched with highest matching scores from the current candidate sets respectively to serve as the optimal matching commodities of the current candidate sets; if the commodity category of the optimal matching commodity of the current candidate set is a preset category, taking the optimal matching commodity as a commodity matching result of the price to be matched corresponding to the current candidate set; if the commodity category of the optimally matched commodity of the current candidate set is not the preset category, the commodity matching result of the price tag to be matched corresponding to the current candidate set is no matching object.

According to the preferred scheme of the embodiment of the invention, iteration optimization is carried out on the candidate set corresponding to each price to be matched and the matching scores of all commodities to be matched in each candidate set according to a global self-adaptive optimization algorithm, when each iteration processing is carried out, the matching scores between any commodity to be matched in the candidate set corresponding to the current price to be matched and the price of the commodity are not equal to the price corresponding to the current price to be matched are deleted in the candidate set corresponding to the current price to be matched, the influence of the commodity price to be matched and the price of the commodity to be matched, which is not matched with the price corresponding to the price, on the matching accuracy of the commodity price to be matched can be avoided, then if all the commodities to be matched in the candidate set corresponding to the current price to be matched are the same, all the commodities to be matched in the candidate set corresponding to be matched are added to the matching scores between the current price to be matched, and the matching scores between any commodity to be matched in the candidate set corresponding to be matched and the current price to be matched are added to finish updating the matching scores, then the matching scores to be completed in the candidate set corresponding to be matched, the current price to be matched is only the current price to be matched, the updating of the commodity to be matched is completed, the matching scores to be matched, the current price to be matched is not matched, the matching item to be matched is set, and the matching item to be matched is set to be matched, and the optimal, and the matching item to be matched, and the matching item to be matched and the matched to be matched with the current and matched and the commodity to be matched and the best and matched is set, as the commodity matching result of the corresponding price tag to be matched, the problem of dislocation of the price tag and the commodity placing part can be effectively solved, and the recognition rate of the commodity associated price is improved.

As a preferred solution, traversing each price to be matched, comparing and analyzing the position information and the matching direction information of the price to be matched and the position information of all the commodities to be matched to obtain a matching score between the price to be matched and each commodity to be matched, and constructing a candidate set corresponding to the price to be matched by using a plurality of commodities to be matched with the matching score not equal to 0, wherein the candidate set comprises:

traversing each price tag to be matched, and screening a plurality of associated commodities corresponding to the current price tag to be matched from all commodities to be matched according to the position information and the matching direction information of the current price tag to be matched and the position information of all commodities to be matched;

according to a preset matching score algorithm, calculating to obtain a matching score between the current price to be matched and each associated commodity according to the position information of the current price to be matched and the position information of each associated commodity, and adding all the associated commodities with the matching score not equal to 0 into a candidate set corresponding to the current price to be matched.

By implementing the preferred scheme of the embodiment of the invention, each price to be matched is traversed, and a plurality of associated commodities corresponding to the current price to be matched are screened out from all commodities to be matched according to the position information and the matching direction information of the current price to be matched and the position information of all commodities to be matched, so that the matching range can be primarily reduced, the subsequent operation and the matching data quantity are reduced, and the commodity price matching efficiency is further improved. In addition, according to a preset matching score algorithm, the degree of association between the price tag and each associated commodity can be evaluated and compared by calculating the matching score between the price tag and each associated commodity, and all associated commodities with the matching score not equal to 0 are added into a candidate set corresponding to the current price tag to be matched, so that the consideration of the commodity to be matched which is irrelevant to the current price tag to be matched or has low matching degree can be avoided, and the accuracy of matching of the commodity price tag is further improved.

As a preferred scheme, traversing each price tag to be matched, and screening a plurality of associated commodities corresponding to the current price tag to be matched from all commodities to be matched according to the position information and the matching direction information of the current price tag to be matched and the position information of all commodities to be matched, wherein the specific steps are as follows:

traversing each price tag to be matched and each commodity to be matched, and judging that the commodity to be matched meets the position condition of the price tag to be matched if the absolute value of the difference value between the vertical coordinate of the top left vertex of the commodity to be matched and the vertical coordinate of the top left vertex of the commodity to be matched is smaller than or equal to the difference value between the vertical coordinate of the bottom right vertex of the commodity to be matched and the vertical coordinate of the top left vertex of the commodity to be matched;

if the matching direction information of the current price tag to be matched is upward matching, when the current commodity to be matched meets the position condition of the current price tag to be matched and the ordinate of the upper left vertex of the current commodity to be matched is smaller than or equal to the ordinate of the lower right vertex of the current price tag to be matched, the current commodity to be matched is used as the associated commodity corresponding to the current price tag to be matched;

if the matching direction information of the current price tag to be matched is downward matching, when the current commodity to be matched meets the position condition of the current price tag to be matched and the ordinate of the lower right vertex of the current commodity to be matched is greater than or equal to the ordinate of the upper left vertex of the current price tag to be matched, the current commodity to be matched is used as the associated commodity corresponding to the current price tag to be matched.

According to the preferred scheme of the embodiment of the invention, according to the matching direction information of the price tag to be matched and based on the position information of the commodity to be matched and the position information of the price tag to be matched, the comparison and judgment of the vertical coordinates of the top points of the price tag to be matched and the commodity to be matched are realized, the associated commodity corresponding to each price tag to be matched can be screened out more accurately, and the risks of incorrect matching and missed matching are reduced.

As a preferred solution, the method for matching commodity identification with commodity price tag further includes:

according to the position information of each price to be matched, carrying out label region segmentation on the image to be identified to obtain price pictures corresponding to each price to be matched;

inputting the price tag pictures corresponding to the price tags to be matched into a price tag OCR recognition model so that the price tag OCR recognition model extracts price information of the price tag pictures corresponding to the price tags to be matched to obtain one or more character areas corresponding to the price tags to be matched and price characters corresponding to the character areas;

extracting the characteristics of each character region, and inputting the characteristic extraction result of each character region into an SVM classifier so that the SVM classifier classifies each character region and outputs a classification result corresponding to each character region; the feature extraction result comprises character height, result confidence and style confidence of price characters corresponding to the character areas, and the classification result corresponding to the character areas is that the character areas are price areas or the character areas are not price areas;

Traversing each price to be matched, and taking the minimum value in price characters corresponding to all character areas classified as price areas in the current price to be matched as the label price of the current price to be matched;

and determining the price to be matched corresponding to each commodity to be matched according to the commodity matching result of each price to be matched, and then taking the price to be matched corresponding to each commodity to be matched and the label price of the price to be matched corresponding to each commodity to be matched as the identification result corresponding to each commodity to be matched.

According to the preferred scheme of the embodiment of the invention, as a certain price to be matched exists and a plurality of price characters are simultaneously identified due to a large amount of text information in the price to be matched, which price character is difficult to distinguish at the moment, the character height, the result confidence and the score of the price character corresponding to each character area are obtained through feature extraction of each character area, the SVM classifier can be effectively assisted in classifying and judging the price area, and then the feature extraction result is input into the SVM classifier, so that the SVM classifier classifies each character area into a price area and a non-price area, thereby effectively extracting the character area related to the price from the price to be matched, providing an accurate area to be detected for subsequent price detection, and finally, for each price to be matched, taking the minimum value of the price characters corresponding to all the character areas classified as the price area in the current price to be matched as the price of the label of the price to be matched, the accurate influence of noise and error identification can be eliminated to a certain extent, and the unique and unique detection result of each price to be provided for users.

As a preferred solution, the identifying of the commodity and detecting of the price tag are performed on the picture to be identified, so as to obtain the position information of each commodity to be matched, and the position information and the matching direction information of each price tag to be matched, specifically:

inputting the pictures to be identified into a sample detection model and a general commodity detection model respectively, so that the sample detection model performs sample category identification on the pictures to be identified, the general commodity detection model performs other commodity detection on the commodities to be matched in the pictures to be identified, and pre-processes the commodity identification result and other commodity detection results of the sample to obtain the position information of each commodity to be matched;

inputting the picture to be identified into a price tag detection model so that the price tag detection model carries out price tag detection on the picture to be identified to obtain position information and matching direction information of each price tag to be matched;

the product detection model is obtained by training a pre-built CascadeRCNN model, the general product detection model is obtained by training a pre-built MaskRCNN model, the products to be matched are products or other products, the products belong to preset categories of products to be matched, and the other products do not belong to the preset categories of products to be matched.

According to the preferred scheme of the embodiment of the invention, the CascadeRCNN model uses a cascade structure, background and false detection targets are gradually screened out by cascading a plurality of detectors, each cascade stage is trained and extracted by utilizing the result of the last stage, and targets which are difficult to detect are more finely distinguished and screened, so that the commodity identification accuracy of the picture to be identified by the product detection model is improved, and meanwhile, the number of candidate targets screened in each stage is gradually reduced, so that the overall higher running speed can be still maintained. In addition, the mask RCNN combines the advantages of target detection and instance segmentation, semantic segmentation can be performed on the picture to be identified on the basis of target detection, and accurate pixel-level segmentation and target detection can be synchronously realized.

As a preferred solution, the price tag detection model specifically includes: the device comprises a feature extraction layer, a feature fusion layer, a detection head and a direction head;

the feature extraction layer is used for extracting features of the picture to be identified to obtain corresponding deep features and shallow features;

the feature fusion layer is used for carrying out depth feature cross fusion on the deep features and the shallow features to obtain corresponding price tag features;

The detection head is used for detecting all price tags to be matched in the image to be identified based on the price tag characteristics to obtain the position information of each price tag to be matched;

and the direction head is used for carrying out matching direction prediction on the pictures to be identified based on the price tag characteristics to obtain the matching direction information of each price tag to be matched.

By implementing the preferred scheme of the embodiment of the invention, the extracted features with different levels or different resolutions are fused together through the feature fusion layer of the price tag detection model so as to obtain richer and multi-scale feature information, and the price tag detection model can simultaneously consider the features from different levels and scales through feature fusion, so that the detection capability and the robustness of price tags with different sizes are improved.

In order to solve the same technical problems, the embodiment of the invention also provides a system for identifying commodity and matching commodity price tags, which comprises:

the identification detection module is used for carrying out commodity identification and price tag detection on the picture to be identified to obtain the position information of each commodity to be matched, and the position information and the matching direction information of each price tag to be matched;

the analysis screening module is used for traversing each price tag to be matched, carrying out comparison analysis on the position information and the matching direction information of the current price tag to be matched and the position information of all the commodities to be matched to obtain the matching score between the current price tag to be matched and each commodity to be matched, and constructing a candidate set corresponding to the current price tag to be matched by utilizing a plurality of commodities to be matched with the matching score not equal to 0;

And the optimization matching module is used for carrying out global self-adaptive optimization on the matching scores of all the candidate sets and all the commodities to be matched in the candidate sets, respectively selecting the commodities to be matched with the highest matching score from the current candidate sets after the optimization is finished as the optimal matching commodity of the current candidate sets, and taking the optimal matching commodity as the commodity matching result of the price to be matched corresponding to the current candidate set if the commodity category of the optimal matching commodity is a preset category, otherwise, taking the commodity matching result of the price to be matched corresponding to the current candidate set as a non-matching object.

In order to solve the same technical problems, the invention also provides a terminal which comprises a processor, a memory and a computer program stored in the memory; wherein the computer program is executable by the processor to implement the method of merchandise identification and merchandise price matching.

To solve the same technical problem, the present invention also provides a computer-readable storage medium including a stored computer program; and controlling the equipment where the computer readable storage medium is located to execute the commodity identification and commodity price matching method when the computer program runs.

Drawings

Fig. 1: a schematic flow chart of a commodity identification and commodity price matching method is provided in the first embodiment of the present invention;

fig. 2: the first embodiment of the invention provides a structure schematic diagram of a commodity identification and commodity price tag matching system.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Embodiment one:

referring to fig. 1, a method for identifying and matching a commodity price tag according to an embodiment of the present invention includes steps S1 to S3, where each step is specifically as follows:

step S1, carrying out commodity identification and price tag detection on the picture to be identified to obtain the position information of each commodity to be matched, and the position information and the matching direction information of each price tag to be matched.

Preferably, step S1 includes steps S11 to S12, and each step is specifically as follows:

And S11, inputting a picture to be identified into the product detection model and the general commodity detection model respectively, so that the product detection model performs product category identification on the picture to be identified, the general commodity detection model performs other commodity detection on the picture to be identified, and the product identification result and other commodity detection results are preprocessed to obtain the position information of each commodity to be matched.

It should be noted that, the product detection model is oriented to a specific customer, for example, is used for identifying a product produced by a certain enterprise (a product produced by a certain enterprise to be identified later is referred to as the product), the products produced by the same enterprise may have different types, different types of products produced by the same enterprise have different SKUs, and for the products produced by other enterprises, the product type of the product identified by the product detection model is a preset type, and the product belonging to the preset type is the product; the general commodity detection model is used for detecting the type and position information of the objects (including commodity and non-commodity materials) in the picture to be identified, the commodity detected by the general commodity detection model is not the commodity, but belongs to other types of commodity except the preset type, the commodity is marked as other commodity, the commodity type of the other commodity is marked as other, and the type of the non-commodity materials such as a sales promotion board, an advertisement paste and the like in the picture is marked as fake. The product detection model is obtained by training a pre-built CascadeRCNN model, and the general commodity detection model is obtained by training a pre-built MaskRCNN model by using a large number of pictures of quick-elimination scenes such as freezers, shelves and the like. The CascadeRCNN model and the MaskRCNN model are both constructed by adopting a swinTransformer as a backbone network and performing feature fusion through FPN (Feature Pyrami d Networks, feature pyramid).

In this embodiment, "preprocessing the product identification result and other product detection results to obtain the position information of each product to be matched" in step S11 includes steps S111 to S114, where each step is specifically as follows:

in step S111, please refer to formula (1), the result of identifying the commodity and the result of detecting other commodities are subjected to the IOU calculation to obtain an IOU threshold.

In the method, in the process of the invention,is the external rectangle of a commodity A in the commodity identification result of the commodity, and is->The external rectangle of a certain article B in the detection results of other commodities.

Step S112, traversing each article B in the other article detection results, and deleting the detection information of the current article B in the other article detection results if the IOU threshold of the current article B and any article A is greater than 0.5.

In step S113, in the other commodity detection results, detection information of the article of the type like fake, that is, detection information of non-commodity materials such as sales cards, advertisement stickers, etc. are removed.

And step S114, combining the detection results of other commodities after finishing the information screening of the steps S112 to S113 with the commodity identification result of the commodity to obtain the position information of each commodity to be matched.

The position information of each commodity to be matched is as follows: the upper left vertex has the coordinate values of (g_x, g_y), and the lower right vertex has the coordinate values of (g_maxx, g_maxy).

And step S12, inputting the picture to be identified into the price tag detection model so that the price tag detection model can perform price tag detection on the picture to be identified to obtain the position information and the matching direction information of each price tag to be matched.

The position information of each price tag to be matched is as follows: the upper left vertex has the coordinate values of (p_x, p_y), and the lower right vertex has the coordinate values of (p_maxx, p_maxy). The matching direction information F of each price tag to be matched is (F_up, F_down); wherein D_up is upward matching, which means that the price tag to be matched should be matched/associated with the commodity to be matched above the price tag to be matched; d_down is a downward match, indicating that the price to be matched should be matched/associated with the goods to be matched below it.

As a preferred embodiment, the price tag detection model mentioned in step S12 specifically includes: the device comprises a feature extraction layer, a feature fusion layer, a detection head and a direction head, wherein each component/network layer is specifically as follows:

the feature extraction layer is used for extracting features of the picture to be identified to obtain corresponding deep features and shallow features; the feature extraction layer is a deep convolution network;

and the direction head is used for predicting the matching direction of the picture to be identified based on the price tag characteristics to obtain the matching direction information of each price tag to be matched. The direction category loss function corresponding to the direction head adopts FocalLoss to relieve the problem of unbalanced distribution of upward matching and downward matching in an actual scene.

Step S2, traversing each price to be matched, comparing and analyzing the position information and the matching direction information of the price to be matched currently and the position information of all commodities to be matched, obtaining matching scores between the price to be matched currently and each commodity to be matched, and constructing a candidate set corresponding to the price to be matched currently by utilizing a plurality of commodities to be matched with the matching score not equal to 0.

Preferably, step S2 includes steps S21 to S22, and each step is specifically as follows:

step S21, traversing each price to be matched, and screening a plurality of associated commodities corresponding to the price to be matched from all commodities to be matched according to the position information and the matching direction information of the price to be matched and the position information of all commodities to be matched.

The position information of each commodity to be matched is as follows: the upper left vertex has the coordinate values of (g_x, g_y), and the lower right vertex has the coordinate values of (g_maxx, g_maxy). The position information of each price tag to be matched is as follows: the upper left vertex has the coordinate values of (p_x, p_y), and the lower right vertex has the coordinate values of (p_maxx, p_maxy). The matching direction information D of each price tag to be matched is (d_up, d_down).

Preferably, step S21 includes steps S211 to S213, and each step is specifically as follows:

step S211, traversing each price tag to be matched and each commodity to be matched, and if the absolute value of the difference value between the ordinate g_y of the top left vertex of the current commodity to be matched and the ordinate p_y of the top left vertex of the current price tag to be matched is smaller than or equal to the difference value between the ordinate g_maxy of the bottom right vertex of the current commodity to be matched and the ordinate g_y of the top left vertex of the current commodity to be matched (i.e., | (g_y-p_y) |is less than or equal to (g_maxy-g_y)), judging that the current commodity to be matched meets the position condition of the current price tag to be matched.

In step S212, if the matching direction information D of the current price to be matched is the upward matching direction d_up, the current price to be matched is used as the associated product corresponding to the current price to be matched when the current price to be matched meets the position condition of the current price to be matched and the ordinate g_y of the top left vertex of the current price to be matched is smaller than or equal to the ordinate p_maxy of the bottom right vertex of the current price to be matched.

In step S213, if the matching direction information D of the current price to be matched is downward matching d_down, the current price to be matched is taken as the associated product corresponding to the current price to be matched when the current price to be matched meets the position condition of the current price to be matched and the ordinate g_maxy of the lower right vertex of the current price to be matched is greater than or equal to the ordinate p_y of the upper left vertex of the current price to be matched.

Step S22, according to a preset matching score algorithm, calculating to obtain the matching score between the current price to be matched and each associated commodity according to the position information of the current price to be matched and the position information of each associated commodity, and adding all the associated commodities with the matching score not equal to 0 into the candidate set corresponding to the current price to be matched.

In this embodiment, in step S22, "according to a preset matching score algorithm, a matching score between a current price to be matched and each associated commodity is calculated according to the position information of the current price to be matched and the position information of each associated commodity", including steps S221 to S222, and the steps are specifically as follows:

in step S221, please refer to equations (2) (3) (4), the distance parameters F1, F2 and f_center between the current price to be matched and each associated commodity are calculated according to the position information of the current price to be matched and the position information of each associated commodity.

F1＝max(p_x,g_x) (2)

F2＝min(p_maxx,g_maxx) (3)

In step S222, please refer to formula (5), a matching score tscore between the current price to be matched and each associated commodity is calculated according to the three distance parameters F1, F2 and f_center calculated in step S221.

And S3, performing global self-adaptive optimization on each candidate set and the matching scores tscore of all the commodities to be matched in each candidate set, and respectively selecting the commodity to be matched with the highest matching score tscore' from each current candidate set after the optimization is finished as the optimal matching commodity of each current candidate set, wherein if the commodity category of the optimal matching commodity is a preset category, the optimal matching commodity is used as the commodity matching result of the price to be matched corresponding to the current candidate set, otherwise, the commodity matching result of the price to be matched corresponding to the current candidate set is a non-matching object.

It should be noted that, the commodity to be matched is an actually sold commodity, in order to conveniently distinguish different types of commodities, a specific service bar code is generally configured for the commodity, and the commodity with the specific service bar code is SKU.

Preferably, step S3 includes steps S31 to S32, and each step is specifically as follows:

and S31, carrying out iterative optimization on each candidate set and the matching scores tscore of all commodities to be matched in each candidate set according to a global self-adaptive optimization algorithm, traversing each price to be matched in the candidate set corresponding to the current price to be matched when each iterative process is carried out, deleting all commodities to be matched which have a binding relation with the current price to be matched and are different from the price corresponding to the current price to be matched, if all commodity categories of the commodities to be matched in the candidate set corresponding to the current price to be matched are the same, adding the matching scores tscore between all commodities to be matched in the candidate set corresponding to the current price to be matched and the current price to be matched, taking the result as the matching score tscore between any commodity to be matched in the candidate set corresponding to the current price to be matched to finish updating the matching score tscore of the commodity to be matched, then only preserving the commodity to be matched for finishing updating in the candidate set corresponding to the current price to be matched, and establishing the binding relation between the commodity to be matched and the current price to be matched until all the candidate sets are not matched.

It should be noted that the candidate set is not changed any more, and the number and the category of the commodities to be matched in the candidate set are not changed any more.

As an example, the candidate set corresponding to the currently traversed price tag a to be matched is marked as a candidate set a ', and the candidate set a ' at this time includes a commodity a to be matched, a commodity b to be matched, a commodity c to be matched, a commodity d to be matched and a commodity e to be matched, where the commodity e to be matched and the price tag a to be matched have a binding relationship, and the commodity price of the commodity e to be matched is not equal to the price corresponding to the price tag a to be matched, and the commodity e to be matched is deleted from the candidate set a '. Then, if the four commodities to be matched, namely, the commodity a to be matched, the commodity b to be matched, the commodity c to be matched and the commodity d to be matched all belong to the same commodity category, adding the matching scores of the four commodities to be matched, namely, the commodity a to be matched, the commodity b to be matched, the commodity c to be matched and the commodity d to be matched and the price tag A to be matched, and then adding the result to be used as a matching score of any one commodity to be matched (in the example, any commodity to be matched can be described by taking the commodity b to be matched as an example) in the candidate set A', so as to update the matching score between the commodity b to be matched and the price tag A to be matched, and establishing the binding relation between the commodity b to be matched and the price tag A to be matched; if the four commodities to be matched, namely the commodity a to be matched, the commodity b to be matched, the commodity c to be matched and the commodity d to be matched do not belong to the same commodity category, the candidate set A' comprising the commodity a to be matched, the commodity b to be matched, the commodity c to be matched and the commodity d to be matched is not processed. In this process, since the number of articles to be matched in the initial candidate set a' changes, iterative optimization needs to be continued.

Step S32, after finishing iteration, selecting the commodities to be matched with highest matching scores from the current candidate sets respectively to serve as the optimal matching commodities of the current candidate sets; if the commodity category of the optimally matched commodity of the current candidate set is a preset category, taking the optimally matched commodity as a commodity matching result of the price tag to be matched corresponding to the current candidate set; if the commodity category of the optimally matched commodity of the current candidate set is not the preset category, the commodity matching result of the price tag to be matched corresponding to the current candidate set is no matching object.

As an example, the preset categories are category a and category b. After finishing the iteration, for a candidate set A ' corresponding to the price tag A to be matched, if the candidate set A ' contains a commodity b to be matched and a commodity c to be matched, wherein the commodity categories of the commodity b to be matched and the commodity c to be matched are different, and the matching score between the commodity b to be matched and the price tag A to be matched is higher than the matching score between the commodity c to be matched and the price tag A to be matched, the commodity b to be matched is taken as the optimal matching commodity of the candidate set A '. At this time, if the commodity category of the commodity b to be matched is one of the category a or the category b, the commodity b to be matched is used as a commodity matching result of the price tag a to be matched; if the commodity category of the commodity b to be matched is neither category A nor category B, namely the commodity category of the commodity b to be matched is not a preset category, the commodity matching result of the price tag A to be matched is no matching object, namely no clear commodity matching result.

As a preferred solution, the method for identifying and matching a commodity price according to the embodiment of the present invention further includes a price detection process, where the process includes steps S01 to S04, and the steps are specifically as follows:

and step S01, carrying out label region segmentation on the picture to be identified according to the position information of each price to be matched to obtain price picture corresponding to each price to be matched.

And step S02, inputting the price tag pictures corresponding to the price tags to be matched into the price tag OCR recognition model so that the price tag OCR recognition model extracts price information of the price tag pictures corresponding to the price tags to be matched to obtain one or more character areas corresponding to the price tags to be matched and price characters corresponding to the character areas.

In this embodiment, the price tag OCR recognition model includes a DBnet text detection model and an SVTR text recognition model. The DBnet text detection model is used for taking charge of character extraction of price tags and obtaining position information of a character area; and the SVTR character recognition model is used for cutting out the character area according to the position information output by the DBnet character detection model, inputting the character area obtained by cutting into the SVTR character recognition model, and extracting the character content of the input character area to obtain the price character corresponding to the character area.

Additionally, since price tags have various conditions such as semantic price tags, promotional prices, handwritten prices, etc., it is necessary to collect various price tag patterns and train a price tag OCR recognition model. Specifically, because semantic tags exist, such as different units (elements and angles) are determined through digital size, decimal points are added to the semantic tags to train the tag OCR recognition model in the training process.

It should be noted that, since the text information in the price tag picture is many, there may be a plurality of price characters identified by one price tag picture, at this time, it is difficult to distinguish which price character is the tag price of the price tag to be matched, so that through steps S03 to S04, only one price character is screened out as the tag price of the price tag to be matched.

And S03, extracting the characteristics of each character area, and inputting the characteristic extraction result of each character area into the SVM classifier so that the SVM classifier can classify each character area and output the classification result corresponding to each character area. The classification result corresponding to the character area is that the character area is a price area or the character area is not a price area.

In the present embodiment, the feature extraction result of each character region includes the following three contents:

(1) And (3) carrying out normalization processing of the character height h of the price character corresponding to each character area according to the formula (6) to obtain h_norm.

In the formula, H_p is the height of the price tag picture corresponding to the current price character.

(2) And the price tag OCR model of the price character corresponding to each character area recognizes the result confidence degree p of the result.

(3) Style confidence score of price character corresponding to each character area. If the current price character has decimal points, referring to formula (7), calculating to obtain a style confidence score of the current price character; if the current price character does not have a decimal point, the style confidence score for the current price character is 0.

Wherein N_dot is the number of price characters with decimal points in the recognition results of all price tag OCR models of the picture to be recognized, and N_n is the number of price characters without decimal points in the recognition results of all price tag OCR models of the picture to be recognized.

Step S04, traversing each price to be matched, and taking the minimum value of price characters corresponding to all character areas classified as price areas in the current price to be matched as the label price of the current price to be matched;

and step S05, determining price tags to be matched corresponding to all the commodities to be matched according to commodity matching results of all the price tags to be matched, and then taking the price tags to be matched corresponding to all the commodities to be matched and the label prices of the price tags to be matched corresponding to all the commodities to be matched as identification results corresponding to all the commodities to be matched.

Referring to fig. 2, a schematic structural diagram of a system for identifying and matching commodity price tags according to an embodiment of the present invention includes an identification detection module M1, an analysis screening module M2, and an optimization matching module M3, where each module is specifically as follows:

the identification detection module M1 is used for carrying out commodity identification and price tag detection on the picture to be identified to obtain the position information of each commodity to be matched, and the position information and the matching direction information of each price tag to be matched;

the analysis and screening module M2 is used for traversing each price tag to be matched, carrying out comparison analysis on the position information and the matching direction information of the current price tag to be matched and the position information of all commodities to be matched to obtain the matching score between the current price tag to be matched and each commodity to be matched, and constructing a candidate set corresponding to the current price tag to be matched by utilizing a plurality of commodities to be matched with the matching score not equal to 0; wherein, one or more commodities to be matched correspond to one SKU to be matched;

the optimization matching module M3 is configured to perform global adaptive optimization on each candidate set and matching scores of all the to-be-matched commodities in each candidate set, and after the optimization is finished, respectively select the to-be-matched commodity with the highest matching score from each current candidate set as the current optimal matching commodity of each candidate set, if the commodity category of the optimal matching commodity is a preset category, then the optimal matching commodity is used as the commodity matching result of the to-be-matched price tag corresponding to the current candidate set, otherwise, the commodity matching result of the to-be-matched price tag corresponding to the current candidate set is no matching object.

It will be clear to those skilled in the art that, for convenience and brevity of description, reference may be made to the corresponding process in the foregoing method embodiment for the specific working process of the above-described system, which is not described herein again.

Additionally, embodiments of the present invention also provide a computer-readable storage medium including a stored computer program; wherein the device in which the computer readable storage medium is located is controlled to execute a method for identifying and matching a commodity price according to the first embodiment when the computer program is run.

Additionally, the embodiment of the invention also provides a terminal, which comprises a processor, a memory and a computer program stored in the memory; the computer program can be executed by a processor to implement a method for identifying and matching a commodity price according to the first embodiment.

Preferably, the computer program may be divided into one or more modules/units (e.g., computer program), which are stored in a memory and executed by a processor to accomplish the present invention. One or more of the modules/units may be a series of computer program instruction segments capable of performing specific functions for describing the execution of the computer program in the terminal.

The processor may be a central processing unit (Central Processing Unit, CPU), or other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc., or the general purpose processor may be a microprocessor, or any conventional processor, which is a control center of the terminal, that interfaces and lines to connect the various parts of the terminal.

The memory mainly includes a program storage area, which may store an operating system, an application program required for at least one function, and the like, and a data storage area, which may store related data and the like. In addition, the memory may be a high-speed random access memory, a nonvolatile memory such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, or the memory may be other volatile solid-state memory devices.

It should be noted that the above terminal may include, but is not limited to, a processor, a memory, and those skilled in the art will appreciate that the above terminal is merely an example and is not limited to the terminal, and may include more or fewer components, or may combine certain components, or different components.

Compared with the prior art, the embodiment of the invention has the following beneficial effects:

the invention provides a commodity identification and commodity price tag matching method, a system, a terminal and a medium, wherein commodity identification and price tag detection are carried out on pictures to be identified to obtain position information of all commodities to be matched, position information and matching direction information of all the price tags to be matched, and then the position information and the matching direction information of all the price tags to be matched and the position information of all the commodities to be matched are subjected to comparison analysis, so that the matching degree between all the price tags to be matched and all the commodities to be matched can be evaluated, the matching score between all the price tags to be matched and all the commodities to be matched is obtained, and the commodities to be matched with the matching degree equal to zero are screened out, namely the commodities to be matched which are not associated with the labels to be matched are screened out, so that the subsequent data processing amount is reduced, and the overall commodity price tag matching efficiency is improved. In addition, global self-adaptive optimization is carried out on the candidate sets corresponding to the price to be matched and the matching scores of all the commodities to be matched in the candidate sets, feedback of the matching scores of the candidate sets and the commodities to be matched can be considered in each iteration, the matching scores of the commodities to be matched and the commodities to be matched in the candidate sets are continuously optimized in a self-adaptive mode, after optimization is finished, the commodities to be matched with the current candidate sets corresponding to the price to be matched are selected as the optimal matching commodities of the current candidate sets, the optimal matching commodities are used as the commodity matching results of the commodities to be matched corresponding to the current candidate sets if the commodity types of the optimal matching commodities are preset types, otherwise, the commodity matching results of the commodities to be matched corresponding to the current candidate sets are non-matching objects, so that each price to be matched with the commodity with the highest matching degree can be ensured, accurate matching of the commodity to be achieved, and accurate commodity information is provided for users.

The foregoing embodiments have been provided for the purpose of illustrating the general principles of the present invention, and are not to be construed as limiting the scope of the invention. It should be noted that any modifications, equivalent substitutions, improvements, etc. made by those skilled in the art without departing from the spirit and principles of the present invention are intended to be included in the scope of the present invention.

Claims

1. A method for identifying and matching a commodity price tag, comprising:

2. The method for identifying and matching commodity price according to claim 1, wherein the matching scores of each candidate set and all the commodities to be matched in each candidate set are globally adaptively optimized, and after the optimization is finished, the commodities to be matched with the highest matching score are selected from the current candidate sets respectively as the current optimal matching commodity of each candidate set, if the commodity category of the optimal matching commodity is a preset category, the optimal matching commodity is used as the commodity matching result of the commodity to be matched corresponding to the current candidate set, otherwise, the commodity matching result of the commodity to be matched corresponding to the current candidate set is no matching object, specifically:

3. The method for identifying and matching commodity price according to claim 1, wherein said traversing each to-be-matched price, comparing and analyzing the position information and matching direction information of the current to-be-matched price and the position information of all the to-be-matched commodities to obtain matching scores between the current to-be-matched price and each to-be-matched commodity, and constructing a candidate set corresponding to the current to-be-matched price by using a plurality of to-be-matched commodities with matching scores different from 0, specifically:

4. The method for identifying and matching commodity price according to claim 3, wherein said traversing each price to be matched, and selecting a plurality of associated commodities corresponding to the current price to be matched from all the commodities to be matched according to the position information and matching direction information of the current price to be matched and the position information of all the commodities to be matched, specifically:

5. The method for matching a commodity identification to a commodity price according to claim 1, further comprising:

6. The method for identifying and matching commodity price tags according to claim 1, wherein the identifying and detecting commodity of the picture to be identified, and obtaining the position information of each commodity to be matched, the position information and the matching direction information of each price tag to be matched, specifically comprises the following steps:

7. The method for matching a commodity identification and a commodity price according to claim 6, wherein said price detection model specifically comprises: the device comprises a feature extraction layer, a feature fusion layer, a detection head and a direction head;

8. A merchandise identification and merchandise price tag matching system, comprising:

9. A terminal comprising a processor, a memory and a computer program stored in the memory; wherein the computer program is executable by the processor to implement a method of item identification and item price matching as claimed in any one of claims 1 to 7.

10. A computer readable storage medium, wherein the computer readable storage medium comprises a stored computer program; wherein the computer program, when run, controls the device in which the computer readable storage medium is located to perform a method for identifying and matching a commodity price according to any one of claims 1 to 7.