WO2020125207A1 - Information promotion method and apparatus - Google Patents

Abstract

An information promotion method and a related apparatus, the method comprising: acquiring a first image, the first image being an image photographed by a photographic apparatus corresponding to a display screen in a shopping mall (S101); detecting whether a trademark is present in the first image (S102); when detecting that a trademark is present, determining feature information of at least one first trademark in the first image (S103); looking up a second trademark matching the feature information of each first trademark in a trademark information database (S104); and displaying promotion information corresponding to the second trademark on the display screen, the promotion information comprising position information of the product corresponding to the second trademark in the shopping mall (S105). The technical solution of the present method can solve the problem of the effect of information promotion not being good enough.

Description

Information promotion method and device

[0001] The present invention relates to the field of information promotion, and in particular, to an information promotion method and device.

[0002] This application requires the priority of the Chinese patent application filed on December 21, 2018, with the application number 201811579857.8 and the invention titled "Information Promotion Method and Device", the entire contents of which are incorporated by reference in this application in.

Background technique

[0003] Trademarks are used by producers and operators of commodities on their production, manufacture, processing, selection or distribution of commodities or service providers on the services they provide, and are used to distinguish the source of commodities or services, including text , Graphics, letters, numbers, three-dimensional signs, color combinations and sounds, as well as the combination of the above-mentioned elements, signs with distinctive characteristics.

[0004] At present, the promotion information about trademarks displayed in shopping malls is generally fixed, and the promotion information seen by each consumer is the same. Because consumers have different interests and hobbies, many consumers promote the fixed broadcast promotion information. May not be interested, resulting in poor information promotion.

Summary of the invention

technical problem

Solution to the problem

Technical solution

[0005] Embodiments of the present invention provide information promotion methods and devices to solve the problem of poor information promotion effects.

[0006] The first aspect provides an information promotion method, including:

[0007] acquiring a first image, where the first image is an image captured by a shooting device corresponding to a display screen in a shopping mall

[0008] detecting whether a trademark exists in the first image;

[0009] in the case where a trademark is detected in the first image, determining feature information of at least one first trademark in the first image;

[0010] Find a second trademark matching the characteristic information of each first trademark in the trademark information database; [0011] The promotion information corresponding to the second trademark is displayed on the display screen, and the promotion information includes location information of commodities corresponding to the second trademark in the mall.

[0012] In a second aspect, an information promotion device is provided, including:

[0013] an acquisition module, configured to acquire a first image, where the first image is an image captured by a shooting device corresponding to a display screen in a shopping mall;

[0014] a detection module, configured to detect whether a trademark exists in the first image;

[0015] an information determination module, configured to determine the characteristic information of at least one first trademark in the first image if a trademark is detected in the first image;

[0016] a search module for searching a second trademark in the trademark information database that matches the feature information of each first trademark;

[0017] a display module, configured to display promotion information corresponding to the second trademark on the display screen, where the promotion information includes location information of commodities corresponding to the second trademark in the mall.

[0018] In a third aspect, an information promotion device is provided, including a processor, a memory, and a communication interface, where the processor, memory, and communication interface are connected to each other, wherein the communication interface is used to input or output data, the The memory is used to store program code, and the processor is used to call the program code to perform the method of the first aspect.

[0019] In a fourth aspect, a computer storage medium is provided, where the computer storage medium stores a computer program, the computer program includes program instructions, and the program instructions, when executed by a processor, perform the method of the first aspect described above .

[0020] In the embodiment of the present invention, by detecting the acquired first image, it is detected whether there is a trademark in the first image, and when a trademark is detected in the image, each first trademark in the first image is determined Search for the second trademark matching the characteristic information of each first trademark in the trademark information database, and display the promotion information corresponding to the second trademark on the display screen of the mall. The promotion information corresponding to the detected trademark is convenient for consumers to view and find the position of the product corresponding to the trademark in the mall. Since the first image is an image captured by a shooting device corresponding to the display screen in the mall, the image is displayed at the exhibition The possibility of the consumer’s image near the screen is high, so the detected trademark is likely to be a trademark on the consumer’s clothing or other parts (such as a backpack), and the consumer’s dress can be used to feedback the user to a certain extent. For product preferences, display promotional information that matches the trademark, It is equivalent to displaying the promotion information of the products that the user likes, and plays the role of targeted promotion information, that is, promotion according to the user's preference. Since promotion is based on the user's preference, accurate delivery of the promotion information can be achieved, which can make The effect of promotion is better.

Beneficial effects of invention

Brief description of the drawings

BRIEF DESCRIPTION

[0021] In order to more clearly explain the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some implementations of the present invention For example, for those of ordinary skill in the art, without paying any creative labor, other drawings can also be obtained based on these drawings.

[0022] FIG. 1 is a schematic diagram of a scenario provided by an embodiment of the present invention;

[0023] FIG. 2 is a schematic flowchart of an information promotion method provided by an embodiment of the present invention;

[0024] FIG. 3 is a schematic structural diagram of detecting a brand name according to an embodiment of the present invention;

[0025] FIG. 4 is a schematic flowchart of another information promotion method provided by an embodiment of the present invention;

[0026] FIG. 5 is a schematic diagram of a convolutional network structure in a target detection model provided by an embodiment of the present invention;

[0027] FIG. 6 is a schematic diagram showing the relationship between the target convolution feature map and the target convolution feature sub-map provided by an embodiment of the present invention;

[0028] FIG. 7 is a schematic diagram of a mapping relationship between a target convolution feature map and a first image provided by an embodiment of the present invention;

[0029] FIG. 8 is a schematic diagram of the relationship between three target convolution feature maps and corresponding target convolution feature sub-graphs obtained after convolution processing based on a target detection model provided by an embodiment of the present invention;

[0030] FIG. 9 is a schematic flowchart of yet another information promotion method provided by an embodiment of the present invention;

[0031] FIG. 10 is a schematic diagram of determining feature information of two first trademarks in a first image provided by an embodiment of the present invention;

[0032] FIG. 11 is a schematic diagram of determining regions corresponding to a plurality of first trademarks in a first image provided by an embodiment of the present invention;

[0033] FIG. 12 is a schematic structural diagram of an information promotion device according to an embodiment of the present invention;

[0034] FIG. 13 is a schematic structural diagram of another information promotion device provided by an embodiment of the present invention. Invention Example

Embodiments of the invention

[0035] The technical solution of the present invention is suitable for detecting an image, thereby obtaining a corresponding trademark detection result, and searching and displaying a scene of promotional information corresponding to the trademark in the trademark information database according to the trademark detection result. In a possible scenario, as shown in FIG. 1, when the shooting device 02 corresponding to the display screen 01 senses that there is a consumer directly opposite it, it periodically consumes the consumer directly opposite and within a preset distance Take pictures of different parts of the wearer’s clothing and send the captured images to the corresponding image processing device 03 for detection. When it is detected that the image contains a trademark, the trademark information database 04 is based on the detected trademark feature information Find the matching trademark, and determine the promotion information corresponding to the trademark, so that the display screen 01 displays the promotion information, so that consumers can better view the information corresponding to the trademark and find the product corresponding to the trademark in the mall. The shooting device may be a camera on the display screen, or may be another shooting device corresponding to the display screen, for example, a device with a photographing function, such as a personal computer, laptop computer, smart phone, tablet computer, and portable wearable device, period It can be any value such as 0.1s, 0.3s, 0.5s, etc. The preset distance can be any value such as 0.5m, lm, 1.5m.

[0036] Please refer to FIG. 2 and FIG. 2 is a schematic flowchart of an information promotion method provided by an embodiment of the present invention. As shown in the figure, the method includes the following steps:

[0037] S101: Acquire a first image, where the first image is an image captured by a shooting device corresponding to a display screen in a shopping mall.

[0038] In the embodiment of the present invention, the photographing device photographs different parts of the consumer's clothing, backpack, etc. to obtain the first image. Among them, the acquired first image may or may not include a trademark. For example, the acquired image may include a trademark and other decorative graphics or background graphics other than the trademark, as shown in FIG. 3, the "cloud" and "rain" shaped images are decorative backgrounds on the article pattern. The shooting device may be a camera above the display screen, as shown by 02 in FIG. 1, or other equipment with a photographing function.

[0039] S102, detecting whether a trademark exists in the first image.

[0040] In the embodiment of the present invention, as shown in FIG. 3, FIG. 3 is a schematic structural diagram of detecting a brand name provided by an embodiment of the present invention, and the first image may be adjusted to a preset size to detect the first image Whether there is a trademark in. Specifically, in the case where the size of the first image is larger than the preset size, the The first image is processed to reduce the first image to the same size as the preset size; in the case that the size of the first image is smaller than the preset size, the first image is processed by upsampling to make The first image is enlarged to the same size as the preset size. The preset size may be an integer multiple of 32*32, for example, the preset size may be 416*416

[0041] For a specific method of detecting whether a trademark exists in the first image, refer to the embodiment corresponding to FIG. 4, and details are not described here.

[0042] S103: In a case where a trademark is detected in the first image, determine feature information of at least one first trademark in the first image.

[0043] In the embodiment of the present invention, when it is detected that the trademark does not exist in the first image, the subsequent steps are not performed; when the trademark is detected in the first image, at least one of the first image is determined Characteristic information of the first trademark. Here, the first trademark is a trademark existing in the first image, and the number of the first trademark may be plural. The characteristic information of the first trademark includes the trademark name of the first trademark and the position coordinates of the first trademark in the first image. For example, when there are two first trademarks in the first image, the characteristic information of the first trademark in the first image includes the trademark names of the two first trademarks and the position coordinates of the two first trademarks in the first image. For example, as shown in FIG. 3, since there is only one trademark in FIG. 3, the characteristic information in the first image determined is the trademark name "NIKE" and the area 10c corresponding to the position coordinates of the trademark in the first image

[0044] S104, searching for a second trademark matching the feature information of each first trademark in the trademark information database.

[0045] In the embodiment of the present invention, since there may be multiple feature information of the first trademark in the first image determined in step S103, the matching information is searched for in the trademark information database according to the feature information of each first trademark. Two trademarks, and the feature information of each first trademark may match multiple second trademarks in the trademark information database, then there may be multiple second trademarks. Finding the second trademark matching the characteristic information of each first trademark in the trademark information database may include the following situations:

[0046] In the first case, a trademark with the same trademark name as the first trademark is searched in the trademark information database based on the trademark name of the first trademark, and the trademark with the same trademark name as the first trademark is determined as the second trademark.

[0047] In the second case, according to the target image corresponding to the position coordinates of the first trademark in the first image, the trademark information database is searched for a trademark pattern whose similarity with the target image is higher than the similarity threshold, and The trademark pattern whose similarity is higher than the similarity threshold is determined as the second trademark. According to the first trademark in the first The target image corresponding to the position coordinates in the image searches the trademark information database for a trademark pattern whose similarity to the target image is higher than the similarity threshold, and determines the trademark pattern as the second trademark. The similarity threshold may be any value such as 80%, 85%, and 90%.

[0048] In the third case, it is also possible to search for a matching trademark in the trademark information database according to the trademark name of the first trademark and the trademark pattern corresponding to the position coordinates of the first trademark in the first image, and search for the found trademark Determined as the second trademark.

[0049] Here, the trademark information database is used to store trademark information, and the trademark information may include a trademark name and a trademark pattern corresponding to the trademark, and the second trademark is a variety of trademarks stored in the trademark information database. It can be understood that the matching here refers to the same or the similarity reaching the similarity threshold can be considered as a match between the two, the similarity threshold can be any value such as 80%, 85%, 90%.

[0050] After finding the second trademark that matches the feature information of the first trademark in the first image, the promotion information corresponding to the second trademark in the trademark information database is determined. The promotion information may include the position information of the commodity corresponding to the second trademark in the shopping mall. The location information of the product corresponding to the second trademark in the shopping mall may specifically be: A rectangular coordinate system is established using the current terminal location as the coordinate origin, and the location information of the trademark in the rectangular coordinate system corresponds to the product corresponding to the second trademark. Location information in the mall. The promotion information may also include the trademark name of the second trademark, where the trademark name may be a Chinese name or an English name, and an abbreviation, etc. For example, the name of "NIKE" may include "NIKE" and "Nike". The promotion information may also include the trademark logo of the second trademark. The color of the trademark logo may be black and white, or it may be color. The pattern may be a flat graphic or a 3D graphic. The promotion information may also include the trademark introduction of the second trademark. The trademark introduction may be the basic information of the trademark, such as the source of the trademark, the moral of the trademark, the applicable group of the trademark, the price of the goods corresponding to the trademark, and the types of goods included in the trademark.

[0051] S105: Display promotion information corresponding to the second trademark on a display screen in a shopping mall, where the promotion information includes location information of commodities corresponding to the second trademark in the mall.

[0052] Step S104 determines the promotion information corresponding to the second trademark. When there are multiple second trademarks, there may be the following two ways to display promotion information corresponding to multiple second trademarks on the display screen of the mall:

[0053] In the first case, the priority may be set according to the order of acquiring the promotion information corresponding to each second trademark, so that the promotion information corresponding to a plurality of second trademarks may be switched and displayed in order. For example, when there are two second trademarks, the corresponding promotional information is also two, and the acquisition time of the promotional information corresponding to the second trademark A is at Before the promotion information corresponding to the second trademark B, priority is given to displaying the promotion information corresponding to the second trademark A within the time threshold. When the time threshold is reached, the promotion information corresponding to the second trademark B is displayed. When the time threshold is reached again, the display is displayed. The promotion information corresponding to the second trademark A is sequentially switched to display the promotion information corresponding to each second trademark. The time threshold can be any value such as 3S, 5S, and 10S.

[0054] In the second case, the promotion information corresponding to multiple second trademarks may be displayed on the display screen of the mall at the same time, for example, the display screen is divided into multiple areas, and the number of the areas corresponds to the number of promotion information corresponding to the second trademark Similarly, the area may be a plurality of areas with different sizes, so as to display promotion information corresponding to multiple second trademarks at the same time.

[0055] In the embodiment of the present invention, by detecting the acquired first image, it is detected whether there is a trademark in the first image, and when a trademark is detected in the image, at least one first trademark in the first image is determined Search for the second trademark matching the characteristic information of each first trademark in the trademark information database, and display the promotion information corresponding to the second trademark on the display screen of the mall. The promotion information corresponding to the detected trademark is convenient for consumers to view and find the position of the product corresponding to the trademark in the mall. Since the first image is an image captured by a shooting device corresponding to the display screen in the mall, the image is displayed at the exhibition The possibility of the consumer’s image near the screen is high, so the detected trademark is likely to be a trademark on the consumer’s clothing or other parts (such as a backpack), and the consumer’s dress can be used to feedback the user to a certain extent. For product preferences, displaying promotional information matching the trademark is equivalent to displaying the promotional information of the product that the user likes, and plays the role of targeted promotional information, that is, promotion according to the user's preferences, because it is based on the user's preferences Promoting can achieve accurate delivery of promotional information, which can make the promotion effect better.

[0056] In a possible implementation manner, the target detection model may be used to detect whether a trademark exists in the first image and determine the feature information of at least one first trademark in the first image, where the target detection model may be regression-based target detection The target detection model of the algorithm may also be a target detection model based on other algorithms, and the specific algorithm adopted by the target detection model is not limited herein. The specific steps of detecting whether there is a trademark in the first image through the target detection model and determining the characteristic information of the first trademark in the first image are shown in FIG. 4, which is a schematic flowchart of an information promotion method provided by an embodiment of the present invention. As shown in the figure, the method includes:

[0057] S201: Perform a convolution process on the first image based on the convolution layer in the target detection model to obtain a target convolution Feature map. The target convolution feature map includes multiple target convolution feature subgraphs.

[0058] In the embodiment of the present invention, the target detection model predicts the category and offset corresponding to the position frame (which position in the figure corresponds to the position frame) by using a convolution kernel on the feature map. The structure of the convolutional network in the target detection model includes multiple general convolutional layers, convolutional feature layers, residual units, upsampling units, and fusion units. It can be understood that in addition to the residual units, In addition to the convolutional layer included in the upsampling unit, the convolutional feature layer, and the fusion unit, the other convolutional layers can all be general convolutional layers. The general convolutional layer is a convolution that only plays a convolutional role in the convolutional network. Layer, the convolutional feature layer is used to generate a target convolutional feature image for image detection. Each convolutional layer corresponds to a convolution kernel of a different size. The convolution kernels of different sizes are used to convolve the image to obtain convolution maps of different sizes. Convolution kernels of different sizes correspond to different multiple prior frames. The a priori frame corresponding to the convolution kernel performs prediction processing on the convolution graph corresponding to the convolution kernel, and multiple bounding boxes can be obtained.

[0059] Here, the convolutional layer in the target detection model performs convolution processing on the first image to obtain multiple target convolution feature maps of different sizes, and the multiple target convolution feature maps of different sizes specifically refer to: The output results of the convolutional feature layers in the detection model, where each convolutional feature layer corresponds to multiple target convolutional feature maps of the same size, the smaller the size of the convolutional feature layer, the corresponding to the convolutional feature layer The greater the number of target convolutional feature maps. For example, the number of target convolution feature maps with size 64*64 is less than the number of target convolution feature maps with size 32*32, and the number of target convolution feature maps with size 32*32 is less than size 16* The number of target convolutional feature maps of 16, the number of target convolutional feature maps of size 16*16 is less than the number of target convolutional feature maps of size 8*8...

[0060] In a specific implementation, the size of the first image may be adjusted to the size of the input image corresponding to the target detection model (the size may be 416*416), and the size of the image is the size of the input image corresponding to the target detection model. Then input the image into the convolutional network of the target detection model, use the image as the input of the first convolutional layer in the convolutional network, and check it in sequence through the convolution corresponding to the convolutional layer in the convolutional network The output result of a convolutional layer is subjected to convolution processing, and then the output result of the convolutional feature layer in the convolutional network is determined as a plurality of target convolutional feature maps with different sizes. Among them, the convolution processing of the convolution layer corresponding to the convolution layer on the result of the previous convolution layer specifically means that the matrix corresponding to the convolution kernel is multiplied by the matrix output by the previous convolution layer, and the convolution layer The corresponding convolution kernel performs convolution processing on the result output by the previous convolution layer. The result obtained is a matrix of the size corresponding to the convolution layer. The corresponding image is the convolution map corresponding to the convolution layer.

[0061] The following describes the process of convolving the first image by the convolution layer in the target detection model. Referring to FIG. 5, FIG. 5 is a convolution network structure in the target detection model provided by the embodiment of the present invention. The schematic diagram of the target detection model includes multiple general convolutional layers, three convolutional feature layers, five residual units, two upsampling units, and two fusion units.

[0062] The first image is resized and input into a convolutional network, and the first image is convoluted by a convolution kernel in a general convolution layer to obtain a general convolution map, and the convolution in the first residual unit The layer performs a convolution process on the general convolution map to obtain a first convolution map, and performs a convolution process on the first convolution map through the convolution layer in the second residual unit to obtain a second convolution map, and a third convolution map. The convolutional layer in the residual unit performs a convolution process on the second convolution map to obtain a third convolution map, and the convolutional layer in the fourth residual unit performs a convolution process on the third convolution map to obtain a third convolution map. For the four convolutional graphs, the fifth convolutional graph is convoluted by the convolutional layer in the fifth residual unit to obtain a fifth convolutional graph. Perform a convolution process on the fifth convolutional map through the convolution kernel in the general convolutional layer to obtain a sixth convolutional map, and perform an upsampling process on the sixth convolutional map through an upsampling unit to obtain a first upsampled convolutional map , Fuse the first up-sampling convolution map and the fourth convolution map by the fusion unit to obtain the first fusion convolution map, and perform convolution processing on the first fusion convolution map by the convolution kernel in the general convolution layer to obtain the first Seven convolutional graphs, the seventh convolutional graph is upsampled by an upsampling unit to obtain a second upsampling convolutional graph, and the second upsampling convolutional graph and the third convolutional graph are fused by a fusion unit to obtain a second Fusion convolution map, convolution processing of the second fusion convolution map through the convolution kernel in the general convolution layer to obtain the eighth convolution map, and then the sixth convolution map, the seventh convolution map, the eighth volume The product graph is determined as the target convolution feature graph.

[0063] Among them, the number of the sixth convolution map, the seventh convolution map, and the eighth convolution map are multiple, and the number of the sixth convolution map, the seventh convolution map, and the eighth convolution map are in order Reduced, that is, the number of sixth convolution maps is greater than the number of seventh convolution maps, and the number of seventh convolution maps is greater than the eighth convolutional layer. That is, the target convolution feature maps obtained after processing are three different sizes of convolution maps, and the size and number of each convolution map are different. Here, there are multiple sixth convolution maps, and each sixth convolution map has the same size; there are multiple seventh convolution maps, each seventh convolution map has the same size; and there are multiple eighth convolution maps, Each eighth convolution graph has the same size.

[0064] One of the residual units processes the image as follows. Taking the first residual unit as an example, the input image is convoluted by the convolution kernel corresponding to the first convolution layer in the first residual unit , Get first Convolution sub-graph, the first convolution sub-layer convolution corresponding to the first convolution sub-convolution of the first convolution sub-graph to obtain a second convolution sub-graph, the first convolution sub-graph and The second convolution subgraph performs residual learning to obtain a first convolution graph corresponding to the first residual unit.

[0065] It should be noted that the examples here are only used to illustrate the process of using a convolution network to perform a convolution process on the first image, and do not limit the embodiment of the present invention. In an alternative embodiment, the convolution network may also It includes more general convolutional layers, convolutional feature layers, residual units, upsampling units and more fusion units.

[0066] Here, the target convolution feature submap refers to the feature unit included in the target convolution feature map, for example, the size of the target convolution feature map is 3*3, then the target convolution feature map may be as shown in FIG. 6, 6 is a schematic diagram of the relationship between a target convolution feature map and a target convolution feature sub-map provided by an embodiment of the present invention. The target convolution feature map includes a total of 9 feature units, and each feature unit is a target convolution feature map A cell is numbered 1-9, that is, the target convolution feature map contains 9 target convolution feature submaps.

[0067] S202: Determine target convolution feature information corresponding to each target convolution feature submap in multiple target convolution feature submaps, respectively.

[0068] Here, the target convolution feature information corresponding to each target convolution feature submap refers to: using the a priori box corresponding to the target convolution feature map as a bounding box to use the target convolution feature sub in the target convolution feature map The picture is the content circled in the center. For example, the convolution feature map is shown in FIG. 6. FIG. 6 is a schematic diagram of the relationship between the target convolution feature map and the target convolution feature sub-map provided by an embodiment of the present invention. For the target in the target convolution feature map Convolution feature sub-picture 9, the target convolution feature information corresponding to the target convolution feature sub-picture 9 is the content of the target convolution feature map corresponding to the three dashed boxes with different sizes in FIG. 6.

[0069] In a specific implementation, the a priori box corresponding to the target convolution feature map may be used as a bounding box to determine the information within the bounding box corresponding to each target convolution feature submap in the target convolution feature map, and the bounding box The information in is determined as the target convolution feature information of the target convolution feature submap corresponding to the bounding box, thereby determining the target convolution feature information corresponding to the target convolution feature map.

[0070] For example, if the target convolution feature map is as shown in FIG. 6, the target convolution feature information corresponding to the target convolution feature map may be: The prior frame corresponding to the convolution feature of size 3*3 is used as the boundary Box, centering the bounding box on feature unit 1 to determine the information corresponding to the bounding box, and determining the information corresponding to the bounding box as the target convolution feature information corresponding to feature unit 1, thereby determining the three boundaries corresponding to feature unit 1 The information corresponding to the box is the three target convolution feature information; the boundary box is centered on the feature unit 2 to determine the boundary box Corresponding information, the information corresponding to the bounding box is determined as the target convolution feature information corresponding to the feature unit 2, thereby determining that the information corresponding to the three bounding boxes corresponding to the feature unit 2 is the three target convolution feature information;... ...; The boundary box is centered on the feature unit 9 to determine the information corresponding to the boundary box, and the information corresponding to the boundary box is determined as the target convolution feature information corresponding to the feature unit 9, thereby determining the three corresponding to the feature unit 9 The information corresponding to each bounding box is three target convolution feature information; finally, the target convolution feature information corresponding to feature unit 1~feature unit 9 is determined as the target convolution feature information corresponding to the target convolution feature map, and 27 can be obtained Target convolution feature information.

[0071] S203, separately determining the matching probability between each target convolution feature information and multiple trademark names in the target detection model, and, the position coordinates corresponding to each target convolution feature information, and respectively in the first image The area corresponding to the position coordinates corresponding to each target convolution feature information is used as the first area corresponding to each target convolution feature information.

[0072] Specifically, the matching probability between each target convolution feature information and the trademark graphics corresponding to multiple trademark names in the target detection model can be calculated by a classifier in the target detection model, so as to determine each target volume separately The matching probability between the product feature information and the trademark graphics corresponding to various trademark names in the target detection model. The number of determined matching probabilities is the same as the number of trademark graphics corresponding to the trademark name in the target detection model.

[0073] Here, since each target convolution feature information is matched with a trademark graphic corresponding to each trademark name in the target detection model, each target convolution feature information corresponds to multiple matching probabilities, and the matching probability is The target convolution feature information is the possibility of trademark graphics corresponding to various trademark names in the target detection model.

[0074] Here, the position coordinates corresponding to each target convolution feature information refer to the position coordinates corresponding to the time when each bounding box corresponding to each target convolution feature information is mapped back to the first image, each target convolution feature information corresponds to four Position coordinates, these four position coordinates correspond to the four vertices of the bounding box, and the coordinates of the four points obtained by mapping the four vertices of the bounding box back to the original image are the position coordinates corresponding to the target convolution feature information, by This can obtain the position coordinates corresponding to the convolution feature information of each target.

[0075] Since each target convolution feature map is obtained from the first image through size adjustment and convolution processing, there is a corresponding relationship between each point in each target convolution feature map and a point or area in the first image, According to this correspondence, the position coordinates of each boundary box corresponding to four points in the first image can be determined, and then each The position coordinates of the four points corresponding to the bounding boxes in the first image are determined as the position coordinates corresponding to the respective target convolution feature information corresponding to each bounding box, and the area formed by the points corresponding to the position coordinates is determined as each target volume The first region corresponding to the product feature information, therefore, it can be obtained that each target convolution feature information corresponds to each first region in the first image.

[0076] For example, for example, the bounding box corresponding to the target convolution feature information is shown in FIG. 7, and FIG. 7 is a schematic diagram of the mapping relationship between the target convolution feature map and the first image provided by an embodiment of the present invention. , The four vertices of the bounding box are al, a2, a3, and a4 respectively, and the four vertices correspond to the points bl, b2, b3, and b4 when mapped back to the first image, and the position coordinates of bl in the first image are (bl l, bl2), b2's position coordinates in the first image are (b21, b22), b3's position coordinates in the first image are (b31, b34), b4's position coordinates in the first image are (b41, b44), then the position coordinates of bl (bl l, bl 2), b2 (b21, b22), b3 (b31, b32) and b4 (b41, b42) are determined as the target volume The position coordinates corresponding to the product feature information determine the region formed in the first image where the points bl, b2, b3, and b4 are located as the first region corresponding to the target convolution feature information.

[0077] In a specific implementation, the position coordinates corresponding to each target convolution feature information can be determined according to the mapping relationship between the target convolution feature map corresponding to the target convolution feature information and the first image, that is, the position coordinates can be determined Corresponding points in the first image, the area combined by the points is the first area.

[0078] S204: Determine the maximum matching probability corresponding to each target convolution feature information according to the matching probability between each target convolution feature information and multiple trademark names in the target detection model, respectively, and corresponding each target convolution feature information The maximum matching probability of is determined as the confidence of the first region corresponding to each target convolution feature information, and the trademark name corresponding to the maximum matching probability corresponding to each target convolution feature information is used as the corresponding to each target convolution feature information The brand name corresponding to the first area.

[0079] In the embodiment of the present invention, since each target convolution feature information is matched with a trademark graphic corresponding to each trademark name in the target detection model, each target convolution feature in the convolution feature information set The information corresponds to multiple matching probabilities, and the maximum matching probability among the multiple matching probabilities corresponding to each target convolution feature information is determined as the confidence of the first region corresponding to each target convolution feature information, then each target volume can be obtained The confidence of the first region corresponding to the product feature information, and using the brand name corresponding to the maximum matching probability corresponding to each target convolution feature information as the brand name corresponding to the first region corresponding to each target convolution feature information. [0080] Here, the confidence of the first area is the largest matching probability among the matching probabilities between the image corresponding to the first area and the trademark graphics corresponding to multiple trademark names in the target detection model.

[0081] Taking one target convolution feature information (ie, information within a bounding box) as an example for illustration, for example, the matching probability of the target convolution feature information A and the trademark pattern corresponding to the trademark name 1 in the target detection model is determined 0.80; the matching probability of the target convolution feature information A and the trademark graphic corresponding to the trademark name 2 in the target detection model is 0.20; the matching probability of the target convolution feature information A and the trademark graphic corresponding to the trademark name 3 in the target detection model Is 0.15, then the trademark pattern corresponding to the trademark name 1 in the target detection model of the target convolution feature information A has the maximum matching probability, and the maximum matching probability is 0.80. Then the maximum matching probability is the confidence of the first area, the confidence is 0.80, and the trademark name corresponding to the first area is the trademark name 1.

[0082] For ease of understanding, the following describes the process in steps S202 to S204 by way of example. For example, as shown in FIG. 8, FIG. 8 is obtained after convolution processing of a target detection model provided by an embodiment of the present invention. A schematic diagram of the relationship between the three target convolution feature maps and the corresponding target convolution feature submaps. Suppose that after processing the input image A by means of S201, three target convolution feature maps B1 and B2 of different sizes are obtained: , B3, assuming that the size of B1 is 3*3, the quantity is 10, and the numbers are 1~10, the size of B2 is 6*6, the quantity is 6, and the numbers are respectively

The size of B3 is 9*9, the quantity is 4, and the numbers are 17~20 respectively. A small square in C1 is a target convolution feature subgraph corresponding to the target convolution feature map in B1, a small square in C2 is a target convolution feature submap corresponding to the target convolution feature map in B2 In the figure, a small square in C3 is a target convolution feature submap corresponding to the target convolution feature map in B3. There are 9 target convolution feature subgraphs in the target convolution feature map numbered 1 in B 1, as shown in C1, each small square in C1 represents a target convolution feature submap, each target Convolutional feature submap corresponds to 3 target convolutional feature information (that is, a small square in C1 corresponds to 3 target convolutional feature information, and a target convolutional feature information corresponds to a bounding box), then the number 1 in B1 is The target convolution feature map corresponds to 27 (3*3*3=27) target convolution feature information, the target convolution feature map numbered 2 in B1 corresponds to 27 target convolution feature information, ... B1 The target convolution feature map numbered 10 in the map corresponds to 27 target convolution feature information. For the target convolution feature map B1 with a size of 3*3, since the number of B1 is 10, 270 (27*10=270) target convolution feature information can be obtained. It can be seen that there are 108 (6*6*3=108) target convolution feature information in the target convolution feature submap numbered 11 in the target convolution feature graph B2 of size 6*6,... , ruler There are 108 target convolution feature information in the target convolution feature sub-number of 16 in the target convolution feature map B2 with the size of 6*6. For the target convolution feature map B2 with the size of 6*6, due to B2 The number is 6, then 648 (108*6=648) target convolution feature information can be obtained. It can be seen that there are 243 (9*9*3=243) target convolution feature information in the target convolution feature subgraph numbered 17 in the target convolution feature graph B3 of size 9*9,... There are 243 target convolution feature information in the target convolution feature sub-picture numbered 20 in the target convolution feature map B3 of size 9*9. For the target convolution feature map B3 of size 9*9, Since the number of B3 is four, 972 (243*4=972) target convolution feature information can be obtained. Here, 270+648+972=1890 target convolution feature information corresponding to three different sizes of target convolution feature maps Bl, B2, and B3 can be obtained.

[0083] Assume that the 1890 target convolution feature information is target convolution feature information 1, ..., target convolution feature information 1890. First, determine the matching probability between the target convolution feature information 1 and the trademark graphics corresponding to multiple trademark names in the target detection model. For example, if there are 50 trademark graphics corresponding to the trademark name in the target detection model, 50 matching probabilities corresponding to the target convolution feature information 1 can be obtained, and the bounding box corresponding to the target convolution feature information 1 can be mapped back into the first image Is determined as the first region 1 corresponding to the target convolution feature information 1. Similarly, the 50 matching probabilities corresponding to the target convolution feature information 2 and the bounding box corresponding to the target convolution feature information 2 can be mapped back to the first region 2 in the first image, ..., the target can be determined The 50 matching probabilities corresponding to the convolution feature information 1890 and the bounding box corresponding to the target convolution feature information 1890 are mapped back to the first region 1890 in the first image.

[0084] Second, taking the target convolution feature information 1 as an example, the maximum matching probability among the 50 matching probabilities corresponding to the target convolution feature information 1 is determined as the confidence of the first region 1, and the maximum matching probability is determined The corresponding brand name is used as the brand name corresponding to the first region 1 to obtain the confidence of the first region 1 corresponding to the target convolution feature information 1 and the brand name corresponding to the first region 1. Similarly, the confidence of the first region 2 corresponding to the target convolution feature information 2 and the brand name corresponding to the first region 2 can be obtained, ..., the first region corresponding to the target convolution feature information 1890 can be obtained The confidence of 1890 and the brand name corresponding to the first area 1890.

[0085] Finally, 1890 first regions can be obtained, each of which has a corresponding confidence level and a corresponding trademark name.

[0086] S205. Determine whether the confidence of each first area is greater than a first threshold. [0087] Here, since the confidence of the first area is the largest matching probability among the matching probabilities between the image corresponding to the first area and the trademark graphics corresponding to multiple trademark names in the target detection model, the confidence of the first area is greater than The first threshold value indicates that the image corresponding to the first area matches the trademark graphics corresponding to multiple trademark names in the target detection model with a high probability of matching, that is, the image corresponding to the first area corresponds to multiple trademark names in the target detection model. Is more likely to have a trademark graphic; the confidence of the first area is less than the first threshold, indicating that the image corresponding to the first area is less likely to be a trademark graphic corresponding to multiple trademark names in the target detection model.

[0088] S206, when the confidence of at least one first area is greater than or equal to the first threshold, it is detected that a trademark exists in the first image.

[0089] S207, when the confidence of each first area is less than the first threshold, it is detected that there is no trademark in the first image.

[0090] As mentioned above, it is assumed that 1890 first regions are determined, and each first region has a corresponding confidence level. Assuming that the first threshold is 0.60, when there is at least one first region in the 1890 first regions When the confidence is greater than or equal to 0.60, it means that there is a trademark in the first image. When the confidence of each of the 1890 first regions is less than 0.60, it means that there is no trademark in the first image. Here, the first threshold may be any value such as 0.60, 0.75, 0.85, etc. So far, it is detected whether the trademark exists in the first image.

[0091] In the embodiment of the present invention, the image is detected by a target detection model, the convolutional network structure in the target detection model includes a residual unit, an upsampling unit, a fusion unit, a convolution feature layer, and a general convolution layer, The residual unit can enable the convolutional network structure to converge in deep conditions. The convolutional layer in the residual unit can compress the features corresponding to the image obtained by each convolution, reducing the amount of parameters and calculations in the model ; The upsampling unit upsamples the image so that the network layer number is very good when the network layer is deep; the fusion unit enables the network to learn deep and shallow features at the same time, so that the convolutional image corresponds to more features; volume The product feature layer outputs three convolutional images of different scales, so that three convolutional images of different scales can be detected, and whether a trademark exists in the image can be detected. Because the amount of parameters and calculations in the model are reduced during the detection process, the efficiency of trademark detection is improved, and the accuracy of trademark detection is improved by detecting three convolutional images of different scales.

[0092] Optionally, before using the above target detection detection model to detect the first image to determine the position coordinates of the first trademark in the first image and the trademark name corresponding to the first trademark, the target may also be trained first Detection model. When training the target detection model, first mark the training sample image, including marking the position of the trademark, trademark name and other information. Then use the marked sample images to train the initial model, and save the model when the model converges and reaches a certain accuracy (referring to the loss function value in the model is less than the loss threshold and the accuracy is greater than the accuracy threshold), the saved model is the target detection detection model.

[0093] In a possible implementation manner, in a case where a trademark is detected in the first image, a method for determining feature information of at least one first trademark in the first image is as follows, as shown in FIG. 9 and FIG. 9 This is a schematic flowchart of an information promotion method provided by an embodiment of the present invention. The method includes:

[0094] S301: Determine a set consisting of a first region with a confidence greater than or equal to a first threshold as a first set.

[0095] As mentioned above, it is assumed that 1890 first regions are determined, and each first region has a corresponding confidence level. Assuming that the first threshold is 0.60, there are 200 first regions in the 1890 first regions. If the confidence is greater than or equal to 0.60, then the first set is the set consisting of the 200 first regions.

[0096] S302. In a case where the number of first regions in the first set is multiple, determine the first region with the highest confidence in the first set as the first target region.

For example, when the first set consists of {first area 1, first area 2, first area 3, and first area 4}, and the confidence of each first area is 0.42, 0.55, 0.85, 0.95, respectively , Then the first area 4 with a confidence level of 0.95 is determined as the first target area.

[0098] S303: When the number of first regions in the first set is one, determine the first region in the first set as the first target region.

[0099] For example, when the first set consists of {first region 1}, then the first region 1 is the first target region.

[0100] S304: Determine the position coordinates corresponding to the first target area and the brand name corresponding to the first target area as feature information of a first trademark.

[0101] For example, if the first target area is the first area 1, the position coordinates corresponding to the first area 1 and the trademark name corresponding to the first area 1 are determined as the characteristic information of the first trademark, that is, the first image Characteristic information of a first trademark.

[0102] Here, through steps S301-S304, an area where the first trademark in the first image is located in the first image is determined, which is the first target area, and by determining the position coordinates and trademark of the first target area The name can determine the position coordinates of the trademark in the first image and the name of the trademark, that is, the characteristic information of the first trademark in the first image. In some possible cases, there may be in the first image For multiple trademarks, after determining the characteristic information of one trademark, it is also possible to determine whether there is a second trademark in the first image through steps S305-S310, and determine the second trademark's presence if the second trademark exists Feature information.

[0103] S305: Determine a set composed of the first area after excluding the first target area in the first set as the second set.

[0104] Assuming that the first set consists of {first area 1, first area 2, first area 3, first area 4}, and the first target area is the first area 4, then the second set is {first Area 1, first area 2, first area 3}.

[0105] S306. Determine whether the degree of coincidence between each first area and the first target area in the second set is greater than or equal to a second threshold.

[0106] In the embodiment of the present invention, the degree of coincidence between each first area and the first target area in the second set may be determined by comparing the IOU, and the calculated degree of coincidence may be compared with the second threshold. I0U is also called the cross-combination ratio, that is, the degree of coincidence of the images in the two regions, and calculating the I0U of the first region 1 and the first region 2 specifically refers to calculating the coincidence of the first region 1 and the first region 2 in the first image Degree, the greater the I0U, the greater the degree of coincidence between the two regions, the smaller the I0U, the smaller the degree of coincidence between the two regions. The I0U of the first area 1 and the first area 2 is equal to the intersection of the first area 1 and the first area 2 divided by the union of the first area 1 and the first area 2, expressed by the formula: I0U=[area (C) fiarea (D) ]/ [area (C) Uarea (D) ], area (C) is the first area 1, and area (D) is the first area 2. In a specific implementation, the intersection ratio of the first area 1 and the first area 2 may be calculated according to the position coordinates of the first area 1 and the position coordinates of the first area 2. Here, the second threshold is a critical point for evaluating the degree of coincidence between two regions, and the second threshold may specifically be 0.60, 0.85, etc. When the I0U of the two regions is greater than the second threshold, it means that the overlap of the two regions is high.

[0107] S307, when the degree of coincidence between each first area and the first target area in the second set is less than the second threshold

To determine the second target area of the second set.

[0108] Here, the second target area of the second set is the first area with the highest confidence in the second set. For example, assuming that the second threshold is 0.65, the second set includes {first area A, first area B, first area C, first area D}, and the confidence levels are 0.62, 0.68, 0.75, 0.85, respectively. A target area is the first area E. When the coincidence of the first area A, the first area B, the first area C, the first area D and the first area E is less than 0.65, the second target area of the second set It is the first area D. [0109] S308, when the degree of coincidence between at least one first area and the first target area in the second set is greater than or equal to the second threshold, the degree of coincidence with the first target area in the second set is greater than or equal to the second threshold The first area of is determined as the second area, and the set of the first area after the second area is excluded from the second set is determined as the third set, and the third target area of the third set is determined.

[0110] Here, the third target area of the third set is the first area with the highest confidence in the third set. For example, assuming that the second threshold is 0.65, the second set includes {first region F, first region G, first region H, first region I}, and the confidence levels are 0.65, 0.69, 0.78, 0.86, respectively. A target area is the first area J, and the degree of coincidence of the first area F, the first area G, and the first area J is less than 0.65, and the degree of coincidence of the first area H, the first area I, and the first area J is greater than 0.65, Then the second area includes the first area H and the first area I, then the third set is a set of the first area after the second area is excluded from the second set, that is, the third set is {first area F, first Area G}, the third target area is the first area G in the third set.

[0111] Optionally, when the degree of coincidence of each first region and the first target region in the second set is greater than or equal to the second threshold, it is determined that the second target region does not exist in the second set, That is, it is determined that the second trademark does not exist in the first image.

[0112] S309: Determine the position coordinates corresponding to the second target area and the brand name corresponding to the second target area as feature information of another first trademark.

[0113] It is determined through steps S305-S308 that there is another first trademark in the first image, that is, there are two first trademarks in the first image, and then the second target area corresponding to the second trademark in the first image The corresponding position coordinates and the trademark name corresponding to the second target area are determined as the characteristic information of the second trademark.

[0114] If it is determined in step S307 that the second target area of the second set is the first area D, then the position coordinates corresponding to the first area D and the trademark name corresponding to the first area D are determined as the characteristics of the second trademark information.

[0115] S310: Determine the position coordinates corresponding to the third target area and the brand name corresponding to the third target area as feature information of another first trademark.

[0116] Here, it can be known that there are two first trademarks in the first image, and the characteristic information of the other first trademark is the characteristic information of the second first trademark in the first image. Assuming that the third target area is determined to be the first area G, the position coordinates corresponding to the first area G and the trademark name corresponding to the first area G are determined as the characteristic information of another first trademark, that is, the second first trademark Characteristic information.

[0117] For example, the second target area and the third target area determined in steps S308-S309 are the first image The area corresponding to another first trademark in the first image (that is, the second first trademark in the first image), and determining the characteristic information of the second first trademark in the first image according to the target area may be as shown in FIG. 10, FIG. 10 is a schematic diagram of determining feature information of two first trademarks in a first image provided by an embodiment of the present invention. Since there are two first trademarks in the figure, the characteristics of one first trademark (first trademark) The information may be the trademark name “NIKE” in FIG. 10 and the area 10e corresponding to the position coordinates of the trademark in the first image, and the characteristic information of another first trademark (second first trademark) may be the trademark in FIG. 10 An area 10d corresponding to the name "Xtep" and the position coordinates of the trademark in the first image.

[0118] Here, through steps S305-S310, an area where another first trademark (the second first trademark) in the first image is located in the first image is determined, and the area is the second target area, which is determined by The position coordinates and the brand name of the second target area can determine the characteristic information of another first trademark (second first trademark) in the first image. Similarly, you can also refer to the specific implementation manner of steps S305-S310 to determine whether there are more first trademarks in the first image and to determine more feature information of the first trademark, until the first image is sequentially determined Characteristic information of all first trademarks. The following describes the process of sequentially determining the characteristic information of all the first trademarks in the first image by way of example.

[0119] Referring to FIG. 11, FIG. 11 is a schematic diagram of determining regions corresponding to multiple first trademarks in a first image provided by an embodiment of the present invention. First, assuming that the second threshold is 0.60, the determined first There are 9 first areas in the first set in the image (the dotted frame in the figure represents the first area), namely A, B, C, D, E, F,

G, H, I, the confidence levels are 0.60, 0.62, 0.65, 0.73, 0.95, 0.75, 0.65, 0.68, 0.60, then the first area E with the confidence level of 0.95 is determined as the first target area, and the second set is In the first set, the set consisting of the first area after excluding the first target area, that is, the second set consists of A, B, C, D, F,

G, H, I composition. Then, the first target area E is an area where a first trademark in the first image is located, and then the area where the first first trademark in the first image is located is determined.

[0120] Second, the degree of coincidence of A, B, C, D, F, G, H, I, and E in the second set is separately determined. When the I0U of A, B, C, D, F, G, H, I, and E are all less than 0.60, the second target area of the second set is F, and the second target area F is the second in the first image (The other) the area where the first trademark is located, then the area where the second first trademark in the first image is located is determined; when A, B, C, D, F, G, H, I in the second set I0U of at least one of the first region and E is greater than or equal to 0.60, assuming that A,

The I0U of B, C, D, F, G, H, and E are greater than or equal to 0.60, then the second area includes A, B, C , D, F, G, H, then the third set is the set of the first area excluding the second area in the second set, that is, the third set is I, the third target area is I, and the third target area I is the area where the second first trademark in the first image is located, and then the area where the second first trademark in the first image is located is determined.

[0121] Assuming that the IOUs of A, B, C, D, and E in the second set are greater than or equal to 0.60, the second area includes A,

B, C, D, the third set is the set after the second set excludes the second area, that is, the third set consists of F, G,

It consists of H and I, and the third target area is the area with the highest confidence in the third set, namely F. Then, the area after excluding F in the third area is determined as the fourth set. Since there are multiple first areas in the fourth set, continue to judge the coincidence of G, H, I in the fourth set with the third target area F Degree, assuming that the coincidence of G, H, I, and F are all greater than 0.60, then the third area includes G, H, and I. The fourth set is the empty set after excluding the third area, and the third target area F is the first The area where the third first trademark in the image is located, then the area where the third first trademark in the first image is located is determined. Finally, it is determined that the areas corresponding to the three first trademarks in the first image, that is, the first target area, the second target area, and the third target area, that is, there are three first trademarks in the first image.

[0122] Finally, the position coordinates and trademark names corresponding to the three target regions corresponding to the three first trademarks existing in the first image are determined as the characteristic information of the three first trademarks corresponding to the first image, that is, the first target The position coordinates and trademark names corresponding to the area are determined as the characteristic information of the first first trademark, the position coordinates and trademark names corresponding to the second target area are determined as the characteristic information of the second first trademark, and the third target area is corresponding The position coordinates and trademark name are determined as the characteristic information of the third first trademark. So far, the characteristic information of all the first trademarks in the first image is determined.

[0123] In the embodiment of the present invention, when a trademark is detected in the image, the duplicate trademark at the same position is removed by comparing the overlapping degree, the area where the trademark is located in the image and the trademark name corresponding to the trademark are determined, and the overlap is compared by comparing The degree method removes duplicate trademarks and improves the speed and accuracy of trademark detection.

[0124] The method of the embodiment of the invention is described above, and the device of the embodiment of the invention is described below.

[0125] Referring to FIG. 12, FIG. 12 is a schematic structural diagram of an information promotion device provided by an embodiment of the present invention. The device 50 includes:

[0126] an acquisition module 501, configured to acquire a first image, where the first image is an image captured by a photographing device corresponding to a display screen in a shopping mall;

[0127] The detection module 502 is configured to detect whether a trademark exists in the first image; [0128] The information determining module 503 is configured to determine the characteristic information of at least one first trademark in the first image when it is detected that the trademark exists in the first image;

[0129] A search module 504, configured to search for a second trademark matching the characteristic information of each first trademark in the trademark information database;

[0130] The display module 505 is configured to display promotion information corresponding to the second trademark on the display screen, where the promotion information includes position information of the commodity corresponding to the second trademark in the mall.

[0131] In a possible design, the characteristic information of each first trademark includes the trademark name of the first trademark and the position coordinates of the first trademark in the first image, the search module 504 is specifically configured to The name of the first trademark is searched for in the trademark information database with the same name as the first trademark, and the trademark with the same name as the first trademark is determined as the second trademark; or, based on A target image corresponding to the position coordinates of the first trademark in the first image, search for a trademark pattern in the trademark information database whose similarity to the target image is higher than a similarity threshold, and compare the The trademark pattern whose similarity of the target image is higher than the similarity threshold is determined as the second trademark.

[0132] In a possible design, the display module 505 is specifically configured to switch and display the plurality of second trademarks in order on a display screen of a shopping mall in the case of multiple second trademarks The promotion information corresponding to the trademark, or, when there are multiple second trademarks, the promotion information corresponding to the multiple second trademarks are displayed simultaneously on the display screen of the mall.

[0133] In a possible design, the detection module 502 includes:

[0134] The processing unit 5021 is configured to perform convolution processing on the first image based on the convolution layer in the target detection model to obtain a target convolution feature map, where the target convolution feature map includes multiple target convolution feature sub Figure;

[0135] The determining unit 5022 is configured to determine target convolution feature information corresponding to each target convolution feature sub-graph in the multiple target convolution feature sub-graphs, respectively;

[0136] The determining unit 5022 is further configured to determine a matching probability between each target convolution feature information and a plurality of trademark names in the target detection model, and a position corresponding to each target convolution feature information Coordinates, and the area corresponding to the position coordinates corresponding to the respective target convolution feature information in the first image is used as the first area corresponding to the respective target convolution feature information;

[0137] The matching unit 5023 is configured to determine the maximum corresponding to each target convolution feature information according to the matching probability between each target convolution feature information and multiple trademark names in the target detection model, respectively Matching probability, respectively determining the maximum matching probability corresponding to each target convolution feature information as the confidence of the first region corresponding to each target convolution feature information, and respectively corresponding to each target convolution feature information The brand name corresponding to the maximum matching probability is used as the brand name corresponding to the first region corresponding to each target convolution feature information;

[0138] The determining unit 5024 is configured to determine whether the confidence of each first area is greater than a first threshold;

[0139] The judgment unit 5024 is further configured to detect the presence of a trademark in the first image when the confidence of at least one first area is greater than or equal to the first threshold;

[0140] The judging unit 5024 is further configured to detect that there is no trademark in the first image when the confidence of each first area is less than the first threshold.

[0141] In a possible design, the matching unit 5023 is specifically configured to:

[0142] determine a set consisting of a first region with a confidence greater than or equal to the first threshold as the first set;

[0143] In a case where the number of first regions in the first set is multiple, the first region with the highest confidence in the first set is determined as the first target region;

[0144] When the number of first regions in the first set is one, determine the first region in the first set as the first target region;

[0145] determining the position coordinates corresponding to the first target area and the trademark name corresponding to the first target area as the first feature information, and determining the first feature information as a feature of a first trademark Information

[0146] In a possible design, the matching unit 5023 is specifically configured to:

[0147] determining the set consisting of the first area after excluding the first target area from the first set as the second set;

[0148] determine whether the degree of coincidence of each first area in the second set with the first target area is greater than or equal to a second threshold;

[0149] When the degree of coincidence between each first area and the first target area in the second set is less than the second threshold, determine the second target area of the second set;

[0150] when the degree of coincidence of at least one first area in the second set with the first target area is greater than or equal to the second threshold, the The first area where the degree of coincidence is greater than or equal to the second threshold is determined as the second area, and the second set is excluded from the second set The set consisting of the first area after the area is determined as the third set, and the third target area of the third set is determined;

[0151] determining the position coordinates corresponding to the second target area and the trademark name corresponding to the second target area as the characteristic information of another first trademark, or determining the position coordinates and the corresponding position of the third target area The trademark name corresponding to the third target area is determined to be another first trademark.

[0152] It should be noted that, for the content that is not mentioned in the embodiment corresponding to FIG. 12, reference may be made to the description of the method embodiment, and details are not repeated here.

[0153] In the embodiment of the present invention, the acquired first image is detected by a target detection model, the convolutional network structure in the target detection model includes a residual unit, an upsampling unit, a fusion unit, a convolution feature layer, and In general convolutional layers, the residual unit can make the convolutional network structure converge in a deep situation. The convolutional layer in the residual unit can compress the features corresponding to the image obtained by each convolution, reducing the model. The amount of parameters and the amount of calculation; the upsampling unit upsamples the image so that the network layer number is very well expressed in deep conditions; the fusion unit enables the network to learn deep and shallow features at the same time, so that the convolved image corresponds to More features; the convolutional feature layer outputs three convolutional images of different scales, so that three convolutional images of different scales can be detected, which improves the efficiency and accuracy of trademark detection. In the case of trademarks, the duplicate trademarks are removed by comparing the coincidences, the characteristic information of each first trademark in the first image is determined, and the trademark information database is searched for matching the characteristic information of each first trademark The second trademark, and display the promotion information corresponding to the second trademark on the display screen of the mall. By displaying the promotion information corresponding to the detected trademark on the mall display screen, it is convenient for consumers to view and find the products corresponding to the trademark in the mall In the location, since the first image is an image captured by a shooting device corresponding to the display screen in the mall, the image is likely to be an image of the consumer near the display screen, then the detected trademark is likely to be The trademark on the consumer's clothing or other parts (such as a backpack), and the consumer's clothing can be used to feedback the user's preference for the product to a certain extent, and display promotional information matching the trademark, which is equivalent to displaying the user's favorite product The promotion information has played the role of targeted promotion information, that is, promotion according to the user's preference, because the promotion is based on the user's preference, can achieve accurate delivery of promotion information, which can make the promotion effect better.

[0154] Referring to FIG. 13, FIG. 13 is a schematic structural diagram of an information promotion device provided by an embodiment of the present invention, The device includes a processor 601, a memory 602, and a communication interface 603. The processor 601 is connected to the memory 602 and the communication interface 603. For example, the processor 601 may be connected to the memory 602 and the communication interface 603 through a bus.

[0155] The processor 601 is configured to support the information promotion apparatus to perform the corresponding functions in the information promotion method described in FIGS. 2, 4, and 9. The processor 601 may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The hardware chip may be an application specific integrated circuit (ASIC), a programmable logic device (programmable logic device, PLD), or a combination thereof. The above PLD may be a complex programmable logic device (CPLD), field-programmable gate array (field-programmable gate array)

array, FPGA), generic array logic (GAL) or any combination thereof.

[0156] The memory 602 is used for storing program codes and the like. The memory 602 may include a volatile memory (vo latile memory, VM), such as a random access memory (random access memory, RAM)

The memory 602 may also include non-volatile memory (non-volatile memory, NVM), such as read-only memory (read-only memory, ROM), flash memory (flash memory), hard disk (hard disk drive, HDD) or Solid-state drive (solid-state drive, SSD); The memory 602 may also include a combination of the aforementioned types of memory.

[0157] The communication interface 603 is used to send or receive data.

[0158] The processor 601 may call the program code to perform the following operations:

[0159] acquiring a first image through the communication interface 603, the first image being an image captured by a shooting device corresponding to a display screen in a shopping mall;

[0160] detecting whether a trademark exists in the first image;

[0161] in the case where a trademark is detected in the first image, determining the characteristic information of at least one first trademark in the first image;

[0162] Find the second trademark matching the characteristic information of each first trademark in the trademark information database;

[0163] The promotion information corresponding to the second trademark is displayed on the display screen, and the promotion information includes location information of the commodity corresponding to the second trademark in the mall.

[0164] It should be noted that the implementation of each operation may also correspond to the method shown in FIG. 2, FIG. 4, and FIG. 9. Corresponding description of the embodiment; the processor 601 can also cooperate with the communication interface 603 to perform other operations in the above method embodiments.

[0165]

Claims

Claims

[Claim i] An information promotion method, characterized in that it includes:

Acquiring a first image, where the first image is an image captured by a shooting device corresponding to a display screen in a shopping mall;

Detecting whether a trademark exists in the first image;

When it is detected that a trademark exists in the first image, determine the feature information of at least one first trademark in the first image; find a second matching the feature information of each first trademark in the trademark information database A trademark; displaying promotion information corresponding to the second trademark on the display screen, where the promotion information includes location information of commodities corresponding to the second trademark in the mall.

[Claim 2] The method according to claim 1, wherein the characteristic information of each first trademark includes the trademark name of the first trademark and the position coordinates of the first trademark in the first image

The searching for the second trademark matching the characteristic information of each first trademark in the trademark information database includes:

Searching for a trademark having the same name as the first trademark in the trademark information database according to the name of the first trademark, and determining the trademark having the same name as the first trademark to be the second trademark; or,

According to the target image corresponding to the position coordinates of the first trademark in the first image, search for a trademark pattern in the trademark information database whose similarity to the target image is higher than a similarity threshold, and compare the The trademark pattern whose similarity of the target image is higher than the similarity threshold is determined as the second trademark.

[Claim 3] The method according to claim 1, wherein the displaying of the promotion information corresponding to the second trademark on the display screen includes:

When there are multiple second trademarks, the promotion information corresponding to the multiple second trademarks is switched and displayed in sequence on the display screen, or,

When there are multiple second trademarks, the promotion information corresponding to the multiple second trademarks is displayed simultaneously on the display screen.

[Claim 4] The method according to claim 1, wherein the detecting whether a trademark exists in the first image includes:

Performing a convolution process on the first image based on the convolution layer in the target detection model to obtain a target convolution feature map, where the target convolution feature map includes multiple target convolution feature submaps; each of the multiple Target convolution feature information corresponding to each target convolution feature subgraph in the target convolution feature subgraph;

Separately determine the matching probability between each target convolution feature information and a plurality of trademark names in the target detection model, and, the position coordinates corresponding to each target convolution feature information, and respectively in the first image The area corresponding to the position coordinates corresponding to the respective target convolution feature information is used as the first area corresponding to the respective target convolution feature information according to the plurality of target convolution feature information and the target detection model respectively. The matching probability between brand names, determining the maximum matching probability corresponding to each target convolution feature information, and determining the maximum matching probability corresponding to each target convolution feature information as the corresponding The confidence of the first area, and the trademark name corresponding to the maximum matching probability corresponding to each target convolution feature information is used as the trademark name corresponding to the first area corresponding to each target convolution feature information; Whether the confidence of each first area is greater than a first threshold; when the confidence of at least one first area is greater than or equal to the first threshold, it is detected that a trademark exists in the first image;

When the confidence of each of the first regions is less than the first threshold, it is detected that there is no trademark in the first image.

[Claim 5] The method according to claim 4, wherein the determining the characteristic information of at least one first trademark in the first image includes:

Determining a set consisting of first regions with a confidence greater than or equal to the first threshold as the first set;

When the number of first regions in the first set is multiple, determine the first region with the highest confidence in the first set as the first target region; When the number of first regions in the first set is one, determine the first region in the first set as the first target region;

The position coordinates corresponding to the first target area and the trademark name corresponding to the first target area are determined as the characteristic information of a first trademark.

[Claim 6] The method according to claim 5, wherein the determining the characteristic information of at least one first trademark in the first image further comprises:

Determining the set consisting of the first area after excluding the first target area in the first set as the second set;

Determine whether the degree of coincidence of each first area in the second set with the first target area is greater than or equal to a second threshold;

When the degree of coincidence between each first area in the second set and the first target area is less than the second threshold, determine the second target area in the second set;

When the degree of coincidence of at least one first area in the second set with the first target area is greater than or equal to the second threshold, the degree of coincidence with the first target area in the second set is greater than Or a first area equal to the second threshold is determined as a second area, and a set of the first area after excluding the second area in the second set is determined as a third set, and the first area is determined Three sets of third target areas; determining the position coordinates corresponding to the second target area and the trademark name corresponding to the second target area as feature information of another first trademark, or corresponding to the third target area The position coordinates of and the brand name corresponding to the third target area are determined to be characteristic information of another first brand.

[Claim 7] An information promotion device, characterized in that it includes:

An obtaining module, configured to obtain a first image, where the first image is an image captured by a shooting device corresponding to a display screen in a shopping mall;

A detection module, configured to detect whether a trademark exists in the first image;

The information determination module is used to determine the characteristic information of at least one first trademark in the first image when a trademark is detected in the first image; the search module is used to search for and search for each trademark in the trademark information database Feature information matching of the first trademark 'S second trademark;

A display module, configured to display promotion information corresponding to the second trademark on the display screen, where the promotion information includes position information of commodities corresponding to the second trademark in the mall

[Claim 8] The apparatus according to claim 7, wherein the detection module includes: a processing unit configured to perform convolution processing on the first image based on the convolution layer in the target detection model, Obtaining a target convolution feature map, where the target convolution feature map includes multiple target convolution feature subgraphs;

A determining unit, configured to separately determine target convolution feature information corresponding to each target convolution feature subgraph in the multiple target convolution feature subgraphs;

The determining unit is further configured to separately determine the matching probability between each target convolution feature information and multiple trademark names in the target detection model, and, the position coordinates corresponding to each target convolution feature information, and respectively Use the area corresponding to the position coordinates corresponding to the respective target convolution feature information in the first image as the first area corresponding to the respective target convolution feature information;

The matching unit is configured to determine the maximum matching probability corresponding to each target convolution feature information according to the matching probability between each target convolution feature information and multiple trademark names in the target detection model, respectively The maximum matching probability corresponding to each target convolution feature information is determined as the confidence of the first region corresponding to each target convolution feature information, and the corresponding maximum matching probability corresponding to each target convolution feature information is respectively The trademark name serves as the trademark name corresponding to the first area corresponding to each target convolution feature information;

A judging unit, used to judge whether the confidence of each first area is greater than a first threshold; the judging unit is also used to detect when the confidence of at least one first area is greater than or equal to the first threshold There is a trademark in the first image; the judgment unit is further configured to detect that there is no trademark in the first image when the confidence of each first area is less than the first threshold.

[Claim 9] The device according to claim 7, wherein the device further comprises a display module , For switching the display of promotion information corresponding to the plurality of second trademarks in sequence on the display screen when there are multiple second trademarks, or when there are multiple second trademarks In the case of, displaying promotion information corresponding to the plurality of second trademarks simultaneously on the display screen.

[Claim 10] The device according to claim 7, wherein the detection module includes a processing monolingual, a determination unit, a matching unit and a judgment unit, the processing unit is used for convolution based on the target detection model The layer performs convolution processing on the first image to obtain a target convolution feature map, where the target convolution feature map includes multiple target convolution feature submaps; the determining unit is configured to determine the multiple target convolutions respectively Target convolution feature information corresponding to each target convolution feature sub-picture in the product feature sub-picture; determining the matching probability between each target convolution feature information and multiple trademark names in the target detection model, and, The position coordinates corresponding to the target convolution feature information, and the regions corresponding to the position coordinates corresponding to the respective target convolution feature information in the first image are used as the first corresponding to the respective target convolution feature information Area; the matching unit is used to determine the maximum matching probability corresponding to each target convolution feature information according to the matching probability between each target convolution feature information and multiple trademark names in the target detection model, Determine the maximum matching probability corresponding to each target convolution feature information as the confidence of the first region corresponding to each target convolution feature information, and respectively determine the maximum matching probability corresponding to each target convolution feature information The corresponding brand name is used as the brand name corresponding to the first area corresponding to each target convolution feature information; the judgment unit is used to determine whether the confidence of each first area is greater than the first threshold; and when at least one When the confidence of the first area is greater than or equal to the first threshold, the presence of a trademark in the first image is detected; when the confidence of each of the first areas is less than the first threshold, the detection is performed There is no trademark in the first image.

[Claim 11] The apparatus according to claim 10, characterized in that the matching unit is further configured to determine a set consisting of a first region with a confidence greater than or equal to the first threshold as the first set; When the number of first regions in the first set is multiple, determine the first region with the highest confidence in the first set as the first target region; in the first set When the number of first regions in is one, determine the first region in the first set as the first target region; map the position coordinates corresponding to the first target region to the first target region The trademark name is determined as the characteristic information of the first trademark.

[Claim 12] The apparatus according to claim 11, wherein the matching unit is further configured to determine a set composed of a first area after excluding the first target area from the first set as the first Two sets; judging whether the degree of coincidence between each first area in the second set and the first target area is greater than or equal to a second threshold; when each first area in the second set and the first When the degree of coincidence of the target areas is less than the second threshold, determine the second target area of the second set; when the degree of coincidence of at least one first area in the second set with the first target area is greater than or When it is equal to the second threshold, determine the first area in the second set whose degree of coincidence with the first target area is greater than or equal to the second threshold as the second area, and determine the second set The set consisting of the first area after excluding the second area is determined as the third set, and the third target area of the third set is determined; the position coordinates corresponding to the second target area and the second The trademark name corresponding to the target area is determined to be characteristic information of another first trademark, or the position coordinates corresponding to the third target area and the trademark name corresponding to the third target area are determined to be characteristic information of another first trademark .

[Claim 13] An information promotion device, comprising a processor, a memory and a communication interface, wherein the processor, the memory and the communication interface are connected to each other, wherein the communication interface is used to input or output data, so The memory is used to store program code, and the processor is used to call the program code to execute the method according to any one of claims 1-6.

[Claim 14] A computer storage medium, characterized in that the computer storage medium stores a computer program, the computer program includes program instructions, and when executed by a processor, the program instructions are executed as claimed in claim 1- 6. The method of any one of the above.