WO2023045535A1 - 识别图片的方法和装置 - Google Patents

识别图片的方法和装置 Download PDF

Info

Publication number
WO2023045535A1
WO2023045535A1 PCT/CN2022/107821 CN2022107821W WO2023045535A1 WO 2023045535 A1 WO2023045535 A1 WO 2023045535A1 CN 2022107821 W CN2022107821 W CN 2022107821W WO 2023045535 A1 WO2023045535 A1 WO 2023045535A1
Authority
WO
WIPO (PCT)
Prior art keywords
picture
code
scanned
scanning
recognition result
Prior art date
Application number
PCT/CN2022/107821
Other languages
English (en)
French (fr)
Inventor
黄莹
黄星
廖群伟
陈景东
王剑
刘家佳
暨凯祥
胡锦华
刘雷
武琳娟
王昊
章鹏
李莎
卢睿
杜金泉
冯成林
张谦
苏煜
林楠
鞠春春
吕炯炯
朱伟
Original Assignee
蚂蚁区块链科技(上海)有限公司
支付宝(杭州)信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 蚂蚁区块链科技(上海)有限公司, 支付宝(杭州)信息技术有限公司 filed Critical 蚂蚁区块链科技(上海)有限公司
Publication of WO2023045535A1 publication Critical patent/WO2023045535A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
    • G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
    • G06K7/14—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
    • G06K7/1404—Methods for optical code recognition
    • G06K7/1408—Methods for optical code recognition the method being specifically adapted for the type of code
    • G06K7/1417—2D bar codes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/018—Certifying business or products
    • G06Q30/0185—Product, service or business identity fraud
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0207—Discounts or incentives, e.g. coupons or rebates
    • G06Q30/0208—Trade or exchange of goods or services in exchange for incentives or rewards
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0207—Discounts or incentives, e.g. coupons or rebates
    • G06Q30/0225—Avoiding frauds

Definitions

  • the present disclosure relates to the field of information security, in particular to a method and device for identifying pictures.
  • the brand In the marketing activities of physical products, the brand usually prints an activity code on the physical product for users to scan and claim rewards after purchase.
  • activity codes are often stolen and sold by black products. Once the activity code is redeemed by the wool party or the volume gang in the black industry (which can be referred to as fraud for short), it will bring huge losses to the brand.
  • the present disclosure provides a method and device for identifying pictures, so as to accurately identify fraud and effectively prevent black products from using activity codes to claim prizes.
  • an identification method includes: receiving a scan code picture, the scan code picture contains an activity code corresponding to a product marketing activity; performing image recognition on the scan code picture to determine the scan code Whether the code picture is a target picture, and the target picture is a picture obtained after scanning the activity code on the entity of the commodity.
  • an identification device comprising: a receiving module, configured to receive a scan code picture, the scan code picture including an activity code corresponding to a product marketing activity; a determination module, configured to scan the code Image recognition is performed on the picture to determine whether the scanned code picture is a target picture, and the target picture is a picture obtained after scanning the activity code on the entity of the product.
  • an identification device including a memory and a processor, wherein executable code is stored in the memory, and the processor is configured to execute the executable code to implement the method as described in the first aspect .
  • a computer-readable storage medium on which executable code is stored, and when the executable code is executed, the method as described in the first aspect can be implemented.
  • a computer program product including executable code, and when the executable code is executed, the method as described in the first aspect can be implemented.
  • An embodiment of the present disclosure provides a method for identifying a picture, which receives a code-scanning picture that contains an activity code corresponding to a marketing activity of a product and performs image recognition on the code-scanning picture to determine whether the code-scanning picture is a product
  • the picture obtained after scanning the activity code on the entity.
  • Figure 1 is a schematic diagram of the operation process of the illegal industry chain.
  • Fig. 2 is a schematic flow chart of a method for identifying a picture provided by an embodiment of the present disclosure.
  • 3A and 3B are diagrams illustrating examples of active codes in an embodiment of the present disclosure.
  • Fig. 4 is another schematic flow chart of the method for identifying a picture provided by an embodiment of the present disclosure.
  • FIG. 5 is an example diagram of an architecture provided by an embodiment of the present disclosure.
  • Fig. 6 is a schematic structural diagram of a visual computing module for image recognition provided by an embodiment of the present disclosure.
  • Fig. 7 is a schematic structural diagram of a classification neural network provided by an embodiment of the present disclosure.
  • Fig. 8 is a schematic diagram of a fusion structure of a tamper detection network provided by an embodiment of the present disclosure.
  • Fig. 9 is a schematic structural diagram of a duplication detection module provided by an embodiment of the present disclosure.
  • Fig. 10 is a schematic diagram of a prevention and control scheme for an activity code on a commodity entity provided by an embodiment of the present disclosure.
  • Fig. 11 is a schematic structural diagram of an apparatus for identifying pictures provided by an embodiment of the present disclosure.
  • Fig. 12 is another schematic structural diagram of an apparatus for identifying pictures provided by an embodiment of the present disclosure.
  • the commodities mentioned in the embodiments of the present disclosure refer to all commodities with entities, and the commodities are regular commodities of the brand owner.
  • the commodity can be drinks, daily necessities, food, toys, etc.
  • Marketing activities refer to the activities carried out by the brand side to promote the sales of goods.
  • a marketing activity can be any kind of promotional activity that includes benefits, for example, it can be to buy a product and get a gift, or it can be to buy a product to get a red envelope or various coupons, or it can also be to collect points for redemption after buying multiple products prize.
  • the brand side will make the marketing activities of the products into activity codes, and print the activity codes on the products. When consumers scan the activity code after purchasing the product, they can obtain the benefits included in the marketing activity.
  • the embodiment of the present disclosure does not specifically limit the category of the activity code, as long as the activity code can be identified and its corresponding marketing activity can be interpreted.
  • the activity code can be a QR code or a bar code.
  • Black production can also be called cyber black production.
  • Cyber black production refers to illegal activities that use the Internet as the medium and network technology as the main means to pose potential threats to the security of computer information systems and social and economic stability. For example, online illegal activities such as redeeming prizes, cashing out, and rebates.
  • Figure 1 shows the operation process of the black industry chain.
  • the main operators of illegal products will collect the physical activity codes of the goods discarded by consumers from waste recycling stations, plastic crushing plants, crowd gathering places, cleaners, etc., or will collect them during the production, transmission and processing of activity codes.
  • the activity code was stolen from the network, and the activity code was sold on the online platform. Once the activity code is redeemed by the wool party or the volume gang in the black industry (which can be referred to as fraud for short), it will bring huge losses to the brand.
  • the list is invalid due to scene dislocation.
  • the existing blacklists are telecom blacklists or credit blacklists, and telecom fraudsters or credit lads will also buy beer and beverages, which belong to real consumers.
  • Using this type of blacklist for marketing anti-fraud scenarios will lead to erroneous matching problem.
  • the evolution of black industry attacks has caused the list to become invalid. For example, in the past, the IP of the hacker attack was the proxy IP, the account number was the small account, the verification code was the code slave, the equipment was the installation farm, and the trader was the studio.
  • Another feasible way in the existing technology is to obtain the number of code scans of users who scan codes to redeem prizes, and judge whether the number of code scans of users exceeds the limit through the preset code scanning rules, so as to prevent scams in illegal activities. A large number of gangs redeemed a large number of prizes.
  • black industry gangs have a large number of accounts and IP pools, the restriction on the number of scans for accounts and IPs only increases the cost of fighting against black industries, and will not prevent black industries from illegally obtaining marketing funds. For example, for every 10 activity codes scanned by black products, one account is exchanged, which is used to cross the rules.
  • black products in order to facilitate storage and sales, black products generally convert the activity codes corresponding to the marketing activities of commodities into pure electronic codes or analog codes and sell them in the form of pictures. That is to say, what black products sell is pictures, and the pictures contain activity codes. If the wool party and the amount-swiping team in the black industry want to redeem the activity codes in the black industry, they need to scan the pictures in the black industry, or print out the pictures in the black industry for scanning. Or in extreme cases, black products can directly use their pictures to claim prizes.
  • the real consumer scans the activity code on the product entity. If the scanned code picture from the real consumer can be distinguished from the scanned code picture from the black industry on the code scanning link, then The problems existing in the above-mentioned prior art can be avoided.
  • the embodiment of the present disclosure provides a new identification scheme, which determines whether the scanned code picture is the picture obtained by scanning the active code on the entity of the commodity by performing image recognition on the scanned code picture, so as to Accurately identify fraud on this code scanning link, effectively preventing black products from using activity codes to redeem prizes.
  • step S210 the scanned code picture is received.
  • the code scanning picture can also be called a prize redemption picture.
  • the scan code image contains the activity code corresponding to the marketing activity of the product.
  • the code-scanning picture may be a picture obtained when the user's mobile device scans the activity code. Therefore, the code-scanning picture may also include a background area (also called a no-code area). The background area can contain other features in the scanned image other than the active code.
  • the scanned code picture can be one of the following types: the picture obtained by scanning the activity code on the physical product; the picture obtained by scanning the picture in the black product and the picture in the black product.
  • the picture in the black product can be a picture containing only the activity code, or it can be a picture of the activity code on the physical product taken by the black product, or it can be a black product that scans the activity code on the physical product And the obtained picture (also can be referred to as simulation picture).
  • the activity code on the physical body of the product is printed on the bottle cap. After scanning the activity code on the physical body of the product, a picture containing the bottle cap can be obtained, and black products can PS the activity code on the bottle cap to form a simulated picture . It is understandable that the pictures in the black products are not limited to these types, as long as the pictures in the black products are used to redeem prizes and contain activity codes.
  • the code scanning pictures have the above-mentioned multiple types, the multiple types of scanning code pictures can be collectively classified into real prize claiming pictures and fraudulent prize claiming pictures.
  • the real prize redemption picture refers to the picture obtained after scanning the activity code on the physical product; the prize redemption picture used for fraud refers to all non-real prize redemption pictures.
  • the embodiment of the present disclosure does not specifically limit the scanned code pictures received, for example, it may be the scanned code pictures obtained directly from the user's mobile terminal by the receiving brand party, or directly obtained by the receiving brand party from the user's mobile terminal
  • the scanned code image obtained by compressing the scanned code image.
  • the code scanning picture may be a picture compressed within 500k, and the recognition efficiency can be improved by compressing the code scanning picture.
  • step S220 image recognition is performed on the code-scanning picture to determine whether the code-scanning picture is a target picture.
  • the target picture is the picture obtained by scanning the activity code on the product entity, that is, the target picture is the real prize redemption picture mentioned above. Therefore, determining whether the code-scanning picture is a target picture can also be understood as determining whether the code-scanning picture is a real prize claiming picture.
  • the embodiment of the present disclosure does not specifically limit the recognition method of image recognition and the determination method of determining the scan code picture. It only needs to be able to determine whether the code-scanning picture is the target picture according to the image recognition.
  • the image features of the scanned code picture do not include screen moiré features.
  • the image features of the scanned code picture may include screen moiré features. Therefore, whether the scanned code picture is the target picture can be determined by identifying whether there is screen moiré in the scanned code picture.
  • the image features of the background area in the scanned code picture may include features of objects in the background area (eg, bottle caps).
  • the image features of the background area in the scanned code picture may not include any other features. Therefore, it can be determined whether the code-scanning picture is the target picture by identifying whether the background area in the code-scanning picture contains features of objects.
  • the image features of the scanned code image only include features of one layer.
  • the image features in the scanned code picture may include features of at least two layers. Therefore, it can be determined whether the code-scanning picture is the target picture by identifying the features of the layers in the code-scanning picture.
  • the embodiments of the present disclosure can accurately identify fraud on the code scanning link by performing image recognition on the scanning code picture to determine whether the scanning code picture is the target picture, and can avoid blacklists such as blacklist prevention and control and rule prevention and control.
  • blacklists such as blacklist prevention and control and rule prevention and control.
  • the scene mismatch, normal consumers’ accidental injury and illegal products’ easy to bypass the rules can effectively prevent black products from using activity codes to redeem prizes.
  • image recognition is performed on the scanned code picture to determine whether the scanned code picture is the target picture.
  • the specific identification method and determination method can also have Many forms.
  • the embodiments of the present disclosure do not specifically limit its implementation.
  • the method in FIG. 4-6 may be used to perform image recognition on the target image. This will be described in detail below.
  • step S220 in FIG. 2 may include step S221 and step S222.
  • step S221 image recognition is performed on the code-scanning picture to obtain a first recognition result and/or a second recognition result, the first recognition result is used to indicate the label of the active code in the code-scanning picture, and the label of the active code includes a physical code and the electronic code, the second recognition result is used to indicate the label of the background area in the scanned code image.
  • step S222 according to the first recognition result and/or the second recognition result, it is determined whether the code-scanning picture is the target picture.
  • the physical code may represent that the active code is an active code on a physical body
  • the electronic code may represent that the active code is an active code on a non-physical (for example, a screen).
  • indicating that the activity code in the scan code picture is a physical code or an electronic code is equivalent to indicating that the scan code picture is obtained by scanning the activity code on the entity or by scanning the activity code on the non-physical The scanned image.
  • the first recognition result may directly indicate that the label of the active code in the scanned code picture is a physical code or an electronic code.
  • the first recognition result may indicate that the active code label in the scanned code image includes a physical code label and an electronic code label, the physical label includes the characteristic value of the physical code, and the electronic code label includes an electronic code label.
  • Code representation value Determine whether the scanned code picture is a physical code or an electronic code by the larger of the representative value of the physical code and the representative value of the electronic code (for example, if the representative value of the physical code in Figure 5 is greater than the representative value of the electronic code, then determine the scanned code image is the target image).
  • the characteristic value may be a probability value indicating that the activity code is a physical code or an electronic code, or may be a similar value indicating that the activity code is a physical code or an electronic code.
  • the scanned image can be identified by pre-training image classification modules using a plurality of pictures containing electronic codes and physical codes respectively, and the image classification module can output the first recognition result to indicate that the active code in the scanned image is an entity Code or electronic code.
  • the background area in the scanned code picture may refer to the part carrying the active code, or it may be understood as the area in the scanned code picture other than the active code (that is, the code-free area in the scanned code picture).
  • Objects in the background area may refer to carriers of active codes.
  • the activity code can be printed in the bottle cap or the pull ring of the beverage, and the bottle cap or the pull ring is the carrier of the activity code.
  • the activity code can be printed on the outer packaging of the bagged commodity or the bottled commodity, and the outer packaging is the carrier of the activity code.
  • the activity code can be printed on a carton containing multiple commodities, and the carton is the carrier of the activity code.
  • the activity code can be printed on a card inside the packaging of the bagged commodity or the boxed commodity, and the card is the carrier of the activity code.
  • the label of the background area may be information used to indicate the attributes of the background area in the scanned image.
  • the label of the background area includes at least one of the following labels: a label for indicating the color of the background area (e.g., red or black), a label for indicating the type of object in the background area (e.g., a bottle cap or carton), a label for indicating the material of the object in the background area (eg, plastic or metal).
  • the tags of the background area in the embodiments of the present disclosure are not limited to the above content, and may also include other content indicating attribute information of the background area, for example, tags used to indicate the shape of objects in the background area, tags used to indicate fonts in the background area Labels, etc., can be set according to the attributes of the product and the needs of detection.
  • the label of the background area in the target picture is a preset label.
  • the default label is: color: red, type: cap, and material: plastic.
  • the number of preset tags and the specific type of representation can be set according to the attributes of the product and the requirements of detection.
  • the second recognition result may include a label of the background area in the scanned code image corresponding to the above-mentioned preset label one-to-one.
  • the method for determining whether the code-scanning picture is the target picture may be specifically, if each of the labels of the background area in the code-scanning picture indicated by the second recognition result is the same as each of the preset labels Identical, then determine that the scanned code picture is the target picture; If any one of the labels of the background area in the scanned code picture indicated by the second recognition result is not the same as any one of the preset labels, then determine that the scanned code picture is not target image.
  • the second recognition result may be obtained through a classification neural network or a combination of multiple classification neural networks.
  • the classification neural network can be a ResNet convolutional neural network, a VGG convolutional neural network or a DenseNet convolutional neural network.
  • the classification neural network in this part may be a part of the image classification module for obtaining the first recognition result, or may be another image classification module.
  • the attack methods of black products gradually evolved, the pictures (especially simulated pictures) in black products became more and more real. Therefore, in some cases, it cannot be accurately determined whether the code-scanning picture is the target picture only by the first recognition result or the second recognition result.
  • the first recognition result will indicate that the scanned code picture is the physical code. If the black product prints out the activity code and sticks it on the bottle cap, and then scans it to redeem the prize, after the scanned code picture in the black product is image-recognized, the first recognition result will also indicate that the scanned code picture is a physical code.
  • the real scan code picture is still the above situation, and the black product redeems the prize by scanning the picture of the activity code on the physical product that is photographed.
  • the second recognition result indicates The label of the background area in the scanned code picture is exactly the same as the preset label. At this time, only relying on the second recognition result to determine the type of the scanned code picture will leave a loophole for the attack of black products.
  • the embodiment of the present disclosure further proposes to determine whether the scanned code picture is the target picture according to the first recognition result and the second recognition result. Specifically, if the first recognition result indicates that the active code is a physical code, and each of the labels of the background area in the scan code picture indicated by the second recognition result is the same as each of the preset labels, it is determined that the code scan The picture is the target picture; if the first recognition result indicates that the active code is an electronic code, or any one of the labels of the background area in the scanned code picture indicated by the second recognition result is different from any one of the preset labels, Make sure the scanned image is a non-target image.
  • the embodiment of the present disclosure does not specifically limit the execution subject that performs image recognition and the execution subject that determines whether the code-scanning picture is the target picture.
  • the execution subject for performing image recognition and determining the scanned code picture may be the same server, or the same service platform, or the same application.
  • the execution entities that perform image recognition and determine the scanned code picture may be two servers or two services within one application.
  • the executing subject for image recognition may be a vision engine
  • the executing subject for determining whether the code-scanning picture is the target picture according to the image recognition result may be a decision engine.
  • An interface can be provided on the decision engine to receive the scanned code picture from the brand side and send the picture to the visual engine.
  • the scanned code picture can be the scanned code picture collected and compressed by the brand side. Setting image recognition and result determination on different engines can reduce the complexity of single-engine work.
  • the image processing method of the visual engine may be any image processing method described in the embodiments of the present disclosure. Therefore, the output result of the visual engine may include the above-mentioned first recognition result and second recognition result.
  • a risk control engine can be set in the decision engine, and the risk control engine can determine whether the scanned code picture is the target picture through the first recognition result and the second recognition result.
  • the risk control engine in FIG. 5 can simultaneously judge the size of the characteristic value of the physical code and the characteristic value of the electronic code and whether the label of the background area in the scanned code picture is a preset label.
  • the wind control engine determines that the characteristic value of the physical code in the first recognition result is greater than the characteristic value of the electronic code and the background area in the scanned code picture indicated in the second recognition result When the label of is a bottle cap, information 1 can be output. And if the wind control engine determines that the characteristic value of the physical code in the first recognition result is smaller than the characteristic value of the electronic code or the label in the background area of the scanned code picture indicated in the second recognition result is not a bottle cap, it can output Information 0.1 means normal and conforms to business logic; 0 means abnormal and does not conform to business logic.
  • the decision engine can also determine the QPS access amount through the decision engine before the visual engine performs image processing on the scanned code image. If the QPS access exceeds an exception (for example, 200 in FIG. 5), then directly The information 0 is output by the decision engine without further image processing. Or the decision engine can also detect the processing time of the vision engine. If the vision engine fails to return the recognition result after a timeout, the risk control engine can also output information 0. The decision engine system can also detect the time of scanning the code and determine whether the time of scanning the code is during the marketing campaign. If it is determined that the code scanning time is an inactive period, then information 99 may be output. When the brand side receives the reminder message 1, it executes the prize redemption, when it receives the reminder message 0, it closes the prize redemption, and when it receives the reminder message 99, it reminds the user of the non-active time.
  • an exception for example, 200 in FIG. 5
  • the decision engine can also detect the processing time of the vision engine. If the vision engine fails to return the recognition result after a timeout,
  • the decision engine only needs to process text information, and the vision engine only needs to process image information, which can increase the recognition speed of scanned code images.
  • first recognition result and second recognition result may be acquired in various ways.
  • first recognition result and the second recognition result may be acquired simultaneously through the image classification module. This will be specifically described below in conjunction with FIG. 6 .
  • the code-scanning image can be recognized by the image classification module in the visual computing module to determine the first recognition result and the second recognition result of the code-scanning picture.
  • the image classification module can be a neural network trained by various positive samples and negative samples. For example, it is a neural network trained by multiple pictures containing electronic codes and physical codes, or in the case of bottle caps, the positive samples of bottle caps with multiple active codes and other types of negative samples. Sample images for training the neural network.
  • a classification neural network ResNet as shown in FIG. 7 may be used to recognize the code-scanned picture.
  • An attention module may be provided in the classification neural network in FIG. 7 to further enhance the saliency of the region of interest in the image.
  • channel attention or spatial attention can be added to any two layers of a convolutional neural network.
  • the embodiments of the present disclosure add channel attention or spatial attention to the classification neural network, so that various attacks of black products can be resisted.
  • the embodiment of the present disclosure does not specifically limit the training method of the classification neural network ResNet, as long as the classification neural network ResNet can realize the required classification.
  • HEM Hard example mining
  • HEM Hard example mining
  • the image classification module can realize multi-dimensional recognition of scanned code pictures. For example, identify the label of the active code in the scanned code picture and identify the label of the background area in the scanned code picture.
  • the recognition result of the image classification module is the recognition result 1, and the recognition result 1 may include the above-mentioned first recognition result and the second recognition result.
  • the above training samples are only examples, and the neural network used for image classification in the embodiments of the present disclosure is not limited to the recognition and classification of the above categories. Its recognition content and the type of category identification in the recognition result can be adjusted according to actual needs, so its training samples can also be adjusted according to needs.
  • the pictures in black products can be counterfeit pictures through PS, or pictures taken on the screen or screenshots. Therefore, a more precise identification method is required.
  • the visual computing module may further include a picture counterfeit detection module.
  • the counterfeit detection module can perform image recognition on the code-scanning picture to obtain the third recognition result (recognition result 2 in Figure 6), and the third recognition result is used to indicate whether the code-scanning picture is: an edited picture, by taking a screenshot The pictures obtained, and/or, the pictures obtained by taking a screen shot.
  • the image forgery detection module may include a target neural network for detecting tampered images, and a neural network for detecting screen captures and/or screenshots.
  • the target neural network for detecting tampered pictures may also be referred to as a neural network for PS detection, and the neural network can detect whether the scanned code picture is an edited picture.
  • black products can erase the QR code of the original bottle cap and paste a new QR code to confuse the prize redemption server.
  • This type of forgery technique is generally defined as image splicing.
  • image stitching is considered to be the most fundamental and main operation, and it is also one of the most common means of tampering with image content. It combines two or more images into one image at the same time. Special image processing methods blur the borders of stitching areas to cover up tampering traces and falsify facts, so it is also called synthetic image tampering.
  • the PS detection neural network can detect whether the picture is a PS picture based on the pixel points, noise distribution, light source information and layer information in the picture.
  • the recognition result of the detection part may indicate whether the code-scanning picture has passed through PS, if the recognition result determines that the code-scanning picture is a picture that has passed through PS, then determine that the code-scanning picture is a non-target picture, otherwise, determine that the code-scanning picture is a target picture.
  • the embodiment of the present disclosure does not specifically limit the neural network for PS detection, for example, it may be an RGB-N tampering detection network, a ManTraNet tampering detection network, and an EXIF-Consistency tampering detection network.
  • PS detection can be realized by using a fusion neural network of at least two neural networks in RGB-N tamper detection network, ManTraNet tamper detection network and EXIF-Consistency tamper detection network.
  • the fusion schematic diagram is shown in Figure 8.
  • Stacking is a method of comprehensively reducing bias and variance by replacing Voting/Averaging of Bagging and Boosting with Meta-Learner.
  • the stacking method is used to fuse RGB-N, ManTraNet and EXIF-Consistency.
  • the AUC and recall of the stacking model are 0.949 and 0.859. Compared with the three models/rules alone, the model performance indicators have been significantly improved.
  • the neural network for detecting screen shots and/or screenshots may be one neural network or two neural networks. Through this neural network, it is possible to identify whether there are edges of mobile phones or computers or screen moiré in the pictures, and it is also possible to identify whether there are some features of screenshots in the pictures, for example, the pictures have battery and signal prompt information on the top of the mobile phone.
  • HOG descriptors may be used to acquire feature regions.
  • HOG Heistogram of oriented gradient
  • HOG is the English abbreviation of histogram of oriented gradient. It is a feature descriptor used in the field of computer vision and image processing for target detection. This technique is used to calculate the statistical value of the direction information of the local image gradient. This method has many similarities with edge orientation histograms, scale-invariant feature transform descriptors, and shape context methods, but the difference with them is:
  • HOG Descriptors are computed on a dense grid of uniformly spaced cells, and overlapping local contrast normalization is used to improve performance.
  • the third recognition result indicates that the code-scanning picture is any one of: an edited picture, a picture obtained by taking a screenshot, and/or a picture obtained by taking a screen shot, then it is determined that the code-scanning picture is a non-target picture. And if the third result indicates that the code-scanning picture is not an edited picture, a picture obtained by taking a screenshot, and/or any one of a picture obtained by taking a screen shot, then the code-scanning picture is determined to be the target picture.
  • the inventors of the present disclosure propose that the scanned code image can be identified and determined based on this.
  • the visual computing module may also include a text recognition module, which can obtain a fourth recognition result (recognition result 3 in FIG. 6 ) by recognizing the code-scanning picture through the text recognition module (OCR recognition).
  • the fourth recognition result is used to indicate whether the text around the active code is abnormal.
  • the text recognition module can detect the comparison result between the text data in the scanned code picture and the preset text data, if the comparison result indicates that the text data in the scanned code picture is consistent with the preset text data, then the fourth recognition result Indicates that the text around the activity code is normal, and the scanned image is determined to be the target image. Otherwise, the fourth recognition result indicates that the text around the active code is abnormal, and it is determined that the scanned code picture is a non-target picture.
  • the preset text data can also be referred to as data of normal business logic.
  • the preset text data is known, for example, it can be initial capital letter + 10 numbers + brand name. If the content or format of the text data in the scanned image is different from this, the comparison result is inconsistent.
  • the vision computing module may also include a repetition detection module.
  • the repetition detection module can identify the similarity between the code-scanning picture and the pictures in the preset image library, and obtain the fifth recognition result (recognition result 4 in FIG. 6 ). If the fifth recognition result indicates that the scanned code picture is highly similar to the picture in the preset image database, then it is determined that the scanned code picture is a non-target picture. If the fifth recognition result indicates that the code-scanning picture is not similar to the pictures in the preset image database, then the code-scanning picture is determined to be the target picture.
  • the duplicate detection module is an image retrieval system CBIR (Content based Image Retrieval).
  • CBIR system is composed of two parts: feature extraction subsystem and query subsystem.
  • the mass image data is converted into embedding information and stored in the image library, the steps are as follows: 1. Preprocessing; for example, image format conversion, regularization, image enhancement and denoising, etc. 2. Extract the area that the user is interested in, and then perform target recognition after extracting features; in some embodiments, this step is not necessary, and can be set according to the research direction. 3. Feature extraction. It can be features based on color, texture, shape, and spatial relationship, or feature extraction through CNN. 4. The database consists of image database, feature database and knowledge database.
  • the image database is digital image information, which is mainly used to return the results after retrieval;
  • the feature database contains automatically extracted content features, which are the key information for retrieval;
  • the knowledge expression in the knowledge base can be replaced to apply to different fields, and the knowledge base is used to Auxiliary query conditions are mainly used for filtering.
  • Query interface provide users with the ability to customize the search, which can be in the form of an interface or an interface wait.
  • Retrieval engine the search engine mainly performs similarity measurement, which contains an effective and reliable similarity measurement function set.
  • Indexing/filtering Fast retrieval through indexing/filtering.
  • Duplication detection is a typical risk in marketing anti-fraud, that is, there are pictures that are completely identical or highly similar to the scanned pictures in the picture base library. Duplication of pictures is called batch risk in business, which means that gangs commit crimes with a high probability. Identifying batch risks has extremely high business value.
  • the recognition method and recognition result of the scanned code image in the embodiment of the present disclosure may be any combination of the above methods and recognition results.
  • the image recognition module in the visual engine in Fig. 5 can be the fusion module of the image classification module, anti-counterfeiting detection module, text recognition module and repetition detection module in Fig. 6, and
  • the output result of the image recognition module can also be the fusion result of the recognition results 1, 2, 3 and 4 in Fig. 6, and the fusion result can be finally expressed as the representative value of the physical code, the representative value of the electronic code and the label of the background area.
  • the recognition method is not limited to the detection method of the four modules in FIG. 6 and may also be other picture detection methods. Through the fusion of multiple identification methods, the scanned code image can be identified from multiple angles to combat common illegal attacks, so that the recognition results are more accurate and the identification effect of illegal fraud is improved.
  • the embodiment of the present disclosure proposes a protection measure based on a digital security chain.
  • the production, storage, transmission, printing, and scrapping of the activity code on the commodity entity are all set on the blockchain to realize encrypted data transmission, decryption restrictions, scanning code traceability, data transmission tracking, etc.
  • a series of security issues can realize the fixed-point tracking of code production, storage, transmission, printing, and scrapping, and reduce the risk of leakage during processing and circulation.
  • Fig. 11 is a schematic structural diagram of a device provided by an embodiment of the present disclosure.
  • the apparatus 1100 may include a receiving module 1110 and a determining module 1120 . These modules are described in detail below.
  • the receiving module 1110 is used to receive the scanned code picture, and the scanned code picture contains the activity code corresponding to the marketing activity of the product; the determining module 1120 is used to perform image recognition on the scanned code picture to determine whether the scanned code picture is a target A picture, the target picture is a picture obtained after scanning the activity code on the entity of the commodity.
  • the determination module is configured to perform image recognition on the code-scanning picture to obtain a first recognition result, the first recognition result is used to indicate the label of the active code in the code-scanning picture, the The label of the activity code includes a physical code and an electronic code; and is used for determining whether the picture scanned by the code is the target picture according to the first recognition result.
  • the determination module is configured to perform image recognition on the code-scanning picture to obtain a second recognition result, the second recognition result is used to indicate the label of the background area in the code-scanning picture, the
  • the label of the background area includes at least one of the following labels: a label for indicating the color of the background area, a label for indicating the type of the object in the background area, and a label for indicating the object in the background area a label of the material; and used for determining whether the scanned code picture is the target picture according to the second identification result.
  • the determining module is configured to perform image recognition on the code-scanning picture to obtain a third recognition result, and the third recognition result is used to indicate whether the code-scanning picture is: an edited picture, through A picture obtained by taking a screenshot, and/or a picture obtained by taking a screen shot; and used for determining whether the scanned code picture is the target picture according to the third identification result.
  • the determination module is configured to use a target neural network to perform image recognition on the code-scanning picture to identify whether the code-scanning picture is an edited picture
  • the target neural network includes at least one of the following neural networks: Two neural networks: RGB-N tamper detection network, ManTraNet tamper detection network and EXIF-Consistency tamper detection network.
  • the determination module is configured to recognize the text around the active code in the code-scanning picture to obtain a fourth recognition result, the fourth recognition result is used to indicate the text around the active code whether there is an abnormality; and determining whether the code-scanning picture is a target picture according to the fourth identification result.
  • the determination module is used to identify the similarity between the scanned code picture and the picture in the preset image library to obtain a fifth recognition result; and is used to determine the scanned code according to the fifth recognition result. Whether the code image is the target image.
  • Fig. 12 is a schematic structural diagram of an apparatus for identifying a picture provided by another embodiment of the present disclosure.
  • the apparatus 1200 may be, for example, a computing device with a computing function.
  • the device 1200 may be a mobile terminal or a server.
  • the apparatus 1200 may include a memory 1210 and a processor 1220 .
  • Memory 1210 may be used to store executable code.
  • the processor 1220 can be used to execute the executable code stored in the memory 1210, so as to realize the steps in the various methods described above.
  • the apparatus 1200 may further include a network interface 1230 through which data exchange between the processor 1220 and external devices may be implemented.
  • all or part of the implementation may be implemented by software, hardware, firmware or other arbitrary combinations.
  • software When implemented using software, it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, all or part of the processes or functions according to the embodiments of the present disclosure will be generated.
  • the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server or data center via wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media.
  • the available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital video disc (Digital Video Disc, DVD)), or a semiconductor medium (such as a solid state disk (Solid State Disk, SSD)), etc. .
  • a magnetic medium such as a floppy disk, a hard disk, a magnetic tape
  • an optical medium such as a digital video disc (Digital Video Disc, DVD)
  • a semiconductor medium such as a solid state disk (Solid State Disk, SSD)
  • the disclosed systems, devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Strategic Management (AREA)
  • Development Economics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Health & Medical Sciences (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Game Theory and Decision Science (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Electromagnetism (AREA)
  • Toxicology (AREA)
  • Image Analysis (AREA)

Abstract

本公开披露了一种识别图片的方法和装置。所述方法包括：接收扫码图片，所述扫码图片包含商品的营销活动对应的活动码；对所述扫码图片进行图像识别，以确定所述扫码图片是否为目标图片，所述目标图片为对所述商品的实体上的所述活动码进行扫描后得到的图片。

Description

[根据细则37.2由ISA制定的发明名称]　识别图片的方法和装置 技术领域
本公开涉及信息安全领域，尤其涉及一种识别图片的方法及装置。
背景技术
在实体商品的营销活动中，品牌方通常会在商品的实体上印刷有供用户购买后进行扫描兑奖的活动码。然而，随着网络黑产的兴起，活动码经常会被黑产窃取和售卖。一旦活动码被黑产中的羊毛党或者刷量团伙进行兑奖(可简称为欺诈)，就会给品牌方带来巨大的损失。
基于此，亟需要一种可以准确的识别欺诈的方案从而有效地防止黑产使用活动码兑奖。
发明内容
有鉴于此，本公开提供一种识别图片的方法及装置，以准确的识别欺诈从而有效地防止黑产使用活动码兑奖。
第一方面，提供一种识别方法，所述方法包括：接收扫码图片，所述扫码图片包含商品的营销活动对应的活动码；对所述扫码图片进行图像识别，以确定所述扫码图片是否为目标图片，所述目标图片为对所述商品的实体上的所述活动码进行扫描后得到的图片。
第二方面，提供一种识别装置，所述装置包括：接收模块，用于接收扫码图片，所述扫码图片包含商品的营销活动对应的活动码；确定模块，用于对所述扫码图片进行图像识别，以确定所述扫码图片是否为目标图片，所述目标图片为对所述商品的实体上的所述活动码进行扫描后得到的图片。
第三方面，提供一种识别装置，包括存储器和处理器，所述存储器中存储有可执行代码，所述处理器被配置为执行所述可执行代码，以实现如第一方面所述的方法。
第四方面，提供一种计算机可读存储介质，其上存储有可执行代码，当所述可执行代码被执行时，能够实现如第一方面所述的方法。
第五方面，提供一种计算机程序产品，包括可执行代码，当所述可执行代码被执行时，能够实现如第一方面所述的方法。
本公开实施例提供了一种的识别图片的方法，其通过接收包含了商品的营销活动对应的活动码的扫码图片并对该扫码图片进行图像识别，以确定扫码图片是否为对商品的 实体上的活动码进行扫描后得到的图片。通过对扫码图片的图像识别而在扫码链路上准确的识别欺诈，从而有效地防止黑产使用活动码兑奖。
附图说明
图1为黑产产业链的运营过程示意图。
图2是本公开实施例提供的识别图片的方法的流程示意图。
图3A和图3B是本公开一实施例中的活动码的示例图。
图4是本公开实施例提供的识别图片的方法的另一流程示意图。
图5是本公开实施例提供的架构示例图。
图6是本公开实施例提供的用于图像识别的视觉计算模块的结构示意图。
图7是本公开实施例提供的分类神经网络的结构示意图。
图8是本公开实施例提供的篡改检测网络的融合结构示意图。
图9是本公开实施例提供的重复检测模块的结构示意图。
图10是本公开实施例提供的商品的实体上的活动码的防控方案的示意图。
图11是本公开实施例提供的识别图片的装置的结构示意图。
图12是本公开实施例提供的识别图片的装置的另一结构示意图。
具体实施方式
下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本公开一部分实施例，而不是全部的实施例。
为了便于理解，先对本公开实施例中涉及的部分概念进行简单的介绍。
本公开实施例提及的商品是指一切具有实体的商品，且商品为品牌方的正规商品。例如，商品可以是酒水饮料、生活用品、食品、玩具等。
营销活动是指品牌方为了促进商品的销售而展开的活动。营销活动可以是任何一种包含福利的促销活动，例如，可以是买商品送赠品，或者可以是买商品送红包或者各种抵用券，或者还可以是买多个商品后凑齐积分以兑换奖品。通常品牌方会将商品的营销活动做成活动码，并将活动码印刷在商品上。当消费者购买商品后对活动码进行扫描就可以获得营销活动中包含的福利。
本公开实施例对活动码的类别不做具体的限定，只要活动码可以被识别并解读出其对应的营销活动即可。例如，活动码可以是二维码或者条形码。
黑产，也可以称为网络黑产。网络黑产是指以互联网为媒介，以网络技术为主要手段，为计算机信息系统安全和社会经济稳定带来潜在威胁的非法行为。例如，兑奖、套 现、返利等网络非法行为。
随着黑产的兴起，用于营销的活动码经常会被黑产窃取和售卖以用于兑奖或套现。如图1所示为黑产产业链的运营过程。黑产的主体运营者会从废品回收站、塑料粉碎厂、人群聚集地、清洁工处等收集消费者丢弃的商品的实体上的活动码，或者会在活动码的生产、传输和加工的环节中窃取到活动码，并将活动码通过线上平台进行售卖。一旦活动码被黑产中的羊毛党或者刷量团伙进行兑奖(可简称为欺诈)，就会给品牌方带来巨大的损失。
为了防止黑产使用活动码兑奖带来的损失，现有技术中的一种可行的方式是，获取扫码兑奖的用户的IP、手机号或者设备，并通过已有的黑名单对其进行判别以判断用户是否为羊毛党或者刷量团伙。如果判别结果确定用户是羊毛党或者刷量团伙则限制其兑奖。
然而，这种方法会造成以下问题：1、场景错位导致的名单失效。例如，已有的黑名单是电信黑名单或者信贷黑名单，而电信诈骗份子或信贷老赖也会购买啤酒饮料，其属于真实消费者，使用该类黑名单用于营销反欺诈场景会导致错配问题。2、黑产攻击形式的演变导致名单失效。例如，过去的黑产攻击的IP为代理IP，账号为小号，验证码为码奴，设备为安装农场、操盘手为工作室。然而随着黑产攻击的演变，现在的黑产的攻击的IP为秒拨，账号为跑分账号，验证码为技术破解，设备为云手机、操盘手为众包。这一系列演变导致现有的黑产的IP、手机号或者设备是真实的和多变的，所以使用黑名单类方法对其失效。
现有技术中的另一种可行的方式是，获取扫码兑奖的用户的扫码次数，通过预设的扫码规则判断用户的扫码的次数是否超过了限制，以预防黑产中的刷量团伙大量的兑奖。
然而，该类方法会造成以下问题：1、正常消费者会被误伤。例如，某饮料的活动码印在两类包装上，一种是纸箱，一种是饮料瓶，一个纸箱内有24瓶饮料。便利店的店主在售卖饮料时，通常是按照瓶售卖，因此纸箱上的活动码都是店主本人扫码领取福利。如果通过限制规则制定的每人限制扫3次，就会误伤到店主，影响其销售积极性。2、黑产容易绕过预设的扫码规则。由于黑产团伙有大量账号和IP池，所以针对账号和IP的扫码次数限制，仅增加了黑产对抗成本，并不会阻止黑产非法获得营销资金的行为。例如，黑产每扫10个活动码换一个账号，用来跨过规则。
基于此，亟需要一种可以准确的识别欺诈的方案从而防止黑产使用活动码兑奖。
然而，本公开的发明人发现，为了方便储存和售卖，黑产一般会将商品的营销活动对应的活动码转换成为纯电子码或者模拟码的形式以图片的形式进行销售。也就是说黑 产售卖的是图片，图片中包含活动码。如果黑产中的羊毛党和刷量团队要对黑产中的活动码进行兑奖，其需要对黑产中的图片进行扫描，或者将黑产中的图片打印出来进行扫描。或者在极端情况下，黑产可以直接使用其图片进行兑奖。而真实的消费者则是对商品的实体上的活动码进行扫描，如果在扫码链路上可以对来自于真实的消费者的扫码图片和来自于黑产的扫码图片进行区分，则可以避免上述现有技术中存在的问题。
有鉴于此，本公开实施例提供了一种新的识别方案，该方案通过对扫码图片进行图像识别以确定扫码图片是否为对商品的实体上的活动码进行扫描后得到的图片，以此扫码链路上准确的识别欺诈，有效地防止黑产使用活动码兑奖。
下面结合附图2，对本公开实施例提供的方法进行详细描述。
在步骤S210，接收扫码图片。
扫码图片也可以称为兑奖图片。扫码图片包含商品的营销活动对应的活动码。扫码图片可以是通过用户的移动设备在对活动码进行扫描时获取的图片，因此，扫码图片中还可以包括背景区域(也可称为无码区域)。背景区域可以包含扫码图片中除过活动码以外的其他特征。
扫码图片可以是以下多种类型中的一个：对商品的实体上的活动码进行扫描后得到的图片；对黑产中的图片进行扫描后得到的图片以及黑产中的图片。
黑产中的图片可以是仅包含活动码的图片，或者还可以是黑产拍摄的商品的实体上的活动码的图片，或者还可以是黑产模拟的对商品的实体上的活动码进行扫描而得到的图片(也可以称为模拟图片)。例如，商品的实体上的活动码印刷在瓶盖上，对商品的实体上的活动码进行扫描后可以得到包含瓶盖的图片，而黑产可以将活动码PS到瓶盖上以形成模拟图片。可以理解的是，黑产中的图片不限于这几种类型，只要黑产中的图片是用于兑奖且包含活动码的图片即可。
需要说明的是，扫码图片虽然具有上述多种类型，但是该多种类型的扫码图片可以统一归类为真实的兑奖图片和用于欺诈的兑奖图片。真实的兑奖图片是指对商品的实体上的活动码进行扫描后得到的图片；用于欺诈的兑奖图片是指所有非真实的兑奖图片。
本公开实施例对接收的扫码图片不做具体的限定，例如，可以是接收品牌方直接从用户的移动终端获取到的扫码图片，也可以是接收品牌方对直接从用户的移动终端获取到的扫码图片进行压缩后的扫码图片。作为一种实现方式，扫码图片可以是压缩在500k以内的图片，通过将扫码图片进行压缩可以提高识别效率。
在步骤S220，对扫码图片进行图像识别，以确定扫码图片是否为目标图片。
目标图片为对商品的实体上的活动码进行扫描后得到的图片，即目标图片为上文所 述的真实的兑奖图片。因此，确定扫码图片是否为目标图片也可以理解为确定扫码图片是否为真实的兑奖图片。
本公开实施例对图像识别的识别方法和确定扫码图片的确定方法不做具体的限定。只要可以根据图像识别确定扫码图片是否为目标图片即可。
例如，扫码图片为目标图片时，扫码图片的图像特征中不包含屏幕摩尔纹特征。而扫码图片为非真实的兑奖图片时，扫码图片的图像特征中可以包含屏幕摩尔纹特征。因此，可以通过识别扫码图片中是否有屏幕摩尔纹来确定扫码图片是否为目标图片。
又如，如图3A所示，扫码图片为目标图片时，扫码图片中的背景区域的图像特征中可以包含背景区域中的物体的特征(例如，瓶盖)。而扫码图片为非真实的兑奖图片时，如图3B所示，扫码图片中的背景区域的图像特征中可以不包含任何其他特征。因此，可以通过识别扫码图片中的背景区域是否包含物体的特征而确定扫码图片是否为目标图片。
再如，扫码图片为目标图片时，扫码图片的图像特征中只包含一个图层的特征。而扫码图片为非真实的兑奖图片时，扫码图片中的图像特征中可以包含至少两个图层的特征。因此，可以通过识别扫码图片中的图层的特征而确定扫码图片是否为目标图片。
本公开实施例通过对扫码图片进行图像识别后确定扫码图片是否为目标图片而在扫码链路上准确的识别欺诈，可以避免如黑名单类防控和规则类防控造成的黑名单的场景错配、正常消费者误伤及黑产容易绕过规则的问题，有效地防止黑产使用活动码兑奖。
由于黑产中的用于欺诈兑奖的图片的形式非常多样化，因此，本公开实施例中对扫码图片进行图像识别，以确定扫码图片是否为目标图片具体识别方法和确定方法也可以具有多种形式。如前所述，本公开实施例对其实现方式不做具体的限定。作为一种实现方式，可以采用图4-6的方法对目标图像进行图像识别。下面对此进行详细的说明。
如图4所示，图2中的步骤S220可以包括步骤S221和步骤S222。
在步骤S221，对扫码图片进行图像识别，以得到第一识别结果和/或第二识别结果，第一识别结果用于指示扫码图片中的活动码的标签，活动码的标签包括实体码和电子码，第二识别结果用于指示扫码图片中的背景区域的标签。
在步骤S222，根据第一识别结果和/或第二识别结果，确定扫码图片是否为目标图片。
在一些实现方式中，可以根据第一识别结果就可以确定扫码图片是否为目标图片。具体来说，如果第一识别结果指示活动码为实体码，则确定扫码图片为目标图片；如果第一识别结果指示活动码为电子码，则确定扫码图片为非目标图片(即非真实的兑奖图片)。
实体码可以代表活动码是实体上的活动码，电子码可以代表活动码是非实体(例如，屏幕)上的活动码。
可以理解的是，指示扫码图片中的活动码为实体码或电子码相当于指示扫码图片是通过对实体上的活动码进行扫描后得到的图片或是通过对非实体上的活动码进行扫描后得到的图片。
本公开实施例对于第一识别结果的表示形式不做具体的限定。例如，第一识别结果可以是直接指示扫码图片中的活动码的标签为实体码或电子码。或者，如图5所示，第一识别结果可以指示扫码图片中的活动码的标签包括实体码的标签和电子码的标签，实体的标签包括实体码的表征值，电子码的标签包括电子码的表征值。通过实体码的表征值与电子码的表征值中的较大者确定扫码图片是实体码还是电子码(例如，图5中的实体码的表征值大于电子码的表征值，则确定扫码图片为目标图片)。表征值可以有多种表示方法，例如，表征值可以是指示活动码为实体码或电子码的概率值或者可以是指示活动码与实体码或电子码的相似值。
本公开实施例对第一识别结果的获取方法不作具体的限定。例如，可以通过预先使用多个分别包含电子码和实体码的图片训练好的图像分类模块对扫码图片进行识别，图像分类模块可以输出第一识别结果以指示扫码图片中的活动码是实体码还是电子码。
在另一些实现方式中，可以根据第二识别结果可以确定扫码图片是否为目标图片。
扫码图片中的背景区域可以是指承载活动码的部分，也可以理解为扫码图片中除过活动码以外的区域(即扫码图片中的无码区域)。背景区域中的物体可以是指活动码的载体。例如，活动码可以印刷在酒水饮料的瓶盖或拉环内，瓶盖或拉环则为活动码的载体。又如，活动码可以印刷在袋装商品或者瓶装商品的外包装上，外包装则为活动码的载体。又如，活动码可以印刷在包含有多个商品的纸箱上，纸箱则为活动码的载体。又如，活动码可以印刷在袋装商品或者盒装商品的包装内部的卡片上，卡片则为活动码的载体。
背景区域的标签则可以是用于指示扫码图片中的背景区域的属性的信息。例如，背景区域的标签包括以下标签中的至少一种：用于指示背景区域的颜色的标签(如，红色或黑色)，用于指示背景区域中的物体的类型的标签(如，瓶盖或纸箱)，用于指示背景区域中的物体的材质的标签背景区域的标签(如，塑料或金属)。本公开实施例对背景区域的标签不限于上述内容，还可以包括其他指示背景区域属性信息的内容，例如，用于指示背景区域中的物体的形状的标签，用于指示背景区域中的字体的标签等等，可以根据商品的属性和检测的需要自行设置。
需要说明的是，由于真实商品的实体上的活动码和活动码的背景是已知的，所以目标图片(即真实兑奖图片)中背景区域的标签为预设标签。例如，对于瓶装可口可乐而言，其活动码印刷在红色的塑料瓶盖上，则预设标签为：颜色：红、类型：瓶盖以及材质：塑料。预设标签的数量和表征的具体类型可以根据商品的属性和检测的需求自行设置。
第二识别结果可以包括扫码图片中的背景区域的与上述预设标签一一对应的标签。根据第二识别结果，确定扫码图片是否为目标图片的方法，具体可以是，如果第二识别结果所指示的扫码图片中的背景区域的标签中的每一个与预设标签中的每一个相同，则确定扫码图片为目标图片；如果第二识别结果所指示的扫码图片中的背景区域的标签中的任何一个与预设标签中的任何一个不相同，则确定扫码图片为非目标图片。
本公开实施例对第二识别结果的获取方式不做具体的限定。例如，可以是通过一个分类神经网络或者多个分类神经网络的组合获取第二识别结果。分类神经网络可以是ResNet卷积神经网络、VGG卷积神经网络或者DenseNet卷积神经网络。该部分的分类神经网络可以是上述用于获取第一识别结果的图像分类模块中的一部分，也可以是另设的图像分类模块。
然而，随着黑产的攻击方式逐渐演变，黑产中的图片(尤其是模拟图片)愈加真实。因此，在一些情况下，仅通过第一识别结果或第二识别结果无法准确地确定扫码图片是否为目标图片。例如，在商品为酒水饮料，活动码印刷在酒水饮料的瓶盖内的情况下，真实消费者的扫码图片经过图像识别后，第一识别结果会指示扫码图片是实体码。如果黑产将该活动码打印出来贴在瓶盖上，然后扫描兑奖时，黑产中的扫码图片经过图像识别后，第一识别结果也会指示扫码图片是实体码。这时，仅靠第一识别结果对扫码图片的类型进行确定，会给黑产的攻击留下漏洞。又如，真实扫码图片还是上述情况，黑产通过扫描拍摄的商品的实体上的活动码的图片而进行兑奖，这时黑产中的扫码图片经过图像识别后，第二识别结果所指示的扫码图片中的背景区域的标签与预设标签完全相同，这时，仅靠第二识别结果对扫码图片的类型进行确定，会给黑产的攻击留下漏洞。
为了解决以上问题，本公开实施例还提出可以根据第一识别结果和第二识别结果，确定扫码图片是否为目标图片。具体来说，如果第一识别结果指示活动码为实体码，且第二识别结果所指示的扫码图片中的背景区域的标签中的每一个与预设标签中的每一个相同，确定扫码图片为目标图片；如果第一识别结果指示活动码为电子码，或，第二识别结果所指示的扫码图片中的背景区域的标签中的任何一个与预设标签中的任何一个不相同，确定扫码图片为非目标图片。
通过同时使用第一识别结果和第二识别结果来确定扫码图片是否为目标图片，可以实现对扫码图片的准确识别，从而避免黑产通过识别漏洞进行欺诈。
本公开实施例对进行图像识别的执行主体和确定扫码图片是否为目标图片的执行主体不做具体的限定。
作为一种实现方式，进行图像识别和确定扫码图片的执行主体可以是同一个服务器，或者同一个服务平台或者同一个应用。
作为另一中实现方式，进行图像识别和确定扫码图片的执行主体可以是两个服务器或者一个应用内的两个服务。例如，如图5所示，进行图像识别的执行主体可以是一个视觉引擎，而根据图像识别结果确定扫码图片是否为目标图片的执行主体可以是一个决策引擎。决策引擎上可以设有接口，以接收来自品牌方的扫码图片并将该图片发送给视觉引擎。扫码图片可以是品牌方采集并压缩后的扫码图片。将图像识别和结果确定分别设置在不同的引擎上，可以减少单引擎工作的复杂度。
以图5为例，视觉引擎的图像处理方法可以是本公开实施例中所述的任何一种图像处理的方法，因此，视觉引擎的输出结果可以包括上述第一识别结果和第二识别结果。而决策引擎中可以设置一个风控引擎，风控引擎可以通过第一识别结果和第二识别结果确定扫码图片是否为目标图片。例如，图5中的风控引擎可以同时判断实体码的表征值和电子码的表征值的大小以及扫码图片中的背景区域的标签是否为预设标签。以商品的活动码印刷在瓶盖为例，风控引擎在确定第一识别结果中的实体码的表征值大于电子码的表征值且第二识别结果中所指示的扫码图片中的背景区域的标签为瓶盖时，可以输出信息1。而如果风控引擎在确定第一识别结果中的实体码的表征值小于电子码的表征值或者第二识别结果中所指示的扫码图片中的背景区域的标签不为瓶盖时，可以输出信息0。1表示正常，符合业务逻辑；0表示异常，不符合业务逻辑。
另外，在一些实现方式中，决策引擎还可以在视觉引擎在对扫码图片进行图像处理之前，通过决策引擎确定QPS访问量，如果QPS访问超过异常(例如，图5中的200)，则直接通过决策引擎输出信息0，而不需要再进行图像处理。或者决策引擎还可以检测视觉引擎的处理时间，如果视觉引擎超时未返回识别结果，风控引擎也可以输出信息0。决策引擎系统还可以检测扫码的时间，并判别扫码时间是否为营销活动期间。如果判别扫码时间为非活动期间，则可以输出信息99。品牌方接收到提醒消息1时执行兑奖，接收到0时关闭兑奖，接收到99时提醒用户此时非活动时间。
通过设置决策引擎和视觉引擎可以分别对不同的数据进行处理，决策引擎只需要处理文本信息，视觉引擎只需要处理图像信息，可以增加扫码图片的识别速度。
如前文所述，上述第一识别结果和第二识别结果可以通过多种方式进行获取。作为一种实现方式，可以通过图像分类模块同时获取第一识别结果和第二识别结果。下文将结合图6对此进行具体的说明。
如图6所示，可以通过视觉计算模块中的图像分类模块对扫码图像进行识别，以确定扫码图片的第一识别结果和第二识别结果。
图像分类模块可以是通过多种正样本和负样本训练好的神经网络。例如，是通过多个包含有电子码和实体码的图片训练的神经网络，或者在载体为瓶盖的情况下，通过多个活动码的载体为瓶盖的正样本及载体为其他类型的负样本的图片训练的神经网络。
在一些实现方式中，可以采用如图7所示的分类神经网络ResNet对扫码图片进行识别。图7中的分类神经网络中可以设有注意力模块，以用于进一步提升图像中感兴趣区域的显著性。例如，可以在卷积神经网络的任意两层中添加通道注意力或空间注意力。本公开实施例通过在分类神经网络中增加了通道注意力或空间注意力，使得可以对抗黑产的多种攻击。本公开实施例对分类神经网络ResNet的训练方式不做具体的限定，只要分类神经网络ResNet可以实现需要的分类即可。在一些实施例中，可以在初步训练好的神经网络上采用负样本中的困难样本对HEM(Hard example mining)对神经网络进行进一步的训练，以提高神经网络的分类效果。
图像分类模块可以实现对扫码图片进行多个维度的识别。例如，对扫码图片中的活动码的标签进行识别和对扫码图片中的背景区域的标签的识别。如图6所示，图像分类模块的识别结果为识别结果1，识别结果1可以包括上文所述的第一识别结果和第二识别结果。
可以理解，上述训练样本仅用于示例，本公开实施例中用于图像分类的神经网络不限于上述类别的识别和分类。其识别内容和识别结果中的类别标识的种类可以根据实际需要自行调整，因此，其训练样本也可以根据需要自行调整。
由于黑产中的图片多种多样，例如，黑产中的图片可以是通过PS伪冒的图片，或者是拍屏的图片及截屏的图片。因此，需要一种更精确的识别方法。
为了解决以上问题，如图6所示，视觉计算模块还可以包括图片伪冒检测模块。通过伪冒检测模块可以对扫码图片进行图像识别，以得到第三识别结果(图6中的识别结果2)，第三识别结果用于指示扫码图片是否为：经过编辑的图片，通过截屏得到的图片，和/或，通过拍屏得到的图片。
图片伪冒检测模块可以包括检测篡改图片的目标神经网络、检测拍屏和/或截图片的神经网络。
检测篡改图片的目标神经网络也可以称为PS检测的神经网络，该神经网络可以检测出扫码图片是否是经过编辑的图片。例如，黑产可以将原瓶盖的二维码擦除，粘贴新二维码用于迷惑兑奖服务器。这类伪造技术通常定义为图像拼接(splicing)。在所有涉及图像非法编辑的操作中，图像拼接被认为是最根本、最主要的操作，也是最为常见的图像内容篡改手段之一，其通过将两幅或者多幅图片合成为一幅图片同时采用特殊的图像处理手段模糊拼接区域边界以达到掩饰篡改痕迹、伪造事实的目的，因此也称为合成图像篡改。通过PS检测神经网络可以基于图片中的像素点、噪声分布、光源信息以及图层信息可以检测出图片是否为PS的图片。该检测部分的识别结果可以是指示扫码图片是否经过PS，如果识别结果确定扫码图片为经过PS的图片，则确定扫码图片为非目标图片，否则，确定扫码图片为目标图片。
本公开实施例对于PS检测的神经网络不做具体的限定，例如可以是RGB-N篡改检测网络、ManTraNet篡改检测网络和EXIF-Consistency篡改检测网络。
作为一种实现方式，可以采用RGB-N篡改检测网络、ManTraNet篡改检测网络和EXIF-Consistency篡改检测网络中的至少两种神经网络的融合神经网络实现PS的检测。融合示意图如图8所示。stacking(堆栈)是通过Meta-Learner取代Bagging和Boosting的Voting/Averaging来综合降低偏差和方差的方法。采用stacking方法将RGB-N、ManTraNet和EXIF-Consistency进行融合。stacking模型的AUC和recall为0.949和0.859，与三种模型/规则单独作用相比，模型性能指标有着显著提升。
检测拍屏和/或截图片的神经网络可以是一个神经网络也可以是两个神经网络。通过该神经网络可以识别出图片中是否有手机或电脑的边缘或屏幕摩尔纹，也可以识别出图片中是否有截屏时的一些特征，例如，图片上具有手机上方的电量和信号提示信息。
本公开实施例对检测拍屏和/或截图片的神经网络的特征获取方式不做具体的限定，例如，可以采用HOG描述符来获取特征区域。HOG(Histogram of oriented gradient)是方向梯度直方图的英文简称，是应用在计算机视觉和图像处理领域，用于目标检测的特征描述器。这项技术是用来计算局部图像梯度的方向信息的统计值。这种方法跟边缘方向直方图(edge orientation histograms)、尺度不变特征变换(scale-invariant feature transform descriptors)以及形状上下文方法(shape contexts)有很多相似之处，但与它们的不同点是：HOG描述符是在一个网格密集的大小统一的细胞单元(dense grid of uniformly spaced cells)上计算，而且为了提高性能，还采用了重叠的局部对比度归一化(overlapping local contrast normalization)技术。
如果第三识别结果指示扫码图片为：经过编辑的图片，通过截屏得到的图片，和/ 或，通过拍屏得到的图片中的任何一种，则确定扫码图片为非目标图片。而如果第三结果指示扫码图片不是经过编辑的图片，通过截屏得到的图片，和/或，通过拍屏得到的图片中的任何一个，则确定扫码图片为目标图片。
对于黑产中的图片，还有一种伪造的可能是，其图片上会有一些作弊说明的文字，如“需下载定位软件定位到xx城市后扫码”，而正常的活动码周围是预设的文本数据。因此，本公开的发明人提出可以基于此对扫码图片进行识别和确定。
如图6所示，视觉计算模块还可以包括文本识别模块，通过文本识别模块(OCR识别)对扫码图片进行识别，可以得到第四识别结果(如图6中的识别结果3)。第四识别结果用于指示所述活动码周围的文本是否存在异常。
具体来说，文本识别模块可以检测扫码图片中的文本数据与预设的文本数据的比较结果，如果比较结果指示扫码图片中的文本数据与预设的文本数据一致，则第四识别结果指示活动码周围的文本没有异常，则确定扫码图片为目标图片。否则，第四识别结果指示活动码周围的文本存在异常，确定扫码图片为非目标图片。
预设的文本数据也可以称为正常业务逻辑的数据，对于品牌方的商品而言，其预设的文本数据是已知的，例如，可以是首字母大写+10个数字+品牌名称。在扫码图片中的文本数据的内容或格式与此不同时，比较结果为不一致。
对于一些极端情况下，黑产中还经常会使用一些图片重复进行兑奖，或者是使用一些已经兑奖过的图片进行兑奖。为了对此进行识别，如图6所示，视觉计算模块还可以包括重复检测模块。
将扫码图片输入重复检测模块后，重复检测模块可以对扫码图片与预设图像库中的图片的相似性进行识别，得到第五识别结果(如图6中的识别结果4)。如果第五识别结果指示扫码图片与预设图像数据库中的图片高度相似，则确定扫码图片为非目标图片。如果第五识别结果指示扫码图片与预设图像数据库中的图片不相似，则确定扫码图片为目标图片。
本公开实施例对重复检测模块的检测方法不作具体的限定。作为一种实现方式，如图9所示，重复检测模块是一个图像检索系统CBIR(Content based Image Retrieval)。CBIR系统是由特征提取子系统和查询子系统两部分组成的。
在特征提取子系统中，将海量图像数据转化成embedding信息存放在图像库当中，步骤如下：1、预处理；例如，图像格式转换、规则化，图像的增强与去噪等。2、提取用户感兴趣的区域，然后在提取特征进行目标识别；在一些实施例中此步骤非必须的，可根据研究方向自行设置。3、特征提取。可以是基于颜色、纹理、形状、空间关系的 特征也可以是通过CNN进行特征提取。4、数据库，由图像库、特征库和知识库组成。图像库为数字化的图像信息，主要用于检索后结果的返回；特征库包含自动提取的内容特征，是检索的关键信息；知识库中知识表达可以更换以便适用不同的领域，知识库是用来辅助查询条件，主要用来过滤。
在查询子系统中，通过用户输入新的图像和查询条件，从图像库中返回相似的图像，其一般步骤是：1、查询接口；提供用户定制化搜索的能力，可以是界面或者接口的形式等。2、检索引擎；检索引擎主要是进行相似性测度，里面包含一个有效可靠的相似性测度函数集。3、索引/过滤。通过索引/过滤实现快速的检索。
重复检测是营销反欺诈中的一类典型风险，即在图片底库中存在与扫码图片完全一致或高度相似的图片。图片重复在业务上称之为批量风险，大概率上意味着团伙作案，识别批量风险有着极高的业务价值。
需要理解的是，本公开实施例中的扫码图片的识别方法和识别结果可以是上述方法和识别结果的任意组合。例如，以图5为例，图5中的视觉引擎中的图像识别模块可以为图6中的图像分类模块、防伪冒检测模块、文本识别模块、重复检测模块的融合模块，并且图5中的图像识别模块的输出结果也可以是图6中的识别结果1、2、3和4的融合结果，其融合结果可以最终表示为实体码的表征值、电子码的表征值以及背景区域的标签。可以理解的是，识别方法不限于图6中的4个模块的检测方法也可以是其他的图片检测方法。通过多种识别方法的融合，可以从多个角度识别扫码图片，以对抗常见的黑产攻击，从而使得识别结果更准确，提高对黑产的欺诈的识别效果。
另外，为了防止商品的实体上的活动码的提前泄漏，本公开实施例提出了一种以数字安全链为基础的防护措施。如图10所示，商品的实体上的活动码的生产、储存、传输、喷印和报废环节都设置在区块链上，实现数据的加密传输、解密限制、扫码溯源、数据传输追踪等一些列安全问题，可以实现码的生产、存储、传输、喷印、报废环节的定点追踪，降低加工和流转过程中的泄露风险。
上文结合图1至图10，详细描述了本公开的方法实施例，下面结合图11，详细描述本公开的装置实施例。应理解，方法实施例的描述与装置实施例的描述相互对应，因此，未详细描述的部分可以参见前面方法实施例。
图11是本公开一实施例提供的的装置的示意性结构图。该装置1100可以包括接收模块1110和确定模块1120。下面对这些模块进行详细介绍。
接收模块1110用于接收扫码图片，所述扫码片包含商品的营销活动对应的活动码；确定模块1120用于对所述扫码图片进行图像识别，以确定所述扫码图片是否为目标图 片，所述目标图片为对所述商品的实体上的所述活动码进行扫描后得到的图片。
可选地，所述确定模块用于对所述扫码图片进行图像识别，以得到第一识别结果，所述第一识别结果用于指示所述扫码图片中的活动码的标签，所述活动码的标签包括实体码和电子码；以及用于根据所述第一识别结果，确定所述扫码图片是否为所述目标图片。
可选地，所述确定模块用于对所述扫码图片进行图像识别，以得到第二识别结果，所述第二识别结果用于指示所述扫码图片中的背景区域的标签，所述背景区域的标签包括以下标签中的至少一种：用于指示所述背景区域的颜色的标签，用于指示所述背景区域中的物体的类型的标签，用于指示所述背景区域中的物体的材质的标签；以及用于根据所述第二识别结果，确定所述扫码图片是否为所述目标图片。
可选地，所述确定模块用于对所述扫码图片进行图像识别，以得到第三识别结果，所述第三识别结果用于指示所述扫码图片是否为：经过编辑的图片，通过截屏得到的图片，和/或，通过拍屏得到的图片；以及用于根据所述第三识别结果，确定所述扫码图片是否为所述目标图片。
可选地，所述确定模块用于利用目标神经网络对所述扫码图片进行图像识别，以识别所述扫码图片是否为经过编辑的图片，所述目标神经网络包括以下神经网络中的至少两种神经网络：RGB-N篡改检测网络、ManTraNet篡改检测网络和EXIF-Consistency篡改检测网络。
可选地，所述确定模块用于对所述扫码图片中的所述活动码周围的文本进行识别，得到第四识别结果，所述第四识别结果用于指示所述活动码周围的文本是否存在异常；以及用于根据所述第四识别结果，确定所述扫码图片是否为目标图片。
可选地，所述确定模块用于对所述扫码图片与预设图像库中的图片的相似性进行识别，得到第五识别结果；以及用于根据所述第五识别结果确定所述扫码图片是否为目标图片。
图12是本公开又一实施例提供的识别图片装置的结构示意图。该装置1200例如可以是具有计算功能的计算设备。比如，装置1200可以是移动终端或者服务器。装置1200可以包括存储器1210和处理器1220。存储器1210可用于存储可执行代码。处理器1220可用于执行所述存储器1210中存储的可执行代码，以实现前文描述的各个方法中的步骤。在一些实施例中，该装置1200还可以包括网络接口1230，处理器1220与外部设备的数据交换可以通过该网络接口1230实现。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其他任意组合来实 现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本公开实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(Digital Subscriber Line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如数字视频光盘(Digital Video Disc，DVD))、或者半导体介质(例如固态硬盘(Solid State Disk，SSD))等。
本领域普通技术人员可以意识到，结合本公开实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。
在本公开所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
以上所述，仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，可轻易想到变化或替换，都应 涵盖在本公开的保护范围之内。因此，本公开的保护范围应以所述权利要求的保护范围为准。

Claims (15)

  1. 一种识别图片的方法，所述方法包括：
    接收扫码图片，所述扫码图片包含商品的营销活动对应的活动码；
    对所述扫码图片进行图像识别，以确定所述扫码图片是否为目标图片，所述目标图片为对所述商品的实体上的所述活动码进行扫描后得到的图片。
  2. 根据权利要求1所述的方法，所述对所述扫码图片进行图像识别，以确定所述扫码图片是否为目标图片，包括：
    对所述扫码图片进行图像识别，以得到第一识别结果，所述第一识别结果用于指示所述扫码图片中的活动码的标签，所述活动码的标签包括实体码和电子码；
    根据所述第一识别结果，确定所述扫码图片是否为所述目标图片。
  3. 根据权利要求1或2所述的方法，所述对所述扫码图片进行图像识别，以确定所述扫码图片是否为目标图片，包括：
    对所述扫码图片进行图像识别，以得到第二识别结果，所述第二识别结果用于指示所述扫码图片中的背景区域的标签，所述背景区域的标签包括以下标签中的至少一种：用于指示所述背景区域的颜色的标签，用于指示所述背景区域中的物体的类型的标签，用于指示所述背景区域中的物体的材质的标签；
    根据所述第二识别结果，确定所述扫码图片是否为所述目标图片。
  4. 根据权利要求1所述的方法，所述对所述扫码图片进行图像识别，以确定所述扫码图片是否为目标图片，包括：
    对所述扫码图片进行图像识别，以得到第三识别结果，所述第三识别结果用于指示所述扫码图片是否为：经过编辑的图片，通过截屏得到的图片，和/或，通过拍屏得到的图片；
    根据所述第三识别结果，确定所述扫码图片是否为所述目标图片。
  5. 根据权利要求4所述的方法，所述对所述扫码图片进行图像识别，包括：
    利用目标神经网络对所述扫码图片进行图像识别，以识别所述扫码图片是否为经过编辑的图片，所述目标神经网络包括以下神经网络中的至少两种神经网络：RGB-N篡改检测网络、ManTraNet篡改检测网络和EXIF-Consistency篡改检测网络。
  6. 根据权利要求1所述的方法，所述对所述扫码图片进行图像识别，以确定所述扫码图片是否为目标图片，包括：
    对所述扫码图片中的所述活动码周围的文本进行识别，得到第四识别结果，所述第四识别结果用于指示所述活动码周围的文本是否存在异常；
    根据所述第四识别结果，确定所述扫码图片是否为目标图片。
  7. 根据权利要求1所述的方法，所述对所述扫码图片进行图像识别，以确定所述扫码图片是否为目标图片，包括：
    对所述扫码图片与预设图像库中的图片的相似性进行识别，得到第五识别结果；
    根据所述第五识别结果确定所述扫码图片是否为目标图片。
  8. 一种识别图片的装置，所述装置包括：
    接收模块，用于接收扫码图片，所述扫码图片包含商品的营销活动对应的活动码；
    确定模块，用于对所述扫码图片进行图像识别，以确定所述扫码图片是否为目标图片，所述目标图片为对所述商品的实体上的所述活动码进行扫描后得到的图片。
  9. 根据权利要求8所述的装置，所述确定模块用于：
    对所述扫码图片进行图像识别，以得到第一识别结果，所述第一识别结果用于指示所述扫码图片中的活动码的标签，所述活动码的标签包括实体码和电子码；
    根据所述第一识别结果，确定所述扫码图片是否为所述目标图片。
  10. 根据权利要求8或9所述的装置，所述确定模块用于：
    对所述扫码图片进行图像识别，以得到第二识别结果，所述第二识别结果用于指示所述扫码图片中的背景区域的标签，所述背景区域的标签包括以下标签中的至少一种：用于指示所述背景区域的颜色的标签，用于指示所述背景区域中的物体的类型的标签，用于指示所述背景区域中的物体的材质的标签；
    根据所述第二识别结果，确定所述扫码图片是否为所述目标图片。
  11. 根据权利要求8所述的装置，所述确定模块用于：
    对所述扫码图片进行图像识别，以得到第三识别结果，所述第三识别结果用于指示所述扫码图片是否为：经过编辑的图片，通过截屏得到的图片，和/或，通过拍屏得到的图片；
    根据所述第三识别结果，确定所述扫码图片是否为所述目标图片。
  12. 根据权利要求11所述的装置，所述确定模块用于：
    利用目标神经网络对所述扫码图片进行图像识别，以识别所述扫码图片是否为经过编辑的图片，所述目标神经网络包括以下神经网络中的至少两种神经网络：RGB-N篡改检测网络、ManTraNet篡改检测网络和EXIF-Consistency篡改检测网络。
  13. 根据权利要求8所述的装置，所述确定模块用于：
    对所述扫码图片中的所述活动码周围的文本进行识别，得到第四识别结果，所述第四识别结果用于指示所述活动码周围的文本是否存在异常；
    根据所述第四识别结果，确定所述扫码图片是否为目标图片。
  14. 根据权利要求8所述的装置，所述确定模块用于：
    对所述扫码图片与预设图像库中的图片的相似性进行识别，得到第五识别结果；
    根据所述第五识别结果确定所述扫码图片是否为目标图片。
  15. 一种识别图片的装置，包括存储器和处理器，所述存储器中存储有可执行代码，所述处理器被配置为执行所述可执行代码，以实现权利要求1-7中任一项所述的方法。
PCT/CN2022/107821 2021-09-22 2022-07-26 识别图片的方法和装置 WO2023045535A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111117739.7A CN113688650B (zh) 2021-09-22 2021-09-22 识别图片的方法及装置
CN202111117739.7 2021-09-22

Publications (1)

Publication Number Publication Date
WO2023045535A1 true WO2023045535A1 (zh) 2023-03-30

Family

ID=78586996

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/107821 WO2023045535A1 (zh) 2021-09-22 2022-07-26 识别图片的方法和装置

Country Status (2)

Country Link
CN (1) CN113688650B (zh)
WO (1) WO2023045535A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113688650B (zh) * 2021-09-22 2022-06-17 支付宝(杭州)信息技术有限公司 识别图片的方法及装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140214514A1 (en) * 2013-01-31 2014-07-31 Wal-Mart Stores, Inc. Methods and systems for automated coupon processing
CN104616163A (zh) * 2013-11-05 2015-05-13 北大方正集团有限公司 抽奖识别码管理方法及装置
CN107403216A (zh) * 2017-05-25 2017-11-28 阿里巴巴集团控股有限公司 一种识别码生成及验证方法及装置
CN111353488A (zh) * 2020-02-25 2020-06-30 支付宝(杭州)信息技术有限公司 识别码图像中的风险的方法、装置以及设备
CN113255857A (zh) * 2021-05-28 2021-08-13 支付宝(杭州)信息技术有限公司 一种图形码的风险检测方法、装置及设备
CN113688650A (zh) * 2021-09-22 2021-11-23 支付宝(杭州)信息技术有限公司 识别图片的方法及装置
CN113963352A (zh) * 2021-09-22 2022-01-21 支付宝(杭州)信息技术有限公司 识别图片和训练神经网络的方法和装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104318277B (zh) * 2014-10-10 2018-12-18 南京大树智能科技股份有限公司 一种商品销售过程检测装置
CN104809622A (zh) * 2015-05-01 2015-07-29 海南亚元防伪技术研究所 超限纹理防伪方法
CN108492129A (zh) * 2018-03-02 2018-09-04 严东军 一种智慧兑奖的方法与平台
CN109345253A (zh) * 2018-09-04 2019-02-15 阿里巴巴集团控股有限公司 资源转移方法、装置及系统
CN110751490A (zh) * 2019-10-22 2020-02-04 中信银行股份有限公司 欺诈识别方法、装置、电子设备及计算机可读存储介质
CN111950673B (zh) * 2020-08-11 2021-06-08 浙江码尚科技股份有限公司 基于二维码的商品防伪验证方法、装置、设备及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140214514A1 (en) * 2013-01-31 2014-07-31 Wal-Mart Stores, Inc. Methods and systems for automated coupon processing
CN104616163A (zh) * 2013-11-05 2015-05-13 北大方正集团有限公司 抽奖识别码管理方法及装置
CN107403216A (zh) * 2017-05-25 2017-11-28 阿里巴巴集团控股有限公司 一种识别码生成及验证方法及装置
CN111353488A (zh) * 2020-02-25 2020-06-30 支付宝(杭州)信息技术有限公司 识别码图像中的风险的方法、装置以及设备
CN113255857A (zh) * 2021-05-28 2021-08-13 支付宝(杭州)信息技术有限公司 一种图形码的风险检测方法、装置及设备
CN113688650A (zh) * 2021-09-22 2021-11-23 支付宝(杭州)信息技术有限公司 识别图片的方法及装置
CN113963352A (zh) * 2021-09-22 2022-01-21 支付宝(杭州)信息技术有限公司 识别图片和训练神经网络的方法和装置

Also Published As

Publication number Publication date
CN113688650B (zh) 2022-06-17
CN113688650A (zh) 2021-11-23

Similar Documents

Publication Publication Date Title
Pacheco et al. Uncovering coordinated networks on social media: methods and case studies
Goel et al. Dual branch convolutional neural network for copy move forgery detection
CN112381775B (zh) 一种图像篡改检测方法、终端设备及存储介质
KR100815530B1 (ko) 유해성 컨텐츠 필터링 방법 및 시스템
CN108734184B (zh) 一种对敏感图像进行分析的方法及装置
CN112231484B (zh) 一种新闻评论审核方法、系统、装置和存储介质
Lago et al. Visual and textual analysis for image trustworthiness assessment within online news
CN107807941A (zh) 信息处理方法和装置
CN112258254B (zh) 基于大数据架构的互联网广告风险监测方法及系统
WO2023045535A1 (zh) 识别图片的方法和装置
CN109284613A (zh) 标识检测及仿冒站点检测方法、装置、设备及存储介质
US9471676B1 (en) System and method for suggesting keywords based on image contents
Nowroozi et al. Impact of media forensics and deepfake in society
CN110533704A (zh) 油墨标签的识别验伪方法、装置、设备及介质
CN111444362B (zh) 恶意图片拦截方法、装置、设备和存储介质
KR102650139B1 (ko) 인공지능 기반 온라인 위조상품 단속 시스템 및 방법
CN113537206A (zh) 推送数据检测方法、装置、计算机设备和存储介质
Golubev et al. Validation of Real Estate Ads based on the Identification of Identical Images
CN113963352B (zh) 识别图片和训练神经网络的方法和装置
Ravindran et al. Towards sustainable North American wood product value chains, part 2: Computer vision identification of ring-porous hardwoods
CN116010707A (zh) 商品价格异常识别方法、装置、设备和存储介质
US11757916B1 (en) Methods and apparatus for analyzing and scoring digital risks
Aydin Comparison of color features on copy-move forgery detection problem using HSV color space
Tanniru et al. Online Fake Logo Detection System
Tay et al. Privobfnet: A weakly supervised semantic segmentation model for data protection

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22871594

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE