CN114418601A - Video image-based method and device for detecting billing, computer equipment and medium - Google Patents
Video image-based method and device for detecting billing, computer equipment and medium Download PDFInfo
- Publication number
- CN114418601A CN114418601A CN202210070730.3A CN202210070730A CN114418601A CN 114418601 A CN114418601 A CN 114418601A CN 202210070730 A CN202210070730 A CN 202210070730A CN 114418601 A CN114418601 A CN 114418601A
- Authority
- CN
- China
- Prior art keywords
- customer
- order
- video image
- judging
- customers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/018—Certifying business or products
- G06Q30/0185—Product, service or business identity fraud
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0633—Lists, e.g. purchase orders, compilation or processing
- G06Q30/0635—Processing of requisition or of purchase orders
Landscapes
- Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device, computer equipment and a medium for detecting a refresh bill based on a video image, wherein the method comprises the steps of obtaining the video image corresponding to an order transaction moment; detecting whether a customer exists in the video image; if so, judging whether the customer is in the set transaction area; if the transaction area is within the transaction area, determining that the customer is valid; counting the number of valid customers; judging whether a new customer exists in the valid customers; if a new customer exists, judging that the current trading order is an effective order; if no new customer exists, the order count is increased by a quantity; counting whether the accumulated order quantity at the order trading moment is greater than the effective customer quantity in the video image corresponding to the order trading moment; and if the number of the effective customers in the video image corresponding to the order trading moment is larger than the number of the effective customers in the video image corresponding to the order trading moment, judging that the current trading order is an invalid order. The invention can detect scenes such as no customer list brushing and multiple list brushing of the same person, does not need to analyze historical data, and has strong attack resistance.
Description
Technical Field
The invention relates to a transaction data processing technology, in particular to a method and a device for detecting a bill swiping based on a video image, computer equipment and a medium.
Background
For the detection of the form brushing, the method in the prior art is to analyze on the basis of historical order data, when a novel form brushing hand appears, the situation that the form brushing cannot be identified may occur, a detection mechanism is easily identified and avoided, and the form brushing can be cracked only by using random prices and commodities at any time and carrying out a small amount of form brushing.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method, a device, computer equipment and a medium for detecting a video image-based brush bill.
In order to achieve the purpose, the invention adopts the following technical scheme:
in a first aspect, a method for video image-based swipe detection, the method comprising:
acquiring a video image corresponding to the order transaction time;
detecting whether a customer exists in the video image;
if no customer exists, judging the current trading order as an invalid order;
if so, judging whether the customer is in the set transaction area;
if not, executing the step of judging the current trade order as an invalid order;
if the transaction area is within the transaction area, determining that the customer is valid;
counting the number of valid customers;
judging whether a new customer exists in the valid customers;
if a new customer exists, judging that the current trading order is an effective order;
if no new customer exists, the order count is increased by a quantity;
counting whether the accumulated order quantity at the order trading moment is greater than the effective customer quantity in the video image corresponding to the order trading moment;
if the number of the effective customers in the video image corresponding to the order trading moment is larger than the number of the effective customers in the video image corresponding to the order trading moment, the step of judging the current trading order to be an invalid order is executed;
and if the number of the effective customers in the video image corresponding to the order trading moment is not greater than the number of the effective customers in the video image corresponding to the order trading moment, executing the step of judging that the current trading order is an effective order.
The further technical scheme is as follows: and detecting whether the customer exists in the video image or not by using a target detection algorithm.
The further technical scheme is as follows: the detecting whether the customer exists in the video image through the target detection algorithm specifically comprises the following steps:
collecting store transaction scene images as a training sample set, wherein the scene images comprise cashiers, customers and cash registers;
framing and marking categories for cashiers, customers and cash registers in the images by using rectangular frames;
inputting the marked training set into a deep neural network for training to obtain three output results, wherein the three output results are respectively a prediction frame of a target, a target category and a category confidence coefficient;
and detecting cash registers, customers and cashiers appearing in the scene images by using the trained neural network model.
The further technical scheme is as follows: the judging whether a new customer exists in the valid customers specifically comprises the following steps:
extracting features of each valid customer using a pedestrian re-identification algorithm based on a deep convolutional neural network;
comparing the characteristics of each valid customer with the characteristics existing in the customer queue one by one;
and if the cosine similarity between a certain customer and all the characteristics in the queue is smaller than the set similarity threshold, the customer is considered as a new customer.
In a second aspect, a video image-based swipe detection apparatus includes an acquisition unit, a detection unit, a first determination unit, a second determination unit, a first statistic unit, a second determination unit, a third determination unit, an accumulation unit, and a second statistic unit;
the acquisition unit is used for acquiring a video image corresponding to the order transaction time;
the detection unit is used for detecting whether a customer exists in the video image;
the first judging unit is used for judging the current trading order as an invalid order;
the first judging unit is used for judging whether a customer is in a set transaction area or not, and if not, the first judging unit is executed;
the second determination unit is used for determining that the customer is valid;
the first statistical unit is used for counting the number of valid customers;
the second judging unit is used for judging whether a new customer exists in the valid customers;
the third judging unit is used for judging the current trading order as an effective order;
the accumulation unit is used for increasing the order count by 1 quantity;
the second counting unit is used for counting whether the order quantity accumulated at the order trading moment is larger than the effective customer quantity in the video image corresponding to the order trading moment, if so, the first judging unit is executed, and if not, the third judging unit is executed.
The further technical scheme is as follows: and in the detection unit, whether a customer exists in the video image is detected through a target detection algorithm.
The further technical scheme is as follows: the detection unit comprises a collection module, a marking module, a training module and a detection module;
the system comprises a collecting module, a training module and a processing module, wherein the collecting module is used for collecting store transaction scene images as a training sample set, and the scene images comprise cashiers, customers and cash registers;
the marking module is used for framing out the cashier, the customer and the cash register in the image by using a rectangular frame and marking the category;
the training module is used for inputting the marked training set into the deep neural network for training to obtain three output results, wherein the three output results are respectively a prediction frame of the target, a target category and a category confidence coefficient;
and the detection module is used for detecting a cash register, a customer and a cashier appearing in the scene image by using the trained neural network model.
The further technical scheme is as follows: the second judgment unit comprises an extraction module, a comparison module and a judgment module;
the extraction module is used for extracting the characteristics of each effective customer by using a pedestrian re-identification algorithm based on a deep convolutional neural network;
the comparison module is used for comparing the characteristics of each effective customer with the characteristics existing in the customer queue one by one;
and the judging module is used for considering a certain customer as a new customer if the cosine similarity between the customer and all the characteristics in the queue is smaller than a set similarity threshold.
In a third aspect, a computer device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method steps as described above when executing the computer program.
In a fourth aspect, a computer-readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method steps as described above.
Compared with the prior art, the invention has the beneficial effects that: according to the invention, the video image is analyzed to carry out the bill-swiping identification, so that scenes such as no customer bill-swiping and multiple bill-swiping of the same person can be detected, historical data does not need to be analyzed, and the attack resistance is strong. The detected list-swiping scene is clear and definite, does not need to be confirmed with the party, avoids the step of monitoring and surveying, can directly keep the list-swiping screenshot, and is convenient, intuitive and clear at a glance.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical means of the present invention more clearly understood, the present invention may be implemented according to the content of the description, and in order to make the above and other objects, features, and advantages of the present invention more apparent, the following detailed description will be given of preferred embodiments.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a method for detecting a video image-based swipe bill according to an embodiment of the present invention;
FIG. 2 is a schematic block diagram of a video image-based swipe detection apparatus according to an embodiment of the present invention;
FIG. 3 is a schematic block diagram of a computer device provided in accordance with an embodiment of the present invention;
fig. 4 is a schematic diagram of scene division according to an embodiment of the present invention.
Detailed Description
In order to more fully understand the technical content of the present invention, the technical solution of the present invention will be further described and illustrated with reference to the following specific embodiments, but not limited thereto.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
The invention is based on the analysis of video images to perform the recognition of the list, therefore, a camera is needed. In practical application, the camera frame of the camera is arranged above the cashier area, so that the requirements of a cashier, a cash register and a customer in a transaction can be met.
As shown in fig. 4, the scene division in the video image is schematically illustrated, in which a circle represents a cashier, a rectangle represents a cash register, a triangle represents a customer, an upper rectangular box in the figure is divided into a customer area, a lower rectangular box in the figure is divided into a cash register area, and a dotted circular area is a transaction area. The pedestrians detected in the cashier area are identified as cashiers, the pedestrians detected in the customer area are identified as shop assistants when the work clothes are worn, and the pedestrians not worn are identified as customers. And setting a transaction range radius R by taking the cash register as a center, and dividing a transaction area, wherein a pedestrian centered in the transaction area is an effective customer, and a pedestrian outside the transaction area is an ineffective customer.
The customer area, the cash register area and the transaction area are divided on the image in advance according to actual scenes of different stores, and the device ID corresponding to each cash register in the image picture is associated.
The invention is described below by means of specific embodiments.
As shown in fig. 1, the method for detecting a video image-based swipe includes the following steps:
and S10, acquiring a video image corresponding to the order transaction time.
Reading order information, obtaining transaction time and transaction cash register equipment ID, obtaining a corresponding camera image according to the transaction time, wherein a common camera is provided with a video recording function and a software interface for calling a video screenshot, and obtaining an image of specified time by accessing the interface.
The cash registers, customers and cashiers in the image are detected by the object detection algorithm and the one used in the transaction is determined in the image based on the device ID.
Before reading the order information, the order count needs to be initialized, namely cleared, and the customer characteristic queue of the store of the day is emptied (the order count and the customer characteristic queue are used for the same person to repeatedly refresh the order scene). The orders are arranged in ascending order of time each day. The pedestrian re-recognition algorithm is used for extracting the pedestrian features of the customers, and the features are stored in a customer feature library of the current day and used for comparing whether the features are the same person or not. The feature queue is the above-mentioned feature library.
S20, detecting whether the video image has the customer, if yes, executing step S30, if no, executing step S301.
And detecting whether a customer exists in the video image through a target detection algorithm.
Specifically, the method for detecting whether a customer exists in a video image through a target detection algorithm specifically comprises the following steps:
s201, collecting store transaction scene images as a training sample set, wherein the scene images comprise cashiers, customers and cash registers;
s202, framing out the cashier, the customer and the cash register in the image by using rectangular frames and marking the categories;
s203, inputting the marked training set into a deep neural network for training to obtain three output results, wherein the three output results are respectively a prediction frame of a target, a target category and a category confidence coefficient;
and S204, detecting a cash register, a customer and a cashier appearing in the scene image by using the trained neural network model.
S301, judging that the current trading order is an invalid order.
S30, whether the customer is in the set trading area is judged, if yes, S40 is carried out, and if not, step S301 is carried out.
If a customer is at the boundary of the divided transaction area, the customer needs to judge according to the proportion of the body of the customer, if the majority of the body is occupied in the transaction area, the customer can be determined to be in the transaction area, if not, the customer is determined not to be in the transaction area, specifically, the judgment can still be analyzed by adopting a target detection algorithm, and redundant description is not repeated herein.
And S40, judging the customer as a valid customer.
And S50, counting the number of valid customers.
Since there are many customers at the same time at the time of order transaction, the number of valid customers is required.
S60, it is determined whether there is a new customer among the valid customers, if yes, step S801 is executed, and if no, step S70 is executed.
Specifically, step S60 specifically includes the following steps:
s601, extracting the characteristics of each effective customer by using a pedestrian re-identification algorithm based on a deep convolutional neural network.
S602, comparing the characteristics of each valid customer with the characteristics existing in the customer queue one by one.
And S603, if the cosine similarity between a certain customer and all the characteristics in the queue is smaller than the set similarity threshold, the customer is considered as a new customer.
S801, judging that the current trading order is an effective order.
And the customer's characteristics need to be added to the customer characteristic queue.
S70, the order count is incremented by one amount.
Since the order count is used for comparison with the number of valid customers, if the number of customers is greater than the number of valid customers, the order is determined to be an invalid order. Assuming that 3 valid customers exist in the scene and the personnel are unchanged, 3 orders are generated, and reasonably, one person buys one order without swiping the order; but if 4 orders are generated by the same 3 persons, the order 4 is judged to be an invalid order if 1 order is reasonably considered to be a refreshed order; if there is a change in personnel among the 3 customers with a new customer in them, the newly generated order can be reasonably considered as a valid order for the new customer, thus preventing the order from being judged as an invalid order.
S80, counting whether the order quantity accumulated at the order transaction time is greater than the effective customer quantity in the video image corresponding to the order transaction time, if yes, executing step S301, and if no, executing step S801.
According to the invention, the video image is analyzed to carry out the bill-swiping identification, so that scenes such as no customer bill-swiping and multiple bill-swiping of the same person can be detected, historical data does not need to be analyzed, and the attack resistance is strong. The detected list-swiping scene is clear and definite, does not need to be confirmed with the party, avoids the step of monitoring and surveying, can directly keep the list-swiping screenshot, and is convenient, intuitive and clear at a glance.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Corresponding to the method for detecting a video image-based swipe form, an embodiment of the present invention further provides a device for detecting a video image-based swipe form.
As shown in fig. 2, the video image-based swipe detecting apparatus 100 includes an acquiring unit 110, a detecting unit 120, a first determining unit 130, a first determining unit 140, a second determining unit 150, a first counting unit 160, a second determining unit 170, a third determining unit 180, an accumulating unit 190, and a second counting unit 200.
The obtaining unit 110 is configured to obtain a video image corresponding to an order transaction time.
Reading order information, obtaining transaction time and transaction cash register equipment ID, obtaining a corresponding camera image according to the transaction time, wherein a common camera is provided with a video recording function and a software interface for calling a video screenshot, and obtaining an image of specified time by accessing the interface.
The cash registers, customers and cashiers in the image are detected by the object detection algorithm and the one used in the transaction is determined in the image based on the device ID.
Before reading the order information, the order count needs to be initialized, namely cleared, and the customer characteristic queue of the store of the day is emptied (the order count and the customer characteristic queue are used for the same person to repeatedly refresh the order scene). The orders are arranged in ascending order of time each day. The pedestrian re-recognition algorithm is used for extracting the pedestrian features of the customers, and the features are stored in a customer feature library of the current day and used for comparing whether the features are the same person or not. The feature queue is the above-mentioned feature library.
The detecting unit 120 is configured to detect whether a customer is present in the video image.
And detecting whether a customer exists in the video image through a target detection algorithm.
Specifically, the detection unit 120 includes a collection module, a labeling module, a training module, and a detection module;
the system comprises a collecting module, a processing module and a processing module, wherein the collecting module is used for collecting store transaction scene images as a training sample set, and the scene images comprise cashiers, customers and cash registers;
the marking module is used for framing out the cashier, the customer and the cash register in the image by using a rectangular frame and marking the category;
the training module is used for inputting the marked training set into the deep neural network for training to obtain three output results, wherein the three output results are respectively a prediction frame of the target, a target category and a category confidence coefficient;
and the detection module is used for detecting a cash register, a customer and a cashier appearing in the scene image by using the trained neural network model.
A first determining unit 130, configured to determine that the current trade order is an invalid order.
The first determining unit 140 is used for determining whether the customer is in the set transaction area, and if the customer is not in the set transaction area, the first determining unit is executed.
If a customer is at the boundary of the divided transaction area, the customer needs to judge according to the proportion of the body of the customer, if the majority of the body is occupied in the transaction area, the customer can be determined to be in the transaction area, if not, the customer is determined not to be in the transaction area, specifically, the judgment can still be analyzed by adopting a target detection algorithm, and redundant description is not repeated herein.
The second determination unit 150 is configured to determine that the customer is a valid customer.
A first statistical unit 160 for counting the number of valid customers.
Since there are many customers at the same time at the time of order transaction, the number of valid customers is required.
The second determining unit 170 is configured to determine whether a new customer exists among the valid customers.
Specifically, the second determination unit 170 includes an extraction module, a comparison module, and a determination module.
And the extraction module is used for extracting the characteristics of each effective customer by using a pedestrian re-identification algorithm based on the deep convolutional neural network.
And the comparison module is used for comparing the characteristics of each effective customer with the characteristics existing in the customer queue one by one.
And the judging module is used for considering that the customer is a new customer if the cosine similarity between the customer and all the characteristics in the queue is smaller than the set similarity threshold.
A third determining unit 180, configured to determine that the current trade order is a valid order.
And the customer's characteristics need to be added to the customer characteristic queue.
And an accumulating unit 190 for increasing the order count by 1 quantity.
Since the order count is used for comparison with the number of valid customers, if the number of customers is greater than the number of valid customers, the order is determined to be an invalid order. Assuming that 3 valid customers exist in the scene and the personnel are unchanged, 3 orders are generated, and reasonably, one person buys one order without swiping the order; but if 4 orders are generated by the same 3 persons, the order 4 is judged to be an invalid order if 1 order is reasonably considered to be a refreshed order; if there is a change in personnel among the 3 customers with a new customer in them, the newly generated order can be reasonably considered as a valid order for the new customer, thus preventing the order from being judged as an invalid order.
The second counting unit 200 is configured to count whether the number of orders accumulated at the order trading time is greater than the number of valid customers in the video image corresponding to the order trading time, if yes, execute the first determining unit, and if not, execute the third determining unit.
According to the invention, the video image is analyzed to carry out the bill-swiping identification, so that scenes such as no customer bill-swiping and multiple bill-swiping of the same person can be detected, historical data does not need to be analyzed, and the attack resistance is strong. The detected list-swiping scene is clear and definite, does not need to be confirmed with the party, avoids the step of monitoring and surveying, can directly keep the list-swiping screenshot, and is convenient, intuitive and clear at a glance.
As shown in fig. 3, the embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the video image-based brush receipt detection method described above are implemented.
The computer device 700 may be a terminal or a server. The computer device 700 includes a processor 720, memory, and a network interface 750, which are connected by a system bus 710, where the memory may include non-volatile storage media 730 and internal memory 740.
The non-volatile storage medium 730 may store an operating system 731 and computer programs 732. The computer program 732, when executed, may cause the processor 720 to perform any of a variety of video image-based swipe detection methods.
The processor 720 is used to provide computing and control capabilities, supporting the operation of the overall computer device 700.
The internal memory 740 provides an environment for the execution of the computer program 732 in the non-volatile storage medium 730, and when the computer program 732 is executed by the processor 720, the processor 720 can be enabled to execute any one of the video image-based swipe detection methods.
The network interface 750 is used for network communication such as sending assigned tasks and the like. Those skilled in the art will appreciate that the configuration shown in fig. 3 is a block diagram of only a portion of the configuration relevant to the present teachings and is not intended to limit the computing device 700 to which the present teachings may be applied, and that a particular computing device 700 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components. Wherein the processor 720 is configured to execute the program code stored in the memory to perform the following steps:
acquiring a video image corresponding to the order transaction time;
detecting whether a customer exists in the video image;
if no customer exists, judging the current trading order as an invalid order;
if so, judging whether the customer is in the set transaction area;
if not, executing the step of judging the current trade order as an invalid order;
if the transaction area is within the transaction area, determining that the customer is valid;
counting the number of valid customers;
judging whether a new customer exists in the valid customers;
if a new customer exists, judging that the current trading order is an effective order;
if no new customer exists, the order count is increased by a quantity;
counting whether the accumulated order quantity at the order trading moment is greater than the effective customer quantity in the video image corresponding to the order trading moment;
if the number of the effective customers in the video image corresponding to the order trading moment is larger than the number of the effective customers in the video image corresponding to the order trading moment, the step of judging the current trading order to be an invalid order is executed;
and if the number of the effective customers in the video image corresponding to the order trading moment is not greater than the number of the effective customers in the video image corresponding to the order trading moment, executing the step of judging that the current trading order is an effective order.
The further technical scheme is as follows: and detecting whether the customer exists in the video image or not by using a target detection algorithm.
The further technical scheme is as follows: the detecting whether the customer exists in the video image through the target detection algorithm specifically comprises the following steps:
collecting store transaction scene images as a training sample set, wherein the scene images comprise cashiers, customers and cash registers;
framing and marking categories for cashiers, customers and cash registers in the images by using rectangular frames;
inputting the marked training set into a deep neural network for training to obtain three output results, wherein the three output results are respectively a prediction frame of a target, a target category and a category confidence coefficient;
and detecting cash registers, customers and cashiers appearing in the scene images by using the trained neural network model.
The further technical scheme is as follows: the judging whether a new customer exists in the valid customers specifically comprises the following steps:
extracting features of each valid customer using a pedestrian re-identification algorithm based on a deep convolutional neural network;
comparing the characteristics of each valid customer with the characteristics existing in the customer queue one by one;
and if the cosine similarity between a certain customer and all the characteristics in the queue is smaller than the set similarity threshold, the customer is considered as a new customer.
It should be understood that, in the embodiment of the present Application, the Processor 720 may be a Central Processing Unit (CPU), and the Processor 720 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Those skilled in the art will appreciate that the configuration of computer device 700 depicted in FIG. 3 is not intended to be limiting of computer device 700 and may include more or less components than those shown, or some components in combination, or a different arrangement of components.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present invention may be implemented in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the above-mentioned apparatus may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the modules or units is only one logical functional division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another device, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A video image-based method for detecting a swipe, the method comprising:
acquiring a video image corresponding to the order transaction time;
detecting whether a customer exists in the video image;
if no customer exists, judging the current trading order as an invalid order;
if so, judging whether the customer is in the set transaction area;
if not, executing the step of judging the current trade order as an invalid order;
if the transaction area is within the transaction area, determining that the customer is valid;
counting the number of valid customers;
judging whether a new customer exists in the valid customers;
if a new customer exists, judging that the current trading order is an effective order;
if no new customer exists, the order count is increased by a quantity;
counting whether the accumulated order quantity at the order trading moment is greater than the effective customer quantity in the video image corresponding to the order trading moment;
if the number of the effective customers in the video image corresponding to the order trading moment is larger than the number of the effective customers in the video image corresponding to the order trading moment, the step of judging the current trading order to be an invalid order is executed;
and if the number of the effective customers in the video image corresponding to the order trading moment is not greater than the number of the effective customers in the video image corresponding to the order trading moment, executing the step of judging that the current trading order is an effective order.
2. The method as claimed in claim 1, wherein the detecting the presence of the customer in the video image comprises detecting the presence of the customer in the video image by an object detection algorithm.
3. The video image-based method for detecting a video image-based billing according to claim 2, wherein the detecting whether the customer exists in the video image by the target detection algorithm specifically comprises:
collecting store transaction scene images as a training sample set, wherein the scene images comprise cashiers, customers and cash registers;
framing and marking categories for cashiers, customers and cash registers in the images by using rectangular frames;
inputting the marked training set into a deep neural network for training to obtain three output results, wherein the three output results are respectively a prediction frame of a target, a target category and a category confidence coefficient;
and detecting cash registers, customers and cashiers appearing in the scene images by using the trained neural network model.
4. The method for detecting a video-image-based ticket swiping according to claim 1, wherein said determining whether a new customer exists among the valid customers specifically comprises:
extracting features of each valid customer using a pedestrian re-identification algorithm based on a deep convolutional neural network;
comparing the characteristics of each valid customer with the characteristics existing in the customer queue one by one;
and if the cosine similarity between a certain customer and all the characteristics in the queue is smaller than the set similarity threshold, the customer is considered as a new customer.
5. The video image-based bill swiping detection device is characterized by comprising an acquisition unit, a detection unit, a first judgment unit, a second judgment unit, a first statistic unit, a second judgment unit, a third judgment unit, an accumulation unit and a second statistic unit;
the acquisition unit is used for acquiring a video image corresponding to the order transaction time;
the detection unit is used for detecting whether a customer exists in the video image;
the first judging unit is used for judging the current trading order as an invalid order;
the first judging unit is used for judging whether a customer is in a set transaction area or not, and if not, the first judging unit is executed;
the second determination unit is used for determining that the customer is valid;
the first statistical unit is used for counting the number of valid customers;
the second judging unit is used for judging whether a new customer exists in the valid customers;
the third judging unit is used for judging the current trading order as an effective order;
the accumulation unit is used for increasing the order count by 1 quantity;
the second counting unit is used for counting whether the order quantity accumulated at the order trading moment is larger than the effective customer quantity in the video image corresponding to the order trading moment, if so, the first judging unit is executed, and if not, the third judging unit is executed.
6. The apparatus according to claim 5, wherein the detection unit detects whether the customer is present in the video image by an object detection algorithm.
7. The video-image-based swipe detection device according to claim 6, wherein the detection unit comprises a collection module, a labeling module, a training module, and a detection module;
the system comprises a collecting module, a training module and a processing module, wherein the collecting module is used for collecting store transaction scene images as a training sample set, and the scene images comprise cashiers, customers and cash registers;
the marking module is used for framing out the cashier, the customer and the cash register in the image by using a rectangular frame and marking the category;
the training module is used for inputting the marked training set into the deep neural network for training to obtain three output results, wherein the three output results are respectively a prediction frame of the target, a target category and a category confidence coefficient;
and the detection module is used for detecting a cash register, a customer and a cashier appearing in the scene image by using the trained neural network model.
8. The apparatus according to claim 5, wherein the second determination unit comprises an extraction module, a comparison module and a determination module;
the extraction module is used for extracting the characteristics of each effective customer by using a pedestrian re-identification algorithm based on a deep convolutional neural network;
the comparison module is used for comparing the characteristics of each effective customer with the characteristics existing in the customer queue one by one;
and the judging module is used for considering a certain customer as a new customer if the cosine similarity between the customer and all the characteristics in the queue is smaller than a set similarity threshold.
9. A computer arrangement comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method steps of any of claims 1-4 when executing the computer program.
10. A computer-readable storage medium, characterized in that the storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to carry out the method steps according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210070730.3A CN114418601A (en) | 2022-01-21 | 2022-01-21 | Video image-based method and device for detecting billing, computer equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210070730.3A CN114418601A (en) | 2022-01-21 | 2022-01-21 | Video image-based method and device for detecting billing, computer equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114418601A true CN114418601A (en) | 2022-04-29 |
Family
ID=81274894
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210070730.3A Pending CN114418601A (en) | 2022-01-21 | 2022-01-21 | Video image-based method and device for detecting billing, computer equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114418601A (en) |
-
2022
- 2022-01-21 CN CN202210070730.3A patent/CN114418601A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107545262B (en) | Method and device for detecting text in natural scene image | |
| CN110866429B (en) | Missing scanning identification method, device, self-service cashing terminal and system | |
| CN109740573B (en) | Video analysis method, device, equipment and server | |
| CN108596109B (en) | Target detection method and device based on neural network and motion vector | |
| CN110598558A (en) | Crowd density estimation method, device, electronic equipment and medium | |
| WO2020062702A1 (en) | Method and device for sending text messages, computer device and storage medium | |
| CN112561080A (en) | Sample screening method, sample screening device and terminal equipment | |
| CN108647587B (en) | People counting method, device, terminal and storage medium | |
| WO2014193220A2 (en) | System and method for multiple license plates identification | |
| CN111783665A (en) | Action recognition method and device, storage medium and electronic equipment | |
| CN111666915A (en) | Monitoring method, device, equipment and storage medium | |
| CN111126119A (en) | Method and device for counting user behaviors arriving at store based on face recognition | |
| CN114913600A (en) | Electricity stealing detection method and device for electricity meter, electronic equipment and storage medium | |
| CN111461905A (en) | Vehicle insurance fraud and claim evasion method and device, computer equipment and storage medium | |
| CN111260685B (en) | Video processing method and device and electronic equipment | |
| CN112115745A (en) | Method, device and system for identifying code missing scanning behaviors of commodities | |
| CN114067403A (en) | Old people self-help money depositing and withdrawing obstacle recognition method and device | |
| KR20240101455A (en) | Information processing program, information processing method, and information processing device | |
| CN111079466A (en) | Vehicle identification method and device, electronic equipment and storage medium | |
| CN110503284B (en) | Statistical method and device based on queuing data | |
| CN114418601A (en) | Video image-based method and device for detecting billing, computer equipment and medium | |
| CN112818802B (en) | Bank counter personnel hand-lifting identification method and system | |
| CN108830183A (en) | A kind of the merchandise control method, apparatus and system of unmanned supermarket | |
| CN112417007A (en) | Data analysis method and device, electronic equipment and storage medium | |
| CN113298100A (en) | Data cleaning method, self-service equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |