WO2019169965A1

WO2019169965A1 - Code-scanning image recognition method, apparatus and device

Info

Publication number: WO2019169965A1
Application number: PCT/CN2019/072772
Authority: WO
Inventors: 杨磊磊; 方刚
Original assignee: 阿里巴巴集团控股有限公司
Priority date: 2018-03-07
Filing date: 2019-01-23
Publication date: 2019-09-12
Also published as: TW201939356A; CN108537085A; TWI769360B

Abstract

A code-scanning image recognition method, apparatus and device. The method comprises: when a code-scanning image is recognized, a GPU performing binarization processing on a current code-scanning image frame, then providing a binarization result to a CPU, and then continuously processing a next code-scanning image frame, meanwhile, the CPU is capable of recognizing the binarization result, and if the recognition fails, recognizing the next binarization result until the recognition succeeds.

Description

Scan code image recognition method, device and device

Cross-reference to related applications

The present application claims priority to Chinese Patent Application No. 201101186076.6, entitled "A Scanning Image Recognition Method, Apparatus, and Apparatus", filed on March 7, 2018, the entire disclosure of which is incorporated by reference. The manner is incorporated herein.

Technical field

The present specification relates to the field of computer software technologies, and in particular, to a method, device and device for scanning code image recognition.

Background technique

The popularity of smartphones has brought convenience to people's lives, and various services can be performed accordingly by using various applications on smartphones.

The smart phone often recognizes the two-dimensional code by scanning, obtains the business information contained in the two-dimensional code, and then interacts with another device based on the service information, thereby implementing a specific service, such as a payment service, an instant communication service, and the like.

In the prior art, when scanning the two-dimensional code, the smart phone uses the camera to align the two-dimensional code, continuously collects the scanned image frame, and the scanned image frame is scanned by the central processing unit (CPU) of the smart phone. Binarization processing is performed to identify until the information contained in the two-dimensional code is successfully identified.

Based on the prior art, a more efficient scan code recognition scheme is needed.

Summary of the invention

The embodiment of the present specification provides a method, a device and a device for scanning image recognition, which are used to solve the following technical problems: a more efficient scanning image recognition scheme is needed.

In order to solve the above technical problem, the embodiment of the present specification is implemented as follows:

A scanning image recognition method provided by an embodiment of the present specification includes:

Obtain the current scan code image frame;

Performing a binarization process on the current scan code image frame by using a graphics processing unit (GPU) to obtain a binarization result;

The binarization result is provided to the CPU to identify the binarization result by the CPU, and obtain a scan image recognition result.

A scanning image recognition device provided by an embodiment of the present specification includes:

Obtaining a module to obtain a current scan image frame;

a binarization module that performs binarization processing on the current scan code image frame by using a GPU to obtain a binarization result;

The identification module provides the binarization result to the CPU to identify the binarization result by using the CPU to obtain a scan image recognition result.

At least one processor; and,

a memory communicatively coupled to the at least one processor; wherein

The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

Obtain the current scan code image frame;

Using the GPU, performing binarization processing on the current scan code image frame to obtain a binarization result;

The above at least one technical solution adopted by the embodiment of the present specification can achieve the following beneficial effects: when identifying a plurality of consecutive scan code image frames, the GPU replaces the CPU for binarization processing, and the CPU only needs to further identify the binarization result. Yes, these two processes can be executed in parallel, which is beneficial to improve the efficiency of scanning image recognition.

DRAWINGS

In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings to be used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a few embodiments described in the present specification, and other drawings can be obtained from those skilled in the art without any creative work.

FIG. 1 is a schematic diagram of an overall architecture involved in an implementation scenario of the present specification;

2 is a schematic flow chart of a method for identifying a scan code image according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of an implementation principle of a scan code image recognition method according to an embodiment of the present disclosure;

FIG. 4 is a schematic structural diagram of a scan code image recognition apparatus corresponding to FIG. 2 according to an embodiment of the present disclosure; FIG.

FIG. 5 is a schematic structural diagram of a scan code image recognition device corresponding to FIG. 2 according to an embodiment of the present disclosure.

Detailed ways

The embodiments of the present specification provide a method, an apparatus, and a device for scanning code image recognition.

In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the specification. The embodiments are only a part of the embodiments of the present application, and not all of them. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present specification without departing from the inventive scope shall fall within the scope of the application.

FIG. 1 is a schematic diagram of an overall architecture involved in an implementation scenario of the present specification. The overall architecture mainly involves a camera, a GPU, and a CPU on a user terminal device. User terminal equipment, for example, includes a smart phone, a tablet computer, a merchant cash register, a point of sale (POS) machine, and the like. The camera is used to continuously collect scan code image frames when the user scans the code, and the GPU is used for binarizing the scan code image frames, and the CPU is used to identify the binarization results, thereby realizing the scan code image recognition.

The scheme of the present specification will be described below mainly based on the exemplary architecture of FIG.

FIG. 2 is a schematic flowchart of a method for identifying a scanned image according to an embodiment of the present disclosure. The flow may also be performed based on a simple operation trigger of the user (such as clicking a “sweep” button), or if necessary, It may also be performed automatically at the right time.

From the perspective of the device, the execution body of the process may be a user terminal device. From a program perspective, the execution body of the process may be a program installed in the above device, and the program may be in the form of a client, a webpage, or a server.

The process in Figure 2 can include the following steps:

S202: Acquire a current scan code image frame.

In the embodiment of the present specification, for the scene of the background art, scanning code mainly refers to scanning a two-dimensional code. Of course, in addition to the two-dimensional code, it is also possible to use a Digital Object Unique Identifier (DOI) such as a barcode, a Uniform Resource Locator (URL), etc., which may be identified by scanning. Refers to scanning the corresponding DOI.

In the embodiment of the present specification, for example, the scan image frame is continuously collected by the camera on the smart phone, and the smart phone is generally operated by the user, and the focus condition and the alignment situation may change at any time during the scanning process. The scanned image frames continuously collected are usually different. The information contained in the code may be successfully identified from some scanned image frames, and the code may not be successfully identified from other scanned image frames. Information contained. It is possible to sequentially identify each of the acquired scan image frames, and after successfully identifying the information contained in the code, stop the subsequent recognition process.

S204: Perform a binarization process on the current scan code image frame by using a GPU to obtain a binarization result.

In the embodiment of the present specification, the GPU is used for the CPU to share the work, and the scan code image frame to be recognized by the GPU is binarized, so that the CPU does not need to perform binarization processing, and the CPU is responsible for identifying the binarization result, thereby reducing The burden of the CPU.

S206: The binarization result is provided to the CPU, so that the binarization result is recognized by the CPU, and the scan code image recognition result is obtained.

In the embodiment of the present specification, assuming that only one scan image frame is used to successfully identify the information contained in the code, the entire scan image recognition process is only a serial operation process, and the GPU performs the scan image frame on the second. The value processing process is further successfully recognized by the CPU for the binarization result of the scanned image frame.

However, if multiple scan image frames are used to successfully identify the information contained in the code, since the GPU and the CPU can work in parallel, the entire scan image recognition process is a serial work plus parallel work process, serial finger to each The processing process of the scan code image frames is serial, and the parallel process means that the following two processes can be parallel: the process of using the CPU to identify the binarization result of a scan code image frame, and using the GPU binarization process The process of scanning a coded image frame behind a coded image frame.

In the embodiment of the present specification, the scan image recognition result is generally a character string included in the code, and the character string indicates, for example, a payment account, a jump URL, a business card, and the like.

Through the method of FIG. 2, when identifying a plurality of consecutive scan image frames, the GPU replaces the CPU for binarization processing, and the CPU only needs to further identify the binarization result, and the two processes can be executed in parallel. It is beneficial to improve the efficiency of scanning image recognition, and it is also beneficial to reduce the burden on the CPU.

Based on the method of FIG. 2, the embodiments of the present specification further provide some specific implementations of the method, and an extended solution, which will be described below.

In the embodiment of the present specification, in order to realize parallel operation of the GPU and the CPU in the process of scanning the image recognition, the work of the GPU and the CPU may be performed based on different threads or different processes without affecting each other.

For example, in step S204, the GPU is used to perform binarization processing on the current scan code image frame, which may include: the first thread uses the GPU to perform binarization processing on the current scan code image frame. . For the step S206, the identifying, by the CPU, the binarization result may include: the second thread using the CPU to identify the binarization result. The first thread and the second thread are different threads.

In the embodiment of the present specification, according to the foregoing description, the current one scan image frame may not be successfully identified. Therefore, after the binarization result is provided to the CPU, the step may be utilized. The GPU continues to binarize the next scanned image frame. Of course, the GPU's continuation process does not need to wait for the CPU to process the binarization result, and the process of using the CPU to identify the binarization result and the process of using the GPU binarization process to scan the coded image frame can be performed in parallel.

In the embodiment of the present specification, for step S206, the specific implementation scheme of providing the binarization result to the CPU is diverse. For example, after the CPU obtains the binarization result, and sends a corresponding notification (for example, to a function module that uses the CPU to identify the binarization result, the function module is referred to herein as a CPU identification module) so that the CPU identification module obtains the The result of the binarization in the result pool, or the binarization result may be sent to the CPU identification module actively, or the CPU identification module may not actively and only passively acquire the binarization result. After the CPU identification module obtains the binarization result, it can be provided to the CPU for use.

In the embodiment of the present specification, the GPU may perform binarization processing on multiple scan code image frames, and accordingly, multiple binarization results are obtained. In order to facilitate the use of the CPU, these binarization results can be uniformly saved. For example, a result pool can be established for storing the binarization results for use by the CPU, and the result pool is not limited in position, such as in memory or cache. In the volatile memory, it can also be in a non-volatile memory such as a hard disk or a flash memory.

For example, for the step S206, the providing the binarization result to the CPU may include: saving the binarization result in a result pool, and sending a corresponding notification for the CPU to obtain the location in the result pool. The binarization result is described. Further, if the CPU fails to identify the binarization result, the CPU may be used to obtain the next binarization result from the result pool for identification until the recognition is successful.

According to the above description, the implementation principle of the above-mentioned scan code image recognition method in an actual application scenario is also shown in FIG. 3 .

In FIG. 3, each scan code image frame is continuously binarized by the GPU, and the obtained binarization results are saved in the result pool, when there is a new binarization result in the binarization result pool. And notifying the CPU identification module to take the binarization result, and the CPU identification module uses the CPU to identify the binarization result, and if the recognition is successful, the result is returned, and if the recognition fails, the next binarization result is obtained from the result pool to continue to identify Until the recognition is successful.

Based on the same idea, the embodiment of the present specification further provides a device and a device corresponding to the foregoing method, as shown in FIG. 4 and FIG. 5.

FIG. 4 is a schematic structural diagram of a scan code image recognition apparatus corresponding to FIG. 2 according to an embodiment of the present disclosure, where the apparatus includes:

Obtaining module 401, acquiring a current scan code image frame;

The binarization module 402 performs binarization processing on the current scan code image frame by using a GPU to obtain a binarization result;

The identification module 403 provides the binarization result to the CPU to identify the binarization result by the CPU to obtain a scan image recognition result.

Optionally, the binarization module 402 performs a binarization process on the current scan code image frame by using a GPU, and specifically includes:

The binarization module 402 performs binarization processing on the current scan code image frame by using a GPU through a first thread;

The identification module 403 uses the CPU to identify the binarization result, and specifically includes:

The identification module 403 identifies the binarization result by using the CPU by using a second thread, where the first thread and the second thread are different threads.

Optionally, after the identification module 403 provides the binarization result to the CPU, the binarization module 402 performs binarization processing on the next scan image frame by using the GPU, where The process of the CPU identifying the binarization result and the process of scanning the coded image frame by the GPU binarization process can be performed in parallel.

Optionally, the identifying module 403 provides the binarization result to the CPU, and specifically includes:

The identification module 403 saves the binarization result in the result pool, and sends a corresponding notification for the CPU to obtain the binarization result in the result pool.

Optionally, if the CPU fails to identify the binarization result, the identifying module 403 uses the CPU to acquire the next binarization result from the result pool for identification.

Optionally, the scan code comprises scanning a two-dimensional code.

FIG. 5 is a schematic structural diagram of a scan code image recognition device corresponding to FIG. 2 according to an embodiment of the present disclosure, where the device includes:

At least one processor; and,

a memory communicatively coupled to the at least one processor; wherein

Obtain the current scan code image frame;

Based on the same idea, the embodiment of the present specification further provides a non-volatile computer storage medium corresponding to FIG. 2, which stores computer executable instructions, and the computer executable instructions are set as:

Obtain the current scan code image frame;

The foregoing description of the specific embodiments of the specification has been described. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the embodiments and still achieve the desired results. In addition, the processes depicted in the figures are not necessarily in a particular order or in a sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The various embodiments in the specification are described in a progressive manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for a device, device, non-volatile computer storage medium embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

The device, the device, the non-volatile computer storage medium and the method provided by the embodiments of the present specification are corresponding, and therefore, the device, the device, and the non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method, since The beneficial technical effects of the method are described in detail, and therefore, the beneficial technical effects of the corresponding device, device, and non-volatile computer storage medium are not described herein.

In the 1990s, improvements to a technology could clearly distinguish between hardware improvements (eg, improvements to circuit structures such as diodes, transistors, switches, etc.) or software improvements (for process flow improvements). However, as technology advances, many of today's method flow improvements can be seen as direct improvements in hardware circuit architecture. Designers almost always get the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented by hardware entity modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is an integrated circuit whose logic function is determined by the user programming the device. Designers program themselves to "integrate" a digital system on a single PLD without having to ask the chip manufacturer to design and fabricate a dedicated integrated circuit chip. Moreover, today, instead of manually making integrated circuit chips, this programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in programming development, but before compiling The original code has to be written in a specific programming language. This is called the Hardware Description Language (HDL). HDL is not the only one, but there are many kinds, such as ABEL (Advanced Boolean Expression Language). AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., are currently the most commonly used VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should also be apparent to those skilled in the art that the hardware flow for implementing the logic method flow can be easily obtained by simply programming the method flow into the integrated circuit with a few hardware description languages.

The controller can be implemented in any suitable manner, for example, the controller can take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (eg, software or firmware) executable by the (micro)processor. In the form of logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, The Microchip PIC18F26K20 and the Silicone Labs C8051F320, the memory controller can also be implemented as part of the memory's control logic. Those skilled in the art will also appreciate that in addition to implementing the controller in purely computer readable program code, the controller can be logically programmed by means of logic gates, switches, ASICs, programmable logic controllers, and embedding. The form of a microcontroller or the like to achieve the same function. Such a controller can therefore be considered a hardware component, and the means for implementing various functions included therein can also be considered as a structure within the hardware component. Or even a device for implementing various functions can be considered as a software module that can be both a method of implementation and a structure within a hardware component.

The system, device, module or unit illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product having a certain function. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or A combination of any of these devices.

For the convenience of description, the above devices are described separately by function into various units. Of course, the functions of the various units may be implemented in one or more software and/or hardware in the implementation of the present specification.

Those skilled in the art will appreciate that embodiments of the specification can be provided as a method, system, or computer program product. Thus, embodiments of the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, embodiments of the present specification can take the form of a computer program product embodied on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present specification. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include non-persistent memory, random access memory (RAM), and/or non-volatile memory in a computer readable medium, such as read only memory (ROM) or flash memory. Memory is an example of a computer readable medium.

Computer readable media includes both permanent and non-persistent, removable and non-removable media. Information storage can be implemented by any method or technology. The information can be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory. (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disk read only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, Magnetic tape cartridges, magnetic tape storage or other magnetic storage devices or any other non-transportable media can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary storage of computer readable media, such as modulated data signals and carrier waves.

It is also to be understood that the terms "comprises" or "comprising" or "comprising" or any other variations are intended to encompass a non-exclusive inclusion, such that a process, method, article, Other elements not explicitly listed, or elements that are inherent to such a process, method, commodity, or equipment. An element defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device including the element.

Those skilled in the art will appreciate that embodiments of the present description can be provided as a method, system, or computer program product. Accordingly, the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment in combination of software and hardware. Moreover, the description may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

This description can be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types. The present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including storage devices.

The various embodiments in the specification are described in a progressive manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

The above description is only for the embodiments of the present specification, and is not intended to limit the application. Various changes and modifications can be made to the present application by those skilled in the art. Any modifications, equivalents, improvements, etc. made within the spirit and scope of the present application are intended to be included within the scope of the appended claims.

Claims

A scanning image recognition method includes:

Obtain the current scan code image frame;

Using the graphics processor GPU, performing binarization processing on the current scan code image frame to obtain a binarization result;

The binarization result is provided to the central processing unit CPU to identify the binarization result by the CPU to obtain a scan code image recognition result.
The method of claim 1, wherein the GPU is used to perform binarization processing on the current scan code image frame, specifically:

The first thread uses a GPU to perform binarization processing on the current scan code image frame;

The identifying, by the CPU, the binarization result includes:

The second thread uses the CPU to identify the binarization result, wherein the first thread and the second thread are different threads.
The method of claim 1, after the providing the result of the binarization to the CPU, the method further comprises:

The next scan code image frame is binarized by the GPU, wherein the process of identifying the binarization result by the CPU and the process of scanning the code image frame by the GPU binarization process can be performed in parallel.
The method of claim 1, wherein the providing the binarization result to the CPU comprises:

The binarization result is saved in the result pool, and a corresponding notification is sent so that the CPU obtains the binarization result in the result pool.
The method of claim 4, if the CPU fails to identify the binarization result, the method further includes:

Using the CPU, the next binarization result is obtained from the result pool for identification.
The method of any one of claims 1 to 5, the scan code comprising scanning a two-dimensional code.
A scan code image recognition device includes:

Obtaining a module to obtain a current scan image frame;

a binarization module, using a graphics processor GPU, performing binarization processing on the current scan code image frame to obtain a binarization result;

The identification module provides the binarization result to the central processing unit CPU to identify the binarization result by the CPU to obtain a scan image recognition result.
The apparatus of claim 7, wherein the binarization module performs a binarization process on the current scan code image frame by using a GPU, and specifically includes:

The binarization module performs binarization processing on the current scan code image frame by using a GPU through a first thread;

The identification module uses the CPU to identify the binarization result, and specifically includes:

The identification module identifies the binarization result by using the CPU by using a second thread, where the first thread and the second thread are different threads.
The apparatus of claim 7, after the identification module provides the binarization result to the CPU, the binarization module uses the GPU to perform binarization processing on the next scan image frame, wherein The process of identifying the binarization result by the CPU and the process of scanning the coded image frame by the GPU binarization process can be performed in parallel.
The apparatus of claim 7, wherein the identification module provides the binarization result to the CPU, specifically:

The identification module saves the binarization result in the result pool, and sends a corresponding notification for the CPU to obtain the binarization result in the result pool.
The apparatus according to claim 10, wherein if the CPU fails to identify the binarization result, the identification module uses the CPU to acquire a next binarization result from the result pool for identification.
The apparatus of any one of claims 7 to 11, the scan code comprising scanning a two-dimensional code.
A scan code image recognition device, comprising:

At least one processor; and,

a memory communicatively coupled to the at least one processor; wherein

The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

Obtain the current scan code image frame;

Using the graphics processor GPU, performing binarization processing on the current scan code image frame to obtain a binarization result;

The binarization result is provided to the central processing unit CPU to identify the binarization result by the CPU to obtain a scan code image recognition result.