WO2020062804A1 - 自然场景中行驶证拍照影像的识别方法、装置及电子设备 - Google Patents

自然场景中行驶证拍照影像的识别方法、装置及电子设备 Download PDF

Info

Publication number
WO2020062804A1
WO2020062804A1 PCT/CN2019/079036 CN2019079036W WO2020062804A1 WO 2020062804 A1 WO2020062804 A1 WO 2020062804A1 CN 2019079036 W CN2019079036 W CN 2019079036W WO 2020062804 A1 WO2020062804 A1 WO 2020062804A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
text
feature
natural scene
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/079036
Other languages
English (en)
French (fr)
Inventor
田野
刘鹏
王健宗
肖京
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Publication of WO2020062804A1 publication Critical patent/WO2020062804A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/148—Segmentation of character regions
    • G06V30/153—Segmentation of character regions using recognition of characters or words
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G06V10/243—Aligning, centring, orientation detection or correction of the image by compensating for image skew or non-uniform image deformations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the present application relates to the field of image processing technology, and in particular, to a method, a device, an electronic device, and a non-volatile readable storage medium for identifying a driving license photograph image in a natural scene.
  • the acquired document image data is obtained based on a fixed shooting angle, a fixed lighting condition, and a fixed background. It has strict constraints on the acquired document image data.
  • the captured credential image data has different image quality, different shooting angles, different lighting conditions, and different backgrounds, making traditional document recognition systems unable to capture credential images captured in natural application scenarios.
  • Image data is used for identification.
  • the identification of ID photographs in natural scenes has a relatively wide range of applications.
  • the photographed image of the vehicle driving license needs to be identified, that is, from the ID image. Extract the text information of the document.
  • the inventors realized that a method for identifying driving license photograph images in natural scenes should be proposed to solve the problem that it is difficult to identify the driving license photograph images in natural scenes in the prior art.
  • this application provides a method, device, electronic device, and non-volatile readable storage medium for identifying driving license photograph images in natural scenes. Realize the flexible recognition of the text in the driving license photo images in different natural scenes.
  • a method for identifying a driving license photograph image in a natural scene includes:
  • the image content of the natural scene image includes a photographed image of the driving license
  • Feature maps obtained by performing feature extraction on the document image through a fixed-width frame traversal to obtain candidate frames, and stitching the target frames selected by the candidate frames according to the text context features to obtain the text area in the document image;
  • Character recognition is performed on the feature sequence of the text area, and a recognition result is output.
  • a device for recognizing a driving license photograph image in a natural scene includes:
  • An image acquisition module configured to obtain a natural scene image, and the image content of the natural scene image includes a photographed image of the driving license;
  • a document extraction module configured to detect a document image from the natural scene image by performing multi-scale feature extraction and target detection on the natural scene image
  • a text extraction module is configured to perform feature extraction on the document image through a fixed-width frame traversal to obtain a feature map to obtain a candidate frame, and stitch the target frame selected by the candidate frame according to the text context feature to obtain the document image.
  • a text recognition module is configured to perform character recognition on a feature sequence of the text area and output a recognition result.
  • An electronic device includes:
  • a memory where computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the method described above is implemented.
  • a non-volatile readable storage medium having computer-readable instructions stored thereon, the computer-readable instructions implementing a method as described above when executed by a processor.
  • the multi-scale feature extraction and target detection are performed on the natural scene image, and the credential image is detected from the natural scene image.
  • the feature image is extracted from the document image to obtain a feature map.
  • the feature frame is traversed through the fixed width frame to obtain candidate frames.
  • the target frames are filtered based on the text context features to obtain the text area in the document image. Character recognition is performed on the feature sequence of the text area, and the recognition result is output.
  • Fig. 1 is a block diagram of a device according to an exemplary embodiment
  • Fig. 2 is a flow chart showing a method for identifying a driving license photograph image in a natural scene according to an exemplary embodiment
  • Fig. 3 is a flow chart showing details of step 330 according to the corresponding embodiment of Fig. 2;
  • Fig. 4 is a document image before correction according to an exemplary embodiment
  • FIG. 5 is a corrected image of a certificate according to the embodiment shown in FIG. 4;
  • Fig. 6 is a pre-correction and post-correction document image according to an exemplary embodiment
  • Fig. 7 is a flow chart showing details of step 350 according to the corresponding embodiment of Fig. 2;
  • Fig. 8 is a flow chart showing details of step 370 according to the corresponding embodiment of Fig. 7;
  • Fig. 9 is a flow chart showing details of step 370 according to the corresponding embodiment of Fig. 8;
  • Fig. 10 is a block diagram of a device for recognizing a driving license photograph image in a natural scene according to an exemplary embodiment.
  • the manual approval method is basically replaced by a computer automatic approval method because it is too labor-intensive, but the existing computer automatic approval technology is too dependent on the image quality, shooting angle, lighting conditions, and background of the document image.
  • the image quality, shooting angle, lighting conditions, and background of the document image are flexible. It is difficult to apply the existing technology to the recognition of driving license photographs in the natural scene.
  • the method and device provided in this application can solve this problem and realize accurate recognition of the driving license photographed image in the natural scene.
  • an implementation environment of the present application includes a terminal device and a server.
  • the terminal device may be a portable mobile device, such as a smart phone, a tablet computer, or a non-portable device, such as a desktop computer.
  • the terminal device transmits to the server a natural scene image that needs to be identified and approved by the server, and the image content of the natural scene image includes a photographed image of the driving license.
  • a server is a device that provides computing services. It can receive the natural scene images transmitted to it by the terminal device, recognize the natural scene images, and generate recognition results. The approval result generated according to the recognition result may be fed back to the terminal device or other devices by the server, or may be stored in the server.
  • the terminal device uploads the natural scene image obtained by the user through the camera to the server, and the server recognizes the natural scene image, generates a recognition result, generates an approval result according to the recognition result, and then sends the approval result Transfer to the terminal device.
  • the implementation environment of the present application is a smart device.
  • Computer-readable instructions are stored on the smart device, and when the computer-readable instructions are executed by the smart device, the recognition of the natural scene image can be realized.
  • Fig. 1 is a block diagram of a device according to an exemplary embodiment.
  • the device may be the above-mentioned terminal device or the above-mentioned smart device.
  • the device 200 may include one or more of the following components: a processing component 202, a memory 204, a power component 206, a multimedia component 208, an audio component 210, a sensor component 214, and a communication component 216.
  • the processing component 202 generally controls overall operations of the device 200, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.
  • the processing component 202 may include one or more processors 218 to execute instructions to complete all or part of the steps of the method described below.
  • the processing component 202 may include one or more modules to facilitate the interaction between the processing component 202 and other components.
  • the processing component 202 may include a multimedia module to facilitate the interaction between the multimedia component 208 and the processing component 202.
  • the memory 204 is configured to store various types of data to support operation at the device 200. Examples of such data include instructions for any application program or method operating on the device 200.
  • the memory 204 may be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (Electrically Erasable Programmable Read-Only Memory (referred to as EEPROM), Erasable Programmable Read Only Memory (referred to as EPROM), Programmable Read Only Memory (Programmable Red-Only Memory (referred to as PROM), read-only memory (referred to as PROM) Read-Only Memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
  • the memory 204 also stores one or more modules, which are configured to be executed by the one or more processors 218 to complete all or part of the steps in the method shown below.
  • the power component 206 provides power to various components of the device 200.
  • the power component 206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device 200.
  • the multimedia component 208 includes a screen that provides an output interface between the device 200 and a user.
  • the screen may include a liquid crystal display (Liquid Crystal Display, LCD for short) and a touch panel. If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user.
  • the touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or slide action, but also detect duration and pressure related to the touch or slide operation.
  • the screen may also include an Organic Light Emitting Display (OLED).
  • OLED Organic Light Emitting Display
  • the audio component 210 is configured to output and / or input audio signals.
  • the audio component 210 includes a microphone (Microphone, MIC for short).
  • the microphone is configured to receive an external audio signal.
  • the received audio signal may be further stored in the memory 204 or transmitted via the communication component 216.
  • the audio component 210 further includes a speaker for outputting audio signals.
  • the sensor component 214 includes one or more sensors for providing status assessment of various aspects of the device 200.
  • the sensor component 214 can detect the open / closed state of the device 200, the relative positioning of the components, and the sensor component 214 can also detect a change in the position of the device 200 or a component of the device 200 and a change in the temperature of the device 200.
  • the sensor component 214 may further include a magnetic sensor, a pressure sensor, or a temperature sensor.
  • the communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices.
  • the device 200 can access a wireless network based on a communication standard, such as WiFi (Wireless-Fidelity, wireless fidelity).
  • the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel.
  • the communication component 216 further includes a Near Field Communication (NFC) module to facilitate short-range communication.
  • NFC Near Field Communication
  • the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra wideband (UWB) technology, Bluetooth technology, and other technologies. .
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra wideband
  • Bluetooth and other technologies.
  • the apparatus 200 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital signal processors, digital signal processing equipment, programmable logic devices, field programmable gate arrays, A controller, microcontroller, microprocessor, or other electronic component is implemented to perform the methods described below.
  • ASICs Application Specific Integrated Circuits
  • digital signal processors digital signal processing equipment
  • programmable logic devices programmable logic devices
  • field programmable gate arrays A controller, microcontroller, microprocessor, or other electronic component is implemented to perform the methods described below.
  • Fig. 2 is a flow chart showing a method for identifying a driving license photograph image in a natural scene according to an exemplary embodiment. This method can be executed by the aforementioned server or by the aforementioned smart device. As shown in FIG. 2, the method includes the following steps.
  • step 310 a natural scene image is obtained, and the image content of the natural scene image includes a photographed image of the driving license.
  • the driving permit consists of a card holder, a home page and an auxiliary page.
  • the front text of the driving license homepage is black and generally contains the following text: license plate number, vehicle type, nature of use, owner, address, brand model, engine number, vehicle identification code, registration date, and date of certification.
  • the color of the front page of the supplementary page is black, and generally contains the following text: number plate number, approved load number, file number, total mass, approved load mass, outline size, and quasi-traction total mass.
  • the natural scene image is an image obtained by shooting a natural scene through a camera device (such as a mobile phone equipped with a camera), and may be a single image or a frame image in a video.
  • the natural scene includes a driving license, and accordingly, the image content of the natural scene image includes a photographed image of the driving license.
  • a device for example, the aforementioned server / smart device that performs the method for identifying a driving license photograph image in a natural scene obtains the natural scene image through a camera installed on the device, or obtains the natural scene image from another device.
  • Step 330 Detect a credential image from the natural scene image by performing multi-scale feature extraction and target detection on the natural scene image.
  • Feature extraction at different scales is performed on natural scene images to obtain several feature maps at different scales.
  • Large targets are detected by small scale feature maps, and small targets are detected by large scale feature maps.
  • a plurality of first preselection boxes with different aspect ratios are generated at different positions on the feature map.
  • a target with a large aspect ratio is detected through a first pre-selection box with a large aspect ratio, and a target with a small aspect ratio is detected through a first pre-selection box with a small aspect ratio.
  • the first preselection box is classified according to the characteristics in the first preselection box, and a classification result is obtained.
  • the classification result includes the confidence degree that the first preselection box is a credential, and also includes the confidence degree that the first preselection box is another different object, which will be
  • the first pre-selected frame whose credibility of the document is higher than the confidence threshold is used as the first candidate frame.
  • the first target frame is filtered from the first candidate frame by calculating the degree of overlap between the first candidate frames, and the identity of the credential image in the natural scene image is determined by the first target frame. position.
  • step 330 may be implemented based on an SSD (Single Shot MultiBox Detector) algorithm.
  • SSD algorithm is a target detection algorithm based on deep learning, which can detect targets from natural scene images.
  • the SSD algorithm is based on the VGG-16 network structure and replaces the fully connected layer with several different feature layers.
  • the target frame is selected by the NMS (Non Maximum Suppression) algorithm.
  • a feature map obtained by performing feature extraction on the credential image is obtained by traversing the frame with a fixed width to obtain a candidate frame, and the target frame is filtered based on the text context feature of the candidate frame to obtain the text area in the credential image.
  • Feature images are extracted from the document image to obtain corresponding feature maps.
  • feature detection is performed with a number of fixed-width pre-selection boxes, and the candidate box corresponding to the features corresponding to the pre-selection box is selected. Preselection box with text.
  • the feature sequence of the candidate frame is spliced into rows to obtain a feature sequence.
  • the text in the document image is a sequence of character lines. According to the feature sequence, the context features of the character sequence contained in the candidate frame can be obtained.
  • the target frame is filtered from the candidate frames, and the target frame is spliced to obtain the text area in the document image.
  • step 350 may be implemented based on a CTPN (Connectionist Text Proposal Network) algorithm.
  • CTPN Connectionist Text Proposal Network
  • the CTPN algorithm performs text detection based on the characteristics of peer text context connections.
  • the CTPN algorithm uses VGG to extract image features, LSTM to extract text context features, and performs regression and classification through fully connected layers.
  • Step 370 Perform character recognition on the feature sequence of the text area, and output the recognition result.
  • the feature region is extracted from the text area, and the extracted features are stitched in rows to obtain the feature sequence. Characters are identified based on feature sequences, and characters are translated into text based on a character dictionary.
  • step 370 may be implemented by a text recognition algorithm.
  • the architecture of the text recognition algorithm is CNN (Convolutional Neural Network, Convolutional Neural Network) combined with LSTM (Long Short Term Memory, Long Short-Term Memory Network) combined with CTC (Connectionist Temporal Classifier).
  • Feature maps are extracted from CNN convolutional layers to obtain feature maps, and feature sequences are detected by bidirectional LSTM combined with CTC, and the detected characters are converted into text by a translation layer.
  • Fig. 3 is a flow chart showing details of step 330 according to the corresponding embodiment of Fig. 2. As shown in FIG. 3, the above step 330 includes:
  • Step 331 Perform feature extraction at different scales on the natural scene image to obtain several feature maps at different scales.
  • the features of the natural scene image are extracted through a convolution layer, and the feature map contains the features extracted above.
  • Features are extracted from images of natural scenes at different scales to obtain feature maps at different scales.
  • Feature maps at different scales have different scales of features.
  • Step 333 Generate a plurality of first preselection boxes with different aspect ratios for the feature map, classify the first preselection box according to the characteristics contained in the first preselection box, and filter out the first preselection box from the first preselection box. A candidate box.
  • Prediction is performed at different positions of the feature map through several first preselection boxes with different aspect ratios, and classification results are obtained according to the characteristics corresponding to the first preselection box.
  • the classification results include the confidence of the credentials contained in the first preselection box.
  • the first pre-selected box whose confidence level of the certificate is higher than the confidence threshold (the classification result is indicated as the certificate) is used as the first candidate box.
  • Step 335 Calculate the degree of overlap between the first candidate frames, filter the first target frame from the first candidate frame according to the classification result, and determine the credential image through the first target frame.
  • the classification result indicates the confidence of the preselected box / candidate box as the credential image.
  • the obtained candidate boxes are sorted according to the confidence degree, the candidate box with the highest confidence is determined, and the overlap of other candidate boxes with the candidate box with the highest confidence If the overlap is greater than the threshold, the candidate frames corresponding to the other candidate frames are filtered out.
  • the first candidate frame is screened by the NMS algorithm, and the first target frame is screened.
  • the method further includes: when the connection between the center of the document and the center of the seal in the document image has a direction deviation from the reference line, the document image is rotated in a direction that reduces the direction deviation, and correction is performed.
  • Document image pose when the connection between the center of the document and the center of the seal in the document image has a direction deviation from the reference line, the document image is rotated in a direction that reduces the direction deviation, and correction is performed. Document image pose.
  • the Huff transform is used to perform straight-line detection on the document image to obtain the edge of the document in the document image.
  • the document edge has a deviation from the edge of the document image
  • the document image is rotated in a direction that reduces the deviation of the direction.
  • Fig. 4 and Fig. 5 are images before rotation and after rotation, respectively, according to an exemplary embodiment.
  • the orientation of the document image becomes + 90 ° or + 180 ° or + 270 ° or 0 °.
  • the connection between the center of the document and the center of the seal has a direction deviation of + 90 ° or + 180 ° or + 270 ° from the reference line.
  • the document image is rotated in a direction that reduces the direction deviation.
  • the orientation deviation is adjusted to 0 °, and the posture of the document image is corrected.
  • Fig. 7 is a flow chart showing details of step 350 according to the corresponding embodiment of Fig. 2. As shown in FIG. 7, the above step 350 includes:
  • Step 351 Perform feature extraction on the document image to obtain a feature map of the document image.
  • a feature map is detected through a frame with a fixed width, and several candidate frames are obtained by filtering according to the features corresponding to the frames, and feature sequences corresponding to the features obtained from the candidate frames are stitched in rows.
  • feature detection is performed using a number of pre-selected boxes of fixed width and different length, and the features contained in the pre-selected box are detected.
  • the pre-selected box with text features is used as a candidate box.
  • the text in the credential image is a sequence of characters, and the character sequence is a number of characters.
  • the characters can be numeric characters, alphabetic characters, and Chinese characters.
  • the context feature of the candidate box refers to the feature that exists for several candidate boxes with characters in the same row based on the arrangement of the characters. By performing feature extraction on the feature sequence, the context features of the candidate frame are obtained.
  • Step 355 After obtaining the context features of the candidate frame according to the feature sequence, the classification result of the candidate frame is obtained through the fully connected layer. According to the context characteristics of the candidate frame, the classification result of the candidate frame is calculated through the fully connected layer. The classification result includes the confidence of the text contained in the candidate frame, and the confidence of the different objects contained in the candidate frame.
  • Step 357 The target frame is filtered from the candidate frames according to the classification result, and the target frame is spliced to obtain a text area.
  • Candidate boxes with confidence higher than the confidence threshold are used as target boxes.
  • the text content of the text area can be known, for example, the text content can be Chinese characters / numbers.
  • the text area is passed through different texts.
  • the recognition algorithm performs recognition to achieve more accurate recognition.
  • Different text recognition algorithms mean that when training the text recognition algorithm so that it can recognize text, several algorithms are trained through different samples respectively. For example, an algorithm capable of identifying a corresponding text area is trained by using a text area whose text content is everyone as a sample, and an algorithm capable of identifying a corresponding text area is trained by using a text area whose text content is a vehicle identification number as a sample.
  • the samples are scaled to enhance the data, that is, for text recognition algorithms with fewer samples, the data is enhanced before training, and for text recognition algorithms with sufficient samples, data is not enhanced.
  • Fig. 8 is a flow chart showing details of step 370 according to the corresponding embodiment of Fig. 7. As shown in FIG. 8, the above step 370 includes:
  • step 371 the block region is extracted from the text region, and the feature sequence extracted by the line stitching is obtained.
  • Segmented feature extraction refers to the process of extracting features from a text area by dividing the text area into several pieces along the direction of the character arrangement and extracting the features in pieces. Divided feature extraction is adapted to the arrangement direction of characters in a text area. Compared with the general method of feature extraction, it is more helpful to identify characters.
  • Step 373 Perform character recognition according to the feature sequence.
  • Step 375 Translate the recognized characters into text according to the character dictionary.
  • step 370 In order to make the recognition result output in step 370 more accurate, compared to directly outputting the text obtained by translating characters at the translation layer as the recognition result, in an exemplary embodiment, for some texts, such as a brand model, the text obtained by translating the translation layer After comparing with the standard text in the text library, the matched standard text is output as the recognition result.
  • Fig. 9 is a flowchart showing details of step 370 according to the corresponding embodiment of Fig. 8. As shown in FIG. 9, the above step 370 further includes:
  • Step 377 Obtain the similarity between the text and the standard text in the text library.
  • the text library is preset and contains several standard texts.
  • a brand model text library is set in advance, and several brand models (such as Tesla Model X) are input into the text library as standard text.
  • Step 379 Output the standard text that matches the text as the recognition result according to the similarity.
  • Sort the calculated similarity determine the standard text with the highest similarity in the text library, and output the standard text as the recognition result.
  • the text translated from the translation layer is Teslo Model X.
  • Teslo Model X the standard text Tesla Model X with the highest similarity to Teslo Model X is obtained from the text library, and Tesla Model X is output as the recognition result.
  • Fig. 10 is a block diagram of a device for recognizing a driving license photograph image in a natural scene according to an exemplary embodiment. As shown in FIG. 10, the device includes:
  • the image acquisition module 410 is configured to obtain a natural scene image, and the image content of the natural scene image includes a photographed image of the driving license.
  • the credential extraction module 430 is configured to detect a credential image from the natural scene image by performing multi-scale feature extraction and target detection on the natural scene image.
  • a text extraction module 450 is configured to perform feature extraction on the document image through a fixed-width frame traversal to obtain a feature map to obtain candidate frames, and stitch the target frames selected by the candidate frames according to text context features to obtain the document image. Text area in.
  • the text recognition module 470 is configured to perform character recognition on a feature sequence of the text area, and output a recognition result.
  • the credential extraction module includes:
  • a feature map generating unit configured to perform feature extraction at different scales on the natural scene image to obtain several feature maps at different scales
  • a preselection box screening unit is configured to generate a plurality of first preselection boxes of different aspect ratios for the feature map, classify the first preselection box according to the characteristics contained in the first preselection box, and select from the preselection box Screening out a number of first candidate frames whose classification results indicate the credentials;
  • a candidate frame screening unit is configured to calculate a degree of overlap between the first candidate frames, filter a first target frame from the first candidate frame according to a classification result, and determine a credential image through the first target frame.
  • the credential extraction module further includes:
  • An attitude correction unit configured to correct the document image by rotating the document image in a direction that reduces the orientation deviation when the line between the document center and the seal center in the document image has a deviation from a reference line Gesture.
  • the text extraction module includes:
  • a feature extraction unit configured to perform feature extraction on the document image to obtain a feature map of the document image
  • a first screening unit configured to detect the feature map through a frame of a fixed width, to obtain a plurality of candidate frames according to the features corresponding to the frames, and to obtain a feature sequence according to the features corresponding to the candidate frames obtained by line stitching;
  • a classification unit configured to obtain a classification result of the candidate frame through a fully connected layer after obtaining a context feature of the candidate frame according to the feature sequence
  • a second filtering unit is configured to filter a target frame from the candidate frames according to the classification result, and stitch the target frames to obtain a text area.
  • the text recognition module includes:
  • a sequence generating unit configured to extract the cut feature from the text area, and obtain the feature sequence by stitching the extracted features according to lines;
  • a character recognition unit configured to perform character recognition according to the feature sequence
  • a translation unit configured to translate the recognized characters into text according to a character dictionary.
  • the text recognition module further includes:
  • a comparison unit configured to obtain a similarity between the text and a standard text in a text library
  • An output unit is configured to output a standard text matching the text as the recognition result according to the similarity.
  • the present application also provides an electronic device, which includes:
  • a memory where computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the method described above is implemented.
  • the present application also provides a non-volatile readable storage medium on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the method described above is implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

一种自然场景中行驶证拍照影像的识别方法，所述方法包括：获得自然场景图像，所述自然场景图像的图像内容包含行驶证的拍照影像（310）；通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像（330）；通过固定宽度的框遍历对所述证件图像进行特征提取所得特征图，得到候选框，拼接所述候选框中根据文本上下文特征筛选出的目标框，得到所述证件图像中的文本区域（350）；对所述文本区域的特征序列进行字符识别，输出识别结果（370）。上述方法能够从自然场景图像中检测出证件图像，准确识别出证件图像中的文本，灵活适应不同的自然场景，解决了现有技术难以识别自然场景中行驶证拍照影像的问题。

Description

自然场景中行驶证拍照影像的识别方法、装置及电子设备
本申请要求2018年9月25日递交、发明名称为“自然场景中行驶证拍照影像的识别方法、装置及电子设备”的中国专利申请CN201811112723.5的优先权，在此通过引用将其全部内容合并于此。
技术领域
本申请涉及图像处理技术领域，特别涉及一种自然场景中行驶证拍照影像的识别方法、装置及电子设备、非易失性可读存储介质。
背景技术
传统证件识别系统多基于固定的应用场景，其所获取的证件图像数据是基于固定的拍摄角度、固定的光照条件、固定的背景获取到的，对所获取的证件图像数据具有严格的约束条件。
在自然的应用场景下，拍摄获取到的证件图像数据具有不同的图像质量、不同的拍摄角度、不同的光照条件、不同的背景，使得传统证件识别系统无法对自然的应用场景下拍摄到的证件图像数据进行证件识别。
当前，自然场景中证件拍照影像的识别又有比较广的应用范围，例如在部分应用场景中，对车主及其车辆进行身份认证时，需对车辆行驶证的拍照影像进行识别，即从证件图像中提取证件的文本信息。
发明人意识到，应提出一种自然场景中行驶证拍照影像的识别方法，解决现有技术难以识别自然场景中行驶证拍照影像的问题。
申请内容
为了解决相关技术中难以识别自然场景中行驶证拍照影像的技术问题，本申请提供了一种自然场景中行驶证拍照影像的识别方法、装置及电子设备、非易失性可读存储介质，以实现灵活识别不同自然场景中行驶证拍照影像中的文本。
一种自然场景中行驶证拍照影像的识别方法，所述方法包括：
获得自然场景图像，所述自然场景图像的图像内容包含行驶证的拍照影像；
通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像；
通过固定宽度的框遍历对所述证件图像进行特征提取所得特征图，得到候选框，拼接所述候选框中根据文本上下文特征筛选出的目标框，得到所述 证件图像中的文本区域；
对所述文本区域的特征序列进行字符识别，输出识别结果。
一种自然场景中行驶证拍照影像的识别装置，所述装置包括：
图像获取模块，用于获得自然场景图像，所述自然场景图像的图像内容包含行驶证的拍照影像；
证件提取模块，用于通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像；
文本提取模块，用于通过固定宽度的框遍历对所述证件图像进行特征提取所得特征图，得到候选框，拼接所述候选框中根据文本上下文特征筛选出的目标框，得到所述证件图像中的文本区域；
文本识别模块，用于对所述文本区域的特征序列进行字符识别，输出识别结果。
一种电子设备，所述电子设备包括：
处理器；以及
存储器，所述存储器上存储有计算机可读指令，所述计算机可读指令被所述处理器执行时实现如前所述的方法。
一种非易失性可读存储介质，其上存储有计算机可读指令，所述计算机可读指令被处理器执行时实现如前所述的方法。
本申请的实施例提供的技术方案可以包括以下有益效果：
获得包含行驶证拍照影像的自然场景图像后，对自然场景图像进行多尺度的特征提取及目标检测，从自然场景图像中检测出证件图像。之后，对证件图像进行特征提取得到特征图，通过固定宽度的框遍历该特征图得到候选框，拼接候选框中根据文本上下文特征筛选出的目标框，得到证件图像中的文本区域。对文本区域的特征序列进行字符识别，输出识别结果。上述方法能够从自然场景图像中检测出证件图像，准确识别出证件图像中的文本，灵活适应不同的自然场景，解决了现有技术难以识别自然场景中行驶证拍照影像的问题。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性的，并不能限制本申请。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并于说明书一起用于解释本申请的原理。
图1是根据一示例性实施例示出的一种装置的框图；
图2是根据一示例性实施例示出的一种自然场景中行驶证拍照影像的识别方法的流程图；
图3是根据图2对应实施例示出的步骤330的细节的流程图；
图4是根据一示例性实施例示出的矫正前的证件图像；
图5是根据图4对应实施例示出的矫正后的证件图像；
图6是根据一示例性实施例示出的矫正前及矫正后证件图像；
图7是根据图2对应实施例示出的步骤350的细节的流程图；
图8是根据图7对应实施例示出的步骤370的细节的流程图；
图9是根据图8对应实施例示出的步骤370的细节的流程图；
图10是根据一示例性实施例示出的一种自然场景中行驶证拍照影像的识别装置的框图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所描述的、本申请的一些方面相一致的装置和方法的例子。
机动车作为现如今的主流代步工具之一，因其本身具有的风险，促进了车险行业的发展，在用户投保流程中，行驶证核准环节需验证用户的行驶证，为降低时间和经济成本，多采取由用户上传行驶证拍照影像，计算机自动/人工根据该拍照影像核准行驶证。
目前，人工核准方式因过于耗费人力，基本被计算机自动核准方式取代，但现有的计算机自动核准技术过于依赖于证件图像的图像质量、拍摄角度、光照条件、背景。
自然场景中的证件，证件图像的图像质量、拍摄角度、光照条件、背景灵活多变，现有技术难以应用于自然场景中行驶证拍照影像的识别。本申请所提供的方法及装置能够解决该问题，实现对自然场景中行驶证拍照影像的准确识别。
在一示例性实施例中，本申请的实施环境包括终端设备和服务器。
终端设备可以是便携移动设备，例如智能手机、平板电脑，也可以是非便携设备，例如台式电脑。终端设备向服务器传输需由服务器进行识别及核准的自然场景图像，该自然场景图像的图像内容包含行驶证的拍照影像。
服务器是提供计算服务的设备，可以接收终端设备向其传输的自然场景 图像，对自然场景图像进行识别，生成识别结果。根据识别结果生成的核准结果可以由服务器反馈到终端设备或其它设备，也可以存储于服务器中。
在一示例性实施例的具体实现中，终端设备将用户通过摄像头拍摄得到自然场景图像上传至服务器，服务器对该自然场景图像进行识别，生成识别结果，根据该识别结果产生核准结果，将核准结果传输至终端设备。
在另一示例性实施例中，本申请的实施环境是智能设备。智能设备上存储有计算机可读指令，该计算机可读指令被智能设备执行时，能够实现自然场景图像的识别。
图1是根据一示例性实施例示出的一种装置的框图。该装置可以是上述终端设备，也可以是上述智能设备。
如图1所示，装置200可以包括以下一个或多个组件：处理组件202，存储器204，电源组件206，多媒体组件208，音频组件210，传感器组件214以及通信组件216。
处理组件202通常控制装置200的整体操作，诸如与显示，电话呼叫，数据通信，相机操作以及记录操作相关联的操作等。处理组件202可以包括一个或多个处理器218来执行指令，以完成下述的方法的全部或部分步骤。此外，处理组件202可以包括一个或多个模块，便于处理组件202和其他组件之间的交互。例如，处理组件202可以包括多媒体模块，以方便多媒体组件208和处理组件202之间的交互。
存储器204被配置为存储各种类型的数据以支持在装置200的操作。这些数据的示例包括用于在装置200上操作的任何应用程序或方法的指令。存储器204可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(Static Random Access Memory，简称SRAM)，电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，简称EEPROM)，可擦除可编程只读存储器(Erasable Programmable Read Only Memory，简称EPROM)，可编程只读存储器(Programmable Red-Only Memory，简称PROM)，只读存储器(Read-Only Memory，简称ROM)，磁存储器，快闪存储器，磁盘或光盘。存储器204中还存储有一个或多个模块，该一个或多个模块被配置成由该一个或多个处理器218执行，以完成如下所示方法中的全部或者部分步骤。
电源组件206为装置200的各种组件提供电力。电源组件206可以包括电源管理系统，一个或多个电源，及其他与为装置200生成、管理和分配电力相关联的组件。
多媒体组件208包括在所述装置200和用户之间的提供一个输出接口的 屏幕。在一些实施例中，屏幕可以包括液晶显示器(Liquid Crystal Display，简称LCD)和触摸面板。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。屏幕还可以包括有机电致发光显示器(Organic Light Emitting Display，简称OLED)。
音频组件210被配置为输出和/或输入音频信号。例如，音频组件210包括一个麦克风(Microphone，简称MIC)，当装置200处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器204或经由通信组件216发送。在一些实施例中，音频组件210还包括一个扬声器，用于输出音频信号。
传感器组件214包括一个或多个传感器，用于为装置200提供各个方面的状态评估。例如，传感器组件214可以检测到装置200的打开/关闭状态，组件的相对定位，传感器组件214还可以检测装置200或装置200一个组件的位置改变以及装置200的温度变化。在一些实施例中，该传感器组件214还可以包括磁传感器，压力传感器或温度传感器。
通信组件216被配置为便于装置200和其他设备之间有线或无线方式的通信。装置200可以接入基于通信标准的无线网络，如WiFi(Wireless-Fidelity，无线保真)。在一个示例性实施例中，通信组件216经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件216还包括近场通信(Near Field Communication，简称NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(Radio Frequency Identification，简称RFID)技术，红外数据协会(Infrared Data Association，简称IrDA)技术，超宽带(Ultra Wideband，简称UWB)技术，蓝牙技术和其他技术来实现。
在示例性实施例中，装置200可以被一个或多个应用专用集成电路(Application Specific Integrated Circuit，简称ASIC)、数字信号处理器、数字信号处理设备、可编程逻辑器件、现场可编程门阵列、控制器、微控制器、微处理器或其他电子元件实现，用于执行下述方法。
图2是根据一示例性实施例示出的一种自然场景中行驶证拍照影像的识别方法的流程图。该方法可以由前述服务器执行，也可以由前述智能设备执行。如图2所示，该方法包括以下步骤。
步骤310，获得自然场景图像，该自然场景图像的图像内容包含行驶证 的拍照影像。
行驶证由证夹、主页、副页三部分组成。行驶证主页正面文字颜色为黑色，一般包含以下文字：号牌号码、车辆类型、使用性质、所有人、住址、品牌型号、发动机号码、车辆识别代号、注册日期、发证日期。副页正面文字颜色为黑色，一般包含以下文字：号牌号码、核定载人数、档案编号、总质量、核定载质量、外廓尺寸、准牵引总质量。
自然场景图像是通过摄像设备(例如装设有摄像头的手机)拍摄自然场景所得图像，可以是单张图像，也可以是视频中的一帧图像。该自然场景中包含有行驶证，相应的，自然场景图像的图像内容包含行驶证的拍照影像。
执行上述自然场景中行驶证拍照影像的识别方法的设备(例如前述服务器/智能设备)通过自身所装设摄像头获得上述自然场景图像，或从其他设备获得上述自然场景图像。
步骤330，通过对自然场景图像进行多尺度的特征提取及目标检测，从自然场景图像中检测出证件图像。
对自然场景图像进行不同尺度的特征提取，得到不同尺度的若干特征图，通过小尺度的特征图检测大目标，通过大尺度的特征图检测小目标。对于所得特征图，在特征图上的不同位置，分别生成若干长宽比不同的第一预选框。通过长宽比大的第一预选框检测长宽比大的目标，通过长宽比小的第一预选框检测长宽比小的目标。
根据第一预选框内的特征对第一预选框进行分类，得到分类结果，分类结果包括第一预选框为证件的置信度，也包括第一预选框为其它不同物体分别的置信度，将为证件的置信度高于置信阈值的第一预选框作为第一候选框。之后，根据所得第一候选框的分类结果，通过计算第一候选框之间的重叠度，从第一候选框中筛选出第一目标框，通过第一目标框确定自然场景图像中证件图像的位置。
在具体实现中，步骤330可以基于SSD(Single Shot MultiBox Detector)算法实现。SSD算法是基于深度学习的目标检测算法，能够从自然场景图像中检测出目标。
SSD算法基于VGG-16网络结构，以若干不同尺度的特征层取代全连接层，通过NMS(Non Maximum Suppression，非极大值抑制)算法筛选出目标框。
步骤350，通过固定宽度的框遍历对证件图像进行特征提取所得特征图，得到候选框，拼接候选框中根据文本上下文特征筛选出的目标框，得到证件图像中的文本区域。
对证件图像进行特征提取得到对应的特征图，在特征图上的不同位置都分别以若干固定宽度的预选框进行特征检测，筛选出预选框对应的特征包括文本特征的候选框，候选框是可能包含文本的预选框。
将候选框所含特征按行拼接得到特征序列，证件图像中的文本是若干行字符序列，根据特征序列能够得到候选框所含字符序列的上下文特征。
根据字符序列的上下文特征，从候选框中筛选出目标框，拼接目标框得到证件图像中的文本区域。
在具体实现中，步骤350可以基于CTPN(Connectionist Text Proposal Network)算法实现。CTPN算法根据同行文本上下文连接的特征，进行文本检测。
CTPN算法通过VGG提取图像特征，通过LSTM提取文本的上下文特征，通过全连接层进行回归及分类。
步骤370，对文本区域的特征序列进行字符识别，输出识别结果。
对文本区域进行切块特征提取，按行拼接提取的特征，得到特征序列。根据特征序列识别字符，将字符根据字符字典转译成文本。
在具体实现中，步骤370可通过文本识别算法实现。例如，文本识别算法的架构为CNN(Convolutional Neural Network，卷积神经网络)结合LSTM(Long Short Term Memory，长短期记忆网络)结合CTC(Connectionist Temporal Classifier)。通过CNN卷积层提取特征得到特征图，通过双向LSTM结合CTC检测特征序列，通过转译层将检测所得字符转换为文本。
通过上述方法，实现了对自然场景中行驶证拍照影像较为准确的识别，解决了现有技术难以识别自然场景中行驶证拍照影像的问题。
图3是根据图2对应实施例示出的步骤330的细节的流程图。如图3所示，上述步骤330包括：
步骤331，对自然场景图像进行不同尺度的特征提取，获得不同尺度的若干特征图。
通过卷积层提取自然场景图像的特征，特征图包含上述提取的特征。对不同尺度的自然场景图像提取特征，得到不同尺度的特征图，不同尺度的特征图所含特征的尺度不同。
步骤333，为特征图生成不同长宽比的若干第一预选框，根据第一预选框所含特征对第一预选框进行分类，从第一预选框中筛选出分类结果指示为证件的若干第一候选框。
在特征图的不同位置分别通过若干不同长宽比的第一预选框进行预测，根据第一预选框对应的特征得到分类结果，分类结果包括第一预选框内含证 件的置信度，将内含证件的置信度高于置信阈值(分类结果指示为证件)的第一预选框作为第一候选框。
步骤335，通过计算第一候选框之间的重叠度，按分类结果从第一候选框中筛选出第一目标框，通过第一目标框确定证件图像。
分类结果指示了预选框/候选框中内容为证件图像的置信度，按置信度对所得候选框进行排序，确定置信度最高的候选框，计算其它候选框与该置信度最高的候选框的重叠度，重叠度大于阈值则筛除上述其它候选框中对应的候选框。在具体实现中，通过NMS算法实现对第一候选框的筛选，筛选出第一目标框。
在一示例性实施例中，上述步骤330之后还包括：证件图像中证件中心与印章中心之间的连线相对于基准线存在方向偏差时，向减小该方向偏差的方向旋转证件图像，矫正证件图像的姿态。
先通过霍夫变换对证件图像进行直线检测，得到证件图像中证件的边缘，证件边缘相对于证件图像边缘存在方向偏差时，向减小该方向偏差的方向旋转证件图像。图4和图5分别为根据一示例性实施例示出的旋转前和旋转后的图像。
通过霍夫变换旋转证件图像后，证件图像的姿态变为+90°或+180°或+270°或0°方向。部分证件图像中证件中心和印章中心的连线相对于基准线存在+90°或+180°或+270°的方向偏差，如图6所示，向减小该方向偏差的方向旋转证件图像，使方向偏差为0°，矫正证件图像的姿态。
通过对证件图像姿态的矫正，实现了对不同姿态证件图像的识别。
图7是根据图2对应实施例示出的步骤350的细节的流程图。如图7所示，上述步骤350包括：
步骤351，对证件图像进行特征提取，得到证件图像的特征图。
步骤353，通过固定宽度的框检测特征图，根据所述框对应的特征筛选得到若干候选框，按行拼接所得候选框对应的特征得到特征序列。
在特征图上的不同位置，分别以若干固定宽度、不同长度的预选框进行特征检测，检测预选框所含特征，检测到文本特征时，将含文本特征的预选框作为候选框。
证件图像中的文本是若干字符序列，字符序列则是若干字符，字符可以是数字字符、字母字符、汉字字符。
部分候选框内含字符的一部分，候选框的上下文特征是指，对于同一行内含字符的若干候选框，基于字符的排列存在的特征。通过对特征序列进行特征提取，得到候选框的上下文特征。
步骤355，根据特征序列获得候选框的上下文特征后，通过全连接层获得候选框的分类结果。根据候选框的上下文特征，通过全连接层计算候选框的分类结果，分类结果包括候选框内含文本的置信度，也包括候选框内含其它不同物体分别的置信度。
步骤357，根据分类结果从候选框中筛选出目标框，拼接目标框得到文本区域。
将置信度高于置信阈值的候选框作为目标框。
文本的内容不同，文本区域所含特征是不同的。根据分类结果，可以获知文本区域的文本内容，例如，文本内容可以是汉字/数字。
在一示例性实施例中，对于文本内容差异较大的文本区域(例如，所有人一般是若干汉字的组合，车辆识别代号则是若干字母及若干数字的组合)，将文本区域通过不同的文本识别算法进行识别，以实现较为准确的识别。
不同的文本识别算法是指，在对文本识别算法进行训练使其能够识别文本时，分别通过不同的样本训练出若干算法。例如，将文本内容为所有人的文本区域作为样本训练出能够识别对应文本区域的算法，将文本内容为车辆识别代号的文本区域作为样本训练出能够识别对应文本区域的算法。
在训练时，按比例对样本进行数据增强，即，对于样本较少的文本识别算法，在训练前对样本进行数据增强，对于样本充足的文本识别算法，则不进行数据增强。
图8是根据图7对应实施例示出的步骤370的细节的流程图。如图8所示，上述步骤370包括：
步骤371，对文本区域进行切块特征提取，按行拼接提取的特征得到特征序列。
切块特征提取是指，对文本区域进行特征提取时，沿字符排列的方向将文本区域切分为若干块，分块提取特征。切块特征提取适应于文本区域内字符的排列方向，与进行特征提取的一般方式相比，更有助于识别字符。
步骤373，根据特征序列进行字符识别。
步骤375，根据字符字典将识别得到的字符转译为文本。
为了使上述步骤370输出的识别结果更准确，相比于将转译层转译字符所得文本作为识别结果直接输出，在一示例性实施例中，对于部分文本，例如品牌型号，将转译层转译所得文本与文本库中的标准文本比对后，将匹配的标准文本作为识别结果输出。
图9是根据图8对应实施例示出的步骤370的细节的流程图。如图9所示，上述步骤370还包括：
步骤377，获得文本与文本库中标准文本的相似度。
文本库是预先设置的，包含若干标准文本。例如，预先设置品牌型号文本库，将若干品牌型号(例如Tesla Model X)作为标准文本输入文本库中。
得到转译层输出的文本后，计算该文本和文本库中标准文本的余弦相似度。
步骤379，根据相似度，将与文本匹配的标准文本作为所述识别结果输出。
对计算所得相似度进行排序，确定文本库中与该文本相似度最高的标准文本，将该标准文本作为识别结果输出。
例如，转译层转译所得文本为Teslo Model X，通过上述步骤377与步骤379，从文本库中获得与Teslo Model X相似度最高的标准文本Tesla Model X，将Tesla Model X作为识别结果输出。
下述为本申请装置实施例，可以用于执行本申请上述方法实施例。对于本申请装置实施例中未披露的细节，请参照本申请方法实施例。
图10是根据一示例性实施例示出的一种自然场景中行驶证拍照影像的识别装置的框图。如图10所示，该装置包括：
图像获取模块410，用于获得自然场景图像，所述自然场景图像的图像内容包含行驶证的拍照影像。
证件提取模块430，用于通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像。
文本提取模块450，用于通过固定宽度的框遍历对所述证件图像进行特征提取所得特征图，得到候选框，拼接所述候选框中根据文本上下文特征筛选出的目标框，得到所述证件图像中的文本区域。
文本识别模块470，用于对所述文本区域的特征序列进行字符识别，输出识别结果。
在一示例性实施例中，所述证件提取模块包括：
特征图生成单元，用于对所述自然场景图像进行不同尺度的特征提取，获得不同尺度的若干特征图；
预选框筛选单元，用于为所述特征图生成不同长宽比的若干第一预选框，根据所述第一预选框所含特征对所述第一预选框进行分类，从所述预选框中筛选出分类结果指示为证件的若干第一候选框；
候选框筛选单元，用于通过计算第一候选框之间的重叠度，按分类结果从第一候选框中筛选出第一目标框，通过所述第一目标框确定证件图像。
在一示例性实施例中，所述证件提取模块还包括：
姿态矫正单元，用于所述证件图像中证件中心与印章中心之间的连线相对于基准线存在方向偏差时，向减小所述方向偏差的方向旋转所述证件图像，矫正所述证件图像的姿态。
在一示例性实施例中，所述文本提取模块包括：
特征提取单元，用于对所述证件图像进行特征提取，得到所述证件图像的特征图；
第一筛选单元，用于通过固定宽度的框检测所述特征图，根据所述框对应的特征筛选得到若干候选框，按行拼接所得候选框对应的特征得到特征序列；
分类单元，用于根据所述特征序列获得候选框的上下文特征后，通过全连接层获得所述候选框的分类结果；
第二筛选单元，用于根据所述分类结果从候选框中筛选出目标框，拼接所述目标框得到文本区域。
在一示例性实施例中，所述文本识别模块包括：
序列生成单元，用于对所述文本区域进行切块特征提取，按行拼接提取的特征得到特征序列；
字符识别单元，用于根据所述特征序列进行字符识别；
转译单元，用于根据字符字典将识别得到的字符转译为文本。
在一示例性实施例中，所述文本识别模块还包括：
比较单元，用于获得所述文本与文本库中标准文本的相似度；
输出单元，用于根据所述相似度，将与所述文本匹配的标准文本作为所述识别结果输出。
本申请还提供一种电子设备，所述电子设备包括：
处理器；以及
存储器，所述存储器上存储有计算机可读指令，所述计算机可读指令被所述处理器执行时实现如前所述的方法。
本申请还提供一种非易失性可读存储介质，其上存储有计算机可读指令，所述计算机可读指令被处理器执行时实现如前所述的方法。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围执行各种修改和改变。本申请的范围仅由所附的权利要求来限制。

Claims (20)

  1. 一种自然场景中行驶证拍照影像的识别方法，其特征在于，所述方法包括：
    获得自然场景图像，所述自然场景图像的图像内容包含行驶证的拍照影像；
    通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像；
    通过固定宽度的框遍历对所述证件图像进行特征提取所得特征图，得到候选框，拼接所述候选框中根据文本上下文特征筛选出的目标框，得到所述证件图像中的文本区域；
    对所述文本区域的特征序列进行字符识别，输出识别结果。
  2. 根据权利要求1所述的方法，其特征在于，所述通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像，包括：
    对所述自然场景图像进行不同尺度的特征提取，获得不同尺度的若干特征图；
    为所述特征图生成不同长宽比的若干第一预选框，根据所述第一预选框所含特征对所述第一预选框进行分类，从所述预选框中筛选出分类结果指示为证件的若干第一候选框；
    通过计算第一候选框之间的重叠度，按分类结果从第一候选框中筛选出第一目标框，通过所述第一目标框确定证件图像。
  3. 根据权利要求1所述的方法，其特征在于，所述通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像之后，还包括：
    所述证件图像中证件中心与印章中心之间的连线相对于基准线存在方向偏差时，向减小所述方向偏差的方向旋转所述证件图像，矫正所述证件图像的姿态。
  4. 根据权利要求1所述的方法，其特征在于，所述通过固定宽度的框遍历对所述证件图像进行特征提取所得特征图，得到候选框，拼接所述候选框中根据文本上下文特征筛选出的目标框，得到所述证件图像中的文本区域，包括：
    对所述证件图像进行特征提取，得到所述证件图像的特征图；
    通过固定宽度的框检测所述特征图，根据所述框对应的特征筛选得到若干候选框，按行拼接所得候选框对应的特征得到特征序列；
    根据所述特征序列获得候选框的上下文特征后，通过全连接层获得所述候 选框的分类结果；
    根据所述分类结果从候选框中筛选出目标框，拼接所述目标框得到文本区域。
  5. 根据权利要求4所述的方法，其特征在于，所述对所述文本区域的特征序列进行字符识别，输出识别结果，包括：
    对所述文本区域进行切块特征提取，按行拼接提取的特征得到特征序列；
    根据所述特征序列进行字符识别；
    根据字符字典将识别得到的字符转译为文本。
  6. 根据权利要求5所述的方法，其特征在于，所述对所述文本区域的特征序列进行字符识别，输出识别结果，包括：
    获得所述文本与文本库中标准文本的相似度；
    根据所述相似度，将与所述文本匹配的标准文本作为所述识别结果输出。
  7. 一种自然场景中行驶证拍照影像的识别装置，其特征在于，所述装置包括：
    图像获取模块，用于获得自然场景图像，所述自然场景图像的图像内容包含行驶证的拍照影像；
    证件提取模块，用于通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像；
    文本提取模块，用于通过固定宽度的框遍历对所述证件图像进行特征提取所得特征图，得到候选框，拼接所述候选框中根据文本上下文特征筛选出的目标框，得到所述证件图像中的文本区域；
    文本识别模块，用于对所述文本区域的特征序列进行字符识别，输出识别结果。
  8. 根据权利要求7所述的装置，其特征在于，所述证件提取模块包括：
    特征图生成单元，用于对所述自然场景图像进行不同尺度的特征提取，获得不同尺度的若干特征图；
    预选框筛选单元，用于为所述特征图生成不同长宽比的若干第一预选框，根据所述第一预选框所含特征对所述第一预选框进行分类，从所述预选框中筛选出分类结果指示为证件的若干第一候选框；
    候选框筛选单元，用于通过计算第一候选框之间的重叠度，按分类结果从第一候选框中筛选出第一目标框，通过所述第一目标框确定证件图像。
  9. 一种电子设备，其特征在于，所述电子设备包括：
    处理器；
    存储器，所述存储器上存储有计算机可读指令，所述计算机可读指令被所 述处理器执行时，实现以下步骤：
    获得自然场景图像，所述自然场景图像的图像内容包含行驶证的拍照影像；
    通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像；
    通过固定宽度的框遍历对所述证件图像进行特征提取所得特征图，得到候选框，拼接所述候选框中根据文本上下文特征筛选出的目标框，得到所述证件图像中的文本区域；
    对所述文本区域的特征序列进行字符识别，输出识别结果。
  10. 根据权利要求9所述的设备，其特征在于，所述通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像，包括：
    对所述自然场景图像进行不同尺度的特征提取，获得不同尺度的若干特征图；
    为所述特征图生成不同长宽比的若干第一预选框，根据所述第一预选框所含特征对所述第一预选框进行分类，从所述预选框中筛选出分类结果指示为证件的若干第一候选框；
    通过计算第一候选框之间的重叠度，按分类结果从第一候选框中筛选出第一目标框，通过所述第一目标框确定证件图像。
  11. 根据权利要求9所述的设备，其特征在于，所述通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像之后，还实现以下步骤：
    所述证件图像中证件中心与印章中心之间的连线相对于基准线存在方向偏差时，向减小所述方向偏差的方向旋转所述证件图像，矫正所述证件图像的姿态。
  12. 根据权利要求9所述的设备，其特征在于，所述通过固定宽度的框遍历对所述证件图像进行特征提取所得特征图，得到候选框，拼接所述候选框中根据文本上下文特征筛选出的目标框，得到所述证件图像中的文本区域，包括：
    对所述证件图像进行特征提取，得到所述证件图像的特征图；
    通过固定宽度的框检测所述特征图，根据所述框对应的特征筛选得到若干候选框，按行拼接所得候选框对应的特征得到特征序列；
    根据所述特征序列获得候选框的上下文特征后，通过全连接层获得所述候选框的分类结果；
    根据所述分类结果从候选框中筛选出目标框，拼接所述目标框得到文本区域。
  13. 根据权利要求12所述的设备，其特征在于，所述对所述文本区域的特征序列进行字符识别，输出识别结果，包括：
    对所述文本区域进行切块特征提取，按行拼接提取的特征得到特征序列；
    根据所述特征序列进行字符识别；
    根据字符字典将识别得到的字符转译为文本。
  14. 根据权利要求13所述的设备，其特征在于，所述对所述文本区域的特征序列进行字符识别，输出识别结果，包括：
    获得所述文本与文本库中标准文本的相似度；
    根据所述相似度，将与所述文本匹配的标准文本作为所述识别结果输出。
  15. 一种非易失性可读存储介质，其上存储有计算机可读指令，该计算机可读指令被处理器执行时，实现以下步骤：
    获得自然场景图像，所述自然场景图像的图像内容包含行驶证的拍照影像；
    通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像；
    通过固定宽度的框遍历对所述证件图像进行特征提取所得特征图，得到候选框，拼接所述候选框中根据文本上下文特征筛选出的目标框，得到所述证件图像中的文本区域；
    对所述文本区域的特征序列进行字符识别，输出识别结果。
  16. 根据权利要求15所述的介质，其特征在于，所述通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像，包括：
    对所述自然场景图像进行不同尺度的特征提取，获得不同尺度的若干特征图；
    为所述特征图生成不同长宽比的若干第一预选框，根据所述第一预选框所含特征对所述第一预选框进行分类，从所述预选框中筛选出分类结果指示为证件的若干第一候选框；
    通过计算第一候选框之间的重叠度，按分类结果从第一候选框中筛选出第一目标框，通过所述第一目标框确定证件图像。
  17. 根据权利要求15所述的介质，其特征在于，所述通过对所述自然场景图像进行多尺度的特征提取及目标检测，从所述自然场景图像中检测出证件图像之后，还实现以下步骤：
    所述证件图像中证件中心与印章中心之间的连线相对于基准线存在方向偏差时，向减小所述方向偏差的方向旋转所述证件图像，矫正所述证件图像的姿态。
  18. 根据权利要求15所述的介质，其特征在于，所述通过固定宽度的框遍历对所述证件图像进行特征提取所得特征图，得到候选框，拼接所述候选框中根据文本上下文特征筛选出的目标框，得到所述证件图像中的文本区域，包括：
    对所述证件图像进行特征提取，得到所述证件图像的特征图；
    通过固定宽度的框检测所述特征图，根据所述框对应的特征筛选得到若干候选框，按行拼接所得候选框对应的特征得到特征序列；
    根据所述特征序列获得候选框的上下文特征后，通过全连接层获得所述候选框的分类结果；
    根据所述分类结果从候选框中筛选出目标框，拼接所述目标框得到文本区域。
  19. 根据权利要求18所述的介质，其特征在于，所述对所述文本区域的特征序列进行字符识别，输出识别结果，包括：
    对所述文本区域进行切块特征提取，按行拼接提取的特征得到特征序列；
    根据所述特征序列进行字符识别；
    根据字符字典将识别得到的字符转译为文本。
  20. 根据权利要求19所述的介质，其特征在于，所述对所述文本区域的特征序列进行字符识别，输出识别结果，包括：
    获得所述文本与文本库中标准文本的相似度；
    根据所述相似度，将与所述文本匹配的标准文本作为所述识别结果输出。
PCT/CN2019/079036 2018-09-25 2019-03-21 自然场景中行驶证拍照影像的识别方法、装置及电子设备 Ceased WO2020062804A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811112723.5A CN109460765A (zh) 2018-09-25 2018-09-25 自然场景中行驶证拍照影像的识别方法、装置及电子设备
CN201811112723.5 2018-09-25

Publications (1)

Publication Number Publication Date
WO2020062804A1 true WO2020062804A1 (zh) 2020-04-02

Family

ID=65606881

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/079036 Ceased WO2020062804A1 (zh) 2018-09-25 2019-03-21 自然场景中行驶证拍照影像的识别方法、装置及电子设备

Country Status (2)

Country Link
CN (1) CN109460765A (zh)
WO (1) WO2020062804A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111401110A (zh) * 2019-01-03 2020-07-10 百度在线网络技术（北京）有限公司 用于提取信息的方法和装置
CN111695638A (zh) * 2020-06-16 2020-09-22 兰州理工大学 一种改进的YOLOv3候选框加权融合选取策略
CN113011497A (zh) * 2021-03-19 2021-06-22 城云科技（中国）有限公司 图像比对方法和系统
CN113705559A (zh) * 2021-08-31 2021-11-26 平安银行股份有限公司 基于人工智能的文字识别方法及装置、电子设备
CN114038004A (zh) * 2021-11-19 2022-02-11 北京恒安嘉新安全技术有限公司 一种证件信息提取方法、装置、设备及存储介质

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109460765A (zh) * 2018-09-25 2019-03-12 平安科技（深圳）有限公司 自然场景中行驶证拍照影像的识别方法、装置及电子设备
CN111832328B (zh) * 2019-04-15 2024-07-16 北京京东乾石科技有限公司 条形码检测方法、装置、电子设备及介质
CN110110722A (zh) * 2019-04-30 2019-08-09 广州华工邦元信息技术有限公司 一种基于深度学习模型识别结果的区域检测修正方法
CN110633649A (zh) * 2019-08-22 2019-12-31 广东博智林机器人有限公司 机械图的审核方法及装置
CN110598715B (zh) * 2019-09-04 2025-02-21 腾讯科技（深圳）有限公司 图像识别方法、装置、计算机设备及可读存储介质
CN110569801B (zh) * 2019-09-10 2023-06-30 四川星盾科技股份有限公司 一种行驶证关键内容识别方法
CN110837838B (zh) * 2019-11-06 2023-07-11 创新奇智(重庆)科技有限公司 基于深度学习的端到端车架号识别系统及识别方法
CN111126389B (zh) * 2019-12-20 2025-04-15 腾讯科技（深圳）有限公司 文本检测方法、装置、电子设备以及存储介质
CN114283432B (zh) * 2021-08-13 2025-06-27 腾讯科技（深圳）有限公司 一种文本块识别方法、装置及电子设备
CN113837287B (zh) * 2021-09-26 2023-08-29 平安科技（深圳）有限公司 证件异常信息识别方法、装置、设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6272245B1 (en) * 1998-01-23 2001-08-07 Seiko Epson Corporation Apparatus and method for pattern recognition
CN107679531A (zh) * 2017-06-23 2018-02-09 平安科技（深圳）有限公司 基于深度学习的车牌识别方法、装置、设备及存储介质
CN107832768A (zh) * 2017-11-23 2018-03-23 盐城线尚天使科技企业孵化器有限公司 基于深度学习的高效阅卷方法和阅卷系统
CN108446621A (zh) * 2018-03-14 2018-08-24 平安科技（深圳）有限公司 票据识别方法、服务器及计算机可读存储介质
CN109460765A (zh) * 2018-09-25 2019-03-12 平安科技（深圳）有限公司 自然场景中行驶证拍照影像的识别方法、装置及电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6272245B1 (en) * 1998-01-23 2001-08-07 Seiko Epson Corporation Apparatus and method for pattern recognition
CN107679531A (zh) * 2017-06-23 2018-02-09 平安科技（深圳）有限公司 基于深度学习的车牌识别方法、装置、设备及存储介质
CN107832768A (zh) * 2017-11-23 2018-03-23 盐城线尚天使科技企业孵化器有限公司 基于深度学习的高效阅卷方法和阅卷系统
CN108446621A (zh) * 2018-03-14 2018-08-24 平安科技（深圳）有限公司 票据识别方法、服务器及计算机可读存储介质
CN109460765A (zh) * 2018-09-25 2019-03-12 平安科技（深圳）有限公司 自然场景中行驶证拍照影像的识别方法、装置及电子设备

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111401110A (zh) * 2019-01-03 2020-07-10 百度在线网络技术（北京）有限公司 用于提取信息的方法和装置
CN111695638A (zh) * 2020-06-16 2020-09-22 兰州理工大学 一种改进的YOLOv3候选框加权融合选取策略
CN113011497A (zh) * 2021-03-19 2021-06-22 城云科技（中国）有限公司 图像比对方法和系统
CN113011497B (zh) * 2021-03-19 2023-06-20 城云科技（中国）有限公司 图像比对方法和系统
CN113705559A (zh) * 2021-08-31 2021-11-26 平安银行股份有限公司 基于人工智能的文字识别方法及装置、电子设备
CN113705559B (zh) * 2021-08-31 2024-05-10 平安银行股份有限公司 基于人工智能的文字识别方法及装置、电子设备
CN114038004A (zh) * 2021-11-19 2022-02-11 北京恒安嘉新安全技术有限公司 一种证件信息提取方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN109460765A (zh) 2019-03-12

Similar Documents

Publication Publication Date Title
WO2020062804A1 (zh) 自然场景中行驶证拍照影像的识别方法、装置及电子设备
CN106462766B (zh) 在预览模式中进行图像捕捉参数调整
CN108885699B (zh) 字符识别方法、装置、存储介质及电子设备
US9418304B2 (en) System and method for recognizing text information in object
CN111126108B (zh) 图像检测模型的训练和图像检测方法及装置
CN112200040B (zh) 遮挡图像检测方法、装置及介质
CN108959462B (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
US11087137B2 (en) Methods and systems for identification and augmentation of video content
CN105678242B (zh) 手持证件模式下的对焦方法和装置
CN106228556B (zh) 图像质量分析方法和装置
WO2017020514A1 (zh) 图片场景判定方法、装置以及服务器
CN105894042B (zh) 检测证件图像遮挡的方法和装置
CN105631406B (zh) 图像识别处理方法和装置
CN106295499B (zh) 年龄估计方法及装置
CN110569835B (zh) 一种图像识别方法、装置和电子设备
CN110869944A (zh) 使用移动设备读取测试卡
CN111523402B (zh) 一种视频处理方法、移动终端及可读存储介质
CN104077597B (zh) 图像分类方法及装置
US20120201470A1 (en) Recognition of objects
CN109697389B (zh) 身份识别方法与装置
US11232616B2 (en) Methods and systems for performing editing operations on media
CN110363111B (zh) 基于镜头失真原理的人脸活体检测方法、装置及存储介质
CN116721115A (zh) 金相组织获取方法、装置、存储介质及芯片
WO2019170038A1 (zh) 一种目标屏幕确定方法、装置及存储介质
CN113592928A (zh) 图像处理方法、图像处理装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19866747

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS (EPO FORM 1205A DATED 09.07.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19866747

Country of ref document: EP

Kind code of ref document: A1