CN116343212B - Customs seal image information acquisition system and method - Google Patents

Customs seal image information acquisition system and method Download PDF

Info

Publication number
CN116343212B
CN116343212B CN202310558261.4A CN202310558261A CN116343212B CN 116343212 B CN116343212 B CN 116343212B CN 202310558261 A CN202310558261 A CN 202310558261A CN 116343212 B CN116343212 B CN 116343212B
Authority
CN
China
Prior art keywords
seal
image
identification
text
package
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310558261.4A
Other languages
Chinese (zh)
Other versions
CN116343212A (en
Inventor
刘德强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Weihai Kaisi Information Technology Co ltd
Original Assignee
Weihai Kaisi Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Weihai Kaisi Information Technology Co ltd filed Critical Weihai Kaisi Information Technology Co ltd
Priority to CN202310558261.4A priority Critical patent/CN116343212B/en
Publication of CN116343212A publication Critical patent/CN116343212A/en
Application granted granted Critical
Publication of CN116343212B publication Critical patent/CN116343212B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/1444—Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
    • G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
    • G06K7/10009—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation sensing by radiation using wavelengths larger than 0.1 mm, e.g. radio-waves or microwaves
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/148—Segmentation of character regions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/18—Extraction of features or characteristics of the image
    • G06V30/1801—Detecting partial patterns, e.g. edges or contours, or configurations, e.g. loops, corners, strokes or intersections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
    • G06V30/19173—Classification techniques
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Toxicology (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Databases & Information Systems (AREA)
  • Electromagnetism (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a customs seal image information acquisition system and a method, wherein the method comprises the following steps: acquiring a package image containing a current package acquired by acquisition equipment; utilizing the trained first artificial neural network model to confirm entity seal types and seal area ranges in the package images; dividing an identification image from the package image according to the identification area range; confirming a sealed text region range in the sealed image by using the trained second artificial neural network model; dividing a seal text image from the seal image according to the seal text area range; and identifying the content of the seal text in the seal text image. The method of the invention has less limitation on the shooting distance and the shooting position of the package, only the package image comprises the seal area of the customs seal, and the efficiency of the customs seal identification work is improved.

Description

Customs seal image information acquisition system and method
Technical Field
The invention relates to the technical field of image processing, in particular to a customs seal image information acquisition system and method.
Background
The customs seal refers to a special mark used by customs to implement sealing and controlling measures with mandatory and constraint force in the entrance and exit inspection and quarantine work, and aims to ensure that goods are safer and more orderly. With the deep learning of image algorithms in recent years becoming deeper in the artificial intelligence security market, the pain points which are caused by slow artificial physical seal examination speed and high working strength and influence the customs clearance speed in the customs examination process are gradually solved. The method can be used for identifying the seal on the carrier or the container, can be used for accurately and orderly monitoring animal and plant epidemic diseases by combining with the related epidemic detection technology, and can easily realize standardization of the supervision process, namely real-time, data statistics analysis automation and the like.
The customs seal types used in the current customs quarantine include lead seal, seal, novel electronic seal and the like, but the current customs seal identification technology is difficult to be compatible with various customs seal types. In addition, when image data acquisition is performed, a fixed area needs to be constructed on a view finding screen of the intelligent terminal, text recognition is performed on seal numbers of lead seals in the fixed area, and shooting distance and position are limited more, so that customs seal recognition efficiency is low.
The information disclosed in this background section is only for enhancement of understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person of ordinary skill in the art.
Disclosure of Invention
The invention aims to provide a customs seal image information acquisition system and a customs seal image information acquisition method, which can improve the efficiency of customs seal identification work.
In order to achieve the above object, in a first aspect, the present invention provides a customs seal image information acquisition method, including:
s1, acquiring a package image containing a current package acquired by acquisition equipment;
s2, confirming entity seal types and seal area ranges in the package images by using the trained first artificial neural network model;
s3, dividing an identification image from the package image according to the identification area range;
s4, confirming the range of the seal text area in the seal image by using the trained second artificial neural network model;
s5, dividing the seal text image from the seal image according to the seal text area range;
s6, identifying the content of the seal text in the seal text image.
The steps S1 to S5 are a default order, but any order of the steps S1 to S5 may be exchanged according to actual situations.
In the prior art, when acquiring package image data in customs seal identification, a fixed area needs to be constructed on a view finding screen of an intelligent terminal, text identification is carried out on seal numbers of lead seals in the area, and shooting distance and shooting position are limited more. However, the distance and the position of shooting are limited less in the step S1 of the invention, and the package image only needs to comprise a seal area of customs seal, the seal area can occupy a smaller area in the package image, and the seal area can also exist at any position of the package image.
It can be appreciated that the invention discloses a customs seal image information acquisition system and a method, wherein the method firstly confirms the seal area range and seal category in the package image through a first artificial neural network model, and reduces the identification area of seal content; and then confirming the range of the area of the seal text in the seal image through a second artificial neural network model, and secondarily shrinking the identification area of the seal content. Therefore, the method has less limitation on the shooting distance and the shooting position of the package, only the seal area of the customs seal is included in the package image, and the efficiency of the customs seal identification work is improved.
In an optional embodiment of the present invention, the determining, by using the trained first artificial neural network model, the entity identification category and the identification area range in the package image includes: extracting image features of the package image through convolution operations at all levels to obtain a package image feature map; generating each target identification region on the parcel image feature map by utilizing a candidate region generation network (RPN) technology; confirming classification results of the contents in the target identification areas; and confirming the target identification area of the entity identification category as the identification area range according to the classification result, wherein the classification result of the entity identification category comprises lead sealing, seal identification and seal identification.
In the embodiment of the invention, after a package image is obtained, the image characteristics of the package image can be extracted by utilizing a convolutional neural network CNN technology to obtain a Feature Map of the package image, and at least one target recognition area is generated on the Feature Map of the package image by utilizing a candidate area generation network (Region Proposal Network, RPN) technology, wherein each target recognition area comprises a single image element.
In the embodiment of the present invention, the confirming the classification result of the content in each target identification area includes:
calculating the classification prediction probability of the content in the target recognition area by using the first probability function; the first probability function is shown as follows:
;
wherein ,is the classification variable of the i-th object recognition area,/->Is a characteristic diagram->Identifying an area for the object->Is the operation sign of each successive multiplication, +.>Is a collection of target recognition regions;
using equationsAnd obtaining the classification result of the content in the target identification area.
In an optional embodiment of the present invention, the identifying the range of the sealed text region in the sealed image using the trained second artificial neural network model includes: extracting image features of the seal image through convolution operations of all levels to obtain a seal image feature map; convolving and extracting secondary features again in the seal image feature map by utilizing a sliding window; and inputting the secondary characteristics into a two-way long-short-term memory network model, and finally obtaining the center coordinates and/or the width-height sizes of the sealed text areas through classification or regression.
Long-term memory network (LSTM) papers were first published in 1997. Due to the unique design structure, LSTM is suitable for processing and predicting very long-spaced and delayed important events in a time series. LSTM generally performs better than time-recursive neural networks and Hidden Markov Models (HMMs), such as used in discontinuous segment continuous handwriting recognition. In 2009, an artificial neural network model constructed with LSTM won ICDAR handwriting recognition of the champion of the race. LSTM is also commonly used for autonomous speech recognition, with a recording of 17.7% error rate achieved in 2013 using the timt natural speech database. As a nonlinear model, LSTM can be used as a complex nonlinear unit to construct larger deep neural networks.
In an optional embodiment of the present invention, the identifying the content of the seal text in the seal text image includes: numbering each seal text region according to the central coordinates of each seal text region; confirming the target number of the target seal text area according to the entity seal category; and identifying the content of the seal text in the seal text image corresponding to the target number.
It can be understood that the customs seal formats of different categories are different, so that in order to accelerate the recognition speed of the key information, seal text contents, such as numbers, sending units, sending dates and the like, corresponding to the key target seal text regions in the customs seal can be directly recognized.
In an alternative embodiment of the present invention, after step S2, the method further comprises: under the condition that the first artificial neural network model does not recognize entity identification, confirming that the current package carries electronic identification; and after confirming that the current package carries the electronic identification, starting a radio frequency receiver to receive and acquire the identification content of the electronic identification.
According to the customs seal image information acquisition method disclosed by the invention, under the condition that the first artificial neural network model does not recognize the entity seal, the current package is confirmed to carry the electronic seal, and a radio frequency receiver is started to read related unsealing, sealing, quarantining and transportation information stored in the radio frequency receiver by using a radio frequency identification (Radio Frequency Identification, RFID) technology, so that multi-sensor fusion is realized, and more types of seals are covered for information acquisition.
In a second aspect, the invention discloses a customs seal image information acquisition system, which is characterized by comprising: a camera, a radio frequency receiver and an information analysis device electrically connected to the camera, the radio frequency receiver, respectively, the information analysis device being adapted to perform the unit of any of the methods of the first aspect.
In an alternative embodiment of the present invention, the present invention discloses a customs seal image information acquisition system, including: the device comprises a camera, a radio frequency receiver and an information analysis device, wherein the information analysis device is electrically connected with the camera and the radio frequency receiver respectively;
the information analysis apparatus comprises a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory being interconnected, wherein the memory is adapted to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform a method according to any of the first aspects.
In a third aspect, the present invention discloses a computer readable storage medium storing a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of any of the first aspects.
Compared with the prior art, the invention discloses a customs seal image information acquisition system and a customs seal image information acquisition method, wherein the method firstly shoots a package through a camera, the selection mode of the distance and the position of the package shooting is more flexible, only a seal area of the customs seal is needed to be included in the package image, and then the range of the seal area in the package image is confirmed through a first artificial neural network model, so that the identification area of seal content is reduced; and then confirming the range of the area of the seal text in the seal image through a second artificial neural network model, and secondarily shrinking the identification area of the seal content. Therefore, the method improves the efficiency of customs seal identification work.
In addition, the customs seal formats of different categories are different, so that in order to accelerate the recognition speed of key information, seal text contents, such as numbers, sending units, sending dates and the like, corresponding to key target seal text areas in the customs seal can be directly recognized.
According to the customs seal image information acquisition method disclosed by the invention, under the condition that the first artificial neural network model does not recognize the entity seal, the current package is confirmed to carry the electronic seal, and a radio frequency receiver is started to read related unsealing, sealing, quarantining and transportation information stored in the radio frequency receiver by using a radio frequency identification (Radio Frequency Identification, RFID) technology, so that multi-sensor fusion is realized, and more types of seals are covered for information acquisition.
Drawings
FIG. 1 is a schematic flow chart of a customs seal image information acquisition method provided by the invention;
FIG. 2 is a schematic view of a prior art scenario in which a parcel image is captured;
FIG. 3 is a schematic illustration of a package image in this embodiment;
FIG. 4 is a feature map of a parcel image corresponding to the parcel image shown in FIG. 3;
FIG. 5 is a schematic illustration of a segmented seal image segmented from the package image shown in FIG. 3;
FIG. 6 is a schematic diagram of an identification process of each of the areas of the seal text in the seal image shown in FIG. 5;
fig. 7 is a schematic diagram of steps of a customs seal image information acquisition method provided by the invention;
fig. 8 is a schematic structural diagram of a customs seal image information acquisition system.
Detailed Description
The following detailed description of embodiments of the invention is, therefore, to be taken in conjunction with the accompanying drawings, and it is to be understood that the scope of the invention is not limited to the specific embodiments.
Throughout the specification and claims, unless explicitly stated otherwise, the term "comprise" or variations thereof such as "comprises" or "comprising", etc. will be understood to include the stated element or component without excluding other elements or components.
The customs seal refers to a special mark used by customs to implement sealing and controlling measures with mandatory and constraint force in the entrance and exit inspection and quarantine work, and aims to ensure that goods are safer and more orderly. With the deep learning of image algorithms in recent years becoming deeper in the artificial intelligence security market, the pain points which are caused by slow artificial physical seal examination speed and high working strength and influence the customs clearance speed in the customs examination process are gradually solved. The method can be used for identifying the seal on the carrier or the container, can be used for accurately and orderly monitoring animal and plant epidemic diseases by combining with the related epidemic detection technology, and can easily realize standardization of the supervision process, namely real-time, data statistics analysis automation and the like.
Several situations require that an encapsulation can be applied: due to port condition limitation and other reasons, customs decides to transport to a designated place for inspection and quarantine; the imported goods are subjected to external packing inspection and quarantine at the port, and are required to be transported to a designated place for production, processing and storage, inspected and quarantined by arrival customs and supervised; returning and destroying prohibited inbound objects according to the regulations of the inbound inspection and quarantine laws; the inspection and quarantine is unqualified, and the treatment such as returning, destroying, pest removing and the like is carried out; the inspection and quarantine is qualified, so that adulteration is avoided, and batch confusion is avoided; vehicles such as ships, airplanes, vehicles and the like which are detected to enter through inspection and quarantine and containers are provided with self-service articles which can prohibit entering or are controlled to be used in China, or infectious disease media (mice and insects) and dangerous diseases and insect pests are detected on the vehicles and need to be controlled in a sealing way to prevent diffusion; for food and production and operation places which have caused food poisoning accidents or have evidence that the food poisoning accidents possibly occur, further port health supervision and investigation treatment are required to be carried out; performing airtight fumigation and pest removal treatment; a carrier, container, loading container, package, etc. loaded with the transit inspection quarantine; sample of receipt transaction and sample of import claim to be signed; foreign trade contractual agreements or government agreements dictate that an identification be applied; and other inspection and quarantine needs sealing.
The customs seal types used in the current customs quarantine include lead seal, seal, novel electronic seal and the like, but the current customs seal identification technology is difficult to be compatible with various customs seal types. In addition, when image data acquisition is performed, a fixed area needs to be constructed on a view finding screen of the intelligent terminal, text recognition is performed on seal numbers of lead seals in the fixed area, and shooting distance and position are limited more, so that customs seal recognition efficiency is low.
In a first aspect, as shown in fig. 1 and 7, the present invention provides a customs seal image information acquisition method, which includes:
s1, acquiring a package image containing a current package acquired by acquisition equipment.
In the prior art, when acquiring package image data in customs seal identification, a fixed area needs to be constructed on a view finding screen of an intelligent terminal, and when character identification is performed on a seal number of a lead seal in the fixed area, the shooting distance and the shooting position are more limited, for example, the position and the angle of a camera need to be adjusted to be opposite to the seal until the front face of the seal completely falls into the fixed area in the view finding screen, as shown in fig. 2. However, in the invention, the step S1 shoots the package image through the camera, and the package image only needs to comprise the seal area of customs seal, the seal area can occupy a smaller area in the package image, and the seal area can also exist at any position of the package image, so that the shooting distance and limitation are less, and the selection mode of the shooting distance and position of the package is more flexible. As shown in fig. 3, which is a schematic diagram of a package image in the present solution, it can be seen that the package image is located at a corner of the package in the picture, and the package image captured in the present solution does not need to be captured by aiming at the package image, so that the identification area can be identified when the package image is captured without aiming at the package image.
S2, confirming entity seal types and seal area ranges in the package images by using the trained first artificial neural network model.
According to the invention, the locating speed is high, the range of the seal area and the seal type in the package image are confirmed by using the trained first artificial neural network model, so that small targets in the image, namely the range of the seal area in the package image, can be rapidly and accurately identified, and the instantaneity of the system is effectively improved.
S3, dividing the seal image from the package image according to the seal area range.
As shown in fig. 5, fig. 5 is a schematic view of a segmented seal image segmented from the package image shown in fig. 3. Fig. 5 shows seal, which includes information such as "owner or agent", "name", "package type", "number/weight", "code-sprayed mark", "transportation means", "container number", "seal type", "seal number", "seal location", "seal cause", etc.
And S4, confirming the range of the seal text area in the seal image by using the trained second artificial neural network model.
S5, dividing the seal text image from the seal image according to the seal text area range.
S6, identifying the content of the seal text in the seal text image.
It can be appreciated that the invention discloses a customs seal image information acquisition system and a method, wherein the method firstly confirms the seal area range and seal category in a package image through a first artificial neural network model, and reduces the identification area of seal content; and then confirming the range of the area of the seal text in the seal image through the second artificial neural network model, and secondarily shrinking the identification area of the seal content. Therefore, the method has less limitation on the shooting distance and the shooting position of the package, only the seal area of the customs seal is included in the package image, and the efficiency of the customs seal identification work is improved.
In an embodiment of the present invention, the determining, by using the trained first artificial neural network model, the entity identification category and the identification area range in the package image includes:
s21, extracting image features of the package image through convolution operation of each level, and obtaining a package image feature map.
S22, generating each target identification area on the parcel image feature map by utilizing a candidate area generation network (RPN) technology, wherein each target identification area contains a single image element.
In the embodiment of the invention, after a package image is obtained, the image characteristics of the package image can be extracted by utilizing a convolutional neural network CNN technology to obtain a Feature Map of the package image, and then at least one target identification area is generated on the Feature Map of the package image by utilizing a candidate area generation network (Region Proposal Network, RPN) technology. Each target recognition area contains a single image element, namely an object to be recognized, as shown in fig. 4, and the image comprises two target recognition areas, wherein one target recognition area contains a seal, and the other target recognition area contains a package whole.
S23, confirming classification results of the contents in each target identification area.
S24, confirming the target identification area of which the classification result belongs to the entity identification category as an identification area range, wherein the classification result of which the entity identification category comprises lead sealing, seal identification and seal identification.
In the embodiment of the present invention, the confirming the classification result of the content in each target identification area includes:
s231, calculating the classification prediction probability of the content in the target recognition area by using the first probability function; the first probability function is shown as follows:
;
wherein ,is the classification variable of the i-th object recognition area,/->Is a characteristic diagram->In order to identify the region for the object,is the operation sign of each successive multiplication, +.>Is a collection of target recognition regions;
s232 utilizes the equationAnd obtaining the classification result of the content in the target identification area.
Wherein, step 231 includes:
s2311, extracting nodes and edge points representing visual features of an object to be identified in a target identification area;
s2312, processing the node and the edge point by using a preset processing method, wherein the preset processing method comprises the following steps: respectively calculating the node input data and the edge point input data of the gate control recursion unit recursion neural network GRU to respectively obtain a node GRU result and an edge GRU result; pooling the node GRU result and the edge GRU result respectively to obtain a node pooling result and an edge pooling result respectively;
s2313, respectively taking the node pooling result and the edge pooling result as edge point input data and node input data of the gate control recursion unit recursion neural network GRU of the next preset processing method, and continuing to process by the preset processing method until the classification prediction probability of the content in the target identification area is obtained.
In step S2312, the node and the edge point are respectively used as node input data and edge point input data of the gate-control recurrent unit recurrent neural network GRU to calculate, so as to obtain a node GRU result and an edge GRU result, which include:
calculating the classification prediction probability of the content in the target recognition area corresponding to the node input data and the edge point input data by using a second probability function; the second probability function is as follows:
；
for the probability function name of each variable x, n is the number of target recognition areas, ++>Is->Classification variable of the individual target recognition areas, +.>For node->Current hidden state, meta-data>Is a visual feature about node i;
and taking the classification prediction probability of the content in the target identification area corresponding to the edge point input data as an edge GRU result.
In an embodiment of the present invention, the determining, by using the trained second artificial neural network model, the range of the sealed text region in the sealed image includes:
s41, extracting image features of the seal image through convolution operations of all levels to obtain a seal image feature map.
S42, the sliding window is utilized to convolve and extract secondary features in the seal image feature map again.
S43, predicting a plurality of targets (anchors) by using the secondary characteristics.
S44, inputting the secondary characteristics into a two-way long-short-term memory network model, and finally obtaining the center coordinates and/or the width and height dimensions of each seal text region through classification or regression.
Long-term memory network (LSTM) papers were first published in 1997. Due to the unique design structure, LSTM is suitable for processing and predicting very long-spaced and delayed important events in a time series. LSTM generally performs better than time-recursive neural networks and Hidden Markov Models (HMMs), such as used in discontinuous segment continuous handwriting recognition. In 2009, an artificial neural network model constructed with LSTM won ICDAR handwriting recognition of the champion of the race. LSTM is also commonly used for autonomous speech recognition, with a recording of 17.7% error rate achieved in 2013 using the timt natural speech database. As a nonlinear model, LSTM can be used as a complex nonlinear unit to construct larger deep neural networks.
When detecting that the traditional lead seal exists, the character on the traditional lead seal is further recognized, the character recognition method utilizes the OCR technology based on deep learning to recognize the seal character, and text characteristics of the seal character are different due to the fact that seals of different types exist. Compared with other general target detection and CTPN type character recognition algorithms, the YOLOv3 combined with CTPN has the advantages of high speed and high positioning accuracy, and can process various forms of text images. And training a model after marking the text data of different types collected in the earlier stage, and performing text secondary fine positioning on the positioning and dividing seal mark images by using the model to obtain a small area with a text sequence, so that the interference of other background information is effectively eliminated, and the obtained character recognition result is accurate and reliable. At this time, the information output by the image algorithm processing module is the identification position, the identification type and the identification number.
In an embodiment of the present invention, the identifying the content of the seal text in the seal text image includes:
s61, numbering each seal text area according to the central coordinates of each seal text area.
As shown in fig. 6, the respective seal text areas are numbered 1 to 18 in accordance with the center coordinates of the respective seal text areas.
S62, confirming the target number of the target seal text area according to the entity seal type.
As shown in fig. 6, according to the entity seal category, a seal text containing key information may be used as a target seal text area, for example, if a "seal number" needs to be acquired, the number 14 is used as a target number; when it is necessary to acquire the "cause of seal", the number 18 is set as the target number.
S63, identifying the content of the seal text in the seal text image corresponding to the target number.
It can be understood that the customs seal formats of different categories are different, so that in order to accelerate the recognition speed of the key information, seal text contents, such as numbers, sending units, sending dates and the like, corresponding to the key target seal text regions in the customs seal can be directly recognized.
Wherein, according to the central coordinates of each seal text area, numbering each seal text area includes:
s611, confirming row coordinates and column coordinates of each center coordinate, and calculating coordinate values corresponding to the center coordinates according to the following formula:, wherein ,/>Representing coordinate values->Representing row coordinates>Representing column coordinates;
and S612, numbering each seal text area according to the sequence of the coordinate values from small to large.
As shown in fig. 7, in the embodiment of the present invention, after step S2, the method further includes:
under the condition that the first artificial neural network model does not recognize the entity seal, confirming that the current package carries the electronic seal;
after confirming that the current package carries the electronic seal, starting the radio frequency receiver to receive and acquire the seal content of the electronic seal.
According to the customs seal image information acquisition method disclosed by the invention, under the condition that the first artificial neural network model does not recognize the entity seal, the current package is confirmed to carry the electronic seal, the radio frequency receiver is started to read the related unsealing, sealing, quarantining and transportation information stored in the radio frequency receiver by using the radio frequency identification (Radio Frequency Identification, RFID) technology, and multi-sensor fusion is realized, so that more types of seals are covered for information acquisition.
When the method is used for collecting and processing the image data, the deep learning network model is utilized to position the lead seal, so that the seal is not required to be placed in a fixed area, the shooting distance and position are limited less, and the selection mode of the shooting distance and position of the package is more flexible.
According to the method and the device, before the identification of the seal text information, the seal styles are additionally classified, the seal styles are respectively identified by characters after the seal styles are obtained, the seal styles in different forms can be processed, and the practicability is higher.
In the invention, after the identification is electronic, the RFID technology is used for reading the related unsealing, sealing, quarantining and transportation information stored in the electronic identification, so that the fusion of multiple sensors is realized, and more types of identifications are covered for information acquisition.
In consideration of the problem of large flow of animals, plants and the like to be detected, which are faced by customs every day, the method utilizes the YOLOv3 model with higher positioning speed to carry out customs seal detection, and the detection speed of the model is close to 100 times of that of FasterRcnn, so that small targets in images can be rapidly and accurately identified, and the real-time performance of a system is effectively improved.
The existing scheme directly carries out character recognition on the target obtained by general target detection, is difficult to adapt to a scene of character detection, cannot accurately position a continuous text sequence, and has the problems that the background noise introduced by the obtained character recognition result is large and the discontinuity exists. According to the invention, after the target detection is segmented, the position of the seal text is positioned by utilizing YOLOv3 combined with CTPN again, so that the secondary accurate positioning of seal characters is realized, and the character recognition result obtained later is more accurate.
In order to realize the whole paperless operation, the invention improves the real-time performance of customs quarantine, avoids the complicated process of manual recording, and obtains the information of the corresponding quarantine object and the loading container thereof by comparing the identified result with the quarantine result storage history corresponding to the number after uploading the identified result in real time. The customs can conduct classification release or corresponding treatment on quarantine objects according to real-time identification results, and quarantine efficiency is improved.
In a second aspect, the invention discloses a customs seal image information acquisition system, which comprises a camera, a radio frequency receiver and an information analysis device, wherein the information analysis device is respectively and electrically connected with the camera and the radio frequency receiver, and the information analysis device is used for executing the module of the method in any one of the first aspects. The specific implementation is similar to that described in the first aspect, and will not be repeated here.
In an alternative embodiment of the invention, the invention provides a customs seal image information acquisition system. As shown in fig. 8, the customs seal image information acquisition system includes: the device comprises a camera 810, a radio frequency receiver 820 and an information analysis device 830, wherein the information analysis device 830 is electrically connected with the camera 810 and the radio frequency receiver 820 respectively.
The information analysis device 830 includes one or more processors 831; one or more input devices 832, one or more output devices 833 and a memory 834. The processor 831, input device 832, output device 833, and memory 834 are connected by a bus 835. The memory 834 is for storing a computer program comprising program instructions, the processor 831 being for executing the program instructions stored by the memory 834. Wherein the processor 831 is configured to invoke the program instructions to perform the operations of any of the methods of the first aspect.
It should be appreciated that in embodiments of the present invention, the processor 831 may be a central processing unit (Central Processing Unit, CPU), which may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The input device 832 may include a touch pad, a fingerprint sensor (for collecting fingerprint information of a user and direction information of a fingerprint), a microphone, etc., and the output device 833 may include a display (LCD, etc.), a speaker, etc.
The memory 834 may include read-only memory and random access memory, and provides instructions and data to the processor 831. A portion of memory 834 may also include non-volatile random access memory. For example, the memory 834 may also store information of device type.
In a specific implementation, the processor 831, the input device 832, and the output device 833 described in the embodiments of the present invention may perform an implementation described in any of the methods of the first aspect, and may also perform an implementation of the terminal device described in the embodiments of the present invention, which is not described herein.
In a third aspect, the present invention provides a computer readable storage medium storing a computer program comprising program instructions which when executed by a processor implement the steps of any of the methods of the first aspect.
The computer readable storage medium may be an internal storage unit of the terminal device of any of the foregoing embodiments, for example, a hard disk or a memory of the terminal device. The computer readable storage medium may be an external storage device of the terminal device, for example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, which are provided in the terminal device. Further, the computer-readable storage medium may further include both an internal storage unit and an external storage device of the terminal device. The computer-readable storage medium is used for storing the computer program and other programs and data required by the terminal device. The above-described computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
Compared with the prior art, the invention discloses a customs seal image information acquisition system and a method, wherein the method firstly confirms the seal area range and the seal category in the package image through a first artificial neural network model, and reduces the identification area of seal content; and then confirming the range of the area of the seal text in the seal image through a second artificial neural network model, and secondarily shrinking the identification area of the seal content. Therefore, the method has less limitation on the shooting distance and the shooting position of the package, only the seal area of the customs seal is included in the package image, and the efficiency of the customs seal identification work is improved.
In addition, the customs seal formats of different categories are different, so that in order to accelerate the recognition speed of key information, seal text contents, such as numbers, sending units, sending dates and the like, corresponding to key target seal text areas in the customs seal can be directly recognized.
According to the customs seal image information acquisition method disclosed by the invention, under the condition that the first artificial neural network model does not recognize the entity seal, the current package is confirmed to carry the electronic seal, and a radio frequency receiver is started to read related unsealing, sealing, quarantining and transportation information stored in the radio frequency receiver by using a radio frequency identification (Radio Frequency Identification, RFID) technology, so that multi-sensor fusion is realized, and more types of seals are covered for information acquisition.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing descriptions of specific exemplary embodiments of the present invention are presented for purposes of illustration and description. It is not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application to thereby enable one skilled in the art to make and utilize the invention in various exemplary embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims and their equivalents.

Claims (8)

1. The customs seal image information acquisition method is characterized by comprising the following steps of:
acquiring a package image containing a current package acquired by acquisition equipment;
utilizing the trained first artificial neural network model to confirm entity seal types and seal area ranges in the package images;
dividing an identification image from the package image according to the identification area range;
confirming a sealed text region range in the sealed image by using the trained second artificial neural network model;
dividing a seal text image from the seal image according to the seal text area range; and
identifying the content of the seal text in the seal text image;
the step of confirming the entity seal type and the seal area range in the package image by using the trained first artificial neural network model comprises the following steps:
extracting image features of the package image through convolution operations at all levels to obtain a package image feature map;
generating each target identification region on the parcel image feature map by utilizing a candidate region generation network (RPN) technology;
confirming classification results of the contents in the target identification areas; and
the target identification area of the entity identification category is identified as the identification area range, and the classification result of the entity identification category comprises lead sealing, seal identification and seal identification;
the step of confirming the range of the seal text area in the seal image by using the trained second artificial neural network model comprises the following steps:
extracting image features of the seal image through convolution operations of all levels to obtain a seal image feature map;
convolving and extracting secondary features again in the seal image feature map by utilizing a sliding window; and
and inputting the secondary characteristics into a two-way long-short-term memory network model, and finally obtaining the center coordinates and/or the width-height sizes of the sealed text areas through classification or regression.
2. The customs seal image information gathering method of claim 1, wherein said confirming the classification result of the content in each of the target identification areas comprises:
calculating the classification prediction probability of the content in the target recognition area by using a first probability function; the first probability function is represented by the following formula:
;
wherein ,is the classification variable of the i-th said object recognition area,/or->For the characteristic map, ++>For the target recognition area,/>Is the operation sign of each successive multiplication, +.>Is a set of the target recognition regions;
using equationsAnd obtaining the classification result of the content in the target identification area.
3. The customs seal image information gathering method of claim 1, wherein the identifying seal text content in the seal text image comprises:
numbering each seal text region according to the central coordinates of each seal text region;
confirming the target number of the target seal text area according to the entity seal category; and
and identifying the content of the seal text in the seal text image corresponding to the target number.
4. A customs seal image information collecting method according to claim 3, wherein said numbering each of said seal text areas according to the center coordinates of each of said seal text areas comprises:
confirming row coordinates and column coordinates of the center coordinates, and calculating coordinate values corresponding to the center coordinates according to the following formula:, wherein ,/>Representing coordinate values->Representing row coordinates>Representing column coordinates;
and numbering each seal text area according to the sequence of the coordinate values from small to large.
5. The customs seal image information gathering method of claim 1, wherein after confirming the entity seal category and seal area range in the package image using the trained first artificial neural network model, the method further comprises:
under the condition that the first artificial neural network model does not recognize entity identification, confirming that the current package carries electronic identification;
and after confirming that the current package carries the electronic identification, starting a radio frequency receiver to receive and acquire the identification content of the electronic identification.
6. A customs seal image information acquisition system, comprising: a camera, a radio frequency receiver and an information analysis device electrically connected to the camera, the radio frequency receiver, respectively, the information analysis device being adapted to perform the method of any one of claims 1 to 5.
7. The customs clearance image information acquisition system of claim 6, wherein the information analysis apparatus comprises a processor, an input device, an output device, and a memory, the processor, the input device, the output device, and the memory being interconnected, wherein the memory is configured to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the method of any of claims 1-5.
8. A computer-readable storage medium comprising,
the computer storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of any of claims 1 to 5.
CN202310558261.4A 2023-05-18 2023-05-18 Customs seal image information acquisition system and method Active CN116343212B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310558261.4A CN116343212B (en) 2023-05-18 2023-05-18 Customs seal image information acquisition system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310558261.4A CN116343212B (en) 2023-05-18 2023-05-18 Customs seal image information acquisition system and method

Publications (2)

Publication Number Publication Date
CN116343212A CN116343212A (en) 2023-06-27
CN116343212B true CN116343212B (en) 2023-08-08

Family

ID=86893250

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310558261.4A Active CN116343212B (en) 2023-05-18 2023-05-18 Customs seal image information acquisition system and method

Country Status (1)

Country Link
CN (1) CN116343212B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1066587A1 (en) * 1998-03-24 2001-01-10 Metrologic Instruments, Inc. Automated system and method for identifying and measuring packages transported through a laser scanning tunnel
CN101131727A (en) * 2006-08-23 2008-02-27 华为技术有限公司 Image collecting method and its application
CN112307919A (en) * 2020-10-22 2021-02-02 福州大学 Improved YOLOv 3-based digital information area identification method in document image
CN115375736A (en) * 2022-10-25 2022-11-22 威海市博华医疗设备有限公司 Image-based pedestrian trajectory tracking method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1066587A1 (en) * 1998-03-24 2001-01-10 Metrologic Instruments, Inc. Automated system and method for identifying and measuring packages transported through a laser scanning tunnel
CN101131727A (en) * 2006-08-23 2008-02-27 华为技术有限公司 Image collecting method and its application
CN112307919A (en) * 2020-10-22 2021-02-02 福州大学 Improved YOLOv 3-based digital information area identification method in document image
CN115375736A (en) * 2022-10-25 2022-11-22 威海市博华医疗设备有限公司 Image-based pedestrian trajectory tracking method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李健 ; 邢立学 ; 赵洪德 ; .浅析人工智能在海关监管领域的应用.科技经济导刊.2020,(第28期),全文. *

Also Published As

Publication number Publication date
CN116343212A (en) 2023-06-27

Similar Documents

Publication Publication Date Title
Ammour et al. Deep learning approach for car detection in UAV imagery
CN110378297B (en) Remote sensing image target detection method and device based on deep learning and storage medium
CN111507327B (en) Target detection method and device
CN111325769B (en) Target object detection method and device
US20190138849A1 (en) Rotation variant object detection in deep learning
CN112132042A (en) SAR image target detection method based on anti-domain adaptation
US12047715B2 (en) Systems and methods for automatic recognition of vehicle information
US20140270366A1 (en) Dimension-Wise Spatial Layout Importance Selection: An Alternative Way to Handle Object Deformation
US20200342247A1 (en) Systems and methods for automatic recognition of vehicle information
CN113160276A (en) Target tracking method, target tracking device and computer readable storage medium
CN111832515A (en) Dense pedestrian detection method, medium, terminal and device
CN114897655A (en) Vision-based epidemic prevention control method and device, storage medium and electronic equipment
CN111444816A (en) Multi-scale dense pedestrian detection method based on fast RCNN
CN111553184A (en) Small target detection method and device based on electronic purse net and electronic equipment
CN108509826A (en) A kind of roads recognition method and its system of remote sensing image
CN116343212B (en) Customs seal image information acquisition system and method
CN112241736A (en) Text detection method and device
CN112150467A (en) Method, system and device for determining quantity of goods
CN104679011B (en) Images match air navigation aid based on Stable Branch characteristic point
CN112418118A (en) Method and device for detecting pedestrian intrusion under unsupervised bridge
CN116823884A (en) Multi-target tracking method, system, computer equipment and storage medium
CN116612272A (en) Intelligent digital detection system for image processing and detection method thereof
KR102416714B1 (en) System and method for city-scale tree mapping using 3-channel images and multiple deep learning
CN112308061B (en) License plate character recognition method and device
CN116434256A (en) Drug image identification method, electronic device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant