CN111898555B - Book checking identification method, device, equipment and system based on images and texts - Google Patents
Book checking identification method, device, equipment and system based on images and texts Download PDFInfo
- Publication number
- CN111898555B CN111898555B CN202010762471.1A CN202010762471A CN111898555B CN 111898555 B CN111898555 B CN 111898555B CN 202010762471 A CN202010762471 A CN 202010762471A CN 111898555 B CN111898555 B CN 111898555B
- Authority
- CN
- China
- Prior art keywords
- book
- spine
- image
- similarity
- keyword
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Character Input (AREA)
Abstract
The invention provides a book checking identification method, device, equipment and system based on images and texts, wherein the method comprises the following steps: comprising the following steps: acquiring a bookshelf image, and cutting the bookshelf image into a plurality of spine images containing spines; identifying the spine images to obtain keywords corresponding to the spines; judging whether the similarity between any keyword corresponding to the spine and book information in a book database reaches a first similarity threshold value: if yes, finishing checking the books corresponding to the spine; if not, taking the keyword corresponding to the spine and at least one keyword corresponding to the spine around the spine as an overall keyword, and continuously judging whether the similarity between the overall keyword and book information in a book database reaches a second similarity threshold value: if yes, finishing checking books corresponding to all the spines forming the integral key words; if not, determining that the checking fails. The invention can assist the automatic book checking of the intelligent library.
Description
Technical Field
The invention belongs to the technical field of intelligent libraries, in particular to the technical field of book management, and particularly relates to a book checking identification method, device, equipment and system based on images and texts.
Background
In recent years, research and application of smart libraries have attracted the eyes of many scholars and companies, and currently, the research and application of smart libraries is still a difficult task. The automatic identification based on the back of the library book is the leading edge core research and development content of intelligent inventory. The visual analysis can acquire accurate book spine text information in real time, and has great prospect for improving book checking efficiency and reducing cost by combining a high-efficiency text matching method.
The existing library book identification technology mainly depends on radio frequency identification technology (RFID) except manual work, and the method has higher accuracy, but has the problems of high cost, privacy disclosure, information conflict, incapability of acquiring book ordering and the like. The application of the technology in the field of efficient library book intelligent inventory is greatly limited. In the current computer application algorithms, there are many published text detection and recognition algorithms, but they are rarely used in library book inventory systems. Text recognition technology based on deep learning has been developed in recent years, efficiency and accuracy are gradually improved, and a large amount of open source software and open APIs are available. Considering that if the popularity of intelligent libraries in the future is increased, it is apparent that intelligent detection of books using visual analysis, deep learning and text intelligent matching techniques is advantageous over existing radio frequency identification techniques.
With the development of text recognition technology based on image processing and the popularization of intelligent devices, the application field of visual analysis is becoming more and more widespread. Books, as the most common and widespread knowledge carrier in daily life, most of the information is usually stored in text form. Meanwhile, books contain a lot of potential visual information and are yet to be discovered and applied. The recognition of the text on the back of a book stored on a bookshelf is one of them. The efficiency of the book checking system is one of the daily problems of the library, the accuracy rate of book checking through the RFID robot is not high, and the actual sequence of drawing book arrangement cannot be obtained effectively.
Disclosure of Invention
In view of the above-mentioned drawbacks of the prior art, the present invention aims to provide a method, a device and a system for identifying book checking based on images and texts, which are used for solving the problems of low book checking efficiency and low accuracy in the prior art.
To achieve the above and other related objects, an embodiment of the present invention provides a book checking identification method based on images and text, including: acquiring a bookshelf image, and cutting the bookshelf image into a plurality of spine images containing spines; identifying the spine images to obtain keywords corresponding to the spines; judging whether the similarity between any keyword corresponding to the spine and book information in a book database reaches a first similarity threshold value: if yes, finishing checking the books corresponding to the spine; if not, taking the keyword corresponding to the spine and at least one keyword corresponding to the spine around the spine as an overall keyword, and continuously judging whether the similarity between the overall keyword and book information in a book database reaches a second similarity threshold value: if yes, finishing checking books corresponding to all the spines forming the integral key words; if not, determining that the checking fails.
In one embodiment of the present invention, an implementation of cutting the bookshelf image into a plurality of spine images including a spine comprises: detecting the edge of the bookshelf image by using an edge detection algorithm; and identifying straight lines in the bookshelf image, and cutting the bookshelf image into a plurality of spine images containing the spine according to the identified straight lines.
In an embodiment of the present invention, the book checking identification method based on images and texts further includes: and carrying out direction correction and/or definition correction on the spine image.
In an embodiment of the present invention, the spine image is identified by using an OCR text model, and a keyword corresponding to each spine is obtained.
The embodiment of the invention also provides a book checking and identifying device based on the images and the texts, which comprises the following steps: the acquisition processing module is used for acquiring a bookshelf image and cutting the bookshelf image into a plurality of spine images containing spines; the identification module is used for identifying the spine images and obtaining keywords corresponding to the spines; the judging and matching module is used for judging whether the similarity between any keyword corresponding to the spine and book information in the book database reaches a first similarity threshold value or not: if yes, finishing checking the books corresponding to the spine; if not, taking the keyword corresponding to the spine and at least one keyword corresponding to the spine around the spine as an overall keyword, and continuously judging whether the similarity between the overall keyword and book information in a book database reaches a second similarity threshold value: if yes, finishing checking books corresponding to all the spines forming the integral key words; if not, determining that the checking fails.
In an embodiment of the present invention, the acquisition processing module includes: an image acquisition unit for acquiring a bookshelf image from an image pickup apparatus; an edge detection unit for detecting an edge of the bookshelf image using an edge detection algorithm; and the cutting unit is used for identifying straight lines in the bookshelf image and cutting the bookshelf image into a plurality of spine images containing the spine according to the identified straight lines.
In an embodiment of the invention, the acquisition processing module further includes: and the correction unit is used for carrying out direction correction and/or definition correction on the spine image.
In an embodiment of the present invention, the recognition module recognizes the spine image by using an OCR text model, and obtains keywords corresponding to each spine.
The embodiment of the invention also provides electronic equipment, which comprises a processor and a memory, wherein the memory stores program instructions; the processor runs program instructions to realize the book checking identification method based on the images and the texts.
The embodiment of the invention also provides a book checking and identifying system, which comprises: the image shooting device is specially arranged on a mobile vehicle and is used for shooting bookshelf images; an electronic device as described above connected to the image pickup device.
As described above, the book checking identification method, device, equipment and system based on images and texts has the following beneficial effects:
according to the invention, the library information is obtained through the visual analysis of the camera and the text matching, the basis and the reference are provided for the automatic book checking of the subsequent library, the automatic book checking of the intelligent library can be assisted, and compared with the traditional schemes such as the RFID technology, the automatic book checking method has the advantages of stronger adaptability, wider application range, lower cost, no dependence on a large amount of manpower for manual checking and good universality.
Drawings
Fig. 1 is a schematic flow chart of the book checking identification method based on images and texts.
FIG. 2 is a flow chart of an implementation of cutting a bookshelf image into a plurality of spine images including a spine in the image and text based book inventory recognition method of the present invention.
Fig. 3 is a schematic diagram showing the overall implementation process of the book checking identification method based on images and texts.
Fig. 4 is a schematic diagram of the schematic structure of the book checking recognition device based on images and texts of the present invention.
Fig. 5 is a schematic diagram of the principle and structure of the acquisition processing module in the book checking and identifying device based on images and texts.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Fig. 7 is a schematic diagram of the principle structure of the book checking and identifying system according to an embodiment of the present application.
Description of element reference numerals
1. Book checking and identifying system
10. Electronic equipment
1101. Processor and method for controlling the same
1102. Memory device
20. Image pickup apparatus
100. Book checking and identifying device based on images and texts
110. Acquisition processing module
111. Image acquisition unit
112. Edge detection unit
113. Cutting unit
114. Correction unit
120. Identification module
130. Judging and matching module
S100-S800 steps
S121 to S122 steps
Detailed Description
Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.
The embodiment aims to provide a book checking identification method, device, equipment and system based on images and texts, which are independent of book types, high in reliability and low in cost, and are used for solving the problems of low book checking efficiency and low accuracy in the prior art.
The book checking and identifying method is a method combining visual identification and text matching, and solves the problems that a library book identifying system based on a radio frequency identification technology is high in cost, privacy is revealed, information is conflicted, and book ordering cannot be obtained.
The principles and embodiments of the image and text-based book inventory identification method, device, equipment and system of the present embodiment will be described in detail below, so that those skilled in the art can understand the image and text-based book inventory identification method, device, equipment and system of the present invention without creative labor.
Example 1
As shown in fig. 1, this embodiment provides a book checking identification method based on images and texts, including:
step S100: acquiring a bookshelf image, and cutting the bookshelf image into a plurality of spine images containing spines;
step S200: identifying the spine images to obtain keywords corresponding to the spines;
step S300: judging whether the similarity between any keyword corresponding to the spine and book information in a book database reaches a first similarity threshold value: if not, the step S500 is continuously executed, and if yes, the step S400 is continuously executed: finishing checking of books corresponding to the spine;
step S500: taking the keywords corresponding to the spine and at least one keyword corresponding to the spine around the spine as an integral keyword;
step S600: continuously judging whether the similarity between the whole keyword and book information in a book database reaches a second similarity threshold value: if yes, go on to step S700: finishing checking books corresponding to all the spines forming the integral key words; if not, go on to step S800: and determining that the checking fails.
The following describes in detail steps S100 to S800 of the book checking identification method based on images and texts in the present embodiment.
Step S100: and acquiring a bookshelf image, and cutting the bookshelf image into a plurality of spine images containing the spine.
Specifically, in this embodiment, as shown in fig. 2, one implementation manner of cutting the bookshelf image into a plurality of spine images including a spine includes:
step S110: detecting the edge of the bookshelf image by using an edge detection algorithm;
step S120: and identifying straight lines in the bookshelf image, and cutting the bookshelf image into a plurality of spine images containing the spine according to the identified straight lines.
The edge detection can be performed by adopting an edge detection algorithm such as a canny algorithm, a Sobel algorithm, a Laplacian algorithm and the like. Edge detection of images is well known to those skilled in the art and will not be described in detail herein.
In this embodiment, the line recognition algorithm for recognizing the line in the bookshelf image may adopt an LSD line detection method or a hough transform method.
In this embodiment, the bookshelf image in the camera is collected in real time, for example, the image is subjected to convolution preprocessing by adopting an optimized convolution operator, and points with gradient amplitude larger than a threshold value are marked as edges, so that edges of the bookshelf image are detected; next, a straight line in the edge of the bookshelf image is recognized using hough transform, and the bookshelf image is cut according to the straight line, thereby cutting the bookshelf image into a plurality of spine images containing the spine.
In this embodiment, the book checking identification method based on the image and the text further includes: and carrying out direction correction and/or definition correction on the spine image.
For example, the spine image is corrected by using functions such as affine functions, so that the text display on the spine image is clear in front, and the corrected spine image is conveniently subjected to text recognition.
Step S200: and identifying the spine images to obtain keywords corresponding to the spines.
In this embodiment, the spine image is identified by using an OCR text model, and keywords corresponding to the spines are obtained. Character recognition by OCR character models is well known to those skilled in the art and will not be described in detail herein.
Step S300: judging whether the similarity between any keyword corresponding to the spine and book information in a book database reaches a first similarity threshold value: if not, the step S500 is continuously executed, and if yes, the step S400 is continuously executed: and finishing checking of books corresponding to the spine.
Step S500: and taking the keyword corresponding to the spine and at least one keyword corresponding to the spine around the spine as an integral keyword.
Step S600: continuously judging whether the similarity between the whole keyword and book information in a book database reaches a second similarity threshold value: if yes, go on to step S700: finishing checking books corresponding to all the spines forming the integral key words; if not, go on to step S800: and determining that the checking fails.
Wherein the keywords and the overall keywords are book information. In the embodiment, text matching is carried out on book information identified by a single book and book information in a book database, and if the similarity between the keyword information identified by the book and the text of a certain book in the book database reaches a set threshold, the matching is considered to be successful; and if not, putting the keywords extracted from books arranged around the books into a book database for integral matching, judging the text similarity again, if the text similarity reaches a threshold value, successfully matching, and outputting the obtained result to a subsequent system after the matching is successful, thereby obtaining book information, and checking books.
As shown in fig. 3, the specific implementation procedure of the book checking identification method based on images and texts in this embodiment is as follows:
firstly, a camera shoots and acquires a bookshelf image, the bookshelf image shot by the camera is collected, then the edge of a book is detected in the bookshelf image, the bookshelf image is cut into a plurality of spine images, and then characters on the spine images are corrected to be clear at the front; processing and identifying OCR (optical character recognition) containing a spine image, extracting keywords, performing text matching on the identified book information and book information in a book database, and if the similarity between the book and the text of a certain book in the database reaches a set threshold, judging that the matching is successful; otherwise, the invention puts the keywords extracted from books arranged around the books into a database for integral matching, and judges the text similarity again, if the text similarity reaches a threshold value, the matching is successful, and if the matching is unsuccessful, the invention informs related personnel to carry out special treatment on books which cannot be identified. Thus obtaining book information and carrying out book identification and checking.
According to the book checking identification method based on the images and the texts, the books are identified by comprehensively utilizing the camera acquisition data, the text identification technology, the computer mode identification technology, the text matching technology and the like, so that the automatic book checking of the intelligent library is assisted, the identification of the books is realized, and library checking is conducted by subsequent equipment library managers. Compared with the traditional schemes such as RFID, the book checking identification method based on the images and the texts has the advantages of stronger adaptability, wider application range, lower cost, no dependence on expensive RFID bar codes, no dependence on a large amount of manpower for manual checking and good universality.
The book checking identification method based on the images and the texts can acquire the book bookshelf images, the book spine characters and the book information so that books can be sorted and checked, the problem that books are misplaced and the like is solved, and meanwhile, operators can conveniently observe and go forward for one-step operation and check.
Example 2
As shown in fig. 4, the book checking identification device 100 based on images and texts of the present embodiment includes: the acquisition processing module 110, the identification module 120 and the judgment matching module 130.
In this embodiment, the acquisition processing module 110 is configured to acquire a bookshelf image and cut the bookshelf image into a plurality of spine images including a spine.
Specifically, as shown in fig. 5, in this embodiment, the acquisition processing module 110 includes: an image acquisition unit 111, an edge detection unit 112, a cutting unit 113, and a correction unit 114.
In the present embodiment, the image capturing unit 111 is configured to capture a bookshelf image from the image capturing apparatus 20; the edge detection unit 112 is configured to detect an edge of the bookshelf image using an edge detection algorithm; the cutting unit 113 is configured to identify a straight line in the bookshelf image, and cut the bookshelf image into a plurality of spine images including a spine according to the identified straight line; the correction unit 114 is configured to perform direction correction and/or sharpness correction on the spine image.
The edge detection can be performed by adopting an edge detection algorithm such as a canny algorithm, a Sobel algorithm, a Laplacian algorithm and the like. Edge detection of images is well known to those skilled in the art and will not be described in detail herein.
In this embodiment, the line recognition algorithm for recognizing the line in the bookshelf image may adopt an LSD line detection method or a hough transform method.
In this embodiment, the bookshelf image in the camera is collected in real time, for example, the image is subjected to convolution preprocessing by adopting an optimized convolution operator, and points with gradient amplitude larger than a threshold value are marked as edges, so that edges of the bookshelf image are detected; next, a straight line in the edge of the bookshelf image is recognized using hough transform, and the bookshelf image is cut according to the straight line, thereby cutting the bookshelf image into a plurality of spine images containing the spine.
In this embodiment, for example, the spine image is corrected by using functions such as affine functions, so that the text display on the spine image is clear, so that the corrected spine image is subjected to text recognition.
In this embodiment, the identifying module 120 is configured to identify the spine image, and obtain keywords corresponding to the spines.
Specifically, in this embodiment, the recognition module 120 recognizes the spine image by using an OCR text model, and obtains keywords corresponding to each spine. Character recognition by OCR character models is well known to those skilled in the art and will not be described in detail herein.
In this embodiment, the determining and matching module 130 is configured to determine whether the similarity between the keyword corresponding to any one of the spine and the book information in the book database reaches a first similarity threshold: if yes, finishing checking the books corresponding to the spine; if not, taking the keyword corresponding to the spine and at least one keyword corresponding to the spine around the spine as an overall keyword, and continuously judging whether the similarity between the overall keyword and book information in a book database reaches a second similarity threshold value: if yes, finishing checking books corresponding to all the spines forming the integral key words; if not, determining that the checking fails.
Wherein the keywords and the overall keywords are book information. In the embodiment, text matching is carried out on book information identified by a single book and book information in a book database, and if the similarity between the keyword information identified by the book and the text of a certain book in the book database reaches a set threshold, the matching is considered to be successful; and if not, putting the keywords extracted from books arranged around the books into a book database for integral matching, judging the text similarity again, if the text similarity reaches a threshold value, successfully matching, and outputting the obtained result to a subsequent system after the matching is successful, thereby obtaining book information, and checking books.
The technical features of the specific implementation of the image and text-based book checking identification device 100 in this embodiment are substantially the same as the image and text-based book checking identification method in the foregoing embodiment 1, and general technical contents between embodiments may not be repeated.
It should be noted that, it should be understood that the division of the modules of the above apparatus is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. And these modules may all be implemented in software in the form of calls by the processing element; or can be realized in hardware; the method can also be realized in a form of calling software by a processing element, and the method can be realized in a form of hardware by a part of modules. For example, the identification module 120 may be a processing element that is set up separately, may be implemented in a chip of an electronic terminal, or may be stored in a memory of the terminal in the form of program codes, and may be called by a processing element of the terminal to execute the functions of the tracking calculation module. The implementation of the other modules is similar. In addition, all or part of the modules can be integrated together or can be independently implemented. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.
For example, the modules above may be one or more integrated circuits configured to implement the methods above, such as: one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), or one or more microprocessors (digital singnal processor, abbreviated as DSP), or one or more field programmable gate arrays (Field Programmable Gate Array, abbreviated as FPGA), or the like. For another example, when a module above is implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU) or other processor that may invoke the program code. For another example, the modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Example 3
As shown in fig. 6, the present embodiment further provides an electronic device 10, where the electronic device 10 includes a processor 1101 and a memory 1102.
The electronic device 10 may be, for example, a stationary terminal, such as a server, desktop, or the like; and may also be a mobile terminal such as a notebook, smart phone or tablet computer.
The memory 1102 is connected to the processor 1101 through a system bus and performs communication with each other, the memory 1102 is used for storing a computer program, the processor 1101 is coupled to the display 1003 and the memory 1002, and the processor 1101 is used for running the computer program to enable the electronic device 10 to execute the image and text based book checking identification method described in embodiment 1. Embodiment 1 has already been described in detail for the book checking identification method based on images and texts, and will not be described here again.
The image and text based book inventory recognition method described may be applied to various types of electronic devices 10. In an exemplary embodiment, the electronic device 10 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, cameras or other electronic components for performing the above-described image and text-based book inventory recognition methods.
It should be noted that the system bus mentioned above may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus. The system bus may be classified into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 6, but not only one bus or one type of bus. The communication interface is used to enable communication between the database access apparatus and other devices (e.g., clients, read-write libraries, and read-only libraries). The memory may comprise random access memory (Random Access Memory, RAM) and may also comprise non-volatile memory (non-volatile memory), such as at least one disk memory.
The processor 1101 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by computer program related hardware. The aforementioned computer program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.
Example 4
As shown in fig. 7, the present embodiment provides a book inventory identification system 1, the book inventory identification system 1 including: an image pickup apparatus 20 and an electronic apparatus 10 as described in embodiment 3.
The image capturing device 20 is specifically disposed on a mobile vehicle, and is connected to the electronic device 10 through a wireless network, and is configured to capture a bookshelf image and transmit the bookshelf image to the electronic device 10. The image capture device 20 is preferably, but not limited to, a camera.
In summary, the invention obtains book information through camera visual analysis and text matching, provides basis and reference for automatic book checking of subsequent libraries, can assist automatic book checking of intelligent libraries, has stronger adaptability, wider application range and lower cost compared with the traditional schemes such as RFID technology, and has good universality without relying on a large amount of manpower for manual checking. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.
Claims (7)
1. A book checking identification method based on images and texts is characterized in that: comprising the following steps:
collecting a bookshelf image and cutting the bookshelf image into a plurality of spine images containing spines, wherein the bookshelf image comprises: detecting the edge of the bookshelf image by utilizing an edge detection algorithm, identifying straight lines in the bookshelf image, and cutting the bookshelf image into a plurality of spine images containing spines according to the identified straight lines;
detecting edges of the bookshelf image using the edge detection algorithm, comprising: carrying out convolution pretreatment on the image by adopting an optimized convolution operator, and marking points with gradient amplitude values larger than a threshold value as edges so as to detect the edges of the book shelf image; performing direction correction and definition correction on the spine image; the direction correction and sharpness correction include: correcting the spine image by using an affine function to enable the text display front on the spine image to be clear; identifying the spine images to obtain keywords corresponding to the spines;
judging whether the similarity between any keyword corresponding to the spine and book information in a book database reaches a first similarity threshold value:
if yes, finishing checking the books corresponding to the spine;
if not, taking the keyword corresponding to the spine and at least one keyword corresponding to the spine around the spine as an overall keyword, and continuously judging whether the similarity between the overall keyword and book information in a book database reaches a second similarity threshold value:
if yes, finishing checking books corresponding to all the spines forming the integral key words;
if not, determining that the checking fails;
wherein the book information includes the keywords and the overall keywords;
the step of judging whether the similarity between the key word corresponding to any spine and the book information in the book database reaches a first similarity threshold value comprises the following steps: performing text matching on the keywords and book information in a book database, and judging whether the text similarity of the keywords and the book information in the book database reaches the first similarity threshold;
the step of judging whether the similarity between the whole keyword and the book information in the book database reaches a second similarity threshold value comprises the following steps: and carrying out overall matching on the overall keyword information and book information in a book database, and judging whether the text similarity between the overall keyword information and the book information in the book database reaches the second similarity threshold.
2. The image and text based book inventory identification method of claim 1, wherein: and identifying the spine image by utilizing an OCR text model to obtain keywords corresponding to the spines.
3. A book checking and identifying device based on images and texts is characterized in that: comprising the following steps:
the acquisition processing module is used for acquiring a bookshelf image and cutting the bookshelf image into a plurality of spine images containing spines, and comprises the following steps: detecting the edge of the bookshelf image by utilizing an edge detection algorithm, identifying straight lines in the bookshelf image, and cutting the bookshelf image into a plurality of spine images containing spines according to the identified straight lines; detecting edges of the bookshelf image using the edge detection algorithm, comprising: carrying out convolution pretreatment on the image by adopting an optimized convolution operator, and marking points with gradient amplitude values larger than a threshold value as edges so as to detect the edges of the book shelf image; performing direction correction and definition correction on the spine image; the direction correction and sharpness correction include: correcting the spine image by using an affine function to enable the text display front on the spine image to be clear;
the identification module is used for identifying the spine images and obtaining keywords corresponding to the spines;
the judging and matching module is used for judging whether the similarity between any keyword corresponding to the spine and book information in the book database reaches a first similarity threshold value or not: if yes, finishing checking the books corresponding to the spine; if not, taking the keyword corresponding to the spine and at least one keyword corresponding to the spine around the spine as an overall keyword, and continuously judging whether the similarity between the overall keyword and book information in a book database reaches a second similarity threshold value: if yes, finishing checking books corresponding to all the spines forming the integral key words; if not, determining that the checking fails; wherein the book information includes the keywords and the overall keywords; the step of judging whether the similarity between the key word corresponding to any spine and the book information in the book database reaches a first similarity threshold value comprises the following steps: performing text matching on the keywords and book information in a book database, and judging whether the text similarity of the keywords and the book information in the book database reaches the first similarity threshold; the step of judging whether the similarity between the whole keyword and the book information in the book database reaches a second similarity threshold value comprises the following steps: and carrying out overall matching on the overall keyword information and book information in a book database, and judging whether the text similarity between the overall keyword information and the book information in the book database reaches the second similarity threshold.
4. The image and text based book inventory recognition device of claim 3, wherein: the acquisition processing module further comprises:
and the image acquisition unit is used for acquiring the bookshelf image from the image shooting equipment.
5. The image and text based book inventory recognition device of claim 3, wherein: and the recognition module recognizes the spine image by utilizing an OCR text model and acquires keywords corresponding to the spines.
6. An electronic device, characterized in that: the system comprises a processor and a memory, wherein the memory stores program instructions; the processor executes program instructions to implement the image and text based book inventory identification method of any one of claims 1 and 2.
7. A book inventory identification system, comprising:
the image shooting device is specially arranged on a mobile vehicle and is used for shooting bookshelf images;
an electronic device as claimed in claim 6, connected to the image capturing device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010762471.1A CN111898555B (en) | 2020-07-31 | 2020-07-31 | Book checking identification method, device, equipment and system based on images and texts |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010762471.1A CN111898555B (en) | 2020-07-31 | 2020-07-31 | Book checking identification method, device, equipment and system based on images and texts |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111898555A CN111898555A (en) | 2020-11-06 |
| CN111898555B true CN111898555B (en) | 2023-05-19 |
Family
ID=73183052
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010762471.1A Active CN111898555B (en) | 2020-07-31 | 2020-07-31 | Book checking identification method, device, equipment and system based on images and texts |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111898555B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112395939A (en) * | 2020-09-07 | 2021-02-23 | 江苏感创电子科技股份有限公司 | Book checking method and system |
| CN112686157A (en) * | 2020-12-30 | 2021-04-20 | 上海书山智能科技有限公司 | Book positioning and identifying method based on deep learning |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106846354A (en) * | 2017-01-23 | 2017-06-13 | 中国人民解放军海军航空工程学院 | A kind of Book Inventory method on frame converted based on image segmentation and random hough |
| CN108875616A (en) * | 2018-06-07 | 2018-11-23 | 广东数相智能科技有限公司 | A kind of Books Chaotic Frame detection method, electronic equipment and storage medium |
| CN108921160A (en) * | 2018-05-04 | 2018-11-30 | 广东数相智能科技有限公司 | A kind of books recognition methods, electronic equipment and storage medium |
| CN109558559A (en) * | 2018-11-30 | 2019-04-02 | 掌阅科技股份有限公司 | Methods of exhibiting, electronic equipment and the computer storage medium of the bookshelf page |
| CN110223340A (en) * | 2019-06-11 | 2019-09-10 | 广西财经学院 | A kind of method, apparatus of robot identification books place frame position |
| CN110909743A (en) * | 2019-11-25 | 2020-03-24 | 章志亮 | Book checking method and book checking system |
| CN111340035A (en) * | 2020-03-24 | 2020-06-26 | 山东浪潮通软信息科技有限公司 | Train ticket identification method, system, equipment and medium |
-
2020
- 2020-07-31 CN CN202010762471.1A patent/CN111898555B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106846354A (en) * | 2017-01-23 | 2017-06-13 | 中国人民解放军海军航空工程学院 | A kind of Book Inventory method on frame converted based on image segmentation and random hough |
| CN108921160A (en) * | 2018-05-04 | 2018-11-30 | 广东数相智能科技有限公司 | A kind of books recognition methods, electronic equipment and storage medium |
| CN108875616A (en) * | 2018-06-07 | 2018-11-23 | 广东数相智能科技有限公司 | A kind of Books Chaotic Frame detection method, electronic equipment and storage medium |
| CN109558559A (en) * | 2018-11-30 | 2019-04-02 | 掌阅科技股份有限公司 | Methods of exhibiting, electronic equipment and the computer storage medium of the bookshelf page |
| CN110223340A (en) * | 2019-06-11 | 2019-09-10 | 广西财经学院 | A kind of method, apparatus of robot identification books place frame position |
| CN110909743A (en) * | 2019-11-25 | 2020-03-24 | 章志亮 | Book checking method and book checking system |
| CN111340035A (en) * | 2020-03-24 | 2020-06-26 | 山东浪潮通软信息科技有限公司 | Train ticket identification method, system, equipment and medium |
Non-Patent Citations (2)
| Title |
|---|
| "Image Recognition with Deep Learning for Library Book Identification";Kaichen Tang et al.;《SpringerLink》;20180918;全文 * |
| "基于视觉的图书馆在架错序图书自动识别技术研究";冯庸;《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》;20170815(第08期);第29-32页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111898555A (en) | 2020-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200342248A1 (en) | Methods for mobile image capture of vehicle identification numbers in a non-document | |
| CN110705405B (en) | Target labeling method and device | |
| WO2020164278A1 (en) | Image processing method and device, electronic equipment and readable storage medium | |
| US9679354B2 (en) | Duplicate check image resolution | |
| CN111898555B (en) | Book checking identification method, device, equipment and system based on images and texts | |
| WO2020155790A1 (en) | Method and apparatus for extracting claim settlement information, and electronic device | |
| CN111695453B (en) | Drawing recognition method and device and robot | |
| CN111191568A (en) | Method, device, equipment and medium for identifying copied image | |
| WO2019144416A1 (en) | Information processing method and system, cloud processing device and computer program product | |
| CN112100430B (en) | Article tracing method and device | |
| CN104750791A (en) | Image retrieval method and device | |
| CN114419363A (en) | Target classification model training method and device based on label-free sample data | |
| CN113111880A (en) | Certificate image correction method and device, electronic equipment and storage medium | |
| Ganapathy et al. | A Malaysian vehicle license plate localization and recognition system | |
| CN107992872B (en) | Method for carrying out text recognition on picture and mobile terminal | |
| CN111931687B (en) | Bill identification method and device | |
| CN113657378B (en) | Vehicle tracking method, vehicle tracking system and computing device | |
| CN114266267A (en) | Automatic identification method and device for collecting two-dimensional codes, documents, certificates and human faces and storage medium | |
| CN114090650A (en) | Sample data identification method and device, electronic equipment and storage medium | |
| CN110942069A (en) | Method, device, storage medium and equipment for extracting key information of vehicle insurance policy | |
| CN112862409A (en) | Picking bill verification method and device | |
| CN113780269A (en) | Image recognition method, device, computer system and readable storage medium | |
| CN113379669A (en) | Method, device and equipment for beverage cabinet image recognition | |
| CN113468532B (en) | Malicious software family inference method and system | |
| US20230130990A1 (en) | Reducing false detections in template-based classification of identity documents |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |