CN114701264A - Book collection and editing preprocessing method, device and system - Google Patents
Book collection and editing preprocessing method, device and system Download PDFInfo
- Publication number
- CN114701264A CN114701264A CN202210402294.5A CN202210402294A CN114701264A CN 114701264 A CN114701264 A CN 114701264A CN 202210402294 A CN202210402294 A CN 202210402294A CN 114701264 A CN114701264 A CN 114701264A
- Authority
- CN
- China
- Prior art keywords
- book
- spine
- orientation
- image
- conveying
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B41—PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
- B41J—TYPEWRITERS; SELECTIVE PRINTING MECHANISMS, i.e. MECHANISMS PRINTING OTHERWISE THAN FROM A FORME; CORRECTION OF TYPOGRAPHICAL ERRORS
- B41J3/00—Typewriters or selective printing or marking mechanisms characterised by the purpose for which they are constructed
- B41J3/407—Typewriters or selective printing or marking mechanisms characterised by the purpose for which they are constructed for marking on special material
- B41J3/4075—Tape printers; Label printers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B41—PRINTING; LINING MACHINES; TYPEWRITERS; STAMPS
- B41J—TYPEWRITERS; SELECTIVE PRINTING MECHANISMS, i.e. MECHANISMS PRINTING OTHERWISE THAN FROM A FORME; CORRECTION OF TYPOGRAPHICAL ERRORS
- B41J3/00—Typewriters or selective printing or marking mechanisms characterised by the purpose for which they are constructed
- B41J3/44—Typewriters or selective printing mechanisms having dual functions or combined with, or coupled to, apparatus performing other functions
- B41J3/445—Printers integrated in other types of apparatus, e.g. printers integrated in cameras
Landscapes
- Character Input (AREA)
Abstract
The invention provides a method, a device and a system for preprocessing book collection and editing, wherein the method for preprocessing book collection and editing comprises the following steps: acquiring an upper surface image of a book on a conveying mechanism; identifying the character direction of the upper surface image; acquiring the orientation of the front cover or the orientation of the back cover of the book, and acquiring the orientation of the spine of the book or the orientation of the opposite side of the spine of the book; and judging the book turning direction based on the character direction of the upper surface of the book, the front cover direction or the back cover direction and the spine direction or the opposite side direction of the spine, wherein the book turning direction comprises left book turning and right book turning. The book page turning direction can be identified, so that subsequent actuating mechanisms such as a marking device and a page turning device can conveniently make corresponding measures according to the book page turning direction, and the working efficiency of the subsequent actuating mechanisms is improved.
Description
Technical Field
The invention relates to the technical field of book processing, in particular to a method, a device and a system for preprocessing book collection and editing.
Background
The books volume in library is huge, and the work load of borrowing and returning books outward is big to make mistakes easily, for convenient more, efficient management books, at books collection and compilation in-process, need print library code label, RFID label or stamp on the page of books. The books in the library comprise a left book turning spine and a right book turning spine, wherein the left book turning spine is arranged on the left side, and the right book turning spine is arranged on the right side. When the system is adopted and compiled, the left page turning and the right page turning which are mixed together need to be processed in a self-adaptive mode. The existing collecting and editing system has poor self-adaptive processing capability, cannot acquire the characteristics of left and right book turning, character orientation and the like of the book before the book enters the executing mechanism, and performs corresponding preprocessing according to the characteristics, so that the processing efficiency is low.
Disclosure of Invention
The invention aims to at least solve the technical problems in the prior art and provides a book collection and editing preprocessing method, device and system.
In order to achieve the above object, according to a first aspect of the present invention, there is provided a book compilation preprocessing method, including: acquiring an image of the upper surface of a book on a conveying mechanism; identifying the character direction of the upper surface image; acquiring the orientation of the front cover or the orientation of the back cover of the book, and acquiring the orientation of the spine of the book or the orientation of the opposite side of the spine of the book; and judging the book turning direction based on the character direction of the upper surface of the book, the front cover direction or the back cover direction and the spine direction or the opposite side direction of the spine, wherein the book turning direction comprises left book turning and right book turning.
The technical scheme is as follows: before the book gathering and editing processing, the book page turning direction can be identified, so that subsequent executing mechanisms such as a marking device and a page turning device can conveniently make corresponding measures according to the book page turning direction, and the working efficiency of the subsequent executing mechanisms is improved.
In a preferred embodiment of the present invention, a plurality of triples are constructed, wherein the triples comprise a text direction, a front cover orientation and a spine orientation of the upper surface of the book, or comprise a text direction, a back cover orientation and a spine orientation of the upper surface of the book, or comprise a text direction, a front cover orientation and a spine opposite side orientation of the upper surface of the book, or comprise a text direction, a back cover orientation and a spine opposite side orientation of the upper surface of the book; and establishing a corresponding relation table of the triples and the book turning direction, and searching the corresponding relation table based on the character direction of the upper surface of the book, the front cover direction or the back cover direction, and the spine direction or the direction of the opposite side of the spine to obtain the book turning direction.
The technical scheme is as follows: by establishing the corresponding relation table, the book turning direction of the book can be quickly acquired, information support is provided for the subsequent execution mechanism operation, and the working efficiency of the subsequent execution mechanism is improved.
In a preferred embodiment of the present invention, invoking a label printing template according to the text direction includes: if the character direction is the same as the conveying direction of the conveying mechanism, calling a first label printing template, wherein the character direction of the first label printing template is the same as the conveying direction; and if the character direction is opposite to the conveying direction of the conveying mechanism, calling a second label printing template, wherein the character direction of the second label printing template is opposite to the conveying direction.
The technical scheme is as follows: the character direction of the upper surface of the book is identified, the corresponding label printing template is called according to the character direction, and the label which accords with the character direction of the book is printed by the subsequent marking machine according to the called label printing template, so that the work complexity of the subsequent marking machine can be reduced, and the work efficiency is improved.
In a preferred embodiment of the present invention, the process of acquiring the orientation of the book cover is: and identifying the ISBN (international standard bn) number of the upper surface image, considering that the book cover faces downwards if the ISBN identification is successful, and considering that the book cover faces upwards if the ISBN identification is unsuccessful.
The technical scheme is as follows: can discern the judgement to the book cover orientation on the transmission device, the corresponding measure is done according to the book cover orientation to the follow-up actuating mechanism of being convenient for, further improves follow-up actuating mechanism's work efficiency.
In a preferred embodiment of the present invention, the method further includes a step of judging the orientation of the cover, which specifically includes: and when the orientation of the book cover is different from the orientation of the set cover, giving an alarm for prompting.
The technical scheme is as follows: and the book with the wrong cover state is reported by warning, so that the working efficiency of a subsequent execution mechanism is further improved.
In a preferred embodiment of the present invention, the process of acquiring the orientation of the book spine comprises: acquiring a side image of a book on the right side of the conveying track, judging whether the side image is a spine image, if so, regarding the spine of the book to face the right side of the conveying track, otherwise, regarding the spine of the book to face the left side of the conveying track; or acquiring a side image of the book at the left side of the conveying track, judging whether the side image is a spine image, if so, regarding the spine of the book to face the left side of the conveying track, otherwise, regarding the spine of the book to face the right side of the conveying track.
The technical scheme is as follows: the orientation of the spine can be acquired, so that the subsequent actuating mechanism such as a page turning device can conveniently make corresponding measures according to the orientation of the spine, and the working efficiency of the subsequent actuating mechanism is further improved.
In order to achieve the above object, according to a second aspect of the present invention, there is provided a book editing pre-processing apparatus comprising: the image acquisition module is used for acquiring an image of the upper surface of the book on the conveying mechanism; the character direction identification module is used for identifying the character direction of the upper surface image; the book cover orientation acquisition module is used for acquiring the orientation of the book cover; the book spine orientation acquisition module is used for acquiring the book spine orientation; and the left-right book turning judging module is used for judging the book turning direction of the book based on the character direction, the cover orientation and the spine orientation of the upper surface of the book, and the book turning direction comprises left book turning and right book turning.
The technical scheme is as follows: the book page turning direction can be identified, so that subsequent actuating mechanisms such as a marking device and a page turning device can conveniently make corresponding measures according to the book page turning direction, and the working efficiency of the subsequent actuating mechanisms is improved.
In order to achieve the above object, according to a third aspect of the present invention, there is provided a book compilation preprocessing system, comprising: the first camera is positioned above the conveying mechanism and used for shooting the upper surface images of the books on the conveying mechanism; and the processor is used for receiving the output image of the first camera and executing the steps of the book editing preprocessing method in the first aspect of the invention.
The technical scheme is as follows: the system can identify the book turning direction of the book, so that subsequent actuating mechanisms such as a marking device and a page turning device can conveniently make corresponding measures according to the book turning direction, and the working efficiency of the subsequent actuating mechanisms is improved.
In order to achieve the above object, according to a fourth aspect of the present invention, the present invention provides a book editing system, which includes a control module, a conveying mechanism, and a camera module and an actuator, which are sequentially installed on the conveying mechanism along a book conveying direction; the camera module is used for acquiring an image of the upper surface of the book on the conveying mechanism; the executing mechanism comprises a left page turning mechanism, a left page turning mechanism driving device, a right page turning mechanism and a right page turning mechanism driving device; the control module is respectively connected with the camera module, the left page turning mechanism driving device and the right page turning mechanism driving device; the control module obtains the book page turning direction according to the book gathering and editing preprocessing method of the first aspect of the invention, starts the left page turning mechanism and closes the right page turning mechanism when the book is left-handed, and starts the right page turning mechanism and closes the left page turning mechanism when the book is right-handed.
The technical scheme is as follows: the operation of the corresponding left book turning mechanism and the right book turning mechanism can be distributed according to the book turning direction of the books, the self-adaptability is strong, and the working efficiency of the system is improved.
In a preferred embodiment of the present invention, the actuator further comprises a marking device; the control module is connected with the marking device, the control module obtains the character direction of the upper surface of the book according to the book editing and preprocessing method in embodiment 1, if the character direction is the same as the conveying direction of the conveying mechanism, the control module controls the marking machine to call a first label printing template to print a label, and if the character direction is opposite to the conveying direction of the conveying mechanism, the control module controls the printing machine to call a second label printing template to print a label.
The technical scheme is as follows: the character direction of the upper surface of the book is identified, the corresponding label printing template is called according to the character direction, and the label which accords with the character direction of the book is printed by the subsequent marking machine according to the called label printing template, so that the work complexity of the subsequent marking machine can be reduced, and the work efficiency is improved.
Drawings
Fig. 1 is a schematic flow chart of a pre-processing method for book compilation in embodiment 1 of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, it is to be understood that the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed in a particular orientation, and be operated, and thus, are not to be construed as limiting the present invention.
In the description of the present invention, unless otherwise specified and limited, it is to be noted that the terms "mounted," "connected," and "connected" are to be interpreted broadly, and may be, for example, a mechanical connection or an electrical connection, a communication between two elements, a direct connection, or an indirect connection via an intermediate medium, and specific meanings of the terms may be understood by those skilled in the art according to specific situations.
Example 1
The embodiment discloses a book collection and editing preprocessing method, as shown in fig. 1, the method includes:
in step S1, an image of the top surface of the book on the transport mechanism is acquired. The books to be processed are transmitted to the editing execution mechanism such as a marking machine, a page turning machine and the like through the conveying mechanism, the books are generally squared on the conveying mechanism, and the upper surface images of the books can be obtained by shooting the books from top to bottom through the camera.
In step S2, the character direction of the top surface image is recognized. The surface image may be a back cover image or a front cover image. When the upper surface image is the back cover image, the character direction of the back cover image can be judged by identifying the number direction of the ISBN number in order to simplify the character direction identification process because the number direction of the ISBN number is the same as the character direction of the back cover. The character direction recognition based on the image is a conventional technology in the art, such as an existing OCR (optical character recognition) algorithm, or a technical solution disclosed in chinese patent with publication number CN102855477A or CN110443239A, and is not described herein again.
In step S3, the book cover orientation and the spine orientation are obtained.
In this embodiment, preferably, the process of acquiring the book cover orientation is: and identifying the ISBN of the upper surface image, if the ISBN is successfully identified, considering that the book cover faces downwards, and if the ISBN is unsuccessfully identified, considering that the book cover faces upwards. Specifically, an ISBN number recognition program may be set to sequentially scan the barcode in each area of the upper surface image, and if the barcode is successfully scanned, the ISBN number is considered to exist and the upper surface image is a back cover, otherwise, the ISBN number is considered to not exist and the upper surface image is a front cover. The upper surface image may be subjected to character recognition, and if "ISBN" is recognized, the upper surface is considered as a back cover, and if "ISBN" is not recognized, the upper surface is considered as a front cover. The book back cover orientation can also be obtained according to the book cover orientation judging process, which is not described herein again.
In this embodiment, step S3 may also be acquiring the book cover orientation and the spine opposite side orientation, or acquiring the book back cover orientation and the spine opposite side orientation.
In this embodiment, it is further preferable that the method further includes a step of judging the orientation of the cover, which specifically includes: and when the orientation of the book cover is different from the orientation of the set cover, giving an alarm for prompting. When the book collection system specifies the cover orientation of the book entering the system, the book with the cover orientation not correct needs to be warned and reported so that the relevant staff can process the book or the corresponding rejecting mechanism rejects the book.
In this embodiment, the direction of the spine can be known in advance, for example, when some editing systems require the spine to be placed in a set orientation when the book is on line, the direction of the spine of the book can be known in advance. The judgment of the spine direction can also be realized by the following scheme that a distance measuring sensor is arranged above the conveying track and used for measuring the distance between the distance measuring sensor and the upper surface of a book below (when the spine direction is judged, the book is required to be static relative to the conveying channel, if the conveying channel stops working), a blowing device blowing towards the side part of the book is arranged on the left side/right side of the conveying track, the spine direction is judged according to whether the output signals of the distance measuring sensor before and after the blowing of the blowing device change, if the signals change, the spine is considered to face the right side/left side of the conveying track, and if the signals do not change, the spine is considered to face the left side/right side of the conveying track. Specifically, the amplitude change of the output signal of the ranging sensor is considered to exceed plus or minus 2%, and the signal is considered to have a change. The range sensor is preferably, but not limited to, an existing laser range sensor. The blowing device is preferably, but not limited to, an existing turbo fan.
In this embodiment, preferably, in order to improve the accuracy of determining the book spine direction, and is easier to implement, the process of acquiring the book spine orientation includes: acquiring a side image of the book on the right side of the conveying track, judging whether the side image is a spine image, if so, regarding the spine of the book to face the right side of the conveying track, otherwise, regarding the spine of the book to face the left side of the conveying track; or acquiring a side image of the book at the left side of the conveying track, judging whether the side image is a spine image, if the side image is the spine image, considering that the spine of the book faces the left side of the conveying track, and if not, considering that the spine of the book faces the right side of the conveying track.
In this embodiment. Preferably, the spine recognition is performed by a machine learning model. The method comprises the steps of obtaining a plurality of book spine images as positive samples, taking a plurality of page images (book spine opposite surfaces) as negative samples, marking 'yes' labels on the positive samples, marking 'no' labels on the negative samples, and dividing all samples into a training set, a verification set and a test set. And establishing a neural network structure, such as an HRNet network. Training the neural network structure through a training set, testing and verifying the trained neural network by using a testing set and a verifying set to obtain a spine recognition model, inputting the side images into the spine recognition model, and outputting yes or no by the spine recognition model. Based on the above process, chinese patent with publication number CN111401252A has appeared in the prior art to realize spine recognition. The technical proposal disclosed in the Chinese patent with the publication number of CN104966081A can also be adopted.
In this embodiment, the above method for determining the orientation of the spine can be correspondingly applied to determine the orientation of the opposite side of the spine (i.e. the page side of the page to be turned), which is not described herein again.
Step S4, determining the book turning direction of the book based on the text direction of the top surface of the book, the front cover direction or the back cover direction, and the spine direction or the opposite side direction, the book turning direction including left book turning and right book turning.
In this embodiment, preferably, to quickly obtain the page turning direction, the following scheme is adopted: constructing a plurality of triples, wherein the triples comprise the text direction, the front cover direction and the spine direction of the upper surface of the book, or the triples comprise the text direction, the back cover direction and the spine direction of the upper surface of the book, or the triples comprise the text direction, the front cover direction and the spine opposite side direction of the upper surface of the book, or the triples comprise the text direction, the back cover direction and the spine opposite side direction of the upper surface of the book; and establishing a corresponding relation table of the triples and the book turning direction, searching the corresponding relation table based on the character direction of the upper surface of the book, the front cover direction or the back cover direction, and the spine direction or the direction of the opposite side of the spine to obtain the book turning direction, and obtaining the book turning direction of the book in a table searching mode in the corresponding relation table.
In this embodiment, preferably, the triplet includes a text direction of the upper surface of the book, a front cover orientation, and a spine orientation, and the specific search process includes:
when the character direction of the upper surface of the book is the same as the conveying direction, the cover faces upwards, and the spine faces towards the right side of the conveying track, the book is considered to be a left-turning book;
when the character direction of the upper surface of the book is the same as the conveying direction, the cover faces downwards, and the spine faces towards the right side of the conveying track, the book is considered to be a right-turning book;
when the character direction of the upper surface of the book is opposite to the conveying direction, the cover faces upwards, and the spine faces towards the right side of the conveying track, the book is considered to be a right-turning book;
when the direction of characters on the upper surface of the book is opposite to the conveying direction, the cover faces downwards, and the spine faces towards the right side of the conveying track, the book is considered to be a left-turning book;
when the character direction of the upper surface of the book is the same as the conveying direction, the cover faces upwards, and the spine faces towards the left side of the conveying track, the book is considered to be a right-turning book;
when the character direction of the upper surface of the book is the same as the conveying direction, the cover faces downwards, and the spine faces towards the left side of the conveying track, the book is considered to be a left-turning book;
when the character direction of the upper surface of the book is opposite to the conveying direction, the cover faces upwards, and the spine faces towards the left side of the conveying track, the book is considered to be a left-turning book;
a book is considered to be a right-handed book when the upper surface of the book has characters oriented opposite the conveying direction, with the cover facing down and the spine facing to the left of the conveying track.
In this embodiment, preferably, the triplet includes a text direction, a back cover orientation, and a spine orientation of the upper surface of the book, and the specific search process includes:
when the character direction of the upper surface of the book is the same as the conveying direction, the back cover is downward, and the spine of the book faces the right side of the conveying track, the book is considered as a left-turning book;
when the character direction of the upper surface of the book is the same as the conveying direction, the back cover is upward, and the spine of the book faces the right side of the conveying track, the book is considered to be a right-turning book;
when the character direction of the upper surface of the book is opposite to the conveying direction, the back cover is downward, and the spine of the book faces the right side of the conveying track, the book is considered to be a right-turning book;
when the character direction of the upper surface of the book is opposite to the conveying direction, the back cover is upward, and the spine of the book faces the right side of the conveying track, the book is considered as a left-turning book;
when the character direction of the upper surface of the book is the same as the conveying direction, the back cover is downward, and the spine of the book faces the left side of the conveying track, the book is considered to be a right-turning book;
when the character direction of the upper surface of the book is the same as the conveying direction, the back cover faces upwards, and the spine faces to the left side of the conveying track, the book is considered to be a left-turning book;
when the character direction of the upper surface of the book is opposite to the conveying direction, the back cover is downward, and the spine of the book faces the left side of the conveying track, the book is considered as a left-turning book;
when the direction of characters on the upper surface of the book is opposite to the conveying direction, the back cover faces upwards, and the spine faces to the left side of the conveying track, the book is considered to be a right-turning book.
In this embodiment, preferably, the triples include the text direction of the upper surface of the book, the front cover orientation, and the opposite side orientation of the spine, and the specific search process includes:
when the direction of characters on the upper surface of the book is the same as the conveying direction, the front cover faces upwards, and the opposite side of the spine faces to the left side of the conveying track, the book is considered to be left-turned;
when the direction of characters on the upper surface of the book is the same as the conveying direction, the front cover faces downwards, and the opposite side of the spine faces to the left side of the conveying track, the book is considered to be a right-turning book;
when the direction of characters on the upper surface of the book is opposite to the conveying direction, the front cover faces upwards, and the opposite side of the spine faces to the left side of the conveying track, the book is considered to be turned right;
when the direction of characters on the upper surface of the book is opposite to the conveying direction, the front cover faces downwards, and the opposite side of the spine faces to the left side of the conveying track, the book is considered to be left-turned;
when the direction of characters on the upper surface of the book is the same as the conveying direction, the front cover faces upwards, and the opposite side of the spine faces to the right side of the conveying track, the book is considered to be a right-turning book;
when the direction of characters on the upper surface of the book is the same as the conveying direction, the front cover faces downwards, and the opposite side of the spine faces to the right side of the conveying track, the book is considered to be left-turned;
when the direction of characters on the upper surface of the book is opposite to the conveying direction, the front cover faces upwards, and the opposite side of the spine faces to the right side of the conveying track, the book is considered to be left-turned;
a book is considered to be right-turned when the upper surface of the book has characters oriented opposite the conveying direction, with the front cover facing downward and the spine opposite the front side facing the right side of the conveying track.
In this embodiment, preferably, the triplet includes a text direction of the upper surface of the book, a back cover orientation, and a spine opposite side orientation, and the specific search process includes:
when the character direction of the upper surface of the book is the same as the conveying direction, the back cover faces downwards, and the opposite side of the spine faces to the left side of the conveying track, the book is considered to be a left-turning book;
when the character direction of the upper surface of the book is the same as the conveying direction, the back cover faces upwards, and the opposite side of the spine faces to the left side of the conveying track, the book is considered to be a right-turning book;
when the character direction of the upper surface of the book is opposite to the conveying direction, the back cover faces downwards, and the opposite side of the spine faces to the left side of the conveying track, the book is considered to be a right-turning book;
when the character direction of the upper surface of the book is opposite to the conveying direction, the back cover faces upwards, and the opposite side of the spine faces to the left side of the conveying track, the book is considered as a left-turning book;
when the character direction of the upper surface of the book is the same as the conveying direction, the back cover faces downwards, and the opposite side of the spine faces the right side of the conveying track, the book is considered to be a right-turning book;
when the character direction of the upper surface of the book is the same as the conveying direction, the back cover faces upwards, and the opposite side of the spine faces the right side of the conveying track, the book is considered to be a left-turning book;
when the character direction of the upper surface of the book is opposite to the conveying direction, the back cover faces downwards, and the opposite side of the spine faces the right side of the conveying track, the book is considered as a left-turning book;
the book is considered to be right-turned when the direction of the characters on the upper surface of the book is opposite to the conveying direction, the back cover faces upwards, and the opposite side of the spine faces towards the right side of the conveying track.
In this embodiment, preferably, the calling the label printing template according to the text direction includes: if the character direction is the same as the conveying direction of the conveying mechanism, calling a first label printing template, wherein the character direction of the first label printing template is the same as the conveying direction; and if the character direction is opposite to the conveying direction of the conveying mechanism, calling a second label printing template, wherein the character direction of the second label printing template is opposite to the conveying direction. Preferably, a memory is provided for storing the first label printing template and the second label printing template.
In this embodiment, it is further preferable that the method further includes a step of obtaining an optimal labeling position, including:
step S101, an upper surface image is acquired. Preferably, in order to improve the acquisition accuracy and reduce interference images, the front image of the page to be labeled is shot by the camera, the upper surface image is segmented from the front image, and the upper surface image can be segmented from the front image by adopting the existing image segmentation method, such as the existing mask method. The top surface image is preferably, but not limited to, the back or front cover of the book.
Preferably, to improve processing efficiency, before step S102 is performed, a text region is obtained by performing text detection on the upper surface image, a graphic region is obtained by performing graphic detection, if a tag cannot be accommodated in a remaining region after the text region and the graphic region are subtracted from the upper surface image, the labeling sub-region is set at a non-first line text or a non-means text of the upper surface image, so that visual influence on a reader can be reduced as much as possible, and if at least one tag can be accommodated in the remaining region, step S102 is performed. The text detection can be realized by adopting the existing algorithm, such as the CTPN algorithm disclosed in the website https:// zhuanlan. zhihu.com/p/34757009, and the technical scheme disclosed in the Chinese patent with the publication number CN105205488A, CN103699895A or CN104050471A, which are not described herein again. The pattern detection is used for detecting various patterns and patterns such as lines, circles, ellipses, triangles and the like in the upper surface image, and the conventional pattern detection algorithm can be adopted, such as the technical scheme disclosed in the chinese patent with the publication number CN1264096A, and the conventional object detection algorithm based on the solov2 neural network, which are not described herein again. And judging whether the label can be accommodated in the residual area, if the length of one sub-area of the residual area is greater than the length of the label and the sum of the length and the width of the sub-area is greater than the width of the label, determining that the label can be accommodated, and otherwise, determining that the label cannot be accommodated.
Step S102, a labeling area is selected from the upper surface image, the labeling area comprises at least one labeling sub-area matched with the size of the label, the ratio of characters and figures in the labeling area is less than or equal to a ratio threshold, and the value range of the ratio threshold is 0-0.7.
In this embodiment, the selecting the labeling area in the upper surface image may specifically include: and acquiring at least one labeling subarea matched with the label in size in the to-be-labeled page area, wherein the ratio of characters to figures in the labeling subarea is less than or equal to a preset ratio threshold value, and the labeling subarea forms a labeling area. Setting a sliding window matched with the size of the label, wherein the sliding window slides on the image on the upper surface in sequence from left to right and from top to bottom, the sliding step can be set to be a label width of one half or one third or one fourth, and a sliding window area with the character and graphic ratio less than or equal to a ratio threshold value is selected as a labeling subarea. Further preferably, in order to reduce the occlusion of the characters and graphics by the label as much as possible, a sliding window area with the smallest character and graphics occupation ratio is selected as the target labeling subarea from all sliding window areas satisfying that the character and graphics occupation ratio is less than or equal to the occupation ratio threshold, for example, the sliding window area with the occupation ratio of 0 is selected as the target labeling subarea. The text detection and the graphic detection in the sliding window region can adopt the existing algorithm, which has been described in detail in the above description and will not be described in detail herein. And counting the sum of pixel points of the character area obtained by character detection and the graphic area obtained by graphic detection, and taking the ratio of the sum of the pixel points to the total number of the pixel points of the sliding window as the ratio of the characters to the images in the sliding window area.
In this embodiment, in order to improve efficiency and make the extracted labeling area more accurate and meet the requirement, preferably, the labeling area is obtained by an intelligent algorithm in step S102, that is, the labeling area is selected from the upper surface image, which specifically includes:
step S1021, acquiring an energy distribution map of the upper surface image, preferably but not limited to acquiring the energy distribution map through the existing image Fourier transform, such as the technical scheme disclosed in the website https:// blog.csdn.net/dengheccdn/article/details/78906126. The energy distribution map can be identified by the neural network model, and the characteristics that characters and graphs on a page image are highlighted, the energy of pixel points is high, the background of the page is not highlighted, and the energy of the pixel points is low are fully considered, so that the characters, the images and the background can be effectively distinguished through the energy distribution map, the neural network learning characteristic rule of the model in a labeling area can be acquired, and the labeling accuracy of the labeling area can be improved.
Step S1022, the energy distribution map is input into the pre-trained labeler region acquisition model.
In step S1023, the energy distribution map of the labeling area output by the model is obtained by the labeling area acquisition module.
Step S1024, mapping the labeling area mark in the energy distribution map of the labeling area to the upper surface image, and outputting the upper surface image of the labeling area. The sliding window of the labeling subarea matched with the size of the label can slide in the labeling area, and the area which can contain the sliding window of the labeling subarea in the labeling area is used as a labeling subarea for outputting.
In this embodiment, it is further preferable that, in order to improve the accuracy, stability and reliability of the model for acquiring the attachable region, the process for constructing the model for acquiring the attachable region includes:
step A1, obtaining a plurality of upper surface images, at least 2000 upper surface images, one upper surface image being a sample, marking a markable area on each sample, preferably but not limited to a marking frame, manually marking, using the upper surface image marking the markable area as a sample label of the original sample, marking a part of the sample with a correct markable area with a character and figure ratio less than or equal to a ratio threshold value, marking the sample with a positive sample label, marking a part of the sample with a wrong markable area with a wrong label with a character and figure ratio greater than the ratio threshold value, and marking the sample with a negative sample label;
step A2, dividing all samples into a training set, a verification set and a test set, wherein the training set comprises positive sample label samples and negative sample label samples, the verification set comprises positive sample label samples, and the test set comprises positive sample label samples;
step A3, constructing a neural network structure of the labeling area acquisition model, wherein the neural network structure is preferably but not limited to an HRNet network structure;
step A4, training a neural network structure by using a training set, verifying the trained neural network structure by using a verification set, and adjusting the parameters of the neural network structure according to a verification result to obtain a region obtaining model capable of labeling;
and step A5, testing the acquired model of the labeling area by using a test set, and verifying the performance of the model.
Step S103, outputting the position information of at least one labeling subarea on the page to be labeled. The position information of the labeling sub-area on the page to be labeled is preferably, but not limited to, the position coordinate of the center point of the labeling sub-area on the coordinate system of the page to be labeled, the coordinate system of the page to be labeled is the coordinate system x 'o' y 'taking the width of the book as the x' axis, the length of the book as the y 'axis, and the lower left corner in the image of the upper surface of the book as the origin o'. Further preferably, in order to facilitate the user's selection, the position information of at least one labeling sub-area on the page to be labeled can be output to the display, and when the position information is displayed, the shape and the position coordinates of the center point of each labeling sub-area are marked on the upper surface image.
In this embodiment, it is further preferable that, in order to reduce the occlusion of the characters and images by the sticker, the method further includes outputting the preferred weight of each sticker sub-area while outputting the position information of the sticker sub-area on the page to be pasted, where the preferred weight of each sticker sub-area is inversely related to the ratio of the characters and the images in the sticker sub-area, and the higher the ratio of the characters and the images in the sticker sub-area is, the lower the preferred weight of the sticker sub-area is. Each labeling subarea is assigned with a preferred weight, the preferred weight range is preferably but not limited to 0 to 1, and the larger the preferred weight is, the better the labeling subarea corresponding to the preferred weight is recommended to be selected.
In this embodiment, it is further preferable that, in order to improve the visual effect of the label and facilitate people to find the label quickly by observation, the method further includes outputting the optimal weight of each labeling sub-region while outputting the position information of the labeling sub-region on the page to be labeled, where the optimal weight of the labeling sub-region is positively correlated with the centering degree of the labeling sub-region in the width direction of the page to be labeled. The centering degree calculation process is as follows: setting the coordinate of the central point of the labeling sub-area as (x ', y '), in the coordinate system x ' o ' y ', the coordinate of the lower left corner of the page in the upper surface image is (0, 0), the coordinate of the lower right corner of the page in the upper surface image is (d, 0), and d is the width of the book, so that the centering degree of the labeling sub-area in the width direction of the page to be labeled is: the value of the degree of centering of the ratio obtained by comparing the minimum value of (d-x ') and x' with the maximum value is in the range of 0 to 1, and the closer to 1, the better the centering of the labeling subarea.
In this embodiment, preferably, in order to integrate the influence of the median of the labeling sub-area and the ratio P of the characters and the graphics in the labeling sub-area on the preferred weight, a (median-P) value is calculated, and a value obtained by normalizing the (median-P) value is used as the preferred weight of the labeling sub-area.
In this embodiment, in the labeling position selecting step, the ratio of the characters and the graphics in the selected labeling area is less than or equal to the ratio threshold, that is, the ratio of the total area occupied by the characters and the graphics in the labeling area to the total area of the labeling area is less than or equal to the ratio threshold, and the ratio threshold can be set according to the user requirements, so that the characters and the graphics on the page to be labeled can be reduced or prevented from being covered by the label, the page readability is improved, and the optimal labeling position can be automatically selected in a self-adaptive manner according to the distribution characteristics of the characters and the graphics on the page to be labeled.
Example 2
The embodiment discloses a pre-processing device for book collection and editing, which corresponds to the embodiment 1 and comprises: the image acquisition module is used for acquiring an image of the upper surface of the book on the conveying mechanism; the character direction identification module is used for identifying the character direction of the upper surface image; the book spine orientation acquisition module is used for acquiring the book spine orientation; and the left-right book turning judging module is used for judging the book turning direction of the book based on the character direction, the cover orientation and the spine orientation of the upper surface of the book, wherein the book turning direction comprises left book turning and right book turning.
Example 3
The embodiment discloses a book collecting and editing preprocessing system, which comprises: the first camera is positioned above the conveying mechanism and used for shooting the upper surface images of the books on the conveying mechanism; and the processor receives the output image of the first camera and executes the steps of the book editing preprocessing method in the embodiment 1.
In this embodiment, it is preferable that a second camera is further included, and the second camera is located on the right side or the left side of the conveying mechanism and is used for shooting a side image of the book on the conveying mechanism.
Example 4
The embodiment discloses a book gathering and editing system which comprises a control module, a conveying mechanism, a camera module and an executing mechanism, wherein the camera module and the executing mechanism are sequentially arranged on the conveying mechanism along the book conveying direction; the camera module is used for acquiring an image of the upper surface of the book on the conveying mechanism; the actuating mechanism comprises a left page turning mechanism, a left page turning mechanism driving device, a right page turning mechanism and a right page turning mechanism driving device; the control module is connected with the camera module, the left page turning mechanism driving device and the right page turning mechanism driving device respectively, obtains the book turning direction of the book according to the book gathering and editing preprocessing method in the embodiment 1, starts the left page turning mechanism and closes the right page turning mechanism when the book is left-handed, and starts the right page turning mechanism and closes the left page turning mechanism when the book is right-handed.
In this embodiment, the camera module preferably, but not limited to, includes a first camera and a second camera, and the first camera and the second camera may be disposed at the same position of the conveying mechanism, and preferably, both are disposed at the starting end of the conveying mechanism. A mounting bracket may be provided which is fixedly mounted above the conveying mechanism and connected to the left and right sides of the conveying mechanism. The first camera is installed above the mounting rack and attached to shoot the upper surface images of the books on the conveying mechanism downwards. The second camera is arranged on the left side or the right side of the conveying mechanism and used for shooting side images of the books.
In the present embodiment, the control module is preferably, but not limited to, a processing control module such as a PLC, a PC computer, an ARM, and an FPGA. The conveying mechanism is preferably, but not limited to, a roller conveying mechanism or a belt conveying mechanism, and the belt conveying mechanism may be the technical solution disclosed in chinese patent publication No. CN 112537515A.
In this embodiment, when the book is a left-turning book, the control module controls the driving device of the left-turning mechanism to start the left-turning mechanism, controls the driving device of the right-turning mechanism to close the right-turning mechanism, and when the book is a right-turning book, the control module controls the driving device of the right-turning mechanism to start the right-turning mechanism, and controls the driving device of the left-turning mechanism to close the left-turning mechanism. The left page turning mechanism and the right page turning mechanism are respectively a first left page turning device and a first right page turning device in chinese patent with publication number CN112537515A, and are not described herein again.
In this embodiment, preferably, the actuator further includes a marking device; the control module is connected with the marking device, obtains the character direction of the upper surface of the book according to the book picking and editing pretreatment method of embodiment 1, controls the marking machine to call a first label printing template to print a label if the character direction is the same as the conveying direction of the conveying mechanism, and controls the printing device to call a second label printing template to print a label if the character direction is opposite to the conveying direction of the conveying mechanism.
In this embodiment, the marking device may also be the first left-side labeling device or the first right-side labeling device disclosed in chinese patent publication No. CN112537515A, or the labeling device disclosed in chinese patent publication No. CN 213566881U.
In this embodiment, preferably, the actuator further includes a left page-turning mechanism and a right page-turning mechanism; the control module obtains the book page turning direction according to the book gathering and editing preprocessing method in embodiment 1, and when the book is a left page turning mechanism, the left page turning mechanism is started, and the right page turning mechanism is closed, and when the book is a right page turning mechanism, the right page turning mechanism is started, and the left page turning mechanism is closed.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
Claims (13)
1. A book collecting and editing preprocessing method is characterized by comprising the following steps:
acquiring an image of the upper surface of a book on a conveying mechanism;
identifying the character direction of the upper surface image;
acquiring the orientation of the front cover or the orientation of the back cover of the book, and acquiring the orientation of the spine of the book or the orientation of the opposite side of the spine of the book;
and judging the book turning direction based on the character direction of the upper surface of the book, the front cover direction or the back cover direction and the spine direction or the opposite side direction of the spine, wherein the book turning direction comprises left book turning and right book turning.
2. The book compilation preprocessing method of claim 1, wherein a plurality of triples are constructed, the triples including a text direction, a front cover orientation, and a spine orientation of the top surface of the book, or the triples including a text direction, a back cover orientation, and a spine orientation of the top surface of the book, or the triples including a text direction, a front cover orientation, and a spine opposite side orientation of the top surface of the book, or the triples including a text direction, a back cover orientation, and a spine opposite side orientation of the top surface of the book; and establishing a corresponding relation table of the triples and the book turning direction, and searching the corresponding relation table based on the character direction of the upper surface of the book, the front cover direction or the back cover direction, and the spine direction or the direction of the opposite side of the spine to obtain the book turning direction.
3. The book editing preprocessing method of claim 1 or 2, wherein calling a label printing template according to the text direction comprises: if the character direction is the same as the conveying direction of the conveying mechanism, calling a first label printing template, wherein the character direction of the first label printing template is the same as the conveying direction; and if the character direction is opposite to the conveying direction of the conveying mechanism, calling a second label printing template, wherein the character direction of the second label printing template is opposite to the conveying direction.
4. The book editing preprocessing method as claimed in claim 1 or 2, characterized in that the process of acquiring the book cover orientation is: and identifying the ISBN (international standard bn) number of the upper surface image, considering that the book cover faces downwards if the ISBN identification is successful, and considering that the book cover faces upwards if the ISBN identification is unsuccessful.
5. The pre-processing method for book cataloging of claim 3, wherein the process of obtaining the book cover orientation is: and identifying the ISBN (international standard bn) number of the upper surface image, considering that the book cover faces downwards if the ISBN identification is successful, and considering that the book cover faces upwards if the ISBN identification is unsuccessful.
6. The book editing preprocessing method of claim 4, further comprising a cover facing judgment step, specifically comprising: and when the orientation of the book cover is different from the orientation of the set cover, giving an alarm for prompting.
7. The book editing preprocessing method as claimed in claim 1, 2, 5 or 6, characterized in that the process of acquiring the book spine orientation is:
acquiring a side image of a book at the right side of the conveying track, judging whether the side image is a spine image, if so, regarding that the spine of the book faces the right side of the conveying track, otherwise, regarding that the spine of the book faces the left side of the conveying track;
or acquiring a side image of the book at the left side of the conveying track, judging whether the side image is a spine image, if the side image is the spine image, considering that the spine of the book faces the left side of the conveying track, and if not, considering that the spine of the book faces the right side of the conveying track.
8. The book editing preprocessing method of claim 3, wherein the process of obtaining the book spine orientation is as follows:
acquiring a side image of a book at the right side of the conveying track, judging whether the side image is a spine image, if so, regarding that the spine of the book faces the right side of the conveying track, otherwise, regarding that the spine of the book faces the left side of the conveying track;
or acquiring a side image of the book at the left side of the conveying track, judging whether the side image is a spine image, if so, regarding the spine of the book to face the left side of the conveying track, otherwise, regarding the spine of the book to face the right side of the conveying track.
9. The book editing preprocessing method of claim 4, wherein the process of obtaining the book spine orientation is as follows:
acquiring a side image of a book at the right side of the conveying track, judging whether the side image is a spine image, if so, regarding that the spine of the book faces the right side of the conveying track, otherwise, regarding that the spine of the book faces the left side of the conveying track;
or acquiring a side image of the book at the left side of the conveying track, judging whether the side image is a spine image, if the side image is the spine image, considering that the spine of the book faces the left side of the conveying track, and if not, considering that the spine of the book faces the right side of the conveying track.
10. The utility model provides a books adopt and compile preprocessing device which characterized in that includes:
the image acquisition module is used for acquiring an image of the upper surface of the book on the conveying mechanism;
the character direction identification module is used for identifying the character direction of the upper surface image;
the book cover orientation acquisition module is used for acquiring the book cover orientation or the back cover orientation;
the book spine orientation acquisition module is used for acquiring book spine orientation or book spine opposite side orientation;
and judging the book turning direction based on the character direction of the upper surface of the book, the front cover direction or the back cover direction and the spine direction or the opposite side direction of the spine, wherein the book turning direction comprises left book turning and right book turning.
11. A book collection and editing preprocessing system is characterized by comprising:
the first camera is positioned above the conveying mechanism and used for shooting images of the upper surface of the book on the conveying mechanism;
a processor receiving an output image of the first camera, performing the steps of the book compilation preprocessing method of any of claims 1-9.
12. A book gathering and editing system is characterized by comprising a control module, a conveying mechanism, a camera module and an executing mechanism, wherein the camera module and the executing mechanism are sequentially arranged on the conveying mechanism along the book conveying direction;
the camera module is used for acquiring an image of the upper surface of the book on the conveying mechanism;
the executing mechanism comprises a left page turning mechanism, a left page turning mechanism driving device, a right page turning mechanism and a right page turning mechanism driving device;
the control module is respectively connected with the camera module, the left page turning mechanism driving device and the right page turning mechanism driving device, the control module acquires the book turning direction according to the book gathering and editing preprocessing method of any one of claims 1 to 9, the left page turning mechanism is started and the right page turning mechanism is closed when the book is left-handed, and the right page turning mechanism is started and the left page turning mechanism is closed when the book is right-handed.
13. The book compilation system of claim 12, wherein the actuator further comprises a marking device;
the control module is connected with a marking device, the control module obtains the character direction of the upper surface of the book according to the book editing pretreatment method of any one of claims 1 to 9, if the character direction is the same as the conveying direction of the conveying mechanism, the control module controls the marking machine to call a first label printing template to print a label, and if the character direction is opposite to the conveying direction of the conveying mechanism, the control module controls the printing device to call a second label printing template to print a label.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210402294.5A CN114701264A (en) | 2022-04-15 | 2022-04-15 | Book collection and editing preprocessing method, device and system |
| CN202210793807.XA CN115157877A (en) | 2022-04-15 | 2022-07-07 | Book collection and editing preprocessing method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210402294.5A CN114701264A (en) | 2022-04-15 | 2022-04-15 | Book collection and editing preprocessing method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114701264A true CN114701264A (en) | 2022-07-05 |
Family
ID=82174101
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210402294.5A Pending CN114701264A (en) | 2022-04-15 | 2022-04-15 | Book collection and editing preprocessing method, device and system |
| CN202210793807.XA Pending CN115157877A (en) | 2022-04-15 | 2022-07-07 | Book collection and editing preprocessing method, device and system |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210793807.XA Pending CN115157877A (en) | 2022-04-15 | 2022-07-07 | Book collection and editing preprocessing method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN114701264A (en) |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2901288B2 (en) * | 1989-11-13 | 1999-06-07 | 富士工業株式会社 | Book direction detection device |
| JPH06183175A (en) * | 1992-02-27 | 1994-07-05 | Ricoh Co Ltd | Method and device for binding corresponding to correct binding position |
| JP3302761B2 (en) * | 1993-02-24 | 2002-07-15 | 株式会社リコー | Book Scanner |
| ITMI20112055A1 (en) * | 2011-11-11 | 2013-05-12 | Sitma Machinery S P A | APPARATUS AND METHOD TO CHECK AND MODIFY THE ORIENTATION OF PRODUCTS IN A BOOK FORMAT CARRIED IN A CONTINUOUS FLOW OF TRANSPORT |
| JP5981876B2 (en) * | 2013-04-24 | 2016-08-31 | 株式会社Pfu | Information processing apparatus, method, and program |
| CN205106817U (en) * | 2015-10-19 | 2016-03-30 | 曲金龙 | Automatic a page reading ware turns over |
| JP6688675B2 (en) * | 2016-05-16 | 2020-04-28 | シャープ株式会社 | Image processing apparatus and image forming apparatus |
| CN107944030A (en) * | 2017-12-13 | 2018-04-20 | 四川九鼎智远知识产权运营有限公司 | A kind of library automatic classification device |
| CN108921160B (en) * | 2018-05-04 | 2021-10-26 | 广东数相智能科技有限公司 | Book identification method, electronic equipment and storage medium |
| CN112109470A (en) * | 2020-09-29 | 2020-12-22 | 广州标控科技有限公司 | Page turning device |
| CN112537515A (en) * | 2020-09-29 | 2021-03-23 | 广州标控科技有限公司 | Book collecting and editing system |
-
2022
- 2022-04-15 CN CN202210402294.5A patent/CN114701264A/en active Pending
- 2022-07-07 CN CN202210793807.XA patent/CN115157877A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| CN115157877A (en) | 2022-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109829467A (en) | Image labeling method, electronic device and non-transient computer-readable storage medium | |
| US7054509B2 (en) | Determining form identification through the spatial relationship of input data | |
| US20150042791A1 (en) | Apparatus, systems and methods for enrollment of irregular shaped objects | |
| CN114919830B (en) | Book labeling position acquisition method, self-adaptive labeling method and system | |
| CN106383131A (en) | Printed matter visual detection method, device and system | |
| CN105512657B (en) | Character identifying method and equipment | |
| EP0820617A1 (en) | Method for locating the position and orientation of a fiduciary mark | |
| CN108388821B (en) | Automatic recognition device and method for express mail bar code | |
| CN114049624A (en) | Intelligent detection method and system for ship cabin based on machine vision | |
| CN113063802A (en) | Printed label defect detection method and device | |
| CN115797811A (en) | Agricultural product detection method and system based on vision | |
| CN109388935A (en) | Document verification method and device, electronic equipment and readable storage medium storing program for executing | |
| CN114701264A (en) | Book collection and editing preprocessing method, device and system | |
| CN112651353A (en) | Target mental calculation positioning and identifying method based on custom label | |
| CN109791597B (en) | Information processing apparatus, system, information processing method, and storage medium | |
| CN115905733A (en) | Mask wearing abnormity detection and trajectory tracking method based on machine vision | |
| CN109325557B (en) | Data intelligence acquisition method based on computer visual image identification | |
| CN111062375B (en) | Question number missing completion method based on context relation | |
| CN109977710B (en) | Method for accurately positioning bookshelf to which book belongs, electronic equipment and storage medium | |
| Araújo et al. | Segmenting and recognizing license plate characters | |
| JP3411796B2 (en) | Character recognition device | |
| JP6533021B1 (en) | Analog meter specification identification device, computer program and analog meter specification recognition method | |
| CN109359705A (en) | A kind of dislocation books search system and its lookup method | |
| EP4080515A1 (en) | A method for classifying an identification tag on a sample tube containing a sample and an automated laboratory system | |
| JP2024000958A (en) | Identification management support device, identification management support terminal, identification management support system device, identification management support method, program and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20220705 |