KR20170087424A - Method of building book database using image of book, method of recommendation using the same, and system of the same - Google Patents

Abstract

The database of the book and the recommendation method of the book are provided. The database construction method and the book recommendation method of the book may include the steps of: collecting a first image in which a first user has taken an image of a bookcase; collecting a second image in which a second user has taken an image of the bookcase; Extracting information on a title and a genre of a book included in the first image and the second image to collect first basic information and second basic information, extracting first and second basic information from the first and second users, The method comprising the steps of: receiving information on a time when a book included in the basic information is read and updating the first and second basic information with first and second book information, Extracting a matching book matching each other from the second book list of the second book information, and using the first book information and the second book information, And recommending the book to the second user by classifying the first book list into a forward book and a backward book at a point in time that is faster than the matching book.

Description

A method for building a database of books using images of books, a method for recommending books using the same, and a system thereof. {Method of building book database using image of book, method of recommendation using same, and system of the same}

The present invention relates to a method and system for building a database by extracting a plurality of book titles from a bookmark photograph, and more particularly, to a method and system for extracting a book title from a bookmark photograph stored in a user terminal .

Global IT companies such as Google and Baidu continue to use their artificial intelligence technology to extract meaningful data from people's everyday recordings. In particular, Google's recent unveiling of Google Photos for free is aimed at collecting large amounts of photos of users with such meaningful information.

Meaningful information is people's concern. It collects data on places visited, items owned, people met, etc., and accumulates insights to understand customers based on these data.

In this context, the bookcase photograph is the most explicit record of the customer 's intellectual taste and interests. In particular, taking pictures of bookshelves and extracting book titles from them to provide metadata can evolve into a great service that can meet the needs of customers who want to manage their book assets.

  The purpose of this technology has been reversed from this need. Customers who want to manage their book assets can build their own online library simply by taking a picture of the bookcase and sending it. This is a breakthrough in the User Experience, where about 30 to 50 books can be entered as a single picture.

SUMMARY OF THE INVENTION The present invention provides a method and system for building a database by extracting a plurality of book titles from a highly reliable bookmark photograph.

It is another object of the present invention to provide a method and system for building a database by extracting a plurality of book titles from bookcase photographs with a minimized error rate.

It is another object of the present invention to provide a method and system for building a database by extracting a plurality of book titles from a bookshelf picture of a simple algorithm.

The technical problem to be solved by the present invention is not limited to the above.

In order to solve the above technical problems, the present invention provides a method of establishing a database of books and recommending books.

According to one embodiment, the method of constructing a database of a book and recommending a book includes the steps of: collecting a first image of a first user, the first image of which the first user has taken an image of the bookcase; Collecting first basic information and second basic information by extracting information on a title and a genre of a book included in the first image and the second image, Receiving information on a time when a book included in the first and second basic information is read and updating the first and second basic information with first and second book information, A matching book matching the first book list of the first book information and the second book information of the second book information is extracted and the matching point of the first book information and the second book information are extracted, Degree To the slower forward books (forward book) and the read time of two minutes to the first list of books to the fast backward books (backward book) than the matching book, it can include like a book to the second user.

According to an embodiment, the step of extracting information on the title and the genre of the book from the first and second images may include extracting a plurality of candidate characters similar to the characters included in the first and second images, And selecting an authentic character from among the plurality of candidate characters, wherein the step of selecting the authentic character comprises the steps of: determining a dummy character and the authentic character from the plurality of candidate characters using an external database Step < / RTI >

In order to solve the above technical problems, the present invention provides a book recommendation system.

According to one embodiment, the book recommendation system receives a first image in which a first user has taken an image of a bookcase and a second image in which a second user has taken an image of a bookcase, 2 book information extracting unit for extracting information on a title and a genre of a book included in the image to generate first basic information and second basic information, a first information extracting unit for extracting, from the first and second users, A book information database for storing the first and second basic information as first and second book information and storing the updated first and second basic information, A matching book extracting unit that extracts matching books matching the first book information and the second book information of the second book information, a matching book extracting unit that extracts a matching book matching the first book information and the second book information, A recommendation book extracting unit for dividing the first book list into a forward book that is slower than the title book and a backward book that is faster than the matching book at a point in time when the user reads the forward book and the forward book, And a book information recommendation unit for recommending the backward book separately.

According to an aspect of the present invention, there is provided a method for building a database by extracting a plurality of book titles from a bookcase photograph.

According to one embodiment, a method of extracting a plurality of book titles from a bookcase photograph to construct a database comprises the steps of: acquiring an image of a bookcase; recognizing characters from the image; And supplementing the character by using the character string.

According to one embodiment, the step of recognizing the character from the image comprises extracting a plurality of candidate characters, wherein the step of supplementing the character comprises, using the external database, And determining the dummy character and the authenticity character from among the dummy character and the authenticity character.

According to an aspect of the present invention, there is provided a system including a database constructed by extracting a plurality of book titles from a bookcase photograph.

According to an embodiment, a system including a database constructed by extracting a plurality of book titles from the bookcase photograph includes an image capturing unit capturing an image of a bookcase, a character recognizing unit recognizing characters from the image of the bookcase, And a database constructed by extracting a plurality of book titles from the character supplementary bookcase photograph supplementing the extracted characters using an external database.

According to an embodiment of the present invention, a method and system for building a database by extracting a plurality of book titles from a highly reliable bookmark photograph with an error rate minimized can be provided.

FIG. 1 illustrates a method of building a database by extracting a plurality of book titles from a bookcase photograph according to an embodiment of the present invention, and a service using the system.
FIG. 2 is a diagram for explaining a method and system for building a database by extracting a plurality of book titles from a bookcase photograph according to an embodiment of the present invention.
3 is a diagram for explaining a book recommendation method according to an embodiment of the present invention.
4 is a view for explaining a book recommendation system according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the technical spirit of the present invention is not limited to the embodiments described herein but may be embodied in other forms. Rather, the embodiments disclosed herein are provided so that the disclosure can be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

Also, while the terms first, second, third, etc. in the various embodiments of the present disclosure are used to describe various components, these components should not be limited by these terms. These terms have only been used to distinguish one component from another. Thus, what is referred to as a first component in any one embodiment may be referred to as a second component in another embodiment. Each embodiment described and exemplified herein also includes its complementary embodiment. Also, in this specification, 'and / or' are used to include at least one of the front and rear components.

The singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. It is also to be understood that the terms such as " comprises "or" having "are intended to specify the presence of stated features, integers, Should not be understood to exclude the presence or addition of one or more other elements, elements, or combinations thereof. Also, in this specification, the term "connection " is used to include both indirectly connecting and directly connecting a plurality of components.

In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Photographs taken in everyday life contain many additional information. Especially, those who like books have taken their bookshelves or book images and shared them with people through social networks or online communities. Therefore, the concept of extracting object names in an image and converting them into metadata has been conceived.

FIG. 1 illustrates a method of building a database by extracting a plurality of book titles from a bookcase photograph according to an embodiment of the present invention, and a service using the system.

 Referring to FIG. 1, when a user takes a picture of a bookcase and uploads the picture to a server through an application, the user can confirm that his or her book list is automatically extracted and stored in the form of metadata in a few minutes to a few tens of minutes.

FIG. 2 is a diagram for explaining a method and system for building a database by extracting a plurality of book titles from a bookcase photograph according to an embodiment of the present invention.

Referring to FIG. 2, the present invention provides a method for extracting a plurality of objects from a plurality of objects including a plurality of objects, each of which includes a plurality of objects having object names in the form of a string, through a method of edge detection, skew correction, And a method and system for extracting title of a book.

In order to achieve the object of the present invention, the present invention provides a device or module for building a database through character recognition and object name extraction from a plurality of object photographs, a terminal device for capturing or inputting a photograph including a plurality of objects to be converted into a database, .

In order to achieve the object of the present invention, there is provided a method of controlling a camera, comprising: capturing or inputting a photograph including a plurality of objects in which the object name is exposed; Detecting an object edge and masking an image on an object-by-object basis to segment the object from the received photographic data; Correcting the inclination in the character direction to optimize the character string recognition included in the segmented object image fragments; Generating a candidate group of converted characters through optical character recognition to convert the character images of the object images with the tilt corrected into character data; And estimating an optimal object name through a similarity matching of a database holding an object name and each character candidate of the character string.

1. Edge detection method of item

Using the difference operator method which is used only for subtraction of four directions among the edge detection method by determining the maximum value by subtracting a series of pixels from the captured image

2. Tilting correction

In order to improve the character recognition accuracy of each item, it is a method of correcting the tilting in the normal direction of the character. By defining the reference line in the vertical direction and defining the line parallel to the long axis of the item edge, The process is as follows.

A. Define a vertical reference line G.

B. Specify the unit item Ix.

C. Define the line Lx parallel to the longest edge of Ix.

D. Calculate the angle t of G and Lx.

E. Reverse image by angle t. (rotated by -t)

3. Object masking method

It is difficult to determine the mask area because edge erasure or edge redundancy (due to edge detection within the item) occurs only with the edge detection method according to the state of the original image. Therefore, when determining the mask boundary, the minimum square including all four edges is determined as a mask. The process of determining the mask boundary is as follows.

Ya, the maximum y value of the left vertical edge Yb, and the maximum y value of the right vertical edge, respectively, are calculated as follows: max (Ya, Yb, Yc)

Left: Calculate x (Xa, Xb, Xc) by defining the x value Xa of the left vertical edge, the minimum x value Xb of the upper horizontal edge, and the minimum x value Xc of the lower horizontal edge, respectively.

(Xa, Xb, Xc) by defining the x value Xa of the right vertical edge, the maximum x value Xb of the upper horizontal edge, and the maximum x value Xc of the lower horizontal edge,

Bottom: Calculate y (Ya, Yb, Yc) by defining the y value Ya of the horizontal edge, the minimum y value Yb of the left vertical edge, and the minimum y value Yc of the right vertical edge.

4. Determining Item Title based on DB Matching

 As a method for converting a string existing in an image into metadata, a recognition candidate group is generated by a general character recognition method and a method of estimating an optimum object name by matching with an object name database

A. Generate a character recognition candidate group, and assign a confidence value to each character unit.

B. Generate a primary string candidate by setting the priority according to each confidence value, but excluding the character that has already been executed, and replacing it with the character having the next consecutive value. If there are no more candidates to replace, terminate.

C. The primary string candidates are matched and retrieved with the object name database, and the match with the search result (e.g., the number of matching characters) is recorded.

D. If the matching degree in C is equal to the number of string lengths of candidate correct answers, it ends.

E. If the matching degree in C is smaller than the number of string lengths of candidate correct answers, it returns to B.

F. Select and return the object name with the highest matching degree as the correct answer.

FIG. 3 is a view for explaining a book recommendation method according to an embodiment of the present invention, and FIG. 4 is a view for explaining a book recommendation system according to an embodiment of the present invention.

3 and 4, the book information extracting unit 302 of the book recommendation system 300 receives the first image IM_1 in which the first user 100 has photographed the image of the bookcase, (200) may receive the second image (IM_2) from which the image of the bookcase is captured.

The book information extracting unit 302 extracts information on a title and a genre of a book included in the first image IM_1 and the second image IM_2 and stores the first basic information FI_1 and the second basic information FI2, Information FI_2 can be generated. According to an embodiment, the step of extracting information on the title and genre of the book from the first and second images IM_1 and IM2 may include extracting information on the title and genre of the book from the first and second images IM_1 and IM_2, Extracting a plurality of similar candidate characters that are similar, and selecting a true character from among the plurality of candidate characters. In addition, the step of selecting the authenticity character may include discriminating the dummy character and the authenticity character from the plurality of candidate characters using the external database 304. [

The book information database 310 stores information on the time when the book included in the first and second basic information FI_1 and FI_2 is read from the first and second users 100 and 200 And can update and store the first and second basic information FI_1 and FI_2 as the first and second book information BI_1 and BI_2, respectively.

The first book information BI_ 1 may include information on the first book list 110 to 130, MB, and 150 to 180 read by the first user 100. According to one embodiment, the first book information BI_ 1 includes a genre of a book of the first book list 110 to 130, MB, 150 to 180, a first book list 110 to 130, 180 of the first user 100 and information on the author of the book of the first book list 110-130, MB, 150-180.

The second book information BI_2 may include information on the second book list 210, the MB 220, and the second book list 220 read by the second user 200. According to one embodiment, the second book information BI_2 may be a genre of the book of the second book list 210, the MB 220, the book of the second book list 210, the MB 220, And may further include information on the time when the user 200 has read and information on the author of the book of the second book list 210, MB, and 220.

The number of books of the first book list 110-180 of the first user 100 may be greater than the number of the second book lists 210,140 and 220 of the second user 200. [ For example, the first user 100 may be a love writer.

The matching book extracting unit 320 may extract a matching book MB using the first book information BI_1 and the second book information BI_2 stored in the book information database 310. [ The matching book MB may be a book matched with the first book list 110 to 130 MB, 150 to 180 and the second book list 210, MB, and 220.

When the matching book (MB) is extracted, the recommendation book extracting unit 320 extracts the matching book BM from the forward book (FB) and the back book (BM) by using the information about the time when the first user The first book list (110 ~ 130, MB, 150 ~ 180) can be divided into the word book (BB).

Specifically, the forward book FB may be a book that is slow in reading the first book 100 from the first book list 110-130, MB, 150-180, have. That is, the forward book FB may be a book that the first user 100 has read after reading the matching book (MB). In FIG. 7, reference numerals 150, 160, 170, and 180 in the first book list 110 to 130, MB, and 150 to 180 may correspond to the forward book FB.

In addition, the backward book (BB) may be a book that is faster than the first user 100 read from the matching book (MB) in the first book list (110 to 130, MB, 150 to 180) have. That is, the backward book (BB) may be a book that the first user (100) has read before reading the matching book (MB). In FIG. 7, reference numerals 110, 120, and 130 in the first book list (110 to 130, MB, 150 to 180) correspond to the backward book (BB).

The book information recommendation unit 330 may recommend the backward book (BB) and the forward book (FB) to the second user (200).

According to one modification, the book information recommendation unit 330 selects books 110 and 120 having genre matching with the matching book (MB) among the forward books (FB) And recommends to the second user 200 the books 150, 170, and 180 having the same genre as the matching book (MB) among the backward books (BB). Accordingly, the second user 200 can acquire the book of the same genre as the matching book (MB) from the first book list 110-130, MB, 150-180 read by the first user 100, Can be recommended.

Depending on the order in which the books are read, the feelings of the readers of the books may vary. That is, depending on whether the first book is read, the second book is read, or the second book is read, the feeling of the reader may be amplified or halved.

In the case of a book writer who is reading a lottery, even if there are several books that he wants to read, he does not read it, but reads the book in order considering the subject, author, etc. of the book. As a result, we have the opportunity to deepen the meaning of the book and amplify the feeling of feeling from the book.

As described above, according to the embodiment of the present invention, when it is confirmed that the second user 200, which is the general reader, has read the same matching book (MB) as the first user 100, 1) the first book list (110 ~ 130, MB, 150~180) read by the user (100), the previous book book (BB) read before the reading of the matching book (MB) (MB), and the forward book (FB) read after reading. Accordingly, it is possible to easily imitate the reading pattern of the first user 100, which is the master, and the quality of reading of the second user 200 can be improved.

In addition, although two users are shown in FIGS. 7 and 8, it is apparent to those skilled in the art that three or more users can use the book recommendation method and book recommendation system according to the embodiment of the present invention.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the scope of the present invention is not limited to the disclosed exemplary embodiments. It will also be appreciated that many modifications and variations will be apparent to those skilled in the art without departing from the scope of the invention.

Claims (3)

Collecting a first image of a first user having taken an image of a bookcase;
Collecting a second image in which a second user has taken an image of a bookcase;
Extracting information on titles and genres of books included in the first image and the second image to collect first basic information and second basic information;
Receiving information on the time when the first and second users read the book included in the first and second basic information, and updating the first and second basic information with the first and second book information, respectively step; And
Extracting a matching book that matches the first book list of the first book information and the second book list of the second book information, and using the first book information and the second book information, A step of discriminating the first book list with a forward book that is slower than the matching book and a backward book that is faster than the matching book at a point in time of reading and recommending the book to the second user To build a database of books and to recommend books.
The method according to claim 1,
Wherein the step of extracting information on a title and a genre of the book from the first and second images comprises:
Extracting a plurality of candidate characters similar to the characters included in the first and second images; And
Selecting an authentic character from among the plurality of candidate characters,
The step of selecting the authenticity character comprises:
Determining a dummy character and the authenticity character from the plurality of candidate characters using an external database; and constructing a database of the book and recommending the book.
A first user who receives a first image of an image of a bookcase and a second image of which a second user has taken an image of a bookcase and receives a title and a genre of the book included in the first image and the second image, A book information extracting unit for extracting information and generating first basic information and second basic information;
Information on the time when the first and second users read the book included in the first and second basic information, and updates the first and second basic information with the first and second book information, respectively A book information database to store;
A matching book extracting unit that extracts a matching book matching the first book list of the first book information and the second book list of the second book information;
The first book information and the second book information are used as a forward book which is slower than the matching book and a backward book which is read faster than the matching book, A recommended book extracting unit for dividing the list; And
And a book information recommendation unit for recommending the forward user and the backward book separately to the second user.

Images