CN114463886A

CN114463886A - Intelligent bookshelf and book management method

Info

Publication number: CN114463886A
Application number: CN202210110528.9A
Authority: CN
Inventors: 刘鹏; 刘石勇; 许丽星; 王昕�; 于仲海; 王凯欣
Original assignee: Hisense Group Holding Co Ltd
Current assignee: Hisense Group Holding Co Ltd
Priority date: 2022-01-29
Filing date: 2022-01-29
Publication date: 2022-05-10
Anticipated expiration: 2042-01-29
Also published as: CN114463886B

Abstract

The application discloses an intelligent bookshelf and a book management method, and belongs to the technical field of electronics. The intelligent bookshelf comprises: the bookshelf body is used for storing books; the camera is used for collecting images of target characters for storing and taking books on the bookshelf body; the controller is used for judging whether any book belongs to the readable books or not based on the information of the readable books corresponding to the reference person after the characteristic similarity between the target person and the reference person is determined to be larger than a first threshold value based on the image and the target person is determined to take out any book from the bookshelf body; and the information output component is used for outputting information for prompting that the target person puts back any book when the any book does not belong to the readable book. The bookshelf solves the problem that the functions of the bookshelf are single. The application is used for storing and managing books.

Description

Intelligent bookshelf and book management method

Technical Field

The application relates to the technical field of electronics, in particular to an intelligent bookshelf and a book management method.

Background

Along with the improvement of the life quality of people, the demand on life convenience is higher and higher, and equipment with rich functions is more favored by people. At present, books are usually placed on a bookshelf in family life, but the bookshelf only has a storage function. However, the existing bookshelf has a single function.

Disclosure of Invention

The application provides an intelligent bookshelf and a book management method, which can solve the problem that the function of the bookshelf is single. The technical scheme is as follows:

in one aspect, an intelligent bookshelf is provided, which includes:

the bookshelf body is used for storing books;

the camera is used for collecting images of target people for storing and taking books on the bookshelf body;

the controller is used for judging whether any book belongs to the readable books or not based on the information of the readable books corresponding to the reference person after determining that the feature similarity between the target person and the reference person is larger than a first threshold value and determining that the target person takes out any book from the bookshelf body;

and the information output component is used for outputting information for prompting that the target person puts back the book when the book does not belong to the readable book.

In another aspect, there is provided a book management method for an intelligent bookshelf, the intelligent bookshelf including: the bookshelf comprises a bookshelf body for storing books, a camera, a controller and an information output assembly, and the method comprises the following steps:

the camera collects images of target people for storing and taking books on the bookshelf body;

the controller determines whether any book belongs to the readable books based on the information of the readable books corresponding to the reference person after determining that the feature similarity between the target person and the reference person is larger than a first threshold based on the image and determining that the target person takes out any book from the bookshelf body;

and the information output component outputs information for prompting the target person to replace the book when the book does not belong to the readable book.

The beneficial effect that technical scheme that this application provided brought includes at least:

in the application, the intelligent bookshelf can determine the feature similarity of the target person and the reference person based on the image acquired by the target person. When the feature similarity is larger than a first threshold, whether the book taken out of the intelligent bookshelf by the target person belongs to the readable book corresponding to the reference person can be judged. If the book taken out by the target person does not belong to the readable book, the intelligent bookshelf can prompt the target person to put back the book. If the target person can be a child, the child can be prevented from reading books which are not suitable for reading, the functions of the bookshelf are enriched, and the using effect of the bookshelf is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic structural diagram of an intelligent bookshelf provided by an embodiment of the present application;

fig. 2 is a schematic structural diagram of a book management system according to an embodiment of the present application;

fig. 3 is a flowchart of a book management method according to an embodiment of the present application;

fig. 4 is a flowchart of another book management method provided in an embodiment of the present application;

FIG. 5 is a schematic diagram of a display interface provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of another display interface provided by an embodiment of the application;

FIG. 7 is a schematic diagram of yet another display interface provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of yet another display interface provided by an embodiment of the present application;

FIG. 9 is a schematic view of a display interface provided by another embodiment of the present application;

fig. 10 is a schematic diagram of another display interface provided in another embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

People can put books in order through the bookshelf in family life. At present, most of bookshelves only have the storage function and are only used for storing books, and the functions of the bookshelves are single. And at present, various books stored in the bookshelf can be randomly accessed by a user. Thus, when a child chooses to read books, the child may read adult books such as novels, violence, confusion or entertainment magazines with curiosity, and the reading of the books may affect the physical and mental health of the child and the academic performance of the child. However, in the process of learning and reading of children in the family environment, parents can hardly accompany the children at all times, and the parents cannot help the children to select reading of books. Therefore, the management effect on the books in the bookshelf is poor at present.

The embodiment of the application provides an intelligent bookshelf and a book management method, functions of the bookshelf can be enriched, parents can be helped to check the reading of books of children, and intelligent application of the bookshelf is optimized.

Fig. 1 is a schematic structural diagram of an intelligent bookshelf provided by an embodiment of the application. As shown in fig. 1, the intelligent bookshelf 10 may include a bookshelf body 101, a camera 102, a controller (not shown in the figure), and an information output component 103. The information output component 103 may include at least one of a display screen and a speaker. The bookshelf body 101 is used for putting the thing, and the bookshelf body 101 can include a plurality of thing check of putting, and books can be placed in each thing check of putting. Each storage compartment has an opening.

Optionally, the intelligent bookshelf 10 may further comprise a door 104, an information input component (not shown), and a memory (not shown). The door 104 is used for shielding the opening of the bookshelf body 101, and the opening of the bookshelf body 101 is also the opening of each storage compartment in the bookshelf body 101. The information input component may include at least one of a display screen, a barcode scanner, and a sound collector (e.g., a microphone). The display screen can be used both for inputting and outputting information. The camera 102 may also serve as an information input component for inputting the captured images. The memory may be used to store information input by the information input component and information processed by the controller.

The controller may include a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or a combination of a CPU and a GPU. The processor may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof.

The memory is connected to the controller via a bus or other means, and at least one instruction, at least one program, a code set, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by the controller to implement the method provided by the following embodiments of the present application. The memory may be a volatile memory (or a nonvolatile memory), a non-volatile memory (or a combination thereof). The volatile memory may be a random-access memory (RAM), such as a Static Random Access Memory (SRAM) or a Dynamic Random Access Memory (DRAM). The nonvolatile memory may be a Read Only Memory (ROM), such as a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), and an electrically erasable programmable read-only memory (EEPROM). The non-volatile memory may also be a flash memory, a magnetic memory, such as a magnetic tape, a floppy disk, or a hard disk. The non-volatile memory may also be an optical disc.

In the embodiment of the present application, the controller may be connected to various components in the intelligent bookshelf 10, such as the camera 102, the information output component 103, the door 104 and the information input component, so as to control the various components accordingly. For example, the controller may control whether the camera 102 acquires an image, or may acquire an image acquired by the camera 102. The controller can also acquire information input by the information input component, such as acquiring character information or image information input on a display screen, acquiring sound information acquired by the sound collector, and acquiring information of a bar code or a two-dimensional code scanned by the bar code scanner. The controller can control the information output component 103 to output corresponding information, such as controlling a display screen to display text or images, and controlling a speaker to output sound. The controller may also control the door 104 to lock or unlock. Alternatively, the controller may be located inside the bookshelf body 101, such as being wrapped by the bookshelf body 101. The controller may also be disposed outside the bookshelf body 101, for example, in other devices, only to ensure that the controller can be in communication connection with each component in the intelligent bookshelf 10, and can control each component.

The camera 102 may be located at the top of the bookshelf body 101, or may be located at the front or side of the bookshelf body 101, where the front is the side of the opening of the bookshelf body 101. Alternatively, the camera 102 may be located outside the bookshelf body 101, for example, fixed on a wall around the bookshelf body 101. Only the camera 102 needs to acquire an image at the opening of the bookshelf body 101, for example, an image (e.g., a face image) of a person of a book is accessed on the bookshelf body 101.

The information output component 103 and at least a part of the information input component may be located outside the bookshelf body 101, such as on the outer surface of the bookshelf body 101, partially embedded in the bookshelf body 101, or located on an object outside the bookshelf body 101. The display screen may be located at the side of the bookshelf body 101 as shown in fig. 1, and the display screen may also be arranged on the door 104. The speaker and the sound collector may be located on the front or side of the bookshelf body 101, or may be located on a wall around the bookshelf body 101. Alternatively, the camera 102, the speaker and the sound collector may be integrated into a sensing module, and are commonly disposed on the top of the bookshelf body 101.

The door 104 may be movably connected to the bookshelf body 101, and is used for shielding the opening of the bookshelf body 101. Fig. 1 illustrates that the door 104 is a vertical hinged door as an example, and the door 104 may also be a sliding door, which is not limited in the embodiment of the present application. Alternatively, the door 104 may be a door made of a transparent material, or may be a door made of an opaque material. In the embodiment of the application, the whole bookshelf body 101 is shown as an example of covering the opening through two doors 104, and optionally, the intelligent bookshelf 10 may also include only one door 104; or the number of the doors 104 in the intelligent bookshelf 10 may also be more than 2, for example, each storage compartment or at least two adjacent storage compartments may correspond to one door.

The intelligent bookshelf 10 can further comprise a locking assembly, which is located between the bookshelf body 101 and the door 104. The door 104 and the bookshelf body 101 are locked by the locking assembly, so that books stored in the bookshelf body 101 are prevented from being randomly accessed. In the embodiment of the present application, the controller is connected to the door 104 to control the locking or unlocking of the door 104, and actually, the controller is connected to the locking component to control the locking component to lock or unlock the door 104. Alternatively, when the intelligent bookshelf 10 comprises a plurality of doors, the controller may control the locking state of each door respectively. For example, the controller may only control a portion of the doors to unlock and still control a portion of the doors to remain locked.

Alternatively, the intelligent bookshelf in the embodiment of the present application may be a bookshelf operating based on Radio Frequency Identification (RFID) technology, and such an intelligent bookshelf may be referred to as an RFID bookshelf. The RFID shelf includes a receiver (not shown) that works in conjunction with an electronic tag (e.g., RFID tag). The electronic tag may emit a signal, such as a broadcast signal, and the receiver may receive the signal emitted by the electronic tag. Books stored in the intelligent bookshelf can carry electronic tags. The electronic tag can be attached to a book or clamped in the book. The electronic tag may leave the factory together with the book or may be set by the user at a later stage, which is not limited in the embodiment of the present application. Each electronic tag may store therein tag information of the electronic tag, where the tag information of the electronic tag at least includes an identifier of the electronic tag. Optionally, the user may also enter other information into the electronic tag, where the tag information of the electronic tag may also include the other information, for example, the other information may include information of a book set by the electronic tag. Optionally, the signal sent by the electronic tag may carry tag information thereof.

The smart bookshelf 10 may include at least one receiver, and each receiver may be embedded inside the bookshelf body 101 or may be attached to the surface of the bookshelf body 101. The at least one receiver may collectively receive signals from the electronic tags in the bookshelf body 101. Optionally, the receiving strength of the signal sent by the receiver to the electronic tag in the bookshelf body 101 may be higher than the receiving strength of the signal sent by the electronic tag outside the bookshelf body 101. Therefore, the position change of the electronic tag can be determined based on the change of the receiving intensity of the electronic tag by the receiver. For example, an intensity threshold may be set, and when the receiving intensity of a signal sent by a certain electronic tag changes from being higher than the intensity threshold to being lower than the intensity threshold, the electronic tag is considered to move from the bookshelf body to the outside of the bookshelf body. Accordingly, when the receiving intensity of the signal sent by the receiver to a certain electronic tag changes from being lower than the intensity threshold value to being higher than the intensity threshold value, the electronic tag is considered to move from the outside of the bookshelf body to the inside of the bookshelf body.

In a first alternative, the outer surface of the bookshelf body 101 and the door 104 are made of materials capable of blocking signal transmission, and the receiver is located in the space enclosed by the outer surface of the bookshelf body 101 and the door 104. For example, the receiver may be embedded in a partition of the bookshelf body 101, or attached to any storage cell of the bookshelf body 101. Due to the signal blocking effect of the outer surface and the door 104, the receiver can be ensured to have higher receiving strength for the signals sent by the electronic tags in the bookshelf body 101 and lower receiving strength for the signals sent by the electronic tags outside the bookshelf body 101.

In a second optional manner, the intelligent bookshelf 10 includes a plurality of receivers distributed at different positions of the bookshelf body 101, and each receiver corresponds to at least one storage cell in the bookshelf body 101. The receiver can have a small signal receiving range, for example, the receiver can only receive signals sent by the electronic tag with the maximum distance of 20-40 cm from the receiver, and the range covers the corresponding storage grid of the receiver. The controller may integrate the receiving strength of the signals sent by the plurality of receivers to the electronic tag to determine whether the electronic tag is located in the bookshelf body 101.

For convenience of description, the receiving strength of the signal sent by the electronic tag by the receiver is simply referred to as: and the receiver receives the signal strength of the electronic tag. For example, when the signal receiving intensity of any receiver of the plurality of receivers for a certain electronic tag is higher than the intensity threshold, and the signal receiving intensity of each receiver for the electronic tag is lower than the intensity threshold, the controller may determine that the electronic tag moves out of the bookshelf body 101. Accordingly, when the signal receiving intensity of each of the plurality of receivers for a certain electronic tag is lower than the intensity threshold, and the signal receiving intensity of any one of the receivers for the electronic tag is higher than the intensity threshold, the controller determines that the electronic tag moves into the bookshelf body 101. Optionally, the controller may also determine the specific position of the electronic tag in the bookshelf body 101. For example, the controller may determine the storage cell corresponding to the receiver having the highest signal reception intensity for the electronic tag and higher than the intensity threshold as the storage cell in which the electronic tag is located.

Alternatively, the controller may store the information of the electronic tag located in the bookshelf body 101, the information of the electronic tag removed from the bookshelf body 101, and the information of the storage cell in which the electronic tag is located in the bookshelf body 101 in the memory. Optionally, the intelligent bookshelf 10 may further include a communication unit (not shown in the figure), and the communication unit may be in communication connection with a device other than the intelligent bookshelf 10. The controller is connected with the communication unit, and the controller can perform information interaction with other equipment through the communication unit.

Fig. 2 is a schematic structural diagram of a book management system according to an embodiment of the present application. As shown in fig. 2, the book management system may include an intelligent bookshelf 10, a server 20, and a terminal 30. The intelligent bookshelf 10 is the intelligent bookshelf shown in fig. 1. The terminal 30 may be a smart phone, a tablet computer, a notebook computer, or other devices with display, information processing, and information interaction functions. The terminal 30 may be in communication connection with the intelligent bookshelf 10 to perform information interaction with the intelligent bookshelf 10. For example, the terminal 30 may manage information of the book in the intelligent bookshelf 10, so that the user can know the information of the book in the intelligent bookshelf 10 through the terminal 30. The terminal 30 may also control the intelligent bookshelf 10.

Both the terminal 30 and the intelligent bookshelf 10 can be in communication connection with the server 20, so that the communication connection between the terminal 30 and the intelligent bookshelf 10 is realized through the server 20. The server 20 may also support information processing or storage of the intelligent bookshelves 10 and the terminal 30. The server 20 may be an edge server deployed in a home lan or may also be a cloud server. The terminal 30 and the intelligent bookshelf 10, and the terminal 30 and the server 20 may be connected through a wired network or a wireless network. Wired networks may include, but are not limited to: universal Serial Bus (USB), wireless networks may include, but are not limited to: wireless Fidelity (WIFI for short), bluetooth, infrared, Zigbee, data network, etc.

Illustratively, a bookshelf management application may be installed in the terminal 30. The terminal 30 may start the bookshelf management application to bind the corresponding intelligent bookshelf 10 through the application, and then manage the intelligent bookshelf 10 through the application. The user may control the intelligent bookshelf 10 to perform certain operations through the terminal 30, and may also obtain information obtained by the intelligent bookshelf 10 or information stored in the intelligent bookshelf 10. For example, a user may remotely obtain an image taken by the camera 102 in the intelligent bookshelf 10 through the terminal 30 to monitor the area where the intelligent bookshelf 10 is located. The user can also remotely control the speaker in the intelligent bookshelf 10 to output corresponding voice through the terminal 30. When the intelligent bookshelf 10 determines that a certain book is taken out or stored, it may send a prompt message indicating the access state of the book to the terminal 30.

Fig. 3 is a flowchart of a book management method provided in an embodiment of the present application, which may be used for the intelligent bookshelf 10 in fig. 1 or fig. 2. As shown in fig. 3, the method may include:

step 301, collecting images of target people accessing books on the bookshelf body.

Wherein step 301 is executed by a camera in the intelligent bookshelf. For example, the camera can continuously collect images in the image collection range of the bookshelf, and the image collection range comprises a certain range in front of the bookshelf body. When a person accesses a book on the bookshelf body, the image collected by the camera can comprise an image of a target person accessing the book. Alternatively, the camera may capture the image under the control of the controller, or the camera may stop capturing the image under the control of the controller.

Step 302, after it is determined that the feature similarity between the target person and the reference person is greater than the first threshold based on the image and it is determined that the target person takes out any book from the bookshelf body, it is determined whether the any book belongs to the readable book based on the information of the readable book corresponding to the reference person.

Wherein step 302 is executed by a controller in the intelligent bookshelf. The controller can analyze the image of the target person acquired by the camera to extract the characteristics of the target person in the image, and then compare the characteristics with the characteristics of the reference person to determine the characteristic similarity of the target person and the reference person. When the feature similarity is greater than the first threshold, the controller may determine that the target person is the reference person. Alternatively, the reference character may be characterized by information stored in a memory of the intelligent bookshelf, or information acquired by the controller from a server or other location through the communication unit.

The reference character may be preset by the user. Optionally, after setting the reference person, the user may set a corresponding readable book for the reference person to control a reading range of the book of the reference person. If the user may be a parent and the reference character may be a child, the readable book may include a book in a reading guidance list for the student issued by the education department. The controller may determine information of the book taken out by the target person based on the signal reception strength of the electronic tag carried by the book by the receiver. After determining that the target person is the reference person based on the feature similarity, the controller may compare the book taken out by the target person with the readable book corresponding to the reference person, and determine whether the book taken out by the target person belongs to the readable book corresponding to the reference person.

And step 303, outputting information prompting the target person to put back the book when the book does not belong to the readable book.

In this case, step 303 is performed by an information output component in the intelligent bookshelf, for example, the information output component may output a prompt message under the control of the controller. For example, a voice prompting the target person to replace the book may be output from a speaker, or a message in the form of a text may be output from a display screen.

In summary, in the book management method provided by the embodiment of the application, the intelligent bookshelf may determine the feature similarity between the target person and the reference person based on the image of the target person acquired by the intelligent bookshelf. When the feature similarity is larger than a first threshold, whether the book taken out of the intelligent bookshelf by the target person belongs to the readable book corresponding to the reference person can be judged. If the book taken out by the target person does not belong to the readable book, the intelligent bookshelf can prompt the target person to put back the book. If the target person can be a child, the child can be prevented from reading books which are not suitable for reading, the functions of the bookshelf are enriched, and the using effect of the bookshelf is improved.

Fig. 4 is a flowchart of another book management method provided in an embodiment of the present application, where the method may be used in the intelligent bookshelf in fig. 1 or fig. 2. As shown in fig. 4, the method may include:

step 401, obtaining facial features and voice features of the reference person and information of the readable book.

In the embodiment of the application, a user can set each family member as a reference character for the intelligent bookshelf, and input information of the reference character to the intelligent bookshelf, so that the intelligent bookshelf records the information of each reference character, and performs corresponding operation based on the recorded information. The information of the reference character may include identification, characteristics, and information of the readable book of the reference character. The feature of the reference character may include at least one of a facial feature and a voice feature. The information of the readable book may include at least one of a name and a type of the readable book. It should be noted that, in the embodiment of the present application, the user inputs both the facial feature and the voice feature of the reference person into the intelligent bookshelf as an example. Optionally, only one of the facial features and the reference features may be input into the intelligent bookshelf, which is not limited in the embodiment of the present application.

Fig. 5 is a schematic diagram of a display interface provided in an embodiment of the present application, fig. 6 is a schematic diagram of another display interface provided in the embodiment of the present application, and fig. 7 is a schematic diagram of another display interface provided in the embodiment of the present application. The process of inputting information of the reference character will be described with reference to fig. 5 to 7. For any reference person to be recorded, the user can set an identifier for the reference person. As shown in fig. 5, identity information can be selected as the identification of the reference person from identity information such as children, students, dad, mom, grandpa, and breast milk. Alternatively, the name or other information of the family member may be used as the reference person identifier.

In one aspect, a user may upload an image of a reference character to the smart bookshelf for the smart bookshelf to extract features of the reference character based on the image. If the image includes a facial image of the reference person, the controller of the intelligent bookshelf may extract facial features of the reference person based on the image. Multiple images can be uploaded for each reference person to ensure that more comprehensive characteristics are extracted. As shown in fig. 6, the image of the reference person may be selected from among the plurality of stored images. Optionally, the user may also position the reference character in front of the camera of the intelligent bookshelf, so that the camera can instantly capture the image of the reference character.

For example, the controller may perform data dimension reduction processing on the face image of the reference person to obtain a one-dimensional face feature vector of the reference person, where the one-dimensional face feature vector is a face feature of the reference person. The face image of the reference character may be an RGB image, and the face image may be an h × k × 3 dimensional image. The digital image data may be represented by a matrix, h representing the number of rows of pixels in the face image matrix, and k representing the number of columns of pixels in the face image matrix. The controller may input the face image of the reference person into the face feature extraction model. The face feature extraction model can process the face image through technologies such as a convolution neural network or a full-connection neural network, and the like, and dimension reduction is carried out on the face image to obtain a one-dimensional face feature vector of a reference person:

the user can trigger the intelligent bookshelf to obtain the face features of all family members in the family in the mode, namely, the one-dimensional face feature vectors of the reference characters are obtained, and then the set of the one-dimensional face feature vectors of the reference characters can be used as a reference face feature library.

On the other hand, the user can also input the voice of the reference character into the intelligent bookshelf, or directly make the reference character speak so that the voice of the reference character is collected by the voice collector of the intelligent bookshelf, so that the controller of the intelligent bookshelf obtains the voice characteristic of the reference character based on the voice analysis. For example, the controller may determine a sound wave pattern of a reference character's wake-up voice. The wake-up voice may be a voice speaking a designated wake-up sentence, such as "hello, bookshelf" for the wake-up sentence, or may be another sentence, such as "hello, bookshelf". The acoustic pattern may be a waveform in a reference coordinate system. The abscissa of the reference coordinate systemTime and frequency on the ordinate. Note that the time indicated on the abscissa is only a relative time of the sound signal, and is not a specific time stamp. The controller can determine n first reference points which are uniformly distributed in the sound wave pattern of the voice of the reference character in the reference coordinate system, wherein n is more than or equal to 2. For example, the controller may take a point at regular intervals in the sound wave pattern as the first reference point to obtain the n first reference points. n may be tens or even hundreds. The controller may use the set of coordinate values of the n first reference points as the sound feature of the reference character. Such as the set C ═ { C ═ C₁，c₂，c₃，c₄，…，c_n}，

I is more than or equal to 1 and less than or equal to n. C in set C_iRepresenting the ith of the n first reference points,

the abscissa representing the ith first reference point,

indicating the ordinate of the ith first reference point. The user can trigger the intelligent bookshelf to acquire the sound characteristics of all family members in the family in the mode, namely the sound characteristics of a plurality of reference characters, and then the set of the sound characteristics of the reference characters can be used as a reference sound characteristic library.

For any reference character, the user can input the names of a plurality of readable books or at least one type of readable bookshelf into the intelligent bookshelf, so that the intelligent bookshelf can record the information of the readable books corresponding to the reference character. As shown in fig. 7, a user may select a book from a plurality of books to be selected, so that the intelligent bookshelf uses the name of the selected book as information of a readable book corresponding to a reference person. The books to be selected may include books stored in the intelligent bookshelf. Optionally, the user may also select a book type from the multiple types of books to be selected, so that the intelligent bookshelf uses the selected book type as a type of a readable book corresponding to the reference character.

Optionally, an application program for book management is installed in the terminal, and the user may trigger the terminal to bind with the intelligent bookshelf through the application program. If the identification of the intelligent bookshelf can be input on the terminal, the terminal is connected with the intelligent bookshelf, and then the binding of the terminal and the intelligent bookshelf is achieved. The binding back terminal can perform information interaction with the intelligent bookshelf and can control the intelligent bookshelf. For example, the user may input the information of the reference person on the terminal, and the terminal may transmit the information of the reference person input by the user to the smart bookshelf, in which case fig. 5 to 7 are display interfaces of the terminal. Or, the user may directly operate on the display screen of the intelligent bookshelf to input the information of the reference character to the intelligent bookshelf, in which case, fig. 5 to 7 are display interfaces of the display screen of the intelligent bookshelf.

Specifically, step 401 is performed by a controller in the intelligent bookshelf. For example, the controller may acquire the information of the reference person from the terminal through the communication unit or acquire the information of the reference person input through the display screen. After the controller acquires the information of the reference person, the information of the reference person can be stored in the memory for subsequent calling.

And step 402, judging whether the feature similarity between the target person and the reference person in the image acquired by the camera is greater than a first threshold value. When the feature similarity is greater than the first threshold, executing step 403; when the feature similarity is less than or equal to the first threshold, the step 402 is continued.

It should be noted that, in the embodiment of the present application, the feature similarity is taken as a face feature similarity, and further, an example of processing an image acquired by a camera is described. Optionally, the feature similarity may also be a sound feature similarity, and at this time, the sound collected by the sound collector may be processed. For the sound feature similarity, reference may be made to the following description of face feature similarity, and the embodiments of the present application are not limited thereto.

The image acquisition scope of the camera of intelligent bookshelf includes bookshelf body the place ahead certain limit at least, and the camera can gather the image of the target personage that appears in its image acquisition scope. The controller may determine the feature of the target person based on the image of the target person captured by the camera, and further determine whether the similarity of the feature of the target person to the reference person is greater than a first threshold. If the first threshold is 90%, 95%, 96%, or other values, the embodiments of the present application are not limited. Alternatively, the controller may collectively determine the characteristics of the target person based on the images of the plurality of frames of target persons captured by the camera.

In the embodiment of the application, whether the feature similarity of the target person and the reference person is greater than a first threshold value is determined, and the determination is used for determining whether the target person is the reference person. If the feature similarity is greater than a first threshold, determining that the target person is a reference person; if the feature similarity is less than or equal to the first threshold, it may be determined that the target person is not the reference person. Alternatively, if the feature similarity between the target person and the reference persons is greater than the first threshold, it may be determined that the target person is the reference person with the largest face feature similarity. If the same person as the target person exists in the set reference persons, go to step 403; if not, continuing to execute step 402 for the images subsequently acquired by the camera.

The camera can collect images of people facing the bookshelf body. The image of the target person may include a facial image of the target person, the features of the target person include facial features, and the feature similarity of the target person to the reference person may include facial feature similarity. Optionally, the features of the target person may also include features other than facial features, such as body type features; the feature similarity may also include similarity of features other than facial features, such as body type similarity of people. The following description will be given by taking only the feature of the target person as the facial feature.

Step 402 may be specifically executed by a controller in the intelligent bookshelf. Alternatively, the camera may continuously capture images, and the controller may perform person detection on each frame of image captured by the camera. When it is determined that a person exists in the image, the controller may further determine whether the image includes a face image of the person, which is the target person. Upon determining that the image includes a facial image of a target person, the controller may extract facial features of the target person based on the facial image. Then, the controller may determine the degree of similarity of facial features of the target person and the set reference person to identify whether the target person is a set reference person.

For example, the controller may input the facial image into a face recognition model to obtain a recognition result output by the face recognition model. The recognition result may characterize whether the feature similarity of the target person and the reference person is greater than a first threshold. The controller can extract the features of the facial image of the target person through the face recognition model and then the face recognition model to obtain the one-dimensional facial feature vector of the target person

The manner of obtaining the one-dimensional face feature vector of the target person may be the same as the manner of determining the one-dimensional face feature vector of the reference person in step 401 described above. The controller can compare the one-dimensional facial feature vector of the target person with the one-dimensional facial feature vectors of a plurality of reference persons in the reference face feature library through the face recognition model to obtain information of the reference persons, which is output by the face recognition model and has the similarity with the facial feature of the target person larger than a first threshold value. If the face recognition model determines that there are a plurality of reference persons having a similarity of facial features to the target person greater than a first threshold, the controller may output only information of the reference person having the greatest similarity of facial features. If there is no reference person whose facial feature similarity with the target person is greater than the first threshold, the face recognition model may output information indicating that recognition has failed.

The controller may determine a cosine value cos θ of an angle between the one-dimensional facial feature vector of the target person and the one-dimensional facial feature vector of the reference person, and determine the target person based on the cosine valueSimilarity of facial features of the target person and the reference person. The closer the cosine value is to 1, the greater the similarity of the facial features. For example, the controller may determine that the degree of similarity of the facial features of the target person and the reference person is greater than a first threshold when the absolute value of the difference between the cosine value and 1 is less than the absolute value threshold. Such as the cosine value

The absolute threshold may be 0.1, 0.05 or some other value.

Optionally, the controller of the intelligent bookshelf may also send the image of the target person collected by the camera to the server through the communication unit, so that the server determines whether the target person is a reference person for the image, and then sends the determined result back to the controller. Therefore, the process of consuming more computing resources can be transferred to a server with stronger functions to be executed, the waste of the computing resources of the controller in the intelligent bookshelf is avoided, and the computing pressure of the controller is reduced.

Alternatively, if the controller determines that the degree of similarity in characteristics of the target person and the reference person is less than or equal to a first threshold value, and the target person is not the reference person having information recorded in the smart bookshelf, the controller may further control the information output component to output information indicating that the target person is not registered. The target person may operate on the display screen of the smart bookshelf to register. If the target person can input own information to the intelligent bookshelf, if the target person inputs own identification through a display screen of the intelligent bookshelf, own images are collected through a camera of the intelligent bookshelf. The controller can send the identifier and the image to a terminal bound with the intelligent bookshelf through the communication unit so that a user of the terminal can check the information of the target object. After seeing the identification and the image of the target object, a user of the terminal can send a registration permission instruction to the intelligent bookshelf, and then registration of a target person on the intelligent bookshelf can be completed. The intelligent bookshelf can store the identification of the target person and the image.

Alternatively, the door of the intelligent bookshelf may be locked or unlocked under the control of the controller. For example, if the controller determines that the target person is not the reference person recorded with information, the controller may control the door of the intelligent bookshelf to remain locked to prevent unregistered persons from freely accessing books in the intelligent bookshelf. After the target person is registered, the controller may control the door of the intelligent bookshelf to be unlocked. If the controller determines that the target person is a reference person, the controller may control the door of the intelligent bookshelf to be unlocked, so that the target person can access books on the intelligent bookshelf. Optionally, if no person exists in the image acquired by the camera of the intelligent bookshelf, the controller may control the doors of the intelligent bookshelf to be locked, so as to prevent a pet or the like from damaging books in the bookshelf.

Optionally, the intelligent bookshelf may include a plurality of doors that can be controlled separately, and each door is used for covering an opening of at least one storage cell in the bookshelf body. The controller can also detect whether the reference person corresponds to the information of the readable book after determining that the target person is a reference person. When the reference character corresponds to a readable book, the controller can determine the position of the readable book in the bookshelf body, and then control the door of the storage grid where the bookshelf body is located to be unlocked. Therefore, the target person can only take the books in the allowed storage lattices, the probability that the target person can read the books is increased, and the probability that the target person cannot take the books which are not allowed to be read is reduced.

And step 403, outputting the historical reading information of the reference person. Step 404 is performed.

In the embodiment of the application, the intelligent bookshelf can record the historical reading information of each reference character aiming at each reference character. The historical reading information may be stored in a memory in the intelligent bookshelf or may be stored in a server connected to the intelligent bookshelf. The historical reading information of the reference person may include: the reading method comprises the following steps of referring to at least one of the name of a historical reading book of a character, the page number of the latest reading page of the historical reading book of the character, the reading progress of the historical reading book of the character, and the reading times of the historical reading book of the character. The reading progress of the reference character on a certain book can be determined based on the total page number of the book and the page number of the reference character on the latest reading page of the book, wherein the latest reading page is the page where the reference character reads the content last time. For example, the reading progress can be the ratio of the page number of the most recently read page to the total page number. Optionally, the reference to the historical reading information of the person may further include: and (4) referring to the total reading time of the character on each book of historical reading.

The controller may obtain the historical reading information of a reference person after determining that the target person is the reference person. The controller can also output the historical reading information through the information output component so that the target person can select books to be read based on the historical reading information.

For example, the controller may control a display screen of the smart bookshelf to display the historical reading information of the reference character. Fig. 8 is a schematic diagram of another display interface provided in the embodiment of the present application. The display screen of the intelligent bookshelf can display the name, reading progress, reading times, the page number of the most recently read page and the total page number of the books which are historically read by the reference character through the display interface shown in fig. 8. As shown in fig. 8, the display screen may display information of respective books historically read in a list manner. For example, the books can be arranged in sequence according to the reading sequence. The display screen can display the reading progress of each book of the reference character through the progress bar, and can also display the percentage representing the reading progress around the progress bar. "282/426" indicates that the most recently read page has page number 282 and total page number 426. Optionally, continuing to see fig. 8, the display screen may also display a last reading record of the target object. If the last read book is recorded by insects, the most recently read page of the book is 79 pages. Optionally, the controller may further control the speaker to broadcast the last reading record of the reference object to assist the reference object in selecting the book to be read based on the last reading record.

And step 404, after the target person is determined to take out the book from the bookshelf body, judging whether the taken out book belongs to the readable book. If the book belongs to a readable book, go to step 405; when the book does not belong to the readable book, step 406 is performed.

For example, the controller of the intelligent bookshelf may determine the electronic tag removed from the bookshelf body by the signal reception strength of the receiver to the electronic tag. Because the image collected by the camera reflects that the target person is in front of the bookshelf body, and the electronic tag is moved out of the bookshelf body at the moment, the controller can determine that the target person takes out the book corresponding to the electronic tag from the bookshelf body. For the way of determining the electronic tag to be removed from the bookshelf body according to the signal receiving strength, please refer to the related description of the receiver in fig. 1, and the embodiments of the present application are not described again.

Alternatively, the target person may scan the taken book at a barcode scanner of the smart bookshelf to enable the controller to determine the taken book by the target person. Alternatively, the controller may capture an image of a book taken out by the target person through the camera, and perform image recognition to determine the book taken out by the target person. Alternatively, the target person may speak the name of the book taken out after taking out the book. The voice collector of the intelligent bookshelf can collect the voice spoken by the target person and perform voice recognition so as to determine the books taken out by the target person.

Since information of a readable book may be set for a reference person, the reading range of the book of the reference person may be limited. After the controller determines that the target person is a preset reference person, the book taken by the target person can be detected to judge whether the target person takes out books beyond the range, namely whether books other than readable books are taken out. For example, the controller may query whether the reference character corresponds to a readable book. When the reference character corresponds to the readable book, the controller may determine whether the book taken by the target character belongs to the readable book, and then perform a corresponding operation based on the determination result. When the reference task does not correspond to a readable book, the controller may directly perform step 405.

For example, the information of the readable books corresponding to the reference person is the names of a plurality of readable books. The controller can determine the name of the book taken by the target person, and further judge whether the name of the book taken by the target person is one of the names of the plurality of readable books. If yes, determining that the books taken by the target person belong to the readable books; if not, determining that the books taken by the target person do not belong to the readable books. Further illustratively, the information of the readable books corresponding to the reference character is at least one type of the readable books. The controller can determine the type of the book taken by the target person, and further judge whether the type belongs to the at least one type. If yes, determining that the books taken by the target person belong to the readable books; if not, determining that the books taken by the target person do not belong to the readable books. In the embodiment of the application, at least one type of the readable books is a relatively detailed classification of the books, such as children's books, primary school textbooks, and the like, and all books of each type in the at least one type are all suitable for being read by the reference character.

Alternatively, if the identity information of the reference person is a child or a student, the user may not set the information of the readable book for the reference person. The controller may directly determine the student recommended by the education department to read the book in the directory as the readable book corresponding to the reference character. When the information of the readable book set by the user exists, the controller preferentially judges whether the book taken by the reference character is the readable book set by the user. Alternatively, the user may manually set the book in the student reading list recommended by the education section as the readable book corresponding to the reference character.

Step 405, recording the information of the reference person to take out the book.

The controller of the intelligent bookshelf can record the information of the book taken out by the reference person when the controller determines that the book taken out by the target person belongs to the readable book. Such as recording the time the reference person takes out the book. The controller may also mark the book stored in the bookcase body among the recorded information of the book to indicate that the book is taken out.

And step 406, outputting information for prompting the target person to put back the book.

When the controller of the intelligent bookshelf determines that the book taken out by the target person does not belong to the readable book, the controller can output information prompting the target person to put back the book through the information output component. For example, the controller may control the speaker to emit a prompt voice, such as "hello, take book out of range, please return book". The controller may also control the display screen to display a prompt to prompt the target person to replace the book. The target person can put the taken book into the bookshelf body based on the prompt voice played by the loudspeaker or the prompt information displayed by the display screen. The method for determining that the book is placed in the bookshelf body by the controller may correspond to the method for determining that the book is removed from the bookshelf body, and may specifically be determined according to the method for determining that the book is removed from the bookshelf body by the controller in step 404, which is not described in detail in this embodiment of the application.

Optionally, when the controller determines that the book taken out by the target person does not belong to the readable book, a prompt message for the target person to take out the book may be sent to the terminal bound to the intelligent bookshelf. The prompt message may also include the name of the book taken by the target person, and information that the book is out of range. When the target person puts the books beyond the range back to the bookshelf body, the controller can also send a prompt message that the books of the target person are put back to the terminal. Therefore, the reading of the target person can be timely monitored by the user of the terminal. Fig. 9 is a schematic diagram of a display interface according to another embodiment of the present application. As shown in FIG. 9, when the target character "glue pudding" takes out the book beyond the range, the terminal can display the prompt message "glue pudding this time takes the book as" dress and beauty ", beyond reading! ". Wherein, the glue pudding is the mark of the target character, and the 'dress and beauty' is the name of the book taken out by the target character.

Alternatively, the controller may record the taking-out time of a certain book when determining that the book is taken out of the bookshelf body. When the book is put back to the bookshelf body again, the time length for taking out the book from the bookshelf body can be determined based on the putting back time and the taking out time of the book. When the duration is greater than the duration threshold, it can be determined that the user has read the book once, and the duration is determined as the duration of the user reading this time, so that the historical reading information of the user on the book can be updated. When the duration is less than or equal to the duration threshold, it can be considered that the user does not read the book, and the reading information is not recorded in the taking-out of the book. If the target person takes out the books except the readable books and immediately puts the books back to the bookshelf body by hearing the prompting voice of the loudspeaker after taking out the books, the taking out of the books is not recorded as one-time reading.

Step 407, after any voice with the similarity of the voice characteristics of the reference person larger than the second threshold value is received, if a book returning instruction triggered by the reference person is received within a specified time, acquiring the page number of the current reading page of the book to be returned.

The user can interact with the intelligent bookshelf through voice, for example, a voice instruction can be sent to the intelligent bookshelf, so that the intelligent bookshelf can perform corresponding operation based on the voice instruction. Before sending a voice instruction to the intelligent bookshelf, a user can firstly speak a wakeup statement of the intelligent bookshelf to trigger the intelligent bookshelf to start a function of receiving the voice instruction. For the introduction of the wake-up statement, reference may be made to the related introduction in step 401, and details are not described herein again in this embodiment. The sound collector in the intelligent bookshelf can continuously collect voice. The controller can compare the voice collected by the voice collector with the awakening voice of the reference person, and specifically can compare the characteristics of the voice with the voice characteristics in the reference voice characteristic library to determine whether the received voice is the awakening voice sent by the reference person recorded with information in the intelligent bookshelf. When the controller determines that the similarity between any voice and the awakening voice sent by the reference person is larger than the second threshold value, namely the similarity between the characteristics of any voice and the sound characteristics of the reference person is larger than the second threshold value, the controller can determine that any voice is the awakening voice sent by the reference person. Optionally, if the similarity between the feature of the speech and the sound features of the multiple reference persons is greater than the second threshold, it may be determined that the speech is emitted by the reference person with the greatest similarity.

For example, the controller may not collect the wake-up voice before the controller detects the wake-up voiceEach voice collected by the voice collector determines the characteristics of the voice so as to determine whether the voice is a wake-up voice sent by a reference person. The manner in which the characteristics of the speech are determined may be the same as the manner in which the characteristics of the sound of the reference character are determined in step 401. For example, the controller may take a point at regular intervals in the sound wave pattern of the voice in the reference coordinate system as the second reference point to obtain n second reference points uniformly distributed in the sound wave pattern of the voice. The adjacent second reference points may be spaced apart by the same time or different time as the adjacent first reference points. The set of coordinate values of the n second reference points may be used as the feature of the speech. If the set Q is Q ═ Q₁，q₂，q₃，q₄，…，q_n}，

Q in the set Q_iRepresenting the ith of the n second reference points,

the abscissa representing the ith second reference point,

indicating the ordinate of the ith second reference point.

The controller may employ a dynamic time warping algorithm to similarity match the speech to the reference character's wake-up speech. The controller may determine a euclidean distance between an ith first reference point in the sound wave pattern of the reference character's wake-up voice and an ith second reference point in the sound wave pattern of the voice. The Euclidean distance

Closer euclidean distance to 0 indicates higher similarity. The controller may determine a number of first reference points for which the euclidean distance between the first reference points and the corresponding second reference points is less than a distance threshold. When more first reference points and corresponding second reference points exist, the Euclidean distances are smaller than the distance threshold value, the controller determines the received voice and the parametersAnd determining that the received voice is the awakening voice sent by the reference person when the similarity of the awakening voice of the reference person is larger than a second threshold value. And if the controller determines that the Euclidean distance between m first reference points and the corresponding second reference points in the n first reference points is smaller than the distance threshold, determining the received voice as the awakening voice sent by the reference person. Wherein m is more than or equal to 1, and m/n is more than a proportional threshold. The ratio threshold may be 90%, 95%, or some other ratio.

After the controller of the intelligent bookshelf determines that the awakening voice sent by the reference figure is received, the controller can correspondingly operate based on the subsequent voice instruction. If the similarity of the voice and the recorded awakening voices sent by all the reference characters is smaller than or equal to a second threshold value, it is determined that the user sending the voice is not the reference character recorded with the information, and the intelligent bookshelf can prohibit the operation from being executed based on a subsequent voice instruction. Alternatively, the controller may further control a speaker or a display screen to output information of the identification failure.

Alternatively, after the user speaks the wake-up voice, the speaker of the intelligent bookshelf may send out a response voice, such as "i am". And then, the user can continue to send out the voice command, and the intelligent bookshelf can execute corresponding operation based on the subsequently received voice command. Optionally, the speaker of the intelligent bookshelf may not send a response voice, and the user may directly send a voice instruction after speaking the wake-up voice. After the intelligent bookshelf determines to receive the awakening voice sent by the reference character, the intelligent bookshelf can directly determine the voice received in the specified time length as the voice sent by the reference character. The controller may analyze the voice instruction to determine an operation indicated by the voice instruction to be performed.

For example, after the reference character reads the book, the book can be put back to the bookshelf body of the intelligent bookshelf. When the reference person puts back the book to be returned, the reference person can speak the awakening voice first and then speak the instruction of returning the book. If a book returning instruction in the form of voice is received within the specified time period, the controller may directly determine that the book returning instruction is issued by the reference character. Optionally, the sound collector may continuously collect the voice, and the controller may determine whether the voice collected by the sound collector carries the book return information. When the collected voice is determined to carry book returning information, the controller can determine that a book returning instruction is received. Optionally, the reference person may also click a book returning button on the display screen, or a barcode scanner of the intelligent bookshelf scans a barcode or a two-dimensional code of a book to be returned, so as to trigger the controller to receive a book returning instruction.

Optionally, if the sound characteristics of the reference person are not recorded in the intelligent bookshelf, the controller of the intelligent bookshelf may further determine whether the target person triggering the book returning instruction is the reference person recorded with the information after the book returning instruction is acquired. Alternatively, the controller may determine the person in the image captured by the camera as the target person for triggering the book returning instruction. The controller may determine whether the target person is a reference person based on the image. For a specific determination manner, reference may be made to related descriptions in step 402, and details are not described in this embodiment of the application.

After determining that the book returning instruction is triggered by the reference person recorded with the information, the controller of the intelligent bookshelf can acquire the page number of the current reading page of the book to be returned so as to record the reading information of the book by the reference person. For example, the reference character may also speak a voice carrying the page number of the current reading page. The sound collector of the intelligent bookshelf can collect the voice spoken by the reference character. The controller can analyze the voice collected by the voice collector to determine the page number of the current reading page carried by the voice. Such as a reference character, can say "read currently to page 79". For another example, the controller may control the information output component to output prompt information for aligning a current reading page of the book to be returned to the camera for photographing recording. The reference person can place the current reading page of the book to be returned under the camera based on the prompt information so that the camera can shoot the image of the current reading page. The controller can identify the image of the current reading page to acquire the page number of the current reading page. The current reading page is the latest reading page.

Optionally, after receiving the book returning instruction, the controller may determine whether the reference person needs to record the relevant information of the reading through the voice based on the voice collected by the voice collector. If the relevant information of the reading is determined to be recorded through the voice, the voice collected subsequently by the sound collector can be analyzed to determine the page number of the current reading page. And if the relevant information of the reading is determined not to be recorded through voice, then controlling a loudspeaker to align the current reading page of the book to be returned to the prompt information of the camera, and determining the page number of the current reading page in an image recognition mode.

Alternatively, the controller of the smart bookshelf may control the information output component to output information prompting the person to register when it is determined that the person who triggered the return instruction is not the reference person. The manner of the person registration may refer to the related introduction of the target person registration in step 402, and the embodiment of the present application is not described again. After the person is registered, the controller acquires the reading information of the book to be returned of the person and stores the reading information.

And step 408, determining the reading progress of the reference person on the book to be returned based on the page number of the current reading page and the total page number of the book to be returned.

Alternatively, the controller of the intelligent bookshelf may acquire the total page number of the book to be returned, for example, the total page number may be input by the user, or the controller of the intelligent bookshelf may acquire the total page number in the internet through the communication unit. For example, the controller of the intelligent bookshelf may determine a ratio of the obtained page number of the current reading page to the total page number of the book to be returned as a user reading progress of the reference character on the book to be returned.

And 409, recording the reading progress of the reference person on the book to be returned after the book to be returned is determined to be stored in the bookshelf body.

The mode that the controller of the intelligent bookshelf determines that the book to be restored is stored in the bookshelf body may refer to the relevant records in step 406, and details are not described in this embodiment of the application. After the controller determines that the book to be returned is stored in the bookshelf body, the reading progress of the book to be returned by the reference person can be recorded. Optionally, the controller may also directly determine that the book to be returned is stored in the bookshelf body after receiving a book return instruction for the book to be returned, and further directly record the reading progress after determining the reading progress of the book to be returned by the reference character.

For example, if the reading information of the reference person on the book to be returned is not recorded in the smart bookshelf, the reading information of the reference person on the book to be returned may be newly added. In the historical reading information, the name of the book which is read by the reference person in the historical reading may include the name of the book to be returned, the number of times of reading the book to be returned by the reference person is 1, and the reading progress of the book to be returned by the reference person is the reading progress of the book. For the history reading information, reference may be made to the related description in step 403, and details of the embodiment of the present application are not repeated. If the reading information of the book to be returned by the reference person is recorded in the intelligent bookshelf, the reading information of the book to be returned by the recorded reference person can be updated. For example, the controller may increase the number of times that the reference person reads the book by one, and replace the original reading progress with the reading progress of the reading. Optionally, the reading information of the book by the reference person may further include a total reading time of the book. The controller can determine the reading time length, and then adds the reading time length to the recorded total reading time length of the book to serve as the updated total reading time length. The reading time is the time difference between the time when the book is taken out of the bookshelf body and the time when the book is put back into the bookshelf body.

Optionally, after the book is put back to the bookshelf body, the controller may further send a prompt message that the reference person has returned the book to the terminal bound to the intelligent bookshelf. The prompt information can include the name of the book, the page number of the current reading page of the book by the reference character and the information of the reading time length, so that the user of the terminal can supervise and control the reading process of the reference character. Fig. 10 is a schematic diagram of another display interface provided in another embodiment of the present application. As shown in fig. 10, the prompt message may be "the glue pudding has put the" insect note "back to the bookshelf, the reading time is 2 hours, and the current reading record is page 97".

An embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When the instructions are run on a computer, the instructions cause the computer to execute the book management method provided by the above-mentioned embodiment.

Embodiments of the present application further provide a computer program product including instructions, which, when the computer program product runs on a computer, cause the computer to execute the book management method provided in the foregoing method embodiments.

It should be noted that, the method embodiments provided in the embodiments of the present application can be mutually referred to with corresponding apparatus embodiments, and each method embodiment and each apparatus embodiment may also be mutually referred to, which is not limited in the embodiments of the present application. The sequence of the steps of the method embodiments provided in the embodiments of the present application can be appropriately adjusted, and the steps can be correspondingly increased or decreased according to the situation, and any method that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application shall be covered by the protection scope of the present application, and therefore, the details are not repeated.

In the present application, the terms "include" and "comprise" are used in an open-ended fashion, and thus should be interpreted to mean "include, but not limited to. In the embodiments of the present application, the terms "first", "second", and the like are used for distinguishing identical items or similar items having substantially the same functions, and it should be understood that the terms "first", "second", and "n" have no logical or temporal dependency, and do not limit the number or execution order. The term "at least one" in the embodiments of the present application means one or more, and the term "a plurality" in the present application means two or more.

The term "and/or" in this application is only one kind of association relationship describing the associated object, and means that there may be three kinds of relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship. Where mathematical formula calculations are involved, the character "/" represents the operator "divide by". The term "at least one of a and B" in the present application is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, at least one of a and B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In this application, "at least one" means one or more, "a plurality" means "two or more".

The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. An intelligent bookshelf, characterized in that the intelligent bookshelf comprises:

the bookshelf body is used for storing books;

the camera is used for collecting images of target characters for storing and taking books on the bookshelf body;

2. The intelligent bookshelf according to claim 1, wherein the information of the readable book comprises: at least one of a name and a type of the readable book.

3. The intelligent bookshelf according to claim 1, wherein any one of the books carries a corresponding electronic tag, and the electronic tag is used for sending a signal; the intelligent bookshelf further comprises a receiver, the receiver is located on the surface or inside the bookshelf body, and the receiver is used for receiving signals sent by the electronic tags;

the controller is further configured to:

determining the receiving strength of the receiver to the signal sent by the electronic tag;

when the receiving intensity of the signal sent by any electronic tag by the receiver changes from being higher than an intensity threshold value to being lower than the intensity threshold value, determining that the book corresponding to any electronic tag is taken out of the bookshelf body.

4. The intelligent bookshelf according to claim 1, wherein the image of the target person comprises a facial image of the target person, and the feature similarity comprises a facial feature similarity; the controller is configured to:

receiving a face image of the reference person;

performing data dimension reduction processing on the face image of the reference person to obtain a one-dimensional face characteristic vector of the reference person;

after the camera acquires the face image of the target person, performing data dimensionality reduction on the face image of the target person to obtain a one-dimensional face feature vector of the target person;

determining a cosine value of an included angle between the one-dimensional facial feature vector of the target person and the one-dimensional facial feature vector of the reference person;

and when the absolute value of the difference value between the cosine value and 1 is smaller than an absolute value threshold, determining that the feature similarity between the target person and the reference person is larger than the first threshold.

5. Intelligent bookshelf according to any one of claims 1-4,

the intelligent bookshelf further comprises a communication unit, and the communication unit is used for: when any book does not belong to the readable book, sending information prompting the target person to take out the book to a terminal bound with the intelligent bookshelf;

and/or the presence of a gas in the gas,

the intelligent bookshelf further comprises a door, and the door is movably connected with the bookshelf body and used for shielding the opening of the bookshelf body; the door is unlocked when the degree of similarity of the features of the target person and the reference person is greater than a first threshold.

6. Intelligent bookshelf according to any one of claims 1-4,

the controller is further configured to: when the feature similarity is larger than the first threshold, obtaining historical reading information of the reference person, wherein the historical reading information of the reference person comprises: at least one of a name of a history reading book of the reference person, a reading progress of the history reading book of the reference person, and a reading number of the history reading book of the reference person;

the information output component is further to: and outputting the historical reading information of the reference person.

7. The intelligent bookshelf of claim 6, wherein the controller is further configured to:

after a book returning instruction is received and the book returning instruction is determined to be triggered by the reference person, acquiring the page number of the current reading page of the book to be returned of the reference person;

determining the reading progress of the reference person on the book to be returned based on the page number of the current reading page and the total page number of the book to be returned;

and recording the reading progress of the reference person on the book to be returned after the book to be returned is determined to be stored in the bookshelf body.

8. The intelligent bookshelf according to claim 7,

the camera is also used for: acquiring an image of the current reading page;

the controller is configured to: identifying the image of the current reading page to acquire the page number of the current reading page;

or,

the intelligent bookshelf further comprises a sound collector, and the sound collector is used for collecting voice;

the controller is configured to: and analyzing the voice to acquire the page number of the current reading page.

9. The intelligent bookshelf according to claim 7, wherein the book return instruction is an instruction in voice form; the controller is configured to:

receiving a wake-up voice of the reference character speaking the wake-up sentence;

n first reference points which are uniformly distributed in the sound wave pattern of the awakening voice in a reference coordinate system are determined, wherein n is more than or equal to 2;

after any voice is received, n second reference points which are uniformly distributed in the sound wave pattern of any voice in the reference coordinate system are determined;

determining the Euclidean distance between the ith first reference point in the n first reference points and the ith second reference point in the n second reference points, wherein i is more than or equal to 1 and less than or equal to n;

when the Euclidean distance between m first reference points and the corresponding second reference points in the n first reference points is smaller than a distance threshold, determining that a book returning instruction received within a specified time length after any voice is triggered by the reference person; wherein m is more than or equal to 1, and m/n is more than a proportional threshold.

10. A book management method for an intelligent bookshelf, the intelligent bookshelf comprising: the bookshelf comprises a bookshelf body for storing books, a camera, a controller and an information output assembly, and the method comprises the following steps:

the controller determines whether any book belongs to the readable books based on the information of the readable books corresponding to the reference person after determining that the feature similarity between the target person and the reference person is greater than a first threshold value based on the image and determining that the target person takes out any book from the bookshelf body;