KR20180000347U - System for providing dynamic reading of publication using mobile device - Google Patents

Abstract

A stereoscopic reading system using a mobile device according to the present invention is a publishing medium on which information that can be read is recorded; A sound recognition unit for recognizing that the information recorded on the publication medium is read, a sound recognition unit for determining whether a sound effect is designated for the recognized information, and a sound effect for executing a specified sound effect when the sound effect is designated A sound effect storage unit for storing the sound effect information stored in the sound effect storage unit; a sound effect storage unit for storing effects that are sequentially executed as the readable information is read; A mobile device having an automatic download module for downloading and storing the downloaded sound effects in the sound effect storage unit; And a sound effect server that can access the mobile device via network means and store sound effects that are executed in accordance with the reading of the contents of the publication medium, wherein the publication medium is recognizable by the mobile device An ISBN, a KC authentication number (an autonomous security check certificate number), a QR code, or an NFC tag for identifying the publication medium.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a system for reading three-

This invention relates to a stereoscopic reading system for publishing media. And more particularly, to a sound effect providing system that provides a sound effect for a publication medium using a mobile device.

The publishing media have been used from the distant past to provide a wide range of information on human society. However, in a multimedia environment that has recently caused extreme interest, conventional publishing media are increasingly being ignored by users.

On the other hand, the development of personal information devices such as computers and smart phones has created a so-called electronic book (E-book) type publication medium in which electronic information is displayed on electronic display means instead of conventional paper. However, the situation in which users are getting out of the picture is not much different in the case of e - book - type publishing media.

For example, a fairy tale book may be provided in the form of a publication or an electronic book. Here, the assimilation book provided as an e-book enables visualization of the image of the assimilation book content, so that it can be felt more fun than the assimilation book provided as a printed matter.

However, as a user who views a fairy tale book, that is, a child has a low concentration and soon becomes bored, a technique for providing a fairy tale book that can induce a child more interesting is needed.

Korean Patent Publication No. 10-2002-0025916

The present invention aims to provide a system for providing a sound effect for a publication medium using a mobile device capable of generating a vivid sound effect in accordance with the reading of information contained in a publication medium.

A stereoscopic reading system using a mobile device according to the present invention is a publishing medium on which information that can be read is recorded; A sound recognition unit for recognizing that the information recorded on the publication medium is read, a sound recognition unit for determining whether a sound effect is designated for the recognized information, and a sound effect for executing a specified sound effect when the sound effect is designated A sound effect storage unit for storing the sound effect information stored in the sound effect storage unit; a sound effect storage unit for storing effects that are sequentially executed as the readable information is read; A mobile device having an automatic download module for downloading and storing the downloaded sound effects in the sound effect storage unit; And a sound effect server that can access the mobile device via network means and store sound effects that are executed in accordance with the reading of the contents of the publication medium, , And may include at least one of a barcode, an ISBN, a KC authentication number (an autonomous security check certificate number), a QR code, or an NFC tag for identifying the publication medium.

In this case, the sound effect server may include a publication medium contents DB storing information related to each of the plurality of publication media; A sound effect information DB storing sound effect information designated for each of the contents; And a service providing unit for receiving the information about the publication medium from the mobile device and for providing the sound effect information included in the content of the input publication medium to the mobile device.

An authentication unit for authenticating the mobile device or the user; A data communication unit forming a data communication channel with the mobile device; And a service management unit for performing subscriber management, charge management, and service management for a service providing service providing the sound effect information.

Here, the mobile device may be any one of a mobile phone, a PDA, a tablet, a notebook, a personal computer (PC), a scanner terminal, an education terminal, or an E-

At this time, the publication medium may display the effect that the sound effect is designated in the readable information.

Here, the predetermined word or words may be the corresponding pronoun for the word to which the sound effect is assigned.

In addition, the sound effect execution unit recognizes that the sound effect is not reproduced in the basic read-aloud situation and that the sound effect recognizes that the predetermined word or words are read before or after the designated word, And an acoustical off mode for reproduction.

The sound effect execution unit may include an error response mode for executing a sound effect next to the sound effect immediately preceding the sound effects sequentially stored in the publication medium when the predetermined word is recognized .

When the stereoscopic reading system using the mobile device of the present invention according to the above configuration is performed, there is an advantage that the listener can feel a vivid sound effect in accordance with the reading aloud in reading the information contained in the publication medium.

1 is a structural view showing a sound effect providing system according to an embodiment of the present invention;
2 is a structural view showing a sound effect providing system according to another embodiment of the present invention;
FIG. 3 is a block diagram illustrating an embodiment of an effect information server for providing effect information according to the idea of the present invention; FIG.
FIG. 4 is a view showing a reading assimilation book as an embodiment of a publication medium capable of executing effect information according to the idea of the present invention; FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings.

In describing the present invention, the terms first and second may be used to describe various components, but the components may not be limited by terms. Terms are for the sole purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being connected or connected to another element, it may be directly connected or connected to the other element, but it may be understood that other elements may be present in between .

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions may include plural expressions unless the context clearly dictates otherwise.

It is to be understood that the term " comprising, " or " comprising " as used herein is intended to specify the presence of stated features, integers, But do not preclude the presence or addition of steps, operations, elements, components, or combinations thereof.

In addition, the shapes and sizes of the elements in the drawings and the like can be exaggerated for clarity.

1 shows a system for providing a sound effect according to an embodiment of the present invention having a smartphone application.

The sound effect providing system shown in FIG. 1 is a publication medium on which information that can be read is recorded, a child / child story book 400; And a smartphone application installed in a smartphone 100 of a reader (caretaker) such as a parent to execute an effect.

The application includes software modules, a voice recognition unit (120) for recognizing information recorded on the publication medium read aloud; An effect execution unit (140) for judging whether an effect is assigned to the recognized information, and for executing a specified effect when an effect is designated; And an effect storage unit 160 storing the effects that are sequentially executed according to the read aloud of the readable information.

The voice recognition unit 120 analyzes sound information input from a microphone provided in the smartphone 100 and determines whether or not the information recorded on the publication medium is voice information read by the user (read aloud) can do. The voice recognition unit 120 can partly analyze and determine the recognized readable information (abbreviated as read aloud information). For example, the speech recognition unit 120 may analyze the recognized read aloud information by syllable unit or word unit, and determine whether an effect according to the idea of the present invention is a designated word or syllable. Various known speech recognition algorithms can be applied to this process, and a detailed description of the known algorithms will be omitted.

The effect execution unit 140 can also apply visual effects, sensory effects such as vibration (touch) and smell, but is advantageous for applying sound effects at low cost. For example, the effect execution unit 140 may drive a speaker provided in the smartphone 100 to output an effect sound assigned to each effect to be executed.

Here, the effect execution unit 140 may find an appropriate effect in the effect storage unit 160. [ The effect storage unit 160 may be implemented as a software module that accesses a data storage unit provided in the smartphone 100. [

Since the human read aloud activity converts the string information into the form of sound waves, the effect of the idea of the present invention is generally executed in a time series and sequentially. Accordingly, the effect storage unit 160 stores the effects sequentially executed according to the order of execution, and the effect storage unit 160 searches for the appropriate effect information in the effect storage unit 160 quickly It is advantageous.

A concrete operation example of the speech recognition unit 120, the effect execution unit 140, and the effect storage unit 160 will be described as follows. If an effect is assigned to a specific word among the read aloud information, the voice recognition unit 120 checks whether the specific word exists in the recognized voice information (read aloud information). If the existence of the specific word is confirmed, The effect execution unit 140 searches the effect storage unit 160 for an effect assigned to the recognized specific word, and executes the searched effect. For example, a thunder sound effect may be assigned to a word 'thunder' during the assimilation, and a thunder sound may be output from the speaker of the smartphone 100 when the reader pronounces thunder.

According to an implementation, the sound effect providing system may further comprise an effect information server 300 for storing effects executed in accordance with the reading of the contents of the publication medium. The effect information server 300 may store a plurality of publishing media and / or a plurality of contents classified according to a predetermined classification rule, and effects executed according to each content read aloud. The smartphone 100 or the terminal device accesses the effect information server 300 via a network means such as the Internet, a wireless communication network, a LAN, a WAN, and the like, and effects the contents and / Read, or download.

On the other hand, since the text itself can occupy a considerable amount of information due to a large number of stories all over the world, the effect storage unit 160 stores each content and effect information included in each content It is advantageous that the effect information server 300 selectively download the necessary effect information.

For example, it is possible to access the effect information server related to the storybook with the smartphone 100 using the wireless data communication network, selectively download the effect information included in the necessary animated contents, and store the effect information in the effect storage 160 desirable. According to another embodiment, the effect information may be read out from the effect information server 300 in real time without the effect storage unit 160, and the effect may be performed.

The effect information is downloaded and stored in units of physical publishing media (for example, a story book) or a story (story content) if an optimum compromise is found in information and storage medium capacity, data network load, user convenience, It is advantageous. To this end, the user (reader) can conveniently have means for downloading the effect information in units of the publishing media or story contents unit. For example, when the QR code or the NFC tag included in the storybook 400 to be read by the reader is recognized by the smartphone 100, the automatic download module 180 may be configured to download the effect information necessary for the read aloud information of the storybook 400 And can be provided in the application. Alternatively, the user can recognize the barcode (this may be a service-dedicated barcode or a barcode for sale) for the assimilation book 400 as an input means such as the smartphone 100, The user can operate the automatic download module 180 in such a manner that the KC authentication number (registered in the book as a unique number assigned to the publication) or the KC authentication number (autonomous safety check report number) is input.

2 shows a sound effect providing system according to another embodiment of the present invention having a dedicated terminal device for effect execution according to the idea of the present invention.

The sound effect providing system shown in FIG. 2 is a publication medium on which information that can be read is recorded, a child / child story book 400; And a terminal device 200 for effecting effects according to the ideas of the present invention.

The terminal device 200 may be a mobile terminal (including a smart phone), a PDA, a tablet, a notebook, a personal computer (PC), a scanner type scanner (a personal information device with a pen having a scanner) , An e-book terminal for publishers, and the like.

The terminal device 200 is a hardware and / or software module, and includes a voice recognition unit 220 for recognizing information recorded on the publication medium read aloud; An effect execution unit (240) for judging whether an effect is designated to the recognized information, and for executing a specified effect when an effect is designated; And an effect storage unit 260 storing the effects that are sequentially executed according to the read aloud of the readable information.

The voice recognition unit 220 may include a microphone for receiving an external sound so as to determine whether the information recorded on the publication medium is voice information read by the user (read aloud). The voice recognition unit 220 can partially analyze and determine the recognized readable information (abbreviated as read aloud information).

The effect execution unit 240 can also apply visual effects and sensory effects such as vibration (touch) and smell, but it is advantageous to apply sound effect for low cost. For example, the effect execution unit 240 may include a speaker to output an effect sound assigned to each effect to be executed. Here, the effect execution unit 240 can find an appropriate effect in the effect storage unit 260. [ The effect storage unit 260 may include a data storage medium such as Flash RAM and EEPROM. It is advantageous for the effect execution unit 240 to quickly search for the effect information in the effect storage unit 260 that the effects sequentially executed in the effect storage unit 260 are stored according to the execution order .

The concrete operation examples of the speech recognition unit 220, the effect execution unit 240, and the effect storage unit 260 are similar to those in the case of FIG. 1, and thus a duplicated description will be omitted.

According to an implementation, the sound effect providing system may further comprise an effect information server 300 for storing effects executed in accordance with the reading of the contents of the publication medium. The effect information server 300 may store a plurality of publishing media and / or a plurality of contents classified according to a predetermined classification rule, and effects executed according to each content read aloud. The terminal device 200 accesses the effect information server 300 via network means such as the Internet, a wireless communication network, a LAN, a WAN, and reads out effects executed according to each contents and / or contents read- You can download it. For example, the terminal device 200 can access the effect information server 300 via the PC via the USB connection means, access the effect information server 300 through the built-in Wi-Fi module and the wireless LAN have.

On the other hand, since the text itself can occupy a considerable amount of information due to a large number of stories from all over the world, the effect storage unit 260 stores the contents and the effect information included in each contents And it is advantageous for the server 300 to selectively download necessary effect information.

The terminal device 200 may have a means by which a user (a reader) can conveniently download the effect information on the basis of the publishing media unit or the story content unit. For example, if the QR code or the NFC tag included in the storybook 400 to be read by the reader is recognized by the terminal 200, the automatic download module 280 may be configured to download the effect information necessary for the read aloud information of the storybook 400 And can be provided in the terminal device.

Depending on the implementation, the sound effect providing system shown in FIG. 1 or FIG. 2 may perform an acousto-off mode (omnidirectional mode) and an error response mode (universal trigger mode). That is, in order to meet the intention of the user (the reader) corresponding to the error temporarily generated in the sound effect output function according to the idea of the present invention or the other educational intention of the reader, the acoustic off mode trigger mode).

In the acoustical off mode, the automatic acoustical output is correctly turned off, and the acoustical output function remains on. However, the condition in which the predetermined effect sound is reproduced is not the sound of the corresponding word itself such as "Thunder" but the sound of the corresponding word + the sounds of the righteous words, such as "Thunder + Urrune".

That is, in this mode, the acoustic effect is not reproduced in a basic read-aloud state, so it is called an acoustic-off mode, and the acoustic effect is not reproduced unless the corresponding pronoun is attached after the word for which the sound effect is set.

Through the acoustical off mode, the user can selectively select and play the sound (for example, it is possible to reproduce only the word desired by the reader out of the total sound DB 100) .

When performing a sound effect output function according to the idea of the present invention, it is possible to temporarily generate sound effects for a specific word and / or a specific time (for example, a large noise passing train) due to the environment in which the reading is performed and the specific pronunciation / The effect may not be reproduced. In this case, re-reading of the part where the reader has not reproduced the sound effect may blur the atmosphere to both the reader and the listener.

If the sound effect is temporarily not reproduced as described above, the reader may generate a specific sound (e.g., a whistle sound) that is easy for the system to recognize, and the sound effect providing system may recognize the specific sound The next sound effect of the sound effect reproduced immediately before is reproduced. This mode can be easily applied when the sound effects are sequentially stored according to the story progress of the book.

FIG. 3 shows an embodiment of a server 300 for providing effect information according to the idea of the present invention to the smartphone 100 application of FIG. 1 or the terminal device 200 of FIG.

The illustrated server 300 includes a publication medium contents DB 320 for storing information about each of a plurality of publication media; An effect information DB 340 for storing effect information (for each content item) provided for each content, as effect information according to the idea of the present invention; According to the idea of the present invention, it is possible to receive information on a publication medium from the smartphone 100 application of FIG. 1 or the terminal apparatus 200 of FIG. 2, A service provider 360 provided to the terminal device 200; An authentication unit 370 for authenticating the smartphone 100 application of FIG. 1 or the terminal device 200 of FIG. 2 or for authenticating users thereof; A data communication unit 390 forming a smartphone 100 application of FIG. 1 or a wire / wireless data communication channel with the terminal device 200 of FIG. 2; And a service management unit 380 for performing subscriber management, charge management, and management of a service target publication medium with respect to a service for providing / executing effect information according to the ideas of the present invention.

The content DB 320 and / or the effect information DB 340 may be implemented as a SW module that manages a storage unit installed in the server 300, or may be implemented as a separate DB server .

The service providing unit 360 and the service managing unit 380 may be implemented in the form of firmware or application (application program) executed in a computing device (CPU or the like) of the server 300.

The authentication unit 370 may be implemented in the form of firmware or application (application program) executed in a computing device (CPU or the like) of the server 300, or may include hardware / software for authentication.

The authentication unit 370, the data communication unit 390, and the service management unit 380 can be implemented by applying techniques known in the art for subscriber authentication and subscriber management and data communication technologies for a server, I will not.

The service providing unit 360 confirms the automatic downloading modules 180 and 280 shown in FIG. 1 or 2 and provides the effect information on the content ID received from the automatic downloading modules 180 and 280. (E.g., E-book) according to the content ID.

Table 1 to Table 3 below illustrate an embodiment of the data storage structure of the content DB 320 and / or the effect information DB 340. According to the implementation, the table 1 may be stored in the contents DB 320, the tables 2 and 3 may be stored in the effect information DB 340, or the tables 1 and 2 may be stored in the contents DB 320 , And stores the table 3 in the effect information DB 340.

Tag ID Content ID title Electronic content address 2A1B 101 Sun and moon xxx.xxx.xxx.xxx AA78 101 Sun and moon xxx.xxx.xxx.xxx F126 174 Thor xxx.xxx.xxx.xxx ... ... ... ...

Table 1 is a table for providing effect information on the content ID received from the automatic download module 180 described above. For example, the tag ID may be information obtained when the QR code or the NFC tag included in the storybook 400 is recognized by the smartphone 100. The content ID may be a code capable of identifying the assimilation book 400. The title is the title of the storybook 400, and may be omitted according to the implementation. According to an embodiment of the present invention, electronic contents corresponding to the content ID (e.g., an e-book) may be provided together. In this case, in order to record information indicating the location of the memory or DB in which the electronic content is stored, And a content address field.

Effect ID Effect contents 424 storm.wav ... ... 428 thunder.wav ... ... 430 mem.wav

Table 2 above stores files necessary to implement effects according to the ideas of the present invention. In other implementations, information may be written indicating the location where the file is stored, rather than the file itself.

In Table 2, although a wave file (* .wav) is described as an effect means, it can be applied to other audio files such as WAV, WMA, MP3, OGG, and the like, or a video file. Further, it may be applied to a file format that outputs vibration, flash, and the like.

Effect Trigger
Effect ID
Expected Interval
Thunder
428
25 seconds
Cicadas
430
55 seconds
storm
424
40 seconds
...
...
...

Table 3 is a table describing effect information belonging to one piece of content defined by one assimilation book (i.e., a publication medium unit) or the content ID of Table 1 above. In Table 3, the order from top to bottom can be the order in which effects are executed according to story (read). The expected interval may be the expected interval from the time the previous effect is executed to the execution of the effect when the story is read aloud. In the case of the first effect, it may be the expected interval from the beginning of the narrative reading to the time the corresponding effect is executed.

Tables 2 and 3 are stored in the effect storage units 160 and 260 through downloading from the server 300 and the like. When recognizing the story aloud sound, the speech recognition units 120 and 220 can request the effect execution units 140 and 240 to execute the corresponding effect IDs, if the words corresponding to the effect triggers of Table 3 are recognized have. The effect execution units 140 and 240 can perform the effect by searching the table 2 for the effect ID requested to be executed and executing the effect file.

When the reader reads the "thunder " while reading the story, the speaker of the smartphone 100 of Fig. 1 or the speaker of the terminal 200 of Fig. 2 outputs a thunder sound, You can give.

Fig. 4 shows a reading assimilation book as an embodiment of a publication medium capable of executing effect information according to the ideas of the present invention.

It can be seen that in the storybook 400 shown as a publication medium, displays 421 and 422 are visually revealed to be effective on the part (word) to which the effect is designated among the readable information (story) . This is to note that the pronunciations should be pronounced more precisely because the pronunciator will have the effect of pronouncing the word.

As shown in the figure, an NFC tag 440 for identifying the storybook itself as the smartphone 100 of FIG. 1 or the terminal device 200 of FIG. 2 may be attached to the cover of the storybook 400. For example, when the NFC tag 440 is recognized by the smartphone 100 of FIG. 1, information such as the content ID of Table 1 may be obtained as a code capable of identifying the assimilation book 400.

Although not shown, other implementations may place QR codes or bar codes instead of NFC tags. For example, when recognizing the NFC tag as a camera module (or a pattern recognition module) included in the terminal device 200 of FIG. 2, as a code capable of identifying the assimilation book 400, Can be obtained. QR codes and bar codes are also useful in e-books. It goes without saying that the present invention can also be applied to a method of inputting an ISBN, a KC authentication number (autonomous safety confirmation report number), etc. as described above.

Although FIGS. 1 and 2 are mainly embodied as a print publication medium, the form of the print publication medium can be used in the form of an E-book, Of course, fall within the scope of the present invention. In this case, the E-book medium and the application may be implemented in one and the same smart device (e.g., tablet or smart phone). Alternatively, the E-book medium may be implemented in the form of an application for a smart device and a separate terminal device.

Although not shown, when a smartphone application and an E-book as a publication medium are installed together in one same smart phone or tablet, the method for identifying the content ID is as follows.

In the first method, when the user reproduces the corresponding E-book content, the content ID is transmitted to the E-book content to be reproduced while the smartphone application corresponding to the idea of the invention is executed by the E-book reproducing apparatus.

In the second method, a smartphone application execution key according to the idea of the present invention is displayed on the E-book reproducing screen. When the user touches the execution key, the smart phone application according to the idea of the present invention is executed, And transmits the content ID to the book content.

Except for identifying the content ID as described above, the E-book medium may implement the idea of this invention similar to a book (fairy tale) medium and the case.

It should be noted that the above-described embodiments are intended to be illustrative, not limiting. It will be appreciated that those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

100: smartphone 120: voice recognition unit
140: effect execution unit 160: effect storage unit
180: automatic download module 300: effect information server
320: publishing medium contents DB 340: effect information DB
360: service providing unit 370: authentication unit
380: service management unit 390: data communication unit
400: Children's Books

Claims (8)

A publication medium on which information that can be read is recorded;
A speech recognition unit for recognizing that the information recorded on the publication medium is read out;
A sound effect executing unit for determining whether a sound effect is designated to the recognized information, and executing a specified sound effect when the sound effect is designated;
A sound effect storage unit for storing effects that are sequentially executed as the readable information is read;
A mobile device having an automatic download module for recognizing a mark provided on the publication medium and downloading sound effect information corresponding to the readable information and storing the downloaded sound effect information in the sound effect storage; And
The mobile device can access the mobile device via the network means, and a sound effect server for storing sound effects executed according to the read-
Lt; / RTI >
The publication medium includes:
A stereoscopic reading system using at least one of a barcode, an ISBN, a KC authentication number (autonomous security check report number), a QR code, or an NFC tag that can be recognized by the mobile device and identifies the publication medium.
The method according to claim 1,
The sound effect server comprises:
A publication medium content DB for storing information related to each content for a plurality of publication media;
A sound effect information DB storing sound effect information designated for each of the contents; And
And a service providing unit for receiving the information about the publication medium from the mobile device and providing the sound effect information included in the content of the input publication medium to the mobile device.
3. The method of claim 2,
An authentication unit for authenticating the mobile device or the user;
A data communication unit forming a data communication channel with the mobile device; And
And a service management unit for performing subscriber management, charge management, and service management for the service providing the sound effect information, and managing the service target publication medium.
The method according to claim 1,
The mobile device is a stereoscopic reading system using a mobile device that is any one of a mobile phone, a PDA, a tablet, a notebook, a personal computer (PC), a scanner terminal, an education terminal, or an E-
The method according to claim 1,
Wherein the publishing medium displays an effect on a portion to which the sound effect is designated among the readable information.
The method according to claim 1,
Wherein the predetermined word or words are the corresponding monolingual word for the word to which the sound effect is assigned.
The method according to claim 1,
The sound effect execution unit plays back the sound effect set on the designated word if the sound effect recognizes that the sound effect is not reproduced in the basic read aloud state and that the predetermined word or words are read before or after the designated word A stereoscopic reading system using a mobile device including an acoustic off mode.
The method according to claim 1,
Wherein the sound effect execution unit includes an error correspondence mode for executing a sound effect next to the sound effect immediately preceding the sound effects sequentially stored in the publication medium when the predetermined specific word is recognized Three-dimensional reading system.

Images