KR102199007B1 - System and method for supporting reading by linking additional content to book - Google Patents

Abstract

The present invention relates to a system and a method for reading assistance. With the present invention, book object-related interactive content containing user voice is output in conjunction with the location of an interaction object. The system includes a reading assistance device. The reading assistance device detects the location of the interaction object as coordinates in an activity area image by analyzing the activity area image including the interaction object, detects the interactive content linked to the location of the interaction object detected by an image processing engine and the activity area image, replaces at least a part of the voice of the interactive content with user voice, outputs the interactive content so as to overlap the interaction object of the activity area image by outputting the interactive content at the location of the interaction object in the activity area image, and outputs the sound of the interactive content containing the user voice.

Description

Reading support system and method linking book objects and interactive contents {SYSTEM AND METHOD FOR SUPPORTING READING BY LINKING ADDITIONAL CONTENT TO BOOK}

The present invention relates to a reading support system and method, and more particularly, to a reading support system and method for supporting a user's reading by providing additional content related to a real book.

In general, reading is an act to acquire knowledge by reading books including text and images. In recent years, as people become accustomed to electronic devices such as smartphones and tablets, the frequency of acquiring knowledge through Internet surfing and Internet video viewing has increased rather than reading real books composed of only text and images (ie, still images). have.

In the publishing industry, real books are replaced with multimedia books composed of text, images, voices, sound effects, and videos, but conventional multimedia books simply provide audio and video related to the contents of the book. That's enough to provide.

Accordingly, the publishing industry is researching various reading support technologies to induce active reading and active reading participation.

The matters described in the background art are intended to help understanding the background of the invention, and may include matters not known as conventional technology to those of ordinary skill in the field to which this technology belongs.

Korean Patent Registration No. 10-1587471 (Name: book system and reading enhancement method for enhancing reading using a wearable computer)

The present invention is proposed in consideration of the above circumstances, and provides a reading support system and method in which interactive content related to a book object is output in conjunction with the location of an interactive object, but the interactive content including the user's voice is output. It is aimed at.

In order to achieve the above object, the reading support system according to an embodiment of the present invention includes a reading support device disposed on a physical surface having an activity area on which an interactive object is disposed, and having at least one processor, and the reading support device Image processing that receives an activity area image including an activity area and one or more interaction objects arranged in the activity area, analyzes the activity area image, and detects the location of the interactive object, which is the coordinates within the activity area image as a result of the image analysis. The engine is connected to the image processing engine for communication, detects the interactive content associated with the active area image and the position of the interactive object detected by the image processing engine, and an output signal including the position of the interactive object and the interactive content A processor that outputs, is connected to an image processing engine and processor for communication, receives a user voice, and replaces at least some of the voice of the interactive content with a user voice, an image processing engine, a processor, and It is connected to the voice processing engine, outputs the active area image, and outputs the interactive content based on the coordinates corresponding to the position of the interactive object in the activity area image in response to the output signal to overlap the interactive object of the activity area image. It includes a display for outputting interactive content and a speaker for outputting sound of the interactive content including a user's voice.

The image processing engine may further detect an interactive object area in which an interactive object is displayed as a result of image analysis on the active area image, and detect a partial area of the interactive object area as an interactive content output area. At this time, the processor outputs an output signal including the interactive content output area and the interactive content, and the display outputs the interactive content to an area corresponding to the interactive content output area among the active area images, and the interaction object of the active area image Interactive content can be output so that it overlaps with a part of.

A reading support system according to an embodiment of the present invention is a camera configured to capture an active area image including an active area of a physical surface and one or more interactive objects disposed in the active area, and transmit the active area image to an image processing engine and a display. It may further include.

The reading support system according to an embodiment of the present invention further includes a microphone configured to receive a user voice and transmit the user voice to the voice processing engine, and the display displays characters or sentences corresponding to the user voice while driving the microphone. can do.

In order to achieve the above object, the reading support method according to an embodiment of the present invention includes outputting an active area image including an active area and one or more interactive objects arranged in the active area by a display, through an image processing engine. , Analyzing the image of the active area, and detecting the position of the interactive object, which is the coordinates in the image of the active area, as a result of the image analysis, and detecting the interactive content linked to the position of the active area image and the interactive object by the processor The step of, receiving a user voice by a voice processing engine, and replacing at least some of the voice of the interactive content with the user voice, outputting an output signal including the location of the interactive object and the interactive content by the processor In response to the output signal, outputting the interactive content based on the coordinates corresponding to the position of the interactive object in the active area image by display, and outputting the interactive content so as to overlap the interactive object of the active area image And outputting the sound of the interactive content including the user's voice by the step and the speaker.

The reading support method according to an embodiment of the present invention includes the steps of detecting, by an image processing engine, an interactive object area in which an interactive object is displayed as a result of image analysis on an active area image, and by the image processing engine, the interactive object area It may further include the step of detecting a partial region of the interactive content output region.

In the reading support method according to an embodiment of the present invention, outputting an output signal including an interactive content output area and an interactive content by a processor, and outputting an interactive content from an active area image in response to the output signal by a display The method may further include outputting the interactive content to overlap with a part of the interactive object of the active area image by outputting the interactive content to the area corresponding to the area.

The reading support method according to an embodiment of the present invention may further include capturing an activity area image including the activity area and one or more interaction objects arranged in the activity area through a camera.

The reading support method according to an embodiment of the present invention includes receiving a user voice that replaces at least a part of the voice of an interactive object by a microphone, and a character corresponding to the user voice before receiving the voice by a display. Alternatively, it may further include displaying a sentence.

According to the present invention, the reading support system and method output interactive content related to the book object in conjunction with the location of the interactive object changed by the user, so that the user can read compared to the prior art, which provides a predetermined additional image at a predetermined time. It has the effect of providing an environment where you can focus on.

In addition, the reading support system and method has the effect of inducing active reading participation of the user using the interactive object by outputting the interactive content related to the book object in connection with the location of the interactive object changed by the user. .

In addition, the reading support system and method induces active reading compared to the prior art, which outputs the interactive content at a preset location by outputting the interactive content related to the book object in conjunction with the location of the interactive object changed by the user. There is an effect that can be done.

In addition, the reading support system and method has an effect of inducing active participation in reading by outputting an interactive object including the user's voice.

The technical effects of the present invention as described above are not limited to the effects mentioned above, and other technical effects that are not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a view for explaining a reading support system according to an embodiment of the present invention.
2 is a view for explaining the reading support device of FIG.
3 and 4 are views for explaining a cradle for using the reading support device of FIG. 1;
5 and 6 are diagrams for explaining an adapter for using the reading support device of FIG. 1;
7 is a view for explaining the configuration of the reading support device of FIG.
8 to 14 are views for explaining the operation of the reading support device of FIG.
15 to 17 are views for explaining an operation of the reading support device of FIG. 1 outputting interactive content in which a user's voice is set.
FIG. 18 is a diagram for explaining the configuration of the service server of FIG. 1;
19 and 20 are diagrams for explaining the operation of the service server of FIG. 1;
Fig. 21 is a diagram for explaining the configuration of the database of Fig. 1;
Fig. 22 is a diagram for explaining the book identifier storage of Fig. 21;
Fig. 23 is a diagram for explaining the book information storage shown in Fig. 21;
FIG. 24 is a diagram for explaining the reading support information storage of FIG. 21;
25 to 27 are flow charts for explaining a reading support method linking a book object and interactive content according to an embodiment of the present invention.
28 is a view for explaining an example of outputting interactive content in which a user's voice is set in a reading support method according to an embodiment of the present invention.

Hereinafter, in order to describe in detail enough that a person having ordinary knowledge in the technical field of the present invention can easily implement the technical idea of the present invention, a most preferred embodiment of the present invention will be described with reference to the accompanying drawings. . First of all, in adding reference numerals to elements of each drawing, it should be noted that the same elements have the same numerals as possible even if they are indicated on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Those of ordinary skill in the art will appreciate that various exemplary blocks, devices, or operations described in connection with the configurations disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. . These blocks, devices, or operations include a processor, digital signal processor (DSP), application-specific integrated circuit (ASIC) or application-specific standard product (ASSP), field-programmable gate array (FPGA) or other programmable logic device, or It may be implemented or performed using gate or transistor logic, individual hardware components, or any combination thereof designed to create a configuration as disclosed herein. The processor may be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors associated with a DSP core, or any other configuration. Computer software or programs include random access memory (RAM), read-only memory (ROM), non-volatile RAM such as flash RAM, erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), registers, and hard drives. It may reside on a disk, removable disk, CD-ROM, or any other type of storage medium known in the art, and may contain machine-readable instructions that can be executed by an array of logical elements such as a processor or DSP. . An exemplary storage medium may be associated with the processor, and the processor may read information from and write information to the storage medium. Alternatively, the storage medium may be integral with the processor. The processor and storage medium can be located within the ASIC. The ASIC can be located in the user's terminal. Alternatively, the processor and storage medium may exist as separate components within the user terminal.

Referring to FIG. 1, a reading support system according to an embodiment of the present invention includes a reading support device 100, a service server 200, and a database 300. Here, in FIG. 1, the service server 200 and the database 300 are shown as being separated in order to easily describe the embodiment of the present invention, but the service server 200 and the database 300 are not limited thereto. The server may be configured, or the service server 200 and the database 300 may be implemented in the form of a module in the reading support apparatus 100.

The reading support device 100 is composed of a terminal having a photographing function. As an example, the reading support device 100 is a smart phone, a tablet, a laptop, etc. equipped with a camera 130 capable of capturing an image. The reading support device 100 is connected to the service server 200 and the database 300 through a network to transmit and receive various pieces of information for providing a reading support service.

Referring to FIG. 2, the reading support device 100 photographs an activity surface (AS) on which a book object (BO) is disposed through a camera 130. Here, the active area AS means a partial area of the physical surface on which the reading support device is disposed.

The reading support device 100 outputs an activity area image (ASV) including book contents (BC) using the display 140. Here, the activity area AS is a partial area of the bottom surface on which the reading support device 100 and the book object BO are disposed, and is an area including an area where the book object BO is disposed. The active area image ASV is an image captured by the camera 130 of the active area AS. The book object BO is a real book, and the book content BC is the content displayed on the display 140, and includes text, images, etc. included in each page of the book object BO.

When an interactive object (IO) is recognized on the activity area image ASV, the reading support device 100 displays interactive contents (IC) corresponding to the recognized interactive object IO ( 140) to print.

In this case, the reading support device 100 outputs the interactive content IC by reflecting the position of the interactive object IO. That is, the reading support device 100 detects the coordinates of the display area of the interactive object IO. The reading support device 100 outputs the interactive content IC using the display 140, but outputs the interactive content IC at a location corresponding to the display coordinates of the interactive object IO. Here, the display area coordinates mean an area in which the interactive object IO is displayed among the entire area of the display 140.

When the user holds the reading support device 100, the reading support device 100 may shake or move. In this case, since the reading support device 100 photographs only a part of the active area AS, the book object BO in which some of the text and images of the book object BO are omitted is output on the display 140 or the active area Blur occurs in the image ASV, making image recognition impossible.

Accordingly, referring to FIGS. 3 and 4, the reading support device 100 photographs the active area AS while being mounted on the cradle 400.

In general, the reading support device 100 is equipped with a camera 130 having a standard angle of view (40 to 50 degrees). In order to photograph all the active areas AS with the camera 130 having a standard angle of view, the camera 130 and the active area AS must be arranged to be spaced apart by a predetermined distance or more.

Accordingly, the cradle 400 is composed of a main body 410 and a support 420, and the main body 410 includes a fixing device 412 for supporting the lower portion of the reading support device 100. The fixing device 412 is disposed at a position spaced apart from the activity area AS by a set height H or more, so that the cradle 400 connects the camera 130 of the reading support device 100 to the activity area AS and a set distance. It is spaced apart so that the camera 130 can photograph all of the active areas AS.

Here, the reading support device 100 has a front surface on which the display 140 is disposed, a rear surface facing the front, and four sides, and the lower portion of the reading support device 100 is a cradle ( When mounted on 400), it means the side that is arranged in the direction of the activity area (AS). Since the set height H may vary according to the configuration of the reading support device 100, the value of the set height H is not separately limited in the embodiment of the present invention.

In addition, in order to enable the camera 130 of the reading support device 100 to capture all of the active area AS, the main body 410 and the support 420 are coupled to rotate around the coupling shaft 430 do. The reading support device 100 is mounted on the front surface of the main body 410, and the support 420 rotates in the rear direction of the main body 410 about the coupling shaft 430. In this case, the angle θ between the main body 410 and the support 420 has an angle within a set angle range in order to secure a field of view capable of capturing all of the active area AS.

Here, the angle is an angle between the body 410 and the support 420 in a state in which the support 420 rotates and is disposed in the rear direction of the body 410. Since the setting angle may vary depending on the structure of the reading support device 100, the main body 410, and the support 420, the numerical value of the setting angle is not separately limited in the embodiment of the present invention.

The reading support device 100 photographs the active area AS using a camera 130 (ie, a front camera) that photographs the front direction in order to allow the user to view the active area image ASV. In general, since the camera 130 of the reading support device 100 is installed to photograph the front direction, it is impossible to photograph the active area AS.

Accordingly, the reading support device 100 photographs the active area AS while the adapter 500 is mounted. The adapter 500 includes a housing 520 and a reflector 540 disposed inside the housing 520.

The housing 520 may be formed of a material having elasticity in order to maintain a fastening force while preventing damage to the reading support device 100 when it is coupled to the reading support device 100. The housing 520 is mounted on the upper portion of the reading support device 100, and is mounted so as to overlap with the camera 130 disposed in front of the reading support device 100. Here, the upper portion of the reading support device 100 refers to a side of the four sides of the reading support device 100 that faces the side disposed in the direction of the activity area AS when mounted on the cradle 400.

The reflector 540 is disposed inside the housing 520 to reflect the active area AS toward the camera 130 of the reading support device 100. The reflector 540 forms an acute angle with the front surface of the reading support device 100 inside the housing 520 and is disposed to face the camera 130 of the reading support device 100.

Referring to FIGS. 5 and 6, the adapter 500 is fitted and coupled to an area in which the camera 130 of the reading support device 100 is disposed. In this case, the adapter 500 blocks another area from being photographed by the camera 130 through the housing 520 and reflects the active area AS to the camera 130 through the reflector 540.

On the other hand, the reading support device 100 may be formed with a guide device that guides the adapter 500 to be mounted in an accurate position. The guide device is disposed on the reading support device 100 and may be formed in one or a combination of two or more of various shapes such as protrusions and grooves. In this case, the guide device may be integrally formed with the housing 520 of the reading support device 100. When the housing 520 of the adapter 500 is coupled with the reading support device 100, a protrusion, a groove, etc. having a shape corresponding to the guide device may be formed in a region overlapping with the guide device.

The reading support device 100 photographs the activity area AS in which the book object BO is disposed through the adapter 500 while being mounted on the cradle 400. The reading support device 100 outputs an interactive content IC linked to the book content BC by using the activity area image ASV generated as a result of the photographing. In this case, the reading support device 100 outputs the interactive content IC so as to overlap (overwrite) the active area image ASV.

Here, the interactive content (IC) is a content that outputs an additional video to a real-time video, such as augmented reality (AR) content, mixed reality (MR) content, and XR (eXtended Reality) content. Take that as an example. At this time, the reading support device 100 outputs interactive content (IC) including a user's voice.

Referring to FIG. 7, the reading support device 100 includes an application 110, a processor 120, a camera 130, a display 140, an image processing engine 150, a first communication module 160, and a storage ( 170), a memory 180, a microphone 191, a voice processing engine 193, and a speaker 195. Application 110, processor 120, camera 130, display 140, image processing engine 150, first communication module 160, storage 170, memory 180, microphone 191, The voice processing engine 193 and the speaker 190 receive or output signals and data through a bus.

The user drives the application 110 installed in the reading support device 100 to use the reading support service. That is, after driving the reading support device 100, the user executes the application 110 supporting the reading support service through manipulations such as touch, click, and double click.

The application 110 is activated by the user's execution and outputs a camera drive request signal to the processor 120. The processor 120 outputs a recording start signal to the camera 130 in response to the camera driving request signal from the application 110. The camera 130 starts shooting of the active area AS in response to the shooting start signal of the processor 120 and transmits the Ack signal to the application 110. The camera 130 generates an active area image ASV photographing the active area AS, and transmits it to the display 140 in real time. The display 140 outputs an active area image (ASV) transmitted from the camera 130 in real time.

In response to the Ack signal of the camera 130, the application 110 transmits a guide output signal to the processor 120 for capturing the cover of the book object BO, which is a reading support service. The processor 120 transmits a cover photographing guide output signal to the display 140 in response to the Ack signal of the camera 130 and the cover guide output signal of the application 110. The display 140 outputs the cover photographing guide CSG to overlap (overwrite) the active area image ASV in response to the guide output signal.

Referring to FIG. 8, the display 140 includes a horizontal straight line (HL) and a vertical straight line (VL) connecting the screen to the center point, and a cover photographing guide (CSG) including a cover recognition area in an active area image (ASV). Output by overlapping (overwriting).

Referring to FIG. 9, when a user places a book object BO such that a book object BO is positioned in a cover recognition area output from the display 140, the camera 130 is an area where the book object BO is placed. Is photographed, and the display 140 outputs an image captured by the camera 130 (hereinafter, an active area image (ASV)).

The image processing engine 150 performs image recognition on an area corresponding to the cover recognition area of the cover photographing guide CSG among all areas of the active area image ASV. The image processing engine 150 generates a cover image as a result of image recognition, and the memory 180 temporarily stores the cover image generated by the image processing engine 150.

The processor 120 generates a book identification request signal including a cover image stored in the memory 180 and transmits it to the first communication module 160. The first communication module 160 transmits the received book identification request signal to the service server 200 through a network.

The first communication module 160 receives a book identification response corresponding to the book identification request signal. The first communication module 160 receives a book identification response including book information corresponding to the book object BO from the service server 200. At this time, the first communication module 160 receives book information including book name and interactive object information from the service server 200, and the interactive object information is an interactive object (IO) used when reading support for books As an example, including the interactive object image (IOI) of.

When the cover image is not the cover of the book object (BO) or is not the book object (BO) registered in the service server 200, the first communication module 160 sends a book identification response including a book recognition error message. You can also receive. The first communication module 160 transmits the book identification response received from the service server 200 to the processor 120.

When a book recognition error message is included in the book identification response, the processor 120 transmits a guide output signal to the display 140. The display 140 outputs the cover photographing guide CSG to overlap (overwrite) the active area image ASV in response to the guide output signal. In this case, the display 140 may output a replacement guide for the book object BO, a location movement guide, etc. together with the cover photographing guide CSG.

When book information is included in the book identification response, the processor 120 transmits a book information output signal to the display 140. In response to the book information output signal, the display 140 outputs the book information and the service start confirmation object of the book object BO to overlap (overwrite) the activity area image ASV.

When the user selects a service initiation confirmation object from among the service initiation confirmation objects displayed on the reading support device 100, the application 110 transmits the reading support service initiation confirmation object to the processor 120.

The processor 120 detects book information from the book identification response in response to the reading support service start confirmation object. The processor 120 generates a reading support information signal including the detected book information and transmits it to the first communication module 160. The first communication module 160 transmits the received reading support information signal to the service server 200 through a network.

The first communication module 160 receives reading support information corresponding to the reading support information signal. At this time, the first communication module 160 includes a page image for page recognition of the book object (BO), an interactive object image (IOI) linked to the page of the book object (BO), and a page or interaction of the book object (BO). Reading support information including an additional video matched with the action object image IIO is received and transmitted to the processor 120.

The processor 120 stores the reading support information transmitted through the first communication module 160 in the storage 170. At this time, the storage 170 maintains the storage state of the reading support information corresponding to the book object BO until the reading support of the book object BO is completed.

Meanwhile, since the reading support information includes images, images, etc., it takes a certain time to receive (download) from the service server 200. Accordingly, the processor 120 transmits, to the display 140, an interactive object information output signal that signals the preparation of the interactive object IO included in the book information.

As shown in FIG. 10, the display 140 outputs an interactive object preparation request screen including interactive object information included in the book information in response to the interactive object information output signal. At this time, the display 140 detects the interactive object image (IOI) from the book information included in the book identification response, and displays the interactive object preparation request screen in which the detected interactive object image (IOI) is arranged according to a predetermined condition. Print. Through this, the display 140 outputs an interactive object preparation request screen until reception of the reading support information is completed.

When reception of the reading assistance information is completed, the processor 120 detects the interactive content IC based on the active area image ASV and the page image. That is, the processor 120 detects a page of the book object BO by comparing the active area image ASV captured by the camera 130 with the page image of the reading support information. The processor 120 transmits a page detection signal including the detected page to the storage 170.

The storage 170 detects a page image corresponding to a page included in the page detection signal from among pre-stored reading support information. The storage 170 detects reading support information including the detected page image. Here, the page image may be information included in the page, such as a specific object image included in the page, a page number, and text, as well as an image of the page.

The storage 170 transmits the detected reading support information to the processor 120, and the processor 120 detects an additional video included in the transmitted reading support information as an interactive content (IC). The processor 120 transmits an interactive content output signal including the detected interactive content IC to the display 140.

As shown in FIG. 11, the display 140 outputs the interactive content IC to overlap (overwrite) the active area image ASV in response to the interactive content output signal of the processor 120. That is, the display 140 outputs an active area image (ASV) of the book object BO photographed by the camera 130 in real time, and the interactive content (IC; that is, an additional video) included in the interactive content output signal ) Is output to overlap (overwrite) the active area image (ASV). In this case, the display 140 may output the interactive content IC on the entire screen or may output the interactive content IC on a part of the screen on which the active area image ASV is output.

On the other hand, when the detected reading support information includes the interactive object image (IOI), the processor 120 detects the interactive object image (IOI). The processor 120 transmits an interactive object output signal including the detected interactive object image IIO to the display 140.

As shown in FIG. 12, the display 140 outputs the interactive object image IIO to overlap (overwrite) the active area image ASV in response to an interactive object image output signal of the processor 120. . That is, the display 140 outputs an active area image (ASV) captured by the camera 130 in real time, and transfers the interaction object image (IOI) included in the interaction object image output signal to the activity area image (ASV). Output to overlap (overwrite). In this case, the display 140 outputs an interactive object image IIO by adding a video effect such as blinking.

The user views the interactive object image IIO output on the display 140 and then places the interactive object IO corresponding to the interactive object image IIO on the upper part of the book object BO. Accordingly, the active area image ASV in which the interactive object IO is disposed on the book object BO is output on the display 140.

The image processing engine 150 detects the interaction object IO from the active area image ASV. That is, the image processing engine 150 detects the type of interaction object and the location of the interaction object from the active area image ASV, generates interaction object detection information including these, and transmits it to the processor 120. Here, the interaction object location is a location where the interaction object IO is detected in the activity area image ASV, and means a location on the area in which the activity area image ASV is output.

The processor 120 generates a reading support information detection signal including an active area image (ASV) and interactive object detection information. At this time, the processor 120 generates a reading support information detection signal including an active area image (ASV) captured by the camera 130 and an interaction object type (shape) and transmits it to the storage 170.

The storage 170 first detects reading support information including a page image corresponding to the activity area image ASV of the interactive content IC among the previously stored reading support information. The storage 170 secondly detects reading support information corresponding to the interactive object type of the interactive content IC from among the first detected reading support information. The storage 170 transmits the second detected reading support information to the processor 120.

The processor 120 detects the additional video included in the transmitted reading support information as interactive content (IC). The processor 120 transmits, to the display 140, an interactive content output signal including the detected interactive content IC and the interaction object position of the interactive object detection information.

As illustrated in FIG. 13, the display 140 outputs the interactive content IC at a location where the interactive object IO is disposed in response to an interactive content output signal of the processor 120. That is, the display 140 detects the location of the interactive object from the interactive content output signal. The display 140 outputs the interactive content IC at a position corresponding to the detected interactive object position among regions in which the active area image ASV is output. Through this, the display 140 outputs the interactive content IC so as to overlap (overwrite) the interactive object IO of the active area image ASV.

Referring to FIG. 14, even after the interactive content IC is output, the image processing engine 150 detects the location of the interaction object from the active area image ASV and transmits it to the processor 120.

The processor 120 determines whether to move by comparing the location of the interaction object detected by the image processing engine 150 with the location of the interaction object detected immediately before. In this case, when the detected interaction object position is different from the previously detected interaction object position, the processor 120 determines the movement of the interaction object IO.

If it is determined that the interaction object IO is moved, the processor 120 transmits an interactive content movement request signal including the location of the interaction object to the display 140. The display 140 outputs the interactive content IC on the active area image ASV based on the location of the interaction object in the interactive content movement request signal. Through this, even if the location of the interactive object IO is changed, the reading support device 100 may output the interactive content IC so as to overlap (overwrite) the interactive object IO.

The reading support device 100 may output the interactive content IC in a partial area of the entire area of the interactive object IO.

The image processing engine 150 detects the interactive object region through image recognition (analysis) of the active region image ASV including the interactive object IO. The image processing engine 150 performs image recognition (analysis) on a region corresponding to the interactive object region among the active region images ASV, and detects the position of the reference symbol as a result of image recognition (analysis). In this case, the image processing engine 150 may detect an area preset with the position of the reference symbol as a reference (center) as the interactive content output area. Through this, the image processing engine 150 detects a partial area of the entire area of the interactive object IO detected from the active area image ASV as the interactive content output area.

The processor 120 detects an additional video included in the reading support information transmitted from the service server 200 as an interactive content (IC). The processor 120 transmits an interactive content output signal including the detected interactive content IC and an interactive content output area to the display 140.

The display 140 outputs the interactive content IC at a location where the interactive object IO is disposed in response to an interactive content output signal of the processor 120. That is, the display 140 detects the interactive object output area from the interactive content output signal. The display 140 outputs the interactive content IC in an area corresponding to the detected interactive object output area among areas in which the active area image ASV is output. Through this, the display 140 outputs the interactive content IC to a partial area of the interactive object IO displayed on the active area image ASV.

The reading support device 100 outputs interactive content (IC) including a user's voice. The reading support device 100 outputs interactive content (IC) in which some of the voices (sounds) are set (replaced) as user voices.

The microphone 191 receives a user's voice. The microphone 191 receives a user voice, which is a sound generated when the user pronounces a predetermined sentence or word. In this case, the display 140 may request input of a user's voice by outputting a screen requesting pronunciation of a preset sentence or word. Here, as an example, the preset sentence or word corresponds to one of sounds included in the interactive content IC.

The voice processing engine 193 stores the user voice input through the microphone 191 in a storage (or memory). The voice processing engine 193 replaces some of the sounds (sounds) included in the detected interactive content IC with the user's voice. The voice processing engine 193 replaces a sound corresponding to a preset sentence or word among sounds of the interactive content IC with a user voice.

The display 140 outputs an image of the interactive content IC, and the speaker 195 outputs a sound (sound) of the interactive content IC including a user's voice.

Referring to FIG. 15, the reading support device 100 outputs a user's voice at a specific point in time T1 of the interactive content IC. As an example, the reading support device 100 outputs interactive content (IC), which is a pansori image, and outputs a user's voice “Eolsu” at a specific time.

As an example, as shown in FIG. 16, it is assumed that the interactive object IO is in the shape of a TV and the interactive content IC is an image in which person A and person B talk to each other or sing together.

The image processing engine 150 detects an area A1 corresponding to a liquid crystal (Brown tube) in the shape of a TV as an interactive content (IC) output area. In this case, the image processing engine 150 detects an area A1 having an area set around the symbol S1 as an interactive content (IC) output area. The display 140 outputs the interactive content IC so as to overlap the area A1.

The voice processing engine 193 replaces the voice of the person A in the interactive content IC with the voice of the user input through the microphone 191. The speaker 195 outputs a conversation or song between a person A as a user's voice and a person B as a preset voice.

In addition, the display 140 may output interactive content (IC) including person A and person B, but may convert the person A's face into a previously input user face image and output it.

As another example, referring to FIG. 17, when the interactive object IO has a lip shape, the voice processing engine 193 detects a user voice input through a microphone from a memory (or storage), and the speaker 195 Outputs the user voice detected by the voice processing engine 193. At this time, when the interaction object IO in the form of an alien is added, the speaker 195 outputs an alien voice that the alien follows the user voice or song along with the user voice.

The service server 200 is configured as a server that provides a reading support service. The service server 200 detects book information, interactive object information, interactive content (IC), etc. from the database 300 according to a signal from the reading support device 100, and uses the detected information to the reading support device 100 Send to.

Referring to FIG. 18, the service server 200 provides a book identifier from the database 300 in response to a request for detecting a book identifier from a second communication module 210 that communicates with the reading support device 100 and the reading support device 100. A book identification engine 220 that detects, a book information detection engine 230 that detects book information from the database 300 based on a book identifier, a reading support information detection engine that detects reading support information based on the book identification information (240) And a control engine 250 that controls the second communication module 210, the book identification engine 220, the book information detection engine 230, and the reading support information detection engine 240 to provide a reading support service Consists of including. Here, the control engine 250, the book identification engine 220, the book information detection engine 230, and the reading support information detection engine 240 may be configured as one processor 120 and operated.

Referring to FIG. 19, the second communication module 210 receives a book identification request signal from the reading support device 100. The second communication module 210 receives a book identification request signal including a cover image of the book object BO and transmits it to the control engine 250.

The control engine 250 detects the cover image from the transmitted book identification request signal. The control engine 250 generates a book identifier detection signal including the detected cover image and transmits it to the book identification engine 220.

The book identification engine 220 detects the book identifier from the cover image in response to the book identifier detection signal. That is, the book identification engine 220 detects the cover image from the book identifier detection signal transmitted from the control engine 250. The book identification engine 220 detects an object image included in the cover image through image recognition of the detected cover image. The book identification engine 220 may detect an object image composed of characters, such as a book name, a publisher, a book group name, etc. included in the cover image through OCR.

The book identification engine 220 generates query data including the detected object image and transmits it to the database 300. Here, as an example, the object image includes at least one of a book name, a cover composition image, a publisher, and a book group name included in the cover image.

The book identification engine 220 receives a query response from the database 300 in response to the query data. At this time, the book identification engine 220 receives a query response including book information including a book identifier. Here, as an example, the book identifier is a unique identifier assigned to each of the book objects (BO) provided by the reading support service.

The book identification engine 220 detects the book identifier from the query response received from the database 300 and transmits the detected book identifier to the control engine 250. The control engine 250 generates a book information detection request signal including the received book identifier and transmits it to the book information detection engine 230.

The book information detection engine 230 detects book information from the database 300 in response to a request signal for detecting book information from the control engine 250. The book information detection engine 230 detects a book identifier from the book information detection request signal. The book information detection engine 230 generates query data including the detected book identifier and transmits it to the database 300.

Here, in FIG. 21, it has been described that the book information detection engine 230 generates query data using the book identifier detected through image recognition of the book identification engine 220, but is not limited thereto, and the cover image is used as query data. Book information can also be detected. In this case, the book identification engine 220 may be excluded from the configuration of the service server 200.

The book information detection engine 230 receives a query response from the database 300 in response to the query data and transmits it to the control engine 250. At this time, the book information detection engine 230 receives a query response including book information including book name and interactive object information. Here, as an example, the interactive object information includes an interactive object image (IOI) of the interactive object (IO) used when supporting reading for a book.

The control engine 250 detects book information from a query response transmitted from the book information detection engine 230. At this time, the control engine 250 detects book information including book name and interactive object information from the query response. The control engine 250 generates a book identification response including the detected book information, and transmits a book identification response signal including the book identification response to the second communication module 210.

The second communication module 210 detects a book identification response from the book identification response signal. The second communication module 210 transmits a book identification response to the reading support device 100 that has transmitted the book identification request signal.

On the other hand, when the cover image is not the cover of the book object (BO) or is not the book object (BO) registered in the service server 200, the book information detection engine 230 does not have a query response to the query data, Query responses without detection results can be received. In this case, the control engine 250 generates a book identification response including a book recognition error message, and the second communication module 210 sends a book recognition error message to the reading support device 100 that has transmitted the book identification request signal. Send a book identification response containing.

Referring to FIG. 20, the second communication module 210 receives a reading support information signal from the reading support device 100. The second communication module 210 receives a reading support information signal including book information and transmits it to the control engine 250.

The control engine 250 detects book information from the transmitted reading support information signal. The control engine 250 generates a reading support information detection signal including the detected book information and transmits it to the reading support information detection engine 240.

The reading assistance information detection engine 240 detects interactive content information from the database 300 in response to a reading assistance information detection signal from the control engine 250. The reading support information detection engine 240 detects book information from the reading support information detection signal, and detects a book identifier from the detected book information. The reading support information detection engine 240 generates query data including the detected book identifier and transmits it to the database 300. The reading support information detection engine 240 receives a query response in response to the query data from the database 300 and transmits it to the control engine 250.

At this time, the reading support information detection engine 240 includes a page image for recognizing a page of the book object BO, an interactive object image IIO linked to the page of the book object BO, a page of the book object BO, or A query response including reading support information including an additional video matched to the interactive object image (IOI) is received.

The control engine 250 detects reading support information from the query response received from the reading support information detection engine 240. The control engine 250 generates a reading support information transmission signal including the detected reading support information and transmits it to the second communication module 210.

The second communication module 210 detects reading support information from the reading support information transmission signal. The second communication module 210 transmits reading support information to the reading support device 100 that has transmitted the reading support information signal.

The database 300 is composed of a server that stores various information for providing a reading support service. The database 300 stores book information, interactive object information, and interactive content (IC), and detects information corresponding to a signal from the service server 200 and transmits it to the service server 200.

Referring to FIG. 21, the database 300 is a query processing engine that generates a query response including information detected from the storage 170 based on the query data transmitted from the service server 200 and transmits the query response to the service server 200. (310), a book identifier storage 320 that stores a book identifier linked to a book object (BO), a book information storage 330 that stores book information associated with a book identifier, and reading support information associated with the book identifier It is configured to include the stored reading support information storage 340.

The query processing engine 310 detects the book identifier from the book identifier storage 320 based on the query data, generates a query response including the detected book identifier, and transmits it to the service server 200.

The query processing engine 310 receives query data including an object image of the cover image. At this time, the query processing engine 310 receives query data including an object image including one or more of a book name, a cover composition image, a publisher, and a book group name included in the cover image.

The query processing engine 310 detects an object image from the query data. The query processing engine 310 detects the book identifier from the book identifier storage 320 based on the object image. The query processing engine 310 detects a book identifier associated with the detected object image. The query processing engine 310 generates a query response including the detected book identifier and transmits it to the service server 200.

As an example, referring to FIG. 22, the book identifier storage 320 stores the book identifiers assigned to each of the book objects (BO) for which the reading support service is available, and the book identifier storage 320 stores the book name (title image) in the book identifier. ), book group name (group image), publisher (publisher image), etc.

Here, in FIG. 24, the book identifier storage 320 is illustrated and described as storing the book name, the book group name, the publisher, etc. in the form of an image.

When the query processing engine 310 receives query data including an object image corresponding to the book name "Title Image #1", the query processing engine 310 detects "a0001" linked to the same book name as a book identifier.

When the query processing engine 310 receives query data including object images corresponding to the book group name "group image #1" and the publisher "B publisher image", the same book group name and "a0002" linked to the publisher Is detected as the book identifier.

The query processing engine 310 may receive query data including a cover image. The query processing engine 310 detects a book identifier associated with the cover image of the query data from the book identifier storage 320. The query processing engine 310 generates a query response including the detected book identifier and transmits it to the service server 200.

The query processing engine 310 detects book information from the book information storage 330 based on the query data, generates a query response including the detected book information, and transmits it to the service server 200.

The query processing engine 310 receives query data including a book identifier from the service server 200 (that is, the book information detection engine 230). The query processing engine 310 detects book information linked to the book identifier of the query data from the book information storage 330. In this case, the query processing engine 310 may receive query data including a cover image from the service server 200 and detect book information linked to the cover image from the book information storage 330. Here, as an example, the book information includes a book name and interactive object information (ie, interactive object image (IOI)).

Referring to FIG. 23, the book information storage 330 stores a book identifier, a book name, and an interactive object image (IOI) in association.

When the query processing engine 310 receives query data including the book identifier "a0002", the book name "title #2" linked to the book identifier "a0002", the interactive object image (IOI) #1, and the interactive object image (IOI)#2 and interactive object image (IOI)#5 are detected.

The query processing engine 310 retrieves book information including the detected "title #2", interactive object image (IOI) #1, interactive object image (IOI) #2, and interactive object image (IOI) #5. And transmits a query response including the same to the service server 200.

The query processing engine 310 detects reading support information from the reading support information storage 340 based on the query data, generates a query response including the detected reading support information, and transmits it to the service server 200.

The query processing engine 310 receives query data including a book identifier from the service server 200 (that is, the reading support information detection engine 240).

The query processing engine 310 detects a book identifier from the query data. The query processing engine 310 detects reading support information from the reading support information storage 340 based on the book identifier.

Referring to FIG. 24, the reading support information storage 340 stores a page image, an interactive object image (IOI), and an additional video in association with a book identifier.

The page image is a reference image for recognizing a page of the book object BO, and is stored in the reading support information storage 340 in association with the book identifier and the page.

The interactive object image IIO is a reference image for recognizing the interactive object IO placed on the book object BO, and is stored in the reading support information storage 340 in association with a book identifier and a page.

The additional video is an augmented reality content, mixed reality content, hyper-realistic content, etc. linked to the interactive object (IO), and is stored in the reading support information storage 340 in connection with the book identifier, page and interactive object image (IOI). do. In this case, the additional video may be a video linked only to the page of the book object BO without being linked with the interactive object IO, and is linked with the book identifier and the page and stored in the reading support information storage 340. Here, the additional video may be real image data or a web address in which real image data is stored.

When query data including the book identifier "a0001" is received, the query processing engine 310 includes at least one of a page, a page image, an interactive object image (IOI), and an additional video linked to the book identifier "a0001". All reading support information is detected. The query processing engine 310 transmits the detected reading support information to the service server 200.

Meanwhile, the query processing engine 310 transmits a query response including “NULL” to the service server 200 when there is no detection result corresponding to the query data of the service server 200.

Hereinafter, a reading support method in which a book object (BO) and an interactive content (IC) are linked according to an embodiment of the present invention will be described with reference to the accompanying drawings. 25 to 27 are flowcharts illustrating a reading support method in which a book object (BO) and an interactive content (IC) are linked according to an embodiment of the present invention.

First, the reading support device 100 is connected to a network. The user drives the application 110 installed in the reading support device 100 to use the reading support service. That is, after driving the reading support device 100, the user executes the application 110 supporting the reading support service through manipulations such as touch, click, and double click.

The application 110 activated by the user outputs a camera drive request signal to the processor 120 (S111), and the processor 120 transmits to the camera 130 and the display 140 in response to the camera drive request signal. A photographing start signal is output (S112).

The camera 130 starts photographing the active area AS in response to the photographing start signal (S113). In this case, when normal shooting of the active area AS starts, the camera 130 outputs an Ack signal for the shooting start signal to the application 110 and the processor 120.

The display 140 outputs an active area image (ASV) captured by the camera 130 in real time in response to the recording start signal (S114).

The application 110 receiving the Ack signal from the camera 130 outputs a guide output signal for capturing the cover of the book object BO to the processor 120 (S115).

Upon receiving the Ack signal of the camera 130 and the guide output signal of the application 110, the processor 120 outputs the cover photographing guide output signal to the display 140 (S116).

The display 140 receiving the cover guide output signal of the processor 120 outputs a cover photographing guide CSG in response to the cover guide output signal (S117). At this time, the display 140 outputs the cover photographing guide CSG to overlap (overwrite) the active area image ASV being output.

The user places the book object BO in the activity area AS so that the cover of the book object BO is photographed. When the book object BO is disposed in the activity area AS, an activity area image ASV photographing the activity area AS in which the book object BO is disposed is displayed on the display 140. At this time, the active area image ASV including the cover of the book object BO is output on the display 140.

When the active area image (ASV) including the cover of the book object (BO) is output (S118; yes), the image processing engine 150 generates a cover image through image recognition of the active area image (ASV) (S119). ). That is, the image processing engine 150 performs image recognition on an area corresponding to the cover recognition area of the cover photographing guide CSG among all areas of the active area image ASV. The image processing engine 150 generates a cover image as a result of image recognition and temporarily stores it in the memory 180.

The processor 120 generates a book identification request signal including a cover image (S120). That is, the processor 120 detects the cover image stored in the memory 180 and generates a book identification request signal including the detected cover image. The processor 120 transmits a book identification request signal to the first communication module 160 through a bus.

The first communication module 160 transmits the book identification request signal generated by the processor 120 to the service server 200 through the network, and receives a book identification response in response to the book identification request signal (S121). In this case, the first communication module 160 may receive a book identification response including one of book information and a book recognition error message.

The book information includes book information including book name and interactive object information, and the interactive object information may include an interactive object image (IOI) of an interactive object (IO) used when supporting reading for a book. .

The book recognition error message is a message indicating that the cover image included in the book identification request signal is not the cover of the book object (BO) or that it is not the book object (BO) registered in the service server 200.

When a book recognition error message is included in the book identification response received from the first communication module 160 (S122; No), the reading support device 100 performs steps S117 to S121 described above again. In this case, the display 140 may output a replacement guide for the book object BO, a location movement guide, etc. together with the cover photographing guide CSG.

When book information is included in the book identification response received from the first communication module 160 (S122; Yes), the processor 120 transmits a book information output signal to the display 140 (S123). That is, the processor 120 detects book information including book name and interactive object information from the book identification response, and generates a book information output signal including the detected book information. The processor 120 transmits the generated book information output signal to the display 140.

The display 140 outputs the book information and the service start confirmation object of the book object BO in response to the book information output signal of the processor 120 (S124). In this case, the display 140 may overlap (overwrite) the book information and the service start confirmation object on the activity area image ASV and output it.

When a service initiation confirmation object is selected by the user (S125; YES), the application 110 transmits a reading support service initiation signal to the processor 120 (S126).

The processor 120 detects book information from the book identification response in response to the reading support service start signal, and generates a reading support information signal including the detected book information (S127). The processor 120 transmits the generated reading support information signal to the first communication module 160.

The first communication module 160 transmits the reading support information signal generated by the processor 120 to the service server 200 through a network, and receives reading support information in response to the reading support information signal (S128). At this time, the reading support information includes a page image for page recognition of the book object (BO), an interactive object image (IOI) linked to the page of the book object (BO), a page of the book object (BO), or an interactive object image ( An example is including an additional video matched with IOI). The first communication module 160 transmits the received reading support information to the processor 120.

The processor 120 stores the reading support information transmitted through the first communication module 160 in the storage 170 (S129). At this time, the storage 170 maintains the storage state of the reading support information corresponding to the book object BO until the reading support of the book object BO is completed.

Since reading support information includes images, images, etc., it takes a certain time to receive (download) from the service server 200. Accordingly, the processor 120 outputs an interactive object information output signal that signals the preparation of the interactive object IO included in the book information to the display 140 (S130).

The display 140 outputs an interactive object preparation request screen including interactive object information included in the book information in response to the interactive object information output signal (S131). At this time, the display 140 detects the interactive object image (IOI) from the book information included in the book identification response, and displays the interactive object preparation request screen in which the detected interactive object image (IOI) is arranged according to a predetermined condition. Print. Through this, the display 140 outputs an interactive object preparation request screen until reception of the reading support information is completed.

When the reception of the reading support information is completed (S132; Yes), the processor 120 compares the book content BC of the active area image ASV captured by the camera 130 with the page image of the reading support information, and The page of (BO) is detected (S133). The processor 120 transmits a page detection signal including a page to the storage 170.

The storage 170 detects a page image corresponding to a page included in the page detection signal from among pre-stored reading support information (S134), and detects reading support information including the detected page image (S135). That is, the storage 170 detects reading support information including the page image detected in step S134 from among the previously stored reading support information. Here, the page image may be information included in the page, such as a specific object image included in the page, a page number, and text, as well as an image of the page. The storage 170 transmits the detected reading support information to the processor 120.

The processor 120 detects the additional video included in the received reading support information as interactive content (IC) (S136), and displays an interactive content output signal including the detected interactive content (IC) (140) It is transmitted to (S137).

The display 140 outputs the interactive content IC of the interactive content output signal by overlapping (overwriting) the active area image ASV including the book content BC (S138).

In the reading support method according to an embodiment of the present invention, interactive content (IC) is output in conjunction with an interactive object location. In the reading support method, the interaction content IC is output by reflecting the location of the interaction object, so that the interaction content IC overlaps (overwrites) the interaction content IC in the activity area image ASV. Here, the interaction object location is a location in the activity area image (ASV), where the interaction object (IO) is detected in the activity area image (ASV), or a specific object and interaction object in the book content (BC) It may be a relative position of (IO).

Referring to FIG. 28, when recording is selected by the user's manipulation, the processor 120 generates a user voice acquisition start signal and transmits the generated user voice acquisition start signal to the microphone 191 (S210).

The microphone 191 receives a user voice (S220), and the storage 170 or the memory 180 stores the user voice input from the microphone 191 (S230). In this case, in step S210, a screen for requesting pronunciation of a preset sentence or word may be output through the display to request input of a user's voice.

The voice processing engine 193 replaces some of the sounds (sounds) included in the interactive content IC with the user's voice (S240). The voice processing engine 193 replaces a sound corresponding to a preset sentence or word among sounds of the interactive content IC with a user voice.

The display 140 and the speaker 195 output interactive content in which a user's voice is set (S250). That is, the display 140 outputs an image of the interactive content IC, and the speaker 195 outputs a sound (sound) of the interactive content IC including the user's voice. Meanwhile, in step S250, a face of a character (person) in which a user's voice is set in the interactive content IC may be converted into a pre-input user face image and output.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description to those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the claims to be described later.

100: reading support device 110: application
120: processor 130: camera
140: display 150: image processing engine
160: first communication module 170: storage
180: memory 191: microphone
193: speech processing engine 195: speaker
200: service server 210: second communication module
220: book identification engine 230: book information detection engine
240: reading support information detection engine
250: control engine 300: database
310: Query Processing Engine 320: Book Identifier Store
330: book information storage 340: reading support information storage
400: cradle 410: main body
412: fixing device 420: support
430: coupling shaft 500: adapter
520: housing 540: reflector
AS: Active area ASV: Active area video
IO: interactive object IOI: interactive object image
IC: Interactive Content BO: Book Object
BC: Book content HL: horizontal straight line
VL: Vertical straight CSG: Cover shooting guide

Claims (12)

A reading support device disposed on a physical surface having an active area on which the interactive object is disposed, and having one or more processors,
The reading support device,
The location of the interaction object, which is a coordinate within the activity area image, as a result of receiving an input of an activity area image including the activity area and at least one interaction object disposed in the activity area, analyzing the activity area image, and analyzing the image An image processing engine that detects;
Interaction including a plurality of characters connected to the image processing engine for communication and associated with the active area image and the location of the interactive object detected by the image processing engine, each of which has different voices A processor configured to detect interaction content and output an output signal including the location of the interaction object and the interaction content;
A plurality of images connected to the image processing engine and the processor for communication, receiving a user voice, and storing the user voice in a storage storing the interactive content, but including the user voice in the interactive content detected by the processor A voice processing engine that sets and stores the voice of one character among the characters of
The image processing engine, the processor, and the voice processing engine are connected for communication, output the active area image, and in response to the output signal, a coordinate corresponding to the position of the interaction object in the active area image is referenced. By outputting the interactive content to output the interactive content so that it overlaps the interactive object of the active area image, the user selects the face of a character whose voice has been replaced by the user's voice among a plurality of characters included in the interactive content. A display that converts and outputs a face image; And
A reading support system comprising a speaker that outputs voices set for each of a plurality of characters included in the interactive content, and outputs the voice of one of the plurality of characters by replacing the voice of the user with the voice of the user. The method of claim 1,
The image processing engine further detects an interactive object area in which the interactive object is displayed as a result of image analysis of the active area image, and detects a partial area of the interactive object area as an interactive content output area . The method of claim 2,
The processor outputs an output signal including the interactive content output area and the interactive content,
The display is a reading support system for outputting the interactive content so as to overlap with a part of the interactive object of the active area image by outputting the interactive content in an area corresponding to the interactive content output area among the active area images. The method of claim 1,
Reading support further comprising a camera configured to capture an active area image including an active area of the physical surface and one or more interactive objects disposed in the active area, and transmit the active area image to the image processing engine and the display system. The method of claim 1,
A reading support system further comprising a microphone configured to receive a user voice and transmit the user voice to the voice processing engine. The method of claim 5,
The display is a reading support system that displays a character or sentence corresponding to the user's voice while driving a microphone. Outputting, by a display, an activity area image including an activity area and at least one interactive object disposed in the activity area;
Analyzing the active area image through an image processing engine, and detecting a position of the interactive object, which is a coordinate within the active area image, as a result of the image analysis;
Detecting, by a processor, interactive content associated with the activity area image and the location of the interactive object, but detecting interactive content including a plurality of characters each having different voices set;
The user voice is received by the voice processing engine, and the user voice is set and stored as the voice of one character among a plurality of characters included in the interactive content detected in the step of detecting the interactive content, and the interaction Replacing at least a portion of the voice of the content with the voice of the user;
Outputting, by the processor, an output signal including the location of the interactive object and the interactive content;
By the display, in response to the output signal, the interaction content is output based on a coordinate corresponding to the position of the interaction object in the activity area image to overlap the interaction object of the activity area image. Outputting content;
By the display, the face of the character whose voice is replaced by the user's voice in the step of replacing the user voice with the user voice among a plurality of characters included in the interactive content output in the step of outputting the interactive content is converted into a user face image. Converting and outputting; And
And outputting, by a speaker, a voice set for each of a plurality of characters included in the interactive content, and outputting the voice of one of the plurality of characters by replacing the voice of the user with the voice of the user. The method of claim 7,
Detecting, by the image processing engine, an interactive object area in which the interactive object is displayed as a result of image analysis of the active area image; And
And detecting, by the image processing engine, a partial area of the interactive object area as an interactive content output area. The method of claim 8,
Outputting, by the processor, an output signal including the interactive content output area and the interactive content; And
By the display, in response to the output signal, the interaction content is output to an area corresponding to the interactive content output area of the activity area image to overlap a part of the interactive object of the activity area image. Reading support method further comprising the step of outputting the content. The method of claim 7,
The method of supporting reading further comprising the step of capturing an image of an activity area including the activity area and at least one interactive object disposed in the activity area through a camera. The method of claim 7,
And receiving, by a microphone, a user voice that replaces at least some of the voices of the interactive object. The method of claim 11,
And displaying, by the display, a text or sentence corresponding to the user's voice before receiving the voice input.

Images