KR20200030228A - Electronic pen system interlocked with artificial intelligent speaker - Google Patents

Abstract

According to one embodiment, provided is an electronic pen system. The electronic pen system includes: an artificial intelligence server recognizing a voice command included in first voice data by analyzing the first voice data, and then, transmitting command data corresponding to the voice command to a command performing device; a content server receiving a content code, and transmitting content data corresponding to the content code to a content output device; an artificial intelligence speaker entering a command standby state upon the recognition of a call voice, and converting a voice recognized in the command standby state into the first voice data to transmit the first voice data to the artificial intelligence server; and an electronic pen photographing a pattern image printed on a printout in a size which cannot be visually recognized, creating a code corresponding to the pattern image, and transmitting the content code of the code to the content server via the artificial intelligence speaker. The artificial intelligence server transmits first command data, corresponding to an electronic pen call command for calling the electronic pen, of the voice command to the artificial intelligence speaker. The artificial intelligence speaker searches for and connects a wireless connection signal wirelessly with the electronic pen and enters an electronic pen service state when receiving the first command data, and transmits the content code to the content server in the electronic pen service state and outputs voice content data of the content data received from the content server through an embedded speaker. Therefore, the electronic pen system is capable of considerably improving user convenience by interlinking an artificial intelligence speaker with an electronic pen.

Description

ELECTRONIC PEN SYSTEM INTERLOCKED WITH ARTIFICIAL INTELLIGENT SPEAKER}

This embodiment relates to an electronic pen interlocked with an artificial intelligence speaker.

There are devices for recognizing the marks printed on the medium and outputting data related to the marks.

A barcode reader is a typical device, and the barcode reader reads barcodes printed on a medium and outputs corresponding data to a display device or the like.

Recently, many electronic pens have been developed to recognize a specific code printed on a medium such as a book and to output a voice corresponding to the recognized code.

The electronic pen includes a code reader device, reads a specific code printed on a medium such as a book, and outputs a voice corresponding to the code through a speaker.

Normally, only text is displayed on a print medium such as a book, so only the type information is provided to the reader. If the electronic pen is used, voice information in addition to the type information can also be provided, thus enhancing the reader's user experience. .

In general, a user input device known as a keyboard, a mouse, and the like is configured independently of a medium, and thus has a disadvantage that a user must separately recognize a medium and an input device, whereas a user input using an electronic pen is performed through a medium. There is an advantage that the media and the input device are integrated and intuitive operation is possible.

On the other hand, an artificial intelligence speaker (also known as an artificial intelligence (AI) speaker) has been developed as an input device having more user convenience than a generally known user input device-a keyboard, a mouse, and the like. The artificial intelligence speaker is a device that recognizes a user's voice, performs a function according to the voice, or outputs an answer to the voice, which is more than a conventional input device in that a user can input a command only with the user's voice without manual intervention. It is evaluated as having user convenience.

However, since the AI speaker's voice recognition level or voice understanding level is not yet high, the function is limited and the user's voice input is required to perform more functions, so the AI speaker does not completely replace the conventional input device. I can't.

Against this background, the purpose of this embodiment is to provide improved user input device technology through an electronic pen interworking with an artificial intelligence speaker.

In order to achieve the above object, an embodiment is to analyze the first voice data to recognize the voice command included in the first voice data, and transmits the command data corresponding to the voice command to the command execution device Intelligent server; A content server that receives a content code and transmits content data corresponding to the content code to a content output device; An artificial intelligence speaker that enters a command waiting state when the call voice is recognized, and converts the voice recognized in the command waiting state into the first voice data and transmits the voice to the artificial intelligence server; And an electronic device that shoots a pattern image printed in a size not visually recognized on the printed object, generates a code corresponding to the pattern image, and transmits the content code among the codes to the content server via the artificial intelligence speaker. The pen includes a pen, and the artificial intelligence server transmits first command data corresponding to an electronic pen calling command to call the electronic pen among the voice commands to the artificial intelligence speaker, and the artificial intelligence speaker comprises the first Upon receiving the command data, the wireless connection signal is searched for wireless connection with the electronic pen, the electronic pen service state is entered, the content code is transmitted from the electronic pen service state to the content server, and the content received from the content server It provides an electronic pen system that outputs voice content data from the data through the built-in speaker.

In the electronic pen system, the artificial intelligence speaker receives TV data in a TV (television) service state and outputs the TV data through a TV device, and receives image content data among the content data in the electronic pen service state. It can be output through a TV device.

In the electronic pen system, the artificial intelligence speaker receives product identification data from the electronic pen wirelessly connected and displays an external image of the electronic pen corresponding to the product identification data through a separate image output device.

In the electronic pen system, the electronic pen may include a pen speaker and turn off the function of the pen speaker when wirelessly connected to the artificial intelligence speaker.

In the electronic pen system, the electronic pen includes an operation unit composed of a plurality of operation buttons, generates a command code corresponding to the operation button, and transmits the command code to the AI speaker, wherein the AI speaker is the command By performing a function corresponding to the code, it is possible to change the output of audio or video.

In the electronic pen system, the content server executes a program corresponding to an interactive code received from the electronic pen via the artificial intelligence speaker, and outputs video and audio output data according to the execution of the program. Is transmitted to the artificial intelligence speaker, the command code is received from the electronic pen via the artificial intelligence speaker from the electronic pen, and the function of the program can be operated.

In the electronic pen system, the electronic pen further includes a microphone and transmits second voice data generated through the microphone to the artificial intelligence server via the artificial intelligence speaker, and the artificial intelligence server comprises: 2 The audio data is converted into text data and transmitted to the content server, and the content server can determine the suitability of the text data and output the determination result through the artificial intelligence speaker.

Then, the electronic pen, before transmitting the second voice data, transmits a content code corresponding to the first conversation message to the content server to the content server, and the content server is previously configured as an answer to the first conversation statement. The determination result may be generated by determining the similarity between the set second conversation text and the text data.

Another embodiment includes an artificial intelligence server that analyzes first voice data to recognize voice commands included in the first voice data, and transmits command data corresponding to the voice commands to a command execution device; A text server that receives a plurality of coordinate codes, generates text by combining the plurality of coordinate codes, and transmits at least one of the text, voice data converted from the text, and image data corresponding to the text; An artificial intelligence speaker that enters a command waiting state when the call voice is recognized, and converts the voice recognized in the command waiting state into the first voice data and transmits the voice to the artificial intelligence server; And a pattern image printed in a size not visually recognized on the printed material, generating a code corresponding to the pattern image, and transmitting the coordinate code among the codes to the text server via the artificial intelligence speaker. The pen includes a pen, and the artificial intelligence server transmits first command data corresponding to an electronic pen calling command to call the electronic pen among the voice commands to the artificial intelligence speaker, and the artificial intelligence speaker comprises the first Upon receiving the command data, it searches for a wireless connection signal, wirelessly connects with the electronic pen, enters an electronic pen service state, transmits the coordinate code to the text server in the electronic pen service state, and converts the voice data converted from the text. Provided is an electronic pen system for outputting through the built-in speaker or transmitting the text to another preselected server. .

The other server may record a memo by date and store the text by date in a storage space corresponding to a unique number of the electronic pen.

As described above, according to the present exemplary embodiment, the artificial intelligence speaker and the electronic pen are interlocked, thereby providing an input device having significantly improved user convenience compared to the prior art.

1 is a view showing the configuration of an electronic pen system according to an embodiment.
2 is a block diagram of an artificial intelligence speaker according to an embodiment.
3 is a block diagram of an electronic pen 120 according to an embodiment.
4 is a flowchart of a method of outputting content from an electronic pen system according to an embodiment.
5 is a view showing a signal flow between each component in the electronic pen system according to an embodiment.
6 is a view showing an operation portion of the electronic pen according to an embodiment.
7 is a diagram illustrating a program execution in an electronic pen system according to an embodiment.
8 is a view showing a process of conversation evaluation in an electronic pen system according to an embodiment.
9 is a view showing the configuration of an electronic pen system according to another embodiment.
10 is a view showing a printed material that can be applied to the electronic pen system according to another embodiment.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. It should be noted that in adding reference numerals to the components of each drawing, the same components have the same reference numerals as possible even though they are displayed on different drawings. In addition, in describing the present invention, when it is determined that detailed descriptions of related well-known configurations or functions may obscure the subject matter of the present invention, detailed descriptions thereof will be omitted.

In addition, in describing the components of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only for distinguishing the component from other components, and the nature, order, or order of the component is not limited by the term. When a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected to or connected to the other component, but another component between each component It should be understood that elements may be "connected", "coupled" or "connected".

1 is a view showing the configuration of an electronic pen system according to an embodiment.

Referring to FIG. 1, the electronic pen system 100 may include an artificial intelligence speaker 110, an electronic pen 120, an artificial intelligence server 130, and a content server 140.

The AI speaker 110 may convert the voice of the user 10 into voice data and transmit it to the AI server 130. Then, the artificial intelligence server 130 may recognize the voice content included in the voice data and perform a function corresponding to the voice content.

For example, the artificial intelligence server 130 may recognize a voice command included in voice data and perform a function corresponding to the voice command. When the user 10 utters the voice "Please connect the radio" to the AI speaker 110, the AI speaker 110 sounds the voice "Connect the radio" to the voice data-voice data is analog signal voice It is converted to digital signals, and can be transmitted to the artificial intelligence server 130. The artificial intelligence server 130 may include a function of extracting or analyzing voice commands from voice data. The artificial intelligence server 130 may include a learned neural network (eg, a neural network, etc.), and input voice data into the neural network and obtain a voice command as its output. Alternatively, the artificial intelligence server 130 includes a function of converting voice data into text, and can substitute text into a rule-based classification program to check voice commands indicated by the text. Then, the AI server 130 may perform a function corresponding to the confirmed voice command, for example, a function of transmitting voice data output from the radio server to the AI speaker 110 in a streaming form.

It is necessary to include a high-capacity storage device and a high-performance computing device in order to recognize the voice content of the voice uttered by the user 10-for example, voice commands-from voice data corresponding to the user 10's voice. . Accordingly, it is difficult for the artificial intelligence speaker 110 having a small size to embed a function for recognizing voice content directly from the voice of the user 10. According to this difficulty, in general, the artificial intelligence speaker 110 performs a function of converting the voice of the user 10 into speech data, and the function of recognizing speech content from the speech data is performed by the artificial intelligence server 130 Can be performed.

On the other hand, the AI speaker 110 does not constantly convert the voice uttered by the user 10 into voice data and transmits it to the AI server 130, but only in a specific state, the voice data corresponding to the voice of the user 10 Can be transmitted to the artificial intelligence server 130. Otherwise, a problem may occur in which the artificial intelligence server 130 responds to the general conversation content of the user 10.

The artificial intelligence speaker 110 may maintain a call waiting state and enter a command waiting state when the call voice is recognized. Since the voice data corresponding to the voice of the user 10 is not transmitted to the AI server 130 in the call waiting state, the AI speaker 110 corresponds to the voice of the user 10 and the voice data corresponding to the voice of the user 10 You can judge for yourself. As described later, the artificial intelligence speaker 110 recognizes the call voice of the user 10 through the triggering module while having a triggering module therein, and when the call voice is recognized, the command standby state is entered. You can enter.

The AI speaker 110 may convert the voice recognized in the command waiting state to voice data and transmit it to the AI server 130. The voice recognized in the command waiting state is hereinafter referred to as first voice data.

The artificial intelligence server 130 may analyze the first voice data and recognize voice commands included in the first voice data. The artificial intelligence server 130 may include a STT (speach to text) engine, convert the first voice data into text, and input the text into an analysis module such as a neural intelligence network to recognize a voice command as its output. Then, the artificial intelligence server 130 may transmit command data corresponding to the voice command to the command execution device. For example, the artificial intelligence server 130 may recognize a voice command “turn off the light” and transmit command data to turn off the light to the light control device.

The artificial intelligence server 130 recognizes the electronic pen calling instruction that calls the electronic pen 120 among the voice commands included in the first voice data, and then the artificial intelligence speaker 110 receives the first instruction data corresponding to the electronic pen calling instruction. Can be sent to For example, when the user 10 utters a voice command "Please connect to the pen," the AI speaker 110 transmits the corresponding first voice data to the AI server 130, and the AI server The 130 may recognize the electronic pen calling command from the first voice data and transmit the first command data corresponding to the electronic pen calling command to the artificial intelligence speaker 110.

When receiving the first command data, the artificial intelligence speaker 110 may search for a wireless connection signal and wirelessly connect to the electronic pen 120. For example, the artificial intelligence speaker 110 and the electronic pen 120 may include wireless communication modules such as Bluetooth and Wi-Fi, and may be wirelessly connected through these wireless communication modules.

The artificial intelligence speaker 110 may enter the electronic pen service state when wirelessly connected to the electronic pen 120 through the first command data.

The electronic pen service state in the AI speaker 110 is a state in which various codes received from the electronic pen 120 can be processed, and in addition to the user input function through voice of the AI speaker 110, the electronic pen 120 It can be understood that the user input function through is added.

The electronic pen 120 may include an object identifier (OID) module and photograph a pattern image through the OID module. The pattern image is an image printed in a size that is not visually recognized on the printed material 20 and is composed of a pattern that can be recognized by the OID module. A visual image of a size visually recognizable may be printed on the printed material 20. As a visual image, for example, letters, figures, photos, and the like may be printed on the printed material 20. On the other hand, the pattern image can be printed so that it overlaps with the visual image at a size that is not visually recognized. Therefore, the user 10 can recognize the visual image printed on the printed object 20, but cannot recognize the pattern image. On the contrary, since the electronic pen 120 has a small photographing area, it can photograph a pattern image and recognize a code corresponding thereto, but may not recognize a visual image printed on a wide area.

The electronic pen 120 may take a pattern image through the OID module and generate a code corresponding to the pattern image. A plurality of pattern images having different patterns may be predefined, and the electronic pen 120 may store codes corresponding to each pattern image. Then, the electronic pen 120 may generate a code corresponding to the pattern image when a specific pattern image is photographed.

The electronic pen 120 may transmit the generated codes to the artificial intelligence speaker 110. Then, the artificial intelligence speaker 110 may transmit the content code among the received codes to the content server 140.

The AI speaker 110 and the electronic pen 120 may be connected through wireless communication, and the AI speaker 110, the AI server 130, and the content server 140 may be connected through the network 30. The artificial intelligence speaker 110, the artificial intelligence server 130, and the content server 140 may have a network identification ID such as an IP (internet protocol) address, and may be connected to the network 30 through the network identification ID. The artificial intelligence speaker 110, the artificial intelligence server 130, and the content server 140 may be connected to a plurality of devices through the network 30. For example, the artificial intelligence speaker 110 may exchange information with the artificial intelligence server 130 through the network 30, and may exchange information with the content server 140. On the other hand, the electronic pen 120 may be wirelessly connected to one device located in the proximity. For example, the electronic pen 120 may be connected to the artificial intelligence speaker 110 through Bluetooth. The electronic pen 120 may use the artificial intelligence speaker 110 to exchange information with the artificial intelligence server 130 and the content server 140. For example, the electronic pen 120 may transmit the content code to the content server 140 through the AI speaker 110, and receive information from the content server 140 through the AI speaker 110. You can.

The electronic pen 120 may transmit the content code among the codes to the content server 140 via the artificial intelligence speaker 110. Then, the content server 140 may receive the content code and transmit content data corresponding to the content code to the content output device. For example, when the electronic page 120 transmits the content code corresponding to one page of a particular book to the content server 140, the content server 140 records the content of the page in voice and the voice content data. The image content data adapted from the content of the page may be transmitted to the artificial intelligence speaker 110.

The artificial intelligence speaker 110 may output voice content data among content data received from the content server 140 through the built-in speaker. Then, the artificial intelligence speaker 110 may display the image content data among the content data received from the content server 140 through a separate image output device.

The separate image output device may be, for example, a TV (television) device 40. The artificial intelligence speaker 110 may receive TV data in a TV service state and output TV data through the TV device 40. In order to output TV data, a set-top box may be built in the artificial intelligence speaker 110. Depending on the embodiment, a set-top box may be attached to the TV device 40 or a set-top box may be embedded.

The artificial intelligence speaker 110 may output the video content data among the content data through the TV device 40 in the electronic pen service state.

The electronic pen 120 may include a speaker (pen speaker) to perform its own function. And, depending on the embodiment, the voice content data may be output from the artificial intelligence speaker 110, it may be output from the electronic pen 120. When the voice content data is output from the AI speaker 110, when the AI speaker 110 and the electronic pen 120 are wirelessly connected, the function of the pen speaker may be turned off.

Meanwhile, the electronic pen system 100 may include a plurality of electronic pens 120, and the artificial intelligence speaker 110 receives product identification data for the plurality of electronic pens 120 through a wireless connection signal, each The external image of the electronic pen 120 corresponding to the product identification data may be displayed through a separate image output device-for example, the TV device 40. Then, when one of the plurality of appearance images is selected by the user 10, the artificial intelligence speaker 110 is wirelessly connected to the selected electronic pen 120 and corresponds to the product identification data received from the electronic pen 120 The external image of the pen 120 may be displayed through an image output device. Selecting one of the plurality of appearance images may be performed by the voice command of the user 10.

2 is a block diagram of an artificial intelligence speaker according to an embodiment.

2, the artificial intelligence speaker 110 includes a voice input unit 210, a voice data conversion unit 220, a triggering unit 230, a control unit 240, a server communication unit 250, and an electronic pen communication unit 260. , A voice output unit 270 and the like.

The voice input unit 210 may include a microphone and convert the voice uttered by the user into an analog electric signal. The voice input unit 210 may further include a filter, if necessary, to remove noise around the user or to remove noise generated by other sources.

The analog electrical signal converted by the audio input unit 210 is transmitted to the audio data conversion unit 220, and the audio data conversion unit 220 can convert the analog electrical signal into digital data.

The triggering unit 230 may include the triggering module and compare voice data with preset call voice data, and when the voice data is similar to the call voice data, the user may determine that the call voice is spoken. When the triggering unit 230 recognizes the call voice, the controller 240 may inform the control unit 240 that the call voice is recognized through a flag or the like.

When the controller 240 is in a call waiting state and receives a signal from the triggering unit 230 that a call voice has been recognized, it may enter a command waiting state. The control unit 240 may output a command waiting state voice through the voice output unit 270 to notify the user that the command waiting state has been entered.

Explained in terms of user behavior, when the user utters a ringing voice-for example, "hey, speaker"-the triggering unit 230 recognizes the ringing voice, and correspondingly, the controller 240 waits for a command The state voice-for example, "yes, please speak"-can be output through the voice output unit 270.

The audio output unit 270 includes a speaker and a digital-to-analog converter (DAC), and converts analog data into analog electrical signals and converts analog electrical signals into sound waves.

The voice input through the voice input unit 210 in the command waiting state may be converted to the first voice data through the voice data conversion unit 220 and then transmitted to the artificial intelligence server through the server communication unit 250. The server communication unit 250 may be connected to the network described with reference to FIG. 1, and may exchange information with an artificial intelligence server or a content server through a network. In addition, the server communication unit 250 may transmit and receive information to and from other servers through a network. For example, the server communication unit 250 may receive TV data through a TV data server and connect with a smart home server. As a result, it is possible to transmit / receive signals / data to control electrical devices in the home.

The artificial intelligence server may recognize the electronic pen call command through the first voice data received through the server communication unit 250. Then, the artificial intelligence server may transmit the first command data corresponding to the electronic pen call command to the artificial intelligence speaker, for example, the server communication unit 250.

The artificial intelligence speaker-for example, the control unit 240-may receive the first command data and search for a wireless connection signal through the electronic communication unit 260 to wirelessly connect to the electronic pen. At this time, the electronic pen communication unit 260 and the electronic pen may be wirelessly connected.

When the electronic pen communication unit 260 and the electronic pen are wirelessly connected, the control unit 240 may enter an electronic pen service state. In addition, in the electronic pen service state, the electronic pen communication unit 260 may receive various codes from the electronic pen, and among these codes, the content code may be transmitted to the content server through the server communication unit 250. Also, among the content data received from the content server, the voice content data may be output through the voice output unit 270.

3 is a block diagram of an electronic pen 120 according to an embodiment.

Referring to FIG. 3, the electronic pen 120 may include an image sensor 310, a processor 320, a communication unit 330, a pen speaker 340, a microphone 350, and an operation unit 360.

The image sensor 310 may include a photoelectric device that converts light into an electrical signal, and photograph a pattern image printed in a size that is not visually recognized on the printed material using the photoelectric device. The image sensor 310 is a kind of camera, and can be viewed as a high-resolution / high-precision camera capable of accurately photographing a local area, and is also called an object identifier (OID) module.

The processor 320 may include a code conversion unit 322 and a pen control unit 324.

The code conversion unit 322 may analyze the pattern of the pattern image photographed by the image sensor 310 and generate a code corresponding to the pattern. The pattern is, for example, about a position where dots are arranged in a virtual grid, and the code conversion unit 322 may generate a code using the positions of the dots recognized in the captured pattern image.

The pen control unit 324 may control various functions of the electronic pen 120. In particular, the pen control unit 324 may check codes received from the code conversion unit 322 and control functions corresponding to each code.

The code may be subdivided into various types, for example, the code may be subdivided into content codes, instruction codes, and the like. The content code is a code that matches the content, and when the content code is confirmed, the pen control unit 324 may control the electronic pen 120 or its peripheral devices so that the content corresponding to the content code can be displayed to the user. . The command code is a code matched with a specific function, and when the command code is confirmed, the pen control unit 324 may control the electronic pen 120 or its peripheral devices to perform a function corresponding to the command code. The command code can be matched with, for example, functions such as volume up, volume down, mute, recording, previous track playback, subsequent track playback, pause, and playback.

The communication unit 330 is a module that performs wireless communication with peripheral devices, and may be connected through an artificial intelligence speaker or a general speaker and wireless communication such as Bluetooth.

The pen speaker 340 is a device for outputting voice, and voice content data may be output among content data corresponding to the content code, and preset voice may be output under the control of the pen control unit 324.

The microphone 350 includes an analog-to-digital converter (ADC), and is a module that converts input voice into voice data, which is a digital signal. The voice input to the microphone 350 may be converted into voice data and then output through the pen speaker 340. Depending on the embodiment, an audio mixer may be included in the processor 320, and the voice data generated in the microphone 350 may be outputted through the pen speaker 340 after being mixed with the voice content data in the audio mixer. have.

When the communication unit 330 is wirelessly connected to an external speaker or an artificial intelligence speaker through wireless communication such as Bluetooth, the function of the pen speaker may be turned off. At this time, the voice data transmitted to the pen speaker 340 may be output by being transmitted to an external speaker or an artificial intelligence speaker connected wirelessly. The audio data mixed by the audio mixer can also be output to an external speaker or an artificial intelligence speaker.

The manipulation unit 360 is a module capable of recognizing a user's manipulation. The manipulation unit 360 may include a plurality of manipulation buttons, and when each manipulation button is pressed, generate different command codes and transmit them to the pen control unit 324. For example, the control unit 360 includes a volume up button, a volume down button, a mute button, a recording button, a transfer track playback button, a subsequent track playback button, a pause button, a playback button, etc. Accordingly, different command codes can be generated.

4 is a flowchart of a method of outputting content from an electronic pen system according to an embodiment.

4, the artificial intelligence speaker may be triggered first (S400). The artificial intelligence speaker may be triggered while recognizing the call voice using the triggering module. The AI speaker may enter a command waiting state after being triggered and wait until a user's voice command is made.

After entering the command-waiting state, the AI speaker may receive an electronic pen call command to call the electronic pen, convert the electronic pen call command to first voice data, and transmit it to the AI server. Then, the artificial intelligence server may recognize the electronic pen call command through analysis of the received first voice data (S402).

When the electronic pen calling command is recognized, the artificial intelligence server may transmit the first command data corresponding to the electronic pen calling command to the artificial intelligence speaker. Then, after receiving the first command data, the artificial intelligence speaker may search for a wireless connection signal and attempt communication connection with the electronic pen (S404). Then, the artificial intelligence speaker may be wirelessly connected to one electronic pen in a wireless connection enabled state (S406).

Then, the artificial intelligence speaker may enter the electronic pen service state and receive a code from the electronic pen in the electronic pen service state (S408).

When the content code is received among the codes, the artificial intelligence speaker may transmit the content code to the content server and receive content data from the content server (S410).

The content data may include video content data and audio content data, and the artificial intelligence speaker may output video content data using a separate video output device such as a TV device, and output the audio content data using a built-in speaker. Yes (S412).

5 is a view showing a signal flow between each component in the electronic pen system according to an embodiment.

Referring to FIG. 5, the artificial intelligence speaker 110 may recognize and trigger a call voice by itself using a built-in triggering module (S400).

Then, the AI speaker 110 may receive a voice command according to the user's speech (S502), convert the voice command into voice data, and transmit the voice command to the AI server 130 (S504).

The artificial intelligence server 130 may analyze voice data, recognize voice commands, and transmit command data corresponding to voice commands to a command execution device. When the voice commands are recognized as electronic pen calling commands, the electronic pen calling commands The first command data corresponding to the may be transmitted to the artificial intelligence speaker 110 (S506).

Upon receiving the first command data corresponding to the electronic pen call command, the artificial intelligence speaker 110 functions as a command execution device. The artificial intelligence speaker 110 searches for a wireless connection signal as a command execution device and searches for an electronic pen ( 120) may be attempted to wirelessly connect (S404).

The artificial intelligence speaker 110 and one electronic pen 120 may be wirelessly connected through a wireless connection signal (S406). Then, the electronic pen 120 may take a pattern image (S508), generate a code for the pattern image, and transmit it to the artificial intelligence speaker 110 (S408).

Then, the artificial intelligence speaker 110 may transmit the content code among the codes to the content server 140 (S510) and receive content data from the content server 140 (S512).

In addition, the artificial intelligence speaker 110 may output audio content data among the content data using a built-in speaker (S514), and output the video content data using a separate video output device such as the TV device 40. (S516).

6 is a view showing an operation portion of the electronic pen according to an embodiment.

Referring to FIG. 6, the electronic pen 120 may include an operation unit 620 composed of a plurality of operation buttons 621 to 625. Then, the electronic pen 120 may generate a command code corresponding to the operation button and transmit it to the artificial intelligence speaker. In addition, the artificial intelligence speaker may change a voice output or a video output by performing a function corresponding to a command code. For example, as an example, when the second operation button 622 is pressed, the electronic pen 120 may generate a second command code and transmit it to the artificial intelligence speaker, and the artificial intelligence speaker may transmit the second command code to the artificial intelligence speaker. Correspondingly, the output of audio or video can be paused. As another example, when the first operation button 621 is pressed, the artificial intelligence speaker may restart the output of the audio or video, and when the third operation button 623 is pressed, the artificial intelligence speaker may display the audio or video. You can stop printing. As another example, when the fourth operation button 624 is pressed, the AI speaker may play the previous track-or previously played content-, and when the fifth operation button 625 is pressed, the AI speaker Can play subsequent tracks or content to be played next.

Meanwhile, a program may be executed in the AI speaker. When such a program is executed in the AI speaker, the command code generated by the operation unit may be used to manipulate the function of the program.

7 is a diagram illustrating a program execution in an electronic pen system according to an embodiment.

Referring to FIG. 7, the artificial intelligence speaker 110 may receive an interactive code from the electronic pen 120. Then, the artificial intelligence speaker 110 may transmit the interactive code to the content server 140, and the content server 140 may execute a program corresponding to the interactive code. After the program is downloaded to the artificial intelligence speaker 110, it may be executed in the artificial intelligence speaker 110, and as it is executed in the content server 140, only the image output data and audio output data as a result of execution are transferred to the artificial intelligence speaker 110. Can be sent. In addition, the artificial intelligence speaker 110 may output image output data through a separate image output device such as the TV device 40, and may output audio output data through a built-in speaker.

Meanwhile, manipulation of the program may be performed by a command code generated in the electronic pen 120. The electronic pen 120 may recognize the pressing of an operation button disposed on the operation unit, generate a command code corresponding to each operation button, and transmit it to the artificial intelligence speaker 110. When the program is executed in the artificial intelligence speaker 110, the artificial intelligence speaker 110 may operate a function of the program by recognizing a command code corresponding to an operation button. Then, when the program is executed in the content server 140, the artificial intelligence speaker 110 transmits a command code corresponding to the operation button to the content server 140, and the content server 140 recognizes the command code and programs You can operate the function.

8 is a view showing a process of conversation evaluation in an electronic pen system according to an embodiment.

Referring to FIG. 8, the electronic pen 120 may transmit the second voice data generated through the microphone to the AI server 130 via the AI speaker 110. Then, the artificial intelligence server 130 may convert the second voice data into text data (STT: speech-to-text) and transmit it to the content server 140. Then, the content server 140 may determine the suitability of the text data, and output the determination result through the artificial intelligence speaker 110.

The electronic pen 120 transmits one content code corresponding to the first conversation sentence to the content server 140 to the content server 140 before transmitting the second voice data, and the content server 140 answers the first conversation question As a result, the similarity between the preset second conversation text and the text data converted through STT may be determined to generate the above-described determination result.

The first conversational text may be visually recognized on the printed material 20, and the pattern image corresponding to the first conversational text may be printed at a size that is not visually recognized. In addition, the answer portion corresponding to the first conversational sentence may be formed on the printed material 20 as a blank.

The user may check the first conversational sentence and instruct the electronic pen 120 to indicate the first conversational sentence. At this time, voice content and / or video content corresponding to the first conversational sentence may be output through the artificial intelligence speaker. In addition, the user may utter a specific voice in response to the first conversational sentence, and the voice may be transmitted to the AI server 130 after being converted into the second voice data in the AI speaker 110.

The artificial intelligence server 130 may convert the second voice data into text data using an STT engine, and transmit the text data to the content server 140. At this time, the AI server 130 may transmit text data via the AI speaker 110, the AI server 110 transmits the text data to the AI speaker 110, and the AI speaker ( 110) may transmit the text data to the content server 140.

In addition, the content server 140 may determine the similarity between the received text data and the preset second conversation to determine whether the user's answer is appropriate, and transmit the determination result to the artificial intelligence speaker 110. At this time, a plurality of second conversational sentences may be set, and the content server 140 may determine the similarity between each of the plurality of second conversational texts and the texter, and generate a determination result as having the highest similarity.

9 is a view showing the configuration of an electronic pen system according to another embodiment.

Referring to FIG. 9, the electronic pen system 900 may include an artificial intelligence speaker 110, an electronic pen 120, an artificial intelligence server 130, a text server 940, and the like.

The printed material 920 may include a pattern image printed in a size that is not visually recognized by the user, and the electronic pen 120 may photograph the pattern image and generate a code corresponding to the pattern image.

In another embodiment, the pattern image printed on the printed object 920 may be a pattern image corresponding to the coordinate code. The electronic pen 120 may photograph such a pattern image and generate a coordinate code and transmit it to the artificial intelligence speaker 110. Then, the artificial intelligence speaker 110 may transmit the coordinate code to the text server 940.

As described with reference to FIG. 1, when the call voice is recognized, the artificial intelligence speaker 110 enters a command waiting state, and converts the voice recognized in the command waiting state into first voice data to convert the artificial intelligence server 130 Can be sent to

Then, the artificial intelligence server 130 transmits command data corresponding to the voice command to the command execution device, and recognizes the electronic pen calling command to call the electronic pen among the voice commands, and then transmits the first command data corresponding thereto to the artificial intelligence speaker. (110).

When receiving the first command data, the artificial intelligence speaker 110 may search for a wireless connection signal, wirelessly connect with the electronic pen 120, and enter the electronic pen service state.

Then, the artificial intelligence speaker 110 may receive the coordinate code from the electronic pen 120 in the electronic pen service state and transmit the coordinate code to the text server 940.

The electronic pen 120 may continuously transmit a plurality of coordinate codes to the AI speaker 110, and the AI speaker 110 may transmit a plurality of coordinate codes to the text server 940.

The text server 940 may receive a plurality of coordinate codes and generate text by combining the plurality of coordinate codes. Then, the text server 940 transmits voice data converted from text and / or image data corresponding to the text to the AI speaker 110, and the AI speaker 110 outputs voice data using the built-in speaker. You can, and send the text to another preselected server.

Here, another server may record the memo by date or time, and store the text by date or time in a storage space corresponding to a unique number of the electronic pen 120 or the AI speaker 110.

10 is a view showing a printed material that can be applied to the electronic pen system according to another embodiment.

Referring to FIG. 10, grids having a size visually recognizable are printed on a printed material, and pattern images having a size not visually recognizable may be printed in different patterns for each location.

The electronic pen 120 includes writing means such as ink, and when the user contacts the electronic pen 120 with the printed material 920, writing means such as ink may come out of the printed material 920.

The user may draw letters or pictures on the printed material 920 using the writing means of the electronic pen 120. The electronic pen 120 may take a writing means such as ink on the printed material 920 and simultaneously photograph the same location using the OID module. Then, the electronic pen 120 may transmit the coordinate code corresponding to the photographed pattern image to the text server via the artificial intelligence speaker. Then, the text server can receive the coordinate codes corresponding to the text or picture drawn on the print 920, and convert the text intended by the user into text.

The printout 920 has a separate unique number, and the other servers described above may store recognized letters or pictures in a storage space corresponding to the unique number.

As described above, according to the present exemplary embodiment, the artificial intelligence speaker and the electronic pen are interlocked, thereby providing an input device having significantly improved user convenience compared to the prior art.

The terms "comprises", "composes" or "haves" as described above mean that the corresponding components can be inherent unless otherwise stated, and do not exclude other components. It should be interpreted that it may further include other components. All terms, including technical or scientific terms, unless otherwise defined, have the same meaning as commonly understood by a person skilled in the art to which the present invention pertains. Commonly used terms, such as predefined terms, should be interpreted as being consistent with the meaning in the context of the related art, and are not to be interpreted as ideal or excessively formal meanings unless explicitly defined in the present invention.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and variations without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the claims below, and all technical spirits within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

Claims (10)

An artificial intelligence server that analyzes first voice data, recognizes voice commands included in the first voice data, and transmits command data corresponding to the voice commands to a command execution device;
A content server that receives a content code and transmits content data corresponding to the content code to a content output device;
An artificial intelligence speaker that enters a command waiting state when the call voice is recognized, and converts the speech recognized in the command waiting state into the first voice data and transmits the voice to the artificial intelligence server; And
An electronic pen that shoots a pattern image printed in a size that is not visually recognized on a print, generates a code corresponding to the pattern image, and transmits the content code among the codes to the content server via the artificial intelligence speaker. Including,
The artificial intelligence server,
The first command data corresponding to the electronic pen call command that calls the electronic pen among the voice commands is transmitted to the artificial intelligence speaker,
The artificial intelligence speaker,
Upon receiving the first command data, the wireless connection signal is searched for wireless connection with the electronic pen, the electronic pen service state is entered, and the content code is transmitted from the electronic pen service state to the content server and received from the content server. An electronic pen system that outputs voice content data from the contents data through a built-in speaker. According to claim 1,
The artificial intelligence speaker,
An electronic pen system that receives TV data in a TV (television) service state, outputs the TV data through a TV device, and outputs video content data among the content data in the electronic pen service state through the TV device. According to claim 1,
The artificial intelligence speaker,
An electronic pen system that receives product identification data from the wirelessly connected electronic pen and displays an external image of the electronic pen corresponding to the product identification data through a separate image output device. According to claim 1,
The electronic pen,
An electronic pen system incorporating a pen speaker and turning off the function of the pen speaker when wirelessly connected to the AI speaker. According to claim 1,
The electronic pen,
It includes an operation unit consisting of a plurality of operation buttons, generates a command code corresponding to the operation button and transmits it to the artificial intelligence speaker,
The artificial intelligence speaker,
An electronic pen system that changes a voice output or a video output by performing a function corresponding to the command code. According to claim 1,
The content server,
The program corresponding to the interactive code received from the electronic pen is executed through the AI speaker, and image output data and audio output data according to the execution of the program are transmitted to the AI speaker, and the electronic An electronic pen system that receives a command code from the electronic pen via an artificial intelligence speaker from the pen and manipulates the functions of the program. According to claim 1,
The electronic pen,
It further includes a microphone and transmits the second voice data generated through the microphone to the AI server via the AI speaker,
The artificial intelligence server,
The second audio data is converted to text data and transmitted to the content server,
The content server,
An electronic pen system that determines the suitability of the text data and outputs the determination result through the artificial intelligence speaker. The method of claim 7,
The electronic pen,
Before transmitting the second voice data, one content code corresponding to the first conversation message is transmitted to the content server to the content server,
The content server,
An electronic pen system that determines the similarity between the second conversational text and the text data set in advance as an answer to the first conversational text to generate the determination result. An artificial intelligence server that analyzes first voice data, recognizes voice commands included in the first voice data, and transmits command data corresponding to the voice commands to a command execution device;
A text server that receives a plurality of coordinate codes, generates text by combining the plurality of coordinate codes, and transmits at least one of the text, voice data converted from the text, and image data corresponding to the text;
An artificial intelligence speaker that enters a command waiting state when the call voice is recognized, and converts the voice recognized in the command waiting state into the first voice data and transmits the voice to the artificial intelligence server; And
An electronic pen that shoots a pattern image printed at a size that is not visually recognized on a print, generates a code corresponding to the pattern image, and transmits the coordinate code among the codes to the text server via the artificial intelligence speaker. Including,
The artificial intelligence server,
The first command data corresponding to the electronic pen call command that calls the electronic pen among the voice commands is transmitted to the artificial intelligence speaker,
The artificial intelligence speaker,
Upon receiving the first command data, the wireless connection signal is searched, wirelessly connected to the electronic pen, and entered into the electronic pen service state. In the electronic pen service state, the coordinate code is transmitted to the text server and the text is converted. An electronic pen system that outputs voice data through a built-in speaker or transmits the text to another preselected server. The method of claim 9,
The other server,
An electronic pen system that records a memo by date and stores the text by date in a storage space corresponding to a unique number of the electronic pen.

Images