KR20190070162A - System and method for realizing multi-device contents based on pen-motion - Google Patents

Abstract

Disclosed are a system and method for implementing multi-device content based on pen motion recognition. According to an embodiment of the present invention, the system comprises: a content provider composed of a plurality of objects including a unique marker; a reader pen determining and controlling a predetermined output form corresponding to a motion generated by user operation when any one of the is selected; and an output device outputting content corresponding to a unique marker included in the selected object in accordance with the output form determined by the reader pen.

Description

[0001] The present invention relates to a pen-motion-aware multi-device content implementation system and method,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a pen-motion-aware multi-device content implementation system and method, and more particularly, to a pen-motion-aware multi-device content implementation system and method for recognizing movement of an electronic pen and thereby reproducing multi- .

When you bring a pen to a book or study wall, electronic pens that read or describe the picture are becoming popular. The electronic pen is useful for language learning because it can listen to the desired words and sentences repeatedly. If the user includes the recording function, the user can directly record his / her voice to compare the pronunciation.

A conventional electronic pen basically brings an electronic pen to a word or a picture on a book or a wall, and the electronic pen recognizes a specific marker which is not shown in a word or a picture and outputs the voice or sound to the speaker.

As electronic pens have gained popularity, various publishing companies and tutoring companies have suggested ways to utilize various electronic pens, and each company has introduced electronic pens changed according to their characteristics.

However, the electronic pen provided by various companies is most likely to be deformed in its appearance. The electronic pen has a function of outputting sound or voice, a function of playing music by storing a music file, and a recording of a sound generated by a user Functions other than the function of using a memory and a speaker basically included in the electronic pen as in the case of the function of the electronic pen.

Korean Patent Laid-Open No. 2009-012558 (Dec. 09, 2009)

According to an aspect of the present invention, there is provided a pen-motion-aware multi-device content implementation system capable of implementing contents through various devices in addition to a built-in speaker of a leader pen according to a motion of a reader pen operated by a user And methods.

The solution of the present invention is not limited to those mentioned above, and other solutions not mentioned can be clearly understood by those skilled in the art from the following description.

As a means for solving the above-mentioned technical problems, according to an embodiment of the present invention, there is provided a contents providing apparatus comprising a content provider composed of a plurality of objects including unique markers, And an output device for outputting a content corresponding to a unique marker included in the selected object in accordance with an output form determined by the leader pen, A recognized multi-device content implementation system is provided.

Preferably, the motion generated by the user's operation may be a motion that occurs immediately before or after an operation in which any one of the objects is selected.

Also preferably, the output device may be at least one of an internal output device including a built-in speaker built in the leader pen, and an external output device including an external speaker, a video output device, and a VR HMD (Virtual Reality Head Mounted Display).

Preferably, the leader pen includes: a sensor unit for recognizing a unique marker included in a plurality of objects; a motion recognition unit for recognizing a motion generated by a user's operation; a local communication unit for supporting an interface with an external output device; And a control unit for determining an output form corresponding to the motion recognized by the motion recognition unit and controlling the internal output unit and the external output unit according to the determined output form.

Preferably, the sensor unit can recognize a unique marker included in any one of the plurality of objects when the object approaches the predetermined recognition range.

Preferably, the control unit controls to output a content corresponding to a unique marker included in the selected object through the built-in built-in speaker when the determined output format is performed by the internal output device, , It is possible to control the corresponding external output device among the external output devices to output a content corresponding to a unique marker included in the selected object through the local communication unit.

According to another embodiment of the present invention, there is provided a method of controlling a robot, comprising the steps of: selecting any one of a plurality of objects from a content provider composed of a plurality of objects including unique markers; And outputting a content corresponding to a unique marker included in the selected object in accordance with the determined output form. The present invention also provides a method for implementing a pen-motion recognition multi-device content.

Preferably, the step of recognizing the motion generated by the operation of the leader pen by the user may further include:

Also, preferably, the step of recognizing motion may be performed immediately before or after the step of selecting any one of the objects.

Further, preferably, when the leader pen is brought close to any one of the plurality of objects within a predetermined recognition range at the step of selecting the object, it is possible to recognize a unique marker included in any one of the objects.

According to the present invention, it is possible to provide various types of information that can not be provided through a limited medium such as a book, a learning wall, etc. to a pen-motion recognition multi-function device capable of providing the reader pen through various media by a simple operation of moving the predetermined pen- There is an effect of providing a device content realization system and method.

When the reader pen is applied to the learning system, the learning information is delivered in various forms through various media, thereby inducing the interest of the learner.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

1 is a configuration diagram of a pen-motion-aware multi-device content implementation system according to a preferred embodiment of the present invention;
2 is a block diagram of a leader pen according to a preferred embodiment of the present invention,
3A to 3C are diagrams illustrating motion of a leader pen according to a preferred embodiment of the present invention,
4 is a flowchart illustrating a method of implementing pen motion recognition multi-device contents according to a preferred embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features, and advantages of the present invention will become more readily apparent from the following description of preferred embodiments with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described herein but may be embodied in other forms. Rather, the embodiments disclosed herein are provided so that the disclosure can be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

In this specification, when an element is referred to as being on another element, it may be directly formed on another element, or a third element may be interposed therebetween. Further, in the drawings, the thickness of the components is exaggerated for an effective description of the technical content.

It is to be understood that when an element, component, apparatus, or system is referred to as comprising a program or a component made up of software, it is not explicitly stated that the element, component, (E.g., memory, CPU, etc.) or other programs or software (e.g., drivers necessary to run an operating system or hardware, etc.)

It is also to be understood that the elements (or elements) may be implemented in software, hardware, or any form of software and hardware, unless the context requires otherwise.

Also, terms used herein are for the purpose of illustrating embodiments and are not intended to limit the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. The terms "comprises" and / or "comprising" used in the specification do not exclude the presence or addition of one or more other elements.

1 is a configuration diagram of a pen motion recognition multi-device contents implementation system according to a preferred embodiment of the present invention.

Referring to FIG. 1, a pen motion recognition multi-device contents realization system according to a preferred embodiment of the present invention comprises a content provider 100, a leader pen 200, and an output device 300.

The content provider 100 provides content composed of a plurality of objects including unique markers. Here, the plurality of objects includes pictures, texts, and backgrounds included in the contents provider 100, and each object includes a unique marker for reproducing various contents that can be expressed for each object. In addition, various contents that can be expressed by objects can be implemented in at least one form of sound, voice, image, and virtual reality.

The content provider 100 may be any one capable of providing contents in various forms such as a paper book, an electronic book, and a story book using a medium such as a computer.

The leader pen 200 is formed in a pen shape so that the user can easily grasp the leader pen 200 by hand and can select any one of a plurality of objects included in the content provider 100 and receive information have.

In the present embodiment, the leader pen 200 does not merely perform a function of simply selecting an object, but can recognize the user's predetermined motion and select the output device 300 corresponding to the motion. The leader pen 200 will be described in more detail later with reference to FIG.

The output device 300 outputs a content corresponding to a unique marker included in the object selected by the user according to the output form determined by the leader pen 200. [ In this embodiment, the output device 300 includes an internal output device and an external output device 300. [

The internal output device may be a built-in speaker built in the leader pen 200. The external output device 300 may include an external speaker capable of short-range communication with the leader pen 200, for example, a video output device such as a Bluetooth speaker and an electronic pad, and a VR HMD (Virtual Reality Head Mounted Display).

When the output device 300 is selected as a built-in speaker or a Bluetooth speaker, the content will be output only as sound or voice, and when selected as the electronic pad, sound and video will be output together. In addition, when the VR HMD is selected, the virtual reality contents are outputted and the user can receive the contents in 3D.

Although not shown in the present embodiment, the leader pen 200 may be provided with a small display. In this case, the internal output device includes an internal speaker and a built-in small display, and sound and image can be output together when the output of the content is selected through the internal output device.

2 is a block diagram of a leader pen according to a preferred embodiment of the present invention.

2, a leader pen 200 according to a preferred embodiment of the present invention includes a sensor unit 210, a motion recognition unit 220, a short range communication unit 230, a built-in speaker 240, a memory unit 250, , And a control unit 260. [

When one of the plurality of objects included in the content provider 100 is selected by the user's operation, the sensor unit 210 recognizes a unique marker included in the selected object.

The sensor unit 210 is provided at the lower end of the body of the leader pen 200 in hardware. Accordingly, when the user holds the body of the leader pen 200 by hand, it is possible to easily perform an operation of bringing the lower end of the body to the object of the contents provider 100, like using a normal pen .

The motion recognition unit 220 recognizes a motion generated by a user's operation. As mentioned above, in the conventional electronic pen, when the electronic pen is brought into a picture or character printed on a paper book, the sound or voice is outputted through a speaker built in the electronic pen.

However, according to the present embodiment, the leader pen 200 used in the present invention can be used in combination with an operation of bringing the leader pen 200 to a desired portion of the content provider 100 such as a paper book, (300) can be selected. Therefore, the motion recognition unit 220 recognizes the motion that the user takes with the leader pen 200 in hand, according to the predetermined motion information.

The motion information recognized by the motion recognition unit 220 can be set in various ways. For example, when the motion in which the leader pen 200 is rotated in the clockwise direction is taken, the voice set in the selected object can be output to the Bluetooth speaker, and the motion in which the leader pen 200 is rotated counterclockwise When the motion of drawing the "V" shape is taken, the leader pen 200 outputs the virtual reality data set in the selected object to the VR HMD .

This is merely an example for understanding the motion that the leader pen 200 can take, and the present invention is not limited thereto, and may be changed in any way considering the convenience of the user. Recognition of the motion of the leader pen 200 by the motion recognition unit 220 will be described later in more detail with reference to FIGS. 3A to 3C.

The short-range communication unit 230 supports an interface between the leader pen 200 and the external output unit 300, and typically Bluetooth can be applied. That is, if the device has a Bluetooth function, the information of the content provider 100 may be provided from the leader pen 200 through the short-range communication unit 230, and the user may provide an output desired by the user.

The built-in speaker 240 corresponds to an internal output unit provided in the leader pen 200. When the control unit 260 determines that the output format is based on the internal output unit, .

Although only the built-in speaker 240 is shown as an internal output device of the leader pen 200 in the present embodiment, another internal output device may be further provided in the leader pen 200 in some cases. For example, when the leader pen 200 is further provided with a small display, the leader pen 200 itself can be configured to provide both video and audio to the user.

The memory unit 250 stores all information necessary for operation of the leader pen 200. For example, the memory unit 250 may store preset motion information for use in recognizing motion in the motion recognition unit 220. When the external output device 300 is registered in the leader pen 200, It is possible to store information on the external output device 300 so that it can be used without any procedure.

The control unit 260 controls all the signals necessary for the operation of the leader pen 200. That is, the control unit 260 controls signal input / output between the sensor unit 210, the motion recognition unit 220, the short range communication unit 230, the internal speaker 240, and the memory unit 250.

The control unit 260 determines an output type corresponding to the motion recognized by the motion recognition unit 220. [ That is, the control unit 260 can compare the motion recognized by the motion recognition unit 220 with the motion information stored in the memory unit 250, and determine which output unit to output.

In addition, the controller 260 controls the internal or external output device 300 according to the determined output format. More specifically, when the output format is performed by the internal output device, the control unit 260 controls to output the content corresponding to the selected object through the built-in speaker. When the output format is performed by the external output device, the controller 260 controls the corresponding external output device of the external output device 300 to output the content corresponding to the selected object through the local communication unit 230.

3A to 3C are diagrams illustrating motion of a leader pen according to a preferred embodiment of the present invention.

3A to 3C, a method of using the leader pen 200 in addition to the content provider 100a including a plurality of objects will be described. A planar content provider 100a in the form of a paper book, a learning wall, or the like is exemplified. In the content provider 100a, objects of various graphic forms are included. That is, the content provider 100a includes a circular object 110a, a triangular object 110b, and a rectangular object 110c, and the circular object 110a, the triangular object 110b, The rectangular object 110c includes respective unique markers that the sensor unit 210 of the leader pen 200 can recognize.

3A, when a user wishes to receive additional voice, image, and virtual reality data while viewing the content provider 100, the user may select a circular object 110a using the leader pen 200, , The triangle object 110b, and the rectangular object 110c can be selected.

As shown in the figure, the user selects one of the plurality of objects by using the leader pen 200, the action of bringing the lower end of the leader pen 200 close to the object, that is, the rectangular object 110c ≪ / RTI > The leader pen 200 waits until the unique marker of the specific object is recognized through the sensor unit 210 until the motion of the leader pen 200 to be performed next is recognized.

Referring to FIG. 3B, the user can move the leader pen 200 as if dragging the leader pen 200 to the right (in the direction of the arrow in the drawing) after bringing the leader pen 200 to the rectangular object 110c. In this case, the motion recognition unit 220 recognizes the motion to be dragged to the right.

Referring to FIG. 3C, after the user brings the leader pen 200 to the rectangular object 110c, the user can rotate the leader pen 200 counterclockwise (arrow direction in the figure). In this case, the motion recognition unit 220 recognizes a motion that rotates counterclockwise.

In the present embodiment, the operation of bringing the leader pen 200 to a specific object is performed first, and then the motion of the leader pen 200 is followed. The user can change the shape of the leader pen 200 so as to follow the action of bringing the leader pen 200 to a specific object after performing the preset motion according to a desired output form while holding the leader pen 200 in hand have.

That is, in this embodiment, the operation of FIG. 3A is performed first and then the operation of FIG. 3B or 3C is performed. However, the operation of FIG. 3B or 3C is performed first, It is also possible to configure the operation to be performed. This can be changed in consideration of the convenience of the user.

4 is a flowchart illustrating a method of implementing pen motion recognition multi-device contents according to a preferred embodiment of the present invention.

Hereinafter, with reference to Figs. 1 to 4, a method of implementing pen motion recognition multi-device contents according to a preferred embodiment of the present invention will be described.

When the user wants to receive other information besides the information displayed on the content provider 100 while using the content provider 100, the user inputs object selection and motion using the leader pen 200 . Accordingly, when the object is selected, the sensor unit 210 recognizes the unique marker included in the object, and when the motion for selecting the output device is inputted, the motion recognition unit 220 recognizes the motion by the user's operation (S400 ). Here, the motion input may be performed immediately before or after the object is selected.

The control unit 260 determines an output type corresponding to the recognized motion through the motion recognition unit 220. [ At this time, the control unit 260 may determine the output type using the motion information pre-stored in the memory unit 250 (S410).

If the output format corresponding to the motion is the internal output motion (S420-Y), the controller 260 controls the internal speaker to output sound or voice corresponding to the selected object (S430).

If the output format corresponding to the motion is the external output motion (S420-N), the control unit 260 transmits the control signal to the corresponding output unit of the external output unit 300 through the local communication unit 230 (S440). Thus, the external output device receiving the control signal outputs sound, voice, image, and virtual reality data corresponding to the selected object (S450).

As described above, the size of the leader pen 200, which is formed in a shape of being held by the user's hand, is limited. Therefore, since a plurality of output devices can not be built in, it is preferable that the built-in speaker is normally built in. In this case, information that can be provided through the leader pen 200 is limited to sound or voice associated with pictures or characters. For example, this is the ability to read a book by voice.

However, in the reader pen 200 according to the present embodiment, in addition to an output device built in the reader pen 200, additional information that can not be provided by the content provider 100 through various output devices capable of short-distance communication such as Bluetooth is provided .

It will be understood by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Therefore, it should be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: content provider 200: leader pen
300: external output device 210:
220: motion recognition unit 230:
240: internal speaker 250: memory unit
260:

Claims (10)

A content provider comprising a plurality of objects including a unique marker;
A reader pen for determining and controlling a predetermined output form corresponding to a motion generated by a user's operation when any one of the plurality of objects is selected; And
And an output unit for outputting a content corresponding to a unique marker included in the selected object according to an output form determined by the leader pen. The method according to claim 1,
Wherein the motion generated by the user's operation is a motion that occurs immediately before or after the operation of selecting any one of the objects. The method according to claim 1,
Wherein the output device is at least one of an internal output device including a built-in speaker built in the leader pen, and an external output device including an external speaker, a video output device, and a VR HMD (Virtual Reality Head Mounted Display) Pen motion recognition multi - device content implementation system. The method of claim 3,
Wherein the leader pen comprises:
A sensor unit recognizing the unique marker included in the plurality of objects;
A motion recognition unit for recognizing a motion generated by the operation of the user;
A short range communication unit for supporting an interface with the external output unit; And
And a control unit for determining the output format corresponding to the motion recognized by the motion recognition unit and controlling the internal output unit and the external output unit according to the determined output format. Implementation system. 5. The method of claim 4,
Wherein the sensor unit recognizes a unique marker included in any one of the plurality of objects when the object approaches the object within a predetermined recognition range, . 5. The method of claim 4,
Wherein the controller controls to output a content corresponding to a unique marker included in the selected object through the embedded speaker when the determined output format is performed by the internal output device, Wherein the control unit controls the external output unit of the external output unit to output a content corresponding to a unique marker included in the selected object through the local communication unit when the external output unit is performed by the external output unit. Content implementation system. Selecting any one of a plurality of objects by a leader pen from a content provider composed of a plurality of objects including unique markers;
Determining a preset output format corresponding to a motion generated by a user's operation; And
And outputting a content corresponding to a unique marker included in the selected object according to the determined output format. 8. The method of claim 7,
And recognizing a motion generated by the operation of the leader pen by the user. 9. The method of claim 8,
Wherein the step of recognizing the motion is performed immediately before or after the step of selecting any one of the objects. 8. The method of claim 7,
Wherein when the leader pen is brought close to any one of the plurality of objects within a predetermined recognition range at the step of selecting the object, the unique marker included in the one object is recognized Method for implementing pen motion recognition multi - device content.

Images