KR20190070568A - Apparatus of interacting with user - Google Patents

Abstract

The present invention provides an apparatus capable of interacting with a user. The apparatus comprises: an image obtaining unit obtaining an image of a target area and a user related image; an image processing unit processing the images obtained from the image obtaining unit; a content providing unit providing content to a user based on a processing result of the image processing unit; and a control unit controlling the image obtaining unit, image processing unit, and content providing unit. The image obtaining unit obtains an image using two or more cameras.

Description

[0001] APPARATUS OF INTERACTING WITH USER [0002]

The present invention relates to a device capable of user interaction, and more particularly, to a device capable of performing user interaction, which controls a content providing section in a direction of increasing the concentration or interest of a user based on a user- ≪ / RTI >

Children's play parishes, not only in Korea but also around the world, are areas where parents are growing steadily due to their love for their children and the heat of education.

According to Korean Patent Laid-Open Publication No. 10-2016-0110354 (published on Sep. 22, 2016, entitled LANGUAGE STUDY TOOL BY BRAIN ACTIVATION), " (N is a natural number) that includes a text corresponding to the target image, and a second sheet that includes text corresponding to the target image. And a second through an N-1th sheet including at least one of a background image, a pattern image, a target image, and text, wherein the first through Nth sheets are different from each other, 1 is an image corresponding to a background excluding the target image in a sheet, and the pattern image is an image including a pattern or a pattern of the target image in the text. According to the present invention, In order for a user including infants and children to learn text corresponding to a specific target image, a connection relationship is formed between the target image and the text using patterned characters (i.e., pattern image) and background image, (Or the target image from the text) is easily remapped. "

In addition, in Korean Registered Utility Model No. 20-0480000 (Registered on Mar. 25, 2016, design name: TEACHING TOOLS), "This design relates to a child learning parish, more specifically, Can be visually confirmed by an infant or a child with the shape of a fruit, an animal, a horse-drawn carriage or an automobile, and particularly, the fruit can be visually checked even in the cross section of the fruit. A child learning paradigm according to the present invention includes a board having guide grooves formed on both sides of a front surface thereof, a board that is axially coupled to the upper portion of the board, A rotation pin, a fixing pin formed on the board, and a subject of learning is printed on the front or rear surface, And a color block inserted into the guide groove formed in the board and moved along the guide groove so that the learner is collinear with the color module so as to distinguish the color Quot ;.

However, such conventional educational diaries are still lacking in various children's diaries, despite the steady growth of the industry or the market. In particular, based on user-related images or user's voices to be dealt with in the present invention It has not been possible to disclose a technique capable of controlling the content providing unit in the direction of increasing the user's concentration or interest.

Korean Patent Laid-Open No. 10-2016-0110354 (published on Sep. 22, 2016, entitled LANGUAGE STUDY TOOL BY BRAIN ACTIVATION) Registered Utility Model No. 20-0480000 (Registered on Mar. 25, 2016, Design Name: TEACHING TOOLS)

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and it is an object of the present invention to provide a user interaction device capable of controlling a content providing unit in a direction of increasing a user's concentration or interest based on a user- And the like.

It is also possible to easily extract the contour, the lattice, the character, the number or the characteristic points for the margins of the block of the object including the character, number or block to be located on the object area, It is another object of the present invention to provide a device capable of user interaction capable of performing user interaction.

Third, it is possible to provide a device capable of user interaction capable of adjusting a display unit to be customized to a user by further including a laminated type fitting structure capable of adjusting the height so as to be detachable to the mounting part in accordance with the size or height of the display part For another purpose.

The problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems which are not mentioned can be clearly understood by those skilled in the art from the following description.

According to another aspect of the present invention, there is provided an apparatus capable of user interaction, including an image acquiring unit acquiring an image of a target region and a user related image; An image processing unit for processing the image acquired by the image acquisition unit; A content providing unit for providing the content to the user based on the processing result of the image processing unit; And a control unit for controlling the image acquiring unit, the image processing unit, and the content providing unit, and the image acquiring unit acquires an image using two or more cameras.

The apparatus also includes: a mount mounted on the specific surface; And an extension formed extending from the mounting portion, wherein the image acquiring unit is composed of two cameras disposed at the end of the extending portion, and each image streamed from the two cameras is merged into a panorama image , The merged panoramic image can be processed in parallel to acquire the image of the object area and the user related image.

The apparatus also includes: a mount mounted on the specific surface; And an extension formed extending from the mounting portion, wherein the image acquiring unit includes: a target area camera disposed at an end of the extending unit to acquire an image of the target area; and a target area camera disposed at a front surface of the placing unit And a user-related camera for acquiring the user-related image, wherein the target area camera acquires an image of the target area, and the user-related camera acquires the user-related image.

A voice acquiring unit for acquiring an auditory signal including the voice of the user; And a voice processing unit for processing the voice acquired by the voice acquiring unit, wherein the content providing unit provides the content to the user based on a processing result of the image processing unit and the voice processing unit, And controls the image acquiring unit, the voice acquiring unit, the image processing unit, the voice processing unit, and the content providing unit.

Also, in order to easily extract the contour, the lattice, the character, the number, or the minutiae for the margin of the block including the character, number, or block to be located on the target area, It can be made of a divided plane.

In addition, the object area may be a planar space defined by a grid consisting of a plurality of blocks piled on a plane partitioned by a lattice.

The user-related image may be biometric information including a user's face, facial expression, eye, nose, mouth, and the content providing unit may be configured to generate a biometric information based on the user- .

The content providing unit may further include a laminated type fitting structure that can be detachably attached to the mounting unit in accordance with the size or height of the display unit, the display providing unit being disposed in the mounting unit.

According to another embodiment of the present invention, there is provided an apparatus capable of user interaction, including: an image acquiring unit acquiring an image of a target area and a user related image; An image processing unit for processing the image acquired by the image acquisition unit; A content providing unit for providing the content to the user based on the processing result of the image processing unit; And a control unit for controlling the image acquisition unit, the image processing unit, and the content providing unit, and the object in the object area is a virtual object.

Of course, the object in the target area may be an actual object, a virtual object, or both a virtual object and an actual object.

The image processing apparatus may further include a projection device for projecting a virtual object on the object area, wherein the image acquisition unit acquires the operation image of the user with respect to the virtual object projected by the projection apparatus, Processing the acquired operation image of the user, and providing the image to the content providing apparatus and the projection apparatus.

Further, the projection apparatus can correct and project information on the virtual object to be projected on the object area, based on the result of the image processing section.

Also, the object of the object area is a virtual object displayed on the touch screen, and the image obtaining unit obtains the operation image of the user with respect to the virtual object on the touch screen, And provides the operation of the user to the contents provider through the communication with the virtual object or the operation of the user with respect to the virtual object on the touch screen.

The apparatus further includes a hologram unit that implements a virtual object hologram object, and the image obtaining unit obtains the operation image of the user with respect to the hologram object, and the image processing unit performs the operation of the obtained user And may provide the operation of the user to the content providing unit by processing the image.

According to the apparatus capable of user interaction according to the present invention,

First, the content providing unit can control the user based on the user-related image or the user's voice in the direction of increasing the concentration or interest of the user.

Second, the object region is made up of planes partitioned into a grid, and the contour, the lattice, the character, the number, or the feature point for the margin of the block of the object including the character, number, It is possible to do.

Thirdly, it is also possible to provide a laminated fitting structure capable of adjusting the height so as to be detachable to the mount portion in accordance with the size or height of the display portion, so that the display portion can be adjusted in a customized manner to the user.

The effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a perspective view of a device capable of user interaction according to a preferred embodiment of the present invention and a planar area of a lattice pattern.
2 is a side view of a user interactable device in accordance with a preferred embodiment of the present invention.
3 is a front view of a user interactable device according to a preferred embodiment of the present invention.
FIG. 4 is a view showing the display unit further including the side view of FIG. 2. FIG.
5 is a diagram illustrating an example of using a device capable of user interaction according to a preferred embodiment of the present invention.
FIG. 6 is a diagram illustrating that an object area and a user-related image are simultaneously acquired as panoramic images using two cameras in an apparatus capable of user interaction according to a preferred embodiment of the present invention.
FIG. 7 is a diagram for explaining a case where a target region in a device capable of user interaction according to a preferred embodiment of the present invention is a three-dimensional or spatial region rather than a plane.
FIG. 8 is a view for explaining a case where a target area is a virtual plane area in a user interactable apparatus according to a preferred embodiment of the present invention.
FIG. 9 is a view for explaining a case where a target area is not a plane but a virtual three-dimensional space or a spatial area in an apparatus capable of user interaction according to a preferred embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. However, it is to be understood that the present invention is not limited to the disclosed embodiments, but may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. It is to be understood by those of ordinary skill in the art that the present invention is not limited to the above embodiments, but may be modified in various ways. Like reference numerals refer to like elements throughout the specification.

Unless defined otherwise, all terms (including technical and scientific terms) used herein may be used in a sense commonly understood by one of ordinary skill in the art to which this invention belongs. In addition, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. The terms " comprises "and / or" comprising "used in the specification do not exclude the presence or addition of one or more other elements in addition to the stated element.

2 is a side view of a device capable of user interaction according to a preferred embodiment of the present invention, and FIG. 2 is a side view of a device capable of user interaction according to a preferred embodiment of the present invention. 3 is a front view of a user interactable device according to a preferred embodiment of the present invention.

As shown in FIG. 1, an apparatus capable of user interaction according to the present invention includes an image acquiring unit, an image processing unit, a voice acquiring unit, a voice processing unit, a content providing unit, and a control unit.

Here, the image acquiring unit acquires images using two or more cameras. The target of the image acquisition is the image of the target area and the user related image.

In this connection, a device capable of user interaction includes a mounting part mounted on a specific surface such as a table, a table, a desk and the like, and an extension part extending from the mounting part. Here, the image acquiring unit is composed of two cameras 10 and 20 disposed at the ends of the extending unit, merging the respective images streamed from the two cameras into a panorama image, parallel processing the merged panorama images, It is possible to acquire an image of a target area and a user-related image.

Or an apparatus capable of user interaction includes a mounting part mounted on a specific surface and an extension formed extending from the mounting part, the image acquiring part includes an object area camera disposed at an end of the extension part and acquiring an image of the object area, (10 or 20), and a user-related camera (30) disposed on the front surface of the mount to acquire a user-related image, wherein the subject area camera acquires an image of the subject area, Can be obtained. Here, only the case where the image acquiring unit has two cameras has been described, but an image may be acquired using two or more cameras.

Here, the mounting portion may further include a fixing member for fixing to a table, a table, a desk or the like. This makes it more stable to mount on certain surfaces.

The image obtained by the image acquisition unit is processed by the image processing unit.

On the other hand, a device capable of user interaction can use only visual image acquisition and image processing as visual information, but also acquires audio signals including a user's voice to acquire an auditory signal (a sound of hitting a table or a user's hand) And a voice processing unit for processing the voice acquired by the voice acquisition unit.

Here, the content providing unit may provide the content to the user based on the results of the video processing unit or the audio processing unit. By using the user-related image such as biometric information including the user's face, facial expression, eyes, eyes, nose, mouth, neck, teeth, ear, hand, etc., Or in a direction to increase the degree of interest. Specifically, by tracking the pupil, the user can ascertain whether he is currently sleepy, intentional or interested. In accordance with this, it is possible to provide contents adaptively in content providing.

Further, although not shown in the drawings, it is possible to acquire additional biometric information such as a brain wave of a user by utilizing the functions of an additional sensor or a smart phone, and to provide personalized contents according to the current state of the user in content providing.

At this time, the control unit controls the image acquiring unit, the sound acquiring unit, the image processing unit, the audio processing unit, and the content providing unit.

As shown in FIG. 1, it is possible to easily extract the contour, the lattice, the character, the number, or the minutiae for the margin of the block including the character, number or block to be positioned on the target area of the plane , And the target area may be a plane 40 partitioned into a lattice. That is, since it is easy for image recognition and image processing of a target object, a grid pattern is utilized.

As shown in FIG. 1, three letters or blocks "A", "C", and "E" are arranged on the grid 40, Quot; ACE ". In the content providing unit, a photograph, a character or a description corresponding to the ACE is displayed, and the English output of "ACE" US, UK, etc., and interact with the user.

FIG. 4 is a view showing the display unit further including the side view of FIG. 2. FIG. The content providing unit is usually composed of a display unit arranged in the mounting unit. And a laminated fitting structure capable of adjusting a height of the display unit so as to be detachable in accordance with the size or height of the display unit. Specifically, the case of using Samsung Galaxy Tab and the case of using Apple's iPad may be different from each other, and it is preferable to provide a laminated type custom structure corresponding to each version.

5 is a diagram illustrating an example of using a device capable of user interaction according to a preferred embodiment of the present invention.

As described above, when a grid pattern is used, it is easy for image recognition and image processing of a target object. For this purpose, it is possible to use a grating pattern grid, but it can also be implemented without a grating pattern grid. For example, it is possible to further include a projection device projecting a grating pattern on the object area so that a planar object area partitioned by the grating is formed by projection by the projection device.

Alternatively, a grid pattern may be formed by further including a light source for irradiating light on the target area, and a grid window located between the light source and the target area. In this case, it is also possible to form a lattice pattern through the shadow of the lattice pattern window by irradiating the object area of the plane partitioned by the lattice with the lattice pattern window of the light source.

5, six characters or blocks 60 of "R", "A", "B", "B", "I" and "T" are arranged, RABBIT "in the content providing unit, displays a rabbit photograph, a character or an explanation corresponding to RABBIT in the content providing unit, and transmits the English pronunciation of" ACE " And so on, thereby interacting with the user. Furthermore, it is also possible to acquire the English pronunciation of the user from a microphone (not shown), to process the voice, and to give feedback to the user about the pronunciation.

FIG. 6 is a diagram illustrating that an object area and a user-related image are simultaneously acquired as panoramic images using two cameras in an apparatus capable of user interaction according to a preferred embodiment of the present invention.

The image acquiring unit is composed of two cameras 10 and 20 disposed at the end of the extension unit. The images obtained by merging the respective images streamed by the two cameras into a panorama image, parallel processing the merged panorama images, Area image and a user-related image can be obtained.

For this purpose, each of the two cameras 10 and 20 (an example of the angle of view of each camera may include about 75 degrees) can set a region of interest (ROI) that does not intersect with each other, Can be processed. In this case, it is possible to reduce distortion as compared with acquiring and processing an image using one wide-angle lens. For reference, the space 70 shown in FIG. 6 represents a spatial area obtained through two cameras.

FIG. 7 is a diagram for explaining a case where a target region in a device capable of user interaction according to a preferred embodiment of the present invention is a three-dimensional or spatial region rather than a plane.

As shown in Fig. 7, the object area may be a planar space defined by a lattice of a plurality of blocks piled on a plane partitioned by a lattice. In FIG. 6, it is possible to acquire stereoscopic, that is, spatial information on a plane in two or more cameras in the image acquiring unit, to recognize blocks piled on the plane. Although only the block is described here, it is possible to acquire a stereoscopic model or the like other than the block through image acquisition and image processing. In this way, it becomes possible to recognize a stereoscopic or spatial object area other than the plane object area described above.

FIG. 8 is a view for explaining a case where a target area is a virtual plane area in a user interactable apparatus according to a preferred embodiment of the present invention.

As shown in Fig. 8, the virtual object is not an actual object such as the block 60 in Fig. 6, but a virtual object is a virtual object such as an iPad 80 or a Galaxy tab 80 "R", "A", "B", "B", "I" in FIG. 5 as shown in FIG. 5 by moving a virtual object on a touch screen and placing it on a predetermined position or a grid pattern on a touch screen. , It is possible to complete the word "T ".

In this case, it is preferable that the control unit also controls the electronic product of the touch screen on which the virtual object exists. Or may operate in conjunction with each other through Bluetooth communication between them. In this case, it is also possible to acquire, process, and output auditory signals by using an iPad or a Galaxy Tab microphone or speaker serving as a touch screen.

That is, as described above, the object in the object area may be an actual object, a virtual object, or both a virtual object and an actual object. It is possible to cope with the case where there is no actual object such as missing an actual object or the case where an actual object is lacked and it is possible to have a virtual object in a form that stimulates children's imagination more.

As shown in FIG. 8, a method of implementing a virtual object is a virtual object displayed on a touch screen, and the image obtaining unit obtains an operation image of the user with respect to a virtual object on the touch screen , The image processing unit processes the obtained operation image of the user and provides the operation of the user to the content providing unit or directly provides the operation of the user of the virtual object on the touch screen to the content providing unit through communication.

The method for implementing a virtual object may further include a projection device for projecting a virtual object on an object area in addition to the form as shown in FIG. 8, and the image acquiring part may acquire an operation image of the virtual object projected by the projection device And the image processing unit can process the obtained operation image of the user and provide the image to the content providing apparatus and the projection apparatus. Here, the projection apparatus may correct the information on the virtual object to be projected on the object area based on the result of the image processing section, and project the information.

FIG. 9 is a view for explaining a case where a target area is not a plane but a virtual three-dimensional space or a spatial area in an apparatus capable of user interaction according to a preferred embodiment of the present invention.

As shown in FIG. 9, as shown in FIG. 7, the object area is expanded not to a plane but to a space, and a hologram object is formed by implementing a target object as a virtual object instead of an actual object as shown in FIG. 8 Lt; / RTI > For the implementation of the hologram object, it is preferable that the extension of the device capable of user interaction according to the present invention has a streamlined shape in which an inner space is created. The actual implementation of the hologram can be implemented utilizing a number of conventional techniques, so that a description of the actual implementation will be omitted here.

As shown in FIG. 9, in another embodiment of implementing a virtual object, an apparatus capable of user interaction further includes a hologram unit that implements a virtual object hologram object, And the image processing unit processes the obtained operation image of the user to provide the operation of the user to the content providing unit.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood. Therefore, it should be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

10 or 20 .... Target area camera
30 .... User Related Camera
40 .... Plane or grating partitioned by a grid

Claims (8)

An image acquiring unit acquiring an image of a target area and a user related image;
An image processing unit for processing the image acquired by the image acquisition unit;
A content providing unit for providing the content to the user based on the processing result of the image processing unit; And
And a control unit for controlling the image acquisition unit, the image processing unit, and the content providing unit,
Wherein the image acquiring unit acquires an image using two or more cameras,
A device capable of user interaction.
The method according to claim 1,
The apparatus comprises:
A mount mounted on a specific surface; And an extension formed extending from the mounting portion,
Wherein the image acquiring unit includes two cameras disposed at the ends of the extension unit, merges each of the images streamed by the two cameras into a panorama image, parallel-processes the merged panorama images, And acquiring the user-
A device capable of user interaction.
The method according to claim 1,
The apparatus comprises:
A mount mounted on a specific surface; And an extension formed extending from the mounting portion,
Wherein the image acquiring unit includes an object area camera disposed at an end of the extension unit to acquire an image of the object area and a user related camera disposed on a front surface of the object unit to acquire the user related image, Wherein the region camera acquires an image of the object region, and the user-related camera acquires the user-
A device capable of user interaction.
The method according to claim 2 or 3,
A voice acquiring unit for acquiring an auditory signal including the voice of the user; And
And a voice processing unit for processing an auditory signal including the voice acquired by the voice acquiring unit,
Wherein the content providing unit provides content to the user based on the processing results of the video processing unit and the audio processing unit,
Wherein the control unit controls the image obtaining unit, the voice obtaining unit, the image processing unit, the voice processing unit, and the content providing unit,
A device capable of user interaction.
The method according to claim 2 or 3,
In order to easily extract the contour line, the lattice, the character, the number, or the minutiae for the margin of the block including the character, number or block to be located on the target area, the target area is divided into a grid Comprising:
A device capable of user interaction.
The method according to claim 2 or 3,
Wherein the object area comprises a planar space defined by a lattice of a plurality of blocks stacked on a plane partitioned by a lattice,
A device capable of user interaction.
The method according to claim 2 or 3,
The user-related image is biometric information including the user's face, facial expression, eye, nose, and mouth,
Wherein the content providing unit controls the user based on the user related image in a direction to increase the user's concentration or interest,
A device capable of user interaction.
The method according to claim 2 or 3,
Wherein the content providing unit comprises a display unit disposed in the receiving unit,
Further comprising a laminated fitting structure capable of adjusting the height of the display unit so as to be detachable in accordance with the size or height of the display unit,
A device capable of user interaction.

Images