KR20190024920A - Voice recognition image feedback providing system and method - Google Patents

Abstract

Disclosed are a system and method for providing voice recognition image feedback that a user may conveniently receive a variety of services. A device for providing voice recognition image feedback included in the system comprises: a voice recognition unit recognizing voice of a user; an image feedback unit outputting an image corresponding to the voice in a direction that the voice is inputted; and a control unit controlling the voice recognition unit and the image feedback unit. The image is outputted to a peripheral location area of the user so that the image is visible to the user. The peripheral location area is an area within a radius of three meters from a point where the user is located. At least part of the image is visible within the radius of three meters.

Description

TECHNICAL FIELD [0001] The present invention relates to a voice recognition system,

The present invention relates to a system and method for providing speech recognition image feedback.

Recently, speech recognition technology has been significantly improved. Various services using the speech recognition technology are emerging.

However, the vast majority of these services simply provide voice, and thus do not satisfy the user's desired service.

KR 10-2017-0081883 A

The present invention provides a system and method for providing voice recognition image feedback in which a user can conveniently receive various services.

According to an aspect of the present invention, there is provided a speech recognition image feedback apparatus including: a speech recognition unit for recognizing a speech of a user; An image feedback unit for outputting an image corresponding to the recognized voice; And a controller for controlling operations of the speech recognition unit and the image feedback unit. Here, at least one of the color, image quality, resolution, and size of the image or the sound that is matched with the image is varied according to the recognized voice, the mood of the user, or the biorhythm of the user, As shown in FIG.

According to another aspect of the present invention, there is provided a speech recognition image feedback apparatus including: a speech recognition unit for recognizing a speech of a user; An image feedback unit for outputting an image; And a controller for controlling operations of the speech recognition unit and the image feedback unit. In this case, even though the request of the user requesting the provision of the image is not included in the voice, the user can not recognize the voice based on at least one of the tone of the voice, the word indicating the mood included in the voice, The corresponding image is automatically provided to the area corresponding to the user's position according to the mood of the user.

The system for providing the voice recognition image feedback according to the present invention and the method for operating the voice recognition image feedback system can recognize the voice of the user and output the image corresponding to the recognized voice to the position of the user. As a result, the user can easily acquire desired information and provide services conveniently, and the voice recognition image feedback providing system can provide various services to the user.

1 is a diagram illustrating a system for providing speech recognition image feedback according to a first embodiment of the present invention.
FIG. 2 illustrates a process of providing image feedback according to an exemplary embodiment of the present invention. Referring to FIG.
3 to 5 are views showing a structure of a speech recognition image feedback apparatus according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating a system for providing speech recognition image feedback according to a second embodiment of the present invention.
7 is a block diagram illustrating a speech recognition image feedback apparatus according to an embodiment of the present invention.
FIG. 8 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a third embodiment of the present invention.
9 is a flowchart illustrating an operation of the system for providing speech recognition image feedback according to the fourth embodiment of the present invention.
10 is a flowchart illustrating an operation of the system for providing speech recognition image feedback according to the fifth embodiment of the present invention.
11 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a sixth embodiment of the present invention.
12 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to a seventh embodiment of the present invention.
13 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to an eighth embodiment of the present invention.
FIG. 14 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a ninth embodiment of the present invention.
15 is a block diagram illustrating the configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.
16 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to a tenth embodiment of the present invention.
17 is a flowchart illustrating an operation of the system for providing speech recognition image feedback according to the eleventh embodiment of the present invention.
18 is a block diagram illustrating the configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.
FIG. 19 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a twelfth embodiment of the present invention.
20 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a thirteenth embodiment of the present invention.
21 is a diagram illustrating an operation of a speech recognition image feedback providing system according to a fourteenth embodiment of the present invention.
22 is a block diagram showing the configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.
23 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to a fifteenth embodiment of the present invention.
24 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a sixteenth embodiment of the present invention.
25 is a diagram illustrating an example of a speech recognition image feedback process of FIG.
26 is a block diagram illustrating the configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.
27 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to a seventeenth embodiment of the present invention.
28 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to an eighteenth embodiment of the present invention.
FIG. 29 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a nineteenth embodiment of the present invention.
30 is a block diagram showing the configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.
31 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a twentieth embodiment of the present invention.
32 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a twenty-first embodiment of the present invention.
33 is a block diagram showing the configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.
FIG. 34 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a twenty-second embodiment of the present invention.
35 is a diagram illustrating a system for providing speech recognition image feedback according to another exemplary embodiment of the present invention.
36 is a view schematically showing a system for providing speech recognition image feedback according to another embodiment of the present invention.
37 is a diagram illustrating a system for providing speech recognition image feedback according to another embodiment of the present invention.
38 is a block diagram illustrating the configuration of a main speech recognition image feedback apparatus according to an embodiment of the present invention.
39 is a block diagram illustrating the configuration of a sub-speech recognition image feedback apparatus according to an embodiment of the present invention.
FIG. 40 is a flowchart illustrating an operation of the speech recognition image feedback providing system of FIG. 37 according to an embodiment of the present invention.
FIG. 41 is a flowchart illustrating an operation of the speech recognition image feedback providing system of FIG. 37 according to another embodiment of the present invention.
42 is a block diagram illustrating a speech recognition image feedback apparatus according to another embodiment of the present invention.
FIG. 43 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a twenty-third embodiment of the present invention.

As used herein, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. In this specification, the terms "comprising ", or" comprising "and the like should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps. Also, the terms "part," " module, "and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software .

The present invention relates to a system and method for providing image feedback to a user's location in a user's speech recognition or corresponding user peripheral location area. Here, the user peripheral location area refers to a peripheral area of the location where the user is located.

That is, the system of the present invention can provide an image corresponding to the inputted command or request to the user peripheral location area when the user inputs a specific command or request by voice. For example, the image may be provided to a user through a beam projection.

Therefore, unlike a system that provides only voice feedback, various and convenient services can be provided to a user through image feedback.

On the other hand, the image includes a two-dimensional image, a three-dimensional image, a hologram, and the like.

Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings. For the sake of convenience of explanation, it is assumed that the image is output through the beam projection, but various methods other than the beam projection capable of providing the image can be used.

FIG. 1 illustrates a system for providing speech recognition image feedback according to a first embodiment of the present invention. FIG. 2 illustrates a process of providing image feedback according to an embodiment of the present invention. 5 is a diagram illustrating a structure of a speech recognition image feedback apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the speech recognition image feedback apparatus 100 of the present embodiment may be located in a building such as a home, for example, and may include a display element 110 for information display or a touch input of a user have. However, the display element 110 is not an essential component.

When the user 102 inputs a voice, the voice recognition image feedback apparatus 100 recognizes the voice of the user 102 and analyzes the voice to extract a command, a request or a wish of the user included in the voice .

For example, if the user 102 enters a voice saying "Show me the weather for the next 10 days" or "Tell me how to go to costco in the sunny valley area" as shown in FIG. 2, The display unit 100 may output an image to the user peripheral location area 120 indicating a 10-day weather or a way to go to the costco.

For example, the voice recognition image feedback apparatus 100 may include a beam projector 200 for providing an image, and may project an image corresponding to the voice to a user peripheral location area 120. [

Here, the user peripheral location area 120 refers to a peripheral area of a location where the user 102 is located rather than a location where the user 102 is located.

For example, the user's perimeter location area 120 may be an area with a radius of a specific distance, e.g., within a radius of 3 meters, based on the location of the user 102 so that the user can visually see the image visually. have. This is because it is inconvenient for the user 102 to view the image even if the user peripheral location area 120 is too close to the point where the user 102 is located or too far. Of course, at least some of the images are within the radius of 3 meters, and not all of the images need be within the radius of 3 meters.

The user peripheral location area 120 may be front, back, side, top, or rear with respect to the user 102, Lt; / RTI > Here, the front area may correspond to a direction in which the voice is input to the voice recognition feedback apparatus 100. [

In addition, although the user perimeter location area 120 may be a floor, ceiling, wall, or the like, it is efficient for the user perimeter location area 120 to be a floor or wall since the location that the user 102 most conveniently sees is the floor or wall .

Accordingly, the user 102 can obtain desired information and the like through the image displayed in the user peripheral location area 120.

In this case, the user peripheral location area 120 may be an area where the user 102 can visually confirm the image, and is not limited to a specific size or shape. However, the image may be set to a size that can be visually confirmed to the user 102 considering the resolution and the like.

Of course, the system of the present invention does not exclude the area where the user 102 is located. However, if the user 102 outputs an image to an area where the user 102 is located, it is difficult for the user 102 to properly check the image. Therefore, it is effective that the image feedback is provided to the user peripheral location area 120 rather than the area where the user 102 is located.

Meanwhile, the speech recognition image feedback apparatus 100 may provide the user 102 with not only the image during the image feedback, but also the audio feedback.

Assuming that only voice feedback is provided, the user 102 is difficult to grasp the desired information and can not enjoy various services. However, since the system for providing the voice recognition image feedback of the present invention provides the user 102 with the desired information and other services through the video, the user 102 can easily grasp the desired information, can do.

In summary, the speech recognition image feedback apparatus 100 of the present invention recognizes the voice of the user 102 and outputs the related image to the user peripheral location area 120 according to the recognized voice.

In order to perform such a function, the voice recognition image feedback apparatus 100 basically includes a microphone, a voice feedback unit for recognizing the voice and grasping the voice contents, a user position tracking unit (position tracking unit), for example, And may include an image output element 200, for example, a beam projector.

Hereinafter, the user position tracking element for grasping the position of the user 102 in order to provide an image will be briefly described.

According to one embodiment, the user location tracking element detects the direction of the recognized voice and analyzes the magnitude of the recognized voice under the detected direction to determine the distance between the voice recognition image feedback device 100 and the user 102 Can be detected. That is, the user location tracking device may track the location of the user 102 based only on the recognized voice. However, this method of location tracking may degrade in terms of accuracy.

According to another embodiment, in order to accurately track the position of the user 102, the user position tracking element may comprise an imaging device (e.g., a camera). Specifically, the user location tracking device may position the camera in the direction of a voice input to the voice recognition image feedback device, and photograph the image through the camera to accurately track the location of the user.

For example, the user location tracking element extracts an object through various methods such as boundary region analysis through pixel change of pixels in the photographed image, and sets a point where the extracted object is located to a position of the user 102 You can decide.

As another example, when the image includes a plurality of objects, the user location tracking element may extract a voice input from the voice recognition image feedback device among the objects (persons) remaining after removing an object that is not a person, It is possible to determine the object that best matches the direction as the location of the user 102. [ In this case, since the objects other than the object corresponding to the location of the user 102 (for example, TV) are removed, the noise can be eliminated. Here, the object corresponding to the person means an object corresponding to the shape of a person.

As another example, if the image includes a plurality of objects, the user location tracking device may determine the location of an object located at a distance corresponding to the size of the voice among the objects corresponding to the person, . In this case, when a plurality of objects exist at a distance corresponding to the size of the voice, the user location tracking device transmits an object matching the direction of the voice input to the voice recognition image feedback device to the location of the user 102 Or may determine an arbitrary object as the location of the user 102.

That is, the voice recognition image feedback apparatus can accurately track (detect) the position of the user 102 through voice and video.

In order to acquire an image of such a user, the user location tracking device may be rotatable or a plurality of user location tracking devices should be installed in the voice recognition image feedback device 100.

Specifically, since the user 102 can be positioned in front of, behind, left, and right of the voice recognition image feedback apparatus 100 and the voice recognition image feedback apparatus 100 is generally fixedly installed, It is effective that the user position tracking device is installed in a rotatable structure or a plurality of user position tracking devices are fixedly installed at different positions.

For example, a guide capable of rotating the user position tracking element may be formed on the body of the speech recognition feedback device 100 and the user position tracking element may be rotated along the guide. In this case, the user position tracking device itself may have a rotatable structure.

As another example, four user position tracking elements may be installed in the user position tracking elements at 90 degree intervals.

Hereinafter, the structure of the image feedback unit for outputting an image will be described.

Since the user 102 may exist at various positions, the image feedback unit may be rotatable or a plurality of image feedback units should be installed at intervals of, for example, 90 degrees.

According to one embodiment, as shown in FIG. 3, a beam projector 200 as an image feedback unit may be installed in a rotatable structure on the body 300 of the voice recognition image feedback apparatus 100.

Specifically, a circular guide 310 is formed on the body 300 along the periphery of the body 300, and the beam projector 200 can move along the guide 302. That is, the beam projector 200 is rotatable 360 degrees along the guide 302. Of course, the beam projector 200 itself can also be rotated.

4, a first guide 400 may be formed on a portion of the body 300 and a second guide 402 may be formed on another portion of the body 300. The guides 400 and 402 The respective beam projectors 200a and 200b may be installed. In this case, the guides 400 and 402 may be formed along the periphery of the body 300 so that the beam projectors 200a and 200b can rotate 360 degrees in all directions. Of course, although two guides are shown in FIG. 4, three or more guides may be formed.

According to another embodiment, a beam projector 500 may be formed outside the body 300 as shown in FIG. Of course, a guide may be formed in the body 300 so that the beam projector 500 can rotate along the periphery of the body 300.

That is, the beam projector may be formed in the voice recognition image feedback apparatus 100 itself as long as it can rotate, or may be installed outside.

According to another embodiment, beam projectors may be formed on the body 300 of the voice recognition image feedback apparatus 100 in the directions of the east, west, north, south, respectively. In this case, the beam projectors can output images to a predetermined area while having a fixed structure.

Of course, the beam projector can also be rotatable in itself, and as a result, the beam projector can freely output an image in a desired direction within a predetermined area.

FIG. 6 is a diagram illustrating a system for providing speech recognition image feedback according to a second embodiment of the present invention.

Referring to FIG. 6, the speech recognition image feedback apparatus 100 may include a beam projector 200 as an image feedback unit, and may provide image feedback according to a user's voice.

However, if the user designates the position to which the image feedback is to be provided by voice, the voice recognition image feedback apparatus 100 may set the position specified by the user, not the user's peripheral position area, Distance, angle, image quality, etc.).

For example, as shown in FIG. 6, when the user says "display on the wall ", the voice recognition image feedback apparatus 100 can output the image to the wall rather than the user's peripheral position area.

According to one embodiment, the speech recognition image feedback apparatus 100 can analyze the voice of the user through natural language analysis, and can grasp the position designated by the user according to the analysis result. Here, the position designated by the user may be specifically specified or may include various positions.

If the location specified by the user is specifically specified, for example, if the user says "the wall where the tv of the living room is located ", the voice recognition image feedback device 100 may provide image feedback to the specified wall . In this case, it is not necessary to grasp the position of the user.

On the other hand, when the location designated by the user includes several locations, for example, when a user speaks a "living room wall ", since there are a plurality of walls in the living room, You can provide visual feedback to the nearest wall. In this case, it is required to grasp the position of the user.

In summary, the speech recognition image feedback apparatus 100 of the present embodiment can provide image feedback to the designated position when a position to which the image is fed back to the user's voice is designated.

In the above, the operation of a basic speech recognition feedback system has been described. Hereinafter, the operation of the system for providing various speech recognition image feedback will be described in detail with reference to the accompanying drawings. However, for convenience of description, reference numerals for components will be omitted.

In addition, since the image feedback can be provided not only to the user peripheral location area but also to the user location area and the user peripheral location area can be widely included in the user location, The location of the user is briefly described.

Hereinafter, various embodiments having a basic structure for recognizing speech and performing image feedback will be described first.

7 is a block diagram illustrating a speech recognition image feedback apparatus according to an embodiment of the present invention.

7, the voice recognition image feedback apparatus of the present embodiment includes a control unit 700, a voice recognition unit 702, a position tracking unit 704, an image feedback unit 706, an audio feedback unit 708, Gt; 710 < / RTI >

The voice recognition unit 702 recognizes the voice of the user and analyzes the voice to grasp the request of the user to which the voice transmits. The voice recognition unit 702 is responsible for voice recognition related functions.

The location tracking unit 704 can track the location of the user in the direction of the voice input to the voice recognition image feedback apparatus. The location tracking unit 704 is responsible for location tracking related functions.

The image feedback unit 706 may provide the user with an image corresponding to the voice. For example, the image feedback section 706 may be a beam projector. The image feedback unit 706 performs functions related to image provisioning.

The voice feedback unit 708 may provide a sound (voice) corresponding to the recognized voice to the user. For example, the audio feedback section 708 may include a speaker. The voice feedback unit 708 is responsible for functions related to voice provisioning.

The storage unit 710 stores various kinds of information such as voice and video.

The control unit 700 generally controls the operation of the components of the speech recognition image feedback apparatus.

Although not shown, the voice recognition image feedback apparatus may further include an environment setting unit and the like.

The environment setting unit sets an environment of the voice recognition image feedback apparatus, and can register, for example, a user, a user voice, and the like.

FIG. 8 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a third embodiment of the present invention.

Referring to FIG. 8, the speech recognition image feedback apparatus recognizes the user's voice (S800). For example, the speech recognition image feedback apparatus extracts natural words included in a user's voice and analyzes the extracted natural words to recognize a user's command, request, and the like.

Then, the speech recognition image feedback apparatus removes noise from the recognized speech (S802).

For example, when a plurality of sounds are received, the voice recognition image feedback device may recognize the voice received with the loudest voice as a voice of the user and filter out the remaining voice.

As another example, when a sound from another apparatus such as a TV is received, the speech recognition image feedback apparatus can filter and remove the mechanical sound, and recognize only the human voice as the user's voice.

Of course, in this case, the user can also recognize the voice which is received as the loudest sound among various sounds.

That is, the noise is all sounds except the user's voice, and the sound is filtered out.

Subsequently, the speech recognition image feedback apparatus tracks the position of the user (S804). For example, the voice recognition image feedback apparatus may track a user's position in a direction in which the voice of the user is input.

That is, the voice recognition image feedback apparatus can track the position of the user based on the size of the voice in the direction in which the voice is input.

According to another embodiment, the voice recognition image feedback apparatus may track the position of the user using the user position tracking element while the user position tracking element is positioned in the direction in which the user's voice is input.

Here, the direction in which the user's voice is input may be the direction in which the voice input with the strongest intensity is input. When the user outputs a voice, the voice recognition image feedback apparatus can micro-input the voice recognition image feedback apparatus in various directions. The voice recognition image feedback apparatus adjusts the direction of the voice inputted with the loudest sound among the directions, You can decide.

Then, the voice recognition image feedback apparatus can output an image corresponding to the voice to a user's peripheral region corresponding to the tracked user's position, and can perform beam projection, for example. Of course, not only the image but also the corresponding sound (spoken feedback) can be output together.

In summary, the speech recognition image feedback apparatus of the present embodiment filters out the remaining sounds except the user's voice as noise, and then provides an image corresponding to the user's voice to the user peripheral location area .

9 is a flowchart illustrating an operation of the system for providing speech recognition image feedback according to the fourth embodiment of the present invention.

Referring to FIG. 9, a user's voice may be registered in advance in a voice recognition image feedback device or an external computing device (for example, a cloud server) (S900).

In general, since the speech recognition image feedback apparatus is highly likely to be used in the home or in the company, it is possible to register the voice of users who are likely to use the speech recognition image feedback apparatus in advance or register the user itself.

Then, the speech recognition image feedback apparatus can recognize the user's voice among a plurality of people (S902 and S904). For example, when a plurality of people's voice is input to the voice recognition image feedback apparatus, the voice recognition image feedback apparatus determines whether there is a voice registered in advance among the input voices, It is possible to recognize the voice as a voice of the user. Here, the comparison of the voices can be made through the coincidence of the waveforms of the voice signals.

On the other hand, if there are two or more pre-registered voices among the input voices, the voice recognition image feedback device can determine the voice inputted with the loudest sound among the pre-registered voices as the user's voice.

According to another embodiment, when a user other than the user's voice is previously registered, the voice recognition image feedback device determines whether there is a pre-registered user among the users who input the voices, It can be determined by the user's voice.

For example, when the user recognizes the voice as "I am withdrawn, and the weather is shown", the voice recognition image feedback device determines whether or not the user is a registered user, and if the user is a registered user, As the voice of the user.

Subsequently, the speech recognition image feedback apparatus can track the position of the user in the direction in which the voice is input (S906).

Then, the speech recognition image feedback apparatus can provide the image to the user peripheral location area corresponding to the tracked user's location (S908).

In summary, the voice recognition image feedback system of the present embodiment can register a voice or a user in advance, determine the voice of the user based on the registration, and provide the user with an image corresponding to the determined voice of the user. Accordingly, it is possible to prevent the use of the voice recognition image feedback providing system of the non-registered user.

According to another embodiment, a gaze recognition method can be used to recognize one of a plurality of users. A user who has provided a voice command to the voice recognition image feedback apparatus is likely to be watching the voice recognition image feedback apparatus at the time of the command. Accordingly, the voice recognition image feedback apparatus can detect the user's gaze using, for example, a camera, and provide the detected image to the user who is looking at the voice recognition image feedback apparatus.

For example, the voice recognition image feedback device may acquire an image of a direction in which the voice is input by a camera, identify users in the obtained image, extract an eye part of the identified users using various algorithms, It is possible to extract a user who directly observes the speech recognition image feedback device.

Meanwhile, when there are a plurality of users viewing the voice recognition image feedback device, the voice recognition image feedback device may directly determine a user who views the voice recognition image feedback device as a user to be provided with an image, Thus, it is possible to sequentially or simultaneously provide the images to all the users detected to view the voice recognition image feedback apparatus.

For example, the voice recognition image feedback apparatus may rotate the image feedback unit or move the image feedback unit of the hinge structure up and down, thereby sequentially providing the images to the detected users.

As another example, when the speech recognition image feedback apparatus includes a plurality of image feedback units, the same image may be simultaneously provided to the detected users through the image feedback units.

10 is a flowchart illustrating an operation of the system for providing speech recognition image feedback according to the fifth embodiment of the present invention.

Referring to FIG. 10, the speech recognition image feedback apparatus recognizes the user's speech and analyzes the recognized speech contents (S1000).

The voice recognition image feedback apparatus determines whether the voice of the user includes a plurality of results (content) according to the analysis (S1002).

For example, the user can input a voice with the voice recognition image feedback device as "Show school album ". In this case, the "school album" may be an elementary school album, a middle school album, a high school album, or a university album. That is, a plurality of results can be presented to the user.

Accordingly, the speech recognition image feedback apparatus determines whether or not the recognized speech includes a plurality of results.

If the recognized voice contains only a single result, that is, if the user's request is clear, the voice recognition image feedback device may track the location of the user (S1006) and provide image feedback to the tracked location ( S1008).

On the other hand, when the recognized voice includes a plurality of results, it is difficult to determine which result should be provided as an image. Therefore, the speech recognition image feedback apparatus can inquire the user about selection of a plurality of results (S1004).

For example, the speech recognition image feedback device may request a selection of a desired one of a plurality of results as a sound or an image. At this time, the user can select the desired result by voice or select the desired result through the display element.

When the user selects a desired result, the voice recognition image feedback apparatus tracks the position of the user in the direction in which the voice is input (S1006), and provides the image corresponding to the result to the tracked position (S1008).

In summary, the speech recognition image feedback system of the present embodiment allows a user to select a desired result when a user's voice containing a plurality of results is input, and then provide the user with an image corresponding to the result.

Although the above description refers to a service providing method when a user's voice includes a plurality of results, the contents (request) according to the user's voice may not be grasped. In this case, the voice recognition image feedback apparatus can not easily grasp the desired result by the user, so that the user can request the user to input definite contents again.

Here, when it is difficult to grasp the contents of the voice, when a user inputs a voice which is not related to the provision of a picture, such as "Let's play, If the user does not exist, the request is unclear such as "Would you like to show an album? &Quot;, and if the voice contains a word registered as a limited word in the voice recognition image feedback device, A case in which the voice recognition image feedback apparatus can not provide the request, and the like.

That is, the voice recognition image feedback system of the present embodiment can request a new input to the user when the voice of the user includes a plurality of results or when the voice recognition image feedback apparatus is difficult to grasp the contents of the voice. The new input may be a selection of results or a new voice input.

In the above, the user has selected a specific result for a plurality of results, but a desired result may be registered in advance in the speech recognition image feedback device. As a result, an image including the specific result can be provided to the user even if the user does not select a specific result.

Specifically, the speech recognition image feedback apparatus can register a desired result in its own memory or an external computing device in advance. For example, the speech recognition image feedback device may register a "high school album" to match the word "album ".

That is, the speech recognition image feedback apparatus can register a desired result in a specific word, phrase or sentence in advance. The registration of the specific word, phrase or sentence may be made by the user designating by voice and may be performed by directly manipulating the display element.

Therefore, the speech recognition image feedback apparatus can recognize that, for example, when the user inputs a voice of "show school album", "album" refers to "high school album".

The voice recognition image feedback apparatus tracks the position of the user in the direction in which the voice is input, and displays an image corresponding to the voice, i.e., an image including a desired result, in a user's peripheral region corresponding to the tracked position Can be output.

On the other hand, the registration of the desired result can be done on a user-by-user basis. For example, user A may match "album" to "high school album", but user B may register "album" to match "university album".

That is, different results may be matched for different users for the same word. In this case, the voice recognition image feedback apparatus can recognize a user corresponding to the voice at the time of voice input, select a result corresponding to the recognized user, and provide the selected result to the user.

11 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a sixth embodiment of the present invention.

Referring to FIG. 11, the speech recognition image feedback apparatus of the present embodiment recognizes the user's voice (S 1100).

Then, the voice recognition image feedback apparatus tracks the position of the user in the direction in which the voice is input (S1102).

Subsequently, the speech recognition image feedback apparatus provides image feedback to a user peripheral location area corresponding to the tracked user's position (S1104).

Then, the speech recognition image feedback apparatus tracks the position of the user again (S1106).

If the user is stationary at a location where he or she has output audio, it is not necessary to trace the location of the user again, but the user can move to another location. In this case, the position of the user is tracked again, so that the image feedback can be provided to the peripheral region of the point where the user is located.

Accordingly, the speech recognition image feedback apparatus can continuously track the position of the user.

Here, the location tracking of the user may be performed through voice analysis of the user continuously input. That is, when the user moves and outputs a voice, a voice will be input to the voice recognition image feedback device in the direction in which the user has moved. Accordingly, the voice recognition image feedback apparatus can continuously track the user's voice and track the location of the user in real time. Of course, the tracking of the user's position may be performed by photographing the image with a camera.

When the user moves to another position, the voice recognition image feedback apparatus provides image feedback to the other position (S1108).

In summary, the speech recognition image feedback system of the present embodiment can continuously track the position of the user and provide the image feedback to the user's peripheral region corresponding to the current position of the user. As a result, when the user moves, the image is also moved and provided.

In the above, although the position of the user is retraced after the provision of the image feedback (S1104), the position of the user may be retraced before S1104.

Of course, although the above is referred to as the location retrace of the user, the user location tracking device can continuously track the user's position until the provision of the image feedback is finished after speech recognition. As a result, image feedback can be efficiently provided to the user at any position of the user.

12 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to a seventh embodiment of the present invention.

Referring to FIG. 12, the speech recognition image feedback apparatus of the present embodiment recognizes the user's voice (S1200).

Then, the speech recognition image feedback apparatus performs a natural language analysis on the recognized speech, and tracks the location of the user in the direction in which the speech is input (S1202).

Subsequently, the speech recognition image feedback apparatus determines whether image feedback is to be provided according to the natural language analysis result (S1204).

There may be a case where the natural language analysis result image (image feedback) needs to be provided and a case where only the sound (spoken feedback) is provided without the image is sufficient. Accordingly, the speech recognition image feedback apparatus determines whether the image needs to be provided through the natural language analysis.

The determination of the image feedback or the provision of the voice feedback may be based on previously registered words, phrases or sentences, or may be based on the number of words included in the results to be provided according to the voice, Or whether the request is related to an image, or whether the request is related to an image.

If it is determined that the image does not need to be provided, the voice recognition image feedback apparatus can output only the sound without outputting the image (S1208).

For example, when the user inputs a voice to know the weather of today, the voice recognition image feedback device determines that the user can be provided with a desired result with only voice output, Can be output.

As another example, if the number of words included in the result of the voice is 5000 words or less, for example, or if there is no image corresponding to the voice, the voice recognition image feedback apparatus can output only the corresponding sound.

On the other hand, if it is determined that the image needs to be provided, the voice recognition image feedback apparatus may provide image feedback (S1206). Of course, in this case, audio output may also be performed together with the image feedback.

For example, when the user inputs a voice to know the daytime weather, the speech recognition image feedback device determines that the information on daytime weather can not be sufficiently provided by voice output, Can be output.

In another example, when the number of words included in the result of the voice exceeds 5000 words, for example, the voice recognition image feedback apparatus can output the corresponding image.

As another example, in the case of a user inputting a voice such as "show a movie ", that is, a voice input in which an image is essentially provided, the voice recognition image feedback apparatus can provide the corresponding image to the user.

In summary, the speech recognition image feedback apparatus of the present embodiment can selectively provide speech feedback or image feedback by analyzing the user's speech.

In this case, the speech recognition image feedback apparatus can automatically provide the speech feedback or the image feedback through the speech analysis.

Of course, if the user explicitly selects whether he or she wants the audio feedback, or if the user desires to input the audio, the voice recognition image feedback apparatus can provide the related information through the method selected by the user.

For example, when the user inputs a voice to provide "today's weather" as an image, the voice recognition image feedback device can provide information on "today's weather" to the user as an image.

13 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to an eighth embodiment of the present invention.

Referring to FIG. 13, the speech recognition image feedback apparatus of the present embodiment recognizes the user's voice (S1300).

Then, the voice recognition image feedback apparatus tracks the location of the user in the direction in which the voice is input (S1302).

Subsequently, the speech recognition image feedback apparatus provides the audio feedback or provides the image feedback to the user peripheral region corresponding to the tracked user position (S1304).

Next, the speech recognition image feedback apparatus provides image feedback or voice feedback (S1306).

In summary, the speech recognition image feedback apparatus of the present embodiment can sequentially provide the image feedback after providing the audio feedback, or sequentially provide the audio feedback after providing the image feedback.

For example, if a user spoken "Tell me the weather", the weather can include today's weather and daytime weather. In this case, the speech recognition image feedback apparatus can provide " day weather "as visual feedback after providing" today's weather "

Therefore, even when the user requests comprehensive information, the voice recognition image feedback apparatus can appropriately provide information through voice and image.

Wherein the request for the comprehensive information includes at least one of the results being provided as spoken feedback when a plurality of results are to be provided to the user in response to the request, May be provided as image feedback, when both the audio feedback and the image feedback can be provided when the determination criterion in the seventh embodiment is applied, and the like.

FIG. 14 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a ninth embodiment of the present invention.

Referring to FIG. 14, the speech recognition image feedback apparatus recognizes the user's voice (S1400).

Then, the speech recognition image feedback apparatus determines a location where the image is to be provided through the speech analysis, for example, natural language analysis included in the speech (S1402). Here, the position may be other than the position of the user. However, if the user specifies his or her own location, the determined location may be the location of the user.

For example, when the user inputs a voice saying "show the weather on the wall ", the voice recognition image feedback device can recognize the natural word" wall " and determine the "wall"

As another example, when the user inputs a voice saying "Show a picture to a friend next to me ", the voice recognition image feedback device recognizes a natural word" friend " You can decide.

As another example, if the user inputs a voice saying "show a picture to one of the friends ", the voice recognition image feedback device recognizes the phrase" one of friends ", randomly selects one of the friends, The area can be determined as the position where the image is to be provided.

The location of the speech recognition image feedback apparatus may include a word extracting unit, a word selecting unit, and a positioning unit for voice analysis.

The word extracting unit extracts words included in the speech.

The word selecting unit removes words other than words related to the position (e.g., a wall, a friend, and a floor) from among the extracted words. Here, the word related to the position may be registered in advance in the speech recognition image feedback apparatus.

The location determination unit may determine a location where the image is to be provided through a word associated with the location and a combination of words, phrases, or sentences combined with the word.

For example, when the user recognizes that the user is speaking, "Show me the weather on the wall while he is sitting on the living room floor ", the location determination section sees" living room "and" Through the combination of the "living room floor" and the " show me "associated with the word " Wall ". Therefore, the speech recognition image feedback apparatus can determine the "wall" as the position where the image is to be provided.

Subsequently, the speech recognition image feedback apparatus can provide image feedback to the determined position (S1404).

In summary, the speech recognition image feedback apparatus of the present embodiment can provide image feedback to a position designated by the user through voice, for example, a position other than the position of the user.

Hereinafter, embodiments related to power operation of the voice recognition image feedback apparatus will be described with reference to the accompanying drawings.

15 is a block diagram illustrating a configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.

15, the voice recognition image feedback apparatus of the present embodiment includes a control unit 1500, a voice recognition unit 1502, a position tracking unit 1504, a mode unit 1506, a power unit 1508, an image feedback unit 1510 ), An audio feedback unit 1512, and a storage unit 1514.

The voice recognition unit 1502 recognizes the voice of the user and analyzes the voice to grasp the request of the user included in the voice. The speech recognition unit 1502 takes charge of functions relating to speech recognition.

The location tracking unit 1504 may track the location of the user in the direction of the voice input to the voice recognition image feedback apparatus. The location tracking unit 1504 is responsible for location tracking related functions.

The mode unit 1506 manages all functions related to the mode such as the sleep mode, the active mode, and the call mode. Particularly, the mode unit 1506 can control a mode for power supply control.

The power supply unit 1508 manages power supply to the components of the voice recognition image feedback apparatus.

The image feedback unit 1510 may provide the user with an image corresponding to the voice. The image feedback unit 1510 is responsible for providing images.

The voice feedback unit 1512 may provide a sound (voice) corresponding to the recognized voice to the user. For example, the audio feedback unit 1512 may include a speaker. The voice feedback unit 1512 is responsible for functions related to voice provisioning.

The storage unit 1514 stores various kinds of information such as voice and video.

The control unit 1500 controls the operation of the components of the speech recognition image feedback device as a whole.

16 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to a tenth embodiment of the present invention.

Referring to FIG. 16, the voice recognition image feedback apparatus of the present embodiment detects whether there is a person around the voice recognition image feedback apparatus in the sleep mode (S1600).

Herein, the sleep mode refers to a state in which the remaining elements except for a sensing element (for example, a human body sensing element or a motion sensing element) for sensing the person and a part of the processor for managing the motion are off .

If there is no user to input a voice, the voice recognition image feedback apparatus does not have to be turned on to save energy, so that the voice recognition image feedback apparatus normally exists in a sleep mode.

Then, when a person is detected, the voice recognition image feedback apparatus is switched from the sleep mode to the active mode, i.e., activated (S1602). That is, all components of the speech recognition image feedback device may be turned on to recognize speech and provide image feedback.

According to another embodiment, among the components of the speech recognition image feedback apparatus, the components related to speech recognition are turned on, while the components related to the image feedback may be in an off state or in a standby mode. The components associated with the image feedback may be turned on when the user's voice is recognized.

Subsequently, the speech recognition image feedback apparatus recognizes the user's voice (S1604).

Then, the voice recognition image feedback apparatus tracks the position of the user in the direction in which the voice is input (S1606).

Subsequently, the speech recognition image feedback apparatus provides image feedback to a user peripheral location area corresponding to the tracked user position (S1608).

In summary, the voice recognition image feedback apparatus of the present embodiment exists in the sleep mode before the user is perceived in the vicinity, and can be switched to the active mode when the user is detected. As a result, unnecessary power dissipation can be prevented.

According to another embodiment, the voice recognition image feedback device may be switched from the sleep mode to the active mode when a specific device such as illumination is activated in a space in which the voice recognition image feedback device is not used .

17 is a flowchart illustrating an operation of the system for providing speech recognition image feedback according to the eleventh embodiment of the present invention.

Referring to FIG. 17, the speech recognition image feedback apparatus of the present embodiment recognizes the user's voice (S1700). In this case, the remaining components of the speech recognition image feedback device except the components related to the speech recognition may be in an off state, i.e., in a sleep mode.

Then, the voice recognition image feedback apparatus determines whether to activate the remaining components (S1702).

For example, the voice recognition image feedback apparatus can determine whether the recognized voice corresponds to the pre-registered voice. That is, the speech recognition image feedback device need not provide image feedback for all users, and can provide image feedback only for previously registered users.

Therefore, in this case, when the voice of the pre-registered user is not detected, it is not necessary to turn on the remaining constituent elements. In order to save energy, only when the voice of the pre-registered user is detected, - It is efficient to turn on.

If the recognized voice is not a pre-registered voice, the voice recognition image feedback device maintains a sleep mode. That is, the remaining components except for the components related to the voice recognition remain in the off state.

On the other hand, if the recognized voice is a pre-registered voice, the remaining components of the voice recognition image feedback device are turned on (i.e., in an active mode), that is, power is supplied to the remaining components, The recognition image feedback apparatus tracks the position of the user in the direction in which the voice is input (S1704).

Subsequently, the speech recognition image feedback apparatus provides image feedback to a user peripheral location area corresponding to the tracked user position (S1706).

In summary, the speech recognition image feedback apparatus of the present embodiment is not activated in response to all users, but can be activated only in response to a pre-registered user.

For example, if only family members are registered in advance in the voice recognition image feedback apparatus, a person other than the family can not operate the voice recognition image feedback apparatus.

The voice recognition image feedback apparatus may image-feed back information of the family members. As a result, if another person can arbitrarily activate the voice recognition image feedback device, the information of the family members may be leaked. Therefore, in order to block such information leakage, the voice recognition image feedback device can be activated in response to only the pre-registered users.

According to another embodiment, the information on the remaining users other than the voice-recognized users among the pre-registered users may be provided in response to the voice-recognized user, even if the voice-recognition image feedback apparatus is activated, . In other words, private information may not be provided to the users other than the user.

Of course, the voice recognition image feedback apparatus can provide the remaining information other than the private information through image feedback to all the previously registered users.

Hereinafter, embodiments related to the operation of the voice recognition image feedback apparatus for detecting a movement of a user will be described with reference to the accompanying drawings.

18 is a block diagram illustrating the configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.

18, the voice recognition image feedback apparatus of the present embodiment includes a control unit 1800, a voice recognition unit 1802, a motion recognition unit 1804, a user analysis unit 1806, a position tracking unit 1808, An audio feedback unit 1812, and a storage unit 1814. [0154]

The voice recognition unit 1802 recognizes the voice of the user and analyzes the voice to grasp the request of the user included in the voice. The voice recognition unit 1802 is responsible for functions related to voice recognition.

The motion recognition unit 1804 can grasp the motion of the user using the motion detection sensor, and can grasp, for example, the hand motion of the user.

The user analysis unit 1806 can grasp the user's intention or mood through the detected movement of the user. The intention or mood according to the user's motion thus detected can be used to assist in determining the position or image size, color, etc. of the image to be fed back.

The location tracking unit 1808 may track the location of the user through the direction of the voice input to the voice recognition image feedback apparatus or the movement of the user. This location tracking unit 1808 is responsible for the location tracking related functions.

The image feedback unit 1810 may provide the user with an image corresponding to the voice. The image feedback unit 1810 performs functions related to image presentation.

The voice feedback unit 1812 may provide a sound (voice) corresponding to the recognized voice to the user. This voice feedback unit 1812 is responsible for functions related to voice provisioning.

The storage unit 1814 stores various kinds of information such as voice and video.

The control unit 1800 controls overall operation of the components of the speech recognition image feedback apparatus.

FIG. 19 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a twelfth embodiment of the present invention.

Referring to FIG. 19, the speech recognition image feedback apparatus of the present embodiment can recognize not only a user's voice but also a motion of a user's gesture or the like (S1900). Of course, for motion recognition, the speech recognition image feedback device must include a motion sensing device, e.g., a camera.

Then, the voice recognition image feedback apparatus can track the location of the user in a direction in which the voice of the user is input or a direction in which the motion is detected (S1902).

Subsequently, the speech recognition image feedback apparatus can provide image feedback to a user peripheral location area corresponding to the tracked user's position (S1904). In this case, the image feedback may be provided reflecting the voice and gesture of the user.

For example, if the user has not spoken of the "wall" as a voice but points to the wall with his finger, the speech recognition image feedback device may provide image feedback to the wall rather than the user's peripheral location area.

As another example, if the user draws a large circle by hand, the speech recognition image feedback device can provide the image in a large size.

That is, the voice recognition image feedback apparatus can provide image feedback by reflecting movement of the user as well as voice. Accordingly, the speech recognition image feedback apparatus can provide a richer variety of images to the user.

Meanwhile, the motion of the user may be recognized after speech recognition or simultaneously with speech recognition.

20 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a thirteenth embodiment of the present invention.

Referring to FIG. 20, the speech recognition image feedback apparatus of the present embodiment recognizes the user's voice (S2000).

Then, the voice recognition image feedback apparatus can grasp the user's mood and the like through the recognized voice (S2002).

According to one embodiment, the user analysis unit may include a voice analysis unit for analyzing a tone of the voice, a word for expressing the mood included in the voice, and the like, and a motion analyzing unit for analyzing the size of the user, And an analysis unit.

For example, the user analysis unit may determine that the user's mood is depressed when the voice is low, and may determine the user's mood to be joyful if the voice includes a laugh sound "haha".

In another example, the user analysis unit may determine the user's mood as joy when the user is perceived as dancing according to the user's motion analysis, and may determine that the mood of the user is sad have.

According to another embodiment, in addition to the recognized voice, the voice recognition image feedback apparatus can grasp the mood, biorhythm, and the like through the user's movement, date of birth, mood input directly by the user, and the like. Here, the biorhythm can be grasped by activating the Internet site by itself or through an external device, and inputting the date of birth on the Internet site.

Subsequently, the speech recognition image feedback apparatus tracks the position of the user in the direction in which the voice is input (S2004).

Then, the speech recognition image feedback apparatus provides image feedback to a user peripheral region corresponding to the tracked user's position (S2006). In this case, the image feedback may be provided reflecting the user's mood or the like.

For example, if the mood of the user is determined to be pleasure through the voice analysis of the user, the voice recognition image feedback device can provide a lighter color image to the user. Of course, the speech recognition image feedback device may also provide bright and cheerful sound with the image.

On the other hand, if it is determined that the mood of the user is sad, the voice recognition image feedback apparatus can provide a darker color image to the user. Basically, the image can be the same regardless of whether it is depressed when the mood is good, except for the color or the atmosphere.

In summary, the voice recognition image feedback providing system of the present embodiment can provide the information or the like through the image, the image, and the sound by reflecting the user's mood or the like when providing information according to the user's voice.

In the above, the voice recognition image feedback apparatus provides the image to the user according to the request included in the voice, but can provide the image to the user without the request.

Specifically, the voice recognition image feedback apparatus can grasp a user's mood through a general voice of a user, not a voice including a specific request, and provide the user with an image reflecting the sensed mood.

According to one embodiment, the speech recognition image feedback apparatus extracts and analyzes words included in the speech and can determine that the speech is a general speech when there is no specific request.

According to another embodiment, the speech recognition image feedback apparatus can determine that the speech is normal speech if the speech input time is longer than a preset time. When the user inputs a specific request to the voice recognition image feedback apparatus, it is highly likely that the voice is within 30 seconds, for example. Therefore, if a voice of 30 seconds or more is continuously inputted, Can be determined as general speech.

For example, when the user makes a phone call with another person, there is a high possibility that a voice is input for 30 seconds or longer. Therefore, the voice recognition image feedback device determines that the voice is a general voice, grasps the user's mood, Or sound can be provided automatically.

That is, the voice recognition image feedback providing system can grasp the mood of the user through the voice of the user and automatically provide the user with an appropriate image or voice even if there is no request from the user. Thus, the user can enjoy unexpected enjoyment.

21 is a diagram illustrating an operation of a speech recognition image feedback providing system according to a fourteenth embodiment of the present invention.

21, a speaker and an image output element (for example, a beam projector) are not included in the speech recognition image feedback apparatus 100, and a speaker 2100 and a beam The projector 2102 can be located.

Of course, the speaker 2100 and the beam projector 2102 may be connected to the voice recognition image feedback apparatus 100 wirelessly or by wire.

Although the speech recognition image feedback apparatus 100, the speaker 2100, and the beam projector 2102 are separated from each other, the operation is the same as in the previous embodiments, and a description thereof will be omitted.

Hereinafter, embodiments related to the operation of the voice recognition image feedback apparatus using not only voice but also image will be described with reference to the accompanying drawings.

22 is a block diagram showing the configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.

22, the voice recognition image feedback apparatus of the present embodiment includes a control unit 2200, a voice recognition unit 2202, an image acquisition unit 2204, a position tracking unit 2206, an image feedback unit 2208, Unit 2210, and a storage unit 2212. [0040]

The voice recognition unit 2202 recognizes the voice of the user and analyzes the voice to grasp the request of the user included in the voice. The speech recognition unit 2202 performs functions related to speech recognition.

The image acquiring unit 2204 acquires an image of the forward direction after being positioned in the direction of a voice input to the voice recognition image feedback apparatus, for example, a camera. The image thus obtained can be used to determine the position of the user and remove noise.

According to another embodiment, the image acquisition unit 2204 may not be included in the voice recognition image feedback apparatus but may be an external device. For example, the image acquisition unit 2204 may be a security camera installed in the house. In this case, the voice recognition image feedback apparatus can acquire a desired image through the image acquisition unit 2204 while communicating with the image acquisition unit 2204.

The location tracking unit 2206 can track the location of the user using the voice and the image. The position tracking unit 2206 performs functions related to position tracking.

The image feedback unit 2208 may provide an image corresponding to the voice to the user. The image feedback unit 2208 performs functions related to image provisioning.

The voice feedback unit 2210 may provide a sound (voice) corresponding to the recognized voice to the user. The voice feedback unit 2210 is responsible for functions related to voice provisioning.

The storage unit 2212 stores various kinds of information such as voice and video.

The control unit 2200 controls overall operation of the components of the voice recognition image feedback apparatus.

23 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to a fifteenth embodiment of the present invention.

Referring to FIG. 23, the speech recognition image feedback apparatus recognizes the user's voice (S2300).

Then, the voice recognition image feedback apparatus captures a user's image using a user location tracking device (e.g., a camera) in the direction of the recognized voice (S2302). In this case, the user position tracking element may be rotatable.

Subsequently, the voice recognition image feedback apparatus tracks the position of the user based on the voice and the image (S2304). Therefore, the voice recognition image feedback apparatus can track the position of the user more accurately than using the voice alone to track the position of the user.

Of course, in the process of tracking the position of the user, a process of filtering noise such as another person, object, TV sound, and the like may be additionally performed. This can be efficiently performed because there is an image captured by the camera.

For example, the voice recognition image feedback apparatus extracts objects from the image acquired by the user location tracking element, removes objects (non-human) from the extracted objects (filtering process) The user can determine an object corresponding to the input direction of the voice and the size of the voice.

As another example, the speech recognition image feedback apparatus removes (filters) the remaining voices except for the largest voices among the voices input, positions the camera in the direction in which the remaining voices are input, The position of the user can be determined using the image.

As another example, the speech recognition image feedback apparatus may extract an object from the image, and determine that the input voice is not the user's voice when the extracted object is a TV or the like.

Then, the speech recognition image feedback apparatus may provide image feedback to a user peripheral region corresponding to the tracked user's position (S2306).

In summary, the speech recognition image feedback apparatus of the present embodiment can accurately track the position of the user using not only the voice but also the image through the image capturing element.

In the above, the position of the user is determined using voice and image, but the position to be fed back may be determined to be a position other than the position of the user.

Specifically, the speech recognition image feedback apparatus tracks a position of a user using the voice and the image, determines a position where image feedback is to be provided around the position of the traced user, It is possible to provide an image corresponding to a voice.

For example, the speech recognition image feedback device may be configured to provide the user with the best viewability and image quality of the user peripheral region using the image acquired by the user position recognition device, rather than simply providing the image feedback in the direction in which the voice is input. It is possible to provide the image feedback to a region where this is good.

Here, the area where the user can view and the image quality may be good may be a region in which the image output from the voice recognition image feedback device is not distorted and can be viewed by the user as it is without much bending.

Accordingly, the speech recognition image feedback apparatus can select a region having a high level of flatness among the user's peripheral region through the image, and determine the selected region as a region to be provided with image feedback.

FIG. 24 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to a sixteenth embodiment of the present invention, and FIG. 25 is a diagram illustrating an example of a speech recognition image feedback process of FIG.

Referring to FIGS. 24 and 25, the speech recognition image feedback apparatus of the present embodiment recognizes the position of the user (S2400).

Then, the voice recognition image feedback apparatus tracks the position of the user in the direction in which the voice is input (S2402).

Subsequently, the speech recognition image feedback apparatus acquires the surrounding image of the user using the camera (S2404). Particularly, when the user designates the object 2500 by voice, for example, when the user inputs a voice of "Cucumbers will be beautifully cut ", the voice recognition image feedback device acquires the image of the object 2500 do.

Then, the voice recognition image feedback apparatus can output an image matching the actual size and shape of the obtained object to the object 2500 (S2406). For example, the speech recognition image feedback apparatus can overlap a cucumber image corresponding to an actual size and shape on a cucumber object 2500, as shown in FIG.

In particular, since the user has said "cut", the cutting line 2510 can be displayed on the cucumber 2500 so that the cucumber can be cut well. Thus, the user can easily cut the cucumber 2500 along the cutting line 2510.

In summary, the speech recognition image feedback system of the present embodiment acquires an image of the object 2500 and can output an image matching the size and shape of the object 2500 with the object 2500 to be overlapped.

As a result, the user can easily realize a desired operation.

In the above, sizing is applied to objects, but can also be applied to people.

Specifically, the voice recognition image feedback apparatus can acquire a user's image using a camera and output an image corresponding to a user's actual body size.

For example, if the user inputs a voice saying "show shirt ", the voice recognition image feedback device outputs an image for the shirt, adjusts the size of the shirt to match the user's body size, .

That is, the speech recognition image feedback apparatus can output an image including an object related to a user corresponding to a user's actual body size.

Although the body size of the user is identified through the camera in the above, the user may register his / her body size or photograph in advance in the voice recognition image feedback device or the cloud server. In this case, the voice recognition image feedback apparatus does not need to secure the image of the user separately.

Hereinafter, embodiments related to the operation of the voice recognition image feedback apparatus for providing an image after grasping a structure of a position where an image is to be provided will be described with reference to the accompanying drawings.

26 is a block diagram illustrating the configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.

26, the speech recognition image feedback apparatus of the present embodiment includes a control unit 2600, a speech recognition unit 2602, a position tracking unit 2604, a structure determination unit 2606, an image feedback unit 2608, Unit 2610, and a storage unit 2612. [0040]

The voice recognition unit 2602 recognizes the voice of the user and analyzes the voice to grasp the request of the user included in the voice. The speech recognition unit 2602 takes charge of functions related to speech recognition.

The location tracking unit 2604 can track the location of the user using the voice. The location tracking unit 2604 is responsible for location tracking related functions.

The structure determination unit 2606 can grasp the structure of the position where the image feedback is to be provided. For example, the structure determination unit 2606 can output the laser to the user peripheral area to grasp the structure of the user peripheral area. For this, the structure determination unit 2606 may include a laser generator.

The image feedback unit 2608 may provide an image corresponding to the voice to the user, and may adjust the resolution of the image based on the detected structure.

The voice feedback unit 2610 may provide a sound (voice) corresponding to the recognized voice to the user. The voice feedback unit 2610 is responsible for functions related to voice provisioning.

The storage unit 2612 stores various kinds of information such as voice and video.

The control unit 2600 controls overall operation of the components of the voice recognition image feedback apparatus.

27 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to a seventeenth embodiment of the present invention.

Referring to FIG. 27, the speech recognition image feedback apparatus of the present embodiment recognizes the user's voice (S2700).

Then, the voice recognition image feedback apparatus tracks the position of the user in the direction in which the voice is input (S2702).

Subsequently, the speech recognition image feedback apparatus detects a peripheral structure by outputting a laser around the position of the tracked user (S2704). Of course, the voice recognition image feedback apparatus may detect a surrounding structure through image capturing, not a laser. That is, the speech recognition image feedback apparatus may include a laser generator for laser output or a camera for image acquisition.

Then, the speech recognition image feedback apparatus provides image feedback to a user's peripheral region corresponding to the tracked user's position (S2706).

In this case, the speech recognition image feedback apparatus can apply a different focus to the same image according to the detected peripheral structure. That is, the speech recognition image feedback apparatus can output an image with a focus optimized for the peripheral structure detected by the laser.

Such focus adjustment can be applied not only to the detection of the surrounding structure but also to the maintenance of the image quality according to the distance. Specifically, the voice recognition image feedback apparatus outputs the image in a direction in which the voice is input, and outputs the image according to a distance between the voice recognition image feedback apparatus and the user, or at a different point depending on a predetermined image output distance The image can be output.

For example, the speech recognition image feedback apparatus may output the same image to the 1-meter point or the 3-meter point from the speech recognition image feedback apparatus depending on the situation. In this case, the image quality of the image may be different depending on the position of 1 meter or 3 meters. Therefore, the voice recognition image feedback apparatus can automatically adjust the focus of the image feedback unit so that the image quality of the image is the same or as close as possible, irrespective of the position of 1 meter or 3 meters. Of course, the voice recognition image feedback apparatus can automatically adjust the focus so that the image quality of the image displayed to the user becomes optimal.

On the other hand, the image output according to the distance can be realized by rotating the feedback unit up and down using a hinge structure, for example, and varying the focus of the image feedback unit according to the output point. Alternatively, the voice recognition image feedback device itself may be rotated up and down, rather than the image feedback section.

Of course, in the above, only the up and down movements are mentioned but not the left and right movements.

In addition, the voice recognition image feedback apparatus may not output the automatic focus adjustment method, but may output images having different image qualities and sizes according to the distance at which the images are output.

28 is a flowchart illustrating an operation of a system for providing speech recognition image feedback according to an eighteenth embodiment of the present invention.

Referring to FIG. 28, the voice recognition image feedback apparatus grasps the internal structure of the home and the like in advance through a camera, a laser, or the like (S2800).

Then, the speech recognition image feedback apparatus designates a position to provide image feedback (S2802). Of course, a specific device may be designated rather than a specific location.

According to one embodiment, the speech recognition image feedback apparatus outputs a plurality of positions capable of image feedback by display or voice, and the user can select a position to receive image feedback among the display or output positions. That is, the position at which the image feedback is to be provided may be determined by the user's selection.

According to another embodiment, the speech recognition image feedback apparatus may automatically designate at least one position suitable for receiving image feedback from the identified internal structure.

Subsequently, the speech recognition image feedback apparatus recognizes the user's voice (S2804).

Then, the voice recognition image feedback apparatus may provide an image corresponding to the voice to the designated location or device, rather than the location of the user (S2806).

For example, the speech recognition image feedback apparatus may provide the image at a position closest to the user's position among the designated positions.

As another example, the speech recognition image feedback device may provide the image at a predetermined position or at a predetermined angle predetermined by the user irrespective of the distance to the user's position. As an example, the user can separately specify themes so that movies are provided on the wall and weather is provided on the living room floor.

When the angle at which the image feedback additional image is output is set, the image feedback unit may be vertically movable through a hinge structure, for example. In this case, the distance between the point where the image is provided and the voice recognition image feedback device will vary depending on the position where the voice recognition image feedback device is installed.

As another example, the speech recognition image feedback apparatus may provide the image to the designated device.

In summary, the speech recognition image feedback system of the present embodiment can preliminarily specify a position or a device to which image feedback is to be provided, and provide image feedback to the designated position or device regardless of the position of the user.

FIG. 29 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a nineteenth embodiment of the present invention.

Referring to FIG. 29, the speech recognition image feedback apparatus recognizes the user's voice (S2900).

Then, the voice recognition image feedback apparatus tracks the location of the user in the direction in which the voice is input (S2902).

Then, the speech recognition image feedback apparatus outputs a laser or the like to the tracked user's position and its periphery to determine the structure of the position where the image is to be provided (S2904).

Then, the speech recognition image feedback apparatus can appropriately change the position where the image or image feedback corresponding to the speech is provided, according to the detected structure (S2906).

For example, when the speech recognition image feedback apparatus is to output a high-resolution image but it is determined that it is efficient to output a low-resolution image according to the structure grasp, the high-resolution image can be changed to a low-resolution image.

As another example, if it is determined that the position to which the image is to be fed is not smooth, the speech recognition image feedback device may increase the brightness of the image.

As another example, the speech recognition image feedback apparatus may change the position where the image feedback is to be provided when the position is not suitable for providing an image or a better position is detected.

Subsequently, the voice recognition image feedback apparatus can output the changed image to the user peripheral region (S2908).

In summary, the speech recognition image feedback system of the present embodiment first grasps the structure of the position where the image feedback is provided, and changes the resolution, size, image quality, etc. of the image or changes the position where the image feedback is provided .

In the above description, the speech recognition image feedback apparatus changes the image. However, the speech recognition image feedback apparatus may prepare the image in accordance with the identified structure.

Hereinafter, embodiments related to the operation of the voice recognition image feedback apparatus for providing an image after grasping a structure of a position where an image is to be provided will be described with reference to the accompanying drawings.

30 is a block diagram showing the configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.

30, the speech recognition image feedback apparatus of the present embodiment includes a control unit 3000, a speech recognition unit 3002, a position tracking unit 3004, an image feedback unit 3006, an audio feedback unit 3008, A control unit 3010, a communication unit 3012, and a storage unit 3014. [

The voice recognition unit 3002 recognizes the voice of the user and analyzes the voice to grasp the request of the user included in the voice. The voice recognition unit 3002 is responsible for voice recognition related functions.

The location tracking unit 3004 can track the location of the user using the voice. The location tracking unit 3004 is responsible for location tracking related functions.

The image feedback unit 3006 provides the user with an image corresponding to the voice.

The voice feedback unit 3008 may provide a sound (voice) corresponding to the recognized voice to the user. This voice feedback unit 3008 is responsible for functions related to voice provisioning.

The external device control unit 3010 can control an external device connected to the voice recognition image feedback device. In particular, the external device control unit 3010 may control an external device so that the image is displayed when the image is provided.

The communication unit 3012 is a path for connecting the voice recognition image feedback apparatus to an external device.

The storage unit 3014 stores various kinds of information such as voice and video.

The control unit (3000) controls overall operation of the components of the voice recognition image feedback device.

31 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a twentieth embodiment of the present invention.

Referring to FIG. 31, the speech recognition image feedback apparatus of the present embodiment recognizes the user's voice (S3100).

Then, the voice recognition image feedback apparatus tracks the location of the user in the direction in which the voice is input (S3102).

Then, the voice recognition image feedback apparatus can simultaneously control external devices while providing image feedback to a user peripheral location area corresponding to the tracked user's position (S3104).

For example, if the user says he or she wants to watch a "Titanic" movie by voice, the voice recognition image feedback device can output the movie. In this case, since the surroundings are dark for the movie viewing, the voice recognition image feedback device can turn off the ambient illumination device (external device) while lowering the illuminance while outputting the movie.

According to one embodiment, the voice recognition image feedback apparatus can directly control the external apparatus while providing the image feedback.

According to another embodiment, the voice recognition image feedback device controls the image feedback, and a separate control device connected to the voice recognition image feedback device may control the external device at the request of the voice recognition image feedback device have. That is, the voice recognition image feedback apparatus can indirectly control the external device.

In summary, the speech recognition image feedback providing system of the present embodiment can control an environment around the image by controlling not only image feedback but also external devices. The control of the external device may be performed at the request of the user, but may be performed automatically without the request of the user.

32 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a twenty-first embodiment of the present invention.

Referring to FIG. 32, the speech recognition image feedback apparatus recognizes the user's speech (S3200).

Next, the speech recognition image feedback apparatus searches for a display element (S3202). Here, the display device is not limited as long as it displays an image, and may include a TV, a smart phone, a tablet PC, and the like.

Subsequently, the voice recognition image feedback apparatus may provide an image corresponding to the recognized voice to the user through the searched display device (S3204). In this case, the display device is connected to the voice recognition image feedback device wirelessly or by wire.

In summary, the speech recognition image feedback apparatus of the present embodiment can provide the user with the related image through the display element in accordance with the voice input. This method can be used when the user is close to the display element or when it is efficient to provide the image through the display element.

That is, the voice recognition image feedback apparatus may provide the corresponding image to the user's peripheral location area corresponding to the user's location or provide the user with the display image through the display device according to the user's voice.

Whether to provide the image to the user peripheral location area or to provide the image through the display device can be determined automatically through the image quality, the user's location, the type of image, and the user's request. The criteria of such selection may be stored in advance by the user.

For example, if a user requests to view a high-definition movie by voice, the user would prefer to output the movie through the display device rather than output to the bottom of the user's surroundings. Therefore, the voice recognition image feedback device can provide a high-quality movie through the display device.

As another example, if the user requests audio news of sports news, the sports news does not require high image quality, so the user will be satisfied even if the image of the sports news is output to the user's neighboring location area. Therefore, in this case, the voice recognition image feedback apparatus can provide the image to the user peripheral location area in consideration of the viewing convenience of the user.

Hereinafter, embodiments related to the operation of the voice recognition image feedback apparatus for providing a video call function will be described with reference to the accompanying drawings.

33 is a block diagram showing the configuration of a speech recognition image feedback apparatus according to another embodiment of the present invention.

33, the speech recognition image feedback apparatus of the present embodiment includes a control unit 3300, a speech recognition unit 3302, a position tracking unit 3304, an image feedback unit 3306, an audio feedback unit 3308, A storage unit 3312,

The voice recognition unit 3302 recognizes the voice of the user and analyzes the voice to grasp the request of the user included in the voice. The voice recognition unit 3302 is responsible for voice recognition related functions.

The location tracking unit 3304 may track the location of the user using the voice. The location tracking unit 3304 is responsible for location tracking related functions.

The image feedback unit 3306 provides the user with an image corresponding to the voice.

The voice feedback unit 3308 may provide a sound (voice) corresponding to the recognized voice to the user.

The video communication unit 3310 provides a video communication using a function of providing voice and video. For the video call, the video call portion 3310 may include a communication function capable of communicating with the device of the other party.

The storage unit 3312 stores various kinds of information such as voice and video.

The control unit 3300 controls overall operation of the components of the voice recognition image feedback apparatus.

FIG. 34 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a twenty-second embodiment of the present invention.

Referring to FIG. 34, the speech recognition image feedback apparatus of the present embodiment recognizes the user's voice (S3400). Here, the voice includes a video call request.

Then, the voice recognition image feedback device dials a number designated by the user according to a video call request of the user (S3402). Here, the number designated by the user may be one of the telephone numbers registered in advance in the voice recognition image feedback apparatus, or may be a telephone number input by the user. Of course, the user may change or update the telephone number registered in the voice recognition image feedback device.

According to one embodiment, the voice recognition image feedback apparatus may include a call unit capable of performing a call function.

According to another embodiment, there is a communication device communicatively connected to the voice recognition image feedback device, and the voice recognition image feedback device can make a call connection to the number designated by the user through the communication device. Here, the communication unit of the voice recognition image feedback apparatus and the communication device are communicatively connected.

Subsequently, the speech recognition image feedback apparatus can output the image of the other party to the user peripheral location area corresponding to the user's position (S3404).

Then, the voice recognition image feedback apparatus receives voice and image of the user (S3406). The voice may be received via a microphone and the image may be received via a camera.

Subsequently, the voice recognition image feedback apparatus transmits the received voice and image to the other party's device (S3408).

Then, the voice recognition image feedback apparatus receives the voice and image of the other party (S3410).

Subsequently, the speech recognition image feedback apparatus provides the user with the voice and image of the received counterpart (S3412).

A video call can be performed through the above method.

In summary, the voice recognition image feedback system of the present embodiment can provide a video call using the function of the voice recognition image feedback apparatus to output a video image.

In the above description, a video call is directly performed while the voice recognition video feedback device provides video feedback.

Specifically, when a user makes a call request or a call request is received from the other party during the provision of image feedback, the voice recognition image feedback apparatus can switch from the normal mode to the call mode. Here, the normal mode means a mode for providing the corresponding image according to the user's voice.

Then, the voice recognition image feedback apparatus performs a video call through the method of FIG. Of course, after the video call is ended, the call mode will be switched to the normal mode.

That is, the voice recognition image feedback providing system of this embodiment can perform switching between the normal mode and the call mode.

Meanwhile, various modes such as a normal mode and a display mode in which only a communication mode is mentioned but an image is output to a display device can be set, and the modes can be switched automatically according to the situation.

35 is a diagram illustrating a system for providing speech recognition image feedback according to another exemplary embodiment of the present invention.

Referring to FIG. 35, the speech recognition image feedback system of the present embodiment may include a speech recognition image feedback apparatus 100 and a user equipment 3500.

The user device 3500 may be a device used by a user, for example, a smart phone, a remote control, or the like. Here, the user device 3500 may be connected to the voice recognition image feedback apparatus 100 wirelessly or by wire.

According to one embodiment, when the user inputs a voice to the user device 3500, the voice recognition image feedback device 100 receives information about the voice from the user device 3500 and provides the image to the user peripheral location area can do.

According to another embodiment, the voice recognition image feedback apparatus 100 can directly recognize the user's voice and provide the user with the image corresponding to the voice through the user device 3500.

According to another embodiment, the user device 3500 may be a control device of the speech recognition image feedback device 100. [ Accordingly, the user can perform power on / off, control setting, and the like of the voice recognition image feedback device 100 using the user device 3500.

According to another embodiment, in order to track the user's position, the voice recognition image feedback apparatus 100 may photograph the position of the user using the camera and the user device 3500 may recognize the user's voice. In this case, the voice recognition image feedback apparatus 100 can determine the position of the user based on the photographed image and the recognized voice.

36 is a view schematically showing a system for providing speech recognition image feedback according to another embodiment of the present invention.

Referring to FIG. 36, the speech recognition image feedback apparatus 100 of the present embodiment may be installed in a ceiling 3600 rather than a floor in a building.

In this case, since the image is output from the ceiling 3600 to the floor, a clearer image can be provided to the user.

According to another embodiment, a voice recognition image feedback device may be installed on the floor, and only the video output device may be installed on the ceiling 3600. [ The image output device can output an image corresponding to the voice to the floor of the user under the control of the voice recognition image feedback device.

According to another embodiment, the speech recognition image feedback apparatus 100 may be installed on a wall surface other than the ceiling 3600. [

In summary, the speech recognition image feedback apparatus 100 of the present embodiment can be installed at various places such as a floor, a ceiling, and a wall.

Hereinafter, the operation of the voice recognition image feedback providing system including the main device and the sub-device will be described in detail.

37 is a diagram illustrating a system for providing speech recognition image feedback according to another embodiment of the present invention.

Referring to FIG. 37, the speech recognition image feedback system of the present embodiment may include a main speech recognition image feedback apparatus 3700 (main apparatus) and at least one sub-speech recognition image feedback apparatus 3702 (sub-apparatus).

The main voice image feedback device 3700 serves to centrally control the sub voice recognition image feedback devices 3702.

The sub-speech recognition image feedback device 3702 is connected to the main audio image feedback device 3700 in a wireless or wired manner and can operate under the control of the main audio image feedback device 3700.

The structural arrangement of the speech recognition image feedback devices 3700 and 3702 is such that the main speech recognition image feedback device 3700 is likely to be located mainly in a central part of the building, for example, a living room, (3702) is likely to be located in each room, toilet, and the like.

Also, the main speech recognition image feedback apparatus 3700 is mainly located at the bottom, but the sub-speech recognition image feedback apparatuses 3702 can be arranged not only on the floor but also on the ceiling, the wall, and the like.

38 is a block diagram illustrating the configuration of a main speech recognition image feedback apparatus according to an embodiment of the present invention.

38, the voice recognition image feedback apparatus of the present embodiment includes a control unit 3800, a voice recognition unit 3802, a position tracking unit 3804, an image feedback unit 3806, an audio feedback unit 3808, A control unit 3810, a communication unit 3812, and a storage unit 3814. [

The voice recognition unit 3802 recognizes the voice of the user and analyzes the voice to grasp the request of the user included in the voice.

In addition, the voice recognition unit 3802 may receive voice from the sub-voice recognition image feedback apparatus and grasp the user's request through the received voice.

The location tracking unit 3804 can track the location of the user using the recognized voice or voice provided from the sub-voice recognition image feedback device.

According to another embodiment, the position tracking unit 3804 may receive information on the tracked position from the sub-speech recognition image feedback apparatus.

The image feedback unit 3806 provides the user with an image corresponding to the voice.

The voice feedback unit 3808 may provide a sound (voice) corresponding to the recognized voice to the user.

The sub-apparatus control unit 3810 controls the operation of the sub-voice recognition image feedback apparatus. For example, the sub-device control unit 3810 may control the sub-voice recognition image feedback device to output an image corresponding to the voice through the sub-voice recognition image feedback device.

The communication unit 3812 is a connection path between the main speech recognition image feedback apparatus and the sub-speech recognition image feedback apparatus.

The storage unit 3814 stores various kinds of information such as voice and video.

The control unit 3800 controls overall operation of the components of the voice recognition image feedback apparatus.

39 is a block diagram illustrating the configuration of a sub-speech recognition image feedback apparatus according to an embodiment of the present invention.

39, the sub-voice recognition image feedback apparatus of the present embodiment includes a control unit 3900, a communication unit 3902, a voice recognition unit 3904, a position tracking unit 3906, an image feedback unit 3910, A main unit 3912 and a main unit 3914. [

The communication unit 3902 is a connection path between the sub-speech recognition image feedback apparatus and the main speech recognition image feedback apparatus.

The voice recognition unit 3904 may receive the user's voice and transmit the received voice to the main voice recognition image feedback device. Of course, the speech recognition unit 3904 may analyze the speech itself and grasp the user's request.

The location tracking unit 3906 can track the location of the user using the voice. This location tracking unit 3906 is responsible for location tracking related functions.

The image feedback unit 3910 may receive an image corresponding to the voice from the main speech recognition image feedback apparatus and provide the received image to a user.

The voice feedback unit 3912 may provide a sound (voice) corresponding to the recognized voice to the user.

The main device unit 3914 is connected to the main speech recognition image feedback apparatus and can perform an operation related to image feedback under the control of the main speech recognition image feedback apparatus.

The control unit 3900 generally controls the operation of the components of the voice recognition image feedback apparatus.

FIG. 40 is a flowchart illustrating an operation of the speech recognition image feedback providing system of FIG. 37 according to an embodiment of the present invention, FIG. 41 is a flowchart of a speech recognition image providing system of FIG. 37 according to another embodiment of the present invention Fig.

40, when the main speech recognition image feedback apparatus (main apparatus) 3700 recognizes the user's speech (S4000), the main speech recognition image feedback apparatus 3700 or the sub-speech recognition image feedback apparatuses (sub- 3702 can track the location of the user (S4002).

Subsequently, the main speech recognition image feedback apparatus 3700 transmits the image corresponding to the speech to the sub-speech recognition image feedback apparatus 3702 closest to the traced user's position, and the sub-speech recognition image feedback apparatus 3702 transmits the sub- May output the transmitted image to a user peripheral location area corresponding to the location of the user (S4004).

41, when the sub-voice recognition image feedback apparatus 3702 receives the user's voice (S4100), the sub-voice recognition image feedback apparatus 3702 transmits information about the received voice to the main speech recognition image feedback apparatus (3702).

Subsequently, the main speech recognition image feedback apparatus 3702 tracks the position of the user through the self or sub-speech recognition image feedback apparatuses 3702, preferably the sub-speech recognition image feedback apparatus 3702 which has received the speech (S4102).

Subsequently, the main speech recognition image feedback apparatus 3700 transmits the image corresponding to the speech to the sub-speech recognition image feedback apparatus 3702 closest to the tracked user position, and the sub-speech recognition image feedback apparatus 3702 May output the transmitted image to a user peripheral location area corresponding to the location of the user (S4104).

According to another embodiment, the main voice recognition image feedback apparatus 3700 recognizes the user's voice, the main voice recognition image feedback apparatus 3700 determines the position of the user in the direction in which the voice is input, And the determined user selects the optimal sub-voice recognition image feedback device 3702 closest to the position.

Subsequently, the main speech recognition image feedback apparatus 3700 transmits the image corresponding to the speech to the optimal sub-speech recognition image feedback apparatus 3702, and the sub-speech recognition image feedback apparatus 3702 transmits the sub- And output the transmitted image to the corresponding user peripheral location area.

According to another embodiment, the main speech recognition image feedback apparatus 3700 can transmit an image according to a user's voice to a plurality of sub-speech recognition image feedback apparatuses 3702, and the sub-speech recognition image feedback apparatuses 3702 May output the transmitted images to the respective user peripheral location areas.

In this case, the images may be overlapped and output to different areas.

In addition, the images output from the sub-speech recognition image feedback devices 3702 may be the same image or different images. When the sub-speech recognition image feedback apparatuses 3702 output different images, the user can receive more information at a time with more stereoscopic feeling.

For example, when the user inputs a voice saying "Show album ", the sub-speech recognition image feedback devices 3702 assign the hologram of the" high school album " .

In this case, when the user selects a specific entity (e.g., icon) of the hologram, the main speech recognition image feedback apparatus 3700 or the sub-speech recognition image feedback apparatus 3700 detects the movement of the user, The user may additionally provide another image corresponding to the object.

The above description does not exclude the fact that the main speech recognition image feedback apparatus 3700 outputs the image, although the above description does not mention the content of the image output by the main speech recognition image feedback apparatus 3700.

In summary, the speech recognition image feedback system of the present embodiment can provide various services including the main speech recognition image feedback apparatus 3700 and the sub-speech recognition image feedback apparatuses 3702 controlled thereby.

42 is a block diagram illustrating a speech recognition image feedback apparatus according to another embodiment of the present invention.

42, the speech recognition image feedback apparatus of the present embodiment includes a control unit 4200, a speech recognition unit 4202, a position tracking unit 4204, an image feedback unit 4206, a virtual object unit 4208, A configuration unit 4210, a configuration unit 4212, and a storage unit 4214. [

The voice recognition unit 4202 recognizes the voice of the user and analyzes the voice to grasp the user's request transmitted by the voice.

The location tracking unit 4204 can track the location of the user in the direction of the voice input to the voice recognition image feedback device.

The image feedback unit 4206 may provide the user with an image corresponding to the voice and including the virtual entity.

The virtual entity 4208 plays a role of creating and managing a virtual entity. Here, the virtual entity is included in the image provided to the user.

The voice feedback unit 4210 may provide a sound (voice) corresponding to the recognized voice to the user.

The environment setting unit 4212 manages various settings of the voice recognition image feedback apparatus and can control settings related to, for example, creation of a virtual object.

The storage unit 4214 stores various kinds of information such as voice and video.

The control unit 4200 controls overall operation of the components of the speech recognition image feedback apparatus.

FIG. 43 is a flowchart illustrating an operation of a speech recognition image feedback providing system according to a twenty-third embodiment of the present invention.

Referring to FIG. 43, the speech recognition image feedback apparatus recognizes the user's voice (S4300).

Then, the voice recognition image feedback apparatus tracks the location of the user in the direction in which the voice is input (S4302).

Subsequently, the speech recognition image feedback apparatus may provide a user with a virtual entity (S4304).

For example, when the user inputs a voice saying "show shirt ", the voice recognition image feedback device provides the user with an image of a shirt with a user object (avatar), in particular a user object having a body size of the actual user .

Thus, the user can have a more realistic feel.

As another example, if the user inputs a voice saying "show day weather ", the voice recognition image feedback device can provide the user with an image including a daylight weather map and a daylight weather forecast for a woman weather forecaster.

At this time, the entity can dynamically guide daytime weather.

That is, the entity is not limited to a user entity but includes all the entities necessary for the user to transmit desired information.

In summary, the speech recognition image feedback system of the present embodiment can use a virtual entity to transmit information to a user.

In the above embodiments, only the contents of the image output by the voice recognition image feedback device are described. However, after outputting the image, the user can input a new request by voice, operation, or display touch. In this case, the speech recognition image feedback apparatus can provide a new image. Here, the provided image may be a still image that displays only simple information, or a dynamic moving image.

Further, although the specific contents set by the user in the voice recognition image feedback apparatus or external apparatus are not mentioned, all the methods set in the apparatus can be applied to the present invention. Particularly, a method of user registration and environment setting through voice or display touch will be suitable for the present invention.

On the other hand, the components of the above-described embodiment can be easily grasped from a process viewpoint. That is, each component can be identified as a respective process. Further, the process of the above-described embodiment can be easily grasped from the viewpoint of the components of the apparatus.

In addition, the above-described technical features may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

It will be apparent to those skilled in the art that various modifications, additions and substitutions are possible, without departing from the spirit and scope of the invention as defined by the appended claims. Should be regarded as belonging to the following claims.

100: speech recognition image feedback device 102: user
110: display device 200: beam projector
300: body 310, 400, 402: guide

Claims (2)

A voice recognition unit for recognizing a voice of the user;
An image feedback unit for outputting an image corresponding to the recognized voice; And
And a controller for controlling operations of the speech recognition unit and the image feedback unit,
Wherein at least one of a color, an image quality, a resolution and a size of the image or a sound matching the image is varied according to the recognized voice, the mood of the user, or the biorhythm of the user, Is output to an area corresponding to the speech recognition unit. A voice recognition unit for recognizing a voice of the user;
An image feedback unit for outputting an image; And
And a controller for controlling operations of the speech recognition unit and the image feedback unit,
The user's perception of the user based on at least one of the tone of the voice, the word indicating the mood contained in the voice, and the movement of the user, even though the request of the user requesting the provision of the image is not included in the voice And the corresponding image is automatically provided as an area corresponding to the position of the user according to mood.

Images