KR101741550B1 - Method and apparatus for providing optimized viewing conditions in multimedia device - Google Patents
Method and apparatus for providing optimized viewing conditions in multimedia device Download PDFInfo
- Publication number
- KR101741550B1 KR101741550B1 KR1020100112530A KR20100112530A KR101741550B1 KR 101741550 B1 KR101741550 B1 KR 101741550B1 KR 1020100112530 A KR1020100112530 A KR 1020100112530A KR 20100112530 A KR20100112530 A KR 20100112530A KR 101741550 B1 KR101741550 B1 KR 101741550B1
- Authority
- KR
- South Korea
- Prior art keywords
- user
- multimedia device
- information
- image
- unit
- Prior art date
Links
Images
Abstract
The present invention relates to an operation method and a multimedia device capable of recognizing a user in a multimedia device and providing an audio / video output environment suitable for the recognized user. That is, in the multimedia device, the user who is using the multimedia device is recognized, the user information of the recognized user is searched, the user's favorite genre is searched based on the searched user information, and based on the searched favorite genre, By setting the beam-forming setting and the audio / video output environment, the multimedia device can recognize the user and set an optimal viewing environment, And to provide a multimedia device capable of receiving a sound, and a multimedia device therefor.
Description
BACKGROUND OF THE
In particular, the present invention relates to an operation method and a multimedia device capable of recognizing a user in a multimedia device and providing an audio / video output environment suitable for the recognized user.
The multimedia device is, for example, a device having a function of receiving and processing a broadcast image that a user can view. The multimedia device displays, for example, a broadcast selected by a user among broadcast signals transmitted from the broadcast station on the display. Currently, broadcasting is changing from analog broadcasting to digital broadcasting around the world.
Digital broadcasting refers to broadcasting in which digital video and audio signals are transmitted. Compared to analog broadcasting, digital broadcasting is strong against external noise and has a small data loss, is advantageous for error correction, has a high resolution, and provides a clear screen. Also, unlike analog broadcasting, digital broadcasting is capable of bidirectional service.
In addition, according to the use of the digital broadcasting, today's multimedia devices are able to use more various contents and services than existing multimedia devices such as game contents, movie contents, music contents, and the like. In order to use the contents, The sound and video output of the camera can be adjusted in more detail.
However, according to the related art, as the elements adjustable in the multimedia device are diversified, it is inconvenient for the user to determine the degree of adjustment of the elements. When a plurality of users use one multimedia device, There has been a problem in that, in order to provide a suitable viewing environment, the setting of the elements must be reset each time the user of the multimedia device is changed.
SUMMARY OF THE INVENTION Accordingly, the present invention has been made in view of the above problems, and it is an object of the present invention to provide a multimedia device capable of recognizing a user currently using the multimedia device and performing an operation for providing a viewing environment suitable for the recognized user, And a multimedia device according to the method are required to be developed.
It is an object of the present invention to provide a multimedia device and an operation method thereof that can improve the usability of a user.
It is another object of the present invention to provide a multimedia device and an operation method thereof that can provide a suitable viewing environment according to a user.
Another object of the present invention is to provide a multimedia device and a method of operating the same that enable sound setting and image setting based on a suitable user when a plurality of users are using the multimedia device.
According to another aspect of the present invention, there is provided a method of providing an optimal viewing environment for a multimedia device, comprising: recognizing a user using the multimedia device; Retrieving user information of the recognized user; Searching for a user's favorite genre based on the searched user information; And setting acoustic beamforming (Beam-forming) based on the retrieved preferred genre.
According to another aspect of the present invention, there is provided a method of providing an optimal viewing environment for a multimedia device, comprising: recognizing a user using the multimedia device; Retrieving user information of the recognized user when a plurality of users are recognized; Searching for a favorite genre of each user based on the searched user information; And configuring acoustic beamforming based on a preference genre of a priority user among the users.
According to another aspect of the present invention, there is provided a multimedia device including: an image sensing unit for acquiring user image information; A recognition unit for recognizing a user based on the data acquired by the image sensing unit; A storage unit for storing user information for each user; An acoustic output unit adapted to adjust sound by applying beam-forming; And searching the user information of the user recognized through the recognition unit through the storage unit, searching for a user preference genre based on the searched user information, and applying sound beamforming of the sound output unit based on the searched favorite genre, And a control unit for controlling the control unit.
According to the embodiment of the present invention, even if the user does not directly adjust the viewing environment, since the multimedia device recognizes the user and sets an optimal viewing environment, the user can select the optimum screen and sound .
According to another embodiment of the present invention, even when a plurality of users simultaneously use one multimedia device, an operation for searching for a priority user and setting an optimal viewing environment is performed, thereby enhancing user convenience .
Meanwhile, according to another embodiment of the present invention, various user interfaces can be provided in the multimedia device, and the multimedia device can be operated conveniently.
1 is a diagram schematically illustrating an example of an overall system including a multimedia device according to an embodiment of the present invention.
FIG. 2 is a diagram showing an example of the multimedia device shown in FIG. 1 in more detail.
FIG. 3 is a view showing a multimedia device using a plurality of heterogeneous image sensors according to an embodiment of the present invention and a photographing screen at the same time.
4 is a diagram illustrating a process of using detection data and recognition data in a plurality of heterogeneous image sensors and multimedia devices according to an embodiment of the present invention.
5 is a diagram for explaining a face vector stored in the database shown in FIG.
6 is a view for explaining an operation of a plurality of heterogeneous image sensors connected to a multimedia device into a hardware area and a software area according to an embodiment of the present invention.
7 is a view showing a plurality of heterogeneous image sensors and a multimedia device according to an embodiment of the present invention, respectively.
8 is a view showing a plurality of heterogeneous image sensors and a multimedia device according to another embodiment of the present invention, respectively.
FIG. 9 is a diagram illustrating a plurality of heterogeneous image sensors according to an exemplary embodiment of the present invention in more detail.
10 is a view showing an example of a first image sensor among a plurality of heterogeneous image sensors according to an embodiment of the present invention.
11 is a view showing another example of the first image sensor among the plurality of heterogeneous image sensors according to the embodiment of the present invention.
12 is a diagram for explaining a method of calculating a distance using the first image sensor shown in FIG.
FIG. 13 is a view showing an example of the multimedia device shown in FIGS. 1 and 2 in more detail.
14 illustrates a user and multimedia device in accordance with an embodiment of the present invention.
15 is a diagram illustrating an image of a depth image sensor that recognizes coordinates of each part of a user's body according to an embodiment of the present invention.
16 is a flowchart illustrating a process of adjusting a viewing environment in a multimedia device according to an embodiment of the present invention.
17 is a flowchart illustrating a process of adjusting a viewing environment when there are a plurality of viewers of a multimedia device according to an exemplary embodiment of the present invention.
18 is a flowchart illustrating in detail the process of setting beamforming according to the preferential user of FIG.
19 is a view illustrating a display screen including a user recognition message according to an embodiment of the present invention.
20 is a view illustrating a display screen including a user setting application message according to an embodiment of the present invention.
FIG. 21 is a diagram illustrating a display screen including a confirmation menu for applying a user setting according to an exemplary embodiment of the present invention. Referring to FIG.
22 is a view illustrating a display screen including a user selection menu according to an embodiment of the present invention.
23 is a view illustrating a display screen including a personalized initial screen according to an embodiment of the present invention.
Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings. Furthermore, the suffix "module" and " part "for components used in the following description are given merely for ease of description, and the" module " It can be designed in hardware or software.
Meanwhile, the multimedia device described herein corresponds to various types of devices that receive and process broadcast data, for example. Further, the multimedia device may correspond to Connected TV. In addition to the broadcast receiving function, the Connected TV is further provided with a wired / wireless communication device and the like, and is more convenient to use than a hand-held input device, a touch screen or a motion- Lt; / RTI > Also, it can be connected to the Internet and a computer by the support of a wired or wireless Internet function, and can perform functions such as e-mail, web browsing, banking or game. A standardized general-purpose OS may be used for these various functions.
Accordingly, the Connected TV can perform various user-friendly functions because various applications can be freely added or deleted on a general-purpose OS kernel, for example. More specifically, the Connected TV may be, for example, a web TV, an Internet TV, an HBBTV, a smart TV, a DTV, or the like, and may be applied to a smartphone according to circumstances.
BRIEF DESCRIPTION OF THE DRAWINGS The above and other features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG.
As used herein, terms used in the present invention are selected from general terms that are widely used in the present invention while taking into account the functions of the present invention, but these may vary depending on the intention or custom of a person skilled in the art or the emergence of new technologies. Also, in certain cases, there may be a term chosen arbitrarily by the applicant, in which case the meaning thereof will be described in the description of the corresponding invention. Therefore, it is intended that the terminology used herein should be interpreted based on the meaning of the term rather than on the name of the term, and on the entire contents of the specification.
1 is a diagram schematically illustrating an example of an overall broadcasting system including a multimedia device according to an embodiment of the present invention. The multimedia device of FIG. 1 may correspond to, for example, Connected TV, but the scope of the present invention is not limited to Connected TV, and the scope of rights of the present invention should be determined by the claims in principle.
1, an overall system including a multimedia device according to an embodiment of the present invention includes a content provider (CP) 10, a service provider (SP) 20, a network provider A network provider (NP) 30 and an HNED 40. The HNED 40 corresponds to, for example, a
The
The
The
On the other hand, the
FIG. 2 is a diagram showing an example of the multimedia device shown in FIG. 1 in more detail.
The
The
The TCP /
The
The PSI & PSIP and / or
In addition, the PSI & PSIP and / or
The audio /
The
The
The
The
The SI &
Meanwhile, the
The plurality of
FIG. 3 is a view showing a multimedia device using a plurality of heterogeneous image sensors according to an embodiment of the present invention and a photographing screen at the same time. Hereinafter, referring to FIG. 3, a multimedia device and a photographing screen using a plurality of heterogeneous image sensors according to an embodiment of the present invention will be described.
In general, the first image sensors associated with depth data processing are not suitable for long-distance facial recognition due to limited resolution (e.g., maximum VGA grade) and recognition distances (e.g., 3.5 m) . In addition, the second image sensors related to color data processing have a disadvantage that they are slow in recognition and are not robust to light conditions. Therefore, in order to compensate for the disadvantages of each image sensor, the multimedia device according to an embodiment of the present invention is designed to be interlocked with a hybrid image sensor module in which a first image sensor and a second image sensor are combined.
As the above-mentioned first image sensor, for example, an IR camera or a depth camera is used. More specifically, for example, TOF (Time Of Flight) method and structured light method are discussed with the IR camera or depth camera. In the TOF method, distance information is calculated using a time difference that is radiated by infrared rays. The structured light method radiates infrared rays in a specific pattern and calculates a distance by analyzing the deformed pattern. However, in the case of the first image sensor, there is an advantage in terms of depth data recognition and processing speed, and it is possible to easily sense objects and people even in a dark place. However, it has a disadvantage that the resolution is low at a long distance.
Furthermore, as the second image sensor described above, for example, a color camera or an RGB camera is used. More specifically, for example, the color camera or the RGB camera, the stereo camera method and the mono camera method are discussed. The stereo camera method detects and tracks a hand or a face on the basis of respective image parallax comparison information photographed through two cameras. The mono camera method detects and tracks a hand or a face based on shape and color information shot through a single camera. However, in the case of the second image sensor, the resolution is improved as compared with the first image sensor. However, the second image sensor is vulnerable to ambient light and is difficult to recognize in the dark. Particularly, there is a problem that accurate depth recognition is difficult.
In order to solve such conventional problems, as shown in FIG. 3, a multimedia device according to an embodiment of the present invention is designed to include both a first image sensor and a second image sensor. However, the image sensors may be designed to be embedded in the multimedia device or may be designed as a separate hardware module. First, as shown in the area (b) of FIG. 3, the first image sensor captures an image including users located in the vicinity of the multimedia device. The specific photographed images are sequentially shown in (1), (2), (3), and (4) in FIG.
On the other hand, when the photographing and data analysis of the first image sensor is completed, the second image sensor photographs an image of a specific user's face, as shown in the area (a) of FIG. The specific photographed images are sequentially shown in (5), (6), and (7) in FIG.
The first image sensor among the plurality of heterogeneous image sensors according to an embodiment of the present invention captures a first image located in the periphery of the multimedia device and extracts depth data from the captured first image. As shown in (1) of FIG. 3, it is possible to design the area of each object to be displayed in different shades according to the distance.
Further, the first image sensor can recognize and recognize at least one face of the user using the extracted depth data. That is, by using a pre-stored database or the like, Extracts the user's body information (e.g., a face, a hand, a foot, a joint, etc.) as shown in FIG. 3B, and further calculates position coordinates and distance information . More specifically, it is designed to calculate x, y, and z values, which are positional information of the face of the user, where x is the position of the face on the abscissa in the captured first image, The position of the face with respect to the vertical axis in the first image, and z denotes the distance between the face of the user and the first image sensor.
In addition, a second image sensor for extracting a color image among a plurality of heterogeneous image sensors according to an embodiment of the present invention photographs a second image of the recognized face of the user, Respectively.
On the other hand, when the second image sensor and the first image sensor shown in FIG. 3 are designed to be adjacent to each other, the error due to the physical position difference may be negligible. However, according to another embodiment of the present invention, the coordinate information or the distance information acquired by the first image sensor is corrected using the information on the physical position difference, It is designed to capture users by using information or distance information. In addition, if the first image sensor and the second image sensor are designed to be horizontal from the ground, information on the phisycal position difference may be set on the basis of a horizontal frame. The feature information is extracted from the photographed second image as shown in (3) of FIG. The feature information is, for example, data corresponding to a specific region (e.g., mouth, nose, eye, etc.) for identifying a plurality of users using the multimedia device. Furthermore, the second image sensor may zoom in on the area of the user's face based on coordinate values (x, y, z values) obtained through photographing of the first image sensor. This means a process of switching from (5) to (6) in FIG.
When the photographing and analysis of the first image sensor and the second image sensor are completed, the multimedia device according to an embodiment of the present invention accesses a memory storing data corresponding to the extracted feature information, And extracts information identifying a particular user stored in the memory.
If information identifying the specific user exists in the memory, the multimedia device provides a predetermined service for the specific user.
On the other hand, if the information identifying the particular user is not present in the memory, the multimedia device is designed to display a guide message for storing the recognized user in the memory.
As described above, according to the embodiment of the present invention, the first image sensor detects the user position information or the coordinate information of the face, and the second image sensor uses the data obtained from the first image sensor, .
Furthermore, according to another embodiment of the present invention, the second image sensor is designed to operate only in a specific condition, not unconditionally using the second image sensor. For example, if the distance information to the user obtained by the operation of the first image sensor is less than or equal to the first reference value, or if the recognition rate of the user's face acquired through the operation of the first image sensor is equal to or greater than the second reference value, Detects and recoginizes a user face located in the vicinity of the multimedia device with only the first image sensor. On the other hand, if the distance information to the user obtained through the operation of the first image sensor exceeds the first reference value, or if the recognition rate of the user's face acquired by the operation of the first image sensor is less than the second reference value, And is further designed to recognize the user's face using the second image sensor.
According to another embodiment of the present invention, in the process of recognizing the user's face, the second image sensor zooms in using the distance information acquired through the first image sensor, And is designed to capture only the face portion using the face coordinate information acquired through the sensor.
Therefore, when a plurality of different image sensors of different types are used, it is possible to realize a long-distance facial recognition, and the data processing speed is further improved than before.
4 is a diagram illustrating a process of using detection data and recognition data in a plurality of heterogeneous image sensors and multimedia devices according to an embodiment of the present invention.
Face detection and face recognition are different processes. The face detection includes a process of detecting a face region in one image. On the other hand, the face recognition is a process of recognizing whether a detected face corresponds to a specific user. Particularly, the process of executing the face detection process using the first image sensor and executing the face recognition process using the second image sensor according to an embodiment of the present invention will be described with reference to FIG.
4, a multimedia device according to an exemplary embodiment of the present invention includes a
Further, the
The
5 is a diagram for explaining a face vector stored in the database shown in FIG.
As shown in FIG. 5, face vectors for each user using a multimedia device according to an exemplary embodiment of the present invention are stored. The face vector is, for example, a data set for feature information appearing on faces of users, and is used for identifying each specific user.
6 is a view for explaining an operation of a plurality of heterogeneous image sensors connected to a multimedia device into a hardware area and a software area according to an embodiment of the present invention.
6, a configuration in which a multimedia device receives an image through a plurality of heterogeneous image sensors and performs an operation includes a
In FIG. 6, the
First, the hardware area may include a
The
In addition, the
Accordingly, the
The
Also, the
The
7 is a view showing a plurality of heterogeneous image sensors and a multimedia device according to an embodiment of the present invention, respectively. Hereinafter, a plurality of heterogeneous image sensors and multimedia devices according to an embodiment of the present invention will be described with reference to FIG. In FIG. 7, a plurality of heterogeneous image sensors and a multimedia device according to an exemplary embodiment of the present invention are shown separately. However, the multiple cameras may be embedded in the multimedia device.
7, the
The
Then, the extracted feature information is transmitted to the face
The receiving unit of the face
Accordingly, the extracting unit of the face
On the other hand, when the face recognition processing module is designed to be performed by the CPU of the multimedia device as shown in FIG. 7, there is an advantageous effect in terms of expanding the features such as various facial recognition and various functions.
8 is a view showing a plurality of heterogeneous image sensors and a multimedia device according to another embodiment of the present invention, respectively. Hereinafter, a plurality of heterogeneous image sensors and multimedia devices according to another embodiment of the present invention will be described with reference to FIG. In FIG. 8, a plurality of heterogeneous image sensors and a multimedia device according to an exemplary embodiment of the present invention are shown separately. However, the multiple cameras may be embedded in the multimedia device.
8, the
8 is different from FIG. 7 in that the face recognition processing module 521 is mounted on the plurality of
Meanwhile, when the face recognition processing module is designed to be performed by a plurality of
FIG. 9 is a diagram illustrating a plurality of heterogeneous image sensors according to an exemplary embodiment of the present invention in more detail. Hereinafter, referring to FIG. 9, a plurality of heterogeneous image sensors according to an embodiment of the present invention will be described in detail.
9, a plurality of heterogeneous image sensors according to an exemplary embodiment of the present invention includes a first
According to an embodiment, the first image sensor may be a depth image sensor.
The depth image sensor is an image sensor in which a pixel value recognized in an image photographed through the depth image sensor is a distance from the depth image sensor.
The first
In order to acquire an image through the first
That is, the multimedia device can grasp the distance information of the subject through the depth image sensor. Particularly, when the subject is a person, skeleton information and coordinate information of each part of the body can be obtained, and information on the concrete operation of the body can be obtained by searching for each partial movement of the body.
Further, under the control of the
In addition, under the control of the
According to an embodiment, the
The
Also, the
The
In addition, when the first
The
The
The
The
Through the above-described configuration, the user can input a predetermined image and voice to the multimedia device, and can control the multimedia device through the input image or voice.
10 is a view showing an example of a first image sensor among a plurality of heterogeneous image sensors according to an embodiment of the present invention. Hereinafter, with reference to FIG. 10, an example of a first image sensor among a plurality of heterogeneous image sensors according to an embodiment of the present invention will be described. The
As shown in FIG. 10, the
11 is a view showing another example of the first image sensor among the plurality of heterogeneous image sensors according to the embodiment of the present invention. Hereinafter, another example of the first image sensor among the plurality of heterogeneous image sensors according to an embodiment of the present invention will be described with reference to FIG. The
As shown in FIG. 11, the light emitted by the
12 is a diagram for explaining a method of calculating a distance using the first image sensor shown in FIG. Hereinafter, a method of calculating the distance using the first image sensor shown in FIG. 11 will be described with reference to FIG.
As shown in the left graph of FIG. 12, the arrival time t value can be obtained through the time difference between the emitted light and the reflected light.
12, the distance between the
FIG. 13 is a view showing an example of the multimedia device shown in FIGS. 1 and 2 in more detail.
The
Referring to FIG. 13, a
The
The
The
The stream signal output from the
The external
The external
The A / V input / output unit includes a USB terminal, a CVBS (Composite Video Banking Sync) terminal, and a component (not shown) so that the video and audio signals of an external device can be input to the
(Analog) terminal, a DVI (Digital Visual Interface) terminal, an HDMI (High Definition Multimedia Interface) terminal, an RGB terminal, a D-SUB terminal, and the like.
The wireless communication unit can perform short-range wireless communication with other electronic devices. The
Also, the external
The
The
The
The
In addition, the
The
FIG. 13 shows an embodiment in which the
The
For example, the
The
The video signal processed by the
The audio signal processed by the
The
The
In addition, the
The
In addition, the
The
Accordingly, the
The
When the user is recognized through the
When there are a plurality of users recognized through the
That is, the user information of each of the recognized plurality of users is searched, and a user whose preference genre of each user matches the genre of the content currently being used by the multimedia device is set as a priority user through the searched user information , A user who first watched the content among the plurality of users may be set as a priority user.
In addition, the
According to an embodiment of the present invention, the
The
In addition, the
The
Also, the
14 illustrates a user and multimedia device in accordance with an embodiment of the present invention.
According to the embodiment, the
In addition, the image sensing unit may include two
As shown in the figure, the
Through the above-described configuration, the multimedia device acquires information on the user and the user location, and can provide a viewing environment suitable for the user based on the obtained information.
15 is a diagram illustrating an image of a depth image sensor that recognizes coordinates of each part of a user's body according to an embodiment of the present invention.
Referring to FIG. 15, the multimedia device can acquire coordinate information for each part of the user's body through the image photographed through the depth image sensor.
That is, when the
For example, coordinate information for only the
When the face of the user is to be recognized, the coordinates of the head part of the obtained distance information is transmitted to the RGB image sensor of the image sensing part, so that the RGB image sensor detects the head part of the user according to the coordinates of the head part To acquire a zooming image for the object.
In addition, when a user moves a body and performs a specific action, the movement of the distance information of each part of the body can be sensed to recognize the action.
16 is a flowchart illustrating a process of adjusting a viewing environment in a multimedia device according to an embodiment of the present invention.
According to an embodiment, the multimedia device recognizes a user currently using the multimedia device (S2001).
In the case where a signal for adjusting the viewing environment is input from the user or the multimedia device automatically executes the viewing environment adjustment by executing the predetermined content in the multimedia device, Recognize the user using the device.
In addition, the multimedia device may include an image sensing unit for recognizing the user. For smooth user recognition, as described in FIG. 9, the image sensing unit acquires distance information of an object from the image sensor as an image A depth image sensor, and an RGB image sensor that acquires color information as an image.
That is, the user currently using the multimedia device is identified through information on the distance between the user and the multimedia device, user skeleton information on the distance information of each part of the user's body, and user color information.
The multimedia device transmits the information obtained through the image sensing unit of the multimedia device to the user recognition unit of the multimedia device for the user identification and transmits the information obtained through the image sensing unit of the multimedia device and the user recognition It is possible to retrieve the corresponding user identification information in comparison with the information.
Also, in order to smoothly perform the user recognition process, a recognition message as shown in FIG. 11 may be displayed.
Next, the multimedia device retrieves information on the recognized user (S2002).
The multimedia device retrieves previously stored user information corresponding to the recognized user when the user using the current multimedia device is recognized.
The user information may be retrieved through a history of contents and services used by the user in the multimedia device, and may include at least one of a user's multimedia device usage time, usage contents and services, sound settings and image quality settings Information.
Next, the multimedia device analyzes user preference genres in the retrieved user information (S2003).
That is, the multimedia apparatus searches through the information included in the user information to find out what content or service the user has used, the time using the content or the service, and, through the search result, You can analyze your favorite genre.
Next, the multimedia apparatus sets beamforming in the audio output of the multimedia apparatus based on the analyzed user preference genre (S2004).
Generally, the beam-forming technique of the acoustic output is used for the purpose of causing the sound output of the multimedia device to exhibit high directivity in a desired direction from the multimedia device. Once the directivity is formed through beamforming, the user can reduce the acoustic signals transmitted from the directions outside of the beam, and selectively acquire acoustic signals from the directions of interest, thereby configuring a more suitable viewing environment. That is, the user can listen to the desired sound signals more smoothly than the noise.
Therefore, the multimedia device can configure beamforming through the user preference genre and the user distance information, and thus the user can be provided with an appropriate viewing environment.
In addition, the multimedia device may perform sound setting and image setting according to the user information of the recognized user.
In other words, the multimedia apparatus may further include an equalizer, a bass enhancement, an ear guard, a music clarity, a noise reduction, and a volume setting of the multimedia device based on the user information Dynamic Booster) and 3D sound (3D sound).
The multimedia device may further include at least one of a menu screen configuration, a brightness, a contrast, a color, a focus, and an exposure of the multimedia device based on the user information And setting of the user preference channel.
In this case, the multimedia device recognizes the user through the image input through the image sensing unit of the multimedia device and sets the viewing environment so as to correspond to the recognized user, without the user having to perform a separate setting process. The multimedia device can be used in a convenient and better environment.
FIG. 17 is a flowchart illustrating a process of adjusting a viewing environment when there are a plurality of viewers of a multimedia device according to an exemplary embodiment of the present invention.
According to an embodiment, the multimedia device recognizes a user currently using the multimedia device (S2101).
The user recognition step S2101 recognizes the user through the image input through the image sensing unit of the multimedia device, similar to the user recognition step S2001 of Fig.
However, if the number of users recognized by the image input through the image sensing unit of the multimedia device is several, the multimedia device recognizes each of the plurality of users.
Next, the multimedia device retrieves the information about each recognized user (S2102).
When a plurality of users using the current multimedia device are recognized, the multimedia device searches pre-stored user information corresponding to each recognized user.
The user information may be retrieved through a history of contents and services used by the user in the multimedia device, and may include at least one of a user's multimedia device usage time, usage contents and services, sound settings and image quality settings Information.
That is, an identification (ID) is assigned to each user, and user information of the corresponding user is retrieved for each user ID.
Next, the multimedia device analyzes user preference genres of each user in the searched user information (S2103).
That is, the multimedia device searches through the information included in the respective pieces of user information to find out what contents or services are used by each user, how much the contents or service use time of each user is, It is possible to search through which genre contents or services each user prefers and to understand the preference genre of each user according to the search result.
Next, the multimedia device sets beamforming in the audio output of the multimedia device based on the analyzed user preference genre based on the priority user among the users (S2104).
That is, since the directivity of the beamforming set according to the position of the user differs, the directivity of the beamforming is set based on the priority user among the plurality of users. A process of setting a priority user among the plurality of users will be described in detail with reference to FIG.
Accordingly, even when there are two or more users currently using the multimedia device, the multimedia device can configure the beamforming according to the priority user through the user preference genre and the user distance information, A suitable viewing environment can be provided.
In addition, it is the same as the viewing environment setting in step S2004 of FIG. 8 that the multimedia device can perform sound setting and image setting according to the user information of the recognized priority user.
FIG. 18 is a flowchart showing in detail the step of setting beamforming (S2104) according to the preferential user of FIG.
According to an embodiment, when there are a plurality of users recognized in the input image through the image sensing unit of the multimedia device, a priority user for setting the viewing environment for the plurality of users can be set.
Accordingly, when the preferred genre of each user is analyzed through the preferred genre analysis step (S2103) of each user, the genre of the currently used content is searched (S2201).
That is, the multimedia device retrieves the genre of the currently-used content through the information included in the additional information of the content currently being used in the multimedia device. The additional information may be information included in a broadcast signal when the content being used is a terrestrial broadcast, and may be information included in the file when the content is a video on demand (VoD) have.
Next, the genre of the content currently used in the searched multimedia device is compared with the preference genre of each user searched in step S2103, and it is searched whether there is a user whose genre matches the genre (S2202).
As a result of the search (S2202), if there is a user having the same genre, the user having the genre matching is set as a priority user among a plurality of users currently using the multimedia device (S2204).
That is, when the genre of the content currently being used in the multimedia device matches the preference genre of the user, the matching user is directly using the content, so that the matching user is set as the priority user, The viewing environment can be set based on the user who uses the content.
According to the embodiment, when there are a plurality of users whose genres match each other, the user selection menu shown in FIG. 22 may be displayed to select a priority user from the user.
On the other hand, if there is no matching user in the genre in step S2202, the multimedia device searches for the first viewer among the recognized plurality of users (S2203).
That is, the multimedia apparatus searches for information on the usage start time among the user information found in step S2101, and determines the first viewer.
Also, the first viewer is set as a priority user (S2205).
That is, if there is no user with the same genre, the user who is currently viewing the multimedia device at the earliest among the users currently using the multimedia device is directly using the content currently being used in the multimedia device, Can be set as a priority user so that the viewing environment can be set based on the user directly using the content.
Next, acoustic beamforming, image setting, and sound setting are performed based on the priority user (S2206).
That is, when the priority user is set, an operation for adjusting the viewing environment such as an acoustic beamforming setting, a video output setting, and a sound output setting according to the user information of the priority user is performed as in the setting step S2104 of FIG. .
Accordingly, even when there are a plurality of users using the multimedia device, the multimedia device can appropriately determine the priority user and perform a proper viewing environment setting.
19 is a view illustrating a
According to the embodiment, when the user inputs a predetermined command for adjusting the viewing environment or automatically adjusts the viewing environment in the multimedia device, the multimedia device performs the user recognition for the adjustment of the viewing environment, A
The multimedia device recognizes the user in the image input through the image sensing unit of the multimedia device, for the purpose of adjusting the viewing environment. Therefore, when the user moves during the input of the image and does not obtain an appropriate image, or when the user executes another operation in the multimedia device while recognizing the user in the input image, the user recognition may not proceed smoothly , The
In addition, if the user desires to cancel the user recognition, the user can cancel the selection by selecting the cancel
Also, when a selection signal of the cancel
20 is a view illustrating a
According to an embodiment of the present invention, when a user is recognized in an image input through the image sensing unit of the multimedia device, a
When the
Accordingly, if it is not desired to adjust the viewing environment, the user can select the cancel
Also, the
In addition, the information about the viewing environment according to the user information of the recognized user may be included.
In addition, the
Accordingly, the user can perform the adjustment of the viewing environment through the background message through the
FIG. 21 is a view showing a
According to an embodiment of the present invention, when the multimedia device recognizes a user in the input image through the image sensing unit of the multimedia device, the multimedia device inputs a confirmation command as to whether or not to set the viewing environment of the multimedia device according to the user information of the recognized user An
The
When the
Also, the
Accordingly, when the user is mistakenly recognized through the image sensing unit of the multimedia device, the user can prevent the viewing environment from being set according to the mistaken user, thereby improving reliability.
22 is a view illustrating a
According to the embodiment, when there are a plurality of users recognized by the image input through the image sensing unit of the multimedia device, among the plurality of recognized users, a selection menu for selecting a priority user in the viewing environment setting The
That is, according to the embodiment, the
The
That is, when a plurality of users are recognized in the input image through the image sensing unit of the multimedia device, the multimedia device extracts the cropped image of the portion for recognizing the user in the image, It is possible to display it in the
Also, it is possible to display information including the recognized user along with the cropped image.
The user can select a priority user via the
The multimedia device loads viewing preference information for the selected user when the selection signal is received, and provides an appropriate viewing environment to the user according to the loaded information.
23 is a view showing a
According to an embodiment, when a user of the multimedia device is recognized, the multimedia device may provide a personalized initial screen for each recognized user.
That is, for example, as shown in FIG. 23, a
The
The
The
The user may also move the
That is, the multimedia device provides the personalized menu to the user currently using the multimedia device according to the user information recognized to configure the viewing environment according to the embodiment of the present invention. The user can conveniently use the multimedia device have.
The multimedia device and its operation method according to the present invention are not limited to the configuration and method of the embodiments described above but the embodiments may be modified so that all or part of each embodiment Or may be selectively combined.
Meanwhile, the method of operating the multimedia device of the present invention can be implemented as a code that can be read by a processor on a recording medium readable by a processor included in the multimedia device. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.
While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the present invention.
100: Multimedia device
105: Broadcast receiver
110: tuner
120: demodulator
130: Network interface unit
135: External device interface unit
140:
150: user input interface unit
170:
180:
185: Audio output section
190: Power supply
200: remote control device
Claims (20)
Recognizing a user using the multimedia device;
Retrieving user information of the recognized user when a plurality of users are recognized;
Searching for a favorite genre of each user based on the searched user information;
Retrieving a genre of content currently in use in the multimedia device;
Setting a user having a matching genre of the currently used content and the searched favorite genre as a priority user; And
Constructing an acoustic beamforming (Beam-forming) based on the preference user's preferred genre
And providing the optimal viewing environment of the multimedia device.
Recognizing a user using the multimedia device;
Retrieving user information of the recognized user when a plurality of users are recognized;
Searching for a favorite genre of each user based on the searched user information;
Retrieving the start time of using the multimedia device of each user;
Setting a user whose usage start time is earliest as a priority user; And
Constructing an acoustic beamforming (Beam-forming) based on the preference user's preferred genre
And providing the optimal viewing environment of the multimedia device.
Retrieving whether the recognized user is changed;
If the user change is detected, returning to the step of recognizing the user
The method comprising the steps of:
A recognition unit for recognizing a user based on the data acquired by the image sensing unit;
A storage unit for storing user information for each user;
A priority setting unit configured to set a priority user among the plurality of users when a plurality of users are recognized through the recognition unit;
An acoustic output unit adapted to adjust sound by applying beam-forming; And
Searching the user information of the user recognized through the recognition unit through the storage unit, searching for a user preference genre based on the searched user information, and controlling sound adjustment applying beamforming of the sound output unit based on the searched favorite genre And a controller,
Wherein the priority setting unit compares a genre of a content currently used in the multimedia device with a preference genre of each recognized user to set a user having the genre as a priority user,
Wherein the control unit controls the beamforming configuration of the sound output unit based on the preference genre of the set priority user,
Multimedia device that can adjust viewing environment
Wherein,
Based on the user information, controlling the acoustic output environment setting of the sound output section
Multimedia device that can adjust viewing environment
Wherein,
And controlling the video output environment setting of the multimedia device based on the user information
Multimedia device that can adjust viewing environment
The image sensing unit includes:
A depth image sensor capable of obtaining the user distance information; And
An RGB image sensor capable of obtaining color information of the user
A multimedia device capable of adjusting the viewing environment
The user information includes:
And at least one of the multimedia device usage time, the usage content, the sound setting, and the image quality setting of each user
Multimedia device that can adjust viewing environment.
The multimedia device includes:
A sensing unit for sensing whether the user is changed,
Further comprising:
Wherein,
A user recognition unit for recognizing the changed user through the image sensing unit and the recognition unit when the user change is detected by the sensing unit, searching through the storage unit for the user information of the re-recognized user, Searching for a genre, and controlling an acoustic re-arrangement to which beamforming of the sound output section is applied based on the searched favorite genre
Multimedia device that can adjust viewing environment.
A recognition unit for recognizing a user based on the data acquired by the image sensing unit;
A storage unit for storing user information for each user;
A priority setting unit configured to set a priority user among the plurality of users when a plurality of users are recognized through the recognition unit;
An acoustic output unit adapted to adjust sound by applying beam-forming; And
Searching the user information of the user recognized through the recognition unit through the storage unit, searching for a user preference genre based on the searched user information, and controlling sound adjustment applying beamforming of the sound output unit based on the searched favorite genre And a controller
Wherein the priority setting unit searches the multimedia device start time of each user to set the user whose start time of use is the earliest as the priority user,
Wherein the control unit controls the beamforming configuration of the sound output unit based on the preference genre of the set priority user,
Multimedia device that can adjust viewing environment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020100112530A KR101741550B1 (en) | 2010-11-12 | 2010-11-12 | Method and apparatus for providing optimized viewing conditions in multimedia device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020100112530A KR101741550B1 (en) | 2010-11-12 | 2010-11-12 | Method and apparatus for providing optimized viewing conditions in multimedia device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| KR20120051210A KR20120051210A (en) | 2012-05-22 |
| KR101741550B1 true KR101741550B1 (en) | 2017-06-15 |
Family
ID=46268329
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020100112530A KR101741550B1 (en) | 2010-11-12 | 2010-11-12 | Method and apparatus for providing optimized viewing conditions in multimedia device |
Country Status (1)
| Country | Link |
|---|---|
| KR (1) | KR101741550B1 (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3422306A1 (en) | 2004-11-17 | 2019-01-02 | Arthur J. Zito, Jr. | User-specific dispensing system |
| US7398921B2 (en) | 2004-11-17 | 2008-07-15 | Zito Jr Arthur J | User-specific dispensing system |
| US11553228B2 (en) | 2013-03-06 | 2023-01-10 | Arthur J. Zito, Jr. | Multi-media presentation system |
| KR102170254B1 (en) | 2013-12-10 | 2020-10-26 | 삼성전자주식회사 | Adaptive beam selection apparatus and method in a wireless communication system |
| KR20150108028A (en) | 2014-03-16 | 2015-09-24 | 삼성전자주식회사 | Control method for playing contents and contents playing apparatus for performing the same |
| KR102342081B1 (en) | 2015-04-22 | 2021-12-23 | 삼성디스플레이 주식회사 | Multimedia device and method for driving the same |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006324809A (en) * | 2005-05-17 | 2006-11-30 | Sony Corp | Information processor, information processing method, and computer program |
| JP2007282191A (en) * | 2006-03-14 | 2007-10-25 | Seiko Epson Corp | Guide apparatus and method of controlling the same |
-
2010
- 2010-11-12 KR KR1020100112530A patent/KR101741550B1/en active IP Right Grant
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006324809A (en) * | 2005-05-17 | 2006-11-30 | Sony Corp | Information processor, information processing method, and computer program |
| JP2007282191A (en) * | 2006-03-14 | 2007-10-25 | Seiko Epson Corp | Guide apparatus and method of controlling the same |
Also Published As
| Publication number | Publication date |
|---|---|
| KR20120051210A (en) | 2012-05-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101731346B1 (en) | Method for providing display image in multimedia device and thereof | |
| EP2453384B1 (en) | Method and apparatus for performing gesture recognition using object in multimedia device | |
| KR20120051212A (en) | Method for user gesture recognition in multimedia device and multimedia device thereof | |
| US8577092B2 (en) | Multimedia device, multiple image sensors having different types and method for controlling the same | |
| US9250707B2 (en) | Image display apparatus and method for operating the same | |
| US9025023B2 (en) | Method for processing image data in television having multiple image sensors and the television for controlling the same | |
| US9390714B2 (en) | Control method using voice and gesture in multimedia device and multimedia device thereof | |
| KR101758271B1 (en) | Method for recognizing user gesture in multimedia device and multimedia device thereof | |
| US8830302B2 (en) | Gesture-based user interface method and apparatus | |
| KR101899597B1 (en) | Method for searching object information and dispaly apparatus thereof | |
| KR101741550B1 (en) | Method and apparatus for providing optimized viewing conditions in multimedia device | |
| US20120268424A1 (en) | Method and apparatus for recognizing gesture of image display device | |
| US20130263048A1 (en) | Display control apparatus, program and display control method | |
| KR101772456B1 (en) | Multimedia device, multiple image sensors having different types and the method for controlling the same | |
| KR101819499B1 (en) | Multimedia device, multiple image sensors having different types and the method for controlling the same | |
| KR20120051213A (en) | Method for image photographing of multimedia device and multimedia device thereof | |
| KR20210002797A (en) | Display device | |
| KR20120050614A (en) | Multimedia device, multiple image sensors having different types and the method for controlling the same | |
| KR101629324B1 (en) | Multimedia device, multiple image sensors having different types and the method for controlling the same | |
| KR20150012677A (en) | multimedia apparatus and method for predicting user command using the same | |
| KR20120074484A (en) | Multimedia device for processing data by using image sensor and the method for controlling the same | |
| KR101759936B1 (en) | Multimedia device using multiple image sensors having different types and method for controlling the same | |
| KR20150059402A (en) | multimedia apparatus and method for displaying pointer thereof | |
| KR20120102337A (en) | Electronic device and method for managing control right | |
| KR20110094727A (en) | Apparatus and method for detecting motion data of infrared ray remocon |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A201 | Request for examination | ||
| E902 | Notification of reason for refusal | ||
| E701 | Decision to grant or registration of patent right | ||
| GRNT | Written decision to grant |