KR102419299B1 - Electronic device - Google Patents

Abstract

According to some embodiments of the present disclosure, an electronic device is disclosed. The electronic device may include an under-display camera unit positioned below the first area of the display unit; and a controller for recognizing a first object photographed through the under-display camera unit and controlling the display unit to display the first object on the first area of the display unit.

Description

Electronic device {ELECTRONIC DEVICE}

The present disclosure relates to an electronic device, and more particularly, to an electronic device having a three-dimensional sensing function capable of object recognition.

Recently, interest in and utilization of object recognition technology such as humans and other objects is increasing, and the general public can easily access various applications using object recognition technology through smart phones. Accordingly, the necessity to accurately identify the shape, position, or movement of an object by means of precise three-dimensional shape recognition is gradually increasing. As one of the methods for this, a three-dimensional sensing technology is being tried, and thereby, precise motion recognition is enabled.

On the other hand, according to the e-Nara indicator, there are approximately 377,000 people with hearing impairments in Korea counted in 2020, the previous year. A deaf person is literally a person who is deaf, and can refer to a person who is deaf and a person who cannot hear or speak. Although hearing impairment is an individual problem, it is not simply an individual problem, but can also cause social problems such as controversy over discrimination.

Accordingly, national institutions are guiding the deaf people to provide subtitles or sign language when providing media so that the hearing impaired can live their daily lives without feeling alienated. However, in the case of subtitles, it may be difficult for hearing impaired people to understand easily, contrary to what ordinary people think. This is a problem that occurs because there is insufficient research in sign language and the sign language and text do not match. Therefore, in an emergency briefing situation such as this Corona-19, a sign language interpreter who can use sign language appeared at the bottom of the screen without exception.

On the other hand, in the above case, a sign language interpreter may be prepared to help communication between the general public and the hearing impaired, but in general, hiring a sign language interpreter for 1:1 conversation between the general public and the hearing impaired is a costly problem. It may not be easy because of In addition, in a service such as a video call, it may be difficult to participate in a conversation even if a sign language interpreter is available.

Therefore, there is a demand for a three-dimensional sensing technology capable of smoothly communicating with the general public and the hearing impaired through a user terminal such as a smart phone or a laptop without a sign language interpreter.

Japanese registered patent JP 6685887

The present disclosure has been made in response to the above-described background technology, and an object of the present disclosure is to provide an electronic device having a three-dimensional sensing function capable of recognizing an object related to a user.

The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to an embodiment of the present disclosure for solving the above-described problems, an electronic device is disclosed. The electronic device may include an under-display camera unit positioned below the first area of the display unit; and a control unit for recognizing a first object photographed through the under-display camera unit and controlling the display unit to display the first object on the first area of the display unit. may include

Also, the first area may be a central area of the display unit.

In addition, when the first object is displayed in a second area different from the first area, the controller is configured to move the under-display camera unit to the second area by moving the under-display camera unit so that the first object is displayed in the first area. can be moved to

In addition, when the first object is a user's hand and the first object is displayed on the second area, the control unit may include the moving unit of the under-display camera unit such that the user's hand is displayed on the first area. can be moved to the second area by

In addition, when the first object is a user's face, the control unit recognizes a third area to which the user's gaze is directed on the display unit, and moves the under-display camera unit to the third area by a moving means. can

In addition, the sign language-to-speech output unit for recognizing a first sign language sensed from the first object and outputting a first voice corresponding to the first sign language; and a voice-signature output unit recognizing a second voice input through a microphone and outputting a second sign language corresponding to the second voice; may further include.

The sign language-voice output unit may include: a sign language recognition unit for recognizing the first sign language sensed from the first object; a first index generator for generating at least one sign language index based on the recognized first sign language; a sign language-word matching unit matching at least one word corresponding to each of the at least one sign language index; a sentence generator for generating at least one sentence using the at least one word; and a voice output unit configured to output the first voice corresponding to the at least one sentence. may include

In addition, the voice-sign output unit, the voice conversion unit for converting the second voice input through the microphone into text; a second index generator for generating at least one sign language index based on the character; a text-image matching unit matching at least one image corresponding to each of the at least one sign language index; a sign language generator for generating the second sign language by using the at least one image; and a sign language output unit for outputting the second sign language; may include

In addition, the under display camera unit may include: a first substrate; a second substrate positioned on the first substrate; a light transmitter including a light source disposed on the first substrate and a light generating module disposed on the second substrate to be coaxially aligned with the light source; and a light receiving unit including a light guide module disposed on the second substrate and a light sensing module disposed on the first substrate to be coaxially aligned with the light guide module; including, wherein the light generating module generates a light pattern irradiated to the outside of the under-display camera unit by the light of the light source, and the light guide module has a light transmitting layer having a predetermined pattern and the light pattern by passing the light reflected from the outside of the under display camera unit to the light sensing module through the light transmitting layer, and the light sensing module senses the light passing through the light guide module to generate a first optical image And, the controller may generate a second optical image by reconstructing the first optical image based on the predetermined pattern.

In addition, the under-display camera unit includes a time-of-flight (TOF) sensor module, a charge-coupled device (CCD) module, a complementary metal-oxide-semiconductor (CMOS) module, or a single photon avalanche diode (SPAD, Single Photon) module. Avalanche Diode) module can be included.

The technical solutions obtainable in the present disclosure are not limited to the above-mentioned solutions, and other solutions not mentioned are clearly to those of ordinary skill in the art to which the present disclosure belongs from the description below. can be understood

According to some embodiments of the present disclosure, an object is to provide an electronic device capable of accurately identifying the shape, position, or movement of an object.

Effects obtainable in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. .

Various aspects are now described with reference to the drawings, wherein like reference numbers are used to refer to like elements throughout. In the following example, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It will be apparent, however, that such aspect(s) may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing one or more aspects.
1 is a block diagram illustrating an example of an electronic device according to some embodiments of the present disclosure;
2 is a view for explaining an example of an under display camera according to some embodiments of the present disclosure.
3 is a view for explaining an example of an electronic device according to some embodiments of the present disclosure.
4 is a cross-sectional view taken along line AA′ of FIG. 3 for explaining an example of a moving means according to some embodiments of the present disclosure.
5 is a flowchart illustrating an example of a method in which a controller of an electronic device moves an under-display camera unit according to some embodiments of the present disclosure.
6 is a view for explaining an example of a first object displayed on a display unit according to some embodiments of the present disclosure;
7 is a flowchart illustrating an example of a method in which a controller of an electronic device moves an under-display camera unit according to another exemplary embodiment of the present disclosure.
8 is a view for explaining an example of a first object displayed on a display unit according to some other exemplary embodiments of the present disclosure.
9 shows a general schematic diagram of an example computing environment in which embodiments of the present disclosure may be implemented.

Various embodiments and/or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of one or more aspects. However, it will also be appreciated by one of ordinary skill in the art that such aspect(s) may be practiced without these specific details. The following description and accompanying drawings set forth in detail certain illustrative aspects of one or more aspects. These aspects are illustrative, however, and some of various methods may be employed in the principles of the various aspects, and the descriptions set forth are intended to include all such aspects and their equivalents. Specifically, as used herein, “embodiment”, “example”, “aspect”, “exemplary”, etc. are not to be construed as advantageous or advantageous over any aspect or design described herein. It may not be.

Hereinafter, the same or similar components are assigned the same reference numerals regardless of reference numerals, and overlapping descriptions thereof will be omitted. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical ideas disclosed in the present specification are not limited by the accompanying drawings.

Although the first, second, etc. are used to describe various elements or elements, these elements or elements are not limited by these terms, of course. These terms are only used to distinguish one element or component from another. Accordingly, it goes without saying that the first element or component mentioned below may be the second element or component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly defined in particular.

In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from context, "X employs A or B" is intended to mean one of the natural implicit substitutions. That is, X employs A; X employs B; or when X employs both A and B, "X employs A or B" may apply to either of these cases. It should also be understood that the term “and/or” as used herein refers to and includes all possible combinations of one or more of the listed related items.

Also, the terms "comprises" and/or "comprising" mean that the feature and/or element is present, but excludes the presence or addition of one or more other features, elements, and/or groups thereof. should be understood as not Also, unless otherwise specified or unless it is clear from context to refer to a singular form, the singular in the specification and claims should generally be construed to mean “one or more”.

In addition, as used herein, the terms “information” and “data” can often be used interchangeably.

When a component is referred to as being “connected” or “connected” to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

References to an element or layer “on” or “on” another component or layer mean that another layer or other component in between as well as directly above the other component or layer. Including all intervening cases. On the other hand, when a component is referred to as “directly on” or “immediately on”, it indicates that another component or layer is not interposed therebetween.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe a component or a correlation with other components. The spatially relative terms should be understood as terms including different orientations of the device during use or operation in addition to the orientation shown in the drawings.

For example, when a component shown in the drawing is turned over, a component described as “beneath” or “beneath” of another component may be placed “above” of the other component. can Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

The suffixes "module" and "part" for the components used in the following description are given or used in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves.

Objects and effects of the present disclosure, and technical configurations for achieving them will become clear with reference to the embodiments described below in detail in conjunction with the accompanying drawings. In describing the present disclosure, if it is determined that a detailed description of a well-known function or configuration may unnecessarily obscure the subject matter of the present disclosure, the detailed description thereof will be omitted. In addition, the terms described below are terms defined in consideration of functions in the present disclosure, which may vary according to intentions or customs of users and operators.

However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. Only the present embodiments are provided so that the present disclosure is complete, and to fully inform those of ordinary skill in the art to which the present disclosure belongs, the scope of the disclosure, and the present disclosure is only defined by the scope of the claims . Therefore, the definition should be made based on the content throughout this specification.

In performing an online conversation such as a video call or a video conference using a smart phone or a laptop computer, the user may mainly look at the front of the screen. However, in the case of a conventional electronic device, in general, the camera may be located at an upper end of each electronic device. In this case, an angle is formed between the direction in which the user looks at the electronic device and the direction in which the camera captures the user, so that the user and the other party in the video call can feel that the user's gaze is downward rather than the front. Some users may perform a video call while looking at the camera instead of looking at the front of the screen of the electronic device. However, in this case, there is a problem in that it is difficult for the user to properly see the counterpart being displayed on his/her electronic device.

On the other hand, the electronic device according to the present disclosure may include an under display camera positioned below the display unit. In this case, as the direction in which the user is photographed coincides with the direction in which the user's gaze is directed, the user's gaze processing may be performed naturally. Hereinafter, an electronic device according to the present disclosure will be described with reference to FIGS. 1 to 9 .

1 is a block diagram illustrating an example of an electronic device according to some embodiments of the present disclosure;

Referring to FIG. 1 , the electronic device 1000 may include an under-display camera unit 100 , a control unit 200 , a sign language-voice output unit 300 , and a voice-sign language output unit 400 . However, since the above-described components are not essential for implementing the electronic device 1000 , the electronic device 1000 may have more or fewer components than those listed above.

The electronic device 1000 may include a personal computer (PC), a notebook (note book), a mobile terminal, a smart phone, a tablet PC, etc. owned by the user, It may include all types of terminals capable of accessing a wired/wireless network. However, the present invention is not limited thereto.

In the present disclosure, the electronic device 1000 may be a user terminal for performing an online conversation such as a video call or a video conference. However, the present invention is not limited thereto.

Meanwhile, the under-display camera unit 100 may be located below the first area of the display unit (not shown). Here, the first area may include various areas of the display unit. For example, the first area may be an area in which an object is displayed on the display. For example, the first area may be a central area of the display unit. In an online conversation, when the object viewed by the user on the display unit is located in the first area, the user's gaze may naturally point toward the under-display camera unit 100 .

Meanwhile, in the present disclosure, the under-display camera unit 100 may be a 2D or 3D sensing module (eg, ToF sensor module) used in a smart phone, a tablet, a smart watch, smart glasses, a car, a robot, or a drone. In addition, the under-display camera unit 100 includes a Charge-Coupled Device (CCD) module, a Complementary Metal-Oxide-Semiconductor (CMOS) module, or a Single Photon Avalanche Diode (SPAD) module. can do. In this case, the under-display camera unit 100 may perform some operations using some components (eg, control units) of the electronic device 1000 . Hereinafter, a description of the under-display camera unit 100 according to the present disclosure will be described with reference to FIG. 2 .

Meanwhile, the control unit 200 may typically process the overall operation of the electronic device 1000 . The control unit 200 may provide or process information or functions appropriate to the user by processing signals, data, information, etc. input or output through the components or by driving an application program stored in a storage unit (not shown).

Meanwhile, in the present disclosure, the controller 200 may recognize the first object photographed through the under-display camera unit 100 . Here, the first object may include various objects. For example, the first object may be a user's face or hand. In this case, the controller 200 may control the display unit to display the first object on the first area of the display unit. Here, the first area may be a central area of the display unit. However, the present invention is not limited thereto.

Meanwhile, according to some embodiments of the present disclosure, the controller 200 may recognize whether the first object is displayed in a second area different from the first area. In addition, when recognizing that the first object is displayed in the second area, the controller 200 may move the under-display camera unit 100 to the second area by a moving means so that the first object is displayed in the first area. have.

Meanwhile, according to some embodiments of the present disclosure, the controller 200 may recognize that the first object is the user's hand. In this case, the controller 200 may recognize the second area where the user's hand is displayed on the display unit. Here, the second area may be a different area from the first area. For example, the second area may be an area that deviates from the center area of the display unit, which is the first area. In addition, the controller 200 may move the under-display camera unit 100 to the second area by a moving means. In this case, the user's hand may be positioned to correspond to the position of the under-display camera unit 100 . Accordingly, it is possible to reduce the case where the hand is not positioned within the shooting angle by moving the position of the hand due to a complicated hand movement such as sign language, so that the under-display camera unit 100 can accurately capture the movement of the hand. However, the present invention is not limited thereto. Hereinafter, an example of a method for the controller 200 according to the present disclosure to move the under-display camera unit 100 to the second area will be described with reference to FIGS. 5 and 6 .

Meanwhile, according to some embodiments of the present disclosure, the controller 200 may recognize that the first object is the user's face. In this case, the control unit 200 may recognize the third area to which the user's gaze is directed on the display unit. Here, the third area may be a different area from the first area. For example, the third area may be an area that deviates from the center area of the display unit, which is the first area. In addition, the controller 200 may move the under-display camera unit 100 to the third area by a moving means. In this case, the user's gaze is directed toward the under-display camera unit 100 , and thus, it is possible to prevent the user's face displayed on the electronic device of the user or other users from feeling unnatural.

Meanwhile, according to some embodiments of the present disclosure, the electronic device 1000 may include a sign language-voice output unit 300 and a voice-sign language output unit 400 . In this case, the electronic device 1000 may recognize and output sign language from the first object that is the user's hand recognized by the controller 200 .

Specifically, the sign language-voice output unit 300 may recognize a first sign language sensed from the first object and output a first voice corresponding to the first sign language. Here, the first sign language may be a sign language recognized from an object that is a hand of a user who speaks the sign language. And, the first voice may be a voice output so that the other party can understand the first sign language. However, the present invention is not limited thereto. Hereinafter, the sign language-voice output unit 300 according to the present disclosure will be described with reference to FIG. 3 .

Meanwhile, the voice-sign output unit 400 may recognize the second voice input through the microphone and output the second sign language corresponding to the second voice. Here, the second sign language may be a sign language generated based on the second voice and displayed on the display unit. However, the present invention is not limited thereto. Hereinafter, the contents of the speech-sign language output unit 400 according to the present disclosure will be described with reference to FIG. 3 .

According to the above configuration, the electronic device 1000 may include the under display camera unit 100 positioned below the first area of the display unit. In this case, as the direction in which the user is photographed coincides with the direction in which the user's gaze is directed, the user's gaze processing may be performed naturally.

2 is a view for explaining an example of an under display camera according to some embodiments of the present disclosure.

Referring to FIG. 2 , the under-display camera unit 100 includes a first substrate 110 , a second substrate 120 , a light transmitter 130 , a light receiver 140 , an optical filter unit 150 , and a driver 160 . ), a first support part 170 and a second support part 180 may be included. However, the present invention is not limited thereto.

According to some embodiments of the present disclosure, the under display camera unit 100 may include a first substrate 110 and a second substrate 120 positioned on the first substrate. For example, the first substrate 110 and the second substrate 120 may be a PCB substrate, but is not limited thereto.

The first substrate 110 may be positioned on one surface 190 of the under-display camera unit 100 . For example, the first substrate 110 may be located on the upper back side of the under-display camera unit 100 . In this case, the second substrate 120 may be located on the first substrate. Here, the upper portion may mean a position relatively distal to the under display camera unit 100 along a direction in which light is irradiated to the outside from the under display camera unit 100 . Accordingly, the lower portion may mean a position relatively proximal to the under display camera unit 100 along a direction in which light is emitted from the under display camera unit 100 to the outside.

The light transmitter 130 and the light receiver 140 are not implemented as separate modules on separate substrates, but are integrally formed on the same substrate (eg, the first substrate 110 and the second substrate 120 ). can be implemented. To this end, the components constituting the light transmitting unit 130 and the light receiving unit 140 are formed of the first substrate 110 or the second substrate 120 according to a direction for irradiating light to the outside and a direction for receiving light from the outside. It can be placed in a corresponding position.

More specifically, according to some embodiments of the present disclosure, the light transmitter 130 is disposed on the second substrate 120 to be coaxially aligned with the light source 131 and the light source 131 disposed on the first substrate 110 . It may include an arranged light generating module 132 . As described above, the light transmitter 130 may be integrally implemented with the light receiver 140 on the first substrate 110 and the second substrate 120 .

The light transmitter 130 may radiate a light pattern for 3D sensing to the outside through the coaxially aligned light source 131 and the light generation module 132 . An exemplary operation of the light transmitter 130 is as follows.

The light source 131 may output light having a wavelength band of a certain range. For example, the light source 131 may output IR (infrared) light. The IR light may be, for example, light having a wavelength band of 800 nm or more. The light source 131 may include at least one laser diode that projects light, a light emitting diode (LED), a vertical cavity surface emitting laser (VCSEL), or an edge emitting laser (EEL), but is limited thereto. doesn't happen

According to some embodiments of the present disclosure, the light generating module 132 may generate a light pattern irradiated to the outside of the under display camera unit 100 by the light of the light source 131 . The light generating module 132 may include various optical elements for generating a light pattern for 3D sensing. For example, the light generating module 132 may include micro-lens arrays (MLA) or diffractive optical elements (DOE). The light generated by the light source 131 may generate a light pattern by passing through the light generating module 132 . For example, when the under display camera unit 100 is a ToF module, the light pattern generated by the light generation module 132 uses a microlens array (eg, pseudo random MLA). It may be a surface light source pattern generated by The surface light source pattern may be a pattern generated by a diffusion effect generated by disposition of a plurality of micro-optical elements in a dot shape. In addition, the light pattern may include a line pattern, a matrix pattern, a dot pattern, and the like. Also, the light pattern may include a pattern of a plurality of dots that are uniformly or randomly arranged. However, the present invention is not limited thereto, and the light pattern generated by the light generating module 132 may include various patterns.

As described above, the light source 131 and the light generating module 132 may be coaxially aligned on the first substrate 110 and the second substrate 120 . Here, the term “coaxially aligned” may mean that light is positioned on the same axis along a direction in which light is radiated from the under-display camera unit 100 to the outside. Accordingly, when the light source 131 and the light generating module 132 are coaxially aligned on the first substrate 110 and the second substrate 120 , the light source 131 and the light generating module 132 are the light sources 131 . It may be positioned in a direction in which the light irradiated from the light can pass through the light generating module 132 to the outside.

In addition, according to some embodiments of the present disclosure, the light receiving unit 140 includes the light guide module 141 disposed on the second substrate 120 , and the first substrate 110 to be coaxially aligned with the light guide module 141 . ) may include a light sensing module 142 disposed in. As described above, the light receiving unit 140 may be integrally implemented with the light transmitting unit 130 on the first substrate 110 and the second substrate 120 .

The light receiver 140 may sense light from the outside through the light guide module 141 and the light sensing module 142 that are coaxially aligned, and may generate an optical image. Similarly to the light transmitter, when the light guide module 141 and the light sensing module 142 are coaxially aligned on the first substrate 110 and the second substrate 120 , the light guide module 141 and the light sensing module Reference numeral 142 may be positioned in a direction in which light incident from the outside may pass through the light guide module 141 to reach the light sensing module 142 .

Exemplary operations of the components of the light receiving unit 140 are as follows.

According to some embodiments of the present disclosure, the light guide module 141 may have a light transmitting layer having a predetermined pattern. In addition, the light guide module 141 may pass the light reflected from the outside of the under display camera unit 100 by the light pattern to the light sensing module 142 through the light transmitting layer.

The light guide module 141 may be, for example, a transmissive film having a light transmitting layer engraved in a predetermined pattern. The light guide module 141 may be a phase mask inducing a phase shift of incident light having a light transmitting layer having a predetermined pattern. In this case, the light guide module 141 may guide light incident from the outside to have an interference pattern (Moirι pattern) generated according to a predetermined pattern. The interference pattern may include an interference pattern, a wave pattern, a grid pattern, and the like. The interference pattern may mean a stripe that is visually created according to a difference in a period when a regularly repeating shape is repeatedly combined several times. The light constituting the interference pattern may be sensed by the light sensing module 142 and used to generate the first optical image.

As another example, the light guide module 141 may be a micro-optical device having a light transmitting layer having a predetermined pattern. In this case, the light guide module may induce externally incident lights to have a 'diffusing pattern' by the structure (eg, dot structure) of the micro-optical device. The light forming the diffusion pattern may be sensed by the light sensing module 142 and used to generate the first optical image. The predetermined pattern of the light transmitting layer may have various patterns. For example, the predetermined pattern may include a plurality of concentric circle patterns having different radii. Here, the plurality of concentric circle patterns may have various radii. Also, intervals between adjacent concentric circles may have various distances. The description of the plurality of concentric circle patterns is specifically discussed in Japanese Patent Application No. 2016-240818 (application date: December 13, 2016, applicant: HITACHI LTD), which is incorporated herein by reference in its entirety.

Further, the predetermined pattern of the light transmitting layer may include a symmetric grating pattern (typically symmetric of an odd number of gratings). For example, the predetermined pattern may be a symmetrical lattice pattern configured in the form of a straight line. For another example, the predetermined pattern may be a symmetrical grid pattern configured in a curved shape. The description of symmetric grating patterns is specifically discussed in US Patent Application Serial No. 15-547155 (filed 12/13/2016, Applicant: Rambus Inc.), which is incorporated herein by reference in its entirety.

In addition, the predetermined pattern of the light transmitting layer may include a surface light source pattern. Here, the surface light source pattern may be a pattern generated by a diffusion effect generated by disposition of a plurality of micro-optical elements in a dot shape.

However, the present invention is not limited thereto, and the predetermined pattern of the light transmitting layer may have various patterns other than the above-described patterns. For example, the predetermined pattern may include a line pattern, a matrix pattern, a dot pattern, and the like. In addition, the predetermined pattern may include a pattern of a plurality of dots arranged uniformly or randomly. Since the predetermined pattern induces the light passing through the light guide module 141 to have an interference pattern or a diffusion pattern according to the predetermined pattern, the predetermined pattern is a reference for reconstructing the first optical image by the control unit 200 can be used

According to some embodiments of the present disclosure, the light sensing module 142 may generate the first optical image by sensing the light passing through the light guide module 141 . Also, the controller 200 may generate the second optical image by reconstructing the first optical image based on the predetermined pattern.

The light sensing module 142 may convert the received light into an electrical signal. The light sensing module 142 may be an image sensor including a photo diode or a complementary metal-oxide semiconductor (CMOS). For example, the light sensing module 142 may be configured to sense a wavelength band of light irradiated from the light source 131 (eg, a wavelength band of IR light).

As described above, the predetermined pattern of the light transmitting layer may induce the light passing through the light guide module 141 to form a unique interference pattern or diffusion pattern according to the predetermined pattern. This light may be sensed by the light sensing module 142 and used to generate the first optical image.

As described above, the first optical image may have an interference pattern or a diffusing pattern generated by a predetermined pattern. The predetermined pattern of the light transmitting layer used herein may be a symmetrical lattice pattern in the form of a straight line.

Meanwhile, the first support unit 170 may protrude upward from a portion of the first substrate 110 . In this case, the first support unit 170 may support the side surface of the second substrate 120 . Accordingly, the first support unit 170 may position the second substrate 120 on the first substrate 110 . In this case, the first support unit 170 may accommodate the light source 131 and the light sensing module 142 therein. The first support unit 170 may be made of a plastic or metal material, but is not limited thereto.

Meanwhile, the second supporter 180 may protrude upward on a portion of the first substrate 110 that is outside the first supporter 170 . Accordingly, the second support 180 may have a shape surrounding the first support. In addition, the second support 180 may accommodate the light generating module 132 and the light guide module 141 therein. The second support unit 180 may be made of a plastic or metal material, but is not limited thereto.

Meanwhile, the optical filter unit 150 may be disposed on one side of the second substrate. The optical filter unit 150 may perform a function of limiting a wavelength band of light reaching the light sensing module 142 . For example, when the light from the light source 131 is IR light, the optical filter unit 150 may be configured to pass light of a wavelength band corresponding to the IR light. The optical filter unit 150 may be disposed on the upper side of the second substrate. In this case, the second support unit 180 may support the side surface of the optical filter unit 150 to be positioned on the second substrate.

Meanwhile, the driver 160 may control the operations of the light transmitter 130 and the light receiver 140 . For example, the driving unit 160 may include the light source 131 and the light generating module 132 of the light transmitting unit 130 according to the information processed by the control unit 200 (eg, the distance to the object, the amount of incident light, etc.). ) can be adjusted. In addition, the driving unit 160 may adjust the amount of light by blocking a portion of the light irradiated from the light transmitting unit 130 using a movable blocking film (not shown) or blocking a portion of the light incident on the light receiving unit 140 . Also, the driving unit 160 may adjust the direction (or angle) to which the light transmitting unit 130 and the light receiving unit 140 face. However, the present invention is not limited thereto, and the driving unit 160 may control various operations of the light transmitting unit 130 and the light receiving unit 140 . Since the under-display camera unit 100 of the present disclosure has a monolithic architecture, the operation of the light transmitter 130 and the light receiver 140 can be controlled by one or a relatively small number of drivers 160 .

Meanwhile, according to some embodiments of the present disclosure, the under-display camera unit 100 may include an under-display camera control unit (not shown). In this case, the under-display camera control unit may process the overall operation of the under-display camera unit 100 . For example, the operation of the under-display camera unit 100 may be processed by the control unit 200 of the electronic device. However, the present invention is not limited thereto.

According to the above configuration, in the under display camera unit 100 , the light receiving unit 140 that emits light and the light receiving unit 140 that receives the reflected light may be implemented in a monolithic architecture. Specifically, the light transmitting unit 130 and the light receiving unit 140 of the under display camera unit 100 are not implemented as separate modules on separate substrates, but on the same substrate (eg, the first substrate 110 and The second substrate 120 may be integrally implemented as a single chip. The monolithic architecture replaces the function of an optical element composed of a plurality of diffractive lenses having a complex structure and large volume through software reconstruction of an optical element and image of a simple structure that generates an interference pattern or diffusion pattern as described above. It can be possible by doing The under-display camera unit 100 having a monolithic architecture may have a simpler product configuration and manufacturing method than in the related art. Accordingly, the under-display camera unit 100 according to the present disclosure may have a low assembly cost and low power consumption.

3 is a view for explaining an example of an electronic device according to some embodiments of the present disclosure.

According to some embodiments of the present disclosure, the electronic device 1000 may include a sign language-voice output unit 300 and a voice-sign language output unit 400 . Also, the electronic device 1000 may provide a real-time online conversation between the hearing impaired and the general public. In the case of a conventional camera, it may be provided in an area at the top of the electronic device. Accordingly, an object such as a user's hand, which is a target of recognition, may be photographed by being biased toward the lower end of the screen photographed by the camera. In this case, in a complex movement such as sign language, there may be a problem in that the movement of the object is not accurately recognized.

On the other hand, referring to FIG. 3 , the under-display camera unit 100 of the electronic device 1000 according to the present invention may be located in a first area in which an object recognized on the display unit is displayed. In this case, the under-display camera unit 100 faces the object, so that the controller 200 can accurately recognize an object such as a user's hand photographed through the under-display camera unit 100 . However, the present invention is not limited thereto.

Meanwhile, according to some embodiments of the present disclosure, the sign language-voice output unit 300 recognizes a first sign language sensed from a first object recognized through the control unit 200, and a first voice corresponding to the first sign language. can be printed out.

To this end, the sign language-to-speech output unit 300 may include a sign language recognition unit, a first index generation unit, a sign language-word matching unit, a sentence generation unit, and a voice output unit. However, the present invention is not limited thereto.

The sign language recognition unit may recognize the first sign language detected from the first object recognized through the control unit 200 . However, the present invention is not limited thereto, and the sign language recognition unit may recognize the first sign language by detecting an object from the image captured by the under-display camera unit 100 .

Meanwhile, the first index generator may generate at least one sign language index based on the recognized first sign language. Here, the at least one sign language index may be for finding a word matching the first sign language in a sign language dictionary in which sign language and words corresponding to the sign language are stored in a database format. However, the present invention is not limited thereto.

Meanwhile, the sign language-word matching unit may match at least one word corresponding to each of the at least one sign language index.

For example, the sign language-word matching unit may match at least one word corresponding to each of the at least one sign language index by using a sign language dictionary stored in a storage unit (not shown). However, the present invention is not limited thereto.

Meanwhile, the sentence generator may generate at least one sentence using at least one word.

In general, sign language may lack propositions such as "silver, is, this, a". Accordingly, when only at least one word matched through the sign language-word matching unit is output as a voice, conversation between users may not be smooth. Accordingly, the sentence generator may generate at least one sentence by using at least one word. However, the present invention is not limited thereto.

Meanwhile, the voice output unit may output a first voice corresponding to the at least one generated sentence.

Meanwhile, in the present disclosure, the sign language-voice output unit may further include a text output unit. In this case, the text output unit may output at least one sentence generated through the sentence generation unit as text. However, the present invention is not limited thereto.

Meanwhile, according to some embodiments of the present disclosure, the voice-sign output unit 400 may recognize the second voice input through the microphone and output the second sign language corresponding to the second voice.

To this end, the speech-sign language output unit 400 may include a speech conversion unit, a second index generation unit, a text-image matching unit, a sign language generation unit, and a sign language output unit. However, the present invention is not limited thereto.

The voice converter may convert the second voice input through the microphone into text.

For example, the voice converter may convert the second voice into text based on a text-to-speech (TTS) engine. However, the present invention is not limited thereto.

Meanwhile, the second index generator may generate at least one sign language index based on the converted character. However, the present invention is not limited thereto.

Meanwhile, the text-image matching unit may match at least one image corresponding to each of the at least one sign language index.

For example, the text-image matching unit may match at least one image corresponding to each of the at least one sign language index by using a sign language dictionary stored in a storage unit (not shown). However, the present invention is not limited thereto.

Meanwhile, the sign language generator may generate the second sign language by using at least one matched image. In this case, the second sign language may be an image or an image. However, the present invention is not limited thereto.

Meanwhile, the sign language output unit may output the generated second sign language.

For example, the second sign language may be output as an image or an image. However, the present invention is not limited thereto.

Meanwhile, in the present disclosure, the speech-sign language output unit may further include a text output unit. In this case, the text output unit may output the converted text through the voice recognition unit. However, the present invention is not limited thereto.

According to the above-described configuration, the electronic device 1000 may allow a conversation between the hearing impaired and the general public. In this case, the control unit 200 may recognize the user's sign language from the image captured by the under-display camera unit 100 positioned in the area displayed on the object on the display unit. In addition, when the area where the object (hand) is displayed on the display unit is changed as the position of the hand is moved and deviated from the shooting screen, the controller 200 controls the under-display camera unit 100 to display the object on the display unit. area can be moved. Accordingly, the under-display camera unit 100 may be disposed in a direction toward the moved hand, and the control unit 200 may accurately recognize the user's sign language and transmit it to the other party by voice or text.

Meanwhile, when a user has an online conversation through the electronic device 1000 , an area viewed by the user on the display unit may vary. For example, in the case of a video meeting in which a plurality of people participate, the display area may be divided in a grid format, and a plurality of counterparts may be displayed in each area. In this case, the user may look at the corner of the display unit to look at the other party. In addition, the user may speak sign language in the direction in which the gaze naturally faces according to the movement of the gaze, that is, at the corner of the display unit. In this case, it may be difficult to recognize the user's hand as the user's hand deviates from the screen being photographed by the camera. Also, since the user's gaze does not face the camera, the user displayed on the electronic device may look unnatural. The under-display camera unit 100 in the present disclosure can move to various areas of the display unit under the control of the control unit 200, so that the above-described problem can be solved. Hereinafter, an example of a method in which the control unit 200 moves the under-display camera unit 100 according to the present disclosure will be described.

4 is a cross-sectional view taken along line A-A' of FIG. 3 for explaining an example of a moving means according to some embodiments of the present disclosure.

Referring to FIG. 4A , the electronic device 1000 may include an under-display camera unit 100 , a housing 500 , and a moving unit 600 . However, the present invention is not limited thereto.

The housing 500 may form the exterior of the electronic device 1000 . In addition, the housing 500 may include a display unit 510 on one surface. However, the present invention is not limited thereto.

The moving means 600 may include one end fixed to the under-display camera unit 100 and the other end fastened to the support member 610 .

For example, the moving means 600 and the support member 610 may be coupled through a rack and pinion gear structure.

Specifically, a motor (not shown) may be provided inside the moving means 600 , and a pinion gear may be coupled to the shaft of the motor. In addition, the pinion gear of the moving means 600 may be coupled to a rack formed at one end of the support member 610 . In this case, the moving means 600 may move in the upper and lower directions of the support member 610 according to the driving of the motor. Here, the upper and lower directions of the support member 610 are the same as the upper and lower directions of the display unit 510 in a direction in which a user generally looks at the display unit 510 to use the electronic device 1000 . can do. However, the present invention is not limited thereto, and the moving means 600 and the support member 610 may be coupled through a rail.

Meanwhile, in the present disclosure, the support member 610 may move in left and right directions. Here, the left and right directions may be the same as the left and right directions of the display unit 510 .

For example, upper and lower ends of the support member 610 may be fastened to a rail (not shown) provided on the inner surface of the housing 500 . In this case, the support member 610 may be moved in the left and right directions of the housing 500 under the control of the controller 200 .

According to the above-described configuration, the under-display camera unit 100 may be moved up, down, left, and right at the bottom of the display unit 510 under the control of the control unit 200 .

Meanwhile, according to some embodiments of the present disclosure, at least one of the under-display camera unit 100 or the moving means 600 may be provided with a tilting device so that an angle can be adjusted.

For example, as described above, the driving unit 160 of the under-display camera unit 100 may further include a tilting unit for adjusting the angles of the light transmitting unit 130 and the light receiving unit 140 . However, the present invention is not limited thereto.

According to the above-described configuration, the under-display camera unit 100 may be moved up, down, left, and right under the control of the control unit 200 in the lower portion of the display unit 510 , and the angle may be adjusted. In this case, as the direction in which the user is photographed coincides with the direction in which the user's gaze is directed, the user's gaze processing may be performed naturally.

5 is a flowchart illustrating an example of a method in which a control unit of an electronic device moves an under-display camera unit according to some embodiments of the present disclosure; 6 is a view for explaining an example of a first object displayed on a display unit according to some embodiments of the present disclosure;

Referring to FIG. 5 , the controller 200 of the electronic device 1000 may recognize whether a first object is displayed in a second area different from the first area ( S110 ).

Specifically, the controller 200 may recognize the first object photographed through the under-display camera unit 100 . In addition, the controller 200 may recognize whether the first object is displayed in a second area different from the first area. Here, the first area may be a central area of the display unit.

For example, referring to FIG. 6A , the first object 700 recognized by the controller 200 may be the user's hand. In addition, the controller 200 may recognize that the first object is being displayed in a second area different from the first area.

Meanwhile, referring to FIG. 5 , when the control unit 200 of the electronic device 1000 recognizes that the first object is being displayed in a second area different from the first area, under the first object to be displayed in the first area. The display camera unit 100 may be moved to the second area by the moving means 600 ( S120 ).

Specifically, that the first object is displayed in the second area means that the first object, such as the user's hand, which is the target of recognition, is photographed biasedly toward one area of the screen photographed by the under-display camera unit 100 . can mean In this case, in a complex movement such as sign language, the controller 200 may not accurately recognize the movement of the first object. Accordingly, the controller 200 may move the under-display camera unit 100 to the second area by the moving means 600 so that the first object is displayed in the first area.

For example, referring to FIG. 6B , the control unit 200 may move the under-display camera unit 100 to the second area where the first object 700 is displayed by the moving means 600 . can In this case, the direction in which the under-display camera unit 100 takes a picture may correspond to the direction in which the user's hand is positioned.

Referring to FIG. 6C , since the position in which the under-display camera unit 100 captures the user's hand corresponds to the direction in which the user's hand is located, the first object 700 is displayed on the display of the electronic device 1000 . It may be displayed in the first area of the sub. In other words, when the under-display camera unit 100 captures the user's hand from the front, the controller 200 may display the first object 700 on the first area of the display unit of the electronic device 1000 .

According to the above-described configuration, the control unit 200 can make the position of the under-display camera unit 100 correspond to a first object such as a user's hand photographed by the under-display camera unit 100 and the position of the first object. have. In this case, the movement of the first object photographed through the under-display camera unit 100 may be accurately recognized.

In addition, as the controller 200 displays the first object on the central region of the display unit, a counterpart who performs an online conversation with the user may also gaze at the first object comfortably.

7 is a flowchart illustrating an example of a method in which a controller of an electronic device moves an under-display camera unit according to another exemplary embodiment of the present disclosure. 8 is a view for explaining an example of a first object displayed on a display unit according to some other exemplary embodiments of the present disclosure.

Referring to FIG. 7 , the control unit 200 of the electronic device 1000 may recognize whether the first object photographed through the under-display camera unit 100 is the user's face ( S210 ).

For example, the controller 200 may recognize whether the first object is the user's face based on the position and shape information of the face contour, eyebrows, nose, mouth, and ears. However, the present invention is not limited thereto.

Meanwhile, when the first object is the user's face, the controller 200 of the electronic device 1000 may recognize a third area on the display unit toward which the user's gaze is directed ( S220 ).

For example, referring to FIG. 8A , the actual user may look at the upper left side of the display unit of the electronic device 1000 . In this case, the user's image displayed on the display unit of the electronic device 1000 may also be displayed as if looking at the upper left side of the display unit. Accordingly, the other party performing an online conversation with the user may feel that the user's gaze is awkward because the user's gaze looks to the upper left side rather than the front. To prevent this, the controller 200 may recognize a third area to which the user's gaze is directed in order to move the under-display camera unit 100 so that the processing of the user's gaze becomes natural. In this case, the control unit 200 may move the under-display camera unit 100 to the third area to which the user's gaze is directed by the moving means 600 .

Specifically, in order to recognize the third area to which the user's gaze is directed, the controller 200 may use the user's gaze position data received from a module such as a gaze collection unit (not shown). Here, the gaze collecting unit may be a module that analyzes the user's gaze through a sensor provided in the display unit and collects gaze position data in real time. For example, the gaze collection unit may be configured to accurately grasp the distance between the display unit and the user's eyes and the movement of the user's pupils through a sensor, a distance sensor, or the like. However, the present invention is not limited thereto, and the gaze collection data may be collected by the under-display camera unit 100 .

Meanwhile, when the third area is recognized, the control unit 200 of the electronic device 1000 may move the under-display camera unit 100 to the third area by the moving means 600 ( S230 ).

For example, referring to FIG. 8B , the control unit 200 may move the under-display camera unit 100 to the third area by the moving means 600 . In this case, the direction in which the under-display camera unit 100 captures the user's face may correspond to the direction in which the user's gaze is directed.

Accordingly, referring to FIG. 8C , the user's gaze displayed on the display unit of the electronic device 1000 may be displayed to face the center of the display unit. That is, as the direction in which the user is photographed coincides with the direction in which the user's gaze is directed, the user's gaze processing can be performed naturally. However, the present invention is not limited thereto.

9 shows a general schematic diagram of an example computing environment in which embodiments of the present disclosure may be implemented.

Although the present disclosure has been described above generally in the context of computer-executable instructions that may be executed on one or more computers, those skilled in the art will appreciate that the present disclosure may be implemented as a combination of hardware and software and/or in combination with other program modules. you will know

Generally, modules herein include routines, procedures, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In addition, those skilled in the art will appreciate that the methods of the present disclosure can be applied to single-processor or multiprocessor computer systems, minicomputers, mainframe computers as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, etc. (each of which is It will be appreciated that other computer system configurations may be implemented, including those that may operate in connection with one or more associated devices.

The described embodiments of the present disclosure may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Computers typically include a variety of computer-readable media. Media accessible by a computer includes volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media. By way of example, and not limitation, computer-readable media may include computer-readable storage media and computer-readable transmission media.

Computer readable storage media includes volatile and nonvolatile media, temporary and non-transitory media, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. includes media. A computer-readable storage medium may be RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disk (DVD) or other optical disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device, or other magnetic storage device. device, or any other medium that can be accessed by a computer and used to store the desired information.

Computer readable transmission media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and Includes all information delivery media. The term modulated data signal means a signal in which one or more of the characteristics of the signal is set or changed so as to encode information in the signal. By way of example, and not limitation, computer-readable transmission media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above are also intended to be included within the scope of computer-readable transmission media.

An example environment 1100 implementing various aspects of the disclosure is shown including a computer 1102 , the computer 1102 including a processing unit 1104 , a system memory 1106 , and a system bus 1108 . do. A system bus 1108 couples system components, including but not limited to system memory 1106 , to the processing device 1104 . The processing device 1104 may be any of a variety of commercially available processors. Dual processor and other multiprocessor architectures may also be used as processing unit 1104 .

The system bus 1108 may be any of several types of bus structures that may further interconnect a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. System memory 1106 includes read only memory (ROM) 1110 and random access memory (RAM) 1112 . A basic input/output system (BIOS) is stored in non-volatile memory 1110, such as ROM, EPROM, EEPROM, etc., the BIOS is the basic input/output system (BIOS) that helps transfer information between components within computer 1102, such as during startup. contains routines. RAM 1112 may also include high-speed RAM, such as static RAM, for caching data.

The computer 1102 may also include an internal hard disk drive (HDD) 1114 (eg, EIDE, SATA) - this internal hard disk drive 1114 may also be configured for external use within a suitable chassis (not shown). Yes—a magnetic floppy disk drive (FDD) 1116 (eg, for reading from or writing to removable diskette 1118), and an optical disk drive 1120 (eg, a CD-ROM) for reading from, or writing to, disk 1122, or other high capacity optical media, such as DVD. The hard disk drive 1114 , the magnetic disk drive 1116 , and the optical disk drive 1120 are connected to the system bus 1108 by the hard disk drive interface 1124 , the magnetic disk drive interface 1126 , and the optical drive interface 1128 , respectively. ) can be connected to The interface 1124 for external drive implementation includes, for example, at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

These drives and their associated computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, and the like. In the case of computer 1102, drives and media correspond to storing any data in a suitable digital format. Although the description of computer readable storage media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will use zip drives, magnetic cassettes, flash memory cards, cartridges, It will be appreciated that other tangible computer-readable storage media and the like may also be used in the exemplary operating environment and any such media may include computer-executable instructions for performing the methods of the present disclosure. .

A number of program modules may be stored in the drive and RAM 1112 , including an operating system 1130 , one or more application programs 1132 , other program modules 1134 , and program data 1136 . All or portions of the operating system, applications, modules, and/or data may also be cached in RAM 1112 . It will be appreciated that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

A user may enter commands and information into the computer 1102 via one or more wired/wireless input devices, for example, a pointing device such as a keyboard 1138 and a mouse 1140 . Other input devices (not shown) may include a microphone, IR remote control, joystick, game pad, stylus pen, touch screen, and the like. Although these and other input devices are connected to the processing unit 1104 through an input device interface 1142 that is often connected to the system bus 1108, parallel ports, IEEE 1394 serial ports, game ports, USB ports, IR interfaces, It may be connected by other interfaces, etc.

A monitor 1144 or other type of display device is also coupled to the system bus 1108 via an interface, such as a video adapter 1146 . In addition to the monitor 1144, the computer typically includes other peripheral output devices (not shown), such as speakers, printers, and the like.

Computer 1102 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1148 via wired and/or wireless communications. Remote computer(s) 1148 may be workstations, server computers, routers, personal computers, portable computers, microprocessor-based entertainment devices, peer devices, or other common network nodes, and are generally Although including many or all of the components described, only memory storage device 1150 is shown for simplicity. The logical connections shown include wired/wireless connections to a local area network (LAN) 1152 and/or a larger network, eg, a wide area network (WAN) 1154 . Such LAN and WAN networking environments are common in offices and companies, and facilitate enterprise-wide computer networks, such as intranets, all of which can be connected to a worldwide computer network, for example, the Internet.

When used in a LAN networking environment, the computer 1102 is coupled to the local network 1152 through a wired and/or wireless communication network interface or adapter 1156 . Adapter 1156 may facilitate wired or wireless communication to LAN 1152 , which LAN 1152 also includes a wireless access point installed therein for communicating with wireless adapter 1156 . When used in a WAN networking environment, the computer 1102 may include a modem 1158 , connected to a communication server on the WAN 1154 , or otherwise establishing communications over the WAN 1154 , such as over the Internet. have the means A modem 1158 , which may be internal or external and a wired or wireless device, is coupled to the system bus 1108 via a serial port interface 1142 . In a networked environment, program modules described for computer 1102 , or portions thereof, may be stored in remote memory/storage device 1150 . It will be appreciated that the network connections shown are exemplary and other means of establishing a communication link between the computers may be used.

The computer 1102 may be associated with any wireless device or object that is deployed and operates in wireless communication, for example, a printer, scanner, desktop and/or portable computer, portable data assistant (PDA), communication satellite, wireless detectable tag. It operates to communicate with any device or place and phone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network or may simply be an ad hoc communication between at least two devices.

Wi-Fi (Wireless Fidelity) makes it possible to connect to the Internet, etc. without a wired connection. Wi-Fi is a wireless technology such as cell phones that allows these devices, eg, computers, to transmit and receive data indoors and outdoors, ie anywhere within range of a base station. Wi-Fi networks use a radio technology called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks may operate in unlicensed 2.4 and 5 GHz radio bands, for example, at 11 Mbps (802.11a) or 54 Mbps (802.11b) data rates, or in products that include both bands (dual band). have.

Those of ordinary skill in the art of the present disclosure will recognize that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein include electronic hardware, (convenience For this purpose, it will be understood that it may be implemented by various forms of program or design code (referred to herein as "software") or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. A person skilled in the art of the present disclosure may implement the described functionality in various ways for each specific application, but such implementation decisions should not be interpreted as a departure from the scope of the present disclosure.

The various embodiments presented herein may be implemented as methods, apparatus, or articles of manufacture using standard programming and/or engineering techniques. The term “article of manufacture” includes a computer program or media accessible from any computer-readable device. For example, computer-readable storage media include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical disks (eg, CDs, DVDs, etc.), smart cards, and flash drives. memory devices (eg, EEPROMs, cards, sticks, key drives, etc.). The term “machine-readable medium” includes, but is not limited to, wireless channels and various other media that can store, hold, and/or convey instruction(s) and/or data.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the embodiments presented herein, but is to be construed in the widest scope consistent with the principles and novel features presented herein.

Claims (10)

an under-display camera unit located below the first area of the display unit; and
a control unit for recognizing a first object photographed through the under-display camera unit and controlling the display unit to display the first object on the first area of the display unit;
including,
The control unit is:
When the first object is a user's face, a third area to which the user's gaze is directed is recognized on the display unit based on gaze position data collected in real time, and
When the third area is different from the first area, the under-display camera unit is moved by the moving means so that the direction in which the under-display camera unit captures the user's face and the direction in which the user's gaze is directed correspond to each other. moving to the third area,
electronic device.
The method of claim 1,
The first area is
The central area of the display unit,
electronic device.
The method of claim 1,
The control unit is
when the first object is displayed in a second area different from the first area, moving the under-display camera unit to the second area by the moving means so that the first object is displayed in the first area;
electronic device.
4. The method of claim 3,
The control unit is
When the first object is a user's hand and the first object is displayed on the second area, the under display camera unit is moved by the moving means so that the user's hand is displayed on the first area. moving to
electronic device.
delete The method of claim 1,
a sign language-voice output unit recognizing a first sign language sensed from the first object and outputting a first voice corresponding to the first sign language; and
a voice-sign language output unit recognizing a second voice input through a microphone and outputting a second sign language corresponding to the second voice;
further comprising,
electronic device.
7. The method of claim 6,
The sign language-voice output unit,
a sign language recognition unit for recognizing the first sign language sensed from the first object;
a first index generator for generating at least one sign language index based on the recognized first sign language;
a sign language-word matching unit matching at least one word corresponding to each of the at least one sign language index;
a sentence generator for generating at least one sentence using the at least one word; and
a voice output unit for outputting the first voice corresponding to the at least one sentence;
containing,
electronic device.
7. The method of claim 6,
The voice-sign output unit,
a voice converter converting the second voice input through the microphone into text;
a second index generator for generating at least one sign language index based on the character;
a text-image matching unit matching at least one image corresponding to each of the at least one sign language index;
a sign language generator for generating the second sign language by using the at least one image; and
a sign language output unit for outputting the second sign language;
containing,
electronic device.
The method of claim 1,
The under display camera unit,
a first substrate;
a second substrate positioned on the first substrate;
a light transmitter including a light source disposed on the first substrate and a light generating module disposed on the second substrate to be coaxially aligned with the light source; and
a light receiver including a light guide module disposed on the second substrate and a light sensing module disposed on the first substrate to be coaxially aligned with the light guide module;
including,
The light generating module generates a light pattern irradiated to the outside of the under display camera unit by the light of the light source,
The light guide module has a light transmitting layer having a predetermined pattern and passes light reflected from the outside of the under display camera unit by the light pattern to the light sensing module through the light transmitting layer,
The light sensing module generates a first optical image by sensing the light passing through the light guide module, and
The control unit generates a second optical image by reconstructing the first optical image based on the predetermined pattern,
electronic device.
The method of claim 1,
The under-display camera unit includes a Time Of Flight (TOF) sensor module, a Charge-Coupled Device (CCD) module, a Complementary Metal-Oxide-Semiconductor (CMOS) module, or a Single Photon Avalanche Diode (SPAD) module. ) containing the module,
electronic device.

Images