KR20230020798A - Apparatus for processing image and methods thereof - Google Patents

Abstract

Disclosed is an electronic device for projecting an image. The electronic device comprises: a projection unit; an acceleration sensor configured to sense an angle at which an electronic device is rotated in the direction of gravity; a plurality of distance sensors configured to sense an angle at which the electronic device is rotated relative to a screen; and a processor which performs keystone correction on the image based on the sensing results by the sensor unit and projects the keystone-corrected image onto the screen by controlling the projection unit. The processor reduces the size of the keystone-corrected image when the size of the image to be displayed on the screen exceeds an image display range on the screen. Accordingly, undistorted images can be provided in the maximum size without a camera.

Description

Electronic device and its image processing method {APPARATUS FOR PROCESSING IMAGE AND METHODS THEREOF}

The present invention relates to an electronic device and method for processing an image, and more particularly, to an electronic device and an image processing method for performing keystone correction to prevent display distortion of an image.

Recently, with the development of electronic technology and optical technology, various projectors are being used. A projector refers to an electronic device that projects light onto a screen (or a projection surface) to form an image on the screen.

When viewing an image using a projector, when the projector is placed upright on a flat surface in the direction of the screen, a rectangular image is displayed on the screen. Otherwise, up/down or left/right distortion occurs or an image in a rotated state is displayed on the screen. This distortion is called the keystone effect.

In the conventional case, a projector uses an image sensor to photograph a screen, and then processes the photographed image to calculate and use a distorted angle between the screen and the projector. In this case, a correction process to match the optical axis of the projector and the measurement axis of the image sensor must be performed. If an error occurs during this correction process, it has a great influence on the angle calculation. In addition, a process of processing an image captured by an image sensor also requires a large amount of calculation. In particular, in the case of an ultra short throw beam projector (Ultra Short Throw Beam Projector) that is being developed recently, a camera with a wide-angle lens is required to view the entire screen with an image sensor because the distance between the projection surface and the projector is very close. However, in this case, since the distortion is severe, it is not easy to calibrate the measurement axis of the optical axis and the image sensor, and the error increases. In the case of the Ultra Short Throw Beam Projector, due to the nature of the product, even a small distortion causes a large distortion of the image projected on the screen, so precise angle measurement is required, but the method using the image sensor has limitations.

Also, when an image subjected to keystone correction is projected onto a screen, the size of the image varies depending on the distance between the screen and the projector. Conventionally, there was also a problem in that it was difficult to fit the image on which keystone correction was performed to the size of the screen.

The present invention is to solve these problems, and an object of the present invention is an electronic device that performs keystone correction using an acceleration sensor and a distance sensor and adjusts the size of an image according to a distance from the screen, and an image processing method thereof is in providing

According to one embodiment of the present invention for achieving the above object, an electronic device includes a sensor unit including a projection unit, an acceleration sensor, and a plurality of distance sensors, and a processor, wherein the acceleration sensor determines that the electronic device is gravitational direction, the plurality of distance sensors detects an angle at which the electronic device is rotated with respect to the screen, and the processor performs keystone correction on the image based on the sensing value obtained by the sensor unit. and controls the projection unit to project the keystone-corrected image onto the screen, and if the image projected on the screen exceeds a preset area within the screen, the size of the keystone-corrected image is set. Control.

Meanwhile, the electronic device further includes a memory in which information about the image display range and the distance to the screen is stored, and the processor stores the distance to the screen obtained from the sensor unit in the memory, and displays the distance to the screen. The predetermined region may be recognized based on coordinates of a plurality of vertices of an image to be displayed.

The processor may control the size of the keystone-corrected image to be displayed at the maximum size within the preset region when all four vertex coordinates of the image to be displayed on the screen are within the image display range.

In addition, when the movement of the electronic device is identified based on a comparison result of a plurality of sensing values obtained by the acceleration sensor, the processor may control the projection unit to project a UI for guiding fixing of the electronic device. .

The processor may control the projection unit to project a UI for guiding fixing of the electronic device when motion of the electronic device is identified based on a comparison result of a plurality of sensing values obtained by the plurality of distance sensors. can

In addition, the processor may not perform the keystone correction when the movement of the electronic device is identified.

Meanwhile, the plurality of distance sensors are disposed on both sides of the projection unit, and the processor determines that the electronic device operates based on a result value sensed by each of the plurality of distance sensors and a distance between the plurality of distance sensors. A rotation angle based on the screen may be calculated.

According to one embodiment of the present invention for achieving the above object, an image processing method of an electronic device including a projection unit provides an angle at which the electronic device is rotated in a gravitational direction and an angle at which the electronic device is rotated with respect to a screen. Obtaining, respectively, performing keystone correction on the image based on the obtained sensing values, and if the image projected on the screen exceeds a predetermined area within the screen, the size of the image for which the keystone correction is performed is determined. Controlling and projecting the size-controlled image on the screen through the projection unit.

Meanwhile, the image processing method may further include detecting a distance to the screen and recognizing the predetermined area based on coordinates of a plurality of vertices of an image to be displayed on the screen.

In the step of controlling the size of the keystone-corrected image, when the coordinates of the four vertices of the image to be displayed on the screen are all within the image display range, the keystone correction is displayed at the maximum size within the preset region. You can control the size of the image.

Meanwhile, the image processing method includes obtaining sensing values of an acceleration sensor provided in the electronic device multiple times, identifying a motion of the electronic device based on a comparison result of the obtained plurality of sensing values, and the electronic device. If the motion of is identified, the method may further include projecting a UI for guiding fixing of the electronic device.

In addition, the image processing method includes obtaining sensing values of a plurality of distance sensors provided in the electronic device multiple times, identifying motion of the electronic device based on a comparison result of the obtained plurality of sensing values, and When the movement of the electronic device is identified, the method may further include projecting a UI for guiding fixing of the electronic device.

Also, the image processing method may further include not performing the keystone correction when the movement of the electronic device is identified.

The image processing method further includes calculating an angle at which the electronic device is rotated with respect to the screen based on result values sensed by a plurality of distance sensors disposed on both sides of the projection unit in the electronic device. can include

According to one embodiment of the present invention for achieving the above object, a non-transitory computer readable storage medium in which a program for performing an image processing method of an electronic device including a projection unit is recorded is rotated in the direction of gravity by the electronic device. obtaining the obtained angle and the rotation angle of the electronic device with respect to the screen, performing keystone correction on the image based on the obtained sensing value, and projecting the image onto the screen to a predetermined area within the screen. If it exceeds, controlling the size of the keystone-corrected image and projecting the size-controlled image on the screen through the projection unit.

As described above, according to various embodiments of the present disclosure, keystone correction can be effectively performed without an image sensor. Also, the keystone-corrected image can be displayed in an optimal size on the screen.

1 is a block diagram showing the configuration of an electronic device according to an embodiment of the present invention;
2 is a diagram for explaining the operation of the electronic device of FIG. 1;
3 is a diagram for explaining a method of calculating a displacement between a screen and an electronic device using two distance sensors;
4 is a diagram for explaining another example using a distance sensor;
5 is a diagram for explaining a keystone correction method;
6 is a diagram for specifically explaining a method of performing keystone correction;
7 is a diagram for explaining keystone correction;
8 is a diagram for explaining image processing according to an embodiment of the present invention;
9 is a flowchart for explaining an image processing method according to an embodiment of the present invention;
10 is a flowchart for explaining an image processing method according to another embodiment of the present invention;
11 is a diagram for comparing before and after image processing;
12 is a perspective view illustrating an external appearance of an electronic device according to an embodiment of the present disclosure;
13 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure;
14 is a perspective view illustrating an external appearance of an electronic device according to other embodiments of the present disclosure;
15 is a perspective view illustrating an external appearance of an electronic device according to another embodiment of the present disclosure;
16 is a perspective view illustrating an external appearance of an electronic device according to another embodiment of the present disclosure;
17A is a perspective view illustrating an external appearance of an electronic device according to another embodiment of the present disclosure;
17B is a perspective view illustrating a rotated state of the electronic device of FIG. 17A.

Since the present embodiments can apply various transformations and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to the specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives of the embodiments of the present disclosure. In connection with the description of the drawings, like reference numerals may be used for like elements.

In describing the present disclosure, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, a detailed description thereof will be omitted.

In addition, the following embodiments may be modified in many different forms, and the scope of the technical idea of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the spirit of the disclosure to those skilled in the art.

Terms used in this disclosure are only used to describe specific embodiments, and are not intended to limit the scope of rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present disclosure, expressions such as “has,” “can have,” “includes,” or “can include” indicate the presence of a corresponding feature (eg, numerical value, function, operation, or component such as a part). , which does not preclude the existence of additional features.

In this disclosure, expressions such as “A or B,” “at least one of A and/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” (1) includes at least one A, (2) includes at least one B, Or (3) may refer to all cases including at least one A and at least one B.

Expressions such as "first," "second," "first," or "second," used in the present disclosure may modify various elements regardless of order and/or importance, and may refer to one element as It is used only to distinguish it from other components and does not limit the corresponding components.

A component (e.g., a first component) is "(operatively or communicatively) coupled with/to" another component (e.g., a second component); When referred to as "connected to", it should be understood that the certain component may be directly connected to the other component or connected through another component (eg, a third component).

On the other hand, when an element (eg, a first element) is referred to as being “directly connected” or “directly connected” to another element (eg, a second element), the element and the above It may be understood that other components (eg, a third component) do not exist between the other components.

The expression “configured to (or configured to)” as used in this disclosure means, depending on the situation, for example, “suitable for,” “having the capacity to.” ," "designed to," "adapted to," "made to," or "capable of." The term "configured (or set) to" may not necessarily mean only "specifically designed to" hardware.

Instead, in some contexts, the phrase "device configured to" may mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "a processor configured (or configured) to perform A, B, and C" may include a dedicated processor (eg, embedded processor) to perform the operation, or by executing one or more software programs stored in a memory device. , may mean a general-purpose processor (eg, CPU or application processor) capable of performing corresponding operations.

In an embodiment, a 'module' or 'unit' performs at least one function or operation, and may be implemented with hardware or software, or a combination of hardware and software. In addition, a plurality of 'modules' or a plurality of 'units' may be integrated into at least one module and implemented by at least one processor, except for 'modules' or 'units' that need to be implemented with specific hardware.

Meanwhile, various elements and regions in the drawings are schematically drawn. Therefore, the technical spirit of the present invention is not limited by the relative size or spacing drawn in the accompanying drawings.

Hereinafter, various embodiments of the present invention will be described in detail using the accompanying drawings.

1 is a block diagram showing the configuration of an electronic device according to an embodiment of the present invention. In this embodiment, since the electronic device 100 can be implemented as a projector for projecting an image onto a wall or a screen or a display device having an image projection function, the electronic device 100 will be collectively referred to as the electronic device 100 in this specification.

The electronic device 100 of FIG. 1 includes a projection unit 110, a processor 170, and a sensor unit 180.

The projection unit 110 is a component for projecting light for expressing an image to the outside. The projection unit 110 may include various detailed components such as a light source, a projection lens, and a reflector. The operating method and detailed configuration of the projection unit 110 will be described in detail later.

The sensor unit 180 may include an acceleration sensor 181 and distance sensors 182-1 to 182-n. As an example, one or more distance sensors 182 - 1 to 182 - n may be included in the sensor unit 180 .

The acceleration sensor 181 is a component for detecting an angle at which the electronic device 100 is rotated in the direction of gravity. For example, a 3-axis acceleration sensor is used. The 3-axis acceleration sensor measures gravitational acceleration for each axis and provides raw data to the processor 170 .

The plurality of distance sensors 182-1 to 182-n are components for detecting a distance to an external object. The distance sensor may be implemented in various types such as an ultrasonic sensor, an infrared sensor, a LIDAR sensor, a radar sensor, and a photodiode sensor. When a plurality of distance sensors are provided, the processor 170 may obtain an angle at which the electronic device 100 is rotated (or distorted) with respect to the screen 10 based on the distance detected by each distance sensor. there is.

The processor 170 is a component for performing overall control operations or image processing operations for the electronic device 100 . The processor 170 performs keystone correction on an image based on the results detected by each sensor. The processor 170 appropriately displays the size of the keystone-corrected image according to the size of the screen. Specifically, the processor 170 calculates the pitch angle, roll angle, and yaw angle of the electronic device 100, respectively, using detection results of the acceleration sensor 181 and the plurality of distance sensors 182-1 to 182-n. do. The processor 170 estimates the posture of the electronic device 100 using the calculated angles, and converts the image based on information such as the estimated posture and the distance to the screen. Accordingly, the maximum possible image can be displayed on the screen 10 without distortion, and when the image is displayed larger than the size of the screen 10 due to a large distance, it is reduced to fit the size of the screen 10. can be displayed As a result, since the user can watch an image of the maximum size and resolution within the size range of the screen 10, viewing satisfaction is increased.

FIG. 2 is a diagram for explaining an example of a sensor arrangement position and operation of the electronic device of FIG. 1 . As shown in FIG. 2 , when x, y, and z axes are defined with respect to the electronic device 100, a pitch angle θ rotates with respect to the y axis. A roll angle (φ) rotating around the x-axis and a yaw angle (ψ) rotating around the z-axis are respectively defined. In FIG. 2 , the acceleration sensor 181 is embedded in the electronic device 100, and two distance sensors 182-1 and 182-2 may be used. The two distance sensors 182 - 1 and 182 - 2 may be disposed on both sides of the projection unit 110 . For example, the two distance sensors 182 - 1 and 182 - 2 may be disposed side by side on both sides of the projection unit 110 .

The processor 170 calculates a pitch angle and a roll angle based on an output value output from the acceleration sensor 181, that is, a detection result. When the electronic device 100 including the acceleration sensor 181 is not moving, only the effect of gravitational acceleration is measured on each axis of the acceleration sensor. Using this value, how much the device including the acceleration sensor tilts toward the direction of gravity You can tell if you're losing An example of a specific operation expression is as follows.

<Equation 1>

In Equation 1, Abx, Aby, and Abz are x, y, and z-axis acceleration values of the acceleration sensor 181, respectively.

3 is a diagram for explaining a method of calculating a rotation angle between a screen and an electronic device using two distance sensors. As shown in FIG. 2, when the electronic device 100 is arranged in a direction different from the screen 10 in a state where the two distance sensors 182-1 and 182-2 are arranged side by side on both sides of the electronic device 100, FIG. As shown in 3, the distances d1 and d2 between the two distance sensors 182-1 and 182-2 and the screen 10 are different.

If the distance between the two distance sensors 182-1 and 182-2 is l, the processor 170 determines the output values of the plurality of distance sensors 182-1 and 182-2, that is, the detection result, and the distance l Calculate the yaw angle (ψ) based on The specific calculation formula is as follows.

<Equation 2>

3 illustrates a case where two distance sensors are disposed in the front direction of the electronic device 100 , however, the distance sensors may vary depending on the appearance or placement of the electronic device 100 . 4, when the electronic device 100 is disposed on the floor and the screen 10 is installed upward from the front of the electronic device 100, the distance sensor 282 is tilted at a certain angle α. may be placed as In this case, the processor 170 may obtain the distance di to the screen using the following equation.

<Equation 3>

In Equation 3, i means the number of the distance sensor, and dio means the measurement value of the ith distance sensor. The processor 170 may calculate a rotation angle with respect to the screen 10 by correcting the measured values of each distance sensor using Equation 3 and then applying the corrected distance value to Equation 2.

Since the distance 1 between the distance sensors or the arrangement angle α corresponds to manufacturing specification information of the electronic device 100, it may be obtained in advance, stored in a memory (not shown) of the electronic device 100, and used for calculation.

As described above, the processor 170 calculates the pitch angle, the roll angle, and the yaw angle based on the detection results of the respective sensors, and then performs keystone correction on the image using the calculated pitch angle, roll angle, and yaw angle.

5 is a diagram for explaining a keystone correction method. If the pitch angle, roll angle, and yaw angle are defined as described above, when the processor 170 controls the projection unit 110 to project an image when the tilt of the electronic device 100 is in a normal state, the screen 10 An image without keystone is displayed. In this specification, the normal state means a case where the rotation angle between the electronic device 100 and the screen and the angle at which the electronic device 100 is tilted are reference values, respectively. In a normal case, the reference value may be 0, but a value other than 0 may be set as the reference value depending on the placed position or surrounding conditions.

When the electronic device 100 is tilted in the x-axis direction and the roll angle is different from the reference value (for example, 0), the image projected on the screen 10 is tilted to the right according to the tilted direction (31). Alternatively, it is displayed tilted to the left (32). The processor 170 rotates the image in a direction opposite to the roll angle change direction, so that the image 30 in a normal form is displayed on the screen.

When the electronic device 100 is tilted in the y-axis direction and the pitch angle is different from the reference value (eg, 0), the image projected onto the screen 10 according to the tilted direction becomes a trapezoid shape (41, 42). In this case, the processor 170 performs correction to increase the length of the upper side or the length of the lower side, so that the normal image 40 is displayed on the screen 10 .

When the electronic device 100 is tilted in the z-axis direction and the yaw angle is different from the reference value (eg, 0), the image projected on the screen 10 according to the tilted direction has a trapezoidal shape with a short left or right side. becomes (51, 52). In this case, the processor 170 increases the length of the left side of the image or performs correction to increase the length of the right side, so that the image 50 in a normal form is displayed on the screen 10 .

6 is a diagram for explaining a method of performing keystone correction in more detail. The electronic device 100 may acquire information necessary for keystone correction through an initial setting operation and store it in advance.

Specifically, in a state in which the screen 10 and the electronic device 100 are arranged as shown in FIG. 6, the rotation angle between the electronic device 100 and the screen is 0 and the tilt angle in the gravitational direction is all 0 (ie, When the roll angle, pitch angle, and yaw angle are all 0), when an image having a known resolution is output from the electronic device 100, an image without a keystone phenomenon is displayed on the screen 10. At this time, the electronic device 100 determines the distance ds between the center of the projection lens of the projection unit 110 and the screen, and the four vertices (v1, v2, The three-dimensional coordinates [v _kx , v _ky , v _kz ] ^T of v3, v4) are measured in advance and stored. Since the electronic device 100 has a limited wide-angle lens for projecting light, the maximum displayable image size varies depending on the distance from the screen. In the initial setting operation, the user may adjust the distance so that the image is displayed as large as possible on the screen 10 . In this way, when a setting menu (not shown) is pressed in a state where the distance is adjusted, the sensed distance information and the coordinate information of the four vertexes of the image displayed at that time are stored. In this specification, when an image without a keystone phenomenon is displayed on the screen 10, a range within the coordinates of the four vertexes is referred to as an image display range.

A method of measuring the distance ds to the screen may be measured in various ways depending on the number and positions of the distance sensors. For example, in the case where two distance sensors are disposed on both sides of the projection lens position, the distance ds from the projection lens position to the screen can be calculated by obtaining an average value of distances measured by the two distance sensors. The processor 170 may calculate 3D coordinates of four vertexes reflected on the screen 10 based on the size and distance ds of the image to be projected. where k is the number of each vertex. Even if the distance ds changes after setting and the 3D coordinates of the four vertexes of the quadrangular image 60 change accordingly, they can be newly acquired through measurement and calculation. As shown in FIG. 6 , the standard of 3D coordinates for expressing the four vertices on the screen is based on the coordinate system (X _p , Y _p , Z _p ) centered on the projection lens of the electronic device 100 .

Meanwhile, as described above, the electronic device 100 may be tilted or placed at an angle with respect to the screen depending on the arrangement state, and thus at least one of the pitch angle, the roll angle, and the yaw angle may not be zero. The processor 170 obtains the following rotation matrix R using the pitch angle, roll angle, and yaw angle calculated using the above equations.

<Equation 4>

The processor 170 transforms the distance ds between the center of the projection lens and the projection surface (screen) measured in a state where all attitude angles (pitch angle, roll angle, yaw angle) are 0, using a rotation matrix R. can be calculated

When the processor 170 obtains the pitch angle, roll angle, and yaw angle calculated based on Equations 1 and 2, the distance ds between the center of the lens and the screen, and the three-dimensional coordinates of the four vertices of the square image before rotation, The following equation can be used to calculate the coordinates of four vertices when keystone occurs.

<Equation 5>

In Equation 5, pk means the three-dimensional coordinates [p _kx , p _ky , p _kz ] ^T of the k-th vertex when keystone occurs. In Equation 5, xk, yk, and zk can be calculated using the following equation.

<Equation 6>

In Equation 6, k means the number of each vertex, and the 3-dimensional coordinates [x _k , y _k , z _k ] ^T is the 3-dimensional coordinates [p _kx , p _ky , p _kz ] ^T of the k-th vertex Rotation matrix These are the coordinates after rotating with Rp. Referring to FIG. 6 and Equations 5 and 6, since the distance of the four vertices to the Zp coordinate axis is always ds, it can be seen that pkz=ds.

The rotation matrix Rp used in Equation 6 can be defined as follows.

<Equation 7>

In this state, among the four vertices (μ1, μ2, μ3, μ4) of the image to be projected on the screen 10, the pixel position coordinates [μ kx , μ ky , 1] of the k-th vertex [μ _kx , μ _ky , 1] ^T and the estimated keystone occurrence The positional relationship of the four vertices (p1, p2, p3, and p4) on the screen is defined by the following equation.

<Equation 8>

In Equation 8, P means a projection matrix. The projection matrix P can be modeled by homography, and its shape can be expressed as follows.

<Equation 9>

Here, the parameters a, b, c, d, e, f, g, h are the pixel positions of the four vertices (μ1, μ2, μ3, μ4) of the image to be projected on the image screen 10 in Equation 8, and the keystone It can be obtained by substituting the pixel positions of the four vertices (p1, p2, p3, and p4) on the screen 10 estimated when this occurs.

As a result, the processor 170 may perform keystone correction by converting the position (xp, yp) of each pixel of the original image using the following equation.

<Equation 10>

In Equation 10, P ^-1 is an inverse matrix of the projection matrix. S denotes a matrix composed of a scale parameter s for size change and parameters tx and ty for position movement in the x-axis direction and the y-axis direction. Specifically, S can be expressed as follows.

<Equation 11>

The processor 170 sets the scale parameter s so that the resolution of the image of the original image can be as large as possible while making it a rectangle in which the vertex is located inside the keystone-generated image. In addition, the processor 170 may adjust at least one of tx and ty when a user manipulation for moving the display position of the image is input or a situation in which the display position needs to be moved occurs.

Specifically, the processor 170 calculates four vertices of an image projected on the screen. If the calculated vertex coordinates deviate from the coordinates of the four vertexes of the screen 10, the processor 170 adjusts the scale parameter s to a smaller value so that the keystone-corrected image does not deviate from the screen 10. The processor 170 sets the scale parameter s so that the keystone-corrected image can be maximally displayed within a range that does not deviate from the screen 10 . In the case of the electronic device 100 having an optical zoom function, the coordinates of the four vertexes of the keystone correction image projected on the screen 10 may be obtained using distance and zoom magnification information. In this case, the zoom factor can be adjusted so that the four vertices are optimized for the screen.

7 and 8 are views for explaining the effect of keystone correction. Referring to FIG. 7 , when the electronic device 100 projects a rectangular image frame 70 , it can be seen that the image frame 71 distorted by the keystone effect is displayed on the screen 10 . Here, the image frame 70 is included in light projected from the projection unit 110 . A pixel x1 in the image frame 70 is displayed at a position x2 on the screen determined by the projection matrix.

The processor 170 converts the original image 80 into a corrected image 81 as shown in FIG. 8 in order to correct the keystone effect. The processor 170 controls the projection unit 110 to project the image frame 70 including the corrected image 81 . Accordingly, even if a keystone effect occurs on the screen 10 and the image frame 70 is distorted, the image 82 displayed on the screen becomes a rectangular shape. The location of one pixel (x1) in the original image 80 is located at the point x2 in the corrected image 81 by the projection matrix P and the scaling matrix S, and the image 82 displayed on the screen 10 ), it can be seen that it is located at the x3 point.

As described above, the processor 170 operates the screen in a steady state (that is, a state in which pitch angle, roll angle, and yaw angle are all fixed to 0 or reference values) at the same distance as the coordinates of the four vertices of the image to be displayed on the screen. By comparing the coordinates of the four vertices of the image displayed on , it is possible to identify whether the image to be actually displayed is within a predetermined area of the screen. If the image projected on the screen exceeds a preset area within the screen, the processor 170 may project the image within the preset area by controlling the size of the keystone-corrected image. For example, the preset area may be an area within a size range of the screen.

If the projected image is located within a predetermined area, the processor 170 sets the scale parameter to be maximum within that range. In this case, if there is a user manipulation, the processor 170 may reduce the image size by adjusting the scale parameter.

Information about the distance measured in the normal state, the coordinates of the four vertices of the image displayed on the screen, etc. may be stored in a memory (not shown) included in the electronic device 100 or may be provided from a separate source. Users can also directly input it.

In addition, if size information of the actual screen 10 or coordinate information of each vertex of the screen is stored in advance, the processor 170 may adjust the size of the keystone-corrected image by comparing the information with these information.

In this embodiment, the keystone correction operation and the image size adjustment operation may be collectively processed through one calculation operation, or may be sequentially processed depending on the implementation method. For example, an image size adjustment operation may be additionally performed according to a user's selection after keystone correction is performed.

9 is a flowchart illustrating an image processing method of an electronic device according to an embodiment of the present invention. According to FIG. 9 , the electronic device obtains an angle of rotation of the electronic device based on the direction of gravity and an angle of rotation of the electronic device based on the screen (S910). Specifically, pitch angle, roll angle, yaw angle, etc. are calculated. Since the method for calculating these angles has been specifically described in the above section, redundant description will be omitted.

The electronic device performs keystone correction using the acquired angles and other preliminary information (S920). Specifically, the electronic device estimates the posture of the electronic device using the calculated angles, and uses information such as the previously stored distance to the screen, the coordinates of each vertex of the screen, or the coordinates of each vertex of an image normally displayed on the screen. Thus, the above-described rotation matrix, projection matrix, image transformation matrix, and scale matrix are respectively determined. The electronic device performs keystone correction by applying the determined matrix to the original image.

Also, the electronic device adjusts the size of the image (S930). Specifically, the electronic device obtains the coordinates of each vertex of the image to be displayed on the screen 10 after keystone correction, and compares the coordinates of each vertex of the screen or the coordinates of each vertex of the image normally displayed on the screen. Thus, it is determined whether an image is to be displayed within a preset area of the screen 10. As a result of the determination, if the image exceeds a predetermined area within the screen 10, the size of the image is controlled by adjusting the scale parameter. For example, the electronic device may reduce the size of the image so that the image is displayed within a predetermined area of the screen. On the other hand, if the image is smaller than the maximum displayable size, the scale parameter is adjusted so that it can be displayed in the maximum size.

When the image processing is completed, the electronic device projects light including the corrected image toward the screen (S940).

In FIG. 9, keystone correction and image size adjustment are illustrated and described as being sequentially performed in different steps, but when processing is performed like an operation using the above-described equations, keystone correction and image size adjustment may be performed simultaneously.

Meanwhile, in another embodiment of the present invention, the above-described image processing task is implemented so that the electronic device can be performed in a fixed state.

10 is a flowchart of an image processing method according to another embodiment of the present invention.

According to FIG. 10 , the electronic device acquires sensor values using an acceleration sensor and a distance sensor, respectively (S1010 and S1035). Depending on the design method, sensor value acquisition can be done sequentially or in parallel.

When a sensor signal is output from the acceleration sensor, the electronic device performs filtering to remove noise of the signal (S1015). If the acceleration sensor is 3-axis, the electronic device reads all sensor values of the 3-axis acceleration sensor. In this case, it may be performed after acquiring the acceleration sensor value for a certain period of time or may be performed in real time.

The electronic device checks the sensor value of the acceleration sensor multiple times. Accordingly, it is determined whether the current measurement value is moved by comparing the previous measurement value with the current measurement value. Although not shown in FIG. 10, normalization may be performed on the difference value using the following equation.

<Equation 12>

Here, Abx, Aby, and Abz are the output values of the x, y, and z-axis acceleration sensors, respectively, and t means time.

When the normalized difference value exceeds the threshold value, the electronic device determines that there is motion (S1020). Even if the slope is fixed, the sensor output value may be generated according to changes in the surrounding magnetic field or environment, so the threshold is not necessarily set to 0 and can be set in consideration of a fine error range. If it is determined that there is movement, the electronic device outputs a UI guiding to fix the electronic device (S1025). If the electronic device performing the operation of FIG. 10 has a projection unit, the electronic device may project the UI onto the screen by controlling the projection unit. If an output means such as a general display or a speaker is provided instead of a projector having a projection unit, a UI (or message) for guiding fixing may be provided using such an output means. Meanwhile, when the movement of the electronic device is identified, keystone correction may not be performed until the movement is stopped.

If the user who checks the message fixes the electronic device, the electronic device calculates the pitch angle and the roll angle using the sensor value of the acceleration sensor (S1030). Since the method of calculating the pitch angle and the roll angle has been described above, redundant description will be omitted.

Meanwhile, when sensor values are obtained from a plurality of distance sensors (S1035), the electronic device filters to remove noise (S1040). The electronic device may perform filtering after obtaining a distance value for a certain period of time or may perform filtering in real time.

After acquiring the sensing values of the distance sensor multiple times, the electronic device compares the current measurement value with the previous measurement value to determine whether there is movement (S1045).

In this case, the electronic device may check the difference between the measured values by normalizing the difference in the following equation.

<Equation 13>

Here, d1 is a sensing value of the first distance sensor, d2 is a sensing value of the second distance sensor, and t means time. The electronic device determines that there is motion when the calculated value is out of the threshold. The threshold is not necessarily set to 0, and can be set in consideration of a fine error range.

When it is determined that the sensor value of the distance sensor has changed, the electronic device outputs a UI (or message) for fixing the screen (S1050). As described above, the UI may be projected on a screen, displayed on a display provided in the electronic device itself, or output as a voice message or the like through a speaker.

If it is determined that the motionless state is fixed, the electronic device calculates the yaw angle (S1055). Since the method for calculating the yaw angle has been described above, redundant description will be omitted.

As described above, when all of the pitch angle, roll angle, and yaw angle are calculated, the electronic device performs image processing operations for keystone correction and scale adjustment (S1060).

Specifically, a projection matrix P representing the relationship between points on an imaginary plane without distortion and points on an actual screen is obtained, and an image to be projected on the screen is pre-warped using the obtained projection matrix to display the image on the screen. Obtain an image transformation matrix that outputs a rectangular image as much as possible. At this time, the image conversion matrix is set to minimize the resolution loss and to be the largest possible within the size range of the screen. Since the image processing method using these matrices has been specifically described in the foregoing, redundant description will be omitted.

The electronic device outputs the processed image in the screen direction (S1065).

As described above, according to various embodiments of the present disclosure, since keystone correction can be performed without using an image sensor, a complicated image processing process can be omitted and hardware can be reduced.

11 is a diagram for explaining a keystone correction effect of an electronic device according to an embodiment of the present invention.

According to FIG. 11 , when an original image 1110 is output as it is, a distorted image 1120 is displayed on the screen 10 . The electronic device 100 transforms the original image before projection to generate and output a corrected image 1111 . Accordingly, an undistorted image 1121 is displayed on the screen 10 . The size of the undistorted image 1121 is maximally displayed within the actual expressible range, but when it is out of the size of the screen 10, it is displayed reduced according to the size of the screen 10. As a result, the user can view the normal state image of the maximum size and maximum resolution possible in the current state without any additional manipulation.

The image processing method described in FIGS. 9 and 10 may be performed in an electronic device having the configuration of FIG. 1 , but is not necessarily limited thereto, and may be performed in an electronic device having various configurations. In addition, although the main body of the electronic device 100 is illustrated as having a rectangular shape in FIG. 2 , the appearance of the electronic device 100 may also be implemented in various shapes. Hereinafter, the appearance and configuration of the electronic device 100 transformed into various shapes will be described.

12 is a perspective view illustrating an external appearance of an electronic device 100 according to an embodiment of the present disclosure.

Referring to FIG. 12 , the electronic device 100 may include a head 103, a body 105, a projection lens 111, a connector 109, or a cover 107.

The electronic device 100 may be a device of various types. In particular, the electronic device 100 may be a projector device that enlarges and projects an image onto a wall or a screen, and the projector device may be an LCD projector or a digital light processing (DLP) projector using a digital micromirror device (DMD). .

In addition, the electronic device 100 may be a home or industrial display device, or a lighting device used in daily life, a sound device including a sound module, a portable communication device (eg, a smartphone), It may be implemented as a computer device, a portable multimedia device, a wearable device, or a home appliance. Meanwhile, the electronic device 100 according to an embodiment of the present disclosure is not limited to the above-described devices, and the electronic device 100 may be implemented as an electronic device 100 having two or more functions of the above-described devices. For example, the electronic device 100 may be used as a display device, a lighting device, or a sound device by turning off a projector function and turning on a lighting function or a speaker function according to manipulation of a processor, and AI including a microphone or communication device. Can be used as a speaker.

The main body 105 is a housing forming an exterior, and may support or protect component parts (eg, the components shown in FIG. 13 ) of the electronic device 100 disposed inside the main body 105 . The body 105 may have a structure close to a cylindrical shape as shown in FIG. 12 . However, the shape of the main body 105 is not limited thereto, and according to various embodiments of the present disclosure, the main body 105 may be implemented in various geometric shapes such as a column, a cone, and a sphere having a polygonal cross section.

The size of the main body 105 may be a size that a user can hold or move with one hand, may be implemented in a very small size for easy portability, and may be implemented in a size that can be placed on a table or coupled to a lighting device.

The material of the main body 105 may be implemented with matte metal or synthetic resin so as not to be stained with user's fingerprints or dust, or the exterior of the main body 105 may be made of a smooth gloss.

A friction area may be formed on a portion of the exterior of the body 105 so that a user can grip and move the main body 105 . Alternatively, the main body 105 may be provided with a bent gripping portion or support 108a (see FIG. 14 ) that the user can grip in at least a portion of the area.

The projection lens 111 is formed on one surface of the main body 105 to project light passing through the lens array to the outside of the main body 105 . The projection lens 111 of various embodiments may be an optical lens coated with low dispersion in order to reduce chromatic aberration. The projection lens 111 may be a convex lens or a condensing lens, and the projection lens 111 according to an embodiment may adjust the focus by adjusting the positions of a plurality of sub lenses.

The head 103 is provided to be coupled to one surface of the body 105 to support and protect the projection lens 111 . The head 103 may be coupled to the main body 105 so as to be swivelable within a predetermined angular range based on one surface of the main body 105 .

The head 103 is automatically or manually swiveled by a user or a processor to freely adjust the projection angle of the projection lens 111 . Alternatively, although not shown in the drawing, the head 103 is coupled to the main body 105 and includes a neck extending from the main body 105, so that the head 103 is tilted or tilted to adjust the projection angle of the projection lens 111. can be adjusted

The electronic device 100 can project light or an image to a desired location by adjusting the direction of the head 103 and adjusting the emission angle of the projection lens 111 while the position and angle of the main body 105 are fixed. there is. In addition, the head 103 may include a handle that the user can grasp after rotating in a desired direction.

A plurality of openings may be formed on the outer circumferential surface of the main body 105 . Audio output from the audio output unit may be output to the outside of the main body 105 of the electronic device 100 through the plurality of openings. The audio output unit may include a speaker, and the speaker may be used for general purposes such as multimedia playback, recording playback, and audio output.

According to an embodiment of the present disclosure, a heat dissipation fan (not shown) may be provided inside the main body 105, and when the heat dissipation fan (not shown) is driven, air or heat inside the main body 105 passes through a plurality of openings. can emit. Therefore, the electronic device 100 can discharge heat generated by driving the electronic device 100 to the outside and prevent the electronic device 100 from overheating.

The connector 109 may connect the electronic device 100 to an external device to transmit/receive electrical signals or receive power from the outside. According to an embodiment of the present disclosure, the connector 109 may be physically connected to an external device. At this time, the connector 109 may include an input/output interface, and may communicate with an external device through wired or wireless communication or receive power. For example, the connector 109 may include an HDMI connection terminal, a USB connection terminal, an SD card receiving groove, an audio connection terminal, or a power outlet, or Bluetooth, Wi-Fi, or wireless connection to an external device wirelessly. A charging connection module may be included.

In addition, the connector 109 may have a socket structure connected to an external lighting device, and may be connected to a socket receiving groove of the external lighting device to receive power. The size and standard of the socket-structured connector 109 may be variously implemented in consideration of a receiving structure of a coupleable external device. For example, according to the international standard E26, the diameter of the junction of the connector 109 may be implemented as 26 mm, and in this case, the electronic device 100 replaces a conventionally used light bulb and uses an external lighting device such as a stand. can be coupled to Meanwhile, when fastened to a socket located on an existing ceiling, the electronic device 100 is projected from top to bottom, and when the electronic device 100 is not rotated by coupling the socket, the screen cannot be rotated either. Accordingly, the electronic device 100 is socket-coupled to the ceiling stand so that the electronic device 100 can rotate even when the socket is coupled and power is supplied, and the head 103 is moved from one side of the main body 105. It swivels and adjusts the emission angle to project the screen to a desired position or rotate the screen.

The connector 109 may include a coupling sensor, and the coupling sensor may sense whether or not the connector 109 is coupled with an external device, a coupling state, or a coupling target, and transmit the sensor to the processor, based on the received detection value. Driving of the electronic device 100 may be controlled.

The cover 107 can be coupled to and separated from the main body 105 and can protect the connector 109 so that the connector 109 is not constantly exposed to the outside. The shape of the cover 107 may have a shape continuous with the body 105 as shown in FIG. 1, or may be implemented to correspond to the shape of the connector 109. The cover 107 can support the electronic device 100, and the electronic device 100 can be used by being coupled to the cover 107 and coupled to or mounted on an external cradle.

In the electronic device 100 according to various embodiments, a battery may be provided inside the cover 107 . A battery may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell or a fuel cell.

Although not shown in the drawings, the electronic device 100 may include a camera module, and the camera module may capture still images and moving images. According to one embodiment, the camera module may include one or more lenses, image sensors, image signal processors, or flashes.

Although not shown in the drawings, the electronic device 100 may include a protective case (not shown) to protect the electronic device 100 and easily carry it, or a stand supporting or fixing the main body 105. (not shown), and may include a bracket (not shown) coupled to a wall or partition.

In addition, the electronic device 100 may provide various functions by being connected to various external devices using a socket structure. As an example, the electronic device 100 may be connected to an external camera device using a socket structure. The electronic device 100 may use the projection unit 110 to provide an image stored in a connected camera device or an image currently being captured. As another embodiment, the electronic device 100 may be connected to a battery module to receive power using a socket structure. Meanwhile, the electronic device 100 may be connected to an external device using a socket structure, but this is merely an example, and may be connected to an external device using another interface (eg, USB, etc.).

13 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 13 , the electronic device 100 includes a projection unit 110, a memory 120, a user interface 130, an input/output interface 140, an audio output unit 150, a power supply unit 160, and a processor 170. ) may be included. Meanwhile, the configuration shown in FIG. 13 is merely an embodiment, and some configurations may be omitted and new configurations may be added.

The projection unit 110 is a component that projects an image to the outside. According to an embodiment of the present disclosure, the projection unit 110 may use various projection methods (eg, a cathode-ray tube (CRT) method, a liquid crystal display (LCD) method, a digital light processing (DLP) method, and a laser method). etc.) can be implemented. As an example, the principle of the CRT method is basically the same as that of a CRT monitor. The CRT method enlarges the image with a lens in front of the cathode ray tube (CRT) and displays the image on the screen. Depending on the number of cathode ray tubes, it is divided into a 1-tube type and a 3-tube type, and in the case of a 3-tube type, red, green, and blue cathode ray tubes can be separately implemented.

As another example, the LCD method is a method of displaying an image by transmitting light from a light source through a liquid crystal. The LCD method is divided into a single-panel type and a three-panel type. In the case of the three-panel type, the light from the light source is separated into red, green, and blue by a dichroic mirror (a mirror that reflects only light of a specific color and passes the rest) and transmits the liquid crystal. After that, the light can gather in one place again.

As another example, the DLP method is a method of displaying an image using a DMD (Digital Micromirror Device) chip. The projection unit of the DLP method may include a light source, a color wheel, a DMD chip, a projection lens, and the like. Light output from a light source may exhibit a color while passing through a rotating color wheel. The light that passed through the color wheel is input to the DMD chip. The DMD chip includes numerous micromirrors and reflects light input to the DMD chip. The projection lens may play a role of enlarging light reflected from the DMD chip to an image size.

As another example, the laser method includes a diode pumped solid state (DPSS) laser and a galvanometer. For lasers that output various colors, three DPSS lasers are installed for each RGB color, and then lasers with optical axes overlapped using special mirrors are used. The galvanometer includes a mirror and a high power motor to move the mirror at high speed. For example, a galvanometer can rotate a mirror at up to 40 KHz/sec. The galvanometer is mounted according to the scanning direction. In general, since the projector scans in a plane, the galvanometer can also be arranged separately in the x and y axes.

Meanwhile, the projection unit 110 may include various types of light sources. For example, the projection unit 110 may include at least one light source among a lamp, LED, and laser.

The projection unit 110 may output an image in a 4:3 aspect ratio, a 5:4 aspect ratio, or a 16:9 wide aspect ratio according to the purpose of the electronic device 100 or the user's settings, and depending on the aspect ratio, WVGA (854*480 ), SVGA(800*600), XGA(1024*768), WXGA(1180*720), WXGA(1180*800), SXGA(1180*1024), UXGA(1600*1100), Full HD(1920*1080) ), etc., can output images at various resolutions.

Meanwhile, the projection unit 110 may perform various functions for adjusting an output image under the control of the processor 170 . For example, the projection unit 110 may perform functions such as zoom, keystone, quick corner (4 corner) keystone, and lens shift.

Specifically, the projection unit 110 may enlarge or reduce the image according to the distance from the screen (projection distance). That is, a zoom function may be performed according to the distance from the screen. In this case, the zoom function may include a hardware method of adjusting the screen size by moving a lens and a software method of adjusting the screen size by cropping an image. Meanwhile, when the zoom function is performed, the focus of the image needs to be adjusted. For example, methods for adjusting the focus include a manual focus method and a motorized method. The manual focus method refers to a method of manually focusing, and the motorized method refers to a method of automatically focusing using a motor built into the projector when a zoom function is performed. When performing the zoom function, the projection unit 110 may provide a digital zoom function through software, and may provide an optical zoom function that performs a zoom function by moving a lens through a driving unit.

Also, the projection unit 110 may perform a keystone function. If the height is not right for the front projection, the screen may be distorted up or down. The keystone function means a function of correcting a distorted screen. For example, if distortion occurs in the left and right directions of the screen, it can be corrected using the horizontal keystone, and if distortion occurs in the vertical direction, it can be corrected using the vertical keystone. The quick corner (4 corner) keystone function corrects the screen when the central area of the screen is normal but the corner area is not balanced. The lens shift function is a function that moves the screen as it is when the screen is out of the screen.

Meanwhile, the projection unit 110 may provide zoom/keystone/focus functions by automatically analyzing the surrounding environment and the projection environment without user input. Specifically, the projection unit 110 determines the distance between the screen and the electronic device 100 detected through sensors (depth camera, distance sensor, infrared sensor, illuminance sensor, etc.) and the space where the electronic device 100 is currently located. Zoom/Keystone/Focus functions can be automatically provided based on information about the image and the amount of ambient light.

Also, the projection unit 110 may provide a lighting function using a light source. In particular, the projection unit 110 may provide a lighting function by outputting a light source using LEDs. According to one embodiment, the projection unit 110 may include one LED, and according to another embodiment, the electronic device may include a plurality of LEDs. Meanwhile, the projection unit 110 may output a light source using a surface-emitting LED according to an implementation example. Here, the surface-emitting LED may refer to an LED having a structure in which an optical sheet is disposed above the LED so that light sources are uniformly distributed and output. Specifically, when the light source is output through the LED, the light source may be evenly dispersed through the optical sheet, and the light source dispersed through the optical sheet may be incident to the display panel.

Meanwhile, the projection unit 110 may provide a user with a dimming function for adjusting the intensity of the light source. Specifically, when a user input for adjusting the intensity of a light source is received from a user through the user interface 130 (eg, a touch display button or a dial), the projection unit 110 displays a light source corresponding to the received user input. It is possible to control the LED to output the intensity of.

Also, the projection unit 110 may provide a dimming function based on the content analyzed by the processor 170 without user input. In detail, the projection unit 110 may control the LED to output the intensity of the light source based on information about currently provided content (eg, content type, content brightness, etc.).

Meanwhile, the projection unit 110 may control the color temperature under the control of the processor 170 . Here, the processor 170 may control the color temperature based on content. Specifically, if the content is identified as being output, the processor 170 may obtain color information for each frame of the content for which output is determined. Also, the processor 170 may control the color temperature based on the obtained color information for each frame. Here, the processor 170 may obtain at least one primary color of a frame based on color information for each frame. Also, the processor 170 may adjust the color temperature based on at least one or more acquired primary colors. For example, the color temperature controllable by the processor 170 may be classified into a warm type or a cold type. Here, it is assumed that a frame to be output (hereinafter referred to as an output frame) includes a scene in which a fire has occurred. The processor 170 may identify (or obtain) that the primary color is red based on color information included in the current output frame. Also, the processor 170 may identify a color temperature corresponding to the identified main color (red). Here, the color temperature corresponding to red may be a warm type. Meanwhile, the processor 170 may use an artificial intelligence model to obtain color information or a primary color of a frame. According to an embodiment, the artificial intelligence model may be stored in the electronic device 100 (eg, the memory 120). According to another embodiment, the artificial intelligence model may be stored in an external server capable of communicating with the electronic device 100. can be stored in

Meanwhile, the electronic device 100 may control a lighting function in conjunction with an external device. Specifically, the electronic device 100 may receive lighting information from an external device. Here, the lighting information may include at least one of brightness information and color temperature information set by an external device. Here, the external device is a device connected to the same network as the electronic device 100 (eg, an IoT device included in the same home/work network) or a device that is not on the same network as the electronic device 100 but can communicate with the electronic device ( For example, a remote control server). For example, it is assumed that an external lighting device (IoT device) included in the same network as the electronic device 100 outputs red light with a brightness of 50. The external lighting device (IoT device) may directly or indirectly transmit lighting information (eg, information indicating that red light is output with a brightness of 50) to the electronic device 100 . Here, the electronic device 100 may control the output of the light source based on lighting information received from an external lighting device. For example, when lighting information received from an external lighting device includes information for outputting red light with a brightness of 50, the electronic device 100 may output red light with a brightness of 50.

Meanwhile, the electronic device 100 may control a lighting function based on biometric information. Specifically, the processor 170 may obtain user's biometric information. Here, the biometric information may include at least one of the user's body temperature, heart rate, blood pressure, respiration, and electrocardiogram. Here, the biometric information may include various types of information in addition to the information described above. For example, an electronic device may include a sensor for measuring biometric information. The processor 170 may obtain user's biometric information through a sensor and control the output of the light source based on the obtained biometric information. As another example, the processor 170 may receive biometric information from an external device through the input/output interface 140 . Here, the external device may refer to a user's portable communication device (eg, a smart phone or a wearable device). The processor 170 may obtain user's biometric information from an external device and control the output of the light source based on the obtained biometric information. Meanwhile, according to an implementation example, the electronic device may identify whether the user is sleeping, and if the user is identified as sleeping (or preparing for sleep), the processor 170 determines the light source based on the user's biometric information. You can control the output.

The memory 120 may store at least one command related to the electronic device 100 . Also, an operating system (O/S) for driving the electronic device 100 may be stored in the memory 120 . Also, various software programs or applications for operating the electronic device 100 may be stored in the memory 120 according to various embodiments of the present disclosure. Also, the memory 120 may include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.

Specifically, various software modules for operating the electronic device 100 may be stored in the memory 120 according to various embodiments of the present disclosure, and the processor 170 executes various software modules stored in the memory 120. Thus, the operation of the electronic device 100 may be controlled. That is, the memory 120 is accessed by the processor 170, and data can be read/written/modified/deleted/updated by the processor 170.

Meanwhile, in the present disclosure, the term memory 120 refers to the memory 120, a ROM (not shown) in the processor 170, a RAM (not shown), or a memory card (not shown) mounted in the electronic device 100 (eg For example, micro SD card, memory stick) may be used as a meaning including.

User interface 130 may include various types of input devices. For example, the user interface 130 may include physical buttons. In this case, the physical button may include a function key, a direction key (eg, a 4-direction key), or a dial button. According to one embodiment, the physical button may be implemented as a plurality of keys. According to another embodiment, the physical button may be implemented as one key. Here, when the physical button is implemented as one key, the electronic device 100 may receive a user input in which one key is pressed for a critical period of time or more. When a user input in which one key is pressed for a critical period of time or longer is received, the processor 170 may perform a function corresponding to the user input. For example, the processor 170 may provide a lighting function based on user input.

Also, the user interface 130 may receive a user input using a non-contact method. When user input is received through the contact method, physical force must be transmitted to the electronic device. Therefore, a method for controlling the electronic device regardless of physical force may be required. Specifically, the user interface 130 may receive a user gesture and perform an operation corresponding to the received user gesture. Here, the user interface 130 may receive a user's gesture through a sensor (eg, an image sensor or an infrared sensor).

Also, the user interface 130 may receive a user input using a touch method. For example, the user interface 130 may receive a user input through a touch sensor. According to an embodiment, the touch method may be implemented as a non-contact method. For example, the touch sensor may determine whether the user's body has approached within a critical distance. Here, the touch sensor may identify a user input even when the user does not contact the touch sensor. Meanwhile, according to another implementation example, the touch sensor may identify a user input in which a user contacts the touch sensor.

Meanwhile, the electronic device 100 may receive user input in various ways other than the above-described user interface. As an example, the electronic device 100 may receive a user input through an external remote control device. Here, the external remote control device may be a remote control device corresponding to the electronic device 100 (eg, an electronic device-specific control device) or a user's portable communication device (eg, a smartphone or a wearable device). Here, the user's portable communication device may store an application for controlling the electronic device. The portable communication device may obtain a user input through a stored application and transmit the acquired user input to the electronic device 100 . The electronic device 100 may receive a user input from a portable communication device and perform an operation corresponding to a user's control command.

Meanwhile, the electronic device 100 may receive a user input using voice recognition. According to an embodiment, the electronic device 100 may receive a user's voice through a microphone included in the electronic device. According to another embodiment, the electronic device 100 may receive a user's voice from a microphone or an external device. Specifically, the external device may acquire a user voice through a microphone of the external device and transmit the obtained user voice to the electronic device 100 . The user's voice transmitted from the external device may be audio data or digital data obtained by converting the audio data (eg, audio data converted into a frequency domain). Here, the electronic device 100 may perform an operation corresponding to the received user voice. Specifically, the electronic device 100 may receive audio data corresponding to a user's voice through a microphone. And, the electronic device 100 may convert the received audio data into digital data. In addition, the electronic device 100 may convert the converted digital data into text data using a speech to text (STT) function. According to an embodiment, the STT (Speech To Text) function may be performed directly in the electronic device 100,

According to another embodiment, a speech to text (STT) function may be performed in an external server. The electronic device 100 may transmit digital data to an external server. The external server may convert digital data into text data and obtain control command data based on the converted text data. The external server may transmit control command data (this time, text data may also be included) to the electronic device 100 . The electronic device 100 may perform an operation corresponding to the user's voice based on the obtained control command data.

Meanwhile, the electronic device 100 may provide a voice recognition function using one assistant (or artificial intelligence assistant, eg, Bixby™, etc.), but this is only an example and through a plurality of assistants. A voice recognition function may be provided. In this case, the electronic device 100 may provide a voice recognition function by selecting one of a plurality of assists based on a trigger word corresponding to the assist or a specific key present on the remote control.

Meanwhile, the electronic device 100 may receive a user input using screen interaction. Screen interaction may refer to a function of identifying whether a predetermined event occurs through an image projected on a screen (or a projection surface) by an electronic device and acquiring a user input based on the predetermined event. Here, the predetermined event may refer to an event in which a predetermined object is identified at a specific location (eg, a location where a UI for receiving a user input is projected). Here, the predetermined object may include at least one of a user's body part (eg, a finger), a pointing stick, and a laser point. When a predetermined object is identified at a location corresponding to the projected UI, the electronic device 100 may identify that a user input for selecting the projected UI has been received. For example, the electronic device 100 may project a guide image to display a UI on the screen. And, the electronic device 100 can identify whether the user selects the projected UI. Specifically, the electronic device 100 may identify that the user has selected the projected UI when a predetermined event is identified at the location of the projected UI. Here, the projected UI may include at least one or more items. Here, the electronic device 100 may perform spatial analysis to identify whether a predetermined event is located at the location of the projected UI. Here, the electronic device 100 may perform spatial analysis through a sensor (eg, an image sensor, an infrared sensor, a depth camera, a distance sensor, etc.). The electronic device 100 may identify whether a predetermined event occurs at a specific location (the location where the UI is projected) by performing spatial analysis. And, if it is identified that a predetermined event occurs at a specific location (the location where the UI is projected), the electronic device 100 may identify that a user input for selecting a UI corresponding to the specific location has been received.

The input/output interface 140 is a component for inputting/outputting at least one of an audio signal and a video signal. The input/output interface 140 may receive at least one of audio and video signals from an external device and output a control command to the external device.

Meanwhile, in an embodiment of the present disclosure, the input/output interface 140 includes HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), USB C-type, DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (Dsubminiature) and DVI (Digital Visual Interface) may be implemented as at least one wired input/output interface. According to an embodiment, the wired input/output interface may be implemented as an interface for inputting/outputting only audio signals and an interface for inputting/outputting only video signals, or may be implemented as one interface for inputting/outputting both audio and video signals.

In addition, the electronic device 100 may receive data through a wired input/output interface, but this is merely an example, and power may be supplied through the wired input/output interface. For example, the electronic device 100 may receive power from an external battery through USB C-type or from an outlet through a power adapter. As another example, an electronic device may receive power from an external device (eg, a laptop computer or a monitor) through a DP.

Meanwhile, in an embodiment of the present disclosure, the input/output interface 140 is Wi-Fi, Wi-Fi Direct, Bluetooth, Zigbee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), and Long Term Evolution (LTE) communication It may be implemented as a wireless input/output interface that performs communication using at least one of the communication methods. Depending on the implementation example, the wireless input/output interface may be implemented as an interface for inputting/outputting only audio signals and an interface for inputting/outputting only video signals, or may be implemented as one interface for inputting/outputting both audio and video signals.

Also, an audio signal may be input through a wired input/output interface, and a video signal may be input through a wireless input/output interface. Alternatively, an audio signal may be input through a wireless input/output interface and a video signal may be input through a wired input/output interface.

The audio output unit 150 is a component that outputs an audio signal. In particular, the audio output unit 150 may include an audio output mixer, an audio signal processor, and a sound output module. The audio output mixer may synthesize a plurality of audio signals to be output into at least one audio signal. For example, the audio output mixer may combine an analog audio signal and another analog audio signal (eg, an analog audio signal received from the outside) into at least one analog audio signal. The sound output module may include a speaker or an output terminal. According to an embodiment, the sound output module may include a plurality of speakers, and in this case, the sound output module may be disposed inside the main body, and the sound emitted by covering at least a part of the diaphragm of the sound output module may be emitted through a sound conduit ( waveguide) and can be transmitted to the outside of the main body. The sound output module includes a plurality of sound output units, and since the plurality of sound output units are symmetrically disposed on the exterior of the main body, sound can be emitted in all directions, that is, in all directions of 360 degrees.

The power supply unit 160 may receive power from the outside and supply power to various components of the electronic device 100 . The power supply unit 160 according to an embodiment of the present disclosure may receive power through various methods. As an example, the power supply unit 160 may receive power using the connector 109 shown in FIG. 1 . In addition, the power supply unit 160 may receive power using a 220V DC power cord. However, the present invention is not limited thereto, and the electronic device may receive power using a USB power cord or a wireless charging method.

In addition, the power supply unit 160 may receive power using an internal battery or an external battery. The power supply unit 160 according to an embodiment of the present disclosure may receive power through an internal battery. For example, the power supply unit 160 may charge power of an internal battery using at least one of a 220V DC power cord, a USB power cord, and a USB C-Type power cord, and may receive power through the charged internal battery. . In addition, the power supply unit 160 according to an embodiment of the present disclosure may receive power through an external battery. For example, when the electronic device and the external battery are connected through various wired communication methods such as a USB power cord, a USB C-Type power cord, and a socket home, the power supply unit 160 may receive power through the external battery. That is, the power supply unit 160 may directly receive power from an external battery, or may charge an internal battery through an external battery and receive power from the charged internal battery.

The power supply unit 160 according to the present disclosure may receive power using at least one of the plurality of power supply methods described above.

Meanwhile, with respect to power consumption, the electronic device 100 may have power consumption equal to or less than a preset value (eg, 43W) due to a socket type and other standards. In this case, the electronic device 100 may vary power consumption to reduce power consumption when using a battery. That is, the electronic device 100 may vary power consumption based on a power supply method and power usage.

Meanwhile, the electronic device 100 according to an embodiment of the present disclosure may provide various smart functions.

Specifically, the electronic device 100 is connected to a portable terminal device for controlling the electronic device 100, and a screen output from the electronic device 100 can be controlled through a user input input from the portable terminal device. For example, the mobile terminal device may be implemented as a smart phone including a touch display, and the electronic device 100 receives and outputs screen data provided by the mobile terminal device from the mobile terminal device, and inputs data from the mobile terminal device. A screen output from the electronic device 100 may be controlled according to a user input.

The electronic device 100 may share content or music provided by the portable terminal device by connecting to the portable terminal device through various communication methods such as Miracast, Airplay, wireless DEX, and Remote PC.

Also, the mobile terminal device and the electronic device 100 may be connected through various connection methods. As an embodiment, the portable terminal device may perform a wireless connection by searching for the electronic device 100 or the electronic device 100 may search for the portable terminal device and perform a wireless connection. And, the electronic device 100 may output content provided by the portable terminal device.

As an embodiment, when a predetermined gesture is detected through the display of the portable terminal device after placing the portable terminal device near the electronic device while specific content or music is being output (eg, motion tap view), the electronic device 100 may output content or music currently being output on the portable terminal device.

In an embodiment, while specific content or music is being output from the portable terminal device, the portable terminal device is brought closer to the electronic device 100 by a predetermined distance or less (eg, non-contact tap view), or the portable terminal device is close to the electronic device 100. When contact is made twice at a short interval (eg, contact tap view), the electronic device 100 may output content or music currently being output on the portable terminal device.

In the above-described embodiment, it has been described that the same screen as that provided by the portable terminal device is provided by the electronic device 100, but the present disclosure is not limited thereto. That is, when a connection is established between the portable terminal device and the electronic device 100, the portable terminal device outputs a first screen provided by the portable terminal device, and in the electronic device 100, the first screen is provided by a different portable terminal device. A second screen may be output. For example, the first screen may be a screen provided by a first application installed on the portable terminal device, and the second screen may be a screen provided by a second application installed on the portable terminal device. For example, the first screen and the second screen may be different screens provided by one application installed in the portable terminal device. Also, as an example, the first screen may be a screen including a remote control type UI for controlling the second screen.

The electronic device 100 according to the present disclosure may output a standby screen. For example, when the electronic device 100 is not connected to an external device or when there is no input received from the external device for a preset time, the electronic device 100 may output a standby screen. Conditions for the electronic device 100 to output the standby screen are not limited to the above examples, and the standby screen may be output under various conditions.

The electronic device 100 may output a standby screen in the form of a blue screen, but the present disclosure is not limited thereto. For example, the electronic device 100 may obtain an irregular object by extracting only the shape of a specific object from data received from an external device, and output an idle screen including the obtained irregular object.

14 is a perspective view illustrating an external appearance of an electronic device 100 according to other embodiments of the present disclosure.

Referring to FIG. 14 , the electronic device 100 may include a support (or referred to as a “handle”) 108a.

The support 108a of various embodiments may be a handle or a hook provided for a user to grip or move the electronic device 100, or the support 108a may be a main body ( 105) may be a stand supporting the.

As shown in FIG. 14, the support 108a may be coupled to or separated from the outer circumferential surface of the main body 105 in a hinge structure, and may be selectively separated and fixed from the outer circumferential surface of the main body 105 according to the user's needs. The number, shape, or arrangement of the supports 108a may be variously implemented without limitation. Although not shown in the drawing, the support 108a is built into the main body 105 and can be taken out and used by the user as needed, or the support 108a can be implemented as a separate accessory and detachable from the electronic device 100. there is.

The support 108a may include a first support surface 108a-1 and a second support surface 108a-2. The first support surface 108a-1 may be a surface facing the outside of the body 105 in a state where the support 108a is separated from the outer circumferential surface of the body 105, and the second support surface 108a-2 is the support (108a) may be a surface facing the inner direction of the main body 105 in a state separated from the outer circumferential surface of the main body 105.

The first support surface 108a-1 is developed from the lower part of the body 105 to the upper part of the body 105 and may move away from the body 105, and the first support surface 108a-1 is flat or uniformly curved. can have a shape. The first support surface 108a-1 is the case where the electronic device 100 is mounted so that the outer surface of the body 105 touches the bottom surface, that is, when the projection lens 110 is disposed facing the front direction, the body ( 105) can be supported. In an embodiment including two or more supports 108a, the angle of exit of the head 103 and the projection lens 110 may be adjusted by adjusting the distance between the two supports 108a or the hinge opening angle.

The second support surface 108a-2 is a surface that comes into contact with the user or an external mounting structure when the support 108a is supported by the user or an external mounting structure, and prevents the user from slipping when the electronic device 100 is supported or moved. It may have a shape corresponding to the gripping structure of the hand or the external mounting structure. The user may direct the projection lens 110 toward the front, fix the head 103, hold the support 108a, move the electronic device 100, and use the electronic device 100 like a flashlight.

The support groove 104 is provided on the main body 105 and is a groove structure that can be accommodated when the support 108a is not in use, and as shown in FIG. It can be implemented as a home structure. Through the support groove 104, the support 108a can be stored on the outer circumferential surface of the main body 105 when the support 108a is not in use, and the outer circumferential surface of the main body 105 can be kept smooth.

Alternatively, the support 108a may have a structure in which the support 108a is stored inside the body 105 and the support 108a is pulled out of the body 105 in a situation where the support 108a is needed. In this case, the support groove 104 may have a structure drawn into the main body 105 to accommodate the support rod 108a, and the second support surface 108a-2 may be in close contact with the outer circumferential surface of the main body 105 or a separate support rod. A door (not shown) that opens and closes the groove 104 may be included.

Although not shown in the drawings, the electronic device 100 may include various types of accessories that help use or store the electronic device 100. For example, the electronic device 100 may include the electronic device 100 A protective case (not shown) may be included to protect and easily transport the electronic device 100 by being coupled to a tripod (not shown) or an external surface that supports or fixes the main body 105. Possible brackets (not shown) may be included.

15 is a perspective view illustrating an external appearance of an electronic device 100 according to another embodiment of the present disclosure.

Referring to FIG. 15 , the electronic device 100 may include a support (or “handle”) 108b.

The support 108b of various embodiments may be a handle or a hook provided for a user to grip or move the electronic device 100, or the support 108b may be a main body ( 105) may be a stand that supports it so that it can be directed at an arbitrary angle.

Specifically, as shown in FIG. 15, the support 108b may be connected to the main body 105 at a predetermined point (eg, 2/3 to 3/4 of the height of the main body) of the main body 105. . When the support 108 is rotated in the direction of the main body, the main body 105 can be supported at an arbitrary angle in a state where the main body 105 is laid down in the lateral direction.

16 is a perspective view illustrating an external appearance of an electronic device 100 according to another embodiment of the present disclosure.

Referring to FIG. 16 , the electronic device 100 may include a support (or referred to as “support”) 108c. The support 108c of various embodiments includes a base plate 108c-1 prepared to support the electronic device 100 on the ground and two support members 108c connecting the base plate 108c-1 and the main body 105. -2) may be included.

In one embodiment of the present disclosure, the two support members (108c-2) have the same height, so that one end surface of the two support members (108c-2) is provided on one outer circumferential surface of the main body 105 and the hinge member (108c). -3) can be combined or separated.

The two supporting members may be hingedly connected to the main body 105 at a predetermined point (eg, 1/3 to 2/4 of the height of the main body) of the main body 105 .

When the two support members and the main body are coupled by the hinge member 108c-3, the main body 105 is rotated based on the imaginary horizontal axis formed by the two hinge members 108c-3 so that the projection lens 110 The emission angle of can be adjusted.

Although FIG. 16 shows an embodiment in which two support members 108c-2 are connected to the body 105, the present disclosure is not limited thereto, and one support member and the body as shown in FIGS. 17A and 17B ( 105) may be connected by one hinge member.

17A is a perspective view illustrating an external appearance of an electronic device 100 according to another embodiment of the present disclosure.

FIG. 17B is a perspective view illustrating a rotated state of the electronic device 100 of FIG. 17A.

17A and 17B, the support 108d according to various embodiments includes a base plate 108d-1, a base plate 108-c, and a main body 105 provided to support the electronic device 100 on the ground. It may include one support member (108d-2) connecting the.

Also, the end face of one support member 108d-2 may be coupled or separated by a groove provided on one outer circumferential surface of the body 105 and a hinge member (not shown).

When one support member (108d-2) and the main body 105 are coupled by one hinge member (not shown), as shown in FIG. 17B, based on a virtual horizontal axis formed by one hinge member (not shown) The body 105 can be rotated.

Meanwhile, the supports shown in FIGS. 14, 15, 16, 17a, and 17b are merely examples, and the electronic device 100 may have supports in various positions or shapes.

As described in FIGS. 12 to 17B , when the electronic device is manufactured in a cylindrical shape, the distance sensor may be disposed in a shape surrounding the periphery of the project lens 111 . In this case, since the distance values measured by the plurality of distance sensors may all vary according to the tilt or twist of the electronic device, the distance from the center of the projector lens 111 to the screen can be calculated by calculating the average value of these distance values. can In addition, when the location of each distance sensor, the distance between the sensors, and the sensing values of each sensor are comprehensively considered, the gradient may be estimated therefrom. The processor may perform keystone correction using the estimated tilt, distance, and the like.

According to various embodiments as described above, the electronic device may estimate the posture of the electronic device, correct the image accordingly, and output the corrected image even without an external device such as a camera or an image sensor attached to the electronic device. Accordingly, an undistorted image can be provided with the maximum size and resolution.

When a program code for performing the image processing method according to various embodiments as described above is installed in an electronic device, the electronic device may execute the program code to perform the above-described method. For example, obtaining an angle at which the electronic device is rotated in the direction of gravity and an angle at which the electronic device is distorted with respect to the screen, respectively, performing keystone correction on an image based on the obtained angles, and performing keystone correction on an image to be displayed on the screen If the size exceeds the display range of the image within the screen, a program code for sequentially performing the steps of adjusting the size of the keystone-corrected image and projecting the scaled image in the direction of the screen through the projection unit is installed in the electronic device. If so, the electronic device may perform an accident report operation.

This program code may be composed of one application and distributed online or in a state loaded on a recording medium, or may be distributed in a state loaded in an electronic device in the form of firmware. An electronic device in which such an application is installed may perform various image processing methods described above.

In addition, various embodiments as described above may be implemented in one device as a whole or in combination for each embodiment, or may be implemented individually for each embodiment.

Such program code may be distributed in a state recorded on various types of computer readable media such as ROM, RAM, memory chip, memory card, external hard drive, hard drive, CD, DVD, magnetic disk, or magnetic tape. A device that downloads such a program code online can also perform the various operations described above.

Although the present invention has been described with reference to the accompanying drawings, the scope of the present invention is determined by the claims to be described later, and should not be construed as being limited to the foregoing embodiments and/or drawings. In addition, those skilled in the art will clearly understand that the present invention can be variously improved, changed, and modified without departing from the spirit of the present invention described in the claims.

110: projection unit 170: processor
180: sensor unit 181: acceleration sensor
182-1 to 182-n: distance sensor

Claims (15)

In electronic devices,
projection unit;
a sensor unit including an acceleration sensor and a plurality of distance sensors; and
Including; processor;
The acceleration sensor detects an angle at which the electronic device is rotated in the direction of gravity,
The plurality of distance sensors detect rotational angles of the electronic device with respect to the screen, and the processor comprises:
Keystone correction is performed on an image based on the sensing value obtained by the sensor unit, and the projection unit controls the projection unit to project the keystone-corrected image on the screen, and the image projected on the screen is displayed on the screen. An electronic device that controls the size of an image on which the keystone correction is performed when it exceeds a predetermined area. According to claim 1,
Further comprising a memory storing information about the image display range and the distance to the screen,
the processor,
The electronic device stores the distance obtained from the sensor unit to the screen in the memory, and recognizes the predetermined area based on coordinates of a plurality of vertices of an image to be displayed on the screen. According to claim 2,
the processor,
The electronic device controls the size of the keystone-corrected image to be displayed in a maximum size within the preset region when the coordinates of four vertices of the image to be displayed on the screen are all within the image display range. According to any one of claims 1 to 3,
the processor,
Controlling the projection unit to project a UI for guiding fixing of the electronic device when motion of the electronic device is identified based on a comparison result of a plurality of sensing values obtained by the acceleration sensor. According to any one of claims 1 to 3,
the processor,
Controlling the projection unit to project a UI for guiding fixing of the electronic device when motion of the electronic device is identified based on a comparison result of a plurality of sensing values obtained by the plurality of distance sensors. According to claim 5,
the processor,
If the movement of the electronic device is identified, the electronic device does not perform the keystone correction. According to any one of claims 1 to 3,
The plurality of distance sensors are disposed on both sides of the projection unit,
The processor calculates an angle at which the electronic device is rotated with respect to the screen based on a result value sensed by each of the plurality of distance sensors and a distance between the plurality of distance sensors. In the image processing method of an electronic device including a projection unit,
acquiring an angle at which the electronic device is rotated in the direction of gravity and an angle at which the electronic device is rotated with respect to the screen;
performing keystone correction on an image based on the acquired sensing values;
controlling the size of the keystone-corrected image when the image projected on the screen exceeds a predetermined area within the screen; and
and projecting the size-controlled image on the screen through the projection unit. According to claim 8,
Detecting a distance to the screen and recognizing the predetermined region based on coordinates of a plurality of vertices of an image to be displayed on the screen. According to claim 9,
In the step of controlling the size of the image on which the keystone correction is performed,
and controlling the size of the keystone-corrected image so that it is displayed in a maximum size within the preset region when the coordinates of the four vertices of the image to be displayed on the screen are all within the image display range. According to any one of claims 8 to 10,
obtaining a sensing value of an acceleration sensor provided in the electronic device a plurality of times;
identifying motion of the electronic device based on a comparison result of the obtained plurality of sensing values; and
If the motion of the electronic device is identified, projecting a UI for guiding fixing of the electronic device; According to any one of claims 8 to 10,
acquiring sensing values of a plurality of distance sensors provided in the electronic device a plurality of times;
identifying motion of the electronic device based on a comparison result of the obtained plurality of sensing values; and
If the motion of the electronic device is identified, projecting a UI for guiding fixing of the electronic device; According to claim 12,
The image processing method further comprising not performing the keystone correction when the movement of the electronic device is identified. According to any one of claims 8 to 10,
Calculating an angle at which the electronic device is rotated with respect to the screen based on result values sensed by a plurality of distance sensors disposed on both sides of the projection unit in the electronic device; further comprising, image processing method. A non-transitory computer-readable storage medium in which a program for performing an image processing method of an electronic device including a projection unit is recorded,
acquiring an angle at which the electronic device is rotated in the direction of gravity and an angle at which the electronic device is rotated with respect to the screen;
performing keystone correction on an image based on the acquired sensing values;
controlling the size of the keystone-corrected image when the image projected on the screen exceeds a predetermined area within the screen; and
A non-transitory computer-readable storage medium having a program recorded thereon for performing an image processing method comprising: projecting the size-controlled image on the screen through the projection unit.

Images