KR102174464B1 - Space touch detecting device and display device having the same - Google Patents

Abstract

The display device includes a display panel. The input detector is disposed in front of the display panel and acquires side images of a sensing space from the entire display surface of the display panel to a predetermined distance. The controller detects an input object located in the sensing space from side images, calculates a three-dimensional position of the input object based on position information in the side images of the input object, and non-contacts based on the three-dimensional position of the input object. Recognizes the spatial touch input of the method.

Description

A spatial touch sensing device, and a display device including the same TECHNICAL FIELD [SPACE TOUCH DETECTING DEVICE AND DISPLAY DEVICE HAVING THE SAME}

The present invention relates to a display device, and more particularly, to a spatial touch sensing device for sensing a spatial touch and a display device including the same.

A touch screen installed on a display device to input data by simply touching it using a finger or a pen is widely used.

The touch screen detects the occurrence of a user input by adding a touch panel to the display panel and recognizing that the characteristics of a corresponding region (ie, a region touched by a finger, etc.) change when a finger or a pen touches it.

Since the touch screen as described above generally employs a method of operating in a two-dimensional plane in which the display screen and the touch panel are disposed on the same surface, there is a limitation in providing various expressions of the user interface.

Accordingly, a problem to be solved by the present invention is to provide a display device capable of sensing a three-dimensional touch input.

Another problem to be solved by the present invention is to provide a spatial touch sensing device capable of sensing a three-dimensional touch input.

The problems of the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

A display device according to an exemplary embodiment for solving the above problem includes: a display panel; An input sensing unit disposed in front of the display panel and acquiring side images of a sensing space from the entire display surface of the display panel to a predetermined distance; And detecting an input object located in the sensing space from the side images, calculating 3D position information of the input object based on position information in the side images of the input object, and calculating the 3D position information of the input object. And a controller for recognizing a non-contact spatial touch input based on dimensional location information.

The input detection unit may include a first camera device disposed toward a first direction crossing the display surface of the display panel to generate a first side image of the sensing space, and a first camera device spaced apart from the first camera device And a second camera device disposed toward a second direction intersecting with the first direction to obtain a second side image of the sensing space, wherein the first side image and the second side image are included in the side images. I can.

The input sensing unit may further include an infrared light source emitting infrared light to the sensing space, and the first camera device and the second camera device may be infrared cameras.

The display device is disposed adjacent to one side of the display panel, the upper cover protrudes toward the front of the display panel, includes an empty space formed therein, and includes an opening formed on a surface adjacent to the display panel ; And a transparent plate covering the opening, and exposing the empty space on both sides of the opening, wherein the infrared light source, the first camera device, and the second camera device are in the empty space of the upper cover. And the infrared light source is covered by the transparent plate, the first camera device is disposed adjacent to one side of the upper cover, exposed by the opening, and the second camera device is the other side of the upper cover Doedoe disposed adjacent to, it may be exposed by the opening.

The display panel is a curved display panel having a concave display surface, and each of the first camera device and the second camera device may be disposed to face the center of an area of the display surface of the display panel.

The sensing space is defined as an area where the photographing region of the first camera device and the photographing region of the second camera device overlap, and the sensing space has the same width in a direction perpendicular to the display surface in the entire display panel. I can.

The display device further includes a cover portion protruding toward the front of the display panel from at least a portion of an edge of the display panel, the cover portion defining the sensing space, and flowing into the sensing space from a side of the display panel Infrared light can be blocked.

The input sensing unit may include a third camera device disposed toward a third direction to generate a third side image of the sensing space, and a third camera device disposed toward a fourth direction to generate a fourth side image of the sensing space. 4 further comprising a camera device, wherein the third direction intersects the second direction, the fourth direction intersects the third direction, and the control unit comprises two side surfaces of the first to fourth side images A first sub-input object may be detected based on the images, and a second sub-input object may be detected based on the remaining two side images of the first to fourth side images.

The display device further includes ultrasonic transducers disposed on one side in front of the display panel to emit ultrasonic waves parallel to the display surface of the display panel, and the closer the input object is to the display panel, The pressure applied to the input object may increase.

A spatial touch sensing apparatus according to an exemplary embodiment for solving the other problem includes: an input sensing unit disposed in front of a screen and acquiring side images of a sensing space from the screen to a predetermined distance; And detecting an input object located in the sensing space from the side images, calculating 3D location information of the input object, and recognizing a non-contact spatial touch input based on the 3D location information of the input object. It includes a control unit.

Details of other embodiments are included in the detailed description and drawings.

The display device according to the exemplary embodiment of the present invention obtains side images of the sensing space from the front of the display unit and calculates a three-dimensional position of the input object in the sensing space from the side images. 3D touch input can be detected.

Effects according to the embodiments of the present invention are not limited by the contents illustrated above, and more various effects are included in the present specification.

1 is a perspective view illustrating a display device according to an exemplary embodiment of the present invention.
2A and 2B are exploded perspective views illustrating an example of a display unit included in the display device of FIG. 1.
2C is a bottom side view illustrating an example of the display device of FIG. 1.
FIG. 2D is an enlarged view of area Q1 of FIG. 1.
3A to 3C are diagrams illustrating an example of a sensing space formed by an input sensing unit included in the display device of FIG. 1.
4A to 4E are diagrams illustrating various arrangement examples of an input sensing unit disposed by the display device of FIG. 1.
5A is a diagram illustrating a display device according to other exemplary embodiments of the present invention.
5B is a plan view illustrating an example of a sensing space formed by the display device of FIG. 5A.
6 is a block diagram illustrating an example of a control unit included in the display device of FIG. 1.
7A to 7F are diagrams illustrating a process of detecting an input object by the controller of FIG. 6.
8A and 8B are schematic diagrams illustrating an input image of a pointer according to a location.
9 is a schematic diagram illustrating a comparison between an original image of a proximity pointer and a converted image.
10 is a schematic diagram showing a method of analyzing a ratio of a converted image.
11A and 11B are diagrams illustrating an example of a sensory providing unit included in the display device of FIG. 1.
12 is a diagram illustrating a display device according to another exemplary embodiment of the present invention.
13 is a diagram illustrating a configuration in which the display device of FIG. 12 detects a multi-input.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various forms different from each other, and only these embodiments make the disclosure of the present invention complete, and common knowledge in the technical field to which the present invention belongs It is provided to completely inform the scope of the invention to those who have it, and the invention is only defined by the scope of the claims.

Although the first, second, and the like are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical idea of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a perspective view illustrating a display device according to an exemplary embodiment.

Referring to FIG. 1, the display device 1 includes a display unit 100, an input detection unit 300, and a control unit 500.

The display unit 100 may provide an image. Here, one surface of the display unit 100 on which an image is displayed is defined as a display surface, and the upper, lower, left, and right sides of the display device 1 are based on the display surface of the display unit 100 shown in FIG. 1 ( For example, based on a surface formed by the first direction D1 and the second direction D2), it is defined as upper, lower, left, and right. In addition, the front (or upper) of the display device 1 is defined as the direction in which the display surface is visible based on a direction perpendicular to the display surface (i.e., the third direction D3), and the rear of the display device 1 (Or, the lower part) is defined as the direction in which the display surface is not visible.

The display unit 100 may have a rectangular planar shape. The display unit 100 may be a curved display module having a specific radius of curvature. For example, the display surface of the display unit 100 may be curved with a radius of curvature of about 1.5 m. As shown in FIG. 1, the display unit 100 may have a concave shape from the front to the rear, or may have a convex shape on the contrary. However, the present invention is not limited thereto, and the display unit 100 may be a flat display module having a flat display surface. Hereinafter, a case where the display unit 100 is a curved display module will be described as an example.

The display unit 100 may be a liquid crystal display device (LCD), an organic light emitting display device (OLED), a plasma display device (PDP), a field emission display device (FED), or the like according to a method of emitting light. A more specific configuration of the display unit 100 will be described later with reference to FIG. 2A.

The input sensing unit 300 may generate sensing data for a sensing space (or sensing region, see FIG. 3A) set in front of the display unit 100. Here, the sensing space has an area equal to or similar to the display surface of the display unit 100 (based on a plane or curved surface parallel to the display surface) and within a preset distance (eg, maximum 1m) from the display surface of the display unit 100. A space in which it is located is a three-dimensional space having a predetermined volume, and may be a space in which a non-contact air touch is input by the input object OBJ. The input object OBJ may include a pointer. The pointer is an indication means indicating a direction or location, and a user's finger, a pen, an indicator rod, a baton, an antenna, a stick, etc. may be used as a pointer. In the drawings, as examples of the pointer, a user's hand (or finger) and a pen are illustrated, but it is obvious that examples of applicable pointers are not limited thereto.

The input sensing unit 300 may generate sensing data for a sensing space in which the input object OBJ is located. The sensing data may be used to generate a touch event by the controller 500 to be described later.

In embodiments, the input sensing unit 300 is disposed in front of the display unit 100 and may acquire side images (or side image data) of the sensing space.

In an embodiment, the input detection unit 300 may include a first camera device 311 and a second camera device 312. The first camera device 311 is disposed in front of the display unit 100, but is disposed toward a first gaze direction DR1 that crosses the display surface of the display unit 100, and generates a first side image of the sensing space. can do. The first gaze direction DR1 may be a direction crossing the display surface in an oblique direction (ie, a direction inclined with respect to the side of the display surface), and, for example, corners (or corners) of the display unit 100 It may be a direction toward the center of the area of the display surface from an edge of one of the parts).

Similarly, the second camera device 312 is disposed to be spaced apart from the first camera device 311 in front of the display unit 100, but toward the second line of sight direction DR2 that crosses the display surface of the display unit 100 It is disposed, and a second side image of the sensing space may be generated. Here, the second gaze direction DR2 intersects the first gaze direction DR1, and for example, the second gaze direction DR2 may be perpendicular to the first gaze direction DR1 or may form an obtuse angle. . The second line of sight direction DR2 may be a direction crossing the display surface in an oblique direction, and for example, may be a direction from the other one of the corners of the display unit 100 toward the center of the area of the display surface.

As shown in FIG. 1, the first camera device 311 may be disposed at the upper left corner of the display unit 100, and the gaze axis of the first camera device 311 may be disposed toward the center of the area of the display surface. have. Similarly, the second camera device 312 may be disposed at the upper right corner of the display unit 100, but may be disposed such that the gaze axis of the second camera device 312 faces the center of the area of the display surface.

The first and second camera devices 300: 311 and 312 may have a view angle (or field of view) of 90 degrees or more. In this case, each of the first and second camera devices 311 and 312 may capture a sensing space in front of the display unit 100 without a shaded area.

For reference, a general camera device used to detect an input object (OBJ) based on an image (or vision) is disposed from one side of the general display device toward the front of the display device to provide a front image of the comparison sensing space. To obtain. Here, the comparative sensing space, which is a space corresponding to the sensing space according to the embodiment, is usually spaced apart from the surface of the display unit 100 by a distance equal to or greater than the minimum focal length of the camera device. That is, according to the focal length of the lens of a general camera device, since the user's hand must be separated from the general camera device by 20 cm or more to accurately detect the user's hand, the input object (OBJ) is Is difficult to detect. In order to overcome this limitation, a camera device having a short focal length lens may be used, but the camera device has a narrow angle of view, so it is difficult to detect the input object OBJ from the entire front of the display device.

On the other hand, since the input sensing unit 300 according to the exemplary embodiments is disposed to be spaced apart from each other and detects the position by acquiring side images of the sensing space, not the front image, a space adjacent thereto from the surface of the display unit 100 For example, all input objects OBJ located within a space within 1m from the display surface of the display unit 100 may be detected.

Meanwhile, in FIG. 1, the first and second camera devices 311 and 312 are shown to be disposed at the upper left corner and the upper right corner of the display unit 100, but these are exemplary and are not limited thereto. Various arrangements of the first and second camera devices 311 and 312 will be described later with reference to FIGS. 4A to 4F.

In addition, in FIG. 1, the input detection unit 300 is shown to include two camera devices 311 and 312, but this is exemplary, and the input detection unit 300 includes three or more camera devices. May be. This will be described later with reference to FIG. 12.

In one embodiment, the input sensing unit 300 may further include an infrared light source 320. The infrared light source 320 may project infrared light into the sensing space. Here, the infrared light has a wavelength of about 940 nm, and can be distinguished from the infrared light having a wavelength of 850 nm used in a surveillance camera. The infrared light source 320 may include infrared LEDs having a relatively wide projection angle or infrared laser light sources having a relatively large projection density.

When the infrared light source 320 is provided, the first camera device 311 and the second camera device 312 may be implemented as infrared cameras, or may include a visible light cut filter that blocks visible light. Accordingly, sensitivity to side images is improved, and the input object OBJ can be more easily detected by infrared light reflected by the input object OBJ. Meanwhile, the display device 1 is applied to an entertainment device such as a slot machine and can be used in an entertainment venue such as a casino. May not be easily detected. Accordingly, the input sensing unit 300 according to the embodiments uses infrared light different from visible light (also, infrared light having a wavelength different from infrared light used in a surveillance camera of an entertainment venue), and thus the input object OBJ ) Can be detected more accurately.

1, the infrared light source 320 may be disposed adjacent to the upper side of the display unit 100. However, this is exemplary, and the infrared light source 320 may be disposed adjacent to the lower side of the display unit 100, or may be disposed on the upper and lower sides of the display unit 100, respectively. In addition, the infrared light source 320 may be disposed on the left side or the right side of the display unit 100.

The control unit 500 detects the input object OBJ from side images (ie, detection data acquired by the input detection unit 300), and inputs the input object based on the position information of the input object OBJ in the side images. 3D location information of the object OBJ is calculated, and a spatial touch input or a touch event may be generated based on the 3D location information. For example, the controller 500 detects an input object by image processing side images, and position information of each of the preset camera devices 311 and 312, and position information (or direction information) of the input object in the side images. ), 3D location information of the input object may be calculated, and a touch event may be generated based on a change in the 3D location information (eg, change in the third direction D3).

Further, the controller 500 provides image data to the display unit 100 and may control or change the image data in response to a touch event. Furthermore, the controller 500 may generate a sensory control signal in response to a touch event.

In some embodiments, the display device 1 may further include a sense providing unit 700. The sense providing unit 700 may provide a specific sense (eg, tactile sense) to the input object OBJ.

The sense providing unit 700 may be disposed in front of the display unit 100. As shown in FIG. 1, the sensory providing unit 700 is disposed adjacent to the lower side of the display unit 100 and may provide a sensory signal to the sensing space. For example, the sense providing unit 700 includes ultrasonic generators (or ultrasonic transducers), and the ultrasonic generators emit ultrasonic waves from the lower side of the display unit 100 toward the upper side of the display unit 100. I can.

When the input object OBJ enters the sensory area (eg, an area affected by ultrasound, a space similar to or included in the sensing space), the end of the input object OBJ (ie, within the sensory area) is The entered part) may vibrate, and the user may recognize that a touch event is generated based on the vibration of the input object OBJ (eg, vibration of the hand).

The sensory providing unit 700 may generate ultrasonic waves based on the sensory control signal provided by the controller 500. That is, the sense providing unit 700 may generate ultrasonic waves only when a touch event occurs. Alternatively, the sensory providing unit 7000 may periodically generate ultrasonic waves irrespective of the sensory control signal.

In an embodiment, the sensory providing unit 700 may change the intensity of the ultrasound (ie, the size of a tactile sense) based on a distance between the input object OBJ and the display unit 100. For example, the sensory providing unit 700 may increase the intensity of ultrasound as the input object OBJ is adjacent to the display unit 100.

As described with reference to FIG. 1, the display device 1 acquires side images of the sensing space through the input detection unit 300, and extracts the input object OBJ from the side images through the control unit 500. And, based on the 3D position of the input object OBJ, a spatial touch input may be recognized or a touch event may be generated.

In addition, the input detection unit 300 detects the input object OBJ adjacent to the display surface of the display unit 100 by acquiring side images of the sensing space, and moves in the third direction D3 of the input object OBJ. The position change (or the distance from the display surface to the input object OBJ) can be more accurately detected.

Further, the first and second camera devices 311 and 312 included in the input detection unit 300 are spaced apart from each other and are disposed adjacent to different corners of the display unit 100, so that the display surface without a shaded area The input object (OBJ) can be detected in the entire sensing space in front of.

Meanwhile, the input sensing unit 300 and the control unit 500 (and the sense providing unit 700) may be implemented as one module (eg, a spatial touch sensing module).

2A and 2B are exploded perspective views illustrating an example of a display unit included in the display device of FIG. 1. 2C is a bottom side view illustrating an example of the display device of FIG. 1. FIG. 2D is an enlarged view of area Q1 of FIG. 1.

2A to 2D, the display unit 100 may include a display module 110 and a housing 130: 131, 132, 133, and 134.

The display module 110 is implemented as a liquid crystal display module, and may include a liquid crystal display panel 111, a light source 113, a back plate 115, and an optical sheet 117, as shown in FIG. 2B.

The liquid crystal display panel 111 may be a curved display panel that is concavely curved left and right with respect to a vertical axis. The liquid crystal display panel 111 may include a pixel electrode and a common electrode facing each other, a liquid crystal layer (or liquid crystal cells) disposed therebetween, transistors that transmit voltage to the pixel electrode, color filters, and polarizing members. .

The light source 113 may provide light to the liquid crystal display panel 111. The light source 113 includes a plurality of LEDs, and the LEDs may be attached to the backplate 115 in a matrix form. 2 shows that the light source 113 is disposed in a direct type, the light source 113 may be disposed in an edge type.

The back plate 115 supports the light source 113 and may cover the liquid crystal display panel 111. The back plate 115 is formed along the circumferential direction of the liquid crystal display panel 111 on a concentric circle of the same origin as the origin of the liquid crystal display panel 111 (that is, the center of a virtual circle obtained by extending the curved surface). It can have a set width. The back plate 115 may include a plurality of rectangular plates, and each of the plates may be disposed to face the origin of the liquid crystal display panel 111.

The optical sheet 117 is disposed between the liquid crystal display panel 111 and the light source 113. The optical sheet 117 may include a diffusion sheet, a prism sheet, and a brightness enhancement sheet.

Meanwhile, although not shown, the display module 110 may further include a window disposed in front of the liquid crystal display panel 111 to protect the liquid crystal display panel 111.

Referring back to FIG. 2A, the housing 130 may accommodate the display module 110 and protect the display module 110 from the rear and side surfaces of the display module 110. In addition, the housing 130 accommodates the input sensing unit 300 and/or the sense providing unit 700, or the input sensing unit 300 and/or the sense providing unit 700 is mounted in the housing 130. ) Can be.

The housing 130 may include a body part 131, an upper cover 132, a side cover 133, and a support part 134.

The main body 131 may have a rectangular planar shape corresponding to the display module 110, may be bent together in a direction in which the display device 1 is bent, and may include a storage space formed by recessing the front surface. The display module 110 may be accommodated in the storage space.

At least a portion of the rear surface of the main body 131 may be cut off or open, and a module-type control unit 150 may be disposed at the cut or open portion of the main body 131.

The upper cover 132 is disposed on the upper side (or upper front surface) of the main body 131 and may include an empty space formed therein between the main body 131 and the main body 131. The input detection unit 300 may be disposed in the empty space. The upper cover 132 and the input sensing unit 300 may be configured as one module, and may be disposed or coupled to the upper side of the main body 131.

As shown in FIG. 2C, an opening OP communicating the empty space and the outside may be formed between the upper cover 132 and the main body 131 at the lower side of the upper cover 132. The housing 130 may further include a transparent plate 132-1 covering at least a portion of the opening. The upper cover 132 is made of an opaque and rigid material, so that the input sensing unit 300 can be prevented from being exposed to the outside. Infrared light emitted from the input sensing unit 300 (or the infrared light source 320) may be transmitted through the transparent plate 132-1 disposed in the opening OP to be provided to the sensing space.

On the other hand, part of the opening OP of the upper cover 132 is not covered by the transparent plate 132-1, and the first and second camera devices 311 and 312 (for example, camera devices ( 311, 312) each lens) may be exposed to the outside. With such a structure, the infrared light reflected by the input object OBJ may be prevented from being totally reflected by the transparent plate 132-1 (or the refractive index of the transparent plate 132-1 ).

Referring back to FIG. 2A, the side cover 133 may have a rectangular planar shape and may be disposed on the side of the body portion 131. The side cover 133 may block light (eg, infrared light) flowing from the side of the display module 110. In addition, the side cover 133 may provide an installation area of the input sensing unit 300 (or the first and second camera devices 321 and 322 ).

In one embodiment, the side cover 133 may define a sensing space. For example, in side images acquired from the first and second camera devices 321 and 322, it may be determined that only the input object OBJ displayed by overlapping with the side cover 133 exists in the sensing area. have.

The support 134 (or the support member) may support the first and second camera devices 311 and 312 and the infrared light source 320. The support part 134 may include a first support part 134-1 for supporting the first and second camera devices 311 and 312 and a second support part (not shown) for supporting the infrared light source 320. .

As shown in FIG. 2D, the first support part 134-1 is from the fixing part 134a fixed to the upper surface of the main body 131 (or the display module 110) and the fixing part 134a. A first extension portion 134b extending along an oblique direction (ie, a fourth direction D4, a direction perpendicular to the line of sight axis of the first camera device 311, or a direction forming an acute angle with the fixing portion 134a) ), and a second extension part 134c extending from the first extension part 134b to the front of the display module 110. The first camera device 311 (or the second camera device 312) may be fixed to the inner surface of the second extension part 134c.

The horizontal angle of the first camera device 311 (that is, the horizontal component of the gaze axis of the first camera device 311) is adjusted according to the alignment of the fixing part 134a and the main body part 131, and the fixing part 134a The vertical angle of the first camera device 311 (ie, a vertical component of the line of sight of the first camera device 311) may be adjusted according to the bent angle of the first extension part 134b based on.

Meanwhile, in FIG. 2D, the first support 134-1 is shown to be disposed on the upper side of the main body 131, but is not limited thereto, and the first support 134-1 is the main body 131 It may be disposed on the side or side cover 133 of the.

The second support part (not shown) may extend upward from the upper side of the main body 131 (or the display module 110 ), and may be formed by bending toward the front of the display module 110. The second support part has a "L" shape (or inverse "L" shape), and an infrared light source on the inner surface of the second support part (not shown) (that is, a surface facing the upper surface of the main body 131) 320 may be placed. The second support part prevents the infrared light emitted from the infrared light source 320 from traveling in the horizontal direction (that is, the infrared light directly entering the first and second camera devices 311 and 312). It may also include a side light blocking member.

In some embodiments, the display unit 100 may further include a three-dimensional filter unit (not shown). The three-dimensional filter unit may be disposed in front of the display module 110. For example, the three-dimensional filter unit may have a radius of curvature that is substantially the same as or similar to that of the display module 110 and may be disposed in front of the display module 110. The three-dimensional filter unit may be disposed in close contact with the front surface of the display module 110 or spaced apart from the front surface of the display module 110.

The three-dimensional filter unit (not shown) may include a lenticular lens sheet. By providing the left eye image of the display module 110 to the user's left eye and the right eye image to the user's right eye by the lenticular lens, a 3D stereoscopic image may be provided to the user. When the lenticular lens is formed to be concave with the same curvature as that of the display module 110, an optical illusion phenomenon such as the existence of an image in the concave space may occur. Accordingly, the image appears to be displayed on the space due to the optical illusion, and the sense of space can be maximized by the user touching the image by putting his hand in a certain depth of the concave space.

As described with reference to FIGS. 2A to 2D, the display unit 100 covers the input sensing unit 300 using the upper cover 132, but at least a part of the upper cover 132 is the input sensing unit 300. The first and second camera devices 311 and 312 of are exposed so that side images reflecting all of the light reflected by the input object OBJ can be generated. In addition, the display unit 100 includes side covers 133a and 133b disposed on the side of the display module 110 to block light from outside, and to more easily define or form a sensing space. I can.

3A to 3C are diagrams illustrating an example of a sensing space formed by an input sensing unit included in the display device of FIG. 1. 3B and 3C illustrate a plane of the sensing space.

First, referring to FIGS. 3A and 3B, the first and second camera devices 311 and 312 are disposed in front of the display unit 110 as described above, but the upper left and upper right corners of the display unit 110 Each can be placed in the corner. In addition, the first and second camera devices 311 and 312 may be disposed such that the gaze axes of the first and second camera devices 311 and 312 face the center of the area of the display surface. The first and second camera devices 311 and 312 may have a view angle of 90 degrees or more, for example, 90 degrees or 140 degrees.

The sensing space SA may be formed in a region where the photographing region of the first camera device 311 and the photographing region of the second camera device 312 overlap. As the sensing space SA is adjacent to the first and second camera devices 311 and 312, the width (or thickness, that is, the length in the direction perpendicular to the display surface of the display unit 110) is smaller, The width may increase as the distance from the first and second camera devices 311 and 312 increases. However, the present invention is not limited thereto, and the shape and width of the sensing space may vary according to the curvature of the display unit 110 and the arrangement direction of the first and second camera devices 311 and 312.

As illustrated in FIG. 3B, the sensing space SA may have a planar shape of an arc (or sector) based on a point (refer to “P0”) positioned in front of the display device 1.

In one embodiment, the edge of the viewing angle of the first camera device 311 and the edge of the viewing angle of the second camera device 312 are the origin of the display device 1 (or the display surface of the display unit 100) (P0 ) Or the origin P0 of the display device 1 may be located within the viewing range of the first camera device 311 and the viewing range of the second camera device 312. In this case, the sensing space SA has the same thickness (i.e., the length in a direction perpendicular to the display surface) over the entire front surface of the display device 1, and is located in the front surface of the display device 1 according to the shape of the sensing space. An input range of spatial touch having the same maximum depth may be provided. For example, when the display device 1 has a display surface of 55 inches and a radius of curvature of about 1.5 m, the first and second camera devices 311 and 312 are located at the center of the area of the display surface (' P1'), the edge of the viewing angle of the first and second camera devices 311 and 312 may pass through the origin P0 of the display device 1. In this case, the sensing space SA has an arc shape with a radius of about 1.5 m, and the display device 1 is based on the user (or the user's gaze) located at the origin P0 of the display device 1 As a result, an input range of spatial touch having the same depth DE1 of at most 1.5m over the entire front area of the display device 1 may be provided.

Meanwhile, when the infrared light source 320 projects infrared light to an area having a certain width (eg, 30 cm) in front of the display device 1, the display device 1 A sensing space SA having the same depth DE2 (eg, a maximum depth of 30 cm) may be formed in the entire area.

As described with reference to FIGS. 3A to 3C, when the viewing angles of the first and second camera devices 311 and 312 are disposed to pass through the origin P0 of the radius of curvature of the display device 1, a separate Without image processing, the sensing space SA having the same depth ('DE2' in FIG. 3C) is formed over the entire front surface of the display device 1, or the same maximum depth (FIG. 3B) over the entire front surface of the display device 1 An input range of spatial touch having'DE1') of may be provided.

4A to 4E are diagrams illustrating various arrangement examples of an input sensing unit disposed by the display device of FIG. 1.

First, referring to FIGS. 1, 3A, and 4A, in that the display device 1_1 of FIG. 4A includes a second camera device 312_1 disposed adjacent to the lower right corner of the display unit 100, FIG. 1 It is different from the display device 1 of. Except for the second camera device 312_1, the display device 1_1 of FIG. 4A is substantially the same as or similar to the display device 1 of FIG. 1, and therefore, a duplicate description will not be repeated.

The first camera device 311 may be disposed adjacent to the upper left corner of the display unit 100.

The sensing space SA1 formed by the first and second camera devices 311 and 312_2 may be substantially the same as or similar to the sensing space SA described with reference to FIG. 3A. However, since the second camera device 312_1 is arranged to be spaced apart from the first camera device 311 as much as possible, even if the size of the display device 1 is reduced, the input object OBJ is used as the first and second camera devices. It may be sufficiently (eg, 30cm or more) spaced apart from at least one of (311, 312_1). Accordingly, the display device 1 can accurately detect the input object OBJ in the sensing space SA1.

Referring to FIG. 4B, the display device 1_2 includes first and second camera devices 311 and 312_2, and the first camera device 311 is disposed adjacent to the upper left corner of the display unit 100 , The second camera device 312_2 may be disposed adjacent to the lower left corner of the display unit 100.

The sensing space SA2 formed by the first and second camera devices 311 and 312_2 may have a thicker thickness as it is adjacent to one side (eg, right side) of the display unit 100. Since the first and second camera devices 311 and 312 are completely blocked from light flowing from the left side of the display unit 100, a separate light source, a display device, and an infrared light source are disposed on the left side of the display unit 100. In the environment, the arrangement of the first and second camera devices 311 and 312 shown in FIG. 4B may be applied. In addition, when a deeper spatial touch input is required on one side of the display unit 100, the arrangement of the first and second camera devices 311 and 312 shown in FIG. 4B may be applied.

Referring to FIG. 4C, the display device 1_3 includes first and second camera devices 311 and 312_3, and the first camera device 311 is disposed adjacent to the upper left corner of the display unit 100 , The second camera device 312_3 may be disposed adjacent to a central portion of the upper side of the display unit 100. Although not shown, the second camera device 312 may be non-overlapping with the infrared light source in the vertical direction.

The second camera device 312_3 may have a field of view of 180 degrees or more, and in this case, the sensing space SA3 may have a greater thickness toward the right side of the display unit 100. That is, a spatial touch input having a relatively large depth on one side (for example, the right side) of the display unit 100 may be allowed.

Referring to FIG. 4D, the display device 1_4 includes first and second camera devices 311_4 and 312_4, and the first camera device 311_4 is disposed at the center of the left side of the display unit 100, The second camera device 312_4 may be disposed adjacent to a central portion of the upper side of the display unit 100. The angle of view of the first and second camera devices 311_4 and 312_4 may be 180 degrees or more. The display device of FIG. 4D may have a sensing space SA4 that is substantially the same as or similar to the sensing space SA3 of the display device of FIG. 4C.

Referring to FIG. 4E, the display device 1_5 includes first and second camera devices 311_5 and 312_5, and the first camera device 311_5 is disposed at the center of the left side of the display unit 100, The second camera device 312_5 may be disposed adjacent to the central portion of the right side of the display unit 100. The angle of view of the first and second camera devices 311_5 and 312_5 may be 180 degrees or more. In this case, the display device 1_5 may have a sensing space SA5 that is substantially the same as or similar to the sensing space SA1 of the display device 1_1 of FIG. 4A. On the other hand, the angle of view of the first and second camera devices 311 and 312 may be 180 degrees or less (for example, 90 degrees), and a partial area (for example, display) instead of the entire front surface of the display device 1 When a spatial touch input is to be detected only for a partial area within a specific distance based on the area center of the surface, the arrangement of the first and second camera devices 311_5 and 312_5 shown in FIG. 4E may be applied.

5A is a diagram illustrating a display device according to other exemplary embodiments of the present invention. 5B is a plan view illustrating an example of a sensing space formed by the display device of FIG. 5A.

1, 5A, and 5B, the display device 1_6 is different from the display device 1 of FIG. 1 in that it includes a flat display unit 100. Except for the flat display unit 100 (and the configurations corresponding to the flat display unit 100), the display device 1_6 of FIG. 5A is substantially the same as or similar to the display device 1 of FIG. Decided not to repeat.

The display device 1_6 includes first and second camera devices 311 and 312, and the first camera device 311 is disposed adjacent to the upper left corner of the display unit 100, and the second camera device ( 312 may be disposed adjacent to the upper right corner of the display unit 100.

However, the first camera device 311 is disposed adjacent to the end of the first side cover 133a (or left side cover) coupled to the left side of the display unit 100, and the second camera device 312 is It may be disposed adjacent to the end of the second side cover 133b (or right side cover) coupled to the right side of the display unit 100. That is, the first and second camera devices 311 and 312 have the width of the first and second side covers 133a and 133b from the display unit 100 (or display surface) (that is, perpendicular to the display surface). It can be arranged spaced apart by the length in the direction).

As the display unit 100 is implemented in a flat type rather than a curved type, when the first and second camera devices 311 and 312 are disposed adjacent to the display surface, based on FIG. 5B, at both sides of the display unit 100 Since the thickness of the sensing space is relatively small, a spatial touch input having a sufficient depth may not be provided on both sides of the display unit 100. On the contrary, when the first and second camera devices 311 and 312 are arranged facing the front, the sensing space SA becomes unnecessarily large and unnecessary information (eg, objects separated by 1 m or more from the display surface) As this is sensed, the load of the display device may increase.

Therefore, in consideration of the depth of the spatial touch input to be provided on both sides of the display unit 100, the first and second camera devices 311 and 312 are spaced forward from the display unit 100, 2 It may be disposed on the side covers 133a and 133b, respectively. For example, as the depth of the spatial touch input to be provided increases, the first and second camera devices 311 and 312 may be disposed more spaced apart from the display unit 100, and the first and second side surfaces The width of the covers 133a and 133b (that is, the length in a direction perpendicular to the display surface) may also be increased.

On the other hand, when the first and second camera devices 311 and 312 are separated from the display unit 100, shadow areas may occur on both sides of the display unit 100. Accordingly, the first and second camera devices 311 and 312 may be arranged to face a reference point located at the rear of the display surface rather than the area center of the display surface.

6 is a block diagram illustrating an example of a control unit included in the display device of FIG. 1. 7A to 7F are diagrams illustrating a process of detecting an input object by the controller of FIG. 6.

7A illustrates an example of a user's hand positioned in the sensing space of the display device 1 as an example of a pointer as an input object, and FIG. 7B shows a first camera device included in the display device 1 of FIG. 7A A first side image acquired at 311 is shown, and FIG. 7C shows a second side image acquired by a second camera device 312 included in the display device 1 of FIG. 7A. Infrared light (that is, the infrared light projected from the infrared light source 320 described with reference to FIG. 1 to the sensing space) is reflected or scattered by the user's hand located in the sensing space, so that the user's hand may appear relatively bright. have.

Referring to FIG. 6, the control unit 500 may include a preprocessing unit 610, an extraction unit 620, and a calculation unit 630. In addition, the control unit 500 may further include an event generating unit 640. Since the image processing processes for the first and second side images IMGAE1 and IMAGE2 are substantially the same or similar to each other, the image processing processes for the first and second side images IMGAE1 and IMAGE2 are mutually common. The process will be described based on the first side image IMAGE1.

The preprocessor 610 may remove the background image from the first side image IMAGE1, remove noise, and also remove the shadow of the input object. That is, the preprocessor 610 may pre-process the first side image so that the hand image can be easily extracted from the first side image IMAGE1.

The preprocessor 610 may include a background removing unit 611, a noise removing unit 612, and a shadow removing unit 613.

The background removal unit 611 may generate or obtain a first preprocessed image IMAGE_B1 by performing a difference calculation on a preset background image IMAGE_BACK from the first side image IMAGE1. Here, the preset background image IMAGE_BACK may be an image of an area in which only the display surface of the display unit 100 is displayed. The background removal unit 611 may convert the pixel value of the corresponding region to 0 (eg, black).

The noise removal unit 612 samples (or binarizes) the first preprocessed image IMAGE_B1 and smoothes it to obtain a first original image IMAEG_N1 from which noise has been removed. For example, the noise removal unit 612 may replace pixel values in the first preprocessed image IMAGE_B1 with 0 (eg, black) or 255 (eg, white) based on the threshold value. .

The shadow removal unit 613 may remove the shadow image by applying a shadow removal algorithm to the first original image IMAGE_N1 and generate the shadow removal image IMAGE_S1. Here, the shadow image may be an image in which a hand is reflected and displayed by the display surface of the display unit 100.

As shown in FIG. 7D, the first original image IMAGE_N1 may include candidate object images OBJ_1 and OBJ_1S. Here, the candidate object images OBJ_1 and OBJ_1S are images that can be recognized as the input object OBJ, and may have a size equal to or larger than a reference size (or a reference area). In this case, the shadow removal unit 613 determines the shadow image OBJ_1S among the candidate object images OBJ_1 and OBJ_1S based on the size, symmetry, and placement position of the candidate object images OBJ_1 and OBJ_1S, and 1 The shadow image OBJ_1S may be removed from the original image IMAGE_N1. For example, when candidate object images OBJ_1 and OBJ_1S having the same or similar shape to each other exist, a shadow image OBJ_1S having a relatively small size and adjacent to the display unit 100 may be selected and removed.

The extraction unit 620 may extract a hand image from the shadow removal image IMAGE_S1 finally output from the preprocessor 610. The extraction unit 620 may include a detection unit 621, a tracking unit 622 and a correction unit 623.

The detector 621 may detect a pointer image by matching preset patterns (eg, features of an input object or object patterns including feature points) with respect to the shadow removal image IMAGE_S1. For example, the detection unit 621 extracts a candidate object image OBJ1 having a preset size (i.e., pixels greater than or equal to a preset number) and matches the candidate object image OBJ1 with a preset pattern to obtain a hand image. Can be detected. Here, the preset patterns are, for example, a pattern corresponding to a hand with only one finger open (a pattern including feature points up to a finger, palm, and wrist), a pattern corresponding to a hand with a fist, and A pattern corresponding to the hand may be included.

In addition, the detection unit 621 of the extraction unit 620, when the image of the pointer as an input object (a hand image in the drawing) is detected, a corresponding pattern (ie, a pattern used to detect a hand image, or a corresponding pattern) A reference point of the input object image may be determined based on the feature point defined in [0104]. For example, as illustrated in FIG. 7D, when a hand image OBJ_1 corresponding to a pattern indicating a state in which one finger is opened is detected, a point at the end of the finger may be determined as the reference point P_OBJ1. For another example, when a hand image corresponding to a pattern indicating a state in which all fingers are opened is detected, the center point of the palm may be determined as the reference point P_OBJ1.

Meanwhile, the shape of the input image of the pointer such as a finger or a pen may vary according to the distance between the first and second camera devices 311 and 312. Even when the shape of the input image is changed as described above, in order to accurately recognize and detect the pointer, the detection unit 621 may further perform shape analysis, ratio (size) analysis, and/or multi-camera weight analysis. To this end, the detection unit 621 may further include a shape analysis unit, a ratio (size) analysis unit, and/or a multi-camera weight analysis unit. 8 to 10 are referred to for a detailed description thereof.

8A and 8B are schematic diagrams illustrating an input image of a pointer according to a location.

Referring to FIGS. 8A and 8B, the image type of the pointer input to the camera device 310 varies according to the distance the pointer is away from the camera device 310. That is, in the case of the user's hand, as shown in FIG. 8A, the shape of the entire hand may be input as an image image at a distance, but the shape of the hand may be recognized as a single mass as a partial shape at a close distance. In addition, in the case of a pen, as shown in FIG. 8B, an input image for the entire shape of the pen can be obtained at a distance, but at a close distance, the shape of the pen is only partially input and thus can be viewed as a single block.

In this way, even when the pointer is located in a short distance, it is possible to identify where the pointer points on the screen by performing shape analysis. 9 is a schematic diagram illustrating a comparison between an original image of a proximity pointer and a converted image. As shown in FIG. 9, when feature points are extracted and connected to an original image in a near distance, the end of the hand or pen has a relatively small curvature compared to the surroundings. By recognizing such a curvature, it is possible to determine where the pointer is pointing in a partial input image at a short distance. In the near-field input image, the end of the pointer may be determined as a point having the same curvature compared to a curvature of a previously stored image pattern, or may be determined as a point having the minimum curvature in a curve formed by the feature points.

10 is a schematic diagram showing a method of analyzing a ratio of a converted image. Referring to FIG. 10, ratio analysis may be performed to distinguish a pointer from a noise image even in the case of a distant image. The ratio analysis may be performed by comparing the ratio of the vertical width (up and down width) and the horizontal width (left and right width) of the image. For example, in the case of a hand or a pen, the width in the horizontal direction is larger than the width in the vertical direction, whereas in the case of a human body, the width in the vertical direction may be analyzed as being larger than the width in the horizontal direction. As another example, the ratio analysis may be performed by comparing the ratio of the first width in the longest direction and the second width in a direction intersecting (usually perpendicular to) the first width. In the illustrated example, the hand or pen has a ratio of the first width and the second width of 2:1 or more, but the human body may be classified as having a ratio of the first width and the second width of less than 2:1. In this way, by comparing the ratio of the width in the vertical direction and the width in the horizontal direction or the ratio of the first width and the second width, the pointer can be accurately recognized and the noise image can be removed.

Meanwhile, an input image inputted to each of the camera devices 311 and 312 by a single pointer is different according to the distance. For accurate object recognition, different weights may be given for each distance to the camera devices 311 and 312. As described above, even a partial input image at a short distance may determine the end of the pointer and a direction in which the pointer is directed through shape analysis, but the accuracy may be lower than that of the entire input image at a distance. In order to compensate for this, the accuracy of object recognition may be improved by adding weight to a far input image among a plurality of input images for one pointer. For example, when the input object OBJ is located relatively closer to the first camera device 311 than the second camera device 312 in the display device 1 illustrated in FIG. 1, the second camera device 312 Object recognition can be performed by giving more weight to the image image input through ). The weight can be adjusted from a ratio of 100:0 to a ratio of 50:50 depending on the distance. The distance away from the camera devices 311 and 312 may be calculated by measuring the size of an image of the pointer, or may be calculated by receiving feedback from the three-dimensional position of the pointer calculated by the calculation unit 630 to be described later.

Referring back to FIGS. 6 and 7, the tracking unit 622 may track a pointer image over time using a pointer (hand in the drawing) tracking algorithm as a target object. For example, the tracking unit 622 may predict the next position of the hand image based on the movement path of the hand image (or reference point) over time, and the hand image may be calculated based on the next position of the predicted hand image. Even if the hand image is not detected quickly or the hand image is temporarily not detected, the hand image (or reference point) can be maintained for a specific time. Through this method, it is possible to prevent the hand image from being temporarily recognized and thus not performing a touch input.

The correction unit 623 may remove noise over time due to a change in the position of the reference point P_OBJ1. For example, when the tracking pointer moves quickly or the recognized reference point P_OBJ1 shakes, the correction unit 623 uses a filter such as a Kalman filter or a low pass filter to determine the location of the reference point P_OBJ1. Can be corrected.

That is, the extraction unit 620 may detect the pointer image (and/or the reference point of the pointer image) from the preprocessed side images, but may track and correct the pointer image suitable for use as a spatial touch input.

The calculator 630 may calculate a 3D position of the pointer in the sensing space based on the pointer image (or the reference point P_OBJ1). The calculation unit 630 may include a first position calculation unit 631 and a second position calculation unit 632.

The first position calculator 631 may calculate a two-dimensional position (or coordinates) of the pointer on a plane corresponding to the display surface of the display unit 100. As shown in FIG. 7A, the first position calculator 631 may calculate the eleventh position angle ANG11 of the hand image OBJ_1 based on the reference point of the hand image OBJ_1. Here, the eleventh position angle ANG11 may be an angle between the input object OBJ from the upper surface of the sensing space with respect to the first camera device 131. The first position calculator 631 may calculate the eleventh position angle ANG11 based on a distance (or the number of pixels) between the reference point of the hand image OBJ_1 from the upper side of the first side image. Similarly, as shown in FIG. 7C, the first position calculator 631 may calculate the 21st position angle ANG21 of the hand image OBJ_2 based on the reference point of the hand image OBJ_2.

Thereafter, as shown in FIG. 7E, the first position calculating unit 631 includes the first camera position information P_C1 of the first camera device 311 and the second camera position of the second camera device 312. Based on the information P_C2, the eleventh position angle ANG11, and the 21st position angle ANG21, the two-dimensional position P_OBJ_XY of the hand as a pointer may be calculated. The two-dimensional position (P_OBJ_XY) of the hand may be a point (or coordinates) where the hand is positioned on the display surface of the display unit 100.

The second position calculator 632 may calculate a depth of the pointer (or a depth value, a distance a hand is separated from the display surface) based on the display surface of the display unit 100. 7A to 7C, the second position calculator 632 calculates the twelfth position angle ANG12 and the 22nd position angle ANG22 based on the reference points of the hand images OBJ_1 and OBJ_2. I can. Here, the twelfth position angle ANG12 may be an angle between the input object OBJ from the front of the sensing space (or the display surface of the display unit 100) with respect to the first camera device 131.

Thereafter, as shown in FIG. 7F, the second position calculating unit 632 includes preset first camera position information (P_C1) of the first camera device 311 and a second camera of the second camera device 312. Based on the location information P_C2, the twelfth position angle ANG12, and the 22nd position angle ANG22, the depth of the hand, which is a pointer, P_OBJ_Z may be calculated.

Referring back to FIG. 6, the second position calculating unit 632 is based on the two-dimensional position (P_OBJ_XY) of the hand and the depth of the pointer (hand) (P_OBJ_Z) calculated by the first position calculating unit 631. A three-dimensional position (P_XYZ) can be created.

Meanwhile, the event generator 640 may generate a touch event based on the three-dimensional position P_XYZ of the hand generated by the calculator 630.

As described with reference to FIGS. 6 to 7F, the controller 600 detects a pointer (or hand) as an input object from side images through image processing, calculates a three-dimensional position of the pointer, and calculates the three-dimensional A touch event may be generated based on the location.

In addition, the control unit 600 may more accurately detect the pointer by removing the shadow reflected from the display surface of the display unit 100.

Furthermore, the controller 600 may generate a more accurate touch event by tracking the pointer image and correcting the pointer image over time.

11A and 11B are diagrams illustrating an example of a sense providing unit included in the display device of FIG. 1.

Referring to FIGS. 1 and 11A, the sensory providing unit 700 may include a plurality of ultrasonic transducers 710 arranged in the front and lower sides of the display unit 100 and arranged along the first direction D1. have. The ultrasonic transducer 710 may generate directional ultrasonic waves (eg, ultrasonic waves having a frequency of 40 kHz) in the second direction D2 to apply pressure to at least a portion of the input object OBJ. That is, the sensory providing unit 700 may provide a tactile sensation to the input object OBJ using the law of acoustic radiation force.

In an embodiment, the sensory providing unit 700 may include a first ultrasonic transducer 711 and a second ultrasonic transducer 712 having different frequencies or different intensities. The first ultrasonic transducer 711 and the second ultrasonic transducer 712 may be arranged as a plurality of rows or columns. For example, the first ultrasonic transducer 711 and the second ultrasonic transducer 712 each form a row and are arranged along the first direction D1, the second ultrasonic transducer 712 is the first The ultrasonic transducer 712 may be disposed adjacent to the display unit 100.

The second ultrasonic transducer 712 may have a lower frequency or higher intensity than the first ultrasonic transducer 711. In this case, the closer the input object OBJ is to the display unit 100, the greater pressure (or tactile sensation) may be provided to the input object OBJ.

In FIG. 11A, the ultrasonic transducer 710 is shown to emit ultrasonic waves in the second direction D2, but the present invention is not limited thereto.

Referring to FIG. 11B, the sense providing unit 700_1 may include third ultrasonic transducers 713 and fourth ultrasonic transducers 714. The third ultrasonic transducers 713 and the fourth ultrasonic transducers 714 may generate and output ultrasonic waves having the same frequency (eg, 40 KHz).

The third ultrasonic transducers 713 are disposed adjacent to the left side from the front lower side of the display unit 100, and are disposed to face the fifth direction D5, so that they can emit ultrasonic waves toward the fifth direction D5. I can. Similarly, the fourth ultrasonic transducers 714 are disposed adjacent to the right side from the front lower side of the display unit 100, and are disposed to face the sixth direction D6, so that the fourth ultrasonic transducers 714 are disposed toward the sixth direction D6. Can radiate. As the sixth direction D6 crosses the fifth direction D5, the ultrasonic waves output from the third ultrasonic transducers 713 are ultrasonic waves output from the fourth ultrasonic transducers 714, and a plurality of points. Can cross in the field.

In this case, the control unit 500 determines a specific point (that is, a point to which tactile sensation should be provided) based on the three-dimensional position of the input object OBJ, and the third and fourth ultrasonic transducers based on the determined point. Some of the 713 and 714 may be operated, or a sensory control signal for operating them may be generated. For example, in order to provide a tactile sensation to the input object OBJ shown in FIG. 11B, the controller 700 may operate the first sub ultrasonic transducer 713a and the second sub ultrasonic transducer 714a. . In this case, a pressure sufficient to be recognized as a tactile sensation to a user may be provided only at a point (or region) where the ultrasonic waves are overlapped. That is, the user can receive a tactile sensation only at the point where the ultrasonic waves overlap.

Meanwhile, in FIG. 11B, the third ultrasonic transducers 713 and the fourth ultrasonic transducers 714 are shown to be arranged to be separated from each other on the lower side of the display unit 100, but this is only an example and is limited thereto. no. For example, the third and fourth ultrasonic transducers 713 and 714 may be alternately arranged with each other. For another example, the third ultrasonic transducers 713 are disposed under the display unit 100, and the fourth ultrasonic transducers 714 are disposed on the left (or right) of the display unit 100, They may emit ultrasonic waves in directions that intersect each other. That is, if the third and fourth ultrasonic transducers 713 and 714 emit ultrasonic waves crossing each other at a plurality of points, their positions are not limited.

In addition, the third and fourth ultrasonic transducers 713 and 714 may be arranged along a plurality of columns (or a plurality of rows), similar to the sensory providing unit 700 illustrated in FIG. 11A.

12 is a diagram illustrating a display device according to another exemplary embodiment of the present invention. 13 is a diagram illustrating a configuration in which the display device of FIG. 12 detects a multi-input. 13 illustrates positions of camera devices 311 to 314 and input objects OBJ1 and OBJ2 on the front side of the display device.

Referring to FIG. 12, the display device 1_7 is different from the display device 1 of FIG. 1 in that it further includes third and fourth camera devices 313 and 314.

The third camera device 313 is disposed to be spaced apart from the first and second camera devices 311 and 312 in front of the display unit 100, and a third line of sight direction DR3 crossing the display surface of the display unit 100 ), it is possible to generate a third side image of the sensing space. Similarly, the fourth camera device 314 is disposed to be spaced apart from the third camera device 313 (and the first and second camera devices 311 and 312) in front of the display unit 100, but the display device ( It is disposed toward the fourth gaze direction DR4 that crosses the display surface of 1) to generate a fourth side image of the sensing space.

The third and fourth camera devices 313 and 314 may be provided in a lower cover that is substantially the same as or similar to the upper cover 132 described with reference to FIG. 2A. Further, the third and fourth camera devices 313 and 314 may be fixed by the support 134 described with reference to FIG. 2D.

The display device 1_7 may detect two input objects (eg, two hands of a user) through the first to fourth camera devices 311, 312, 313, and 314.

When the first and second input objects OBJ1 and OBJ2 are located in the sensing space in front of the display device 1_7, one of the first and second camera devices 311 and 312 is An image of one of the input objects OBJ1 and OBJ2 may not be acquired. That is, as shown in FIG. 12, when the first input object OBJ1 and the second input object OBJ2 are located along the first gaze direction DRR1 (or the third gaze direction DRR3), Based on the first camera device 311, the second input object OBJ2 is covered by the first input object OBJ1, so the first camera device 311 generates an image for the second input object OBJ2. It may not be possible. As the first input object OBJ1 has a volume rather than a point, as shown in FIG. 13, a shadow area SZ for the first camera device 311 may occur.

Accordingly, the display device 1_7 further includes third and fourth camera devices 313 and 314, and not only the first input object OBJ1 using the third and fourth camera devices 313 and 314 In addition, the second input object OBJ2 located in the shaded area SZ may be detected.

6 and 13, position information of a first reference point P_OBJ1 for a first input object OBJ from a first side image obtained from the first camera device 311 (for example, the 11th position Only angle (ANG11)) can be obtained. Similarly, from the fourth side image acquired from the fourth camera device 314, only the positional information of the second reference point P_OBJ2 for the second input object OBJ2 (for example, the 42nd position angle ANG42) Can be obtained. Meanwhile, location information for each of the first and second input objects OBJ1 and OBJ2 may be obtained from the second and third side images acquired from the second and third camera devices 312 and 313. .

In this case, the controller 600 determines a camera device from which only location information on one input object is obtained and a camera device from which location information on two input objects is obtained as one group, and Based on the three-dimensional position of the one input object can be calculated. In addition, the remaining two camera devices may be determined as another group, and a 3D position of the other of the two input objects may be calculated based on information of the corresponding group.

For example, the second camera device 312 is grouped together with the first camera device 311 into a first group, and the first and second camera devices 311 and 312 are the first camera devices 312 of the display device 1_7. As it is positioned on the long side LS1, a three-dimensional position of the first input object OBJ1 adjacent to the first long side LS1 among the first and second input objects may be calculated. Based on the second camera device 312, since the 21st position angle ANG21 of the first input object OBJ1 is smaller than the 22nd position angle ANG22 of the second input object OBJ2, the controller 500 Is determined by determining the 21st position angle ANG21 as the position angle of the first input object OBJ1, and based on the eleventh position angle ANG11 and the 21st position angle ANG21, 3 of the first input object OBJ1 You can calculate the dimensional position.

Similarly, the third camera device 313 is grouped together with the fourth camera device 314 into a second group, and the third and fourth camera devices 313 and 314 are the second long side of the display device 1_7. According to the position at LS2, a 3D position of the second input object OBJ2 adjacent to the second long side LS2 among the first and second input objects may be calculated. Based on the third camera device 313, since the 32nd position angle ANG32 of the second input object OBJ2 is smaller than the 31st position angle ANG31 of the first input object OBJ1, the controller 500 Is determined as the 32nd position angle ANG32 as the position angle of the second input object OBJ2, and based on the 32nd position angle ANG32 and the 42nd position angle ANG42, 3 of the second input object OBJ2 You can calculate the dimensional position.

For another example, the third camera device 313 is grouped together with the first camera device 311 into a first group, and the first and third camera devices 311 and 313 are As adjacent to the first short side SS1, a 3D position of the first input object OBJ1 adjacent to the first short side SS1 among the first and second input objects may be calculated. Based on the third camera device 313, since the 31st position angle ANG31 of the first input object OBJ1 is larger than the 32nd position angle ANG32 of the second input object OBJ2, the controller 500 Determines the 31st position angle ANG31 as the position angle of the first input object OBJ1, and based on the eleventh position angle ANG11 and the 31st position angle ANG31, 3 of the first input object OBJ1 You can calculate the dimensional position.

Similarly, the second camera device 312 is grouped together with the fourth camera device 314 into a second group, and the second and fourth camera devices 312 and 314 are the second short sides of the display device 1_7. As it is positioned at SS2, a 3D position of the second input object OBJ2 adjacent to the second short side SS2 among the first and second input objects may be calculated. Based on the second camera device 312, since the 22nd position angle ANG32 of the second input object OBJ2 is larger than the 21st position angle ANG31 of the first input object OBJ1, the controller 500 Determines the 22nd position angle ANG22 as the position angle of the second input object OBJ2, and based on the 22nd position angle ANG22 and the 42nd position angle ANG42, 3 of the second input object OBJ2 You can calculate the dimensional position.

That is, the control unit 500 groups the first to fourth camera devices 311, 312, 313, and 314 into first and second groups, and the first input object among side images corresponding to the first group (Or, only information on the input object) is obtained to calculate the 3D position of the first input object, and only the second input object is obtained from the side images corresponding to the second group to calculate the 3D position of the second input object can do.

As described with reference to FIGS. 12 and 13, the display device 1_7 may set two camera devices as a group and detect one input object for each group. Accordingly, the display device 1_7 Can detect a plurality of input objects.

Meanwhile, in FIGS. 12 and 13, the display device 1_7 is shown to detect two input objects using four camera devices, but this is only an example and is not limited thereto. The display device 1_7 may accurately detect N input objects using 2N (where, N is an integer greater than or equal to 2) camera devices arranged to be spaced apart from each other.

Embodiments of the present invention have been described above with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features of the present invention. You can understand. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting.

1: display device
10: spatial touch detection module
100: display
130: housing
131: main body
132: top cover
133: side cover
134: support
300: input detection unit
311, 312, 313, 314: first to fourth camera devices
320: infrared light source
500: control unit
700: sensory provision unit

Claims (10)

Display panel;
An input sensing unit disposed in front of the display panel and acquiring side images of a sensing space from the entire display surface of the display panel to a predetermined distance; And
Detects an input object located in the sensing space from the side images, calculates 3D position information of the input object based on the position information in the side images of the input object, and calculates the 3D position information of the input object Including a control unit for recognizing a non-contact type spatial touch input based on the location information,
The input detection unit,
A first camera device disposed toward a first direction crossing the display surface of the display panel to generate a first side image of the sensing space, and
A second camera device spaced apart from the first camera device and disposed toward a second direction crossing the first direction to obtain a second side image of the sensing space,
The first side image and the second side image are included in the side images,
The first direction and the second direction each face an area center of the display surface,
The angles of view of the first camera device and the second camera device each have 130 degrees to 150 degrees,
The first camera device and the second camera device are respectively positioned above the display panel, and are not disposed to overlap the display panel. delete The method of claim 1,
The input sensing unit further includes an infrared light source for emitting infrared light to the sensing space,
The first camera device and the second camera device are infrared cameras. The method of claim 3,
An upper cover disposed adjacent to one side of the display panel, protruding to the front of the display panel, including an empty space formed therein, and including an opening formed on a surface adjacent to the display panel; And
Covering the opening, further comprising a transparent plate exposing the empty space at both sides of the opening,
The infrared light source, the first camera device, and the second camera device are disposed in an empty space of the upper cover,
The infrared light source is covered by the transparent plate,
The first camera device is disposed adjacent to one side of the upper cover, and is exposed by the opening,
The second camera device is disposed adjacent to the other side of the upper cover, and is exposed by the opening. The method of claim 4,
The display panel is a curved display panel having a concave display surface,
Each of the first camera device and the second camera device is disposed to face the center of an area of the display surface of the display panel. The method of claim 5,
The sensing space is defined as a region in which the photographing region of the first camera device and the photographing region of the second camera device overlap,
The sensing space has the same width in a direction perpendicular to the display surface across the entire display panel. The method of claim 3,
Further comprising a cover portion protruding toward the front of the display panel from at least a portion of the edge of the display panel,
The cover part defines the sensing space, and blocks infrared light flowing into the sensing space from a side of the display panel. The method of claim 1,
The input detection unit,
A third camera device disposed toward a third direction to generate a third side image of the sensing space, and
Further comprising a fourth camera device disposed toward a fourth direction to generate a fourth side image of the sensing space,
The third direction intersects the second direction,
The fourth direction intersects the third direction,
The control unit detects a first sub-input object based on two side images of the first to fourth side images, and a first sub-input object is detected based on the remaining two side images of the first to fourth side images. 2 Display device for detecting sub-input objects. The method of claim 1,
Further comprising ultrasonic transducers disposed on one front side of the display panel to emit ultrasonic waves parallel to the display surface of the display panel,
A display device in which pressure applied to the input object by the ultrasonic wave increases as the input object is adjacent to the display panel. An input sensing unit disposed in front of the screen and obtaining side images of the sensing space from the screen to a predetermined distance; And
Detecting an input object located in the sensing space from the side images, calculating 3D location information of the input object, and recognizing a non-contact spatial touch input based on the 3D location information of the input object Including a control unit,
The input detection unit,
A first camera device disposed toward a first direction to generate a first side image of the sensing space, and
A second camera device spaced apart from the first camera device and disposed toward a second direction crossing the first direction to obtain a second side image of the sensing space,
The first side image and the second side image are included in the side images,
The cross angle between the first direction and the second direction is 40 degrees to 50 degrees,
A spatial touch sensing device having an angle of view of the first camera device and the second camera device of 130 degrees to 150 degrees, respectively.

Images