KR20120029230A - Transparent display device and method for detecting an object - Google Patents

Abstract

PURPOSE: A transparent display device and a method for detecting an object are provided to easily obtain information about rearward object without an additional search process. CONSTITUTION: An eye direction recognition unit determines an eye direction(S1020). An object detection unit searches a visible object on a first and a second locations of the transparent display from the selected eye direction(S1030). A CPU(Central Processing Unit) controls the information acquisition of the searched object(S1040). The transparent display displays the obtained object information(S1050).

Description

Transparent display device and method for detecting an object}

The present invention relates to a transparent display device and an object detection method, and more particularly, to a transparent display device and an object detection method for providing information through the transparent display.

Electronic devices such as broadcast receivers, personal computers, notebook computers, mobile phones, and the like may be configured to perform various functions. Examples of such various functions include broadcast reception function, data and voice communication function, taking a picture or video through a camera, a voice recording function, a music file playing function through a speaker system, and an image or video display function. Some terminals may include additional functionality to play games.

In addition, with the development of image technology and transparent electronic device development technology, transparent display panels capable of displaying objects and displaying behind them are being actively researched, and electronic devices display images and data using transparent display panels as display panels. Doing. As such a transparent display panel, an organic light emitting panel or a plasma panel using self-luminescence is widely used.

An object of the present invention is to provide a transparent display device and an object detection method for providing information on an object selected by a user among objects visible through the transparent display.

Another object of the present invention is to provide a transparent display device and an object detection method for providing information on an object when an object corresponding to a specific condition is seen through the transparent display.

According to an aspect of the present invention, there is provided a method for detecting an object, the method comprising: receiving object selection information, detecting an object indicated by the input object selection information in a landscape viewed through a transparent display, and detecting the detected object. Acquiring information about the object and displaying the obtained information on the transparent display. Here, the object may include at least one of a person, an animal, an object, a car, a building, and a natural object. The information on the object may include at least one of object name, object type information, object history information, object appearance information, object weight information, object location information, and detailed description of the object. The displayed information may be moved according to the position where the information is displayed and the recognized gesture.

Receiving the object selection information may include detecting a down event at a first position on the transparent display, and detecting an up event at a second position on the transparent display. The object indicated by the input object selection information may be an object visible through an area including the first position and the second position on the transparent display.

The object selection information may include at least one of an image, an object name, and an object property. That is, in the detecting of the object, the object having the same image as the image of the object selection information may be detected, the object having the object name of the object selection information may be detected, and the object attribute of the object selection information may be detected. Can detect objects

A display may be displayed for highlighting the shape of the detected object seen on the transparent display. That is, the perimeter of the shape may be displayed in a specific color, and the perimeter of the shape or the area of the shape may be displayed to be shiny.

The acquiring may include requesting a server to search for information on the object and receiving a search result from the server. In the requesting of the search, at least one of the image and the location information of the detected object may be transmitted.

The acquiring may include retrieving information about the object from data stored in a storage medium.

The acquiring may include: retrieving information about the object from data stored in a storage medium; requesting a server to retrieve information about the object when the search fails in the retrieving; and from the server. Receiving the search results.

In order to achieve the above another technical problem, a transparent display device according to the present invention, detects an object indicated by the object selection information received in the transparent display, the shape of the object, the landscape seen through the transparent display, And a control unit for obtaining information about the detected object and controlling the obtained information to be displayed on the transparent display. The object may include at least one of a person, an animal, an object, a car, a building, and a natural object. The information on the object may include at least one of object name, object type information, object history information, object appearance information, object weight information, object location information, and detailed description of the object. The displayed information may be moved according to the position where the information is displayed and the recognized gesture.

The controller may detect a down event at a first position on the transparent display and detect an up event at a second position on the transparent display, wherein the input object selection information is instructed. The object may be an object visible through an area including the first position and the second position on the transparent display.

The object selection information may include at least one of an image, an object name, and an object property. That is, the controller may detect an object having an image of the object selection information, detect an object having an object name, and detect an object corresponding to an object property.

The controller may control the display to highlight a shape of the detected object displayed on the transparent display. In this case, the display may be displayed on an edge of the shape, may be displayed in a specific color, and may be displayed in the form of a thick line. In addition, the controller may display the edge of the shape or the area occupied by the shape to be shiny.

The control unit may transmit a search request signal for requesting information search for the object to a server, and control reception of a search result from the server. The search request signal may include at least one of an image and location information of the detected object.

The controller may search for information about the object in data stored in a storage medium. If retrieving the information on the object from the storage medium, the controller may request a server to retrieve the information on the object.

According to the transparent display device and the object detection method according to the present invention, since the user provides information on the object selected by the user from the objects visible through the transparent display, the user does not need to perform a separate search procedure for the information on the rear object When an object corresponding to a certain condition is displayed on a transparent display, the information about the object is provided so that the user can easily obtain information about the desired object without having to continuously observe the appearance of the specific object. can do.

1 is a view showing a preferred embodiment configuration for an object information providing system according to the present invention,
2 is a diagram illustrating an example of a situation in which information about an object indicated by object selection information is displayed;
3A to 3C are diagrams illustrating one preferred embodiment of a method of inputting object selection information;
4 is a view showing another preferred embodiment of the manner in which object selection information is input;
5 is a diagram illustrating another example of a situation in which information on an object indicated by object selection information is displayed;
6A to 6D illustrate screens on which a display for highlighting the shape of an object indicated by the object selection information is displayed;
7 is a diagram illustrating a screen on which information about a detected object is displayed;
8 is a block diagram showing the configuration of a preferred embodiment of a transparent display device according to the present invention;
9 is a block diagram showing a configuration of a preferred embodiment of the control unit of the transparent display device;
10 is a block diagram showing a process of performing a preferred embodiment of the object detection method according to the present invention;
11A to 11E illustrate screens on which information about a detected object and a recognized gesture are displayed;
12 is a diagram illustrating a screen displaying a gesture for requesting display of information about a detected object;
FIG. 13 is a flowchart illustrating a preferred embodiment of an object detection process; FIG.
14 is a view showing a process of performing a preferred embodiment for the process of obtaining object information;
FIG. 15 is a flowchart illustrating another preferred embodiment of a process for obtaining object information; FIG.
16 is a view showing the implementation of another preferred embodiment of the object detection method according to the present invention;
17 is a block diagram showing the configuration of another preferred embodiment of a transparent display device according to the present invention.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention that can specifically realize the above object will be described. At this time, the configuration and operation of the present invention shown in the drawings and described by it will be described as at least one embodiment, by which the technical spirit of the present invention and its core configuration and operation is not limited.

The terms used in the present invention have been selected as general terms widely used as possible in consideration of the functions in the present invention, but may vary according to the intention or custom of a person skilled in the art or the emergence of a new technology. In addition, in certain cases, there is also a term arbitrarily selected by the applicant, in which case the meaning will be described in detail in the description of the invention. Therefore, it is intended that the terms used in the present invention should be defined based on the meanings of the terms and the general contents of the present invention rather than the names of the simple terms.

1 is a diagram showing a preferred embodiment configuration for an object information providing system according to the present invention.

Referring to FIG. 1, the object information providing system 1 according to the present invention may include an object information providing server 10, a location application server 20, a network 50 and 60, and a transparent display device 100. have.

The object information providing server 10 may store the object image, the location information, the information about the object, the association information relating the object image and the information about the object, and the association information relating the position information and the information about the object. The object may include at least one of a person, an animal, an object, a car, a building, and a natural object. The information about the object may include at least one of object name, object type information, object history information, object appearance information, object weight information, object location information, and detailed description of the object. The information about the object is called object information, and the object image is an image in which the object is photographed.

The association information is information mapping information and information. For example, the association information may be one table name in a relational database system. The object information providing server 10 may search for object information that maps to a specific object image through related information, and may search for object information that maps to location information through related information.

The object information providing server 10 may receive a search request signal from the transparent display device 100. The search request signal may include at least one of an image and location information of the detected object. The detected object image may be a photograph taken by the transparent display apparatus 100, or may be an object image extracted from the photographed photograph.

When the search request signal including the image of the detected object is received from the transparent display device 100, the object information providing server 10 may recognize the object from the received image and search for information on the recognized object. have. The object information providing server 10 may transmit the information about the found object to the transparent display device 100. Here, the object information providing server 10 may recognize the object by comparing the received image with the stored object image. That is, the object information providing server 10 may find an object image that is the same or most similar to the object image included in the received image, and search for information on the recognized object by using the found object image and the stored related information. have.

When receiving the search request signal including the location information of the detected object from the transparent display device 100, the object information providing server 10 recognizes the object mapped to the received location information, and the information on the recognized object You can search. The object information providing server 10 may transmit the information about the found object to the transparent display device 100. Here, the object information providing server 10 may search for the information on the recognized object by using the received location information and the stored related information.

When receiving the search request signal including the image and location information of the detected object from the transparent display device 100, the object information providing server 10 recognizes the object based on the image and location information of the received object, You can retrieve information about recognized objects. The object information providing server 10 may transmit the information about the found object to the transparent display device 100. Here, the object information providing server 10 may detect object information corresponding to the received object information and object information mapping among the object information mapped to the object image included in the image of the object as the information on the recognized object. have.

The location application server 20 may provide a location based service (LBS) to the transparent display device 100. The location application server 20 may provide location information on a specific object to the transparent display device 100, and provide map and terrain information of a specific location.

The object information providing server 10, the location application server 20, and the transparent display device 100 may be connected to a network. The network may be the Internet or an intranet and may be a Broadband Convergence Network (BCN) or an Ultra Broadband Convergence Network (uBCN).

The network may be composed of a backbone network 60 and a subscriber network 50. The backbone network 60 may be composed of one or a plurality of integrated networks of an X.25 network, a Frame Relay network, an ATM network, a Multi Protocol Label Switching (MPLS) network, and a Generalized Multi Protocol Label Switching (GMPLS) network. Subscriber network 50 includes Fiber To The Home (FTTH), Asymmetric Digital Subscriber Line (ADSL), cable network, Wireless LAN (IEEE 802.11b, IEEE 802.11a, IEEE802.11g, IEEE802.11n), WIBro (Wireless Broadband) , Wimax, and High Speed Downlink Packet Access (HSDPA). The transparent display device 100 may be connected to the subscriber network 50.

The transparent display apparatus 100 receives object selection information and detects an object indicated by the object selection information input in a landscape viewed through the transparent display. The transparent display apparatus 100 displays information about the detected object.

2 is a diagram illustrating an example of a situation in which information about an object indicated by object selection information is displayed.

Referring to FIG. 2, when a person 230 inputs object selection information for selecting an object 210 visible through a transparent display, the transparent display apparatus 100 may display an object 210 indicated by the input object selection information. ) Can be detected. In addition, the transparent display apparatus 100 may display the information 215 about the detected object 210 on the transparent display. The object selection information input here may be a signal for selecting a specific point 251 on the screen of the transparent display device 100. In addition, the object 210 indicated by the object selection information may be an object whose shape is displayed on the point 251 based on the eye direction when the person 230 looks at the point 251 selected by the person 230.

3A to 3C illustrate exemplary embodiments of a method of inputting object selection information.

3A to 3C, object selection information may be input by a gesture. The gesture here may be a single point or multipoint gesture, a static or dynamic gesture, a continuous or segmented gesture, and / or the like. A single point gesture is a gesture performed with a single point of contact, for example, this gesture is performed with a single touch, for example from one finger, palm or stylus. Multipoint gestures are gestures that can be performed with multiple points. For example, this gesture is performed with multiple touches, for example from multiple fingers, fingers and palms, fingers and stylus, multiple stylus and / or any combination thereof. Static gestures are gestures that do not include movement, and dynamic gestures are gestures that do include movement. Continuous gestures are gestures performed in a single stroke, and distinct gestures are gestures performed in a sequence of separate steps or strokes.

The contact of a finger with the touch screen typically has several different patterns. For example, a single point gesture on a touch screen includes a down event followed by an up event at or about the same location as the down event. Dynamic gestures on the touch screen include a down event followed by at least one dragging event, followed by an up event.

In some embodiments, parameters are used to describe the process by which a finger approaches a touch screen display, contacts a touch screen display, and moves away from the touch screen display. The touch screen display may be a touch screen integrated with the transparent display. This parameter may be at least one of the distance between the finger and the touch screen display, the pressure at which the finger presses the touch screen display, the contact area between the finger and the touch screen, the voltage between the finger and the touch screen, the capacitance between the finger and the touch screen display, or at least one of the physical parameters. It may be a function of.

In some embodiments, depending on the magnitude of a parameter (eg, capacitance) between the finger and the touch screen display, when this parameter exceeds a predetermined threshold, a down event occurs and the parameter exceeds the predetermined threshold and While the finger's corresponding cursor position moves from position A to position B, a dragging event occurs, and an up event occurs when this parameter falls below the threshold level.

The transparent display apparatus 100 detects a down event at the first location 310. Finger 301 moves from first position 310 to second position 320. In this case, the finger 301 may move in contact with the touch screen display 300 of the transparent display device 100, and the parameter between the finger 301 and the touch screen display 300 exceeds a predetermined threshold. I can move it. When the transparent display device 100 detects a dragging event at the second location 320, the transparent display device 100 displays a display indicating an area including the first location 310 and the second location 320. 305 may be displayed on the touch screen display 300.

When the finger 301 moves to the third position 330 and the finger is moved away from the touch screen display 300 or the parameter is smaller than a predetermined threshold, the transparent display device 100 is moved to the third position 330. Detect up event. Upon detecting the up event, the transparent display device 100 displays a display 307 indicating an area including the first location 310 and the third location 330 and in the area 307 the touch screen display 300. The visible object 350 may be detected through. Here, the down event of the first location 310 and the up event of the third location 330 may be recognized as one object selection information, and the object indicated by the input object selection information may have a shape in the region 307. It may be an object 350.

4 illustrates another exemplary embodiment of a method of inputting object selection information.

Referring to FIG. 4, the object selection information may be input through a Graphic User Interface (GUI). The object selection information may include at least one of an image, an object name, and an object property, and the GUI may provide an interface for receiving an image, an object name, and an object property.

The GUI 400 is an example of a GUI for inputting object selection information indicating a specific vehicle. The user may input "AAAAAA" (model name), which is the name of the car, to be detected in the input box 410, and input the index "Black" (attribute) of the car, which is to be detected, in the input box 420. Can be.

When the object selection information is received through the GUI, the transparent display apparatus 100 may store the input object selection information and detect whether an object indicated by the stored object selection information is displayed through the transparent display. The gaze direction shown here may be a vertical direction of the display of the transparent display device 100, may be a direction specified by the user, and the transparent display device 100 at a point where the user's eyes are located while the object selection information is input. Direction of the furnace.

5 is a diagram illustrating another example of a situation in which information on an object indicated by object selection information is displayed.

Referring to FIG. 5, the transparent display apparatus 100 detects whether or not a car 510 having a model name of “AAAAAA” and a color of “Black” is shown through the transparent display, and when the car 510 is detected, Information 511 about 510 is displayed.

6A to 6D illustrate screens on which a display for highlighting the shape of an object indicated by the object selection information is displayed.

6A to 6D, when the object 610 is detected, the transparent display apparatus 100 may display a display for notifying the position of the detected object 610 on the transparent display 600.

In some embodiments, the transparent display apparatus 100 may display a border line 601 on the transparent display 600 that highlights an area in which the shape of the detected object 610 is located. Here, the border line 601 may be displayed to flicker.

In some embodiments, the transparent display apparatus 100 may display the display 602 in an area in which the shape of the detected object 610 is located. In this case, the indication 602 may be hatched, may be a specific color, and the indication 602 may be displayed to flicker.

In some embodiments, the transparent display device 100 may display a dotted line 603 along the outer line of the shape of the detected object 610. The dotted line 603 may be displayed to blink.

In some embodiments, the transparent display device 100 may display the solid line 604 along the outline of the shape of the detected object 610. Here, the solid line 604 may be displayed in bold, may be displayed in a specific color, and may be displayed to flicker.

7 is a diagram illustrating a screen on which information about a detected object is displayed.

Referring to FIG. 7, when the transparent display apparatus 100 detects an animal 710, information about the detected animal 710 may be displayed. The transparent display device 100 displays the Yorkshire Terrier as the object type information, displays the UK as the object history information, displays about 23 cm in height as the object appearance information, displays 3.5 kg as the object weight information, and displays the object for the object. The detailed description may display introductory information, appearance information, and personality information.

The transparent display apparatus 100 may retrieve information about the detected object from a storage device directly and obtain the information by requesting a search from the object information providing server 10.

The transparent display apparatus 100 may transmit a search request signal for requesting a search for the detected object to the object information providing server 10. The transmitted search request signal may include an image of the detected object and location information of the detected object. The image of the detected object may be an original image captured by the photographing apparatus, or may be an image generated by extracting a region occupied by the object from the original image by the transparent display apparatus 100.

The transparent display apparatus 100 may obtain the location information of the detected object from the location application server 20, and calculate the location information of the detected object from its location information.

The transparent display apparatus 100 may position its position through radio waves received from the base station 40 using network-based technologies such as Cell ID scheme, AOA scheme, and TDOA scheme. In addition, the transparent display apparatus 100 uses a handset-based positioning technique such as a time of arrival (TOA) method, a global positioning system (GPS) method, and a hybrid positioning method, to position itself through a signal received from the satellite 30. Can be located.

The position of the detected object around the transparent display apparatus 100 may be positioned by using image recognition-based technology. For example, the transparent display apparatus 100 may position the object from a picture including the objects photographed from the plurality of photographing apparatuses using a teacher learning algorithm. As another example, the transparent display apparatus 100 may position the object using Vision VR based technology. By using the image recognition technology to locate the object, the position of the object can be detected with a precision of several centimeters, so that the position of the object can be detected more accurately.

The position of the detected object around the transparent display apparatus 100 may be positioned by using an ultrasound-based technique. The transparent display apparatus 100 may calculate the distance of the detected object through the transmitted ultrasound by transmitting ultrasound in the recognized gaze direction, and determine the azimuth angle of the object through the direction in which the ultrasound is transmitted. The transparent display apparatus 100 may position the detected object based on the calculated distance and the determined azimuth angle.

8 is a block diagram showing the configuration of a preferred embodiment of a transparent display device according to the present invention.

Referring to FIG. 8, the transparent display apparatus 100 includes a tuner unit 110, a demodulator 120, a network interface unit 130, a signal processor 140, a transparent display 150, and an audio output unit 160. , An input device 170, a storage unit 180, a controller 190, and an external signal receiver 135. The transparent display device 100 may be a personal computer system such as a desktop, laptop, tablet or handheld computer. In addition, the transparent display device 100 may be a mobile terminal such as a mobile phone, a smart phone, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, or the like. Can be.

The tuner unit 110 selects an RF broadcast signal corresponding to a channel selected by a user from among radio frequency (RF) broadcast signals received through an antenna, and converts the selected RF broadcast signal into an intermediate frequency signal or a baseband video or audio signal. To convert. For example, if the selected RF broadcast signal is a digital broadcast signal, it is converted into a digital IF signal (DIF). If the analog broadcast signal is converted into an analog baseband video or audio signal (CVBS SIF). That is, the tuner 110 may process a digital broadcast signal or an analog broadcast signal. The analog baseband video or audio signal CVBS SIF output from the tuner 110 may be directly input to the signal processor 140.

In addition, the tuner unit 110 may receive a single broadcast RF broadcast signal according to an ATSC (Advanced Television System Committee) scheme or a multiple broadcast RF broadcast signal according to a digital video broadcasting (DVB) scheme.

According to another embodiment of the present invention, the transparent display device 100 may include at least two tuner parts. In the case of having at least two tuner units, the second tuner unit selects an RF broadcast signal corresponding to a channel selected by a user from among RF broadcast signals received through an antenna, similarly to the first tuner unit, and intermediates the selected RF broadcast signal. Convert to frequency signal or baseband video or audio signal.

In addition, the second tuner may sequentially select the RF broadcast signals of all the broadcast channels stored through the channel memory function among the received RF broadcast signals and convert them to intermediate frequency signals or baseband video or audio signals. The second tuner may periodically perform the conversion of all broadcast channels. Therefore, the transparent display device 100 may display images of the broadcast signal converted through the first tuner unit and provide images of various channels converted through the second tuner unit in the form of thumbnails. In this case, the first tuner unit converts the main RF broadcast signal selected by the user into an intermediate frequency signal or a baseband video or audio signal, and the second tuner unit sequentially / periodically selects all the RF broadcast signals except for the main RF broadcast signal. This may be converted into an intermediate frequency signal or a baseband video or audio signal.

The demodulator 120 receives the digital IF signal DIF converted by the tuner 110 and performs a demodulation operation. For example, when the digital IF signal output from the tuner unit 110 is an ATSC scheme, the demodulator 120 performs 11-VSGI (11-Vestigial Side Band) demodulation. As another example, when the digital IF signal output from the tuner 110 is a DVB scheme, the demodulator 120 performs coded orthogonal frequency division modulation (COFDMA) demodulation.

Also, the demodulation unit 120 may perform channel decoding. To this end, the demodulator 120 includes a trellis decoder, a de-interleaver, and a reed solomon decoder to perform trellis decoding, deinterleaving, Solomon decoding can be performed.

The demodulation unit 120 may perform demodulation and channel decoding, and then output a stream signal TS. In this case, the stream signal may be a signal multiplexed with a video signal, an audio signal, or a data signal. For example, the stream signal may be an MPEG-2 Transport Stream (TS) multiplexed with an MPEG-2 standard video signal, a Dolby AC-3 standard audio signal, and the like. Specifically, the MPEG-2 TS may include a header of 4 bytes and a payload of 184 bytes.

The stream signal output from the demodulator 120 may be input to the signal processor 140. The signal processor 140 performs demultiplexing and signal processing, and then outputs an image to the transparent display 150, and outputs an audio to the audio output unit 160.

In addition, in the case of a digital broadcast receiver having at least two tuners, a number of demodulators corresponding thereto may be provided. The demodulator may be provided separately according to the ATSC scheme and the DVB scheme.

The network interface unit 130 receives the packets received from the network and transmits the packets to the network. The network interface unit 130 may transmit an IP packet transmitting a search request signal to the object information providing server 10 through a network, and may transmit an IP packet delivering a search result from the object information providing server 10. In addition, the network interface unit 130 may transmit an IP packet transmitting a location information request signal to the location application server 20 through a network, and receive an IP packet delivering location information from the location application server 20. .

In addition, the network interface unit 130 receives an IP packet for transmitting broadcast data from a service providing server through a network. Here, the broadcast data includes content, update messages indicating whether to update the content, metadata, service information, software code, and A / V data. In addition, the service information may include service information for the real-time broadcast service and service information for the Internet service. The Internet service includes content services such as CoD (Content's on Demand), YouTube service, weather, news, local information and search, entertainment services such as games and karaoke, TV mail, and TV SMS (Short Message Service). This means a service that can be provided through the Internet. Accordingly, in the present invention, the digital broadcast receiver includes a network TV, a web TV and a broadband TV. In addition, the broadcast service may include an internet service as well as a broadcast service provided through terrestrial, satellite, and cable.

The controller 190 executes a command and performs an operation associated with the transparent display apparatus 100. For example, the controller 190 may control input and output and reception and processing of data between components of the transparent display apparatus 100 by using a command retrieved from the storage unit 180. The controller 190 may be implemented on a single chip, multiple chips, or multiple electrical components. For example, various architectures may be used for the controller 190, including dedicated or embedded processors, single purpose processors, controllers, ASICs, and the like.

The controller 190 executes computer code together with an operating system to generate and use data. The operating system is generally known and will not be described in more detail. By way of example, the operating system may be a Window based OS, Unix, Linux, Palm OS, DOS, Android, Macintosh, and the like. The operating system, other computer code, and data may be present in the storage unit 180 that operates in conjunction with the control unit 190.

The storage unit 180 generally provides a place for storing program codes and data used by the transparent display apparatus 100. For example, the storage unit 180 may be implemented as a read only memory (ROM), a random access memory (RAM), a hard disk drive, or the like. The program code and data may reside in a removable storage medium and may be loaded or installed onto the transparent display device 100 as needed. Removable storage media herein include CD-ROMs, PC-CARDs, memory cards, floppy disks, magnetic tape, and network components.

The transparent display 150 may operate in connection with the controller 190. The transparent display 150 may be implemented as an organic light emitting panel or a plasma panel.

The transparent display 150 may display a graphical user interface (GUI) 153 that provides an easy-to-use interface between a user of the transparent display device and an operating system or an application running on the operating system. The GUI 153 presents the program, file, and operation options in a graphical image. A graphical image may include a window, a field, a dialog box, a menu, an icon, a button, a cursor, a scroll bar, and the like. Such images may be arranged in a predefined layout or may be dynamically generated to help with the particular action the user is taking. During operation, a user can select and activate an image to present functions and tasks associated with various graphical images. By way of example, a user may select a button that opens, closes, minimizes, or maximizes a window, or an icon that launches a particular program.

The input device 170 may be a touch screen disposed on or in front of the transparent display 150. The touch screen may be integrated with the transparent display 150 or may be a separate component. As the touch screen is disposed in front of the transparent display 150, the user may directly manipulate the GUI 153. For example, the user may only place his finger on the object to be controlled. In the touchpad, there is no one-to-one relationship like this.

In the touchpad, the touchpad lies generally in a different plane away from the transparent display 150. For example, the transparent display 150 is generally located in a vertical plane and the touchpad is generally located in a horizontal plane. This makes its use less intuitive and therefore more difficult when compared to a touch screen. In addition to being a touch screen, the input device 170 may be a multipoint input device.

The controller 190 may recognize the gesture 171 applied to the input device 170 and control the transparent display apparatus 100 based on the recognized gesture 171. Here, the gesture may be defined as a stylized interaction with the input device 170 that maps to one or more specific computing operations. The gesture 171 may be performed through various hands, and more specifically, through finger movements. As another alternative or in addition, the gesture may be done with a stylus.

The input device 170 receives the gesture 171, and the controller 190 executes commands for performing operations associated with the gesture 171. In addition, the storage unit 180 may include a gesture operating program 181 which may be part of an operating system or a separate application. Gesture operator 181 generally recognizes the occurrence of gesture 171 and in turn responds to gesture 171 and / or gesture 171 to inform one or more software agents of what action (s) should be taken. Contains the command of.

When the user performs one or more gestures, the input device 170 transmits gesture information to the controller 190. Using commands from the storage unit 180, more specifically the gesture operation program 181, the controller 190 interprets the gesture 171 and stores the storage unit 180, the transparent display 150, and the voice output unit. Different components of the transparent display device 100 such as the 160, the signal processor 140, the network interface 130, and the input device 170 are controlled. The gesture 171 performs an operation in an application stored in the storage unit 180, modifies a GUI object displayed on the transparent display 150, modifies data stored in the storage unit 180, and a network interface unit ( 130 may be identified as a command for performing an operation in the signal processor 140. By way of example, these commands may be associated with zooming, panning, scrolling, page turning, rotation, resizing, video channel change, content reception, Internet connection, and the like. As a further example, the command may also be used to launch a particular program, open a file or document, view a menu, make a selection, execute a command, log on to an Internet site system, authorized person Permitting access to restricted areas of the computer system, loading a user profile associated with a user's preferred array of desktops, and / or the like.

The controller 190 may detect the object based on the object selection information received from the input device 170. If the object selection information is a down event and an up event, the controller 190 may detect an object whose shape is located at a first position where the down event occurs and a second position where the up event occurs. Herein, the controller 190 recognizes a gaze direction of a person heading to the transparent display through a picture received from the photographing apparatus 90, and an object visible on an area including the first position and the second position in the recognized gaze direction. Can be detected. The controller 190 may control object information acquisition of the detected object and control the transparent display 150 to display the obtained object information.

The controller 190 may receive object selection information from the input device 170 and check whether the object indicated by the received object selection information is visible through the transparent display 150. Here, the controller 190 may check whether the specific object is newly photographed by comparing the current image frame with the previous image frame from the photographing apparatus 90. When a specific object is newly photographed, the controller 190 may extract an image of the photographed object to determine whether the photographed object is an object indicated by the object selection information. The controller 190 may control the acquisition of the object information by using the image of the photographed object, and compare the image of the photographed object or the acquired object information with the object selection information and the object to which the photographed object is indicated by the object selection information. You can check if it is.

If the object indicated by the object selection information is visible, the controller 190 may display a display indicating an area where the shape of the object is located on the transparent display 150. In addition, the controller 190 may control the object information acquisition of the object, and may control the transparent display 150 to display the obtained object information.

The external signal receiver 135 may provide an interface for connecting the external device to the transparent display device 100. Here, the external device refers to various kinds of video or audio output devices such as a digital versatile disk (DVD), a Blu-ray, a game device, a camcorder, a computer (laptop), and the like. The transparent display apparatus 100 may control the video signal and the audio signal received from the external signal receiver 135 to be displayed, and may store or use the data signal.

Also, the external device may be the photographing device 90. The photographing apparatus 90 may include a plurality of cameras. The imaging device 90 can image a person. The photographing apparatus 90 may recognize a human face area, focus on the face area, and zoom in to capture an image, and recognize the human hand area to focus on the hand area, and zoom in to capture an image. In this case, the captured hand may be recognized as a gesture. That is, the controller 190 may execute commands for performing operations associated with the recognized hand.

The photographing apparatus 90 may photograph an object behind the transparent display 150. That is, the photographing apparatus 90 may capture an object visible through the transparent display 150 in the direction of the human eye according to the control signal of the transparent display apparatus 100. The photographing apparatus 90 may recognize the object area, focus on the object area, and zoom in to capture an image. The object may also be imaged by the imaging device 90 at a plurality of different locations.

9 is a block diagram illustrating a configuration of a preferred embodiment of the control unit of the transparent display device.

Referring to FIG. 9, the controller 190 may include a gaze direction recognizer 910, a central processor 920, and an object detector 930.

The gaze direction recognizing unit 910 may detect a face region in the image frame photographed by the photographing apparatus 92. The gaze direction recognition unit 910 may detect an eye image in the detected face region, and determine a gaze direction using the detected eye image.

In one embodiment for determining the gaze direction, the gaze direction recognizer 910 calculates a difference value between each of the stored eye images and the detected eye image, so that the difference value between the detected eye image is best among the stored eye images. You can select a small eye image. The gaze direction recognizer 910 may determine the gaze direction of the eye image detected by the gaze direction indicated by the gaze direction information mapped to the selected eye image.

The central processor 920 may detect a hand region in an image frame captured by the photographing apparatus 92. The central processing unit 920 may select a hand shape such as a hand shape displayed on the detected hand area among the stored hand shapes by comparing each of the stored hand shapes with the hand shapes displayed on the detected hand area. The CPU 920 may execute a command to perform an operation associated with the selected hand.

The object detector 930 controls the movement of the focus of the photographing apparatus 94 as an object indicated by the object selection information and controls the imaging of the photographing apparatus 94.

10 is a block diagram showing a process of performing an embodiment of the object detection method according to the present invention.

Referring to FIG. 10, the central processing unit 920 detects a down event at a first position on the transparent display 150 (S1000).

The central processor 920 detects an up event at a second position on the transparent display 150 (S1010).

The gaze direction recognition unit 910 determines the gaze direction (S1020). Here, the gaze direction recognition unit 910 may determine a direction toward the area including the first position and the second position in the space where the person is located as the gaze direction.

The object detector 930 detects an object visible through an area including a first position and a second position on the transparent display 150 in the gaze direction determined by the gaze direction recognizer 910 (S1030). The object may include at least one of a person, an animal, an object, a car, a building, and a natural object. In another embodiment, the object may be an object viewed through an area including a first position and a second position on the transparent display 150 in a vertical direction of the transparent display 150.

The central processing unit 920 controls the acquisition of information on the detected object (S1040). The object information, which is information about the object, may include at least one of object name, object type information, object history information, object appearance information, object weight information, object location information, and detailed description of the object.

Before obtaining information about the object, the central processing unit 920 may calculate location information of the detected object. The CPU 920 may control the displayed position information to be displayed, and use the calculated position information to obtain information about the detected object. Here, the central processing unit 920 may obtain the location information of the detected object from the location application server 20, and calculate the location information of the detected object from its location information. The central processing unit 920 may position the display apparatus 100 through radio waves received from the base station 40 using network-based technologies such as Cell ID method, AOA method, and TDOA method. . In addition, the central processing unit 920 uses a handset-based positioning technique such as a time of arrival (TOA) method, a global positioning system (GPS) method, and a hybrid positioning method to determine its position through a signal received from the satellite 30. Can be positioned. In addition, the central processing unit 920 may position the detected object based on the display apparatus 100 using image recognition technology or ultrasound technology.

The transparent display 150 displays the obtained object information (S1050). Here, the object information may be displayed as shown in the screens shown in FIGS. 2 and 7.

The central processing unit 920 checks whether an information movement command has been received (S1060). The command for moving the information may be a command generated through a gesture recognized through a hand shape included in an image frame captured by the photographing apparatus 92, and generated through a dynamic gesture recognized as a down event, a dragging event, or an up event. May be a command.

When the information movement command is received, the central processing unit 920 controls the movement of the position of the information displayed on the transparent display 150 according to the received information movement command (S1070).

11A to 11E are diagrams illustrating a screen on which information about a detected object and a recognized gesture are displayed, and FIG. 12 is a diagram illustrating a screen displaying a gesture for requesting display of information on a detected object. .

11A through 11E and 12, the object information 1113 about the object 1111 detected on the screen 1110 is located in an area where an image of the detected object 1111 is located. When the gesture 1115 is recognized while the screen 1110 is displayed, the screen 1110 is switched to the screen 1120. The recognized gesture 1115 may be displayed on the screen, and may disappear from the screen when a command associated with the gesture 1115 is performed. In some embodiments, the recognized gesture 1115 may not be displayed on the screen.

On the screen 1120, the object information 1113 is moved in the direction indicated by the finger of the gesture 1115 and displayed on the area 1123. When the gesture 1125 is recognized while the screen 1120 is displayed, the screen 1120 is switched to the screen 1130.

In the name 1130, the object information 1123 is moved in the direction indicated by the finger of the gesture 1125 and displayed in the area 1133. When the gesture 1135 is recognized while the screen 1130 is displayed, the screen 1130 is switched to the screen 1140.

On the screen 1140, the object information 1133 is moved in the direction indicated by the finger of the gesture 1135 and displayed on the area 1143. When the gesture 1145 is recognized while the screen 1140 is displayed, the screen 1140 is switched to the screen 1150.

On the screen 1150, the object information 1143 is moved in the direction indicated by the finger of the gesture 1145 and displayed on the area 1153. When the gesture 1155 is recognized while the screen 1150 is displayed, the screen 1150 is switched to the screen 1210.

In the screen 1210, the object information 1153 is not displayed according to the display stop command, which is a command associated with the gesture 1155. When the gesture 1215 is recognized while the screen 1210 is displayed, the object information 1153 may be displayed on the screen 1210. In this case, the object information may be displayed in the region 1153 of the previous screen 1150, and the object information may be displayed in the region designated as a default.

FIG. 13 is a flowchart illustrating a preferred embodiment of an object detection process. FIG.

Referring to FIG. 13, the object detector 930 controls the movement of the focus of the photographing apparatus so as to capture an object (S1300). Here, the photographing apparatus 94 may be configured of a plurality of cameras, the object detector 930 may control the movement of each camera.

The object detector 930 controls imaging of the photographing apparatus 94 at the moved focus (S1310).

The object detector 930 controls reception of an image captured by the object from the photographing apparatus 94 (S1320). The object detector 930 may extract an object image from an image frame output from the photographing apparatus 94. In addition, the object detector 930 may calculate the position of the object through the image frame captured by the photographing apparatus 94.

14 is a diagram illustrating a preferred embodiment of the process of obtaining the object information.

Referring to FIG. 14, the central processing unit 920 reads an image stored in the storage unit 180 (S1400). The stored image may be an image of an object around the transparent display device 100 in advance.

The central processor 920 compares the read image with the picked-up image (S1410). Here, when the outline of the object included in the read image and the outline of the object included in the captured image have the same pattern, the object included in the read image and the object included in the captured image may be recognized as the same object. have. That is, if the pattern of each outer line is matched with the same size, the object included in the read image and the object included in the captured image may be recognized as the same object.

The central processing unit 920 checks whether the same object is found (S1420).

If the same object is found, the central processing unit 920 reads out information associated with the image found in the storage unit 180 (S1430). The read information may be used as information about an object included in the captured image. Also, the read information may be information in which object information of objects around the transparent display apparatus 100 is stored in advance.

FIG. 15 is a flowchart illustrating another exemplary embodiment of an object information acquisition process. Referring to FIG.

Referring to FIG. 15, the central processing unit 920 requests a search for information on an object from a server (S1500). The server may be the object information providing server 10. In addition, the central processing unit 920 may transmit a search request signal to the object information providing server 10. The search request signal may include an object image and may include object location information.

The central processing unit 920 controls the reception of the search result from the server (S1510).

16 is a view showing the implementation of another preferred embodiment of the object detection method according to the present invention.

Referring to FIG. 16, the input apparatus 170 receives object selection information in operation S1600. The object selection information may include at least one of an image, an object name, and an object property and may be input through the GUI 400 illustrated in FIG. 4.

The central processor 920 controls the input object selection information to be stored in the storage 130 (S1610).

The central processing unit 920 checks whether an object indicated by the stored object selection information is detected (S1620). Here, the central processing unit 920 may check whether the object indicated by the stored object selection information is visible through the transparent display 150. The central processor 920 may determine whether the object is visible from the object image output by the object detector 930. The object detector 930 may check whether the specific object is newly photographed by comparing the current image frame with the previous image frame from the photographing apparatus 90. It is possible to control the movement of the focal point 90 and obtain an image frame in which the object is captured from the photographing apparatus 90, and extract the object image of the object from the obtained image frame.

The controller 190 may control the acquisition of the object information by using the object image extracted by the object detector 930, and compare the image of the photographed object or the acquired object information with the object selection information so that the newly photographed object is an object. You can check whether the selection information indicates an object.

When the object indicated by the object selection information is detected, the central processing unit 920 controls acquisition of information on the detected object (S1630). The object information, which is information about the object, may include at least one of object name, object type information, object history information, object appearance information, object weight information, object location information, and detailed description of the object. The detailed execution process of step S1630 may be performed according to the object information acquisition process shown in FIG. 14 or FIG. 15.

Before obtaining information about the object, the central processing unit 920 may calculate location information of the detected object. The CPU 920 may control the displayed position information to be displayed, and use the calculated position information to obtain information about the detected object. Here, the central processing unit 920 may obtain the location information of the detected object from the location application server 20, and calculate the location information of the detected object from its location information. The central processing unit 920 may position the display apparatus 100 through radio waves received from the base station 40 using network-based technologies such as Cell ID method, AOA method, and TDOA method. . In addition, the central processing unit 920 uses a handset-based positioning technique such as a time of arrival (TOA) method, a global positioning system (GPS) method, and a hybrid positioning method to determine its position through a signal received from the satellite 30. Can be positioned. In addition, the central processing unit 920 may position the detected object based on the display apparatus 100 using image recognition technology or ultrasound technology.

The transparent display 150 displays the obtained object information (S1640). Here, the object information may be displayed as shown in the screens shown in FIGS. 5 and 7. In addition, the central processing unit 920 may control to display a display indicating the area where the shape of the detected object is located on the transparent display 150.

Although the object detecting method of FIG. 16 is not shown, the method may further include steps S1060 and S1070 of the object detecting method of FIG. 10.

17 is a block diagram showing the configuration of another preferred embodiment of a transparent display device according to the present invention.

The transparent display device 100 of FIG. 17 includes a tuner unit 110, a demodulator unit 120, a network interface unit 130, a signal processor 140, a transparent display 150, a voice output unit 160, and an input device. 170, a storage unit 180, a controller 190, an external signal receiver 1735, and a photographing apparatus 1790. The controller 190 controls the movement, focus, zoom in and zoom out, and imaging of the photographing apparatus 1790. The photographing apparatus 1790 photographs a rear object visible through the transparent display 150 and a person located around the transparent display 150 under the control of the controller. The photographing apparatus 150 may include a plurality of cameras.

17 is different from the transparent display apparatus 100 of FIG. 8 in that the transparent display apparatus 100 of FIG. 17 includes a photographing apparatus 1790, and the remaining components of the transparent display apparatus 100 of FIG. In the correspondence with the components of the transparent display apparatus 100 of FIG. 8, detailed description of each component will be omitted below.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer apparatus is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . The computer-readable recording medium may also be distributed to networked computer devices so that computer readable code can be stored and executed in a distributed manner.

Although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific preferred embodiments described above, and the present invention belongs to the present invention without departing from the gist of the present invention as claimed in the claims. Various modifications can be made by those skilled in the art, and such changes are within the scope of the claims.

Claims (20)

Receiving object selection information;
Detecting an object indicated by the input object selection information in a landscape viewed through a transparent display;
Obtaining information about the detected object; And
And displaying the obtained information on the transparent display. The method of claim 1,
Receiving the object selection information,
Detecting a down event at a first location on the transparent display; And
Detecting an up event at a second location on the transparent display;
And an object indicated by the input object selection information is an object visible through an area including the first position and the second position on the transparent display. The method of claim 1,
The object selection information includes at least one of an image, an object name, and an object property. The method of claim 1,
Displaying an indication for highlighting the shape of the detected object visible on the transparent display. The method of claim 1,
The obtaining step,
Requesting a server to retrieve information about the object; And
Receiving a search result from the server; object detection method comprising a. 6. The method of claim 5,
In requesting the search,
Transmitting at least one of an image and location information of the detected object. The method of claim 1,
The obtaining step,
And retrieving information about the object from data stored in a storage medium. The method of claim 1,
The object detection method comprising at least one of a person, an animal, an object, a car, a building, and a natural object. The method of claim 1,
The information about the object includes at least one of object name, object type information, object history information, object appearance information, object weight information, object location information, and detailed description of the object. The method of claim 1,
Wherein the displayed information is moved according to the displayed position and the recognized gesture. A transparent display that transmits the shape of the object;
Receive selection information, detect an object indicated by the received object selection information in a landscape viewed through the transparent display, obtain information about the detected object, and display the obtained information on the transparent display; And a control unit for controlling the transparent display device. 12. The method of claim 11,
The control unit,
Detecting a down event at a first location on the transparent display and detecting an up event at a second location on the transparent display, wherein the object indicated by the input object selection information is the transparent display; And an object visible through an area including the first position and the second position on the image. 12. The method of claim 11,
The object selection information includes at least one of an image, an object name, and an object property. 12. The method of claim 11,
The control unit,
And display a display for highlighting the shape of the detected object visible on the transparent display. 12. The method of claim 11,
The control unit,
And transmitting a search request signal for requesting information search for the object to a server, and controlling reception of a search result from the server. 16. The method of claim 15,
The search request signal includes at least one of an image and location information of the detected object. 12. The method of claim 11,
The control unit,
And retrieving information about the object from data stored in a storage medium. 12. The method of claim 11,
The object may include at least one of a person, an animal, an object, a car, a building, and a natural object. 12. The method of claim 11,
The information about the object includes at least one of object name, object type information, object history information, object appearance information, object weight information, object position information, and detailed description of the object. 12. The method of claim 11,
And wherein the displayed information is moved according to the displayed position and the recognized gesture.

Images